Commit Graph

302 Commits

Author SHA1 Message Date
b70343dc2b Fixing Image struct to no longer use Graph.
Signed-off-by:  John Howard <jhoward@microsoft.com>
Upstream-commit: 9001ea26e7821239207586ad23c78c67d6926bfc
Component: engine
2015-07-20 13:59:53 -07:00
4cbf8e8120 Merge pull request #13681 from tiborvass/carry-11784
Carry 11784: rmi dangling is unsafe when pulling
Upstream-commit: 7f353a11e4d40d08d1bd15df24c87a95d8cd9a02
Component: engine
2015-07-17 16:17:18 -07:00
371476f565 fix 8926: rmi dangling is unsafe when pulling
Signed-off-by: Ma Shimiao <mashimiao.fnst@cn.fujitsu.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 1b67c38f6f4a709e1d159445b56f0beee8cfad3a
Component: engine
2015-07-17 11:39:57 -04:00
e2d31e5c38 graph: clarify the need for named error
Signed-off-by: Vincent Batts <vbatts@redhat.com>
Upstream-commit: a40e337882dfe3f34af44a9f2aec2ed96dcce455
Component: engine
2015-07-17 10:01:52 -04:00
8755c824aa Merge pull request #14664 from calavera/fix_load_tag_with_digest
Check if a tag name to load is a valid digest.
Upstream-commit: 212525f951d9c92ad4cbf7628e7024930da080ac
Component: engine
2015-07-16 12:46:05 -07:00
98b526f041 Check if a tag name to load is a valid digest.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Upstream-commit: 1ec25653d87fa86517a319992ee89eb3e5ee1223
Component: engine
2015-07-16 10:53:56 -07:00
1492d5ce64 Update graph to use vendored distribution client for the v2 codepath
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 19515a7ad859b28c474d81e756ac245afcd968e3
Component: engine
2015-07-16 13:13:47 -04:00
0a78587632 remove pkg/transport and use the one from distribution
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 276c640be4b4335e3b8d684cb3562a56d3337b39
Component: engine
2015-07-16 13:13:46 -04:00
89b6e2370a Merge pull request #14661 from LK4D4/vet_warns
Fix some formatting calls
Upstream-commit: cd642973fa50531d3da6c13eb0255b6f4cc8aa9c
Component: engine
2015-07-15 16:41:18 -07:00
46a0415407 Fix some formatting calls
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
Upstream-commit: a5142f6ac31c6c21b96c6895fff525c46e8056a6
Component: engine
2015-07-15 12:25:50 -07:00
06996e8f7f Set canonical name correctly
Currently canonical name gets set to the local name and displayed in the errors.
Canonical name should be the unique and canonical name for an image.
Use docker.io as the canonical domain for images on the public registry.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: 7f48cd7dce6fdc077bcde0962e0aa0e73fb63225
Component: engine
2015-07-14 17:45:49 -07:00
1296feade8 Windows: Graph driver implementation
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: 52f4d09ffb376ffaa6677cb1e0413c6a97f53f24
Component: engine
2015-07-10 14:33:11 -07:00
017d6047d5 added ability to iterate over all indexes and use index.Iterate() instead of ReadDir() to walk over the graph
Signed-off-by: Roman Strashkin <roman.strashkin@gmail.com>
Upstream-commit: cc955ae73c8aa234e0c41bdb65f111be79b32b90
Component: engine
2015-07-07 22:13:28 +03:00
36eb2b9f9d Fix duplicate layers in manifest
Currently the layer array is initialized with the first layer then the first layer is appened to the layer list. Adding the first layer twice causes the layer to appear twice in the manifest, making a duplicate push and pull attempt occur.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: 35081ea4b6711b1cfccb67b0f0d691b7adc85ff6
Component: engine
2015-07-06 17:31:06 -07:00
842ad32482 Replace latest log by logrus
Related to #11618 and #11614

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 10e114fb956db1b1a8bc9308cc6d14cbf30a5bab
Component: engine
2015-07-02 16:11:52 +02:00
205cf6d70e Attempt to protect on disk image store with mutex
During `(*Graph).Register, there was no protection on adding new layers
concurrently. In some cases, this resulted in corruption of a layer by creating
the directory but not the underlying data. This manifested in several different
IO errors reported in the client.  This attempts to fix this by adding a mutex
by Image ID to protect the Register operation.

We do not completely understand the root cause of this corruption other than
the result is somehow tied to this particular function.  This fix has been
confirmed to address the issue through testing.

Unfortunately, this fix does not address existing corruption. The user will
have to remove and re-pull the corrupt layer to stop the error from happening
in the future. This change only ensures that the layer will not become corrupt.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: 7eac23cf8de08f281c84c36390f734dbaef8c69a
Component: engine
2015-06-25 20:16:37 -07:00
35e8f92672 Remove dead code
Signed-off-by: Antonio Murdaca <runcom@linux.com>
Upstream-commit: 927d13bc3c2030bb0e0429dbc500f13d72e7ccf6
Component: engine
2015-06-20 19:14:15 +02:00
b387e8ade7 Merge pull request #13575 from mattmoor/consistent-push-fallback
Make v2 push have v1-fallback behavior consistent with pull.
Upstream-commit: d86345b9f722c187afd4af8a92583fc07506b5d9
Component: engine
2015-06-17 13:41:58 -07:00
0b8d2a4fe1 Merge pull request #13870 from lindenlab/pull-single-tag
Only request a single repository tag when pulling a specific image:tag
Upstream-commit: c107e9d7903d148692f9e727bf6d5ce6e88e2f72
Component: engine
2015-06-17 15:29:39 -04:00
2d2c80b6fc Merge pull request #13975 from stevvooe/move-setup-init-layer
Move graph.SetupInitLayer to daemon package where it is used
Upstream-commit: a27d8f9aa4dbdbad5bbfd83c8d8321e7a976305c
Component: engine
2015-06-16 22:37:51 -04:00
fe0ca49e61 Move graph.SetupInitLayer to daemon package where it is used
An inspection of the graph package showed this function to be way out of place.
It is only depended upon by the daemon code. The function prepares a top-level
readonly layer used to provide a consistent runtime environment for docker
images.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: b7f887a9a24a2df61fa53612ae42b7e9d62fe858
Component: engine
2015-06-16 16:50:56 -07:00
d527901687 Merge pull request #13685 from yuchangchun1/tag_regx
add a more accurate error description for invalid tag name
Upstream-commit: ba9db62e68581b2f9187b15b13150396961e23e9
Component: engine
2015-06-16 16:33:17 -07:00
8810db85be Merge pull request #13198 from rhvgoyal/extend-docker-inspect
docker-inspect: Extend docker inspect to export image metadata related to graph driver
Upstream-commit: e69df2589c1220217054abdc6a361b55990b3c0b
Component: engine
2015-06-16 15:03:14 -05:00
4dbf1e0f71 Only pulling single repository tag on pull for a specific tag. extending TestGetRemoteTags unit test
Splitting out GetRemoteTag from GetRemoteTags.  Adding registry.ErrRepoNotFound error

Signed-off-by: Don Kjer <don.kjer@gmail.com>
Upstream-commit: b349a74c71fb072f9f23f508b8c698d0590abb12
Component: engine
2015-06-16 07:10:09 +00:00
778f11fd97 Store layer digests on pull
Currently digests are not stored on pull, causing a simple re-tag or re-push to send up all layers. Storing the digests on pull will allow subsequent pushes to the same repository to not push up content.
This does not address pushing content to a new repository. When content is pushed to a new repository, the digest will be recalculated. Since only one digest is currently stored, it may cause a new content push to the original repository.

Fixes #13883

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: a98ea87e463383a77d7244c78c9da3ac9f2cd086
Component: engine
2015-06-15 11:08:21 -07:00
4a6f8e2e63 docker-inspect: Extend docker inspect to export image/container metadata related to graph driver
Export image/container metadata stored in graph driver. Right now 3 fields
DeviceId, DeviceSize and DeviceName are being exported from devicemapper.
Other graph drivers can export fields as they see fit.

This data can be used to mount the thin device outside of docker and tools
can look into image/container and do some kind of inspection.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Upstream-commit: 407a626be62996cd6385ea4d80e669ab83f5f04d
Component: engine
2015-06-15 14:05:10 -04:00
3e78e2ecb7 Merge pull request #13773 from dmcgowan/refactor-1-image-graph-separation
refactor: separate graph from image
Upstream-commit: 00b8fec75f3a113c851013e0f0d559cc6d96b84d
Component: engine
2015-06-11 17:44:37 -07:00
08db54d962 Show actual number of elements restored
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: 141cd2a1f26f3455a1daf20420ca1efb5a9e1a16
Component: engine
2015-06-10 15:18:51 -07:00
c2f09cc42b Update graph to use digest type
Update get and set functions to use digests.
Update push code to use the digest type instead of string

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: c0b44218196ab597d22eb6344e8770fdee73067b
Component: engine
2015-06-05 18:07:41 -07:00
6c407af390 Update graph walkhistory to pass by value
Remove unused graph history function

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: bb50a4159bda8e6f2d47b69ddd476ab32ac6be14
Component: engine
2015-06-05 18:06:09 -07:00
6825ac1c62 Separate graph from image
Move graph related functions in image to graph package.
Consolidating graph functionality is the first step in refactoring graph into an image store model.
Subsequent refactors will involve breaking up graph into multiple types with a strongly defined interface.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: 2b58b677a54950bc78c13760ef79fe6284154847
Component: engine
2015-06-05 18:06:09 -07:00
edbdadf562 Make the v2 logic fallback on v1 when v2 requests cannot be authorized.
Signed-off-by: Matt Moore <mattmoor@google.com>
Upstream-commit: bd2575cc4f16bdce1748e90bdef63823cff3a35d
Component: engine
2015-06-02 21:22:59 -07:00
0cd5e0e48f add a more accurate error description for invalid tag name
Signed-off-by: yuchangchun <yuchangchun1@huawei.com>
Upstream-commit: e0475d331ba59bb742903e6a1b88251b9b27f416
Component: engine
2015-06-03 09:38:39 +08:00
49338919a2 Merge pull request #13576 from stevvooe/verify-digests
Properly verify manifests and layer digests on pull
Upstream-commit: 274baf70bf806678d12adc1e1de2c8d7774de645
Component: engine
2015-06-02 11:16:23 -07:00
8ce2553351 If no endpoint could be established with the given mirror configuration,
fallback to pulling from the hub as per v1 behavior.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
Upstream-commit: 6e4ff1bb13c82fd530b27c85ae9b0ca476cddfcd
Component: engine
2015-06-01 15:18:56 -07:00
e03ebf99d3 Break down loadManifest function into constituent parts
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: 84413be3c9f908c08f79e3080d9a74fb8ad34505
Component: engine
2015-06-01 13:02:50 -07:00
882b9f99a2 Add tests for loadManifest digest verification
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: 74528be903d2f6c9e07ab9a3931b050d5dae6562
Component: engine
2015-05-29 16:30:21 -07:00
412d1581d2 Attempt to retain tagging behavior
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: 1e653ab645b513966c3fe9b13db7a5a9c9a8acb7
Component: engine
2015-05-29 15:20:37 -07:00
395ceacd61 Properly verify manifests and layer digests on pull
To ensure manifest integrity when pulling by digest, this changeset ensures
that not only the remote digest provided by the registry is verified but also
that the digest provided on the command line is checked, as well. If this check
fails, the pull is cancelled as with an error. Inspection also should that
while layers were being verified against their digests, the error was being
treated as tech preview image signing verification error. This, in fact, is not
a tech preview and opens up the docker daemon to man in the middle attacks that
can be avoided with the v2 registry protocol.

As a matter of cleanliness, the digest package from the distribution project
has been updated to latest version. There were some recent improvements in the
digest package.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: 06612cc0fee103bf6f46429e3cd572725ef72948
Component: engine
2015-05-29 15:20:28 -07:00
553a8c069a Merge pull request #13374 from RichardScothern/v2-mirror
V2 mirror support
Upstream-commit: 2daede5a9cc6cc171f2f1f3aebcae561a2421d18
Component: engine
2015-05-27 21:15:26 -04:00
430d1ade67 Review feedback:
- Match verbiage with other output
    - Remove dead code and clearer flow

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
Upstream-commit: e817e08481b91638037e55b8d4855f56814c81f5
Component: engine
2015-05-27 17:29:24 -07:00
f4eb0aeb64 Restore the stripped registry version number
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
Upstream-commit: f6f7d3524862563f8388e1382bb28ecbde8cf265
Component: engine
2015-05-27 15:15:47 -07:00
b58daceb88 Merge pull request #12881 from nakedible/patch-1
Prevent fallback to v1 registry for digest pulls
Upstream-commit: 54b514735c592de77f3d79ec80c2d37bd589da6c
Component: engine
2015-05-26 19:24:37 -04:00
416a489d28 Allow mirroring only for the official index
Strip authconfig from session to keep credentials from being sent to the mirror.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: c19962ade10619e5edd8057249566c494d4719bb
Component: engine
2015-05-26 11:08:19 -07:00
521590a1df Registry v2 mirror support.
The v2 registry will act as a pull-through cache, and needs to be
handled differently by the client to the v1 registry mirror.

See docker/distribution#459 for details

Configuration

Only one v2 registry can be configured as a mirror. Acceptable configurations
in this chanage are: 0...n v1 mirrors or 1 v2 mirror. A mixture of v1 and v2
mirrors is considered an error.

Pull

If a v2 mirror is configured, all pulls are redirected to that mirror. The
mirror will serve the content locally or attempt a pull from the upstream mirror,
cache it locally, and then serve to the client.

Push

If an image is tagged to a mirror, it will be pushed to the mirror and be
stored locally there. Otherwise, images are pushed to the hub. This is
unchanged behavior.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
Upstream-commit: 13deed380127d4a0e7982bb0d4f6529d0962116b
Component: engine
2015-05-26 11:08:19 -07:00
04b3252768 Add transport package to support CancelRequest
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 73823e5e56446b23ce01bb8e44a9670ab2552b0a
Component: engine
2015-05-18 23:31:42 -04:00
14d2347fac registry: Refactor requestfactory to use http.RoundTrippers
This patch removes the need for requestFactories and decorators
by implementing http.RoundTripper transports instead.

It refactors some challenging-to-read code.

NewSession now takes an *http.Client that can already have a
custom Transport, it will add its own auth transport by wrapping
it.

The idea is that callers of http.Client should not bother
setting custom headers for every handler but instead it should
be transparent to the callers of a same context.

This patch is needed for future refactorings of registry,
namely refactoring of the v1 client code.

Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: a01cc3ca7729c3ce635fef7c1db837b5c6ae1028
Component: engine
2015-05-18 21:51:53 -04:00
acd82df11a Prevent fallback to v1 registry for digest pulls
The intention of the user is to download a verified image if explicitly
pulling with a digest and falling back to v1 registry circumvents that
protection.

Signed-off-by: Nuutti Kotivuori <naked@iki.fi>
Upstream-commit: 642e6a377324c7873f278c6bd7fd5e60201139e2
Component: engine
2015-05-18 08:26:26 +03:00
56ea157723 Revert "Fix inconsistent date formats in API"
This reverts commit 945fc9d882324ac87505e34bb74e6ebe30be1309.

Signed-off-by: Victor Vieux <victorvieux@gmail.com>
Upstream-commit: b261ce5fb007371892bc40a5853337db09a1c8ae
Component: engine
2015-05-14 17:31:45 -07:00
2e9a18b752 Fix inconsistent date formats in API
Prior to this patch, the response of
- GET /images/json
- GET /containers/json
- GET /images/(name)/history

display the Created Time as UNIX format which doesn't make sense.

These should be more readable as CLI command `docker inspect` shows.

Due to the case that an older client with a newer version daemon, we
need the version check for now.

Signed-off-by: Hu Keping <hukeping@huawei.com>
Upstream-commit: 945fc9d882324ac87505e34bb74e6ebe30be1309
Component: engine
2015-05-14 18:58:55 +08:00