The `digest` data type, used throughout docker for image verification
and identity, has been broken out into `opencontainers/go-digest`. This
PR updates the dependencies and moves uses over to the new type.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: 7a855799175b6b984886ef1cfa337d6df1d4c668
Component: engine
Signed-off-by: Jie Luo <luo612@zju.edu.cn>
typo
Signed-off-by: Jie Luo <luo612@zju.edu.cn>
fix some typos
Signed-off-by: Jie Luo <luo612@zju.edu.cn>
Upstream-commit: ea2dd4b5d0b41552d047814d9e39ddaa3662ab41
Component: engine
Translate pull errors to provide a more consistent and user friendly
error message.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: 19a93a6e3d4213c56583bb0c843cf9e33d379752
Component: engine
Using hub only works for amd64 platforms.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: 641c1808e187ed86d8b614e49e8e6da5146201e3
Component: engine
The current error message is "Error: image [name] not found". This makes
sense from the perspective of the v1 pull, since we found the repository
doesn't exist over the v1 protocol. However, in the vast majority of
cases, this error will be produced by fallback situations, where we
first try to pull the tag with the v2 protocol, and then fall back the
v1 protocol, which probably isn't even supported by the server.
Including the tag in the error message makes a lot more sense since the
actual repository may exist on v2, but not the tag.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 745892a7b271cf0f1770a8ec3698aaf61573e5f2
Component: engine
- cherry-pick from 1.10.3 branch: 0186f4d4223a094a050d06f456355da3ae431468
- add token service test suite
- add integration test (missing in 1.10.3 branch)
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
Upstream-commit: 1b5c2e1d722757a55364fb45cf3fcec7f2c75fb4
Component: engine
- add test for pull from private registry with no credentials
- add test for push to docker hub with no credentials
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
Signed-off-by: Antonio Murdaca <amurdaca@localhost.localdomain>
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
Upstream-commit: 6d23c3c57a733436d902b7883f2f0b00846df9e2
Component: engine
This PR fix the DockerHubPullSuite.TestPullNonExistingImage test
in #19425. The majority of the execution time in this test is
from multiple executions of 'docker pull', each of which takes
more than one second even though it tries to pull a non-existing
image.
Without changing the behavior of the 'docker pull' itself, this
fix tries to execute the 'docker pull' command in parallel in
order to speed up the execution of the overall test.
Since each 'docker pull' is independent, executions in parallel
should not alter the purpose of the test.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 461976d2affe3ed4a354608d1dcb266e06f1d2b9
Component: engine
Retries after v1 fallbacks were added in #20411. The test still appears
to be flaky. There are two potential problems. The initial pull was not
protected against pulling from v1, so it could be giving us a different
hello-world image to compare against. Also, after experiencing a v1
fallback, we need to restore the original image before doing the next
pull, because otherwise the "Image is up to date for hello-world:latest"
message will not show up as expected.
See #17214.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 0d270cadd4b65623b1f1ae02c4fe5bcc5f81fcd3
Component: engine
Sometimes transient network issues will cause
TestPullFromCentralRegistryImplicitRefParts to end up pulling with the
v1 protocol. This violates the assumptions behind the test. To make the
test more robust, allow a few retries if any pull ends up using the v1
protocol.
Fixes#17214
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 884201115315abb0c7815e44e6728e590476aeb8
Component: engine
Revert the portions of #17617 that report all errors when a pull
falls back, and go back to just reporting the last error. This was nice
to have, but causes some UX issues because nonexistent images show
additional "unauthorized" errors.
Keep the part of the PR that handled ENOSPC, as this appears to work
even without tracking multiple errors.
Fixes#19419
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 87338bf0fa97c905d2d707393c95f73aea398cad
Component: engine
Allow v1 protocol fallback when pulling all tags from a repository unknown to v2 registry
Upstream-commit: 6c30931b06ff44e1ce002002430577b7135d2049
Component: engine
This is a followup to #18839. That PR relaxed the fallback logic so that
if a manifest doesn't exist on v2, or the user is unauthorized to access
it, we try again with the v1 protocol. A similar special case is needed
for "pull all tags" (docker pull -a). If the v2 registry doesn't
recognize the repository, or doesn't allow the user to access it, we
should fall back to v1 and try to pull all tags from the v1 registry.
Conversely, if the v2 registry does allow us to list the tags, there
should be no fallback, even if there are errors pulling those tags.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 589a5226e7d4880ab7afa9e31202849db08a4bfa
Component: engine
PR #18590 caused compatibility issues with registries such as gcr.io
which support both the v1 and v2 protocols, but do not provide the same
set of images over both protocols. After #18590, pulls from these
registries would never use the v1 protocol, because of the
Docker-Distribution-Api-Version header indicating that v2 was supported.
Fix the problem by making an exception for the case where a manifest is
not found. This should allow fallback to v1 in case that image is
exposed over the v1 protocol but not the v2 protocol.
This avoids the overly aggressive fallback behavior before #18590 which
would allow protocol fallback after almost any error, but restores
interoperability with mixed v1/v2 registry setups.
Fixes#18832
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 9d6acbee92016c47796ee8751dce9c59056f850d
Component: engine
If we detect a Docker-Distribution-Api-Version header indicating that
the registry speaks the V2 protocol, no fallback to V1 should take
place.
The same applies if a V2 registry operation succeeds while attempting a
push or pull.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: a57478d65f1f5782cc746c260f9d841a0907ce54
Component: engine
This commit adds a transfer manager which deduplicates and schedules
transfers, and also an upload manager and download manager that build on
top of the transfer manager to provide high-level interfaces for uploads
and downloads. The push and pull code is modified to use these building
blocks.
Some benefits of the changes:
- Simplification of push/pull code
- Pushes can upload layers concurrently
- Failed downloads and uploads are retried after backoff delays
- Cancellation is supported, but individual transfers will only be
cancelled if all pushes or pulls using them are cancelled.
- The distribution code is decoupled from Docker Engine packages and API
conventions (i.e. streamformatter), which will make it easier to split
out.
This commit also includes unit tests for the new distribution/xfer
package. The tests cover 87.8% of the statements in the package.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 572ce802306a4e919802e5b77cbeca94acda7c0a
Component: engine
This test was directly comparing lines of output from "docker images".
Sometimes, when busybox had been pushed to the hub recently, the
relative creation times would differ like this:
... obtained []string = []string{"busybox", "latest", "d9551b4026f0", "27", "minutes", "ago", "1.113", "MB"}
... expected []string = []string{"busybox", "latest", "d9551b4026f0", "26", "minutes", "ago", "1.113", "MB"}
Fixing by removing the time-since-creation fields from the comparison.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: d17669999fff55219f5ed236f00fdf8a08d09304
Component: engine
Add distribution package for managing pulls and pushes. This is based on
the old code in the graph package, with major changes to work with the
new image/layer model.
Add v1 migration code.
Update registry, api/*, and daemon packages to use the reference
package's types where applicable.
Update daemon package to use image/layer/tag stores instead of the graph
package
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: 4352da7803d182a6013a5238ce20a7c749db979a
Component: engine
Generate a hash chain involving the image configuration, layer digests,
and parent image hashes. Use the digests to compute IDs for each image
in a manifest, instead of using the remotely specified IDs.
To avoid breaking users' caches, check for images already in the graph
under old IDs, and avoid repulling an image if the version on disk under
the legacy ID ends up with the same digest that was computed from the
manifest for that image.
When a calculated ID already exists in the graph but can't be verified,
continue trying SHA256(digest) until a suitable ID is found.
"save" and "load" are not changed to use a similar scheme. "load" will
preserve the IDs present in the tar file.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 504e67b867865a2835e8002c01087a2cfd7bfd0e
Component: engine
- utils_test.go and docker_utils_test.go
- Moved docker related function to docker_utils.go
- add a test for integration-cli/checker
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 5109071706cab06257b21172627ce51e5fa3a88c
Component: engine
Update and migrate existing tests to the `DockerHubPullSuite`. Most
tests were preserved, but refactored and made more exhaustive. One test
was deliberately removed (`TestPullVerified`) as it is unreliable and
that the feature was obsoleted by content trust.
Move all trust related tests to `docker_cli_pull_trusted_test.go`.
Move tests depending on a local registry to `docker_cli_pull_local_test.go`.
Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
Upstream-commit: f324f4851f28edfd8dab82cd624a4ec1f01cd207
Component: engine
Currently some notary tests change the system clock to check for expiration.
Skip these tests until the code can be refactored to not rely on updating the system clock.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: bf3c1e6a3afc951e41643b8d55d4ac25fa9cb06d
Component: engine
Update help line to allow 90 characters instead of 80
The trust flag pushes out the help description column wider, requiring more room to display help messages.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: 259cadb0b19a9049302e78fec529b0105efabb1d
Component: engine
Clean up tests to remove duplicate code
Add tests which run pull and create in an isolated configuration directory.
Add build test for untrusted tag
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: 871d2b96ed5cf234c41a5e731a34fc9deda4e9f1
Component: engine
Created date util function
Signed-off-by: Nathan McCauley <nathan.mccauley@docker.com>
Upstream-commit: 1406cb35fd814a987b91f92ed1a4b61a21205736
Component: engine
Added notary server to docker base image.
Created trust suite which runs trust server for running trusted commands.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: 58a1de9b59594948df152f0003e759b77bcaa56a
Component: engine
To avoid manually creating and destroying registrys in tests.
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
Upstream-commit: f696b1071a296bee1f4ac7cafa807ce337fb9f2c
Component: engine
It prints test name and duration for each test.
Also performs deleteAllContainers after each test.
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
Upstream-commit: dc944ea7e48d11a2906e751d3e61daf08faee054
Component: engine
