Translate pull errors to provide a more consistent and user friendly
error message.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: 19a93a6e3d4213c56583bb0c843cf9e33d379752
Component: engine
Remove the following comment in pullV2Tag:
// NOTE: not using TagService.Get, since it uses HEAD requests
// against the manifests endpoint, which are not supported by
// all registry versions.
This is actually not an issue, because TagService.Get does a fallback to
GET if HEAD fails. It has done this ever since TagService was added to
the distribution API, so this comment was probably based on an early
version of TagService before it was merged, or was always a
misunderstanding.
However, we continue to use ManifestService.Get instead because it
saves a round trip. The manifest can be retrieved directly instead of
resolving the digest first.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 99c59d5988d20a6722224da918fdec299d3aaded
Component: engine
Allow built images to be squash to scratch.
Squashing does not destroy any images or layers, and preserves the
build cache.
Introduce a new CLI argument --squash to docker build
Introduce a new param to the build API endpoint `squash`
Once the build is complete, docker creates a new image loading the diffs
from each layer into a single new layer and references all the parent's
layers.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 362369b4bbea38881402d281ee2015d16e8b10ce
Component: engine
This method return a map associating a chain ID to its layer object
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
Upstream-commit: 148aef9199ef0af6d03ea53e616c9fbd23b7c5ec
Component: engine
Classify blobs into three categories based on size.
Use a very limited number of mount attempts and no existence check for
small blobs. Use more attempts for bigger blobs.
Also remember blob associations during layer existence check.
Blob digests are now checked in the target repository from newest to
latest. If the blob exists and the metadata entry does not, it will be
created. If the blob is not found, the metadata entry will be removed.
Signed-off-by: Michal Minář <miminar@redhat.com>
Upstream-commit: 81f7b1f1e50252e9fdd52111f0899d59f19ecb87
Component: engine
Don't fallback back immediately to blob upload if the cross-repo mount
fails and layer upload is initiated by registry.
Instead cancel the upload and re-try cross-repo mount from different
source repository before doing full re-upload.
Signed-off-by: Michal Minář <miminar@redhat.com>
Upstream-commit: c6dd51c32cfccc06e77b4f7cb2358f788753df72
Component: engine
to avoid unnecessary blob re-uploads.
Cross repository mount from particular repo will most probably fail if
the user pushing to the registry is not the same as the one who pulled
or pushed to the source repo.
This PR attempts first to cross-repo mount from the source repositories
associated with the pusher's auth config. Then it falls back to other
repositories sorted from the most similar to the target repo to the
least.
It also prevents metadata deletion in cases where cross-repo mount fails
and the auth config hashes differ.
Signed-off-by: Michal Minář <miminar@redhat.com>
Upstream-commit: 0928f3f2e3eda75a295b651d27f9dd992fd951a4
Component: engine
Write the foreign layer URLs being downloaded into the debug log.
Signed-off-by: John Starks <jostarks@microsoft.com>
Upstream-commit: 6b7d028085e0e6ac0c5f224f0a493839e2beeba3
Component: engine
This moves the types for the `engine-api` repo to the existing types
package.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Upstream-commit: 91e197d614547f0202e6ae9b8a24d88ee131d950
Component: engine
Windows base layers are no longer the special "layers+base" type, so we can remove all the special handling for that.
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
Upstream-commit: f342b27145d8f5af27cd5de1501551af275e899b
Component: engine
Ensure that cancellation of a pull propagates rather than continuing to
container creation. This ensures that the `Prepare` method is properly
re-entrant.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: d99c6b837ffd18ffe5bce801feb4936bf0edd2aa
Component: engine
This fixes the hard coded restriction for non-linux platforms to v2 registries. Previously, the check was above the flag parsing, which would overwrite the hard coded value and prevent correct operation. This change also removes the related daemon flag from Windows to avoid confusion, as it has no meaning when the value is going to always be hard coded to true.
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
Upstream-commit: adee28458c23581ac9afb163b7cce8c6bb1d2dee
Component: engine
Updates the v1 search endpoint to also support v2 auth when an identity token is given.
Only search v1 endpoint is supported since there is not v2 search currently defined to replace it.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: 19d48f0b8ba59eea9f2cac4ad1c7977712a6b7ac
Component: engine
If the remote registry responds with a NAME_UNKNOWN error, treat this as
a fatal error and don't retry the push.
Tested against an ECR registry.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: a12ab28e0afb8dd0a05187989dd95c9b9ca3fd68
Component: engine
Always attempt to add digest even when tag already exists.
Ensure digest does not currently exist.
When image id is mismatched, output an error log.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: 33984f256b1a281b1130ac7e8edb7bc311750ccf
Component: engine
Layer uploads are deduplicated by a "key" made up of the layer DiffID
and the repository name. The repository name being used to form this key
was a remote version of the name that didn't include the name of the
registry. Consequently, pushes of the same layer in a repository with
the same remote name to different registries would wrongly be
deduplicated.
Correct the key by using the full name of the repository, which includes
the registry hostname as well as the image's name.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 1333ef3ca381976438d4ae10f6b0e9be21e1a9ba
Component: engine
This patch introduces a new experimental engine-level plugin management
with a new API and command line. Plugins can be distributed via a Docker
registry, and their lifecycle is managed by the engine.
This makes plugins a first-class construct.
For more background, have a look at issue #20363.
Documentation is in a separate commit. If you want to understand how the
new plugin system works, you can start by reading the documentation.
Note: backwards compatibility with existing plugins is maintained,
albeit they won't benefit from the advantages of the new system.
Signed-off-by: Tibor Vass <tibor@docker.com>
Signed-off-by: Anusha Ragunathan <anusha@docker.com>
Upstream-commit: f37117045c5398fd3dca8016ea8ca0cb47e7312b
Component: engine
This fix tries to fix logrus formatting by removing `f` from
`logrus.[Error|Warn|Debug|Fatal|Panic|Info]f` when formatting string
is not present.
This fix fixes#23459.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: a72b45dbec3caeb3237d1af5aedd04adeb083571
Component: engine
Replace use of foreign sources with descriptors and describable
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: 2c60430a3d1431e0879aa1c66ca23143de987b35
Component: engine
This is used to support downloading Windows base images from Microsoft
servers.
Signed-off-by: John Starks <jostarks@microsoft.com>
Upstream-commit: 05bd04350b8348b3c3bbe3156420257313e4e804
Component: engine
… and refactor a little bit some daemon on the way.
- Move `SearchRegistryForImages` to a new file (`daemon/search.go`) as
`daemon.go` is getting pretty big.
- `registry.Service` is now an interface (allowing us to decouple it a
little bit and thus unit test easily).
- Add some unit test for `SearchRegistryForImages`.
- Use UniqueExactMatch for search filters
- And use empty restore id for now in client.ContainerStart.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 636c276f67b3cd96a95dec2f6cfc419b7f219892
Component: engine
This fix tries to address issues raised in #20936 and #22443
where `docker pull` or `docker push` fails because of the
concurrent connection failing.
Currently, the number of maximum concurrent connections is
controlled by `maxDownloadConcurrency` and `maxUploadConcurrency`
which are hardcoded to 3 and 5 respectively. Therefore, in
situations where network connections don't support multiple
downloads/uploads, failures may encounter for `docker push`
or `docker pull`.
This fix tries changes `maxDownloadConcurrency` and
`maxUploadConcurrency` to adjustable by passing
`--max-concurrent-uploads` and `--max-concurrent-downloads` to
`docker daemon` command.
The documentation related to docker daemon has been updated.
Additional test case have been added to cover the changes in this fix.
This fix fixes#20936. This fix fixes#22443.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 7368e41c07c21a1e2c6a49abecd1c2fc76404e49
Component: engine
Use sockets.DialerFromEnvironment, as is done in other places,
to transparently support SOCKS proxy config from ALL_PROXY
environment variable.
Requires the *engine* have the ALL_PROXY env var set, which
doesn't seem ideal. Maybe it should be a CLI option somehow?
Only tested with push and a v2 registry so far. I'm happy to look
further into testing more broadly, but I wanted to get feedback on
the general idea first.
Signed-off-by: Brett Higgins <brhiggins@arbor.net>
Upstream-commit: 207027087e71f5587ca407e20cc252f7958bc1b3
Component: engine
@nwt noticed that the media type specified in the config section of a
schema2 manifest is application/octet-stream, instead of the correct
value application/vnd.docker.container.image.v1+json.
This brings in https://github.com/docker/distribution/pull/1622 to fix
this.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: c18d03a7783bb78b7568bf3c23644888995e9a5d
Component: engine
