Allow v1 protocol fallback when pulling all tags from a repository unknown to v2 registry
Upstream-commit: 6c30931b06ff44e1ce002002430577b7135d2049
Component: engine
Move NoBaseImageSpecifier to a common spot and then use it instead of
"scratch" in a couple of places.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Upstream-commit: e6806223e81c916c9b24580b19207271f1a36965
Component: engine
- Stop serializing JSONMessage in favor of events.Message.
- Keep backwards compatibility with JSONMessage for container events.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Upstream-commit: 72f1881df102fce9ad31e98045b91c204dd44513
Component: engine
Support restoreCustomImage for windows with a new interface to extract
the graph driver from the LayerStore.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: f5916b10ae02c7db83052a97205ac345a3d96300
Component: engine
This is a followup to #18839. That PR relaxed the fallback logic so that
if a manifest doesn't exist on v2, or the user is unauthorized to access
it, we try again with the v1 protocol. A similar special case is needed
for "pull all tags" (docker pull -a). If the v2 registry doesn't
recognize the repository, or doesn't allow the user to access it, we
should fall back to v1 and try to pull all tags from the v1 registry.
Conversely, if the v2 registry does allow us to list the tags, there
should be no fallback, even if there are errors pulling those tags.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 589a5226e7d4880ab7afa9e31202849db08a4bfa
Component: engine
RWLayer will now have more operations and be protected through a referenced type rather than always looked up by string in the layer store.
Separates creation of RWLayer (write capture layer) from mounting of the layer.
This allows mount labels to be applied after creation and allowing RWLayer objects to have the same lifespan as a container without performance regressions from requiring mount.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: d04fa49a0dec89d2f71a813ce8eaa182184139c5
Component: engine
PR #18590 caused compatibility issues with registries such as gcr.io
which support both the v1 and v2 protocols, but do not provide the same
set of images over both protocols. After #18590, pulls from these
registries would never use the v1 protocol, because of the
Docker-Distribution-Api-Version header indicating that v2 was supported.
Fix the problem by making an exception for the case where a manifest is
not found. This should allow fallback to v1 in case that image is
exposed over the v1 protocol but not the v2 protocol.
This avoids the overly aggressive fallback behavior before #18590 which
would allow protocol fallback after almost any error, but restores
interoperability with mixed v1/v2 registry setups.
Fixes#18832
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 9d6acbee92016c47796ee8751dce9c59056f850d
Component: engine
- Use layer DiffIDs for progress output in v1 push. This makes the
output consistent with v2 pushes, which means that a fallback to v1
won't start progress bars for a different set of IDs.
- Change wording used in v1 status updates to be consistent with v2.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 891bbc17ec2623897e32a3b9116cb2047ba70fdd
Component: engine
If we detect a Docker-Distribution-Api-Version header indicating that
the registry speaks the V2 protocol, no fallback to V1 should take
place.
The same applies if a V2 registry operation succeeds while attempting a
push or pull.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: a57478d65f1f5782cc746c260f9d841a0907ce54
Component: engine
Add function to get metadata from layer store for a mutable layer
fixes#18614
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: a7e096832123280d26df3c121ecad8dd012060b9
Component: engine
This commit adds a transfer manager which deduplicates and schedules
transfers, and also an upload manager and download manager that build on
top of the transfer manager to provide high-level interfaces for uploads
and downloads. The push and pull code is modified to use these building
blocks.
Some benefits of the changes:
- Simplification of push/pull code
- Pushes can upload layers concurrently
- Failed downloads and uploads are retried after backoff delays
- Cancellation is supported, but individual transfers will only be
cancelled if all pushes or pulls using them are cancelled.
- The distribution code is decoupled from Docker Engine packages and API
conventions (i.e. streamformatter), which will make it easier to split
out.
This commit also includes unit tests for the new distribution/xfer
package. The tests cover 87.8% of the statements in the package.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 572ce802306a4e919802e5b77cbeca94acda7c0a
Component: engine
The v1 push code was querying the size of the layer chain up to the
layer it was pushing, rather than just that layer. This made the
progress indicator inaccurate.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 741924384ee7a617af1a0275e2dc674e86b9d0d9
Component: engine
We were calling Stat for each layer to get the size so we could indicate
progress, but https://github.com/docker/distribution/pull/1226 made it
possible to get the length from the GET request that Open initiates.
Saving one round-trip per layer should make pull operations slightly
faster and more robust.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 39589800b4750bf28078efe57f7f1e74d971248f
Component: engine
This change allows API clients to retrieve an authentication token from
a registry, and then pass that token directly to the API.
Example usage:
REPO_USER=dhiltgen
read -s PASSWORD
REPO=privateorg/repo
AUTH_URL=https://auth.docker.io/token
TOKEN=$(curl -s -u "${REPO_USER}:${PASSWORD}" "${AUTH_URL}?scope=repository:${REPO}:pull&service=registry.docker.io" |
jq -r ".token")
HEADER=$(echo "{\"registrytoken\":\"${TOKEN}\"}"|base64 -w 0 )
curl -s -D - -H "X-Registry-Auth: ${HEADER}" -X POST "http://localhost:2376/images/create?fromImage=${REPO}"
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 8dce8e9901501863a80caf47d71713d4d36925a3
Component: engine
Currently, the resources associated with the io.Reader returned by
TarStream are only freed when it is read until EOF. This means that
partial uploads or exports (for example, in the case of a full disk or
severed connection) can leak a goroutine and open file. This commit
changes TarStream to return an io.ReadCloser. Resources are freed when
Close is called.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 21278efaee563b356851a530b08b0537fee095d7
Component: engine
