If a reference passed to the pull code contains both a tag and a digest,
currently the tag is used instead of the digest in the request to the
registry. This is the wrong behavior. Change it to favor the digest.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 0bff591bb0bc5a11ec22eb4f0b6104a79dea0819
Component: engine
The diff id resolution currently relies on a stored mapping for
archive digest to diff id. This mapping could be derived from
the image configuration if the image configuration is available.
On linux the image config is pulled in parallel and may not be
available. On windows, however, it is always pulled first and can
be used to supplement the stored mapping for images which may not
have this mapping from being side loaded. This becomes useful when
combined with side loaded foreign layers.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: 633f9252b8e066ef7a1a738ddc84c3970379844e
Component: engine
Without this fix the error the client might see is:
target is unknown
which wasn't helpful to me when I saw this today. With this fix I
now see:
MediaType is unknown: 'text/html'
which helped me track down the issue to the registry I was talking to.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Upstream-commit: c127d9614f5b30bd73861877f8540a63e7d869e9
Component: engine
Per request for more debug info on how the engine deals with
multi-platform "manifest list" images, this adds information about the
manifest list entries and whether it found an os/arch match, and the
digest of the match.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
Upstream-commit: 9a8cb9313c9c5eb535cf0bce25ca27f84bbfc570
Component: engine
Fallback errors are not an error, but an
informational message.
This changes those errors to be logged
as "Info" instead of "Error".
After this patch, debug logs look like this;
DEBU[0050] Calling GET /_ping
DEBU[0050] Calling POST /v1.27/images/create?fromImage=localhost%3A5000%2Ffoo&tag=latest
DEBU[0050] Trying to pull localhost:5000/foo from https://localhost:5000 v2
WARN[0050] Error getting v2 registry: Get https://localhost:5000/v2/: http: server gave HTTP response to HTTPS client
INFO[0050] Attempting next endpoint for pull after error: Get https://localhost:5000/v2/: http: server gave HTTP response to HTTPS client
DEBU[0050] Trying to pull localhost:5000/foo from http://localhost:5000 v2
INFO[0050] Attempting next endpoint for pull after error: manifest unknown: manifest unknown
DEBU[0050] Trying to pull localhost:5000/foo from https://localhost:5000 v1
DEBU[0050] attempting v1 ping for registry endpoint https://localhost:5000/v1/
DEBU[0050] Fallback from error: Get https://localhost:5000/v1/_ping: http: server gave HTTP response to HTTPS client
INFO[0050] Attempting next endpoint for pull after error: Get https://localhost:5000/v1/_ping: http: server gave HTTP response to HTTPS client
DEBU[0050] Trying to pull localhost:5000/foo from http://localhost:5000 v1
DEBU[0050] [registry] Calling GET http://localhost:5000/v1/repositories/foo/images
ERRO[0050] Not continuing with pull after error: Error: image foo:latest not found
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 86061441593f8c768781681db2268bc1ab6d043e
Component: engine
Remove forked reference package. Use normalized named values
everywhere and familiar functions to convert back to familiar
strings for UX and storage compatibility.
Enforce that the source repository in the distribution metadata
is always a normalized string, ignore invalid values which are not.
Update distribution tests to use normalized values.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: 3a1279393faf78632bf169619d407e584da84b66
Component: engine
fix typo I found AMAP in integration-cli/*
fix typo mentioned by Allencloud
Signed-off-by: Aaron.L.Xu <likexu@harmonycloud.cn>
Upstream-commit: 40af5691648c5b9d07b1231e3ed3be29fd66521a
Component: engine
The `digest` data type, used throughout docker for image verification
and identity, has been broken out into `opencontainers/go-digest`. This
PR updates the dependencies and moves uses over to the new type.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: 7a855799175b6b984886ef1cfa337d6df1d4c668
Component: engine
Move configurations into a single file.
Abstract download manager in pull config.
Add supports for schema2 only and schema2 type checking.
Add interface for providing push layers.
Abstract image store to generically handle configurations.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: 3c7676a057a4c0103895f793e407dc6736df139a
Component: engine
Remove the following comment in pullV2Tag:
// NOTE: not using TagService.Get, since it uses HEAD requests
// against the manifests endpoint, which are not supported by
// all registry versions.
This is actually not an issue, because TagService.Get does a fallback to
GET if HEAD fails. It has done this ever since TagService was added to
the distribution API, so this comment was probably based on an early
version of TagService before it was merged, or was always a
misunderstanding.
However, we continue to use ManifestService.Get instead because it
saves a round trip. The manifest can be retrieved directly instead of
resolving the digest first.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 99c59d5988d20a6722224da918fdec299d3aaded
Component: engine
Windows base layers are no longer the special "layers+base" type, so we can remove all the special handling for that.
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
Upstream-commit: f342b27145d8f5af27cd5de1501551af275e899b
Component: engine
Always attempt to add digest even when tag already exists.
Ensure digest does not currently exist.
When image id is mismatched, output an error log.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: 33984f256b1a281b1130ac7e8edb7bc311750ccf
Component: engine
This fix tries to fix logrus formatting by removing `f` from
`logrus.[Error|Warn|Debug|Fatal|Panic|Info]f` when formatting string
is not present.
This fix fixes#23459.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: a72b45dbec3caeb3237d1af5aedd04adeb083571
Component: engine
Replace use of foreign sources with descriptors and describable
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: 2c60430a3d1431e0879aa1c66ca23143de987b35
Component: engine
This is used to support downloading Windows base images from Microsoft
servers.
Signed-off-by: John Starks <jostarks@microsoft.com>
Upstream-commit: 05bd04350b8348b3c3bbe3156420257313e4e804
Component: engine
@nwt noticed that the media type specified in the config section of a
schema2 manifest is application/octet-stream, instead of the correct
value application/vnd.docker.container.image.v1+json.
This brings in https://github.com/docker/distribution/pull/1622 to fix
this.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: c18d03a7783bb78b7568bf3c23644888995e9a5d
Component: engine
Now that we are checking if the image and host have the same architectures
via #21272, this value should be null so that the test passes on non-x86
machines
Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
Upstream-commit: 1f59bc8c03df18686b93a0cd619cf2c55cbcf421
Component: engine
These fields are needed to specify the exact version of Windows that an
image can run on. They may be useful for other platforms in the future.
This also changes image.store.Create to validate that the loaded image is
supported on the current machine. This change affects Linux as well, since
it now validates the architecture and OS fields.
Signed-off-by: John Starks <jostarks@microsoft.com>
Upstream-commit: 194eaa5c0f843257e66b68bd735786308a9d93b2
Component: engine
Close could be called twice on a temporary download file, which could
have bad side effects.
This fixes the problem by setting to ld.tmpFile to nil when the download
completes sucessfully. Then the call to ld.Close will have no effect,
and only the download manager will close the temporary file when it's
done extracting the layer from it. ld.Close will be responsible for
closing the file if we hit the retry limit and there is still a partial
download present.
Fixes#21675
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 930ae3dbcb11091955ca936c280d13f24494b245
Component: engine
With the --insecure-registry daemon option (or talking to a registry on
a local IP), the daemon will first try TLS, and then try plaintext if
something goes wrong with the push or pull. It doesn't make sense to try
plaintext if a HTTP request went through while using TLS. This commit
changes the logic to keep track of host/port combinations where a TLS
attempt managed to do at least one HTTP request (whether the response
code indicated success or not). If the host/port responded to a HTTP
using TLS, we won't try to make plaintext HTTP requests to it.
This will result in better error messages, which sometimes ended up
showing the result of the plaintext attempt, like this:
Error response from daemon: Get
http://myregistrydomain.com:5000/v2/: malformed HTTP response
"\x15\x03\x01\x00\x02\x02"
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 5e8af46fda3f4e17e06726237fc6b9ab6957e3ea
Component: engine
Several improvements to error handling:
- Introduce ImageConfigPullError type, wrapping errors related to
downloading the image configuration blob in schema2. This allows for a
more descriptive error message to be seen by the end user.
- Change some logrus.Debugf calls that display errors to logrus.Errorf.
Add log lines in the push/pull fallback cases to make sure the errors
leading to the fallback are shown.
- Move error-related types and functions which are only used by the
distribution package out of the registry package.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 8f26fe4f59ce515c68440da1443ace4c96e89d4a
Component: engine
This makes the behavior consistent with having incorrect credentials.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 7b81bc147cf75cb32697e8fdf88e05ae879cb879
Component: engine
This will allow it to be reused between download attempts in a
subsequent commit.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: f425529e7e0a6b15c8cc43f0c1dbb7a42572e30d
Component: engine
Currently, the temporary file storing downloaded layer data is only
removed after a successful download or a digest verification error. A
transport-level error does not cause it to be removed. This is a
regression from 1.9 that could cause disk usage to grow until the Docker
daemon is restarted.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 5a363ce60bee3dc26a433c7e2cee6dc76939849e
Component: engine
Also renames BlobSumService to V2MetadataService, BlobSum to
V2Metadata
Signed-off-by: Brian Bland <brian.bland@docker.com>
Upstream-commit: 63099477189ea14f3122f6aa37fa7c60d33562c7
Component: engine
Tracks source repository information for each blob in the blobsum
service, which is then used to attempt to mount blobs from another
repository when pushing instead of having to re-push blobs to the same
registry.
Signed-off-by: Brian Bland <brian.bland@docker.com>
Upstream-commit: 7289c7218e2101eb94fb90f2cb22e1412d016984
Component: engine
