Also remove unused func `newTempFile` and prevent a possible deadlock
between pull_v2 `attemptIDReuse` and graph `register`
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
Upstream-commit: f6577be1c93150149c291f9d18375d7bcae9ebb1
Component: engine
Ensure v2 registries are given more than 5 seconds to return a ping and avoid an unnecessary fallback to v1.
Elevates log level about failed v2 ping to a warning to match the warning related to using v1 registries.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: f8ea4ad16d51f6f062be2c250be26d731b92bd3c
Component: engine
Add a unit test for validateManifest which ensures extra data can't be
injected by adding data to the JSON object outside the payload area.
This also removes validation of legacy signatures at pull time. This
starts the path of deprecating legacy signatures, whose presence in the
very JSON document they attempt to sign is problematic. These
signatures were only checked for official images, and since they only
caused a weakly-worded message to be printed, removing the verification
should not cause impact.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 011bfd666eeb21a111ca450c42a3893ad03c9324
Component: engine
Generate a hash chain involving the image configuration, layer digests,
and parent image hashes. Use the digests to compute IDs for each image
in a manifest, instead of using the remotely specified IDs.
To avoid breaking users' caches, check for images already in the graph
under old IDs, and avoid repulling an image if the version on disk under
the legacy ID ends up with the same digest that was computed from the
manifest for that image.
When a calculated ID already exists in the graph but can't be verified,
continue trying SHA256(digest) until a suitable ID is found.
"save" and "load" are not changed to use a similar scheme. "load" will
preserve the IDs present in the tar file.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 504e67b867865a2835e8002c01087a2cfd7bfd0e
Component: engine
progressreader.Broadcaster becomes broadcaster.Buffered and
broadcastwriter.Writer becomes broadcaster.Unbuffered.
The package broadcastwriter is thus renamed to broadcaster.
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 2391233404e2e6892c79a24f31cc99715c086b21
Component: engine
Fix: Docker Daemon didn't send actual actions client ask for to Token Server
Upstream-commit: d25dada639fe475ff6496b1d357d2250a0e014e3
Component: engine
The original purpose of this was to cancel downloads if pullV2Tag
returns an error, preventing an associated crash (see #15353). The
broadcaster now accomplishes the same thing that the pipe does, making
the pipe redundant. When pullV2Tag returns, all broadcasters are closed,
which means all further writes to those broadcasters will return errors.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 50c355777e31c08d3a97ec70b67bbe5a3bc883af
Component: engine
Before, this only waited for the download to complete. There was no
guarantee that the layer had been registered in the graph and was ready
use. This is especially problematic with v2 pulls, which wait for all
downloads before extracting layers.
Change Broadcaster to allow an error value to be propagated from Close
to the waiters.
Make the wait stop when the extraction is finished, rather than just the
download.
This also fixes v2 layer downloads to prefix the pool key with "layer:"
instead of "img:". "img:" is the wrong prefix, because this is what v1
uses for entire images. A v1 pull waiting for one of these operations to
finish would only wait for that particular layer, not all its
dependencies.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 23e68679f080fee7ceb25cf791832f523a3a024a
Component: engine
Close the pipeWriter even if there was no error.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: 3037e4f7c4eec06cf3a5afca6f58ec58b4bdd63e
Component: engine
The Docker Daemon should send actual actions client ask for to issue tokens,
not all the permissions that client is guaranteed.
Signed-off-by: xiekeyang <xiekeyang@huawei.com>
Upstream-commit: e451ddbf4657cbe72f539e1f9513a5bc3a31f3d6
Component: engine
These don't get seen, so they are unnecessary.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 9b9d70ad271b46a67bde57b8a807121f6e85b75f
Component: engine
- Rename to Broadcaster
- Document exported types
- Change Wait function to just wait. Writing a message to the writer and
adding the writer to the observers list are now handled by separate
function calls.
- Avoid importing logrus (the condition where it was used should never
happen, anyway).
- Make writes non-blocking
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 26c9b585042ac7dce8db83478a69fd01a4b003d7
Component: engine
Previously, its other return value was used even when it returned an
error. This is awkward and goes against the convention. It also could
have resulted in a nil pointer dereference when an error was returned
because of an unknown pool type. This changes the unknown pool type
error to a panic (since the pool types are hardcoded at call sites and
must always be "push" or "pull"), and returns a "found" boolean instead
of an error.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 80513d85cfc0e46f8202fc3030f11052bbfeea7a
Component: engine
Based on #12874 from Sam Abed <sam.abed@gmail.com>. His original commit
was brought up to date by manually porting the changes in pull.go into
the new code in pull_v1.go and pull_v2.go.
Fixes#8385
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 572f008e892b06262963ccb75c631f2f5d6c6fcd
Component: engine
* use downloadInfo pointers everywhere
* use downloads slice only for things that we really download
* cleanup tmp files in all cases
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
Upstream-commit: 9b2426c606b92fa37b7484d9d884edd43352b69c
Component: engine
The process of pulling an image spawns a new goroutine for each layer in the
image manifest. If any of these downloads fail we would stop everything and
return the error, even though other goroutines would still be running and
writing output through a progress reader which is attached to an http response
writer. Since the request handler had already returned from the first error,
the http server panics when one of these download goroutines makes a write to
the response writer buffer.
This patch prevents this crash in the daemon http server by waiting for all of
the download goroutines to complete, even if one of them fails. Only then does
it return, terminating the request handler.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Upstream-commit: d80c4244d322fe0a9cbdd996d23e37fb5b089544
Component: engine
We noticed a regression since the 1.7.1 patch after some refactoring. This
patch corrects the behavior and adds integration tests for modified manifest
and rootfs layer blobs.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Upstream-commit: de52a3bcaa1742054be184cc3465f0933f6d383a
Component: engine
Some structures use int for sizes and UNIX timestamps. On some
platforms, int is 32 bits, so this can lead to the year 2038 issues and
overflows when dealing with large containers or layers.
Consistently use int64 to store sizes and UNIX timestamps in
api/types/types.go. Update related to code accordingly (i.e.
strconv.FormatInt instead of strconv.Itoa).
Use int64 in progressreader package to avoid integer overflow when
dealing with large quantities. Update related code accordingly.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 1f61084d83aea37b212468aaa975020094b7f7c9
Component: engine
Add a trusted flag to force the cli to resolve a tag into a digest via the notary trust library and pull by digest.
On push the flag the trust flag will indicate the digest and size of a manifest should be signed and push to a notary server.
If a tag is given, the cli will resolve the tag into a digest and pull by digest.
After pulling, if a tag is given the cli makes a request to tag the image.
Use certificate directory for notary requests
Read certificates using same logic used by daemon for registry requests.
Catch JSON syntax errors from Notary client
When an uncaught error occurs in Notary it may show up in Docker as a JSON syntax error, causing a confusing error message to the user.
Provide a generic error when a JSON syntax error occurs.
Catch expiration errors and wrap in additional context.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: ed13c3abfb242905ec012e8255dc6f26dcf122f6
Component: engine
