This patch splits LookupEndpoints into LookupPullEndpoints and
LookupPushEndpoints so that mirrors added with --registry-mirror are
skipped in the list returned by LookupPushEndpoints.
Fixes https://github.com/docker/distribution/issues/823
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: b899977ee283d6bccf35ad9a079ed92a291ad34c
Component: engine
The process of pulling an image spawns a new goroutine for each layer in the
image manifest. If any of these downloads fail we would stop everything and
return the error, even though other goroutines would still be running and
writing output through a progress reader which is attached to an http response
writer. Since the request handler had already returned from the first error,
the http server panics when one of these download goroutines makes a write to
the response writer buffer.
This patch prevents this crash in the daemon http server by waiting for all of
the download goroutines to complete, even if one of them fails. Only then does
it return, terminating the request handler.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Upstream-commit: d80c4244d322fe0a9cbdd996d23e37fb5b089544
Component: engine
We noticed a regression since the 1.7.1 patch after some refactoring. This
patch corrects the behavior and adds integration tests for modified manifest
and rootfs layer blobs.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Upstream-commit: de52a3bcaa1742054be184cc3465f0933f6d383a
Component: engine
Some structures use int for sizes and UNIX timestamps. On some
platforms, int is 32 bits, so this can lead to the year 2038 issues and
overflows when dealing with large containers or layers.
Consistently use int64 to store sizes and UNIX timestamps in
api/types/types.go. Update related to code accordingly (i.e.
strconv.FormatInt instead of strconv.Itoa).
Use int64 in progressreader package to avoid integer overflow when
dealing with large quantities. Update related code accordingly.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 1f61084d83aea37b212468aaa975020094b7f7c9
Component: engine
Expand the godoc documentation for the graph package.
Centralize DefaultTag in the graphs/tag package instead of defining it
twice.
Remove some unnecessary "config" structs that are only used to pass
a few parameters to a function.
Simplify the GetParentsSize function - there's no reason for it to take
an accumulator argument.
Unexport some functions that aren't needed outside the package.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: d4836cd7ec1c085c5a5caa7eb7f5eda4ace10eb6
Component: engine
TL;DR: check for IsExist(err) after a failed MkdirAll() is both
redundant and wrong -- so two reasons to remove it.
Quoting MkdirAll documentation:
> MkdirAll creates a directory named path, along with any necessary
> parents, and returns nil, or else returns an error. If path
> is already a directory, MkdirAll does nothing and returns nil.
This means two things:
1. If a directory to be created already exists, no error is returned.
2. If the error returned is IsExist (EEXIST), it means there exists
a non-directory with the same name as MkdirAll need to use for
directory. Example: we want to MkdirAll("a/b"), but file "a"
(or "a/b") already exists, so MkdirAll fails.
The above is a theory, based on quoted documentation and my UNIX
knowledge.
3. In practice, though, current MkdirAll implementation [1] returns
ENOTDIR in most of cases described in #2, with the exception when
there is a race between MkdirAll and someone else creating the
last component of MkdirAll argument as a file. In this very case
MkdirAll() will indeed return EEXIST.
Because of #1, IsExist check after MkdirAll is not needed.
Because of #2 and #3, ignoring IsExist error is just plain wrong,
as directory we require is not created. It's cleaner to report
the error now.
Note this error is all over the tree, I guess due to copy-paste,
or trying to follow the same usage pattern as for Mkdir(),
or some not quite correct examples on the Internet.
[v2: a separate aufs commit is merged into this one]
[1] https://github.com/golang/go/blob/f9ed2f75/src/os/path.go
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Upstream-commit: a83a76934787a20e96389d33bd56a09369f9b808
Component: engine
This allow us to avoid entropy usage in non-crypto critical places.
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
Upstream-commit: 6bca8ec3c9ccc169c53b3d7060fe5c8ba8670aac
Component: engine
In 1.6.2 we were decoding inspect API response into interface{}.
time.Time fields were JSON encoded as RFC3339Nano in the response
and when decoded into interface{} they were just strings so the inspect
template treated them as just strings.
From 1.7 we are decoding into types.ContainerJSON and when the template
gets executed it now gets a time.Time and it's formatted as
2015-07-22 05:02:38.091530369 +0000 UTC.
This patch brings back the old behavior by typing time.Time fields
as string so they gets formatted as they were encoded in JSON -- RCF3339Nano
Signed-off-by: Antonio Murdaca <runcom@linux.com>
Upstream-commit: c9207bc0aa57745876a3422d2cbc290be7c53da8
Component: engine
Add a trusted flag to force the cli to resolve a tag into a digest via the notary trust library and pull by digest.
On push the flag the trust flag will indicate the digest and size of a manifest should be signed and push to a notary server.
If a tag is given, the cli will resolve the tag into a digest and pull by digest.
After pulling, if a tag is given the cli makes a request to tag the image.
Use certificate directory for notary requests
Read certificates using same logic used by daemon for registry requests.
Catch JSON syntax errors from Notary client
When an uncaught error occurs in Notary it may show up in Docker as a JSON syntax error, causing a confusing error message to the user.
Provide a generic error when a JSON syntax error occurs.
Catch expiration errors and wrap in additional context.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: ed13c3abfb242905ec012e8255dc6f26dcf122f6
Component: engine
* Add godoc documentation where it was missing
* Change identifier names that don't match Go style, such as INDEX_NAME
* Rename RegistryInfo to PingResult, which more accurately describes
what this structure is for. It also has the benefit of making the name
not stutter if used outside the package.
Updates #14756
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 4fcb9ac40ce33c4d6e08d5669af6be5e076e2574
Component: engine
pushV2Tag already deduplicates layers, but the scope of this
deduplication is only for a particular tag. If we are pushing all tags
in a repository, we may check layers several times. Fix this by moving
the layersSeen map from the pushV2Tag function to the v2Pusher struct.
In addition to avoiding some useless round-trips, this makes the "docker
push" output less confusing. It formerly could contain many repeated
lines like:
124e2127157f: Image already exists
124e2127157f: Image already exists
...
Add test coverage based on the "docker push" output: a hash should not
appear multiple times when pushing multiple tags.
Fixes#14873
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 810d3b2642d4a82dc186d6eff8c2e487ee624bc5
Component: engine
with the current duplication of code in the grap.go split-up, this puts
all assembly/disassembly logic into isolated functions
Signed-off-by: Vincent Batts <vbatts@redhat.com>
Upstream-commit: 22347fdb636c0b7936c87f6cf422a2e4a30adf9e
Component: engine
if there is a tar-data.json.gz present for an image layer, then use it
to create the tar archive, instead of the traditional graphdriver Diff.
Signed-off-by: Vincent Batts <vbatts@redhat.com>
Conflicts:
graph/graph.go
Upstream-commit: 5a00326d29efb161826ef13dfd63ed2732017cd1
Component: engine
and add a comment.. :-)
Signed-off-by: Vincent Batts <vbatts@redhat.com>
Conflicts:
graph/graph.go
Upstream-commit: ba1f76cbfa2c137abfbc607725460e376e6f44d3
Component: engine
Preserve the entries from the tar archive for layers added to the graph.
With these entries and relative filesystem path, the tar archives can be
reassembled later.
Signed-off-by: Vincent Batts <vbatts@redhat.com>
Upstream-commit: 5d9f06599c4f0cde9ad266dff77e797ea99c3ac3
Component: engine
Currently canonical name gets set to the local name and displayed in the errors.
Canonical name should be the unique and canonical name for an image.
Use docker.io as the canonical domain for images on the public registry.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: 7f48cd7dce6fdc077bcde0962e0aa0e73fb63225
Component: engine
Currently the layer array is initialized with the first layer then the first layer is appened to the layer list. Adding the first layer twice causes the layer to appear twice in the manifest, making a duplicate push and pull attempt occur.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: 35081ea4b6711b1cfccb67b0f0d691b7adc85ff6
Component: engine
Related to #11618 and #11614
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 10e114fb956db1b1a8bc9308cc6d14cbf30a5bab
Component: engine
During `(*Graph).Register, there was no protection on adding new layers
concurrently. In some cases, this resulted in corruption of a layer by creating
the directory but not the underlying data. This manifested in several different
IO errors reported in the client. This attempts to fix this by adding a mutex
by Image ID to protect the Register operation.
We do not completely understand the root cause of this corruption other than
the result is somehow tied to this particular function. This fix has been
confirmed to address the issue through testing.
Unfortunately, this fix does not address existing corruption. The user will
have to remove and re-pull the corrupt layer to stop the error from happening
in the future. This change only ensures that the layer will not become corrupt.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: 7eac23cf8de08f281c84c36390f734dbaef8c69a
Component: engine
An inspection of the graph package showed this function to be way out of place.
It is only depended upon by the daemon code. The function prepares a top-level
readonly layer used to provide a consistent runtime environment for docker
images.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: b7f887a9a24a2df61fa53612ae42b7e9d62fe858
Component: engine
Splitting out GetRemoteTag from GetRemoteTags. Adding registry.ErrRepoNotFound error
Signed-off-by: Don Kjer <don.kjer@gmail.com>
Upstream-commit: b349a74c71fb072f9f23f508b8c698d0590abb12
Component: engine
