This file was not well documented and had very high cyclomatic complexity.
This patch completely rearranges this file and the ImageDelete method to
be easier to follow and more maintainable in the future.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Upstream-commit: 111d2f348767692a34366ff5cd01f607cf54ad57
Component: engine
The practice of buffering to a tempfile during a pushing contributes massively
to slow V2 push performance perception. The protocol was actually designed to
avoid precalculation, supporting cut-through data push. This means we can
assemble the layer, calculate its digest and push to the remote endpoint, all
at the same time.
This should increase performance massively on systems with slow disks or IO
bottlenecks.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: 349aef06d51a9425f70a877585fe45283366c2c4
Component: engine
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
Windows: add support for images stored in alternate location.
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
Upstream-commit: dfbb5520e3b35030f3eef38d5a2d86ad20ea0a2f
Component: engine
Some structures use int for sizes and UNIX timestamps. On some
platforms, int is 32 bits, so this can lead to the year 2038 issues and
overflows when dealing with large containers or layers.
Consistently use int64 to store sizes and UNIX timestamps in
api/types/types.go. Update related to code accordingly (i.e.
strconv.FormatInt instead of strconv.Itoa).
Use int64 in progressreader package to avoid integer overflow when
dealing with large quantities. Update related code accordingly.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 1f61084d83aea37b212468aaa975020094b7f7c9
Component: engine
Expand the godoc documentation for the graph package.
Centralize DefaultTag in the graphs/tag package instead of defining it
twice.
Remove some unnecessary "config" structs that are only used to pass
a few parameters to a function.
Simplify the GetParentsSize function - there's no reason for it to take
an accumulator argument.
Unexport some functions that aren't needed outside the package.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: d4836cd7ec1c085c5a5caa7eb7f5eda4ace10eb6
Component: engine
TL;DR: check for IsExist(err) after a failed MkdirAll() is both
redundant and wrong -- so two reasons to remove it.
Quoting MkdirAll documentation:
> MkdirAll creates a directory named path, along with any necessary
> parents, and returns nil, or else returns an error. If path
> is already a directory, MkdirAll does nothing and returns nil.
This means two things:
1. If a directory to be created already exists, no error is returned.
2. If the error returned is IsExist (EEXIST), it means there exists
a non-directory with the same name as MkdirAll need to use for
directory. Example: we want to MkdirAll("a/b"), but file "a"
(or "a/b") already exists, so MkdirAll fails.
The above is a theory, based on quoted documentation and my UNIX
knowledge.
3. In practice, though, current MkdirAll implementation [1] returns
ENOTDIR in most of cases described in #2, with the exception when
there is a race between MkdirAll and someone else creating the
last component of MkdirAll argument as a file. In this very case
MkdirAll() will indeed return EEXIST.
Because of #1, IsExist check after MkdirAll is not needed.
Because of #2 and #3, ignoring IsExist error is just plain wrong,
as directory we require is not created. It's cleaner to report
the error now.
Note this error is all over the tree, I guess due to copy-paste,
or trying to follow the same usage pattern as for Mkdir(),
or some not quite correct examples on the Internet.
[v2: a separate aufs commit is merged into this one]
[1] https://github.com/golang/go/blob/f9ed2f75/src/os/path.go
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Upstream-commit: a83a76934787a20e96389d33bd56a09369f9b808
Component: engine
This allow us to avoid entropy usage in non-crypto critical places.
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
Upstream-commit: 6bca8ec3c9ccc169c53b3d7060fe5c8ba8670aac
Component: engine
with the current duplication of code in the grap.go split-up, this puts
all assembly/disassembly logic into isolated functions
Signed-off-by: Vincent Batts <vbatts@redhat.com>
Upstream-commit: 22347fdb636c0b7936c87f6cf422a2e4a30adf9e
Component: engine
if there is a tar-data.json.gz present for an image layer, then use it
to create the tar archive, instead of the traditional graphdriver Diff.
Signed-off-by: Vincent Batts <vbatts@redhat.com>
Conflicts:
graph/graph.go
Upstream-commit: 5a00326d29efb161826ef13dfd63ed2732017cd1
Component: engine
and add a comment.. :-)
Signed-off-by: Vincent Batts <vbatts@redhat.com>
Conflicts:
graph/graph.go
Upstream-commit: ba1f76cbfa2c137abfbc607725460e376e6f44d3
Component: engine
During `(*Graph).Register, there was no protection on adding new layers
concurrently. In some cases, this resulted in corruption of a layer by creating
the directory but not the underlying data. This manifested in several different
IO errors reported in the client. This attempts to fix this by adding a mutex
by Image ID to protect the Register operation.
We do not completely understand the root cause of this corruption other than
the result is somehow tied to this particular function. This fix has been
confirmed to address the issue through testing.
Unfortunately, this fix does not address existing corruption. The user will
have to remove and re-pull the corrupt layer to stop the error from happening
in the future. This change only ensures that the layer will not become corrupt.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: 7eac23cf8de08f281c84c36390f734dbaef8c69a
Component: engine
An inspection of the graph package showed this function to be way out of place.
It is only depended upon by the daemon code. The function prepares a top-level
readonly layer used to provide a consistent runtime environment for docker
images.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: b7f887a9a24a2df61fa53612ae42b7e9d62fe858
Component: engine
Update get and set functions to use digests.
Update push code to use the digest type instead of string
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: c0b44218196ab597d22eb6344e8770fdee73067b
Component: engine
Move graph related functions in image to graph package.
Consolidating graph functionality is the first step in refactoring graph into an image store model.
Subsequent refactors will involve breaking up graph into multiple types with a strongly defined interface.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: 2b58b677a54950bc78c13760ef79fe6284154847
Component: engine
When buffering to file add support for compressing the tar contents. Since digest should be computed while writing buffer, include digest creation during buffer.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: 851c64725d0b1b37e51fa0d0df744bbe82ad4c7b
Component: engine
Added method in StreamFormatter to handle calls
from progressreader. Solves #10959
Signed-off-by: bobby abbott <ttobbaybbob@gmail.com>
Upstream-commit: 12b278d3540bc32699e8c2197b556188fd98b77b
Component: engine
I was confused earlier when I did:
```
docker push localhost.localdomain:1234/foo
```
Because docker told me:
```
No such id: localhost.localdomain:1234/foo
```
I actually had buried in my mind the solution to this, but the error
message
confused me because I had recently had some fun trying to get the
registry
working and therefore thought it was telling me that I didn't have an
account
on the registry.
This pull request makes it unambiguous that the error is that the
specified
image is unknown.
/cc @cpuguy83
Docker-DCO-1.1-Signed-off-by: Peter Waller <p@pwaller.net> (github: pwaller)
Upstream-commit: 3a08bbc4d29835fe5b8f3d2c5bde442f6a7014a0
Component: engine
The argument specified the json data to save to disk when registering
a new image into the image graph. If it is nil, then the given image
is serialized to json and that is written by default. This default
behavior is sufficient if the given image was originally deserialzed
from this jsonData to begin with which has always been the case.
Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Upstream-commit: 26184de8ab1dfe812094c55c9becd8ebb60ed7be
Component: engine
