This matches what tar does, and without it the tarsum created
by the registry will not match the docker one.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
Upstream-commit: f918fca3bf77ebe5845593281382f68e5b166ab3
Component: engine
This uses a plain filepath.Walk + addTarFile to create a tar file,
optionially compressing it with gzip.
Unfortunately go only has gzip compression support, not bzip2 or xz.
However, this is not a regression, as docker currently uses *no*
compression for TarFilter(). The only compression of tarfiles
currently happens in utils/tarsum.go, and that manually does gzip
compression.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
Upstream-commit: 5ea48aa7f8d3839877b869fdfcea2d5de0972fbf
Component: engine
We can use tar.FileInfoHeader to do much of the work.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
Upstream-commit: bab8efbf050e1bc2d5c2ff64c6161ef4d323bc3f
Component: engine
This is the code that takes a normal file and adds it to a TarWriter.
We extract it so that we can share it with Tar().
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
Upstream-commit: 5b77e51e0a15eddefcb40380673df8c0c24f95d1
Component: engine
This simplifies that code that calls out to tar by removing support
for now unused features.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
Upstream-commit: 4fb1db7f742fb34a7a06621d0698063de87a572c
Component: engine
stat.Rdev and time.* is 32bit on OSX, which breaks cross builds with eg:
cannot use stat.Rdev (type int32) as type uint64 in function argument
We fix this with an extra conversion to uint64.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
Upstream-commit: ba52130873395a44d637fc57f98ed174f0ac87bb
Component: engine
This replaces the shelling out to tar with a reimplementation of untar
based on the archive/tar code and the pre-existing code from ApplyLayer
to create real files from tar headers.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
Upstream-commit: a4868e233c7ebce259fc02d3dbfb241c23471a4a
Component: engine
This way we can reuse it for Untar()
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
Upstream-commit: 710d5a48fb751623fbf77a51b89f2dfbf0edac68
Component: engine
This changes ExportChanges to use the go tar support so we can
directly create tar layer files. This has several advantages:
* We don't have to create the whiteout files on disk to get them
added to the layer
* We can later guarantee specific features (such as xattrs) being
supported by the tar implementation.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
Upstream-commit: d54ce8087aba23663856c81a3fb5475979bdf453
Component: engine
This reverts commit 733bf5d3ddbfb6dba7c2c0996c4af47a765e4593.
This is needed to fix "no such file" errors `docker build` errors for
devicemapper.
This fixes issue #3449.
Docker-DCO-1.0-Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com> (github: unclejack)
Upstream-commit: d003cfea25c276904dbe1e972c0cf71d5c25e689
Component: engine
This shells out to the xz binary to support .tar.xz layers, as
there is no compression/xz support in go.
Upstream-commit: b8a4f570fb31091f43caeba5b824ae38a5bc69e8
Component: engine
When pulling from a registry we get a compressed tar archive, so
we need to wrap the stream in the right kind of compress reader.
Unfortunately go doesn't have an Xz decompression method, but I
don't think any docker layers use that atm anyway.
Upstream-commit: a96a26c62f6333e61134759256db43a7f5946631
Component: engine
The TestLookupImage test seems to use a layer that contains
/etc/postgres/postgres.conf, but not e.g. /etc/postgres.
To handle this we ensure that the parent directory always
exists, and if not we create it.
Upstream-commit: 78c22c24b353d77fdab3e1616d9986a8ae95a7c2
Component: engine
The default gnu tar format has no sub-second precision mtime support,
and the golang tar writer currently doesn't support that either.
This means if we export the changes from a container we will not
get zeron in the sub-second precision field when the change is applied.
This means we can't compare that to the original without getting a
spurious change. So, we detect this case by treating a case where the
seconds match and either of the two nanoseconds are zero as equal.
Upstream-commit: 10cd902f900392a2f10a6f8763bba70607ea0d41
Component: engine
Rather than calling out to tar we use the golang tar parser
to directly extract the tar files. This has two major advantages:
1) We're able to replace an existing directory with a file in the
new layer. This currently breaks with the external tar, since
it refuses to recursively remove the destination directory in
this case, and there are no options to make it do that.
2) We avoid extracting the whiteout files just to later remove them.
Upstream-commit: 818c249bae8d29842834bf765299c86c09e6913e
Component: engine
This moves Driver.Size() to Differ.DiffSize(), removing the empty
implementations in devmapper and dummy, and renaming the one in aufs.
Then we fall back to a container.Changes() implementation in the non-aufs
case.
Upstream-commit: 5d76681c3d0cbb744205a397420603ce029ce754
Component: engine
This adds 3 tests:
Verify that ChangesDirs() returns nothing when run on a copy of the
same directory.
Verify that various mutations of a directory get reported with the
right changes.
Verify that ExportChanges() + ApplyLayer() of the above mutation gets
an identical directory.
Unfortunately the last test is disabled because it fails in multiple
ways atm. But I want to get it in so that we can fix it.
Upstream-commit: f7238f94e8b5b31e78360c342676ca84b5984806
Component: engine
This code was duplicated in two places, one which was unused. This
syncs the code and removes the unused version.
Upstream-commit: d69a6a20f0b6657821638ee591920d071a784b0e
Component: engine
rmTargetPath can be a directory, so we have to use RemoveAll() or we
will fail to whiteout non-empty directories.
Upstream-commit: 006e2a600ce689770ba2c49805bc4f634976f365
Component: engine
When directories are white-outed we can get called with the previously
removed directories. Handle this with os.IsNotExist(error).
Upstream-commit: 6f3c32eb18f0a604ba2638b6a9f19f8824412fd1
Component: engine
This replaces the current version with the latest version from the
dm branch. Advantages in this version:
We only scan each directory tree once, which means less i/o and less chance
of container changes inbetween the two scans causing inconsistencies.
We avoid comparing some fields for change detection:
* Inode
* size-in-blocks
These can change during a copy operation (e.g. in the dummy backend)
without needing to actually reflect a change in content or metadata.
* Ctime
Any copy operation will create a new Ctime value, and there is no
API to change it to the "source" value.
* size for directories
The size of a directory doesn't have to be the same just because you
recreated the same content as another director. Internal details in
the filesystem may make these different with no "real" change.
Upstream-commit: b6ef4bc9521346dc5066d71821c6cadfbeced9d3
Component: engine
Some aufs metadata are directories, and whiteouts can be for
directories, so use os.RemoveAll() for these.
Upstream-commit: 484804abffda2692492e295d4691f90564d05eb2
Component: engine
plugin.
The merge is inert, in other words the devmapper code is not called
and the primary aufs backend is untouched.
Upstream-commit: 36c7a7ae943220ffb1bea9cd47d2f586db00eab2
Component: engine
