Currently when overlay creates a whiteout file then the overlay2 layer is archived,
the correct tar header will be created for the whiteout file, but the tar logic will then attempt to open the file causing a failure.
When tar encounters such failures the file is skipped and excluded for the archive, causing the whiteout to be ignored.
By skipping the copy of empty files, no open attempt will be made on whiteout files.
Fixes#23863
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: bd13c53f8dc1504c9e681dfabef2afc1685ccec7
Component: engine
This fix tries to fix logrus formatting by removing `f` from
`logrus.[Error|Warn|Debug|Fatal|Panic|Info]f` when formatting string
is not present.
This fix fixes#23459.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: a72b45dbec3caeb3237d1af5aedd04adeb083571
Component: engine
There might be other (valid) reasons for setxattr(2) to fail, so only
ignore it when it's a not supported error (ENOTSUP). Otherwise, bail.
Signed-off-by: Aleksa Sarai <asarai@suse.de>
Upstream-commit: e6d856df43ac2877ebbcff26e9f3ee755f52bce9
Component: engine
Since certain filesystems don't support extended attributes, ignore
errors produced (emitting a warning) when attempting to apply extended
attributes to file.
Signed-off-by: Aleksa Sarai <asarai@suse.de>
Upstream-commit: 75a271603c761c18c7deb186d08943dcf85b739d
Component: engine
If the destination does not exist, it needs to be created with ownership
mapping to the remapped uid/gid ranges if user namespaces are enabled.
This fixes ADD operations, similar to the prior fixes for COPY and WORKDIR.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
Upstream-commit: 06d95003b3ddd22d26121280d645e48582a4f931
Component: engine
Closes#20470
Before this PR we used to scan the entire build context when there were
exclusions in the .dockerignore file (paths that started with !). Now we
only traverse into subdirs when one of the exclusions starts with that dir
path.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Upstream-commit: 842b8d8784b132279003580eedd0e9c12b885815
Component: engine
During "COPY" or other tar unpack operations, a target/destination
parent dir might not exist and should be created with ownership of the
root in the right context (including remapped root when user namespaces
are enabled)
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
Upstream-commit: 7a61b9ae425e5c100da2bb32b929031c6302b3fb
Component: engine
When execute `docker export -o path xxx` and path is a directory docker
has no privilege to write to, daemon will print lots of error logs that
most of them are duplicated and redundant.
This will remove unnecessary error logs and print only once.
Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
Upstream-commit: 439433099e261d504561d834839fcb936fb7ea95
Component: engine
Save was failing file integrity checksums due to bugs in both
Windows and Docker. This commit includes fixes to file time handling
in tarexport and system.chtimes that are necessary along with
the Windows platform fixes to correctly support save. With this
change, sysfile_backups for windowsfilter driver are no longer
needed, so that code is removed.
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
Upstream-commit: 041a9510c671ecf4724bf15388fdb9de549b04ed
Component: engine
aufs kernel module creates whiteout files on upper layer delete (and
other situations) and those files already are 'translated' regarding
ownership in host terms (e.g. they are already "0:0" owned), so when
these layers are copied around with pkg/archive we don't want to try and
translate these files regarding ownership.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
Upstream-commit: 1626c9dae0f285185e3ef4c1e2be55a807e8c0ed
Component: engine
Moved a defer up to a better spot.
Fixed TestUntarPathWithInvalidDest to actually fail for the right reason
Closes#18170
Signed-off-by: Doug Davis <dug@us.ibm.com>
Upstream-commit: 7bb9fc415ac50abf5f754005264bfe152eeca7d6
Component: engine
Fixes#16555
Original docker `cp` always copy symbol link itself instead of target,
now we provide '-L' option to allow docker to follow symbol link to real
target.
Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
Upstream-commit: 92600bdec1284f9031868751f61bef476d2e1dbd
Component: engine
Fixes#17766
Previously, opaque directory whiteouts on non-native
graphdrivers depended on the file order, meaning
files added with the same layer before the whiteout
file `.wh..wh..opq` were also removed.
If that file happened to have subdirs, then calling
chtimes on those dirs after unpack would fail the pull.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: db3070ab1b9e394e45b1db22b8ed8352272d4410
Component: engine
The race is between pools.Put which calls buf.Reset and exec.Cmd
doing io.Copy from the buffer; it caused a runtime crash, as
described in #16924:
``` docker-daemon cat the-tarball.xz | xz -d -c -q | docker-untar /path/to/... (aufs ) ```
When docker-untar side fails (like try to set xattr on aufs, or a broken
tar), invokeUnpack will be responsible to exhaust all input, otherwise
`xz` will be write pending for ever.
this change add a receive only channel to cmdStream, and will close it
to notify it's now safe to close the input stream;
in CmdStream the change to use Stdin / Stdout / Stderr keeps the
code simple, os/exec.Cmd will spawn goroutines and call io.Copy automatically.
the CmdStream is actually called in the same file only, change it
lowercase to mark as private.
[...]
INFO[0000] Docker daemon commit=0a8c2e3 execdriver=native-0.2 graphdriver=aufs version=1.8.2
DEBU[0006] Calling POST /build
INFO[0006] POST /v1.20/build?cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&memory=0&memswap=0&rm=1&t=gentoo-x32&ulimits=null
DEBU[0008] [BUILDER] Cache miss
DEBU[0009] Couldn't untar /home/lib-docker-v1.8.2-tmp/tmp/docker-build316710953/stage3-x32-20151004.tar.xz to /home/lib-docker-v1.8.2-tmp/aufs/mnt/d909abb87150463939c13e8a349b889a72d9b14f0cfcab42a8711979be285537: Untar re-exec error: exit status 1: output: operation not supported
DEBU[0009] CopyFileWithTar(/home/lib-docker-v1.8.2-tmp/tmp/docker-build316710953/stage3-x32-20151004.tar.xz, /home/lib-docker-v1.8.2-tmp/aufs/mnt/d909abb87150463939c13e8a349b889a72d9b14f0cfcab42a8711979be285537/)
panic: runtime error: slice bounds out of range
goroutine 42 [running]:
bufio.(*Reader).fill(0xc208187800)
/usr/local/go/src/bufio/bufio.go:86 +0x2db
bufio.(*Reader).WriteTo(0xc208187800, 0x7ff39602d150, 0xc2083f11a0, 0x508000, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:449 +0x27e
io.Copy(0x7ff39602d150, 0xc2083f11a0, 0x7ff3960261f8, 0xc208187800, 0x0, 0x0, 0x0)
/usr/local/go/src/io/io.go:354 +0xb2
github.com/docker/docker/pkg/archive.func·006()
/go/src/github.com/docker/docker/pkg/archive/archive.go:817 +0x71
created by github.com/docker/docker/pkg/archive.CmdStream
/go/src/github.com/docker/docker/pkg/archive/archive.go:819 +0x1ec
goroutine 1 [chan receive]:
main.(*DaemonCli).CmdDaemon(0xc20809da30, 0xc20800a020, 0xd, 0xd, 0x0, 0x0)
/go/src/github.com/docker/docker/docker/daemon.go:289 +0x1781
reflect.callMethod(0xc208140090, 0xc20828fce0)
/usr/local/go/src/reflect/value.go:605 +0x179
reflect.methodValueCall(0xc20800a020, 0xd, 0xd, 0x1, 0xc208140090, 0x0, 0x0, 0xc208140090, 0x0, 0x45343f, ...)
/usr/local/go/src/reflect/asm_amd64.s:29 +0x36
github.com/docker/docker/cli.(*Cli).Run(0xc208129fb0, 0xc20800a010, 0xe, 0xe, 0x0, 0x0)
/go/src/github.com/docker/docker/cli/cli.go:89 +0x38e
main.main()
/go/src/github.com/docker/docker/docker/docker.go:69 +0x428
goroutine 5 [syscall]:
os/signal.loop()
/usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
/usr/local/go/src/os/signal/signal_unix.go:27 +0x35
Signed-off-by: Derek Ch <denc716@gmail.com>
Upstream-commit: b48f4bf5239280b99c781a33000b4d4687967c34
Component: engine
All the go-lint work forced any existing "Uid" -> "UID", but seems to
not have the same rules for Gid, so stat package has calls UID() and
Gid().
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
Upstream-commit: 79240b9eafa08001912d5b360a654b1ad9cd1d20
Component: engine
Adds support for the daemon to handle user namespace maps as a
per-daemon setting.
Support for handling uid/gid mapping is added to the builder,
archive/unarchive packages and functions, all graphdrivers (except
Windows), and the test suite is updated to handle user namespace daemon
rootgraph changes.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
Upstream-commit: 442b45628ee12ebd8e8bd08497896d5fa8eec4bd
Component: engine
This fixes the case where directory is removed in
aufs and then the same layer is imported to a
different graphdriver.
Currently when you do `rm -rf /foo && mkdir /foo`
in a layer in aufs the files under `foo` would
only be be hidden on aufs.
The problems with this fix:
1) When a new diff is recreated from non-aufs driver
the `opq` files would not be there. This should not
mean layer differences for the user but still
different content in the tar (one would have one
`opq` file, the others would have `.wh.*` for every
file inside that folder). This difference also only
happens if the tar-split file isn’t stored for the
layer.
2) New files that have the filenames before `.wh..wh..opq`
when they are sorted do not get picked up by non-aufs
graphdrivers. Fixing this would require a bigger
refactoring that is planned in the future.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: 00e3277107a63218b656a36457caf380f9cabf37
Component: engine
… and fixes the last bits that were missing :3.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 0c70eb83813af6d5266769bc908593f9660b13f1
Component: engine
