add regression case for the issue#37870
Signed-off-by: Wei Fu <fuweid89@gmail.com>
(cherry picked from commit 8e25f4ff6d89888a1bcd578f3f8f7aab89dce24d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ae6284a623bac86ac6ab718fa4a369dd8c0a3cfc
Component: engine
[18.09 backport] move the syslog syscall to be gated by CAP_SYS_ADMIN or CAP_SYSLOG
Upstream-commit: 6f1145e740f35a0b805a11d5d29daa89d2a27ed0
Component: engine
Bump libnetwork to 6da50d1978302f04c3e2089e29112ea24812f05b which
is the current tip of libnetwork's bump_18.09 branch to get the DSR load
balancing mode option changes for the 18.09 branch of Docker CE.
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: fd1fe0b702571865cc77d66937e4ca570b5b9cc3
Component: engine
For some reason, shared mount propagation between the host
and a container does not work for btrfs, unless container
root directory (i.e. graphdriver home) is a bind mount.
The above issue was reproduced on SLES 12sp3 + btrfs using
the following script:
#!/bin/bash
set -eux -o pipefail
# DIR should not be under a subvolume
DIR=${DIR:-/lib}
MNT=$DIR/my-mnt
FILE=$MNT/file
ID=$(docker run -d --privileged -v $DIR:$DIR:rshared ubuntu sleep 24h)
docker exec $ID mkdir -p $MNT
docker exec $ID mount -t tmpfs tmpfs $MNT
docker exec $ID touch $FILE
ls -l $FILE
umount $MNT
docker rm -f $ID
which fails this way:
+ ls -l /lib/my-mnt/file
ls: cannot access '/lib/my-mnt/file': No such file or directory
meaning the mount performed inside a priviledged container is not
propagated back to the host (even if all the mounts have "shared"
propagation mode).
The remedy to the above is to make graphdriver home a bind mount.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 16d822bba8ac5ab22c8697750f700403bca3dbf3)
Upstream-commit: fa8ac946165b8004a15e85744e774ed6ba99fd38
Component: engine
This function ensures the argument is the mount point
(i.e. if it's not, it bind mounts it to itself).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 8abadb36fa8149cd44e76b0e7fdedd6f1f2eccd0)
Upstream-commit: 2199ada691dc635cac5cdd065d909a539dd0b793
Component: engine
1. There is no need to specify rw argument -- bind mounts are
read-write by default.
2. There is no point in parsing /proc/self/mountinfo after performing
a mount, especially if we don't check whether the fs is mounted or
not -- the only outcome from it could be an error from our mountinfo
parser, which makes no sense in this context.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit f01297d1ae352bc2bf01ebf62e879c1c83cdbee4)
Upstream-commit: fd7611ff1f1d61d5b4b45b2c0bd83976cbccf174
Component: engine
The `overlay` storage driver is deprecated in favor of the `overlay2` storage
driver, which has all the benefits of `overlay`, without its limitations (excessive
inode consumption). The legacy `overlay` storage driver will be removed in a future
release. Users of the `overlay` storage driver should migrate to the `overlay2`
storage driver.
The legacy `overlay` storage driver allowed using overlayFS-backed filesystems
on pre 4.x kernels. Now that all supported distributions are able to run `overlay2`
(as they are either on kernel 4.x, or have support for multiple lowerdirs
backported), there is no reason to keep maintaining the `overlay` storage driver.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 31be4e0ba11532e5d6df8401f5d459e292c58f36)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c20e8dffbbb4dd328e4c00d18e04eecb80cf4d4e
Component: engine
The `devicemapper` storage driver is deprecated in favor of `overlay2`, and will
be removed in a future release. Users of the `devicemapper` storage driver are
recommended to migrate to a different storage driver, such as `overlay2`, which
is now the default storage driver.
The `devicemapper` storage driver facilitates running Docker on older (3.x) kernels
that have no support for other storage drivers (such as overlay2, or AUFS).
Now that support for `overlay2` is added to all supported distros (as they are
either on kernel 4.x, or have support for multiple lowerdirs backported), there
is no reason to continue maintenance of the `devicemapper` storage driver.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 06fcabbaa0d4f25db0580d3604bbed708223fb06)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 734e7a8e55e41b064025273776dc5a76e45bf2e1
Component: engine
[18.09] backport fix denial of service with large numbers in cpuset-cpus and cpuset-mems
Upstream-commit: 4b8336f7cf091fd5c4742286bda1e34c45667d78
Component: engine
As pointed out in https://github.com/moby/moby/issues/37970,
Docker overlay driver can't work with index=on feature of
the Linux kernel "overlay" filesystem. In case the global
default is set to "yes", Docker will fail with EBUSY when
trying to mount, like this:
> error creating overlay mount to ...../merged: device or resource busy
and the kernel log should contain something like:
> overlayfs: upperdir is in-use by another mount, mount with
> '-o index=off' to override exclusive upperdir protection.
A workaround is to set index=off in overlay kernel module
parameters, or even recompile the kernel with
CONFIG_OVERLAY_FS_INDEX=n in .config. Surely this is not
always practical or even possible.
The solution, as pointed out my Amir Goldstein (as well as
the above kernel message:) is to use 'index=off' option
when mounting.
NOTE since older (< 4.13rc1) kernels do not support "index="
overlayfs parameter, try to figure out whether the option
is supported. In case it's not possible to figure out,
assume it is not.
NOTE the default can be changed anytime (by writing to
/sys/module/overlay/parameters/index) so we need to always
use index=off.
[v2: move the detection code to Init()]
[v3: don't set index=off if stat() failed]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 8422d85087bfa770b62ef4e1daaca95ee6783d86)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 690e097fedd7362f3b2781c32ca872ad966d286e
Component: engine
This simplifies the code a lot.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit a55d32546a8556f9e6cabbc99836b573b9944f0c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: dc0a4db7c9dc593a8568a8e30e4e21e118c2839d
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
(cherry picked from commit c907c2486c0850030f8ac1819ac8c87631472c68)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 7184074c0880c656be00645007588a00ec2266cd
Component: engine
The stdin fifo of exec process is created in containerd side after
client calls Start. If the client calls CloseIO before Start call, the
stdin of exec process is still opened and wait for close.
For this case, client closes stdinCloseSync channel after Start.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
(cherry picked from commit c7890f25a9eaae8d07614bd85b2b3231b03e54ec)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 6679a5faeb724f1ad060f2fdf6d189f1005924b9
Component: engine
Using a value such as `--cpuset-mems=1-9223372036854775807` would cause
`dockerd` to run out of memory allocating a map of the values in the
validation code. Set limits to the normal limit of the number of CPUs,
and improve the error handling.
Reported by Huawei PSIRT.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f8e876d7616469d07b8b049ecb48967eeb8fa7a5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0922d32bce74657266aff213f83dfa638e8077f4
Component: engine
This updates the containerd dependencies to match
the versions used by the vendored containerd version
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 31a9c9e79101cdf38d383104afbc1b48ede75291)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5070e418b806cc96ad0f5b3ac32c8d416ff8449a
Component: engine
With containerd reaching 1.0, the runtime now
has a stable API, so there's no need to do a check
if the installed version matches the expected version.
Current versions of Docker now also package containerd
and runc separately, and can be _updated_ separately.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c65f0bd13c85d29087419fa555281311091825e7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 054c3c2931cec5dca8bb84af97f1457c343ec02f
Component: engine
We check for max value for -default-addr-pool-mask-length param as 32.
But There won't be enough addresses on the overlay network. Hence we are
keeping it 29 so that we would be having atleast 8 addresses in /29 network.
Signed-off-by: selansen <elango.siva@docker.com>
(cherry picked from commit d25c5df80e60cdbdc23fe3d0e2a6808123643dc7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9406f3622d18a0d9b6c438190e8fdd8be53d3b22
Component: engine
Addressing few review comments as part of code refactoring.
Also moved validation logic from CLI to Moby.
Signed-off-by: selansen <elango.siva@docker.com>
(cherry picked from commit 148ff00a0a800fad99de11ee3021d4c5d4869157)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9816bfcaf58a609d64d648043c10817c27dcfa36
Component: engine
[18.09] Fix long startup on windows, with non-hns governed Hyper-V networks
Upstream-commit: 6e5ed2ccce0923950dae7a6cb9bd1dd851ed5af8
Component: engine
Similar to a related issue where previously, private Hyper-V networks
would each add 15 secs to the daemon startup, non-hns governed internal
networks are reported by hns as network type "internal" which is not
mapped to any network plugin (and thus we get the same plugin load retry
loop as before).
This issue hits Docker for Desktop because we setup such a network for
the Linux VM communication.
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
(cherry picked from commit 6a1a4f97217b0a8635bc21fc86628f48bf8824d1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 54bd14a3fe1d4925c6fa88b24949063d99067c07
Component: engine
The parameter name was wrong, which may mislead a user.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit c378fb774e413ba8bf5cadf655d2b67e9c94245a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c9ddc6effc444c54def41d498b359a9a986ad79d
Component: engine
This call is what is used to implement `dmesg` to get kernel messages
about the host. This can leak substantial information about the host.
It is normally available to unprivileged users on the host, unless
the sysctl `kernel.dmesg_restrict = 1` is set, but this is not set
by standard on the majority of distributions. Blocking this to restrict
leaks about the configuration seems correct.
Fix#37897
See also https://googleprojectzero.blogspot.com/2018/09/a-cache-invalidation-bug-in-linux.html
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
(cherry picked from commit ccd22ffcc8b564dfc21e7067b5248819d68c56c6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 16836e60bc87abb3e9ab16f33c2038931c1d473b
Component: engine
These messages were enhanced to include the path that was
missing (in df6af282b9048dfedcd7b7a9a89126aca887f4e1), but
also changed the first part of the message.
This change complicates running e2e tests with mixed versions
of the engine.
Looking at the full error message, "mount" is a bit redundant
as well, because the error message already indicates this is
about a "mount";
docker run --rm --mount type=bind,source=/no-such-thing,target=/foo busybox
docker: Error response from daemon: invalid mount config for type "bind": bind mount source path does not exist: /no-such-thing.
Removing the "mount" part from the error message, because
it was redundant, and makes cross-version testing easier :)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 574db7a53782c57554089c9606505af1c108df0b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b499acc0e834e11882909269238407c65f68f034
Component: engine
Andrew Hsu
Tonis Tiigi
Sebastiaan van Stijn
Wei Fu
Drew Erny
Madhu Venugopal
Tibor Vass
Chris Telfer
Kir Kolyshkin
Tibor Vass
John Howard
Akihiro Suda
Eli Uriegas
Eli Uriegas
Justin Cormack
selansen
Sebastiaan van Stijn
Dani Louca
Simon Ferquel