The promise package represents a simple enough concurrency pattern that
replicating it in place is sufficient. To end the propagation of this
package, it has been removed and the uses have been inlined.
While this code could likely be refactored to be simpler without the
package, the changes have been minimized to reduce the possibility of
defects. Someone else may want to do further refactoring to remove
closures and reduce the number of goroutines in use.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: 0cd4ab3f9a3f242468484fc62b46e632fdba5e13
Component: engine
The change in 7a7357dae1bcccb17e9b2d4c7c8f5c025fce56ca inadvertently
changed the `defer` error code into a no-op. This restores its behavior
prior to that code change, and also introduces a little more error
logging.
Signed-off-by: Euan Kemp <euan.kemp@coreos.com>
Upstream-commit: 639ab92f011245e17e9a293455a8dae1eb034022
Component: engine
This is kernel config available in RHEL7.4 based kernels that enables
mountpoint removal where the mountpoint exists in other namespaces.
In particular this is important for making this pattern work:
```
umount -l /some/path
rm -r /some/path
```
Where `/some/path` exists in another mount namespace.
Setting this value will prevent `device or resource busy` errors when
attempting to the removal of `/some/path` in the example.
This setting is the default, and non-configurable, on upstream kernels
since 3.15.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 83c2152de503012195bd26069fd8fbd2dea4b32f
Component: engine
Fixing ‘docker cp’ to allow new target file name in a host symlinked directory
Upstream-commit: 149f3aceb277ecccd33ee815e9c54d6aec782ab5
Component: engine
[Builder] Introduce a typed command system and 2 phase parse/dispatch build
Upstream-commit: 22e15721b1f33f32f07615e034cf3a7b4c865aa3
Component: engine
libcontainerd has a bunch of platform dependent code and huge interfaces
that are a pain implement.
To make the plugin manager a bit easier to work with, extract the plugin
executor into an interface and move the containerd implementation to a
separate package.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: c85e8622a4813d7b72d74517faa03ab5de4c4550
Component: engine
When using a volume via the `Binds` API, a shared selinux label is
automatically set.
The `Mounts` API is not setting this, which makes volumes specified via
the mounts API useless when selinux is enabled.
This fix adopts the same selinux label for volumes on the mounts API as on
binds.
Note in the case of both the `Binds` API and the `Mounts` API, the
selinux label is only applied when the volume driver is the `local`
driver.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 5bbf5cc671ec8007bf8e0416799fff01d6a79b7e
Component: engine
To avoid a zombie apocalypse, use cmd.Wait() to properly finish
the processes we spawn by Start().
Found while investigating DockerSuite.TestLogsFollowSlowStdoutConsumer
failure on ARM (see
https://github.com/moby/moby/pull/34550#issuecomment-324937936).
[v2: don't expect no error from Wait() when process is killed]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 14f0a1888f92667f82bea548bfa2fe4a890a75e8
Component: engine
It was causing the error message to be
'overlay' is not supported over <unknown>
instead of
'overlay' is not supported over ecryptfs
Signed-off-by: Iago López Galeiras <iago@kinvolk.io>
Upstream-commit: ddb31b4fdf7311ad6c7e62aa15b5f6da16518e77
Component: engine
We run our CI on Scaleway C1 machine, which is pretty slow,
including I/O. This test was failing on it, as it tried to
write 100000 lines of log very fast, and the loggerCloseTimeout
(defined and used in container/monitor.go) prevents the
daemon to finish writing it within this time frame,
Reducing the size to 150000 characters (75000 lines) should
help avoiding hitting it, without compromising the test case
itself.
Alternatively, we could have increased the timeout further. It was
originally set to 1s (commit b6a42673a) and later increased 10x
(commit c0391bf55). Please let me know if you want me to go that way.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 1bc93bff221bd30e80f776cc620a8937314569ef
Component: engine
This is a work base to introduce more features like build time
dockerfile optimisations, dependency analysis and parallel build, as
well as a first step to go from a dispatch-inline process to a
frontend+backend process.
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
Upstream-commit: 669c0677980b04bcbf871bb7c2d9f07caccfd42b
Component: engine
This commit reverts a hunk of commit 2f5f0af3f ("Add unconvert linter")
and adds a hint for unconvert linter to ignore excessive conversion as
it is required on 32-bit platforms (e.g. armhf).
The exact error on armhf is this:
19:06:45 ---> Making bundle: dynbinary (in bundles/17.06.0-dev/dynbinary)
19:06:48 Building: bundles/17.06.0-dev/dynbinary-daemon/dockerd-17.06.0-dev
19:10:58 # github.com/docker/docker/daemon/graphdriver/overlay
19:10:58 daemon/graphdriver/overlay/copy.go:161: cannot use stat.Atim.Sec (type int32) as type int64 in argument to time.Unix
19:10:58 daemon/graphdriver/overlay/copy.go:161: cannot use stat.Atim.Nsec (type int32) as type int64 in argument to time.Unix
19:10:58 daemon/graphdriver/overlay/copy.go:162: cannot use stat.Mtim.Sec (type int32) as type int64 in argument to time.Unix
19:10:58 daemon/graphdriver/overlay/copy.go:162: cannot use stat.Mtim.Nsec (type int32) as type int64 in argument to time.Unix
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: b569f57890d4cad132be437e5dac55130b6c76e9
Component: engine
When running 'make all' on armhf, I got this:
> ---> Making bundle: .integration-daemon-start (in bundles/17.06.0-dev/test-docker-py)
> Using test binary docker
> INFO: Waiting for daemon to start...
> Starting dockerd
> .
> Traceback (most recent call last):
> File "/usr/local/lib/python2.7/dist-packages/_pytest/config.py", line
> 320, in _importconftest
> mod = conftestpath.pyimport()
> File "/usr/local/lib/python2.7/dist-packages/py/_path/local.py", line
> 662, in pyimport
> __import__(modname)
> File "/docker-py/tests/integration/conftest.py", line 6, in <module>
> import docker.errors
> File "/docker-py/docker/__init__.py", line 2, in <module>
> from .api import APIClient
> File "/docker-py/docker/api/__init__.py", line 2, in <module>
> from .client import APIClient
> File "/docker-py/docker/api/client.py", line 11, in <module>
> from .build import BuildApiMixin
> File "/docker-py/docker/api/build.py", line 6, in <module>
> from .. import auth
> File "/docker-py/docker/auth.py", line 6, in <module>
> import dockerpycreds
> ImportError: No module named dockerpycreds
> ERROR: could not load /docker-py/tests/integration/conftest.py
The fix for this was already provided by commit 0ec8f56a3 and
commit c7c923594, but for some reason it did not made its way
to Dockerfiles for all architectures.
While at it, remove excessive comments.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 7439d360fd567e063b8e9c4174a5c21b9fbc06aa
Component: engine
Instead of providing a generic message listing all possible reasons
why xfs is not available on the system, let's be specific.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: c21245c9200ab39a9219b28f8185573b78a55074
Component: engine
If mount fails, the reason might be right there in the kernel log ring buffer.
Let's include it in the error message, it might be of great help.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 46833ee1c353c247e3ef817a08d5a35a2a43bdf3
Component: engine
Since the update to Debian Stretch, devmapper unit test fails. One
reason is, the combination of somewhat old (less than 3.16) kernel and
relatively new xfsprogs leads to creating a filesystem which is not supported
by the kernel:
> [12206.467518] XFS (dm-1): Superblock has unknown read-only compatible features (0x1) enabled.
> [12206.472046] XFS (dm-1): Attempted to mount read-only compatible filesystem read-write.
> Filesystem can only be safely mounted read only.
> [12206.472079] XFS (dm-1): SB validate failed with error 22.
Ideally, that would be automatically and implicitly handled by xfsprogs.
In real life, we have to take care about it here. Sigh.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 6b01bc5adb1255cb48e22e755ab86fd2c3305211
Component: engine
Presumably after switch to debian-stretch as a base, the following
errors happens in Jenkins:
10:48:03 ---> Making bundle: test-docker-py (in
bundles/17.06.0-dev/test-docker-py)
10:48:03 ---> Making bundle: .integration-daemon-start (in
bundles/17.06.0-dev/test-docker-py)
10:48:03 Using test binary docker
10:48:03 # DOCKER_EXPERIMENTAL is set: starting daemon with experimental
features enabled!
10:48:03 /etc/init.d/apparmor: 130: /etc/init.d/apparmor:
systemd-detect-virt: not found
10:48:03 Starting AppArmor profiles:Warning from stdin (line 1):
/sbin/apparmor_parser: cannot use or update cache, disable, or
force-complain via stdin
10:48:03 Warning failed to create cache: (null)
10:48:03 .
10:48:03 INFO: Waiting for daemon to start...
10:48:03 Starting dockerd
10:48:05 .
10:48:06 Traceback (most recent call last):
10:48:06 File
"/usr/local/lib/python2.7/dist-packages/_pytest/config.py", line 320, in
_importconftest
10:48:06 mod = conftestpath.pyimport()
10:48:06 File
"/usr/local/lib/python2.7/dist-packages/py/_path/local.py", line 662, in
pyimport
10:48:06 __import__(modname)
10:48:06 File "/docker-py/tests/integration/conftest.py", line 6, in
<module>
10:48:06 import docker.errors
10:48:06 File "/docker-py/docker/__init__.py", line 2, in <module>
10:48:06 from .api import APIClient
10:48:06 File "/docker-py/docker/api/__init__.py", line 2, in <module>
10:48:06 from .client import APIClient
10:48:06 File "/docker-py/docker/api/client.py", line 6, in <module>
10:48:06 import requests
10:48:06 ImportError: No module named requests
10:48:06 ERROR: could not load /docker-py/tests/integration/conftest.py
10:48:06
and
00:38:55 File "/docker-py/docker/transport/ssladapter.py", line 21, in
<module>
00:38:55 from backports.ssl_match_hostname import match_hostname
00:38:55 ImportError: No module named backports.ssl_match_hostname
00:38:55 ERROR: could not load /docker-py/tests/integration/conftest.py
To fix, install the missing python modules.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: ce2a0120c1925492a9bb7f6339cdbf716a4c50e0
Component: engine
Since the update to Debian Stretch, this test fails. The reason is dynamic
binary, which requires i386 ld.so for loading (and apparently it is no longer
installed by default):
> root@09d4b173c3dc:/go/src/github.com/docker/docker# file exit32-test
> exit32-test: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, BuildID[sha1]=a0d3d6cb59788453b983f65f8dc6ac52920147b6, stripped
> root@09d4b173c3dc:/go/src/github.com/docker/docker# ls -l /lib/ld-linux.so.2
> ls: cannot access '/lib/ld-linux.so.2': No such file or directory
To fix, just add -static.
Interestingly, ldd can'f figure it out.
> root@a324f8edfcaa:/go/src/github.com/docker/docker# ldd exit32-test
> not a dynamic executable
Other tools (e.g. objdump) also show it's a dynamic binary.
While at it, remove the extra "id" argument (a copy-paste error I
guess).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 771256b305c8c06fca5eb1d041b60fbe093c0e1b
Component: engine
Static build with devmapper is impossible now since libudev is required
and no static version of libudev is available (as static libraries are
not supported by systemd which udev is part of).
This should not hurt anyone as "[t]he primary user of static builds
is the Editions, and docker in docker via the containers, and none
of those use device mapper".
Also, since the need for static libdevmapper is gone, there is no need
to self-compile libdevmapper -- let's use the one from Debian Stretch.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 84f1c054e66d3999aaf2751062cda8a77925b7ae
Component: engine
The main gain here is that they all use exactly the same distro; previously
arm64 was using Ubuntu Xenial because Debian jessie was too old.
Does not seem that we can change any of the downloaded dependencies still,
as eg libseccomp is still not the version we are using.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Upstream-commit: e89a5e5e91476102a471797fc2a81aa2f0f2b3fb
Component: engine
