Commit Graph

33664 Commits

Author SHA1 Message Date
ea7e1c4aaa builder: fix duplicate mount release
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 2732fe527f9258561c7310c128914b4b456c8404)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5853cd510c3272755ca5d6605ca8039d54a5ba15
Component: engine
2018-10-24 20:11:51 +02:00
7d67a90c9e Bump swarmkit
Signed-off-by: Drew Erny <drew.erny@docker.com>
Upstream-commit: 1222a7081ac9ebb0830a6c8008142258c49800b5
Component: engine
2018-10-22 15:10:20 -05:00
cd3e286dd2 Merge pull request #64 from thaJeztah/18.09_backport_syslog
[18.09 backport] move the syslog syscall to be gated by CAP_SYS_ADMIN or CAP_SYSLOG
Upstream-commit: 6f1145e740f35a0b805a11d5d29daa89d2a27ed0
Component: engine
2018-10-22 08:24:03 -07:00
96aa81d195 Merge pull request #93 from ctelfer/18.09-backport-dsr
[18.09] Bump libnetwork to 6da50d19 for DSR load balancing changes
Upstream-commit: ef87a664ef2554af9afa85ed25ffb82c39105624
Component: engine
2018-10-19 09:37:11 -07:00
d5fc44041a Bump libnetwork to 6da50d19 for DSR changes
Bump libnetwork to 6da50d1978302f04c3e2089e29112ea24812f05b which
is the current tip of libnetwork's bump_18.09 branch to get the DSR load
balancing mode option changes for the 18.09 branch of Docker CE.

Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: fd1fe0b702571865cc77d66937e4ca570b5b9cc3
Component: engine
2018-10-18 10:52:57 -04:00
5521475b61 builder: fix private pulls on buildkit
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit c693d45acf74b87680ace0db8615f97bd6853598)
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: fdaf08a57b2348623f33e0b9855c488421fc7bf6
Component: engine
2018-10-17 17:54:13 -07:00
8eac62569b Merge pull request #86 from kolyshkin/18.09-backport-btrfs-prop
[18.09] backport Fix mount propagation for btrfs
Upstream-commit: 4d0b8cc2d701e2a09b8e1f889b98c08d225d8145
Component: engine
2018-10-12 18:28:24 -07:00
23c67fa29f Merge pull request #82 from tiborvass/18.09-buildkit-cherry-picks
[18.09 backport] builder: treat unset keep-storage as 0
Upstream-commit: 7c63f178e7f2405337893c14b3c8c748b5cc1897
Component: engine
2018-10-12 11:01:20 -07:00
4c92b55e3f Merge pull request #83 from thaJeztah/18.09_backport_bump_buildkit
[18.09 backport] bump buildkit to c7bb575343df0cbfeab8b5b28149630b8153fcc6
Upstream-commit: b811212ccdbfd752675de1161e59ca194c798da4
Component: engine
2018-10-12 10:43:01 -07:00
d00ceca6f0 btrfs: ensure graphdriver home is bind mount
For some reason, shared mount propagation between the host
and a container does not work for btrfs, unless container
root directory (i.e. graphdriver home) is a bind mount.

The above issue was reproduced on SLES 12sp3 + btrfs using
the following script:

	#!/bin/bash
	set -eux -o pipefail

	# DIR should not be under a subvolume
	DIR=${DIR:-/lib}
	MNT=$DIR/my-mnt
	FILE=$MNT/file

	ID=$(docker run -d --privileged -v $DIR:$DIR:rshared ubuntu sleep 24h)
	docker exec $ID mkdir -p $MNT
	docker exec $ID mount -t tmpfs tmpfs $MNT
	docker exec $ID touch $FILE
	ls -l $FILE
	umount $MNT
	docker rm -f $ID

which fails this way:

	+ ls -l /lib/my-mnt/file
	ls: cannot access '/lib/my-mnt/file': No such file or directory

meaning the mount performed inside a priviledged container is not
propagated back to the host (even if all the mounts have "shared"
propagation mode).

The remedy to the above is to make graphdriver home a bind mount.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 16d822bba8ac5ab22c8697750f700403bca3dbf3)
Upstream-commit: fa8ac946165b8004a15e85744e774ed6ba99fd38
Component: engine
2018-10-12 09:29:38 -07:00
de86551520 pkg/mount: add MakeMount()
This function ensures the argument is the mount point
(i.e. if it's not, it bind mounts it to itself).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 8abadb36fa8149cd44e76b0e7fdedd6f1f2eccd0)
Upstream-commit: 2199ada691dc635cac5cdd065d909a539dd0b793
Component: engine
2018-10-12 09:29:38 -07:00
0d643ed639 pkg/mount: simplify ensureMountedAs
1. There is no need to specify rw argument -- bind mounts are
   read-write by default.

2. There is no point in parsing /proc/self/mountinfo after performing
   a mount, especially if we don't check whether the fs is mounted or
   not -- the only outcome from it could be an error from our mountinfo
   parser, which makes no sense in this context.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit f01297d1ae352bc2bf01ebf62e879c1c83cdbee4)
Upstream-commit: fd7611ff1f1d61d5b4b45b2c0bd83976cbccf174
Component: engine
2018-10-12 09:29:38 -07:00
2d00f9e2eb builder: treat unset keep-storage as 0
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit d6ac2b0db00455824c400394f316bdbc5adf8867)
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: dbfc648a94569d8dbc8c6468d56ec93559363bb0
Component: engine
2018-10-11 20:35:43 +00:00
d02c5a3248 Merge pull request #75 from thaJeztah/18.09_backport_bump_containerd_client_1.2.0_rc.1
[18.09] backport update containerd client and dependencies to v1.2.0-rc.1
Upstream-commit: 8e67dfab97ec71c66ee04a0c39f89e0a57ce2a0f
Component: engine
2018-10-11 13:27:48 -07:00
0746eb40ef Merge pull request #73 from thaJeztah/18.09_backport_addr_pool
[18.09] backport default-addr-pool-mask-length param max value check
Upstream-commit: b38d454861a8b4343142a62094cc3cbf4b157e00
Component: engine
2018-10-11 13:27:22 -07:00
921a9476d4 Merge pull request #70 from thaJeztah/18.09_backport_upstream_dos_fix
[18.09] backport fix denial of service with large numbers in cpuset-cpus and cpuset-mems
Upstream-commit: 4b8336f7cf091fd5c4742286bda1e34c45667d78
Component: engine
2018-10-11 13:25:55 -07:00
6a1a0dbf9e Merge pull request #72 from thaJeztah/18.09_backport_esc-879
[18.09] backport masking credentials from proxy URL
Upstream-commit: 2697d2b687a629f63d3c5b8fe802c56f3d5dc1a7
Component: engine
2018-10-11 13:25:30 -07:00
127d4b5bad bump buildkit to c7bb575343df0cbfeab8b5b28149630b8153fcc6
Relevant changes:

- buildkit#667 gateway: check for `ReadDir` and `StatFile` caps on client side
- buildkit#668 dockerfile: fix ssh required option
- buildkit#669 dockerfile: update default copy image
- buildkit#670 solver: specify SSH key ID in error message when required key was not forwarded
- buildkit#673 solver: fix possible nil dereference
- buildkit#672 fix setting uncompressed label on content
- buildkit#680 dockerfile: fix empty dest directory panic

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9cfce302145a493571df138888485999cd6172ce)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f58f8421433d18e0fb9a51567068a2ddc1b13a1b
Component: engine
2018-10-11 21:55:49 +02:00
681595daf2 bump up buildkit
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
(cherry picked from commit 837b9c6214cd43828473d15175b0a3c6e1db8507)
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Upstream-commit: 90c72824c36369efd8be52bedd731d12b3415508
Component: engine
2018-10-11 03:01:18 +09:00
2e2a73446a Switch copy image to a docker org based one
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 5cfd110c306d6a370307178b84d2b98d8598acc4)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 7b54720ccbfa5d8242e896f27e8b36ee58612401
Component: engine
2018-10-05 18:01:10 +00:00
00c518411b Fix denial of service with large numbers in cpuset-cpus and cpuset-mems
Using a value such as `--cpuset-mems=1-9223372036854775807` would cause
`dockerd` to run out of memory allocating a map of the values in the
validation code. Set limits to the normal limit of the number of CPUs,
and improve the error handling.

Reported by Huawei PSIRT.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f8e876d7616469d07b8b049ecb48967eeb8fa7a5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0922d32bce74657266aff213f83dfa638e8077f4
Component: engine
2018-10-05 15:13:43 +02:00
8074d84ff1 Update containerd client and dependencies to v1.2.0-rc.1
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit dd622c81a4ea8f4e625f03ee9f13ad47b8746638)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 148d9f0e58bc180fefffcfc0a9e7a00b4276a67a
Component: engine
2018-10-05 14:53:33 +02:00
8be1c880ff Update containerd dependencies
This updates the containerd dependencies to match
the versions used by the vendored containerd version

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 31a9c9e79101cdf38d383104afbc1b48ede75291)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5070e418b806cc96ad0f5b3ac32c8d416ff8449a
Component: engine
2018-10-05 14:38:34 +02:00
c642332896 Fix for default-addr-pool-mask-length param max value check
We check for max value for -default-addr-pool-mask-length param as 32.
But There won't be enough addresses on the  overlay network. Hence we are
keeping it 29 so that we would be having atleast 8 addresses in /29 network.

Signed-off-by: selansen <elango.siva@docker.com>
(cherry picked from commit d25c5df80e60cdbdc23fe3d0e2a6808123643dc7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9406f3622d18a0d9b6c438190e8fdd8be53d3b22
Component: engine
2018-10-04 21:59:25 +02:00
b4ba6169d9 Global Default AddressPool - Update
Addressing few review comments as part of code refactoring.
Also moved validation logic from CLI to Moby.

Signed-off-by: selansen <elango.siva@docker.com>
(cherry picked from commit 148ff00a0a800fad99de11ee3021d4c5d4869157)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9816bfcaf58a609d64d648043c10817c27dcfa36
Component: engine
2018-10-04 21:59:09 +02:00
15addd7d74 Merge pull request #66 from thaJeztah/18.09_backport_fix-dm-errmsg
[18.09] backport: gd/dm: fix error message
Upstream-commit: 52d6ad2a68f84bc83560a4f9971ad6746502abf9
Component: engine
2018-10-04 21:28:22 +02:00
a2008ac193 Masking credentials from proxy URL
Signed-off-by: Dani Louca <dani.louca@docker.com>
(cherry picked from commit 78fd9784542a302c6cae0ab072563c68f9f62711)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 58e51512704b6d7656952e140332472a4c37e46f
Component: engine
2018-10-04 21:20:54 +02:00
39e8b55008 Fix long startup on windows, with non-hns governed Hyper-V networks
Similar to a related issue where previously, private Hyper-V networks
would each add 15 secs to the daemon startup, non-hns governed internal
networks are reported by hns as network type "internal" which is not
mapped to any network plugin (and thus we get the same plugin load retry
loop as before).

This issue hits Docker for Desktop because we setup such a network for
the Linux VM communication.

Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
(cherry picked from commit 6a1a4f97217b0a8635bc21fc86628f48bf8824d1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 54bd14a3fe1d4925c6fa88b24949063d99067c07
Component: engine
2018-10-03 15:24:34 +02:00
1fc8f05911 gd/dm: fix error message
The parameter name was wrong, which may mislead a user.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit c378fb774e413ba8bf5cadf655d2b67e9c94245a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c9ddc6effc444c54def41d498b359a9a986ad79d
Component: engine
2018-10-03 14:01:13 +02:00
cad393d146 Move the syslog syscall to be gated by CAP_SYS_ADMIN or CAP_SYSLOG
This call is what is used to implement `dmesg` to get kernel messages
about the host. This can leak substantial information about the host.
It is normally available to unprivileged users on the host, unless
the sysctl `kernel.dmesg_restrict = 1` is set, but this is not set
by standard on the majority of distributions. Blocking this to restrict
leaks about the configuration seems correct.

Fix #37897

See also https://googleprojectzero.blogspot.com/2018/09/a-cache-invalidation-bug-in-linux.html

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
(cherry picked from commit ccd22ffcc8b564dfc21e7067b5248819d68c56c6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 16836e60bc87abb3e9ab16f33c2038931c1d473b
Component: engine
2018-10-02 20:33:38 +02:00
475e03f71a Merge pull request #62 from thaJeztah/18.09_backport_tweak_error_message
[18.09] backport: tweak bind mount errors
Upstream-commit: e44436c31fce06f9ad69cbb325b781be54516d3e
Component: engine
2018-09-28 14:13:41 -07:00
683cfd7a61 Merge pull request #56 from thaJeztah/18.09_backport_more_permissive_daeon_conf_dir
[18.09] backport loosen permissions on /etc/docker directory
Upstream-commit: 34b3cf4b0cf6867c81cb370b4e2056e0947ee4fd
Component: engine
2018-09-28 11:42:01 -07:00
530db7f45a Tweak bind mount errors
These messages were enhanced to include the path that was
missing (in df6af282b9048dfedcd7b7a9a89126aca887f4e1), but
also changed the first part of the message.

This change complicates running e2e tests with mixed versions
of the engine.

Looking at the full error message, "mount" is a bit redundant
as well, because the error message already indicates this is
about a "mount";

    docker run --rm --mount type=bind,source=/no-such-thing,target=/foo busybox
    docker: Error response from daemon: invalid mount config for type "bind": bind mount source path does not exist: /no-such-thing.

Removing the "mount" part from the error message, because
it was redundant, and makes cross-version testing easier :)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 574db7a53782c57554089c9606505af1c108df0b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b499acc0e834e11882909269238407c65f68f034
Component: engine
2018-09-28 14:35:55 +02:00
f4dc4cdbaa vendor buildkit to 8f4dff0d16ea91cb43315d5f5aa4b27f4fe4e1f2
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit e161a8d1e9e84f50410860d903205e6d2dc86110)
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 67541d5841e645f3408b01f189ec4339df449edc
Component: engine
2018-09-27 22:46:57 +00:00
3be9a1e274 fix daemon tests that were using wrong containerd socket
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 52b60f705ca3f12df01b0cf1002860b842770c74)
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 6bf8dfc4d89461228031a595d63482b9603c8899
Component: engine
2018-09-25 23:09:25 +00:00
1c28fdbba1 hack/make: remove 'docker-' prefix when copying binaries
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 361412c79eeacf11b07871870e875f9e5bce3ea7)
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: e090646d477f2e7d00aba971bcc187f3af7948a3
Component: engine
2018-09-25 23:09:25 +00:00
b48ad13f28 Remove 'docker-' prefix for containerd and runc binaries
This allows to run the daemon in environments that have upstream containerd installed.

Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 34eede0296bce6a9c335cb429f10728ae3f4252d)
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: b3bb2aabb8ed5a8af0a9f48fb5aba3f39af38e0d
Component: engine
2018-09-24 22:35:36 +00:00
42f77ded0c Merge pull request #51 from thaJeztah/18.09_backport_fix-libcontainerd-startup-error
[18.09] backport: Add fail fast path when containerd fails on startup
Upstream-commit: e69efe2ef5d24c3e7a4bd139d34cfa1ff7180869
Component: engine
2018-09-22 00:11:43 -07:00
47296e79f8 vendor: remove boltdb dependency which is superseded by bbolt
This also brings in these PRs from swarmkit:
- https://github.com/docker/swarmkit/pull/2691
- https://github.com/docker/swarmkit/pull/2744
- https://github.com/docker/swarmkit/pull/2732
- https://github.com/docker/swarmkit/pull/2729
- https://github.com/docker/swarmkit/pull/2748

Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: cce1763d57b5c8fc446b0863517bb5313e7e53be
Component: engine
2018-09-22 01:24:11 +00:00
f3dec68e23 builder: vendor buildkit to 39404586a50d1b9d0fb1c578cf0f4de7bdb7afe5
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit d0f00bc1fb2c349ad72ecc2394b8474937de84dd)
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: 3d67dd046539f8e04db82ce07ea56f97b832676b
Component: engine
2018-09-21 17:06:25 -07:00
4f76283213 builder: use buildkit's GC for build cache
This allows users to configure the buildkit GC.

The following enables the default GC:
```
{
  "builder": {
    "gc": {
      "enabled": true
    }
  }
}
```

The default GC policy has a simple config:
```
{
  "builder": {
    "gc": {
      "enabled": true,
      "defaultKeepStorage": "30GB"
    }
  }
}
```

A custom GC policy can be used instead by specifying a list of cache prune rules:
```
{
  "builder": {
    "gc": {
      "enabled": true,
      "policy": [
        {"keepStorage": "512MB", "filter": ["unused-for=1400h"]]},
        {"keepStorage": "30GB", "all": true}
      ]
    }
  }
}
```

Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 4a776d0ca76c4bdf4399aef8c102361d6c2819eb)
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: 73e2f72a7c5bd6d6f8306e0ffe4371e1c3b00a21
Component: engine
2018-09-21 17:06:25 -07:00
dc48089436 add support of registry-mirrors and insecure-registries to buildkit
Signed-off-by: Anda Xu <anda.xu@docker.com>
(cherry picked from commit 171d51c861b479af8798fbe7c51906bcecdffea0)
(cherry picked from commit a72752b2f74467333b4ebe21c6c474eb0c2b99e0)
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: 2926a45be6b9315d2ddeec27d1193278b6bbae91
Component: engine
2018-09-21 17:06:25 -07:00
b69d4cb4ec update vendor
Signed-off-by: Anda Xu <anda.xu@docker.com>
(cherry picked from commit 308701fac6f716160283ca0868da59aba2ea7eb5)
(cherry picked from commit b48afc216f46c8e786560b807528699012e1627b)
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: b73fd4d936864998451cdd37f45694541e43006e
Component: engine
2018-09-21 17:06:25 -07:00
248d11e3f9 daemon/images: removed "found leaked image layer" warning, because it is expected now with buildkit
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 5aa222d0fe5a420dae96be0c04a33308823e0d0f)
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: bb2adc4496f2fd1b755fc701dbed5dab33175efd
Component: engine
2018-09-21 17:06:25 -07:00
ff79736ce2 vendor: update bolt to bbolt
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: b501aa82d5151b8af73d6670772cc4e8ba94765f
Component: engine
2018-09-21 17:06:25 -07:00
a4444691f3 vendor: add bbolt v1.3.1-etcd.8
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: 46a703bb3bfe75e99de2cc457dc0d568a1976a6b
Component: engine
2018-09-21 17:06:25 -07:00
4c429518af Merge pull request #52 from thaJeztah/18.09_backport_fix-TestServiceWithDefaultAddressPoolInit
[18.09] backport TestServiceWithDefaultAddressPoolInit: avoid panic
Upstream-commit: ff9340ca2cc003d0829216524c6f96b98ce62f16
Component: engine
2018-09-21 11:19:57 -07:00
812694c0cc fixed the dockerd won't start bug when 'runtimes' field is defined in both daemon config file and cli flags
Signed-off-by: Anda Xu <anda.xu@docker.com>
(cherry picked from commit 8392d0930b511402aa7aa71ccfe6c0d4a8159237)
Upstream-commit: 66ed41aec82dbcdfbc38027e3d800e429af1cd58
Component: engine
2018-09-20 10:54:47 -07:00
43b6de9f82 Merge pull request #50 from AntaresS/cherry-pick-moby
[18.09] backport propagate the dockerd cgroup-parent config to buildkitd
Upstream-commit: ea2e2c54272d906d527949c5635b95ac8e23b613
Component: engine
2018-09-18 16:36:12 -07:00
97c18c02cb create newBuildKit function separately in daemon_unix.go and daemon_windows.go for cross platform build
Signed-off-by: Anda Xu <anda.xu@docker.com>
(cherry picked from commit 66ac92cdc65be350ec53f1de51052374846dfe24)
Upstream-commit: a5d731edecc75927f602c7f15e5ba9f5f77d3655
Component: engine
2018-09-18 11:19:51 -07:00