This test is failing on Windows currently:
```
11:59:47 --- FAIL: TestHealthKillContainer (8.12s)
11:59:47 health_test.go:57: assertion failed: error is not nil: Error response from daemon: Invalid signal: SIGUSR1
``
That test was added recently in https://github.com/moby/moby/pull/39454, but
rewritten in a commit in the same PR:
f8aef6a92f
In that rewrite, there were some changes:
- originally it was skipped on Windows, but the rewritten test doesn't have that skip:
```go
testRequires(c, DaemonIsLinux) // busybox doesn't work on Windows
```
- the original test used `SIGINT`, but the new one uses `SIGUSR1`
Analysis:
- The Error bubbles up from: 8e610b2b55/pkg/signal/signal.go (L29-L44)
- Interestingly; `ContainerKill` should validate if a signal is valid for the given platform, but somehow we don't hit that part; f1b5612f20/daemon/kill.go (L40-L48)
- Windows only looks to support 2 signals currently 8e610b2b55/pkg/signal/signal_windows.go (L17-L26)
- Upstream Golang looks to define `SIGINT` as well; 77f9b2728e/src/runtime/defs_windows.go (L44)
- This looks like the current list of Signals upstream in Go; 3b58ed4ad3/windows/types_windows.go (L52-L67)
```go
const (
// More invented values for signals
SIGHUP = Signal(0x1)
SIGINT = Signal(0x2)
SIGQUIT = Signal(0x3)
SIGILL = Signal(0x4)
SIGTRAP = Signal(0x5)
SIGABRT = Signal(0x6)
SIGBUS = Signal(0x7)
SIGFPE = Signal(0x8)
SIGKILL = Signal(0x9)
SIGSEGV = Signal(0xb)
SIGPIPE = Signal(0xd)
SIGALRM = Signal(0xe)
SIGTERM = Signal(0xf)
)
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit eeaa0b30d47e6b9dac8d8ea2ced6d5ce44c24463)
Signed-off-by: Dani Louca <dani.louca@docker.com>
Upstream-commit: f481d4c02366093b337e9aebfbbf23b1ff3968fe
Component: engine
Docker daemon always stops healthcheck before sending signal to a
container now. However, when we use "docker kill" to send signals
other than SIGTERM or SIGKILL to a container, such as SIGINT,
daemon still stops container health check though container process
handles the signal normally and continues to work.
Signed-off-by: Ruilin Li <liruilin4@huawei.com>
(cherry picked from commit da574f93432e600fda561da5e6983e7f69b364a9)
Signed-off-by: Dani Louca <dani.louca@docker.com>
Upstream-commit: afee55c3740644be40882d883d8287782f2bc6c2
Component: engine
For now, just verifying that an error is returned, but not checking the
error message itself, because those are not under our control, and may
change with different Go versions.
```
=== Failed
=== FAIL: opts TestParseDockerDaemonHost (0.00s)
hosts_test.go:87: tcp tcp:a.b.c.d address expected error "Invalid bind address format: tcp:a.b.c.d" return, got "parse tcp://tcp:a.b.c.d: invalid port \":a.b.c.d\" after host" and addr
hosts_test.go:87: tcp tcp:a.b.c.d/path address expected error "Invalid bind address format: tcp:a.b.c.d/path" return, got "parse tcp://tcp:a.b.c.d/path: invalid port \":a.b.c.d\" after host" and addr
=== FAIL: opts TestParseTCP (0.00s)
hosts_test.go:129: tcp tcp:a.b.c.d address expected error Invalid bind address format: tcp:a.b.c.d return, got parse tcp://tcp:a.b.c.d: invalid port ":a.b.c.d" after host and addr
hosts_test.go:129: tcp tcp:a.b.c.d/path address expected error Invalid bind address format: tcp:a.b.c.d/path return, got parse tcp://tcp:a.b.c.d/path: invalid port ":a.b.c.d" after host and addr
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit de1523d221)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: feb68f9055
Component: cli
go1.11.13 (released 2019/08/13) includes security fixes to the net/http and net/url packages.
See the Go 1.11.13 milestone on our issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.13
- net/http: Denial of Service vulnerabilities in the HTTP/2 implementation
net/http and golang.org/x/net/http2 servers that accept direct connections from untrusted
clients could be remotely made to allocate an unlimited amount of memory, until the program
crashes. Servers will now close connections if the send queue accumulates too many control
messages.
The issues are CVE-2019-9512 and CVE-2019-9514, and Go issue golang.org/issue/33606.
Thanks to Jonathan Looney from Netflix for discovering and reporting these issues.
This is also fixed in version v0.0.0-20190813141303-74dc4d7220e7 of golang.org/x/net/http2.
net/url: parsing validation issue
- url.Parse would accept URLs with malformed hosts, such that the Host field could have arbitrary
suffixes that would appear in neither Hostname() nor Port(), allowing authorization bypasses
in certain applications. Note that URLs with invalid, not numeric ports will now return an error
from url.Parse.
The issue is CVE-2019-14809 and Go issue golang.org/issue/29098.
Thanks to Julian Hector and Nikolai Krein from Cure53, and Adi Cohen (adico.me) for discovering
and reporting this issue.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9a7c1a1e5f0858ac54f1c7577c4fca8665f70496
Component: packaging
go1.11.13 (released 2019/08/13) includes security fixes to the net/http and net/url packages.
See the Go 1.11.13 milestone on our issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.13
- net/http: Denial of Service vulnerabilities in the HTTP/2 implementation
net/http and golang.org/x/net/http2 servers that accept direct connections from untrusted
clients could be remotely made to allocate an unlimited amount of memory, until the program
crashes. Servers will now close connections if the send queue accumulates too many control
messages.
The issues are CVE-2019-9512 and CVE-2019-9514, and Go issue golang.org/issue/33606.
Thanks to Jonathan Looney from Netflix for discovering and reporting these issues.
This is also fixed in version v0.0.0-20190813141303-74dc4d7220e7 of golang.org/x/net/http2.
net/url: parsing validation issue
- url.Parse would accept URLs with malformed hosts, such that the Host field could have arbitrary
suffixes that would appear in neither Hostname() nor Port(), allowing authorization bypasses
in certain applications. Note that URLs with invalid, not numeric ports will now return an error
from url.Parse.
The issue is CVE-2019-14809 and Go issue golang.org/issue/29098.
Thanks to Julian Hector and Nikolai Krein from Cure53, and Adi Cohen (adico.me) for discovering
and reporting this issue.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: aed09dc7eb
Component: cli
go1.11.13 (released 2019/08/13) includes security fixes to the net/http and net/url packages.
See the Go 1.11.13 milestone on our issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.13
- net/http: Denial of Service vulnerabilities in the HTTP/2 implementation
net/http and golang.org/x/net/http2 servers that accept direct connections from untrusted
clients could be remotely made to allocate an unlimited amount of memory, until the program
crashes. Servers will now close connections if the send queue accumulates too many control
messages.
The issues are CVE-2019-9512 and CVE-2019-9514, and Go issue golang.org/issue/33606.
Thanks to Jonathan Looney from Netflix for discovering and reporting these issues.
This is also fixed in version v0.0.0-20190813141303-74dc4d7220e7 of golang.org/x/net/http2.
net/url: parsing validation issue
- url.Parse would accept URLs with malformed hosts, such that the Host field could have arbitrary
suffixes that would appear in neither Hostname() nor Port(), allowing authorization bypasses
in certain applications. Note that URLs with invalid, not numeric ports will now return an error
from url.Parse.
The issue is CVE-2019-14809 and Go issue golang.org/issue/29098.
Thanks to Julian Hector and Nikolai Krein from Cure53, and Adi Cohen (adico.me) for discovering
and reporting this issue.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: d122605850fca5df1ff8babe7ee9f1dfed2a335b
Component: engine
1. Use "in-place" variables for if statements to limit their scope to
the respectful `if` block.
2. Report the error returned from sd_journal_* by using CErr().
3. Use errors.New() instead of fmt.Errorf().
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 20a0e58a794cfb9b1a1f757d222248e22555f7f0)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: b32bc11a00a82584138ae78e083930918dcb0b36
Component: engine
From the first glance, `docker logs --tail 0` does not make sense,
as it is supposed to produce no output, but `tail -n 0` from GNU
coreutils is working like that, plus there is even a test case
(`TestLogsTail` in integration-cli/docker_cli_logs_test.go).
Now, something like `docker logs --follow --tail 0` makes total
sense, so let's make it work.
(NOTE if --tail is not used, config.Tail is set to -1)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit dd4bfe30a8ac1b31630310090dc36ae3d9253444)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 4427414624c56c459431d3dbbde81f3396cb02b0
Component: engine
If we take a long time to process log messages, and during that time
journal file rotation occurs, the journald client library will keep
those rotated files open until sd_journal_process() is called.
By periodically calling sd_journal_process() during the processing
loop we shrink the window of time a client instance has open file
descriptors for rotated (deleted) journal files.
This code is modelled after that of journalctl [1]; the above explanation
as well as the value of 1024 is taken from there.
[v2: fix CErr() argument]
[1] https://github.com/systemd/systemd/blob/dc16327c48d/src/journal/journalctl.c#L2676
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit b73fb8fd5d521081c92b5c2cce334c21b2e0ff5f)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 48160719f2c5734b513aec7cedbbca0c72f4a085
Component: engine
TL;DR: simplify the code, fix --follow hanging indefinitely
Do the following to simplify the followJournal() code:
1. Use Go-native select instead of C-native polling.
2. Use Watch{Producer,Consumer}Gone(), eliminating the need
to have journald.closed variable, and an extra goroutine.
3. Use sd_journal_wait(). In the words of its own man page:
> A synchronous alternative for using sd_journal_get_fd(),
> sd_journal_get_events(), sd_journal_get_timeout() and
> sd_journal_process() is sd_journal_wait().
Unfortunately, the logic is still not as simple as it
could be; the reason being, once the container has exited,
journald might still be writing some logs from its internal
buffers onto journal file(s), and there is no way to
figure out whether it's done so we are guaranteed to
read all of it back. This bug can be reproduced with
something like
> $ ID=$(docker run -d busybox seq 1 150000); docker logs --follow $ID
> ...
> 128123
> $
(The last expected output line should be `150000`).
To avoid exiting from followJournal() early, add the
following logic: once the container is gone, keep trying
to drain the journal until there's no new data for at
least `waitTimeout` time period.
Should fix https://github.com/docker/for-linux/issues/575
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit f091febc942859ffbc881f3a3aa327366603ae65)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: be568f93432ed1399a0aacca5fbd2c1046ccbc85
Component: engine
1. The journald client library initializes inotify watch(es)
during the first call to sd_journal_get_fd(), and it make sense
to open it earlier in order to not lose any journal file rotation
events.
2. It only makes sense to call this if we're going to use it
later on -- so add a check for config.Follow.
3. Remove the redundant call to sd_journal_get_fd().
NOTE that any subsequent calls to sd_journal_get_fd() return
the same file descriptor, so there's no real need to save it
for later use in wait_for_data_cancelable().
Based on earlier patch by Nalin Dahyabhai <nalin@redhat.com>.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 981c01665bcb2c9fc5e555c5b976995f31c2a6b4)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 349e199eab5337f03b442f38720293143e1b1fca
Component: engine
In case the LogConsumer is gone, the code that sends the message can
stuck forever. Wrap the code in select case, as all other loggers do.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 79039720c8b7352691350bd56be3cc226d67f205)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 56a8a516127a2630c3a68ab31c416f223b91e9df
Component: engine
In case Tail=N parameter is requested, we need to show N lines.
It does not make sense to walk backwards one by one if we can
do it at once. Now, if Since=T is also provided, make sure we
haven't jumped too far (before T), and if we did, move forward.
The primary motivation for this was to make the code simpler.
This also fixes a tiny bug in the "since" implementation.
Before this commit:
> $ docker logs -t --tail=6000 --since="2019-03-10T03:54:25.00" $ID | head
> 2019-03-10T03:54:24.999821000Z 95981
After:
> $ docker logs -t --tail=6000 --since="2019-03-10T03:54:25.00" $ID | head
> 2019-03-10T03:54:25.000013000Z 95982
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit ff3cd167ea4d089b7695a263ba2fc4caa0a0750c)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 2a124db7da440f1efd4c2957320d8b25d9d9ce36
Component: engine
Minor code simplification.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit e8f6166791c097deb15c39f8dddf6f97be65b224)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 1d336dc53bd7bf5941596ffeb253d102de609a51
Component: engine
Clean up a deferred function call in the journal reading logic.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
(cherry picked from commit 1ada3e85bf89201910c28f2ff6892c00cee0f137)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: e700930ca521d0c004b6a3ed8bdd35a2d538aa15
Component: engine
As in other similar drivers (jsonlog, local), use a set
(i.e. `map[whatever]struct{}`), making the code simpler.
While at it, make sure we remove the reader from the set
after calling `ProducerGone()` on it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit b2b169f13f681cd0d591ccb06d6cfff97933db77)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: fe85c72a2eac4cbf249d2c4c754684bb447eefdd
Component: engine
There are a few more places, apparently, that List operations against
Swarm exist, besides just in the List methods. This increases the max
received message size in those places.
Signed-off-by: Drew Erny <drew.erny@docker.com>
(cherry picked from commit a84a78e9767d82abd4744dad9ce4fb3f64141a8f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 41da428d065ab15fd2d8aba1fbd0d0056136a43b
Component: engine
full diff: 19e791fd6d...142a73731c
included:
- docker/swarmkit#2872 [19.03 backport] Only update non-terminal tasks on node removal
- backport of docker/swarmkit#2867 Only update non-terminal tasks on node removal
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 7fcfdbaab6d8c6a2d55ad7b72a851c5e92b360ac
Component: engine
The Golang base images switch to buster, which causes some breakage
in networking and packages that are no longer available; (`btrfs-tools`
is now an empty package, and `libprotobuf-c0-dev` is gone).
Some of out tests also start faiilng on stretch, and will have to be
investigated further;
```
15:13:06 --- FAIL: TestRenameAnonymousContainer (3.37s)
15:13:06 rename_test.go:168: assertion failed: 0 (int) != 1 (inspect.State.ExitCode int): container a7fe866d588d65f353f42ffc5ea5288e52700384e1d90850e9c3d4dce8657666 exited with the wrong exitcode:
15:13:38 --- FAIL: TestHostnameDnsResolution (2.23s)
15:13:38 run_linux_test.go:128: assertion failed:
15:13:38 --- ←
15:13:38 +++ →
15:13:38 @@ -1 +1,2 @@
15:13:38 +ping: bad address 'foobar'
15:13:38
15:13:38
15:13:38 run_linux_test.go:129: assertion failed: 0 (int) != 1 (res.ExitCode int)
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ed672bb523cb255d0b2b79837d9c45a7c3255000)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 3c1bc29b2c372d0f1e37dc8aac33cdc01b623aa0
Component: engine
This check was used to make sure we don't bump Go versions independently
(Linux/Windows). The Dockerfile switched to using a build-arg to allow
overriding the Go version, which rendered this check non-functional.
It also fails if Linux versions use a specific variant of the image;
08:41:31 ERROR: Failed 'ERROR: Mismatched GO versions between Dockerfile and Dockerfile.windows. Update your PR to ensure that both files are updated and in sync. ${GO_VERSION}-stretch ${GO_VERSION}' at 07/20/2019 08:41:31
08:41:31 At C:\gopath\src\github.com\docker\docker\hack\ci\windows.ps1:448 char:9
08:41:31 + Throw "ERROR: Mismatched GO versions between Dockerfile and D ...
08:41:31 + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This patch fixes the check by looking for the value of `GO_VERSION` instead
of looking at the `FROM` line (which is harder to parse).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4fa57a8191b1d23c6466725b688519f83c0ac5dd)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: a2d32c5654a206954fd0d9a5fd08abd8c03d4523
Component: engine
This was an oversight when changing the Dockerfile to use a build-arg;
the Windows Dockerfile downloads the Go binaries, which never have a
trailing `.0`.
This patch makes sure that the trailing zero (if any) is removed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c5bd6e3dc7680d6c683496f63dafb1f30f87eaa7)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 8f1a27c51f93698b5fe17f0fb043f47794e11cca
Component: engine
This allows overriding the version of Go without making modifications in the
source code, which can be useful to test against multiple versions.
For example:
make GO_VERSION=1.13beta1 shell
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c6281bc4383b7f9eab617fd73601e8594c93365b)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Conflicts:
* Dockerfile, Dockerfile.e2e, Dockerfile.simple, Dockerfile.windows:
(due to Go version difference, missing CROSS etc.)
Upstream-commit: d9ba337adb0c3880acd34ffe9b0a15717990876b
Component: engine
Protect access to q.quotas map, and lock around changing nextProjectID.
Techinically, the lock in findNextProjectID() is not needed as it is
only called during initialization, but one can never be too careful.
Fixes: 52897d1c092 ("projectquota: utility class for project quota controls")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 1ac0a66a64a906911d0708cd0e5fa397a2f0b595)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 7027bb9bedae63879c1e41894739ba0ea2deedc1
Component: engine
The docker-in-docker image now enables TLS by default (added in
docker-library/docker#166), which complicates testing in our
environment, and isn't needed for the tests we're running.
This patch sets the `DOCKER_TLS_CERTDIR` to an empty value to
disable TLS.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b1a3c1aad1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9c8ac0a123
Component: cli
The edge channel is deprecated and no longer updated
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 08fd6dd63c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a88330d9dd
Component: cli
[18.09] Initialize nss libraries in Glibc so that the dynamic libraries are l…
Upstream-commit: 8ff992ae26ecedb7b80ee71be902d0fbc0879861
Component: engine
The fastly cdn mirror we're using also mirrors the debian security
repository;
```
Welcome to deb.debian.org (fastly instance)!
This is deb.debian.org. This service provides mirrors for the following Debian archive repositories:
/debian/
/debian-debug/
/debian-ports/
/debian-security/
The server deb.debian.org does not have packages itself, but the name has SRV records in DNS that let apt in stretch and later find places.
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c8f43b5f6f7c83cfb5570f7f013c7efaa430d285)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Conflicts:
* Dockerfile (GO_VERSION value differs, and CROSS (#39010) is absent)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 6e603e957eec3a46db4d8d69c3a7946c4d2e9c59
Component: engine
As of Alpine Linux 3.3 (or 3.2?) there exists a new --no-cache
option for apk. It allows users to install packages with an index
that is updated and used on-the-fly and not cached locally.
This avoids the need to use --update and remove /var/cache/apk/*
when done installing packages.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 3b44dd66a4b316049df86521c491b50466d9b55d)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: a4ae2af0e59018a309d4bc71ced729b34e906564
Component: engine
This helps merge conflicts in situations where downstream
projects have additional dependencies.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8c5460a2cc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 7b3d023163
Component: cli
To make it better readable, and to encourage pinning
by sha, but "align" to a tagged release.
similar to 6026ce4a8b (#1822)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1764ffafca
Component: cli
I would like to use the regex matcher
Signed-off-by: Ian Campbell <ijc@docker.com>
(cherry picked from commit 986196e3e3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c1c3add698
Component: cli
Path-specific rules were removed, so this is no longer used.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 530e63c1a61b105a6f7fc143c5acb9b5cd87f958)
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit f8a0f26843bc5aff33cf9201b75bd4bdbb48a3ad)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a11c3098a3d5106b2d7c90d971c9300099556a49
Component: engine
Commit 77b8465d7e68ca102d7aae839c7b3fe0ecd28398 added a secret update
endpoint to allow updating labels on existing secrets. However, when
implementing the endpoint, the DebugRequestMiddleware was not updated
to scrub the Data field (as is being done when creating a secret).
When updating a secret (to set labels), the Data field should be either
`nil` (not set), or contain the same value as the existing secret. In
situations where the Data field is set, and the `dockerd` daemon is
running with debugging enabled / log-level debug, the base64-encoded
value of the secret is printed to the daemon logs.
The docker cli does not have a `docker secret update` command, but
when using `docker stack deploy`, the docker cli sends the secret
data both when _creating_ a stack, and when _updating_ a stack, thus
leaking the secret data if the daemon runs with debug enabled:
1. Start the daemon in debug-mode
dockerd --debug
2. Initialize swarm
docker swarm init
3. Create a file containing a secret
echo secret > my_secret.txt
4. Create a docker-compose file using that secret
cat > docker-compose.yml <<'EOF'
version: "3.3"
services:
web:
image: nginx:alpine
secrets:
- my_secret
secrets:
my_secret:
file: ./my_secret.txt
EOF
5. Deploy the stack
docker stack deploy -c docker-compose.yml test
6. Verify that the secret is scrubbed in the daemon logs
DEBU[2019-07-01T22:36:08.170617400Z] Calling POST /v1.30/secrets/create
DEBU[2019-07-01T22:36:08.171364900Z] form data: {"Data":"*****","Labels":{"com.docker.stack.namespace":"test"},"Name":"test_my_secret"}
7. Re-deploy the stack to trigger an "update"
docker stack deploy -c docker-compose.yml test
8. Notice that this time, the Data field is not scrubbed, and the base64-encoded secret is logged
DEBU[2019-07-01T22:37:35.828819400Z] Calling POST /v1.30/secrets/w3hgvwpzl8yooq5ctnyp71v52/update?version=34
DEBU[2019-07-01T22:37:35.829993700Z] form data: {"Data":"c2VjcmV0Cg==","Labels":{"com.docker.stack.namespace":"test"},"Name":"test_my_secret"}
This patch modifies `maskSecretKeys` to unconditionally scrub `Data` fields.
Currently, only the `secrets` and `configs` endpoints use a field with this
name, and no other POST API endpoints use a data field, so scrubbing this
field unconditionally will only scrub requests for those endpoints.
If a new endpoint is added in future where this field should not be scrubbed,
we can re-introduce more fine-grained (path-specific) handling.
This patch introduces some change in behavior:
- In addition to secrets, requests to create or update _configs_ will
now have their `Data` field scrubbed. Generally, the actual data should
not be interesting for debugging, so likely will not be problematic.
In addition, scrubbing this data for configs may actually be desirable,
because (even though they are not explicitely designed for this purpose)
configs may contain sensitive data (credentials inside a configuration
file, e.g.).
- Requests that send key/value pairs as a "map" and that contain a
key named "data", will see the value of that field scrubbed. This
means that (e.g.) setting a `label` named `data` on a config, will
scrub/mask the value of that label.
- Note that this is already the case for any label named `jointoken`,
`password`, `secret`, `signingcakey`, or `unlockkey`.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c7ce4be93ae8edd2da62a588e01c67313a4aba0c)
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 73db8c77bfb2d0cbdf71ce491f3d3e66c9dd5be6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 32b40c53662e733b4627b0b303c71b52484a31f4
Component: engine
1. Adds `docker events` description info on the two scope types of events.
2. Adds `docker events` note in two places about backlog limit of event log.
Further info and background info in Issue 727
Signed-off-by: Bret Fisher <bret@bretfisher.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 988b9a0d96)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 74d51dc13b
Component: cli
When deploying a stack using a relative path as bind-mount
source in the compose file, the CLI converts the relative
path to an absolute path, relative to the location of the
docker-compose file.
This causes a problem when deploying a stack that uses
an absolute Windows path, because a non-Windows client will
fail to detect that the path (e.g. `C:\somedir`) is an absolute
path (and not a relative directory named `C:\`).
The existing code did already take Windows clients deploying
a Linux stack into account (by checking if the path had a leading
slash). This patch adds the reverse, and adds detection for Windows
absolute paths on non-Windows clients.
The code used to detect Windows absolute paths is copied from the
Golang filepath package;
1d0e94b1e1/src/path/filepath/path_windows.go (L12-L65)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d6dd08d568)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a757fad956
Component: cli
go1.11.11 (released 2019/06/11) includes a fix to the crypto/x509 package.
See the Go 1.11.11 milestone on the issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.11
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f33c398df1eabda69ca05ff6c5c45351790a8696
Component: packaging
go1.11.3 (released 2018/12/14)
- crypto/x509: CPU denial of service in chain validation golang/go#29233
- cmd/go: directory traversal in "go get" via curly braces in import paths golang/go#29231
- cmd/go: remote command execution during "go get -u" golang/go#29230
See the Go 1.11.3 milestone on the issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 517a30b48d7b483e27eb3c3328356a8e83272988)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 17565b93d361b1b189962b3e98c431316a7cc628
Component: packaging
[18.09] switch to balenalib/rpi-raspbian because resin/rpi-raspbian is deprecated
Upstream-commit: e8cb60baf6abb7b2cd50c9991056891be462b839
Component: packaging
version 2.9 is really old; this sets the same minimal version
as is used for the containerd.io package
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a246b19b0723d41a8e58ff42d3af52fd57a7e94d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 95dea749d4cb7e474194bbe209f052e94da5df2e
Component: packaging
This code was attempting to check Linux file permissions
to determine if the key was accessible by other users, which
doesn't work, and therefore prevented users on Windows
to load keys.
Skipping this check on Windows (correspinding tests
were already skipped).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 15d361fd77)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 6be8fce6f8
Component: cli
Full diff: https://github.com/golang/go/compare/go1.11.5...go1.11.9
go1.11.6 (released 2019/03/14) includes fixes to cgo, the compiler, linker,
runtime, go command, and the crypto/x509, encoding/json, net, and net/url
packages. See the Go 1.11.6 milestone on our issue tracker for details.
https://github.com/golang/go/issues?q=milestone%3AGo1.11.6
go1.11.7 (released 2019/04/05) includes fixes to the runtime and the net
packages. See the Go 1.11.7 milestone on our issue tracker for details.
https://github.com/golang/go/issues?q=milestone%3AGo1.11.7
go1.11.8 (released 2019/04/08) was accidentally released without its
intended fix. It is identical to go1.11.7, except for its version number.
The intended fix is in go1.11.9.
go1.11.9 (released 2019/04/11) fixes an issue where using the prebuilt
binary releases on older versions of GNU/Linux led to failures when linking
programs that used cgo. Only Linux users who hit this issue need to update.
See golang/go#31293 for details
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2cc1df06ee264ab5d1769bd4aae9e417feda9402
Component: engine
go1.11.4 (released 2018/12/14) includes fixes to cgo, the compiler, linker,
runtime, documentation, go command, and the net/http and go/types packages. It
includes a fix to a bug introduced in Go 1.11.3 that broke go get for import
path patterns containing "...".
See the Go 1.11.4 milestone for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.4+label%3ACherryPickApproved
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3770f386477e5eea4e6ba02516393a1edd5cd28b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e05367a1ba3cbf2cd7f99fef73908b900b57b929
Component: engine
go1.11.13 (released 2018/12/14)
- crypto/x509: CPU denial of service in chain validation golang/go#29233
- cmd/go: directory traversal in "go get" via curly braces in import paths golang/go#29231
- cmd/go: remote command execution during "go get -u" golang/go#29230
See the Go 1.11.3 milestone on the issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6b7c093b0de21d574ce120aee891e60187749174)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 19d37c9a337e82e0e8ce0ff28271739e8ec78e83
Component: engine
Includes fixes to the compiler, linker, documentation, go command, and the
database/sql and go/types packages. See the Go 1.11.2 milestone on the issue
tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e80ee5206e6e62521c8149381d06bbb0bea88169)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f9a71f917f0979742b1d7ee606fb596e873d773c
Component: engine
go1.11.1 (released 2018/10/01) includes fixes to the compiler,
documentation, go command, runtime, and the crypto/x509, encoding/json,
go/types, net, net/http, and reflect packages.
See the Go 1.11.1 milestone on our issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.1
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0347751117513312be72e8b0d03f32319027f145)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 6b397d155bf464a94fa05620a485d969edf42393
Component: engine
As per https://github.com/etcd-io/etcd/blob/fa57f7fbc787b4/Gopkg.lock
List of packages required by subset of etcd used is provided by:
go list -f '{{join .Deps "\n"}}' \
github.com/docker/docker/vendor/github.com/coreos/etcd/... \
| grep -F . | grep -v coreos/etcd | sort | uniq
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 7008ac01fa4dbde9a069cc647a09c982f3648fc3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c4bf0803154de1b26ee3437c7fd9307722e2a5e4
Component: engine
We can do that now as we're no longer carrying archive/tar.
Note that latest vndr removes vendor/ subdir so we don't have to,
thus the change in hack/validate/vendor.
While at it, re-run a new vndr version to make sure everything
that should be there is.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit ce858feb6ac124b3a42400b857e480ef98710164)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2f2588f137f642b0c1520e8a41ec731040d0b336
Component: engine
It's that time of year again! Go 1.11 is released, time to use it.
This commit also
* removes our archive/tar fork, since upstream archive/tar
is fixed for static builds, and osusergo build tag is set.
* removes ENV GO_VERSION from Dockerfile as it's not needed
anymore since PR #37592 is merged.
[v2: switch to beta2]
[v3: switch to beta3]
[v4: rc1]
[v5: remove ENV GO_VERSION as PR #37592 is now merged]
[v6: rc2]
[v7: final!]
[v8: use 1.11.0]
[v9: back to 1.11]
[v8: use 1.11.0]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 10fd0516b9f9f04d0f0e2c0755e704303f1a487f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ebf396050d6e977df2669d5e7d3f38098719f7c2
Component: engine
We would like to use a version with .0 suffix (like 1.11.0) in
Dockerfile, so that once a .1 version is out (like 1.11.1) we
won't accidentally switch to it.
Unfortunately it's not possible to use .0 suffix currently
as it breaks the check in make.ps1. This patch fixes that.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 8ad648b59a2993156586458cc9ff8dcbc89b86e4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: dc5371b4134dd6474a467308da85daaa27914a64
Component: engine
Instead of installing golang from sources, it's easier to use
golang image which is based on Debian Stretch.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 32a23311033dc5bfdfbc80c54a159cc92990efd2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2e82d410bef320096e316ef7a3db8797694f88fe
Component: engine
To include the following backported fix:
https://github.com/kolyshkin/ugorji-go/commit/1cf431c13dec46596
which should fix this:
> 13:40:53 vendor/github.com/ugorji/go/codec/gen-helper.generated.go:1:
> possible malformed +build comment%!(EXTRA []interface {}=[])
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 09921ca33fb1cdb7648143e787a2a6222d844eb0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ca9d80ea6a07f54b6171cd3cb1a5125948fd5fdc
Component: engine
This should eliminate a bunch of new (go-1.11 related) validation
errors telling that the code is not formatted with `gofmt -s`.
No functional change, just whitespace (i.e.
`git show --ignore-space-change` shows nothing).
Patch generated with:
> git ls-files | grep -v ^vendor/ | grep .go$ | xargs gofmt -s -w
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 9b0097a69900009ab5c2480e047952cba60462a7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ee28567c7066368207a947e02c6242db7a4adb16
Component: engine
This is set only if it is not already set.
This should give a little speedup to CI builds.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 1275a001a68722494d090d5beca6749a83710cc2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e64cd6abed308a26d9048d4d0e4c52207c2ad5df
Component: engine
comparing PR commit(s) to HEAD of moby/moby master branch and if founds
new (or renamed) integration tests will run stress tests for them.
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
(cherry picked from commit 8a8fd37f6fb53716cb4b3a7e93e1e3cf385927e2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e5c0923b277055c977994ebfdc689346c3104a16
Component: engine
This patch;
- adds support for multiple newlines
- removes the 1-space indentation of wrapped lines
- allows numerical characters in targets (0-9)
Given these targets:
```Makefile
.PHONY: foobar
foobar: ## runs the foobar lorum ipsum.\nand so pn\nand so on
echo foobar
.PHONY: e2e-tests
e2e-tests: ## runs the end-to-end tests
echo e2e-tests
```
Before this change, the output of `make help` was
```
foobar runs the foobar lorum ipsum.
and so pn\nand so on
```
After this change, the output is:
```
foobar runs the foobar lorum ipsum.
and so pn
and so on
e2e-tests runs the end-to-end tests
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 202c9d8c98614e7cce2017f5c99d3d783fe8b509)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 733b86683f347f922affc714d305d9f03fe9b47e
Component: engine
build the final stage of the Dockerfile (including COPY ...) if no BIND_DIR
is used.
if BIND_DIR is used, build the dev stage, thus skipping the COPY.
Original author: @thaJeztah
Signed-off-by: Mohammad Nasirifar <farnasirim@gmail.com>
(cherry picked from commit e6d7df2e5d313800414b955e10a26d6687e7a1bf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e5a039169f389120be3fbc2a9db45ef8bc99eae5
Component: engine
Through the env variable of the same name.
The idea here is pretty simple: I/O perf on native mounted disks
on non-Linux (notably Mac OS) is just terrible, thus making it
a real pain to develop: one has to choose between re-building
the image after every single change (eg to run a test) or just
work directly inside the same container (eg with vim, but even then
one would have to re-configure their dev container every time
it gets destroyed - containers, after all, are not supposed to
be long-lived).
Allowing to override `DOCKER_MOUNT` makes it easy for everyone
to decide what their volume/syncing strategy is; for example
one can choose to use [docker-sync](https://github.com/EugenMayer/docker-sync)
This patch won't change anything for anyone who doesn't
set the `DOCKER_MOUNT` env variable in their environment.
Signed-off-by: Jean Rouge <jer329@cornell.edu>
(cherry picked from commit aea6fdf3d340835a1b0af208839ce42ace3a5b89)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ecc423df7fde7f0edc963f03d70da133f03bb63c
Component: engine
Looks like -i (together with DOCKER_INCREMENTAL_BINARY etc)
were used to get faster incremental builds.
Nowdays (since Go 1.10) this is no longer the case, as
go build cache is used [1]. Here's a quote:
> You do not have to use "go test -i" or "go build -i" or
> "go install" just to get fast incremental builds. We will
> not have to teach new users those workarounds anymore.
> Everything will just be fast.
To enable go cache between builds, add a volume for /root/.cache.
[1] https://groups.google.com/forum/#!msg/golang-dev/qfa3mHN4ZPA/X2UzjNV1BAAJ
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit bdcd81d3301a053eefc320de16ac842ec47ed459)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0f190f798f7b9f90bd008fe8fb0fc087ec965642
Component: engine
[18.09 backport] Consider WINDOWS_BASE_IMAGE_TAG override when setting Windows base image for tests
Upstream-commit: bb54c5bf2a3ab1c698fda6b3d67773d25755ae45
Component: engine
Previously, getWalkRoot("/", "foo") would return "//foo"
Now it returns "/foo"
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 7410f1a859063d4ed3d8fca44f27bdde4c2cb5a3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8677bbe3f31a8e4516e1ba413ce1063c803ea10c
Component: engine
Before 7a7357da, archive.TarResourceRebase was being used to copy files
and folders from the container. That function splits the source path
into a dirname + basename pair to support copying a file:
if you wanted to tar `dir/file` it would tar from `dir` the file `file`
(as part of the IncludedFiles option).
However, that path splitting logic was kept for folders as well, which
resulted in weird inputs to archive.TarWithOptions:
if you wanted to tar `dir1/dir2` it would tar from `dir1` the directory
`dir2` (as part of IncludedFiles option).
Although it was weird, it worked fine until we started chrooting into
the container rootfs when doing a `docker cp` with container source set
to `/` (cf 3029e765).
The fix is to only do the path splitting logic if the source is a file.
Unfortunately, 7a7357da added support for LCOW by duplicating some of
this subtle logic. Ideally we would need to do more refactoring of the
archive codebase to properly encapsulate these behaviors behind well-
documented APIs.
This fix does not do that. Instead, it fixes the issue inline.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 171538c190ee3a1a8211946ab8fa78cdde54b47a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 70a837138aa1c5c0260a005b470ac0c5717984b4
Component: engine
- moby/buildkit#952 [18.09 backport] Have parser error on dockerfiles without instructions
- backport of moby/buildkit#771 Have parser error on dockerfiles without instructions
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a1d76e0585c498f5167b28b9909fd8c88fd3370d
Component: engine
Make sure adapter.removeNetworks executes during task Remove
adapter.removeNetworks was being skipped for cases when
isUnknownContainer(err) was true after adapter.remove was executed
This fix eliminates the nil return case forcing the function
to continue executing unless there is a true error
Fixes https://github.com/moby/moby/issues/39225
Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
(cherry picked from commit 70fa7b6a3fd9aaada582ae02c50710f218b54d1a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 75887d37e1ddbef579e239ff0b1b7a2508e486fd
Component: engine
make sure the LB sandbox is removed when a service is updated
with a --network-rm option
Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
(cherry picked from commit 680d0ba4abfb00c8ac959638c72003dba0826b46)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 280b8dff7df137dd21fcf27cacf6d12ed531770a
Component: engine
This fix moves multiple places of serviceRunningTasksCount
to one location in integration/internal/swarm, so that
code duplication could be removed.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
(cherry picked from commit e485a60e2bcc59860f387c94f6afaa0130ea7040)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c087f681d40332bd7d158baf615c03f6a72274f1
Component: engine
Tests which will re-deploy containers uses function serviceIsUpdated() to
make sure that service update really reached state UpdateStateCompleted.
Tests which will not re-deploy container uses function
serviceSpecIsUpdated to make sure that service version is increased.
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
(cherry picked from commit b868ada474b5c214ed9bddb792dd36f794dc1fa6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ccc1abea092a54d99dec51345ef9303b4629572d
Component: engine
[18.09 backport] explicitly set filesystem type for mount to avoid 'invalid argument' error on arm
Upstream-commit: 51ebfcbe423c679f06cfb37f7e6be565e0955639
Component: engine
[18.09 backport] bugfix: fetch the right device number which great than 255
Upstream-commit: b236a1e78d35a62b164d00f4e19f59a556da8831
Component: engine
[18.09 backport] Ensure all integration daemon logging happens before test exit
Upstream-commit: 08f6e9c14fd0de408db93914058d75df5b32ac67
Component: engine
go1.11.9 (released 2019/04/11) fixes an issue where using the prebuilt
binary releases on older versions of GNU/Linux led to failures when linking
programs that used cgo. Only Linux users who hit this issue need to update.
See golang/go#31293 for details
Full diff: https://github.com/golang/go/compare/go1.11.8...go1.11.9
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1b3ed84535
Component: cli
go1.11.6 (released 2019/03/14) includes fixes to cgo, the compiler, linker,
runtime, go command, and the crypto/x509, encoding/json, net, and net/url
packages. See the Go 1.11.6 milestone on our issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.6
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1500105975)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9959062d9a
Component: cli
go1.11.4 (released 2018/12/14) includes fixes to cgo, the compiler, linker,
runtime, documentation, go command, and the net/http and go/types packages. It
includes a fix to a bug introduced in Go 1.11.3 that broke go get for import
path patterns containing "...".
See the Go 1.11.4 milestone for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.4+label%3ACherryPickApproved
go1.11.3 (released 2018/12/14)
- crypto/x509: CPU denial of service in chain validation golang/go#29233
- cmd/go: directory traversal in "go get" via curly braces in import paths golang/go#29231
- cmd/go: remote command execution during "go get -u" golang/go#29230
See the Go 1.11.3 milestone on the issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit deaf6e13ab)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a378a00954
Component: cli
go1.11.2 (released 2018/11/02) includes fixes to the compiler, linker,
documentation, go command, and the database/sql and go/types packages.
See the milestone on the issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 58f0bfcf51)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 534c774fab
Component: cli
With a docker build cache already primed with the build image I am seeing
`time make build -f docker.Makefile DOCKER_BUILDKIT=1 GO_BUILD_CACHE=n` takes
more than 1 minute.
By contrast `time make build -f docker.Makefile DOCKER_BUILDKIT=1
GO_BUILD_CACHE=y` takes less than 10s with a hot cache irrespective of whether
the source tree has changed
Signed-off-by: Ian Campbell <ijc@docker.com>
(cherry picked from commit d5de8358f0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c1c4b46f29
Component: cli
When building the Dockerfiles for development, those images are mainly used to
create a reproducible build-environment. The source code is bind-mounted into
the image at runtime; there is no need to create an image with the actual
source code, and copying the source code into the image would lead to a new
image being created for each code-change (possibly leading up to many "dangling"
images for previous code-changes).
However, when building (and using) the development images in CI, bind-mounting
is not an option, because the daemon is running remotely.
To make this work, the circle-ci script patched the Dockerfiles when CI is run;
adding a `COPY` to the respective Dockerfiles.
Patching Dockerfiles is not really a "best practice" and, even though the source
code does not and up in the image, the source would still be _sent_ to the daemon
for each build (unless BuildKit is used).
This patch updates the makefiles, circle-ci script, and Dockerfiles;
- When building the Dockerfiles locally, pipe the Dockerfile through stdin.
Doing so, prevents the build-context from being sent to the daemon. This speeds
up the build, and doesn't fill up the Docker "temp" directory with content that's
not used
- Now that no content is sent, add the COPY instructions to the Dockerfiles, and
remove the code in the circle-ci script to "live patch" the Dockerfiles.
Before this patch is applied (with cache):
```
$ time make -f docker.Makefile build_shell_validate_image
docker build -t docker-cli-shell-validate -f ./dockerfiles/Dockerfile.shellcheck .
Sending build context to Docker daemon 41MB
Step 1/2 : FROM debian:stretch-slim
...
Successfully built 81e14e8ad856
Successfully tagged docker-cli-shell-validate:latest
2.75 real 0.45 user 0.56 sys
```
After this patch is applied (with cache)::
```
$ time make -f docker.Makefile build_shell_validate_image
cat ./dockerfiles/Dockerfile.shellcheck | docker build -t docker-cli-shell-validate -
Sending build context to Docker daemon 2.048kB
Step 1/2 : FROM debian:stretch-slim
...
Successfully built 81e14e8ad856
Successfully tagged docker-cli-shell-validate:latest
0.33 real 0.07 user 0.08 sys
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 166856ab1b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 206ea57da8
Component: cli
gofmt/goimports changed some heuristics in 1.11 and the code is now
formatted slightly differently.
No functional change, just whitespace.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 906c2d161a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: edbc0e0613
Component: cli
In case go build will see a need to call C++ (rather than C)
compiler, CXX env var need to be properly set (to osxcross wrapper).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit ee461303f9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f9a42a4024
Component: cli
Fix this warning from go-1.11
> cli/registry/client/fetcher.go:234: Debugf format %s has arg
> repoEndpoint of wrong type client.repositoryEndpoint
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 51848bf3bb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a13ec91543
Component: cli
Currently the API spec would allow `"443/tcp": [null]`, but what should
be allowed is `"443/tcp": null`
Signed-off-by: Dominic Tubach <dominic.tubach@to.com>
(cherry picked from commit 32b5d296ea5d392c28affe2854b9d4201166bc27)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 3ee1e060fc9a31844064035cc3e8b76fddb8b341
Component: engine
This test runs on a daemon also used by other tests
so make sure we don't get failures if another test
doesn't cleanup or is running in parallel.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 915acffdb4cf95b934dd9872c1f54ea4487819b7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 05599804157a38db2eda84d3e8060a879b755ab7
Component: engine
Running a bundled aufs benchmark sometimes results in this warning:
> WARN[0001] Couldn't run auplink before unmount /tmp/aufs-tests/aufs/mnt/XXXXX error="exit status 22" storage-driver=aufs
If we take a look at what aulink utility produces on stderr, we'll see:
> auplink:proc_mnt.c:96: /tmp/aufs-tests/aufs/mnt/XXXXX: Invalid argument
and auplink exits with exit code of 22 (EINVAL).
Looking into auplink source code, what happens is it tries to find a
record in /proc/self/mounts corresponding to the mount point (by using
setmntent()/getmntent_r() glibc functions), and it fails.
Some manual testing, as well as runtime testing with lots of printf
added on mount/unmount, as well as calls to check the superblock fs
magic on mount point (as in graphdriver.Mounted(graphdriver.FsMagicAufs, target)
confirmed that this record is in fact there, but sometimes auplink
can't find it. I was also able to reproduce the same error (inability
to find a mount in /proc/self/mounts that should definitely be there)
using a small C program, mocking what `auplink` does:
```c
#include <stdio.h>
#include <err.h>
#include <mntent.h>
#include <string.h>
#include <stdlib.h>
int main(int argc, char **argv)
{
FILE *fp;
struct mntent m, *p;
char a[4096];
char buf[4096 + 1024];
int found =0, lines = 0;
if (argc != 2) {
fprintf(stderr, "Usage: %s <mountpoint>\n", argv[0]);
exit(1);
}
fp = setmntent("/proc/self/mounts", "r");
if (!fp) {
err(1, "setmntent");
}
setvbuf(fp, a, _IOLBF, sizeof(a));
while ((p = getmntent_r(fp, &m, buf, sizeof(buf)))) {
lines++;
if (!strcmp(p->mnt_dir, argv[1])) {
found++;
}
}
printf("found %d entries for %s (%d lines seen)\n", found, argv[1], lines);
return !found;
}
```
I have also wrote a few other C proggies -- one that reads
/proc/self/mounts directly, one that reads /proc/self/mountinfo instead.
They are also prone to the same occasional error.
It is not perfectly clear why this happens, but so far my best theory
is when a lot of mounts/unmounts happen in parallel with reading
contents of /proc/self/mounts, sometimes the kernel fails to provide
continuity (i.e. it skips some part of file or mixes it up in some
other way). In other words, this is a kernel bug (which is probably
hard to fix unless some other interface to get a mount entry is added).
Now, there is no real fix, and a workaround I was able to come up
with is to retry when we got EINVAL. It usually works on the second
attempt, although I've once seen it took two attempts to go through.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit ae431b10a9508e2bf3b1782e9d435855e3e13977)
Upstream-commit: c303e63ca6c3d25d29b8992451898734fa6e4e7c
Component: engine
Do not use filepath.Walk() as there's no requirement to recursively
go into every directory under mnt -- a (non-recursive) list of
directories in mnt is sufficient.
With filepath.Walk(), in case some container will fail to unmount,
it'll go through the whole container filesystem which is both
excessive and useless.
This is similar to commit f1a459229724f5e8e440b49f ("devmapper.shutdown:
optimize")
While at it, raise the priority of "unmount error" message from debug
to a warning. Note we don't have to explicitly add `m` as unmount error (from
pkg/mount) will have it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 8fda12c6078ed5c86be0822a7a980c6fbc9220bf)
Upstream-commit: b85d4a4f09badf40201ef7fcbd5b930de216190d
Component: engine
In case there are a big number of layers, so that mount data won't fit
into a single memory page (4096 bytes on most platforms, which is good
enough for about 40 layers, depending on how long graphdriver root path
is), we supply additional layers with O_REMOUNT, as described in aufs
documentation.
Problem is, the current implementation does that one layer at a time
(i.e. there is one mount syscall per each additional layer).
Optimize the code to supply as many layers as we can fit in one page
(basically reusing the same code as for the original mount).
Note, per aufs docs, "[a]t remount-time, the options are interpreted
in the given order, e.g. left to right" so we should be good.
Tested on an image with ~100 layers.
Before (35 syscalls):
> [pid 22756] 1556919088.686955 mount("none", "/mnt/volume_sfo2_09/docker-aufs/aufs/mnt/a86f8c9dd0ec2486293119c20b0ec026e19bbc4d51332c554f7cf05d777c9866", "aufs", 0, "br:/mnt/volume_sfo2_09/docker-au"...) = 0 <0.000504>
> [pid 22756] 1556919088.687643 mount("none", "/mnt/volume_sfo2_09/docker-aufs/aufs/mnt/a86f8c9dd0ec2486293119c20b0ec026e19bbc4d51332c554f7cf05d777c9866", 0xc000c451b0, MS_REMOUNT, "append:/mnt/volume_sfo2_09/docke"...) = 0 <0.000105>
> [pid 22756] 1556919088.687851 mount("none", "/mnt/volume_sfo2_09/docker-aufs/aufs/mnt/a86f8c9dd0ec2486293119c20b0ec026e19bbc4d51332c554f7cf05d777c9866", 0xc000c451ba, MS_REMOUNT, "append:/mnt/volume_sfo2_09/docke"...) = 0 <0.000098>
> ..... (~30 lines skipped for clarity)
> [pid 22756] 1556919088.696182 mount("none", "/mnt/volume_sfo2_09/docker-aufs/aufs/mnt/a86f8c9dd0ec2486293119c20b0ec026e19bbc4d51332c554f7cf05d777c9866", 0xc000c45310, MS_REMOUNT, "append:/mnt/volume_sfo2_09/docke"...) = 0 <0.000266>
After (2 syscalls):
> [pid 24352] 1556919361.799889 mount("none", "/mnt/volume_sfo2_09/docker-aufs/aufs/mnt/8e7ba189e347a834e99eea4ed568f95b86cec809c227516afdc7c70286ff9a20", "aufs", 0, "br:/mnt/volume_sfo2_09/docker-au"...) = 0 <0.001717>
> [pid 24352] 1556919361.801761 mount("none", "/mnt/volume_sfo2_09/docker-aufs/aufs/mnt/8e7ba189e347a834e99eea4ed568f95b86cec809c227516afdc7c70286ff9a20", 0xc000dbecb0, MS_REMOUNT, "append:/mnt/volume_sfo2_09/docke"...) = 0 <0.001358>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit d58c434bffef76e48bff75ede290937874488dd3)
Upstream-commit: 75488521735325e4564e66d9cf9c2b1bc1c2c64b
Component: engine
Apparently there is some kind of race in aufs kernel module code,
which leads to the errors like:
[98221.158606] aufs au_xino_create2:186:dockerd[25801]: aufs.xino create err -17
[98221.162128] aufs au_xino_set:1229:dockerd[25801]: I/O Error, failed creating xino(-17).
[98362.239085] aufs au_xino_create2:186:dockerd[6348]: aufs.xino create err -17
[98362.243860] aufs au_xino_set:1229:dockerd[6348]: I/O Error, failed creating xino(-17).
[98373.775380] aufs au_xino_create:767:dockerd[27435]: open /dev/shm/aufs.xino(-17)
[98389.015640] aufs au_xino_create2:186:dockerd[26753]: aufs.xino create err -17
[98389.018776] aufs au_xino_set:1229:dockerd[26753]: I/O Error, failed creating xino(-17).
[98424.117584] aufs au_xino_create:767:dockerd[27105]: open /dev/shm/aufs.xino(-17)
So, we have to have a lock around mount syscall.
While at it, don't call the whole Unmount() on an error path, as
it leads to bogus error from auplink flush.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 5cd62852fa199f272b542d828c8c5d1db427ea53)
Upstream-commit: bb4b9fe29eba57ed71d8105ec0e5dacc8e72129e
Component: engine
1. Use mount.Unmount() which ignores EINVAL ("not mounted") error,
and provides better error diagnostics (so we don't have to explicitly
add target to error messages).
2. Since we're ignoring "not mounted" error, we can call
multiple unmounts without any locking -- but since "auplink flush"
is still involved and can produce an error in logs, let's keep
the check for fs being mounted (it's just a statfs so should be fast).
2. While at it, improve the "can't unmount" error message in Put().
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 4beee98026feabe4f4f0468215b8fd9b56f90d5e)
Upstream-commit: 5b68d00abc4cad8b707a9a01dde9a420ca6eb995
Component: engine
Both mount and unmount calls are already protected by fine-grained
(per id) locks in Get()/Put() introduced in commit fc1cf1911bb
("Add more locking to storage drivers"), so there's no point in
having a global lock in mount/unmount.
The only place from which unmount is called without any locking
is Cleanup() -- this is to be addressed in the next patch.
This reverts commit 824c24e6802ad3ed7e26b4f16e5ae81869b98185.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit f93750b2c4d5f6144f0790ffa89291da3c097b80)
Upstream-commit: 701939112efc71a0c867a626f0a56df9c56030db
Component: engine
The function is not needed as it's just a shallow wrapper around
unix.Mount().
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 2f98b5f51fb0a21e1bd930c0e660c4a7c4918aa5)
Upstream-commit: 023b63a0f276b79277c1c961d8c98d4d5b39525d
Component: engine
The errors returned from Mount and Unmount functions are raw
syscall.Errno errors (like EPERM or EINVAL), which provides
no context about what has happened and why.
Similar to os.PathError type, introduce mount.Error type
with some context. The error messages will now look like this:
> mount /tmp/mount-tests/source:/tmp/mount-tests/target, flags: 0x1001: operation not permitted
or
> mount tmpfs:/tmp/mount-test-source-516297835: operation not permitted
Before this patch, it was just
> operation not permitted
[v2: add Cause()]
[v3: rename MountError to Error, document Cause()]
[v4: fixes; audited all users]
[v5: make Error type private; changes after @cpuguy83 reviews]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 65331369617e89ce54cc9be080dba70f3a883d1c)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 7f1c6bf5a745c8faeba695d3556dff4c4ff5f473
Component: engine
Moby currently sorts uid and gid ranges in id maps. This causes subuid
and subgid files to be interpreted wrongly.
The subuid file
```
> cat /etc/subuid
jonas:100000:1000
jonas:1000:1
```
configures that the container uids 0-999 are mapped to the host uids
100000-100999 and uid 1000 in the container is mapped to uid 1000 on the
host. The expected uid_map is:
```
> docker run ubuntu cat /proc/self/uid_map
0 100000 1000
1000 1000 1
```
Moby currently sorts the ranges by the first id in the range. Therefore
with the subuid file above the uid 0 in the container is mapped to uid
100000 on host and the uids 1-1000 in container are mapped to the uids
1-1000 on the host. The resulting uid_map is:
```
> docker run ubuntu cat /proc/self/uid_map
0 1000 1
1 100000 1000
```
The ordering was implemented to work around a limitation in Linux 3.8.
This is fixed since Linux 3.9 as stated on the user namespaces manpage
[1]:
> In the initial implementation (Linux 3.8), this requirement was
> satisfied by a simplistic implementation that imposed the further
> requirement that the values in both field 1 and field 2 of successive
> lines must be in ascending numerical order, which prevented some
> otherwise valid maps from being created. Linux 3.9 and later fix this
> limitation, allowing any valid set of nonoverlapping maps.
This fix changes the interpretation of subuid and subgid files which do
not have the ids of in the numerical order for each individual user.
This breaks users that rely on the current behaviour.
The desired mapping above - map low user ids in the container to high
user ids on the host and some higher user ids in the container to lower
user on host - can unfortunately not archived with the current
behaviour.
[1] http://man7.org/linux/man-pages/man7/user_namespaces.7.html
Signed-off-by: Jonas Dohse <jonas@dohse.ch>
(cherry picked from commit c4628d79d26c47bfbac9a3b22d684ee5fd78973c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 601f11300ed9995ae1d9117a52fd38b3e86563d8
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
(cherry picked from commit 293c74ba79f0008f48073985507b34af59b45fa6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 91f5be57af815df372371e1e989d00963ce4d02f
Component: engine
It has been pointed out that we're ignoring EINVAL from umount(2)
everywhere, so let's move it to a lower-level function. Also, its
implementation should be the same for any UNIX incarnation, so
let's consolidate it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 90be078fe59a8cfeff2bcc5dc2f34a00309837b6)
Upstream-commit: 47c51447e1b6dacf92b40574f6f929958ca9d621
Component: engine
As standard mount.Unmount does what we need, let's use it.
In addition, this adds ignoring "not mounted" condition, which
was previously implemented (see PR#33329, commit cfa2591d3f26)
via a very expensive call to mount.Mounted().
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 77bc327e24a60791fe7e87980faf704cf7273cf9)
Upstream-commit: 893b24b80db170279d5c9532ed508a81c328de5e
Component: engine
[18.09 backport] Pass root to chroot to for chroot Tar/Untar (CVE-2018-15664)
Upstream-commit: 72797601723f6a8847027a1abbd1a3cea2667718
Component: engine
Increases the max recieved gRPC message size for Node and Secret list
operations. This has already been done for the other swarm types, but
was not done for these.
Signed-off-by: Drew Erny <drew.erny@docker.com>
(cherry picked from commit a0903e1fa3eca32065c7dbfda8d1e0879cbfbb8f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a987f31fbc42526bf2cccf1381066f075b18dfe0
Component: engine
Previously only unpack operations were supported with chroot.
This adds chroot support for packing operations.
This prevents potential breakouts when copying data from a container.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 3029e765e241ea2b5249868705dbf9095bc4d529)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 61e0459053c359e322b8d5c017e855f616fd34c0
Component: engine
This is useful for preventing CVE-2018-15664 where a malicious container
process can take advantage of a race on symlink resolution/sanitization.
Before this change chrootarchive would chroot to the destination
directory which is attacker controlled. With this patch we always chroot
to the container's root which is not attacker controlled.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit d089b639372a8f9301747ea56eaf0a42df24016a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 155939994f453559676656bc4b05635e83ebef56
Component: engine
[18.09] Backport Forcing a nil IP specified in PortBindings to IPv4 zero (0.0.0.0).
Upstream-commit: c882c0011f0857da5e8cef1ea48e0aa80019abf1
Component: engine
As pointed out by Tonis, there's a race between ReleaseRWLayer()
and GetRWLayer():
```
----- goroutine 1 ----- ----- goroutine 2 -----
ReleaseRWLayer()
m := ls.mounts[l.Name()]
...
m.deleteReference(l)
m.hasReferences()
... GetRWLayer()
... mount := ls.mounts[id]
ls.driver.Remove(m.mountID)
ls.store.RemoveMount(m.name) return mount.getReference()
delete(ls.mounts, m.Name())
----------------------- -----------------------
```
When something like this happens, GetRWLayer will return
an RWLayer without a storage. Oops.
There might be more races like this, and it seems the best
solution is to lock by layer id/name by using pkg/locker.
With this in place, name collision could not happen, so remove
the part of previous commit that protected against it in
CreateRWLayer (temporary nil assigmment and associated rollback).
So, now we have
* layerStore.mountL sync.Mutex to protect layerStore.mount map[]
(against concurrent access);
* mountedLayer's embedded `sync.Mutex` to protect its references map[];
* layerStore.layerL (which I haven't touched);
* per-id locker, to avoid name conflicts and concurrent operations
on the same rw layer.
The whole rig seems to look more readable now (mutexes use is
straightforward, no nested locks).
Reported-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit af433dd200f8287305b1531d5058780be36b7e2e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1ebe324c6a481de86a5b84494057a639773624f1
Component: engine
This is an additon to commit 1fea38856a ("Remove v1.10 migrator")
aka PR #38265. Since that one, CreateRWLayerByGraphID() is not
used anywhere, so let's drop it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit b4e9b507655e7dbdfb44d4ee284dcd658b859b3f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1576eaba3357b7aab5121b51b9ebc34e731082fe
Component: engine
Goroutine stack analisys shown some lock contention
while doing massively (100 instances of `docker rm`)
parallel image removal, with many goroutines waiting
for the mountL mutex. Optimize it.
With this commit, the above operation is about 3x
faster, with no noticeable change to container
creation times (tested on aufs and overlay2).
kolyshkin@:
- squashed commits
- added description
- protected CreateRWLayer against name collisions by
temporary assiging nil to ls.mounts[name], and treating
nil as "non-existent" in all the other functions.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 05250a4f0094e6802dd7d338d632ea632d0c7e34)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 78cbf4d1388a4eef155f08da0d2422e3a2cad11f
Component: engine
Add a mutex to protect concurrent access to mountedLayer.references map.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit f73b5cb4e8b9a23ad6700577840582d66017a733)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: bb80a60be2a9236b0db27222310f568e1d3fba11
Component: engine
As of Go 1.12, the `testing` package panics if a goroutine logs to a
`testing.T` after the relevant test has completed. This was not
documented as a change at all; see the commit
95d06ab6c982f58b127b14a52c3325acf0bd3926 in the Go repository for the
relevant change.
At any point in the integration tests, tests could panic with the
message "Log in goroutine after TEST_FUNCTION has completed". This was
exacerbated by less direct logging I/O, e.g. running `make test` with
its output piped instead of attached to a TTY.
The most common cause of panics was that there was a race condition
between an exit logging goroutine and the `StopWithError` method:
`StopWithError` could return, causing the calling test method to return,
causing the `testing.T` to be marked as finished, before the goroutine
could log that the test daemon had exited. The fix is simple: capture
the result of `cmd.Wait()`, _then_ log, _then_ send the captured
result over the `Wait` channel. This ensures that the message is
logged before `StopWithError` can return, blocking the test method
so that the target `testing.T` is not marked as finished.
Signed-off-by: Daniel Sweet <danieljsweet@icloud.com>
(cherry picked from commit 7546322e994c4a23ea3cae0cf0a2a8019de12c03)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b2168eec8b37619a1d28d0bb44cf052d4f9883c7
Component: engine
Checks for environment variable VERSION if it exists then it sets dockerVersion to VERSION
Signed-off-by: corbin-coleman <corbin.coleman@docker.com>
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit edc639e99f21d0dd814581858d02787c23ba1599)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 238951ac3a78a8517d7764de0f48b8ded28441e2
Component: engine
append(newRoot.DiffIDs) without element does nothing,
so it's probably not what was intended. Changed code
to perform a slice copying instead.
Fixes#38834.
Signed-off-by: Iskander Sharipov <quasilyte@gmail.com>
(cherry picked from commit 3429e9993085520ac902fef2ef6aabd57080bd36)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1c755a73bfd63f63d6fe3363d8237f9db634633d
Component: engine
This test starts a new daemon, which will fail when testing
against a remote daemon;
--- FAIL: TestInfoAPIWarnings (0.00s)
info_test.go:53: failed to start daemon with arguments [-H=0.0.0.0:23756 -H=unix:///tmp/docker-integration/d5153ebcf89ef.sock] : [d5153ebcf89ef] could not find docker binary in $PATH: exec: "dockerd": executable file not found in $PATH
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 056840c2a644cc37f91392103f5c3ae05d14a695)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5e024a2fcd10585611ab1ea792d0256c6477c87c
Component: engine
This test is dependent on the search results returned by Docker Hub, which
can change at any moment, and causes this test to be unpredictable.
Removing this test instead of trying to catch up with Docker Hub any time
the results change, because it's effectively testing Docker Hub, and not
the daemon.
Unit tests are already in place to test the core functionality of the daemon,
so it should be safe to remove this test.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 21e662c77468d1e3fc4efaa049021e2ad0e49b4d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0d052bc40e908f1f596b0f16840b4e93fc6a9ec2
Component: engine
Errors were being ignored and always telling the user that the path
doesn't exist even if it was some other problem, such as a permission
error.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit ebcef288343698dd86ff307f5b9c58aa52ce9fdd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0ac09de395d61d8cecdd6496aa3fb9ceb174f295
Component: engine
`chmod` is a legacy syscall, and not present on arm64, which
caused this test to fail.
Add `fchmodat` to the profile so that this test can run both
on x64 and arm64.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4bd8964b23f616a1aef63ea0acebbdc4b6ea90e6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1b642b8c45a734d4c2d9af2f6849289db556d003
Component: engine
Sometimes this test fails (allegedly due to problems with Docker Hub),
but it fails later than it should, for example:
> 01:20:34.845 assertion failed: expression is false: strings.Count(outSearchCmdStars, "[OK]") <= strings.Count(outSearchCmd, "[OK]"): The quantity of images with stars should be less than that of all images: <...>
This, with non-empty list of images following, means that the initial
`docker search busybox` command returned not enough results. So, add
a check that `docker search busybox` returns something.
While at it,
* raise the number of stars to 10;
* simplify check for number of lines (no need to count [OK]'s);
* improve error message.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 4f80a1953d66265d9a1cbf4e172b8fa58e3e403a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8a369c735b2bf07da53275fe9fe61d429320f233
Component: engine
Today `$ docker service create --limit-cpu` configures a containers
`CpuPeriod` and `CpuQuota` variables, this commit switches this to
configure a containers `NanoCpu` variable instead.
Signed-off-by: Olly Pomeroy <olly@docker.com>
(cherry picked from commit 8a60a1e14a5e98b7ed0fbea57369615a766a1ae9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 675f4256ae2b85556b4243da44ff627900660463
Component: engine
GCR does not currently support 401 response from blob endpoints.
This detects the case where no manifest requests have been
performed for the current resolver and does a dummy request
to enable authorization.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit bcd8298c35f53f79b42c6e089e8da114ddb5c57e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 21cce932b05a4b979359624668f7f09e83dc90cd
Component: engine
full diff: c9029898e3...872f0a83c9
- docker/libnetwork#2354 [18.09 backport] Cleanup the cluster provider when the agent is closed
- backport of docker/libnetwork#2307 Fix for problem where agent is stopped and does not restart
- fixesdocker/for-linux#495 Docker swarm overlay networking not working after --force-new-cluster
- docker/libnetwork#2369 [18.09 BACKPORT] Pick a random host port if the user does not specify a host port
- backport of docker/libnetwork#2368 (windows) Pick a random host port if the user does not specify a host port
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5354408039681020f9ad6afe4bf696fc90f9ce69
Component: engine
[18.09 backport] test-fixes and updates, and fix images filter when use multi reference filter
Upstream-commit: 82a2b8882e5a9dc2ba4389f589ad28dc9cee0d57
Component: engine
In case of failure, stale out was printed.
Fixes: 6212ea669b4e92b3
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 1921753b4b30dcca4fe772e7c1b0bc3f7bb7cd62)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b60ecd32a1d3ba7debf02d15fe346ea5ca607bbd
Component: engine
Move the test case from integration-cli to integration.
The test logic itself has not changed, except these
two things:
* the new test sets default-ipc-mode via command line
rather than via daemon.json (less code);
* the new test uses current API version.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 9fd765f07cc08ccc2ea991d21835bf50ece9318b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b228a498d55fcd47c741d51a510a7f5eab0794f7
Component: engine
NOTE TestUpdateRestartPolicy is left as is as otherwise
it will decrease its readability.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit f664df01d1836686c7a8a917e9da81d56c758d74)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2c546aace40872169b03e18f07294d3fbfa3d452
Component: engine
Since container.Create() already initializes HostConfig
to be non-nil, there is no need for this code. Remove it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 17022b3ad2d95863e5acd17ebaf943b7761623cc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ed48a92a580b7ccc5c5837c7d802f787000146eb
Component: engine
This test sometimes failed because the number of events received did not
match the expected number:
FAIL: docker_cli_events_test.go:316: DockerSuite.TestEventsFilterLabels
docker_cli_events_test.go:334:
c.Assert(len(events), checker.Equals, 3)
... obtained int = 2
... expected int = 3
This patch makes the test more stable, by:
- use a wider range between `--since` and `--until`. These options were set
so that the client detaches after events were received, but the actual
range should not matter. Changing the range will cause more events to be
returned, but we're specifically looking for the container ID's, so this
should not make a difference for the actual test.
- use `docker create` instead of `docker run` for the containers. the
containers don't have to be running to trigger an event; using `create`
speeds up the test.
- check the exit code of the `docker create` to verify the containers were
succesfully created.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0e15c02465c87c82908bcc45b0c1d6bd38f27c32)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 26f7e0a8b08b2890d079d4083735c05377770ca8
Component: engine
I noticed that this test failed, because the node was in status "pending".
The test checks for the node's status immediately after it was restarted, so
possibly it needs some time to unlock.
14:07:10 FAIL: docker_cli_swarm_test.go:1128: DockerSwarmSuite.TestSwarmLockUnlockCluster
...
14:07:10 docker_cli_swarm_test.go:1168:
14:07:10 checkSwarmLockedToUnlocked(c, d)
14:07:10 docker_cli_swarm_test.go:1017:
14:07:10 c.Assert(getNodeStatus(c, d), checker.Equals, swarm.LocalNodeStateActive)
14:07:10 ... obtained swarm.LocalNodeState = "pending"
14:07:10 ... expected swarm.LocalNodeState = "active"
This patch adds a `waitAndAssert` for the node's status, with a 1 second timeout.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 973ca00d60712ef644b5b37abf7fa01078bb4ade)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 954425f8bdb5db95355cb20be465d248f7c3b1f1
Component: engine
When starting docker daemons for swarm testing, we disable iptables
and use lo for communication (in order to avoid network conflicts).
The problem is, these options are lost on restart, that can lead
to any sorts of network conflicts and thus connectivity issues
between swarm nodes.
Fix this. This does not fix issues with swarm test failures, but
it seems they appear are less often after this one.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 2ed512c7faea938b0b07e69187b8a132e2ecb66a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 553b09684cb68962403ebdc5495fb548364f778f
Component: engine
This is repeated 6 times in different tests, with slight
minor variations. Let's factor it out, for clarity.
While at it, simplify the code: instead of more complex
parsing of "docker swarm init|update --autolock" output (1)
and checking if the key is also present in
"docker swarm unlock-key" output (2), get the key
from (2) and check it is present in (1).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 24cbb9897193894f4716583d1861091ab2fa1ae2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 6ec991ec8371bdba94480c587eea6ea24f3e9d43
Component: engine
A timer is leaking on every daemon start and stop.
Probably nothing major, but given the amount of
daemon starts/stops during tests, it's better to
be accurate about it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 6016520162fdcb19f50d08c4f0b54b06a7a6eac0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 784a89354b118ef68f1f20fb4ff383c6282ea03c
Component: engine
Since commit 17173efbe00 checkSwarmLockedToUnlocked() no longer
require its third argument, so remove it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 66cb1222d6559e120d9d1a29932aa778aa517894)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2a02a2e9dae0045d550e98fa849aceafc0c41a0e
Component: engine
1. Using MNT_FORCE flag does not make sense for nsfs. Using MNT_DETACH
though might help.
2. When -check.vv is added to TESTFLAGS, there are a lot of messages
like this one:
> unmount of /tmp/dxr/d847fd103a4ba/netns failed: invalid argument
and some like
> unmount of /tmp/dxr/dd245af642d94/netns failed: no such file or directory
The first one means directory is not a mount point, the second one
means it's gone. Do ignore both of these.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 73baee2dcf546b2561bdd9a500b0af08cb62b1be)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 90cb9b6545fbf7f91cf3eb427f627a7353ea684f
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
(cherry picked from commit b1b9937bc75f0db9c804838ecce9bb6792a42525)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 533b0f602d10af8fe325c3544a2411f5060a47b8
Component: engine
It looks like the logic of the test became wrong after commit
ae0883c ("Move TestAttachDetach to integration-cli").
The original logic was:
* (a few first steps skipped for clarity)
* send escape sequence to "attach";
* check "attach" is exiting (i.e. escape sequence works);
* check the container is still alive;
* kill the container.
Also, timeouts were big at that time, in the order of seconds.
The logic after the above mentioned commit and until now is:
* ...
* send escape sequence to "attach";
* check the container is running (why shouldn't it?);
* kill the container;
* checks that the "attach" has exited.
So, from the "let's check detach using escape sequence is working"
the test became something like "let's check that attach is gone
once we kill the container".
Let's fix the above test, also increasing the timeout waiting
for attach to exit (which fails from time to time on power CI).
Now, the second test, TestAttachDetachTruncatedID, does the exact
same thing, except it uses a truncated container ID. It does not
seem to be of much value, so let's remove it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 9f3a343a5101ab661a6a97c9e149a0b11ccc320a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 17ae129697c10e9f866ffc253bc211f8912588df
Component: engine
This introduces `NoTasksForService` and `NoTasks` poller checks, that
can be used to check if no tasks are left in general, or for a specific
service.
Some redundant checks were also removed from some tests.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 56a68c15f8a093b1761e77a74d8b7acdfbcb30a2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8f6421457d3be6280236aa466e533fb32cb4509b
Component: engine
Clean up and refactor this test;
- make `serviceRunningTasksCount` to use a `desired-state` filter
- use subtests, and inline the `validNetworkVerbose` checks; also use
asserts for the individual checks, so that any failure will log exactly
what failed
- remove helper functions that are no longer needed
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 60d93aab2e68b13b3f22c43add4762d3e8108227)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 046352162f1a23f10a05b59e4fac9d932bcd4f35
Component: engine
This fix removes code duplication and consolidates networkIsRemoved
into one place.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
(cherry picked from commit 28b7824caa9c8f1acc0471136a8a4fd80e51f491)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: dd20556e037b262473f3f8163dfc7c21e9161c8e
Component: engine
These tests are run on a local Linux daemon only, so no need
to do a platform-check.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 69c0b7e47682a2a7a850122a9a2f711259fbb25a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: bb6db57acc8a63f7f85fafe4be0cfe7fa929e3a1
Component: engine
These tests can only be run on a local Linux daemon, so there's
no need to build them on Windows
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 84224935ea78d93e21a21c0e1e0aa3e83a5c7853)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b4c0a7efd46d5fb13df7214aface1f291ba356d2
Component: engine
A client is already created in testenv.New(), so we can just
as well use that one, instead of creating a new client.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0de62d9bbcb92e9b7c73ee4cdef51c2229878e05)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e438d4799d6e233fe631d332973dcac1d977fef6
Component: engine
`testEnv` is a package-level variable, so protecting / restoring
`testEnv` in parallel will result in "concurrent map write" errors.
This patch removes `t.Parallel()` from tests that use this
functionality (through `defer setupTest(t)()`).
Note that _subtests_ can still be run in parallel, as the defer
will be called after all subtests have completed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4d88a95d6730383624570f8730aa203a56caadc3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 05ecb140c4f3c4c7cbb860baa425104fa6f132ea
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
(cherry picked from commit bc80e8df3e6300354305f81a4c0c48b855a1776b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: d5a0ffa172a012e388d5b31edf332acc264b39b6
Component: engine
I noticed this error in CI:
```
20:37:25 INFO: Non-base image count on control daemon to delete is 9
20:37:25 "docker rmi" requires at least 1 argument.
20:37:25 See 'docker rmi --help'.
20:37:25
20:37:25 Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
20:37:25
20:37:25 Remove one or more images
```
Which indicated that the PowerShell script managed to find images to delete, but
not actually passing the images to `docker rmi`.
The reason for this failing was that the script attempted to convert the
collection/array to a string, which produces;
```powershell
Write-Output $(docker images --format "{{.Repository}}:{{.ID}}" | `
>> select-string -NotMatch "windowsservercore" | `
>> select-string -NotMatch "nanoserver" | `
>> select-string -NotMatch "docker" `
>> ).ToString()
System.Object[]
```
Which, when trying to split by the chosen separator (`:`), will return the same;
```powershell
Write-Output "System.Object[]".Split(":")[0]
```
This patch:
- Adds an intermediate variable (`$allImages`) to make the code better readable
- Switches the separator to `#`, to prevent breaking on images pulled from a
repository with a port in its name (`myregistry:5000/my/image`)
- Switches to use a comma-separated list for `-NotMatch` (for readability)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5580b79813d3a5d2a7b83bcf1607839a5b0f468d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 74fe2c044f56e58356e9f598e5b08d94e676bbd4
Component: engine
This patch replaces PowerShell aliases for their real commands, see https://blogs.technet.microsoft.com/heyscriptingguy/2012/04/21/when-you-should-use-powershell-aliases/
For example;
- use `Get-Location` instead of `pwd`
- use `Set-Location` instead of `cd`
- use `ForEach-Object` instead of the `%` shorthand
- use `Write-Output` instead of `echo`
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6130c89ccec5262c25b91b3226a6ef2240bfde3b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 4f0911d9122bf73d7efb90a50444cd6e52a09a96
Component: engine
Fixed some mixed/tabs spaces for indentation, and used
tabs for auto-generated Go code.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0f8b616c0c8f959ac4b29f29a8716b8cb7ea28fd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 73e5951b2e3c13ce84781d28b08cdc1958fb6224
Component: engine
Old selector was wrong (it watched for the label we applied to child
resources when reconciling the stack, instead of the stack itself)
This should be back-ported to older version of the CLI
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
(cherry picked from commit 8cd74eb33a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f89d05edcb
Component: cli
Vendoring can take some time, depending on network-speed, so
reduce flakiness by increasing the default timeout, to prevent:
make[1]: Entering directory '/go/src/github.com/docker/cli'
rm -rf vendor
bash -c 'vndr |& grep -v -i clone'
2019/03/18 11:38:26 Collecting initial packages
Too long with no output (exceeded 10m0s)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit dba90e4999)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 90fa621791
Component: cli
This patch switches the shellcheck image to use the official image
from Docker Hub.
Note that this does not yet update shellcheck to the latest version (v0.5.x);
Shellcheck v0.4.7 added some new checks, which makes CI currently fail, so will
be done in a follow-up PR. Instead, the v0.4.6 version is used in this PR, which
is closest to the same version as was installed in the image before this change;
```
docker run --rm docker-cli-shell-validate shellcheck --version
ShellCheck - shell script analysis tool
version: 0.4.4
license: GNU General Public License, version 3
website: http://www.shellcheck.net
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 388646eab0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b59752479b
Component: cli
When building the Dockerfiles for development, those images are mainly used to
create a reproducible build-environment. The source code is bind-mounted into
the image at runtime; there is no need to create an image with the actual
source code, and copying the source code into the image would lead to a new
image being created for each code-change (possibly leading up to many "dangling"
images for previous code-changes).
However, when building (and using) the development images in CI, bind-mounting
is not an option, because the daemon is running remotely.
To make this work, the circle-ci script patched the Dockerfiles when CI is run;
adding a `COPY` to the respective Dockerfiles.
Patching Dockerfiles is not really a "best practice" and, even though the source
code does not and up in the image, the source would still be _sent_ to the daemon
for each build (unless BuildKit is used).
This patch updates the makefiles, circle-ci script, and Dockerfiles;
- When building the Dockerfiles locally, pipe the Dockerfile through stdin.
Doing so, prevents the build-context from being sent to the daemon. This speeds
up the build, and doesn't fill up the Docker "temp" directory with content that's
not used
- Now that no content is sent, add the COPY instructions to the Dockerfiles, and
remove the code in the circle-ci script to "live patch" the Dockerfiles.
Before this patch is applied (with cache):
```
$ time make -f docker.Makefile build_shell_validate_image
docker build -t docker-cli-shell-validate -f ./dockerfiles/Dockerfile.shellcheck .
Sending build context to Docker daemon 41MB
Step 1/2 : FROM debian:stretch-slim
...
Successfully built 81e14e8ad856
Successfully tagged docker-cli-shell-validate:latest
2.75 real 0.45 user 0.56 sys
```
After this patch is applied (with cache)::
```
$ time make -f docker.Makefile build_shell_validate_image
cat ./dockerfiles/Dockerfile.shellcheck | docker build -t docker-cli-shell-validate -
Sending build context to Docker daemon 2.048kB
Step 1/2 : FROM debian:stretch-slim
...
Successfully built 81e14e8ad856
Successfully tagged docker-cli-shell-validate:latest
0.33 real 0.07 user 0.08 sys
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 166856ab1b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8997667aa2
Component: cli
When manually stopping a container with a restart-policy, the container
would show as "restarting" in `docker ps` whereas its actual state
is "exited".
Stopping a container with a restart policy shows the container as "restarting"
docker run -d --name test --restart unless-stopped busybox false
docker stop test
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7e07409fa1d3 busybox "false" 5 minutes ago Restarting (1) 4 minutes ago test
However, inspecting the same container shows that it's exited:
docker inspect test --format '{{ json .State }}'
{
"Status": "exited",
"Running": false,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 0,
"ExitCode": 1,
"Error": "",
"StartedAt": "2019-02-14T13:26:27.6091648Z",
"FinishedAt": "2019-02-14T13:26:27.689427Z"
}
And killing the container confirms this;
docker kill test
Error response from daemon: Cannot kill container: test: Container 7e07409fa1d36dc8d8cb8f25cf12ee1168ad9040183b85fafa73ee2c1fcf9361 is not running
docker run -d --name test --restart unless-stopped busybox false
docker stop test
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d0595237054a busybox "false" 5 minutes ago Restarting (1) 4 minutes ago exit
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8c0ecb638705e89746a81fa1320aafaa7ff701b2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 00f0b9df0dde80b2c40023b12a29dd476f2a77e2
Component: engine
When running a container in the host's network namespace, the container
gets a copy of the host's resolv.conf (copied to `/etc/resolv.conf` inside
the container).
The current code always used the default (`/etc/resolv.conf`) path on the
host, irregardless if `systemd-resolved` was used or not.
This patch uses the correct file if `systemd-resolved` was detected
to be running.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8364d1c9d590d4266871cd820b76ef12e2b934ed)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 04ae160eca4a8bc134ca9abafa044191b893080b
Component: engine
The integration test TestExportContainerWithOutputAndImportImage in moby/moby is the same as TestExportContainerAndImportImage,
except for the output file option. Adding a unit test to cover the output file option of the export command here allows
the removal of the redundant integration test TestExportContainerWithOutputAndImportImage.
Signed-off-by: Arash Deshmeh <adeshmeh@ca.ibm.com>
(cherry picked from commit fc1e11d46a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 079adf3f23
Component: cli
Tagger was not called for BuildKit-mode.
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
(cherry picked from commit 7fc0f820ea1e9036a1466ee8ef7a7395b792623f)
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Upstream-commit: 8d87a2a4bc88b235f06e3995bb33f978d7c28cdf
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 9d2e97ac6e20b17477947fc63e70299938606a38)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 91703956dffcf1d9c997aea83f9489a8c768412d
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
(cherry picked from commit 80fce6d747c5208b42e94ac9e3f22cef28dd8afe)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: fd4670161de8f9947c524138c477177f935c98f2
Component: engine
There's little way of knowing what each exit status means at present
because it's not documented. I'm assuming they are the same as docker
run.
Signed-off-by: Eric Curtin <ericcurtin17@gmail.com>
(cherry picked from commit 23670968cc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 27ab7cc3d6
Component: cli
Noticed a typo in this markdown file: "instead" instead of "in stead"
Signed-off-by: Ryan Wilson-Perkin <ryanwilsonperkin@gmail.com>
(cherry picked from commit 7a9fc782c5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1d314f2227
Component: cli
Copies the improved description from the man page
to the online reference docs.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 89bc5fbbae)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 275ab1f063
Component: cli
The `docker image rm` command can be used not only
to remove images but also remove tags.
This update improves the documentation to make
this clear.
Signed-off-by: Filip Jareš <filipjares@gmail.com>
(cherry picked from commit 2ba9601ef1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0717f261ed
Component: cli
By default, exec uses the environment of the current process, however,
if `exec.Env` is not `nil`, the environment is discarded:
e73f489494/src/os/exec/exec.go (L57-L60)
> If Env is nil, the new process uses the current process's environment.
When adding a new environment variable, prepend the current environment,
to make sure it is not discarded.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6c4fbb7738)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: fc8717799f
Component: cli
Relevant changes:
- containerd/containerd#51 Fix empty device type
- containerd/containerd#52 Remove call to unitName
- Calling unitName incorrectly appends -slice onto the end of the slice cgroup we are looking for
- addresses containerd/containerd#47 cgroups: cgroup deleted
- containerd/containerd#53 systemd-239+ no longer allows delegate slice
- containerd/containerd#54 Bugfix: can't write to cpuset cgroup
- containerd/containerd#63 Makes Load function more lenient on subsystems' checking
- addresses containerd/containerd#58 Very strict checking of subsystems' existence while loading cgroup
- containerd/containerd#67 Add functionality for retrieving all tasks of a cgroup
- containerd/containerd#68 Fix net_prio typo
- containerd/containerd#69 Blkio weight/leafWeight pointer value
- containerd/containerd#77 Check for non-active/supported cgroups
- addresses containerd/containerd#76 unable to find * in controller set: unknown
- addresses docker/for-linux#545 Raspbian: Error response from daemon: unable to find "net_prio" in controller set: unknown
- addresses docker/for-linux#552 Error response from daemon: unable to find "cpuacct" in controller set: unknown
- addresses docker/for-linux#545 Raspbian: Error response from daemon: unable to find "net_prio" in controller set: unknown
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 386b06eacd689bb98fb4117087614466c6e130f3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a36f6bfc36d2425c693097102b91152fa8f80012
Component: engine
Notable Updates
- Fix an issue that non-existent parent directory in image layers is created with permission 0700. containerd#3017
- Fix an issue that snapshots of the base image can be deleted by mistake, when images built on top of it are deleted. containerd#3087
- Support for GC references to content from snapshot and container objects. containerd#3080
- cgroups updated to dbea6f2bd41658b84b00417ceefa416b97 to fix issues for systemd 420 and non-existent cgroups. containerd#3079
- runc updated to 2b18fe1d885ee5083ef9f0838fee39b62d653e30 to include the improved fix for CVE-2019-5736. containerd#3082
- cri: Fix a bug that pod can't get started when the same volume is defined differently in the image and the pod spec. cri#1059
- cri: Fix a bug that causes container start failure after in-place upgrade containerd to 1.2.4+ or 1.1.6+. cri#1082
- cri updated to a92c40017473cbe0239ce180125f12669757e44f. containerd#3084
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 25cdae293fedb18f3538788d629d6e4475acaa4b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 6c715008627a9ff2073e932d5c792dc7bc1f2f01
Component: engine
After switching to Go 1.12, the format-string causes an error;
```
=== Errors
cli/config/config_test.go:154:3: Fatalf format %q has arg config of wrong type *github.com/docker/cli/cli/config/configfile.ConfigFile
cli/config/config_test.go:217:3: Fatalf format %q has arg config of wrong type *github.com/docker/cli/cli/config/configfile.ConfigFile
cli/config/config_test.go:253:3: Fatalf format %q has arg config of wrong type *github.com/docker/cli/cli/config/configfile.ConfigFile
cli/config/config_test.go:288:3: Fatalf format %q has arg config of wrong type *github.com/docker/cli/cli/config/configfile.ConfigFile
cli/config/config_test.go:435:3: Fatalf format %q has arg config of wrong type *github.com/docker/cli/cli/config/configfile.ConfigFile
cli/config/config_test.go:448:3: Fatalf format %q has arg config of wrong type *github.com/docker/cli/cli/config/configfile.ConfigFile
DONE 1115 tests, 2 skipped, 6 errors in 215.984s
make: *** [Makefile:22: test-coverage] Error 2
Exited with code 2
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d4877fb225)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 7ea48a16e3
Component: cli
This was leaking the fd.
Signed-off-by: Ian Campbell <ijc@docker.com>
(cherry picked from commit 186e7456ac)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 75e9075591
Component: cli
This happens on Windows when dialing a named pipe (a path which is used by CLI
plugins), in that case some debugging shows:
DEBU[0000] conn is a *winio.win32MessageBytePipe
DEBU[0000] conn is a halfReadCloser: false
DEBU[0000] conn is a halfWriteCloser: true
the raw stream connection does not implement halfCloser
In such cases we can simply wrap with a nop function since closing for read
isn't too critical.
Signed-off-by: Ian Campbell <ijc@docker.com>
(cherry picked from commit 8919bbf04d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 69e1094f5a
Component: cli
... in preference to `chsh`, since in recent alpine 3.9.2 images that can fail
with:
Password: chsh: PAM: Authentication token manipulation error
Which seems to relate to the use of `!` as the password for `root` in `/etc/shadow`gq
Signed-off-by: Ian Campbell <ijc@docker.com>
(cherry picked from commit 5de2d9e8a9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b666e9a090
Component: cli
[18.09 backport] Fix TestBuildWithSession, TestBuildSquashParent using wrong daemon during test
Upstream-commit: c97602ef9903fd18e65b341a7359287e684cde84
Component: engine
These tests were spinning up a new daemon, but after the daemon was spun up,
the default test-daemon was used by the client.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9a3911ced83331b6efe698528a4480bd5ff3d7a6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 605bc1ffc5adc5a3968ebe7a58060d2542cc28f3
Component: engine
relevant changes:
- swarmkit#2826 [18.09 backport] use a custom grpc dialer when managers are joining (backport of swarmkit#2802)
- swarmkit#2801 [18.09 backport] Include old error-message for backward compatibility (backport of swarmkit#2797)
- swarmkit#2788 [18.09 backport] Return correct error-codes on conflicting names (backport of swarmkit#2779)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1a60fdbdcefbecf762e063fbb5b3593e53e877bb
Component: engine
As reported in docker/for-linux/issues/484, since Docker 18.06
docker cp with a destination file name fails with the following error:
> archive/tar: cannot encode header: Format specifies USTAR; and USTAR cannot encode Name="a_very_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_long_filename_that_is_101_characters"
The problem is caused by changes in Go 1.10 archive/tar, which
mis-guesses the tar stream format as USTAR (rather than PAX),
which, in turn, leads to inability to specify file names
longer than 100 characters.
This tar stream is sent by TarWithOptions() (which, since we switched to
Go 1.10, explicitly sets format=PAX for every file, see FileInfoHeader(),
and before Go 1.10 it was PAX by default). Unfortunately, the receiving
side, RebaseArchiveEntries(), which calls tar.Next(), mistakenly guesses
header format as USTAR, which leads to the above error.
The fix is easy: set the format to PAX in RebaseArchiveEntries()
where we read the tar stream and change the file name.
A unit test is added to prevent future regressions.
NOTE this code is not used by dockerd, but rather but docker cli
(also possibly other clients), so this needs to be re-vendored
to cli in order to take effect.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit f55a4176febbd0dffd6e5eb65beb70bc32912d0b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 989e7f5d3a3f40ebb936376245b770f766ea42e9
Component: engine
In applyTar, if the driver's ApplyDiff returns an error, the function
returns early without calling io.Copy.
As a consequence, the resources (a goroutine and some buffers holding
the uncompressed image, the digest, etc...) allocated or referenced by
NewInputTarStream above aren't released, as the worker goroutine only
finishes when it finds EOF or a closed pipe.
Signed-off-by: Sergio Lopez <slp@redhat.com>
(cherry picked from commit 5846db10af9fb37061ab92a07c3d82fbea92b2e0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f660ef2c25590f21fdd10357c08a542a4876d6c3
Component: engine
This commit ensures that we test the correct branches of Docker
when making packaging changes.
Signed-off-by: Dave Tucker <dt@docker.com>
(cherry picked from commit 1ef9fb3f0b2b8016b50c6c71f2d14bea16faff35)
Upstream-commit: 5e59ff8a5e8ea44444aa4f22a8baf12e091bcc76
Component: packaging
Signed-off-by: John Howard <jhoward@microsoft.com>
This is a follow-on from https://github.com/moby/moby/pull/38277
but had to be done in a couple of stages to ensure that CI didn't
break. v1.1 of the busybox image is now based on a CMD of "sh"
rather than using an entrypoint. And it also uses the bin directory
rather than `c:\busybox`. This makes it look a lot closer to the
Linux busybox image, and means that a couple of Windows-isms in
CI tests can be reverted back to be identical to their Linux
equivalents.
(cherry picked from commit 561e0f6b7fc256c160292b32695cf1d6150741db)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 613c2f27ed2e7d65474c2f3e786d9e24e757d99d
Component: engine
- Fixes a vulnerability in runc that allows a container escape (CVE-2019-5736)
6635b4f0c6,
- Includes security fix for `runc run --no-pivot` (`DOCKER_RAMDISK=1`):
28a697cce3
(NOTE: the vuln is attackable only when `DOCKER_RAMDISK=1` is set && seccomp is disabled)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f03698b69a7777b8d30b9c5897504f8704b87676)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c7fca75c035ba0b750f46a9676a376f8e4409f15
Component: engine
This reverts commit eb137ff1765faeb29c2d99025bfd8ed41836dd06, reversing
changes made to a79fabbfe84117696a19671f4aa88b82d0f64fc1.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 88330c9aac5556d0abc7a5afcb4d906604a07fa2
Component: engine
[18.09 backport] Graphdriver: fix "device" mode not being detected if "character-device" bit is set
Upstream-commit: 2e4c5c57c30522dc4b33b5cd5371f294ae3fee82
Component: engine
[18.09 backport] Allow overriding repository and branch in validate scripts, and no need to git fetch in CI
Upstream-commit: 02c953cf36fa4f017be402451106d9c273023b4f
Component: engine
- TestAPISwarmLeaderElection
- TestAPISwarmRaftQuorum
- TestSwarmClusterRotateUnlockKey
because they are known to be flaky.
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
(cherry picked from commit 02157c638ba0c325d8fd1debc1678e7e99eacfc1)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 278f1a130b66de23f07e472792b70b640f777666
Component: engine
Due to a bug in Golang (github.com/golang#27640), the "character device"
bit was omitted when checking file-modes with `os.ModeType`.
This bug was resolved in Go 1.12, but as a result, graphdrivers
would no longer recognize "device" files, causing pulling of
images that have a file with this filemode to fail;
failed to register layer:
unknown file type for /var/lib/docker/vfs/dir/.../dev/console
The current code checked for an exact match of Modes to be set. The
`os.ModeCharDevice` and `os.ModeDevice` bits will always be set in
tandem, however, because the code was only looking for an exact
match, this detection broke now that `os.ModeCharDevice` was added.
This patch changes the code to be more defensive, and instead
check if the `os.ModeDevice` bit is set (either with, or without
the `os.ModeCharDevice` bit).
In addition, some information was added to the error-message if
no type was matched, to assist debugging in case additional types
are added in future.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c7a38c2c06f7ab844a48c6c447942913131b83d6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 3744b45ba8ad93f1a21cbc80420856b04efc4593
Component: engine
* Replaces `cocks` with `cerf` as the former might be perceived as
offensive by some people (as pointed out by @jeking3
[here](https://github.com/moby/moby/pull/37157#commitcomment-31758059))
* Removes a duplicate entry for `burnell`
* Re-arranges the entry for `sutherland` to ensure that the names are in
sorted order
* Adds entries for `shamir` and `wilbur`
Signed-off-by: Debayan De <debayande@users.noreply.github.com>
(cherry picked from commit e50f791d42d1167a5ef757b1aa179e84f0f81bba)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a818442de73b16d8ad756c74e5e660d132e97848
Component: engine
By default, exec uses the environment of the current process, however,
if `exec.Env` is not `nil`, the environment is discarded:
e73f489494/src/os/exec/exec.go (L57-L60)
> If Env is nil, the new process uses the current process's environment.
When adding a new environment variable, prepend the current environment,
to make sure it is not discarded.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b84bff7f8ad1562a7d05f21bd84179d3306f2b4b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 19e733f89f7652f58b567b5178bacc10ef2940b5
Component: engine
Here's what happens:
1. One runs `make binary` once
2. Days go by...
3. One makes changes to dockerd sources
4. One runs `make test-integration` to test the changes
5. One spends a long time figuring out why on Earth
those changes in step 3 are ignored by step 4.
6. One writes this patch
7. ...
8. PROFIT!!
OK, so `make test-integration` builds a dockerd binary
in bundles/dynbinary-daemon/, when starts a daemon instance
for testing. The problem is, the script that starts the
daemon sets PATH to try `bundles/binary-daemon/` first,
and `bundles/dynbinary-daemon/` second.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 228bc35e824d09a6f40d60619808462124f0af31)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e9ecd5e486c591979e34782025bb849f7faf8eba
Component: engine
CIs are assumed to do a git fetch and git merge before running tests.
Therefore, no need for a git fetch inside our validate scripts in CI.
If VALIDATE_ORIGIN_BRANCH is set, then git fetch is skipped and
VALIDATE_ORIGIN_BRANCH is used in validate scripts.
Otherwise, behavior is unchanged.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit feb70fd5c9e2fb3f300e953dd83053f0830f3895)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 7b9ec00eec7ffe745ebd2f807daa50d84b3e10e7
Component: engine
When running CI in other repositories (e.g. Docker's downstream
docker/engine repository), or other branches, the validation
scripts were calculating the list of changes based on the wrong
information.
This lead to weird failures in CI in a branch where these values
were not updated ':-) (CI on a pull request failed because it detected
that new tests were added to the deprecated `integration-cli` test-suite,
but the pull request did not actually make changes in that area).
This patch allows overriding the target repository (and branch)
to compare to (without having to edit the scripts).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2a08f33166247da9d4c09d4c6c72cbb8119bf8df)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 748f37022df465c39a76461c4970f4c678f629e3
Component: engine
containerd has two objects with regard to containers.
There is a "container" object which is metadata and a "task" which is
manging the actual runtime state.
When docker starts a container, it creartes both the container metadata
and the task at the same time. So when a container exits, docker deletes
both of these objects as well.
This ensures that if, on start, when we go to create the container metadata object
in containerd, if there is an error due to a name conflict that we go
ahead and clean that up and try again.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 5ba30cd1dc6000ee53b34f628cbff91d7f6d7231)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1d0353548a21b0b60595ad44859a7072f90ea6c6
Component: engine
for windows all networks are re-populated in the store during network controller initialization. In current version it also regenerate network Ids which may be referenced by other components and it may cause broken references to a networks. This commit avoids regeneration of network ids.
Signed-off-by: Andrey Kolomentsev <andrey.kolomentsev@docker.com>
(cherry picked from commit e017717d96540dd263d95f90fdb2457928909924)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 317e0acc4e602f978e4d9c0130a113d179026c8e
Component: engine
containerd is now running as a separate service, and should
no longer be started as a managed child-process of dockerd.
The dockerd service already specifies that it should be started
`After` the containerd.service, but there is still a race
condition, where containerd is started, but its socket is not yet
created.
In that situation, `dockerd` detects that the containerd socket
is missing, and will start a new instance of containerd (as a
managed child-process), which causes live-restore to fail.
This patch explicitly sets the `--containerd` daemon option.
If this option is set, `dockerd` will not start a new instance
of containerd.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1985463b1337e55c0dfcad2fb985fb5af5a10c78)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 15653df497646518ec5af5cab55ebb4461ee5b80
Component: packaging
A git bundle allows us keep the same SHA, giving us the ability to
validate our patch against a known entity and allowing us to push
directly from our private forks to public forks without having to
re-apply any patches.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 03dfb0ba53cc5f64b746a25aa5ed8a48763ea223
Component: engine
- Fix a bug that a container can't be stopped or inspected when its corresponding image is deleted
- Fix a bug that the cri plugin handles containerd events outside of k8s.io namespace
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 27cc170d2891f25ab066620e8fe94d62f81b9aa3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f80c6d7ae15443f15c014ddbd17e30cfac47b906
Component: engine
RHEL/CentOS 3.10 kernels report that kernel-memory accounting is supported,
but it actually does not work.
Runc (when compiled for those kernels) will be compiled without kernel-memory
support, so even though the daemon may be reporting that it's supported,
it actually is not.
This cause tests to fail when testing against a daemon that's using a runc
version without kmem support.
For now, skip these tests based on the kernel version reported by the daemon.
This should fix failures such as:
```
FAIL: /go/src/github.com/docker/docker/integration-cli/docker_cli_run_unix_test.go:499: DockerSuite.TestRunWithKernelMemory
assertion failed:
Command: /usr/bin/docker run --kernel-memory 50M --name test1 busybox cat /sys/fs/cgroup/memory/memory.kmem.limit_in_bytes
ExitCode: 0
Error: <nil>
Stdout: 9223372036854771712
Stderr: WARNING: You specified a kernel memory limit on a kernel older than 4.0. Kernel memory limits are experimental on older kernels, it won't work as expected and can cause your system to be unstable.
Failures:
Expected stdout to contain "52428800"
FAIL: /go/src/github.com/docker/docker/integration-cli/docker_cli_update_unix_test.go:125: DockerSuite.TestUpdateKernelMemory
/go/src/github.com/docker/docker/integration-cli/docker_cli_update_unix_test.go:136:
...open /go/src/github.com/docker/docker/integration-cli/docker_cli_update_unix_test.go: no such file or directory
... obtained string = "9223372036854771712"
... expected string = "104857600"
----------------------------------------------------------------------
FAIL: /go/src/github.com/docker/docker/integration-cli/docker_cli_update_unix_test.go:139: DockerSuite.TestUpdateKernelMemoryUninitialized
/go/src/github.com/docker/docker/integration-cli/docker_cli_update_unix_test.go:149:
...open /go/src/github.com/docker/docker/integration-cli/docker_cli_update_unix_test.go: no such file or directory
... value = nil
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1e1156cf67233cf8eaee2da9c17465ff0d9c2aa0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e042692db1316a60be35bfdca10d7e08d20f50ad
Component: engine
Revert "[18.09 backport] API: fix status code on conflicting service names"
Upstream-commit: ce8b8f1cf33bb33811077be3b0b972dd2dbc4c79
Component: engine
This was removed in 662e248f680eb49a9951a8b34125506b8f82dfed. The
removal breaks usage on Debian derivatives that do not use systemd
(see https://github.com/docker/for-linux/issues/482).
Upstream-commit: 196c4028288ad241f52201d0f171c251ed13b98f
Component: packaging
The CLI package does not provide the functionality of the
"selinux" and "engine" packages (it does _conflict_ with
older engine packages though).
This removes the "obsoletes" from the CLI package, as the
Engine package already obsoletes the other ones.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d4e1ddb963a43ff9f0a06e4ed1b37307c8f26a17)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9e6f124b6b5deba3878ad242661d1991881f33b9
Component: packaging
Currently running the e2e tests produces a warning/error:
$ make -f docker.Makefile test-e2e
«...»
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock docker-cli-e2e
./scripts/test/e2e/run: line 20: test: : integer expression expected
This is from:
test "${DOCKERD_EXPERIMENTAL:-}" -eq "1" && «...»
Where `${DOCKERD_EXPERIMENTAL:-}` expands to the empty string, resulting in
`test "" -eq "1"` which produces the warning. This error is enough to trigger
the short-circuiting behaviour of `&&` so the result is as expected, but fix
the issue nonetheless by provdiing a default `0`.
Signed-off-by: Ian Campbell <ijc@docker.com>
(cherry picked from commit 4f483276cf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c462e06fcd
Component: cli
This includes a patch on top of containerd 1.2.1 to handle fifo
timeouts.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
(cherry picked from commit e5d9d721626958a37dccfa0b234d9fc96d8c2bfb)
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Upstream-commit: d161dfe1a36929a03ee3dfa916d296abfd4ccef1
Component: engine
go1.10.6 (released 2018/12/14)
- crypto/x509: CPU denial of service in chain validation golang/go#29233
- cmd/go: directory traversal in "go get" via curly braces in import paths golang/go#29231
- cmd/go: remote command execution during "go get -u" golang/go#29230
See the Go 1.10.6 milestone on the issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.10.6
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8afe9f422dc0183ce48e1db09189ccbde634080a
Component: engine
[18.09 backport] fixes display text in Multiple IDs found with provided prefix
Upstream-commit: a8572d3e8e1e3f738d6e9bc3bf0f137d64bab616
Component: engine
This avoids the corner case where `via` is not nil, but has a length of 0,
so the updated code does not panic in that situation.
Signed-off-by: Iskander Sharipov <quasilyte@gmail.com>
(cherry picked from commit a5c185b99404ea3fbab47ff9d7ba143392566bc1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 3482a3b14a6414977bd9860c513abf06dedd6bf7
Component: engine
This updates the swarmkit vendoring to the latest version in the bump_v18.09
branch
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ad7105260f3c2ff32a375ff78dce9a96e01d87cb
Component: engine
We don't use these spec files to actually build from anymore so we
should just go through and remove them.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit b98d7aad1e153c18da5b488afc40ba751f90cb36)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9d68fe72f3649fad5e3f902602c8ae9ab4a6abe5
Component: packaging
Without this the docker.socket would not start by default when starting
the docker.service leading to failures to start.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 88885d18b1bb0ef91eab4ad3311773f9c40838b7)
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Upstream-commit: 93f125a2602b7eb72128cbc16d9d35c9d92ed18a
Component: packaging
go1.10.5 (released 2018/11/02) includes fixes to the go command, linker,
runtime and the database/sql package. See the milestone on the issue
tracker for details:
List of changes; https://github.com/golang/go/issues?q=milestone%3AGo1.10.5
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 871d24d3fc
Component: cli
Removes the systemd drop-in unit file for socket activation and instead
prefers socket activation by default for both RHEL based and DEBIAN
based distributions.
Socket activation for RHEL based distributions was tested on CentOS 7 and Fedora 28.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 91c85cd38197b9d92d5b3e8a1b577d3178d73fcc)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 6a7e302c260a2ff7e4d8b7a6d01249a09c140618
Component: packaging
go1.10.5 (released 2018/11/02) includes fixes to the go command, linker,
runtime and the database/sql package. See the milestone on the issue
tracker for details:
List of changes; https://github.com/golang/go/issues?q=milestone%3AGo1.10.5
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 00ad8e7c5730f3c50ae2e548b47d1340202f72b2
Component: engine
@sw-pschmied originally post this in #38285
While looking through the Moby source code was found /proc/asound to be
shared with containers as read-only (as defined in
https://github.com/moby/moby/blob/master/oci/defaults.go#L128).
This can lead to two information leaks.
---
**Leak of media playback status of the host**
Steps to reproduce the issue:
- Listen to music/Play a YouTube video/Do anything else that involves
sound output
- Execute docker run --rm ubuntu:latest bash -c "sleep 7; cat
/proc/asound/card*/pcm*p/sub*/status | grep state | cut -d ' ' -f2 |
grep RUNNING || echo 'not running'"
- See that the containerized process is able to check whether someone
on the host is playing music as it prints RUNNING
- Stop the music output
- Execute the command again (The sleep is delaying the output because
information regarding playback status isn't propagated instantly)
- See that it outputs not running
**Describe the results you received:**
A containerized process is able to gather information on the playback
status of an audio device governed by the host. Therefore a process of a
container is able to check whether and what kind of user activity is
present on the host system. Also, this may indicate whether a container
runs on a desktop system or a server as media playback rarely happens on
server systems.
The description above is in regard to media playback - when examining
`/proc/asound/card*/pcm*c/sub*/status` (`pcm*c` instead of `pcm*p`) this
can also leak information regarding capturing sound, as in recording
audio or making calls on the host system.
Signed-off-by: Jonathan A. Schweder <jonathanschweder@gmail.com>
(cherry picked from commit 64e52ff3dbdb31adc0a9930b3ea74b04b0df8d86)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5fffdb32261145b1178f571e25fbd71572769d58
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
(cherry picked from commit 14c8b67e51a8340284dfd8919843ce9edd1e5c71)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9c93de59da8eaa0e0e0377578a33b465b9465bb2
Component: engine
This is needed for implementing bash completion for the `docker manifest`
command family.
Signed-off-by: Harald Albers <github@albersweb.de>
(cherry picked from commit a183c952c6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2a36695037
Component: cli
This preapares bash completion for more context sensitivity:
- experimental cli features
- orchestrator specific features
Also renames _daemon_ to _server_ where used in context of `docker version`
because the fields there are grouped unter _Server_.
Signed-off-by: Harald Albers <github@albersweb.de>
(cherry picked from commit 564d4da06e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: dc74fc81f2
Component: cli
A similar change was made in the CLI itself, but is not
inherited by the code that generates the YAML docs.
Before this patch is applied;
```
usage: docker container exec [OPTIONS] CONTAINER COMMAND [ARG...] [flags]
```
With this patch applied:
```
usage: docker container exec [OPTIONS] CONTAINER COMMAND [ARG...]
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 44d96e9120)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 7e90635652
Component: cli
The RPM packages list containerd as a hard dependency. While
having containerd installed allows certain features (e.g., allow
you to run `docker engine activate`), this should not be a requirement
for installing the Docker CLI, as it limits the use of this package
for situations where the CLI is installed to connect to a remote
daemon.
This patch removes the containerd dependency from the RPM packages
(the deb packages don't have this dependency, so no change is needed
in those packages)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 037349c5dc3f18d3bea3be0217e249f716a8be25)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: bc3867a8f6e59c56c967720c33c6ce9a65b64b7b
Component: packaging
During the migration to "image based builds", some dependencies
were removed.
This patch brings back those dependencies.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a6ff66f1a51af62c9fd17a8b931f8a9b0283247a
Component: packaging
This requirement was originally added in 86f76496ce33bd6eff1737348bc44add4723ddd2,
but got removed in the migration to the new image-based packaging.
Commit f2ceca98823548e401c6cca125b7057df28c4892 added this requirement
back for `.deb` packages, but did not include the same changes for
RPMs.
This patch adds back the requirement for RPM packages as well.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 86ed6eb2cef4e7386979fc73397c1844531eb3a9
Component: packaging
Nightly builds wouldn't install correctly since our versioning scheme
for nightly builds is 0.0.0~ and 0 < 18.
Should be backwards compatible with 18.09 builds so there's no need to
actually backport this to 18.09.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit de312224beff9c6bc7fd6def46459d531cf2ce4a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 3e383bee967aa07d35ea717544bc18aafbbad4f7
Component: packaging
This is to keep the UX consistent. `unused-for` is still accepted and a synonym.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 369da264bac15769ae944cc880e66abaf158612b)
Upstream-commit: 0b2d88d328ca88c8732dc11c72873b53be3bd2f8
Component: engine
This is cleaner because running system prune twice in a row
now results in a no-op the second time.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 6c10abb247)
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 4a4a1f3615
Component: cli
These options were added in API 1.39, so should be ignored
when using an older version of the API.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7632ccbc66c82179547b96524672bd5082ae7481)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 4cc45d91eb44cbaddcfd75335c3f72ede035c440
Component: engine
In newer kernels, AppArmor will reject attempts to send signals to a
container because the signal originated from outside of that AppArmor
profile. Correct this by allowing all unconfined signals to be received.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Signed-off-by: Aleksa Sarai <asarai@suse.de>
(cherry picked from commit 4822fb1e2423d88cdf0ad5d039b8fd3274b05401)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 67c602c3fe3d5d817fb7210e50d7ed1688b28801
Component: engine
The previous update used a commit from master. Now that
all the fixes are backported to the containerd 1.2 release
branch, we can switch back to that branch.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2fb5de68a9bd05b1dbf3ae3f7ae82bcd5e64dc5c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: db7f375d6a2aaf6d79f5c690e2f302c640bdde04
Component: engine
release notes: https://github.com/containerd/containerd/releases/tag/v1.2.0
- New V2 Runtime with a stable gRPC interface for managing containers through
external shims.
- Updated CRI Plugin, validated against Kubernetes v1.11 and v1.12, but it is
also compatible with Kubernetes v1.10.
- Support for Kubernetes Runtime Class, introduced in Kubernetes 1.12
- A new proxy plugin configuration has been added to allow external
snapshotters be connected to containerd using gRPC.-
- A new Install method on the containerd client allows users to publish host
level binaries using standard container build tooling and container
distribution tooling to download containerd related binaries on their systems.
- Add support for cleaning up leases and content ingests to garbage collections.
- Improved multi-arch image support using more precise matching and ranking
- Added a runtime `options` field for shim v2 runtime. Use the `options` field to
config runtime specific options, e.g. `NoPivotRoot` and `SystemdCgroup` for
runtime type `io.containerd.runc.v1`.
- Some Minor API additions
- Add `ListStream` method to containers API. This allows listing a larger
number of containers without hitting message size limts.
- Add `Sync` flag to `Delete` in leases API. Setting this option will ensure
a garbage collection completes before the removal call is returned. This can
be used to guarantee unreferenced objects are removed from disk after a lease.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8674930c84140c990451adb148165422d008b661)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e137337fe6083da91fd6d83d699cff3a857f636e
Component: engine
Containerd should be "leading" when specifying which version of runc to use.
From the RUNC.MD document in the containerd repository
(https://github.com/containerd/containerd/blob/b1e202c32724e82779544365528a1a082
b335553/RUNC.md);
> We depend on a specific runc version when dealing with advanced features. You
> should have a specific runc build for development. The current supported runc
> commit is described in vendor.conf. Please refer to the line that starts with
> github.com/opencontainers/runc.
This patch adds a note to vendor.conf and runc.installer to describe the order
in which runc should be updated.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit da3810d235bc0bd0197243d0128f258394554704)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c9c87d76d651d57d72e52c575a2c9600170b5212
Component: engine
log-opts are passed to logging-drivers as-is, so the daemon is not
aware what value-type each option takes.
For this reason, all options must be provided as a string, even if
they are used as numeric values by the logging driver.
For example, to pass the "max-file" option to the default (json-file)
logging driver, this value has to be passed as a string;
```json
{
"log-driver": "json-file",
"log-opts": {
"max-size": "10m",
"max-file": "3"
}
}
```
When passed as a _number_ (`"max-file": 3`), the daemon will invalidate
the configuration file, and fail to start;
unable to configure the Docker daemon with file /etc/docker/daemon.json: json: cannot unmarshal number into Go value of type string
This patch adds an example to the daemon.json to show these values
have to be passed as strings.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit fd33e0d933)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9b3eea87ee
Component: cli
- `--force` is not available in `service create`
- `--force` is a boolean option
Signed-off-by: Harald Albers <github@albersweb.de>
(cherry picked from commit 5fa5eb1da6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 046ffa4e87
Component: cli
On kernels without this options set, publishing ports for swarm
services does not work, making the published port not accessible
("connection refused")
Thanks to Wenbo Wang for reporting, and Tianon for finding this.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 44e1c6ce81ff62c76ba630caf16a73922950abc3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 25bec4665b6f011e52f7b2765ba1579c7430481d
Component: engine
Re-adds the docker.socket file for debian based distributions.
TAR-144 +review
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 69bd2eea398e0396c24f97d0697f9befd0f2464e)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: aaf035970219d6ae19e22bb94adaf51e61a9ff82
Component: packaging
In case we're running on RHEL7 kernel, which has non-working
and broken kernel memory controller, add 'nokmem' build tag
so that runc never enables kmem accounting.
For more info, see the following runc commit:
https://github.com/opencontainers/runc/commit/6a2c1559684
This behavior can be overriden by having `RUNC_NOKMEM` environment
variable set (e.g. to empty value to disable setting nokmem).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 8972aa9350d52e4a7e58242447b7a9d2f0c27f37)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8486ea11ae800a1e6d634b741dfb007ba29f6003
Component: engine
client.checkResponseErr() was hanging and consuming infinite memory
when the serverResp.Body io.Reader returns infinite stream.
This commit prohibits reading more than 1MiB.
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
(cherry picked from commit 1db4be0c32267b7a8a27998089ce96440bed492e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 49556e047059d81b64f6cd12f4c602c9a9c471c7
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
This is a fix for a few related scenarios where it's impossible to remove layers or containers
until the host is rebooted. Generally (or at least easiest to repro) through a forced daemon kill
while a container is running.
Possibly slightly worse than that, as following a host reboot, the scratch layer would possibly be leaked and
left on disk under the dataroot\windowsfilter directory after the container is removed.
One such example of a failure:
1. run a long running container with the --rm flag
docker run --rm -d --name test microsoft/windowsservercore powershell sleep 30
2. Force kill the daemon not allowing it to cleanup. Simulates a crash or a host power-cycle.
3. (re-)Start daemon
4. docker ps -a
PS C:\control> docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7aff773d782b malloc "powershell start-sl…" 11 seconds ago Removal In Progress malloc
5. Try to remove
PS C:\control> docker rm 7aff
Error response from daemon: container 7aff773d782bbf35d95095369ffcb170b7b8f0e6f8f65d5aff42abf61234855d: driver "windowsfilter" failed to remove root filesystem: rename C:\control\windowsfilter\7aff773d782bbf35d95095369ffcb170b7b8f0e6f8f65d5aff42abf61234855d C:\control\windowsfilter\7aff773d782bbf35d95095369ffcb170b7b8f0e6f8f65d5aff42abf61234855d-removing: Access is denied.
PS C:\control>
Step 5 fails.
(cherry picked from commit efdad5374465a2a889d7572834a2dcca147af4fb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 02fe71843e0e45ddc986a6c8182370b042349a27
Component: engine
The CloudWatch Logs API defines its limits in terms of bytes, but its
inputs in terms of UTF-8 encoded strings. Byte-sequences which are not
valid UTF-8 encodings are normalized to the Unicode replacement
character U+FFFD, which is a 3-byte sequence in UTF-8. This replacement
can cause the input to grow, exceeding the API limit and causing failed
API calls.
This commit adds logic for counting the effective byte length after
normalization and splitting input without splitting valid UTF-8
byte-sequences into two invalid byte-sequences.
Fixes https://github.com/moby/moby/issues/37747
Signed-off-by: Samuel Karp <skarp@amazon.com>
(cherry picked from commit 1e8ef386279e2e28aff199047e798fad660efbdd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 757650e8dcca87f95ba083a80639769a0b6ca1cc
Component: engine
These tests don't seem to have anything Linux-specific,
so enable them on Windows
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b334198e652b503d432832ec6f4faab7707d0eb4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e8eb3ca4eef1c913563195787eb1f2527c9febf3
Component: engine
Some improvements in this test;
- use the volume-information that's returned by VolumeCreate as "expected"
- don't use an explict name for the volume, as it was only used to reference
the volume for inspection
- improve the test-output on failure, so that "expected" and "actual" values
are printed
Without this patch applied;
=== RUN TestVolumesInspect
--- FAIL: TestVolumesInspect (0.02s)
volume_test.go:108: assertion failed: false (bool) != true (true bool): Time Volume is CreatedAt not equal to current time
FAIL
With this patch applied;
=== RUN TestVolumesInspect
--- FAIL: TestVolumesInspect (0.02s)
volume_test.go:95: assertion failed: expression is false: createdAt.Truncate(time.Minute).Equal(now.Truncate(time.Minute)): CreatedAt (2018-11-01 16:15:20 +0000 UTC) not equal to creation time (2018-11-01 16:15:20.2421166 +0000 UTC m=+13.733512701)
FAIL
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8e8cac82637952d49f8df42a3164478f13978c90)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 65bf95f3df84de5901479091685715f227c333ce
Component: engine
- Add windows CI entrypoint script.
Signed-off-by: John Howard <jhoward@microsoft.com>
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
(cherry picked from commit d3cc071bb98669545d4e3043c9bd85879292b815)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 59be98043a02f44b63b26f159461fed08292e027
Component: engine
Fixes a compatibility issue on recent debian versions, where iptables now uses
nft by default.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e1783a72d1b84bc3e32470c468d14445e5fba8db
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
(cherry picked from commit d03ab106624ebc30b69080d0092702ad1fe1285c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 41f3cea42f2f45244051b7f829c0ef9c27383c26
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
This removes the need for an SVM in the LCOW driver to ApplyDiff.
This change relates to a fix for https://github.com/moby/moby/issues/36353
However, it found another issue, tracked by https://github.com/moby/moby/issues/37955
(cherry picked from commit bde99960657818efabf313991867c42921882275)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9cf6464b639aef09df9b16999cc340a1276d0bf8
Component: engine
Only the filters the user specified should be added as cache filters to buildkit.
Make an AND operation of the provided filters.
ID filter now does prefix-matching.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit b6137bebb83e886aef906b7ff277778b69616991)
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 52a3c39506b883f713694ce39d1a4fd9f5638800
Component: engine
`d.Sock()` already returns the socket-path including the
`unix://` scheme.
Also removed `--iptables=false`, as it didn't really seem
nescessary for this test.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 14342046477b7ed87a9c5c451bb4520c7f6cabcc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c40a7d393bca990e07973024e71034b4b6fc05e5
Component: engine
4MB client side limit was introduced in vendoring go-grpc#1165 (v1.4.0)
making these requests likely to produce errors
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 489b8eda6674523df8b82a210399b7d2954427d0)
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Upstream-commit: 6ca0546f2571cf4acdc1f541bccfac23a78cb8d2
Component: engine
Hopefully this removes some confusion as to what this version number
should be.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 5bdfa19b8646839f9d704307aa6589c7d686db44)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 64b0c76151ceb7b26f9c7477f3044dac747d227b
Component: engine
Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
(cherry picked from commit f962bd06ed8824d1f75d8546b428965cd61bdf7f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5591f0b1ee7dec101b490228258613cd7caf64ee
Component: engine
- remove features/fixes that were backported to 18.06, so already shipped
- re-organised some change to be in a more suitable section
- add information about deprecated features, and distros
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Packages are no longer built for Ubuntu 17.10 since
commit eb635bdb3044b008344c8318c8d727ac94662526, so
remove it from the README as well
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9cab4c43c4dd30e86efed1dfb9199ed91d766eca)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 89705eb29d9f8782cf218a6d4692dd8a34a95857
Component: packaging
Docker no longer ships packages for Ubuntu 14.04,
so remove the build scripts as well.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3a8fa7fc0acc5c3fb468ec5ec41e236e741c804e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 90fadd7b66e8696cc0d3479293bc62ea730ec51e
Component: packaging
Docker is no longer packaged for Debian Jessie,
so remove the scripts.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 49524c88c2b2ab1751941992913adcff36487f7a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 146e29aa88e8598d5a89ae29a091ff303fd4f3e5
Component: packaging
Packages for Wheezy are no longer built, so remove it
from the list of packages in the README.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c8935f360bc083ed61984a691bcfe19e9cfa1749)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9e724cc033c392764c1bf3b37d40ccb4d2e66bcd
Component: packaging
Includes fixes to the go command, linker, and the net/http, mime/multipart,
ld/macho, bytes, and strings packages. See the Go 1.10.4 milestone on the
issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.10.4
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit fe1fb7417c360f185ce608e0bfda41c0ef9fe364)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 72368177254811e5816f03a4773deaafb9df5202
Component: engine
I am attempting to push a tag to a private repository. The documentation for `docker tag` has an explicit example to for how ["To push an image to a private registry"](https://docs.docker.com/engine/reference/commandline/tag/#tag-an-image-referenced-by-name). My colleague clarified that this command does not in fact push anything, so I thought this PR might save some future novice the same confusion.
Signed-off-by: Jake Lambert <jake.lambert@volusion.com>
(cherry picked from commit 4ed484bac4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1df47ffb4d
Component: cli
ddadd3db49 mass standardized the
formatting, with some errors.
This commit fixes errors on `login.md`:
- revert wrong `Logging out` headline
- restore correct level for some headlines (relative to parent
headline level change)
- re-add `Usage` headlines, with better name
Also add `related commands` headline on `login` and `logout`.
Signed-off-by: Thomas Riccardi <thomas@deepomatic.com>
(cherry picked from commit a0e3ec8790)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2e7e529a18
Component: cli
add regression case for the issue#37870
Signed-off-by: Wei Fu <fuweid89@gmail.com>
(cherry picked from commit 8e25f4ff6d89888a1bcd578f3f8f7aab89dce24d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ae6284a623bac86ac6ab718fa4a369dd8c0a3cfc
Component: engine
Avoid testing for session support in non-buildkit builder to support
servers that falsely report as `1.39` compatible
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 3e8c41beb0)
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 4c68a9666f
Component: cli
[18.09 backport] move the syslog syscall to be gated by CAP_SYS_ADMIN or CAP_SYSLOG
Upstream-commit: 6f1145e740f35a0b805a11d5d29daa89d2a27ed0
Component: engine
Removes the billing profile flow which is now handled on the back-end.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 8e565d0399
Component: cli
Bump libnetwork to 6da50d1978302f04c3e2089e29112ea24812f05b which
is the current tip of libnetwork's bump_18.09 branch to get the DSR load
balancing mode option changes for the 18.09 branch of Docker CE.
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: fd1fe0b702571865cc77d66937e4ca570b5b9cc3
Component: engine
The `devicemapper` storage driver is deprecated in favor of `overlay2`, and will
be removed in a future release. Users of the `devicemapper` storage driver are
recommended to migrate to a different storage driver, such as `overlay2`, which
is now the default storage driver.
The `devicemapper` storage driver facilitates running Docker on older (3.x) kernels
that have no support for other storage drivers (such as overlay2, or AUFS).
Now that support for `overlay2` is added to all supported distros (as they are
either on kernel 4.x, or have support for multiple lowerdirs backported), there
is no reason to continue maintenance of the `devicemapper` storage driver.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 662441ba31)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: fde819236b
Component: cli
The `overlay` storage driver is deprecated in favor of the `overlay2` storage
driver, which has all the benefits of `overlay`, without its limitations (excessive
inode consumption). The legacy `overlay` storage driver will be removed in a future
release. Users of the `overlay` storage driver should migrate to the `overlay2`
storage driver.
The legacy `overlay` storage driver allowed using overlayFS-backed filesystems
on pre 4.x kernels. Now that all supported distributions are able to run `overlay2`
(as they are either on kernel 4.x, or have support for multiple lowerdirs
backported), there is no reason to keep maintaining the `overlay` storage driver.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8bc2aa45a6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: aa6314c663
Component: cli
For some reason, shared mount propagation between the host
and a container does not work for btrfs, unless container
root directory (i.e. graphdriver home) is a bind mount.
The above issue was reproduced on SLES 12sp3 + btrfs using
the following script:
#!/bin/bash
set -eux -o pipefail
# DIR should not be under a subvolume
DIR=${DIR:-/lib}
MNT=$DIR/my-mnt
FILE=$MNT/file
ID=$(docker run -d --privileged -v $DIR:$DIR:rshared ubuntu sleep 24h)
docker exec $ID mkdir -p $MNT
docker exec $ID mount -t tmpfs tmpfs $MNT
docker exec $ID touch $FILE
ls -l $FILE
umount $MNT
docker rm -f $ID
which fails this way:
+ ls -l /lib/my-mnt/file
ls: cannot access '/lib/my-mnt/file': No such file or directory
meaning the mount performed inside a priviledged container is not
propagated back to the host (even if all the mounts have "shared"
propagation mode).
The remedy to the above is to make graphdriver home a bind mount.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 16d822bba8ac5ab22c8697750f700403bca3dbf3)
Upstream-commit: fa8ac946165b8004a15e85744e774ed6ba99fd38
Component: engine
This function ensures the argument is the mount point
(i.e. if it's not, it bind mounts it to itself).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 8abadb36fa8149cd44e76b0e7fdedd6f1f2eccd0)
Upstream-commit: 2199ada691dc635cac5cdd065d909a539dd0b793
Component: engine
1. There is no need to specify rw argument -- bind mounts are
read-write by default.
2. There is no point in parsing /proc/self/mountinfo after performing
a mount, especially if we don't check whether the fs is mounted or
not -- the only outcome from it could be an error from our mountinfo
parser, which makes no sense in this context.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit f01297d1ae352bc2bf01ebf62e879c1c83cdbee4)
Upstream-commit: fd7611ff1f1d61d5b4b45b2c0bd83976cbccf174
Component: engine
The `overlay` storage driver is deprecated in favor of the `overlay2` storage
driver, which has all the benefits of `overlay`, without its limitations (excessive
inode consumption). The legacy `overlay` storage driver will be removed in a future
release. Users of the `overlay` storage driver should migrate to the `overlay2`
storage driver.
The legacy `overlay` storage driver allowed using overlayFS-backed filesystems
on pre 4.x kernels. Now that all supported distributions are able to run `overlay2`
(as they are either on kernel 4.x, or have support for multiple lowerdirs
backported), there is no reason to keep maintaining the `overlay` storage driver.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 31be4e0ba11532e5d6df8401f5d459e292c58f36)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c20e8dffbbb4dd328e4c00d18e04eecb80cf4d4e
Component: engine
The `devicemapper` storage driver is deprecated in favor of `overlay2`, and will
be removed in a future release. Users of the `devicemapper` storage driver are
recommended to migrate to a different storage driver, such as `overlay2`, which
is now the default storage driver.
The `devicemapper` storage driver facilitates running Docker on older (3.x) kernels
that have no support for other storage drivers (such as overlay2, or AUFS).
Now that support for `overlay2` is added to all supported distros (as they are
either on kernel 4.x, or have support for multiple lowerdirs backported), there
is no reason to continue maintenance of the `devicemapper` storage driver.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 06fcabbaa0d4f25db0580d3604bbed708223fb06)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 734e7a8e55e41b064025273776dc5a76e45bf2e1
Component: engine
Unlike `docker build --secret`, `docker build --ssh` allows the build container to
use SSH keys with passphrases.
$ eval $(ssh-agent)
$ ssh-add ~/.ssh/id_rsa
(Input your passphrase here)
$ docker build --ssh default=$SSH_AUTH_SOCK ...
This feature requires the daemon with `CapExecMountSSH` build capability (moby/moby#37973) .
Currently, the official Dockerfile frontend does not provide the syntax for using the SSH forwarder.
However, the experimental `RUN --mount=type=ssh` syntax can be enabled by using
the Dockerfile frontend image built with the `BUILDTAGS="dfrunmount dfssh"`, via the `# syntax =` "shebang".
The Dockerfile for the Dockerfile frontend is available at github.com/moby/buildkit/frontend/dockerfile/cmd/dockerfile-frontend)
The pre-built image is also available as `tonistiigi/dockerfile:ssh20181002` .
An example Dockerfile with `RUN --mount=type=ssh`:
# syntax = tonistiigi/dockerfile:ssh20181002
FROM alpine
RUN apk add --no-cache openssh-client
RUN mkdir -p -m 0700 ~/.ssh && ssh-keyscan gitlab.com >> ~/.ssh/known_hosts
RUN --mount=type=ssh ssh git@gitlab.com | tee /hello
# "Welcome to GitLab, @GITLAB_USERNAME_ASSOCIATED_WITH_SSHKEY" should be printed here
More info available at moby/buildkit#608, moby/buildkit#655
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
(cherry picked from commit db7399a016)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e942084530
Component: cli
[18.09] backport fix denial of service with large numbers in cpuset-cpus and cpuset-mems
Upstream-commit: 4b8336f7cf091fd5c4742286bda1e34c45667d78
Component: engine
As pointed out in https://github.com/moby/moby/issues/37970,
Docker overlay driver can't work with index=on feature of
the Linux kernel "overlay" filesystem. In case the global
default is set to "yes", Docker will fail with EBUSY when
trying to mount, like this:
> error creating overlay mount to ...../merged: device or resource busy
and the kernel log should contain something like:
> overlayfs: upperdir is in-use by another mount, mount with
> '-o index=off' to override exclusive upperdir protection.
A workaround is to set index=off in overlay kernel module
parameters, or even recompile the kernel with
CONFIG_OVERLAY_FS_INDEX=n in .config. Surely this is not
always practical or even possible.
The solution, as pointed out my Amir Goldstein (as well as
the above kernel message:) is to use 'index=off' option
when mounting.
NOTE since older (< 4.13rc1) kernels do not support "index="
overlayfs parameter, try to figure out whether the option
is supported. In case it's not possible to figure out,
assume it is not.
NOTE the default can be changed anytime (by writing to
/sys/module/overlay/parameters/index) so we need to always
use index=off.
[v2: move the detection code to Init()]
[v3: don't set index=off if stat() failed]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 8422d85087bfa770b62ef4e1daaca95ee6783d86)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 690e097fedd7362f3b2781c32ca872ad966d286e
Component: engine
This simplifies the code a lot.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit a55d32546a8556f9e6cabbc99836b573b9944f0c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: dc0a4db7c9dc593a8568a8e30e4e21e118c2839d
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
(cherry picked from commit c907c2486c0850030f8ac1819ac8c87631472c68)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 7184074c0880c656be00645007588a00ec2266cd
Component: engine
The stdin fifo of exec process is created in containerd side after
client calls Start. If the client calls CloseIO before Start call, the
stdin of exec process is still opened and wait for close.
For this case, client closes stdinCloseSync channel after Start.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
(cherry picked from commit c7890f25a9eaae8d07614bd85b2b3231b03e54ec)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 6679a5faeb724f1ad060f2fdf6d189f1005924b9
Component: engine
We'll be using a redirect from this URL to the back-end docs system for
hosting release notes. Final location confirmed with Docs team and PM.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: f93908213a
Component: cli
Using a value such as `--cpuset-mems=1-9223372036854775807` would cause
`dockerd` to run out of memory allocating a map of the values in the
validation code. Set limits to the normal limit of the number of CPUs,
and improve the error handling.
Reported by Huawei PSIRT.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f8e876d7616469d07b8b049ecb48967eeb8fa7a5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0922d32bce74657266aff213f83dfa638e8077f4
Component: engine
This updates the containerd dependencies to match
the versions used by the vendored containerd version
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 31a9c9e79101cdf38d383104afbc1b48ede75291)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5070e418b806cc96ad0f5b3ac32c8d416ff8449a
Component: engine
With containerd reaching 1.0, the runtime now
has a stable API, so there's no need to do a check
if the installed version matches the expected version.
Current versions of Docker now also package containerd
and runc separately, and can be _updated_ separately.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c65f0bd13c85d29087419fa555281311091825e7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 054c3c2931cec5dca8bb84af97f1457c343ec02f
Component: engine
We check for max value for -default-addr-pool-mask-length param as 32.
But There won't be enough addresses on the overlay network. Hence we are
keeping it 29 so that we would be having atleast 8 addresses in /29 network.
Signed-off-by: selansen <elango.siva@docker.com>
(cherry picked from commit d25c5df80e60cdbdc23fe3d0e2a6808123643dc7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9406f3622d18a0d9b6c438190e8fdd8be53d3b22
Component: engine
Addressing few review comments as part of code refactoring.
Also moved validation logic from CLI to Moby.
Signed-off-by: selansen <elango.siva@docker.com>
(cherry picked from commit 148ff00a0a800fad99de11ee3021d4c5d4869157)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9816bfcaf58a609d64d648043c10817c27dcfa36
Component: engine
[18.09] Fix long startup on windows, with non-hns governed Hyper-V networks
Upstream-commit: 6e5ed2ccce0923950dae7a6cb9bd1dd851ed5af8
Component: engine
Similar to a related issue where previously, private Hyper-V networks
would each add 15 secs to the daemon startup, non-hns governed internal
networks are reported by hns as network type "internal" which is not
mapped to any network plugin (and thus we get the same plugin load retry
loop as before).
This issue hits Docker for Desktop because we setup such a network for
the Linux VM communication.
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
(cherry picked from commit 6a1a4f97217b0a8635bc21fc86628f48bf8824d1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 54bd14a3fe1d4925c6fa88b24949063d99067c07
Component: engine
The parameter name was wrong, which may mislead a user.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit c378fb774e413ba8bf5cadf655d2b67e9c94245a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c9ddc6effc444c54def41d498b359a9a986ad79d
Component: engine
This call is what is used to implement `dmesg` to get kernel messages
about the host. This can leak substantial information about the host.
It is normally available to unprivileged users on the host, unless
the sysctl `kernel.dmesg_restrict = 1` is set, but this is not set
by standard on the majority of distributions. Blocking this to restrict
leaks about the configuration seems correct.
Fix#37897
See also https://googleprojectzero.blogspot.com/2018/09/a-cache-invalidation-bug-in-linux.html
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
(cherry picked from commit ccd22ffcc8b564dfc21e7067b5248819d68c56c6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 16836e60bc87abb3e9ab16f33c2038931c1d473b
Component: engine
This should ensure we don't produce builds without a valid "Built:"
field in the the version payload.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 9a071a3fbe316ea6d70bc1a5111cbe14ae99aec9
Component: packaging
During the refactoring for 18.09 the activate/update flows no longer
restart the engine explicitly but let the user do that when they're ready,
so the health check logic is no longer required.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: f2b2061cc3
Component: cli
This is a follow up PR to #1381 to address some of the review comments
we didn't get to.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: c12e23a4c1
Component: cli
Help the user understand which license they're about
to load in case they have multiple licenses they need to
figure out.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 5a97a93ae1
Component: cli
These messages were enhanced to include the path that was
missing (in df6af282b9048dfedcd7b7a9a89126aca887f4e1), but
also changed the first part of the message.
This change complicates running e2e tests with mixed versions
of the engine.
Looking at the full error message, "mount" is a bit redundant
as well, because the error message already indicates this is
about a "mount";
docker run --rm --mount type=bind,source=/no-such-thing,target=/foo busybox
docker: Error response from daemon: invalid mount config for type "bind": bind mount source path does not exist: /no-such-thing.
Removing the "mount" part from the error message, because
it was redundant, and makes cross-version testing easier :)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 574db7a53782c57554089c9606505af1c108df0b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b499acc0e834e11882909269238407c65f68f034
Component: engine
People blow away `/var/lib/docker` all the time so we probably shouldn't
store important data there.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 9391057c9472ba24049b8645c251a4c63894522f)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 865140fc4155481b20f694a4528e04e48eb76de4
Component: packaging
Epoch needs to be included in the Replaces / Breaks sections
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit f6a3e274b4892f8f501e1191a73d8b8935a56aa5)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: bebe7028c6b41ecee80278cdfc46de1e7704d199
Component: packaging
For static builds.
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 43ebdc0845840c13c2f92c7b564ddb56eb38d428
Component: packaging
Due to a typo, substitution would not work if the given
environment-variable was set.
Given the following docker compose file;
```yaml
version: "3.7"
services:
app:
image: nginx:${version:-latest}
```
Deploying a stack with `$version` set would ignore the `$version`
environment variable, and use the default value instead;
```bash
version=alpine docker stack deploy -c docker-compose.yml foobar
Creating network foobar_default
Creating service foobar_app
docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
rskkjxe6sm0w foobar_app replicated 1/1 nginx:latest
```
This patch also fixes "soft default" not detecting empty environment variables,
only non-set environment variables.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ec3daea021)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b8702b8a9a
Component: cli
This allows to run the daemon in environments that have upstream containerd installed.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 34eede0296bce6a9c335cb429f10728ae3f4252d)
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: b3bb2aabb8ed5a8af0a9f48fb5aba3f39af38e0d
Component: engine
This allows to provide more information for build cache disk usage.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit a90b99edfc)
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 9de1318e36
Component: cli
[18.09] backport fixing daemon won't start when "runtimes" option defined in both config file and cli
Upstream-commit: 90a90ae2e1fdb1f9782a51e41fe2b7dc80d48952
Component: engine
The packages will deliver this as a link so lets make sure we don't
write through the link to the underlying packaged file.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 0f22d7e295
Component: cli
If, for some reason, the certs directory has permissions that are
inaccessible by docker, we should still be able to fetch manifests using
the `insecure` flag.
Since the cli doesn't access the engine's list of insecure registries,
the registry client should make a singleton list of the registry being queried with the
`insecure` flag.
Closes#1358
Signed-off-by: Christy Norman <christy@linux.vnet.ibm.com>
(cherry picked from commit d57adbc034)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f9d666b057
Component: cli
These files are not used, and having them in repo is misleading.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: fca673ad8051d1b377435821ec6c76c0de24b663
Component: packaging
Set the PATH to what appears to be the standard on latest Ubuntu (18.04)
and Debian (9), fixing the following two issues:
1. PATH did not contain /bin (leading to ContainerTop/ps not working
on newer distros, among the other things).
2. $PATH can't be specified in Environment directives in .service files.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 62d9a0d1253049a2fab10f938542a8e09dd0804a
Component: packaging
Adapt the CLI to the host install model for 18.09.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 342afe44fb
Component: cli
The `/etc/docker` directory is used both by the dockerd daemon
and the docker cli (if installed on the saem host as the daemon).
In situations where the `/etc/docker` directory does not exist,
and an initial `key.json` (legacy trust key) is generated (at the
default location), the `/etc/docker/` directory was created with
0700 permissions, making the directory only accessible by `root`.
Given that the `0600` permissions on the key itself already protect
it from being used by other users, the permissions of `/etc/docker`
can be less restrictive.
This patch changes the permissions for the directory to `0755`, so
that the CLI (if executed as non-root) can also access this directory.
> **NOTE**: "strictly", this patch is only needed for situations where no _custom_
> location for the trustkey is specified (not overridden with `--deprecated-key-path`),
> but setting the permissions only for the "default" case would make
> this more complicated.
```bash
make binary shell
make install
ls -la /etc/ | grep docker
dockerd
^C
ls -la /etc/ | grep docker
drwxr-xr-x 2 root root 4096 Sep 14 12:11 docker
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cecd9817177093be99c1c9bb0dcf43ccec14ad1d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: fc576226b24e8b5db6e95e48967d56c5808f9fe9
Component: engine
devicemapper cannot be enabled on a statically compiled binary so we
dynamically compile it on a rhel based distribution in order to enable
devicemapper usage.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 7cb8d358947974abae3e3c346b09523904030609)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: c80ac56dfa393c405c2cd33af94ad6ac1366ece2
Component: packaging
iptables is sometimes placed in `/usr/sbin`
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit c86a958d6d344d6bf568523c28d6643d7b49ba43)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 36a08784a0b7bc49487005d323bf4e41fd50d36d
Component: packaging
[18.09] backport always hornor client side to choose which builder to use with DOCKER_…
Upstream-commit: 5fb0a7ced7bcdb9e86e048fe3b24d0d4297d6155
Component: engine
I could not reproduce the panic in #37735, so here's a bandaid.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 7dac70324d0ce6acd23458b0bef06f099837d648)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2c26eac56628527ed64c79ce9145ed97583cbeca
Component: engine
Saw this in moby ci:
> 00:22:07.582 === RUN TestServiceWithDefaultAddressPoolInit
> 00:22:08.887 --- FAIL: TestServiceWithDefaultAddressPoolInit (1.30s)
> 00:22:08.887 daemon.go:290: [d905878b35bb9] waiting for daemon to start
> 00:22:08.887 daemon.go:322: [d905878b35bb9] daemon started
> 00:22:08.888 panic: runtime error: index out of range [recovered]
> 00:22:08.889 panic: runtime error: index out of range
> 00:22:08.889
> 00:22:08.889 goroutine 360 [running]:
> 00:22:08.889 testing.tRunner.func1(0xc42069d770)
> 00:22:08.889 /usr/local/go/src/testing/testing.go:742 +0x29d
> 00:22:08.890 panic(0x85d680, 0xb615f0)
> 00:22:08.890 /usr/local/go/src/runtime/panic.go:502 +0x229
> 00:22:08.890 github.com/docker/docker/integration/network.TestServiceWithDefaultAddressPoolInit(0xc42069d770)
> 00:22:08.891 /go/src/github.com/docker/docker/integration/network/service_test.go:348 +0xb53
> .....
Apparently `out.IPAM.Config[0]` is not there, so to avoid panic, let's
check the size of `out.IPAM.Config` first.
Fixes: f7ad95cab9c
[v2: add logging of data returned by NetworkInspect()]
[v3: use assert.Assert to fail immediately]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 69d3a8936b8d91e43290aadb9f7ac2cd2929dc61)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f43fc6650cf5a452157fe081086098c124a426fd
Component: engine
Prevents looping of startup errors such as containerd
not being found on the path.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
(cherry picked from commit ce0b0b72bcc58de82ba1e0e7127499d07f678122)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 85361af1f749517c8bdfd3d36b0df94a92e29b2b
Component: engine
… so moving it in test files for now.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
(cherry picked from commit 37ca5d6813)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 62aed95bc1
Component: cli
osusergo build tag is needed for better chances to have
a proper static binary when Go >= 1.11 is used, and is
harmless otherwise.
seccomp build tag is needed so the resulting dockerd binary
has seccomp support.
[v2: add libseccomp-dev installation]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 97596405f4300a7ae6eb7664f6c58785955dd8d3
Component: packaging
Should generate correct bits whether or not the suffix is present.
Working example:
❯ ./gen-deb-ver . 18.09.0-beta1
18.09.0~1.1.beta1 18.09.0-beta1
❯ ./gen-deb-ver . 18.09.0-ce-beta1
18.09.0~ce~1.1.beta1 18.09.0-ce-beta1
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 9de3f6fb573806abc2e546447c41552ff90731a4)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: cc336d0804076b31d6d059b164e9b4e3b3cec886
Component: packaging
Includes fixes related to upgrade cycles
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 83a20d53f14adcd2bcc9e932435aa66b6e056afa)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: befa3c61954d12f2413df56a384989d7ae2cb9b3
Component: packaging
A bit overdue
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 11643a30e0bdde4016afe42ab5f1f85e0fc83ecf)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: f38206fa477c59ec39b06b8ddde40be1f65c7cf0
Component: packaging
docker load doesn't support OCI based tar files so we have to make 2
different ones.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 696cdc009ca73b070d8877ee5cdd13c4b0e4a39b)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: ea811c67ee1e558ecec1b394fc591458b57f7ed1
Component: packaging
The official access point for the Q3 engine images will
be prefixed by store.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 3597d75281
Component: cli
This is set in a top-level makefile, but apparently
isn't getting passed all the way through as expected.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: a5ad5471419cb2a0935148f891c5ada8b13b4779
Component: packaging
Also cleans up the target in rpm and deb Makfiles because
the tar target already output a tar.
Signed-off-by: Jose Bigio <jose.bigio@docker.com>
(cherry picked from commit 8a4744f48aa13bea0b4e9174165be4add0a5a42f)
Signed-off-by: Jose Bigio <jose.bigio@docker.com>
Upstream-commit: 8e4a92dc667e2143b6d5707e19c5564b7da46ba1
Component: packaging
… this removes a whole lot of dependencies from people depending on docker/cli…
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
(cherry picked from commit 2d344b2f61)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: fc3dc8f058
Component: cli
… this is, for now, the only platform that is supported
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
(cherry picked from commit a3a955f204)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2a46a3d46c
Component: cli
This will make it easier for anybody to build with a specific version of a distro they want to build on.
They could lock into a specific sha of a docker image if they wanted, as an example.
Signed-off-by: corbin-coleman <corbin.coleman@docker.com>
Upstream-commit: 7afc35b95ac46bfef383788341ddb0b157af14eb
Component: packaging
This should accomodate the new versioning fix
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: be00095e1d0d5674665a63b6e5932178632feb0e
Component: packaging
Removes the need for the offline installer to install the shim process
and instead installs the shim process as part of the packaging.
May be easier in the future to just package the shim process on it's own
but that'll come after this 18.09 release
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit f8bd366d58f8bdf8a82b9a033353ca5bf4eda948)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 31d0cb047b98ab43f661bd026bdd63deef62543d
Component: packaging
This should test that
- all the messages produced are delivered (i.e. not lost)
- followLogs() exits
Loosely based on the test having the same name by Brian Goff, see
https://gist.github.com/cpuguy83/e538793de18c762608358ee0eaddc197
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit f845d76d047760c91dc0c7076aea840291fbdbc5)
Upstream-commit: 2a82480df9ad91593d59be4b5283917dbea2da39
Component: engine
When daemon.ContainerLogs() is called with options.follow=true
(as in "docker logs --follow"), the "loggerutils.followLogs()"
function never returns (even then the logs consumer is gone).
As a result, all the resources associated with it (including
an opened file descriptor for the log file being read, two FDs
for a pipe, and two FDs for inotify watch) are never released.
If this is repeated (such as by running "docker logs --follow"
and pressing Ctrl-C a few times), this results in DoS caused by
either hitting the limit of inotify watches, or the limit of
opened files. The only cure is daemon restart.
Apparently, what happens is:
1. logs producer (a container) is gone, calling (*LogWatcher).Close()
for all its readers (daemon/logger/jsonfilelog/jsonfilelog.go:175).
2. WatchClose() is properly handled by a dedicated goroutine in
followLogs(), cancelling the context.
3. Upon receiving the ctx.Done(), the code in followLogs()
(daemon/logger/loggerutils/logfile.go#L626-L638) keeps to
send messages _synchronously_ (which is OK for now).
4. Logs consumer is gone (Ctrl-C is pressed on a terminal running
"docker logs --follow"). Method (*LogWatcher).Close() is properly
called (see daemon/logs.go:114). Since it was called before and
due to to once.Do(), nothing happens (which is kinda good, as
otherwise it will panic on closing a closed channel).
5. A goroutine (see item 3 above) keeps sending log messages
synchronously to the logWatcher.Msg channel. Since the
channel reader is gone, the channel send operation blocks forever,
and resource cleanup set up in defer statements at the beginning
of followLogs() never happens.
Alas, the fix is somewhat complicated:
1. Distinguish between close from logs producer and logs consumer.
To that effect,
- yet another channel is added to LogWatcher();
- {Watch,}Close() are renamed to {Watch,}ProducerGone();
- {Watch,}ConsumerGone() are added;
*NOTE* that ProducerGone()/WatchProducerGone() pair is ONLY needed
in order to stop ConsumerLogs(follow=true) when a container is stopped;
otherwise we're not interested in it. In other words, we're only
using it in followLogs().
2. Code that was doing (logWatcher*).Close() is modified to either call
ProducerGone() or ConsumerGone(), depending on the context.
3. Code that was waiting for WatchClose() is modified to wait for
either ConsumerGone() or ProducerGone(), or both, depending on the
context.
4. followLogs() are modified accordingly:
- context cancellation is happening on WatchProducerGone(),
and once it's received the FileWatcher is closed and waitRead()
returns errDone on EOF (i.e. log rotation handling logic is disabled);
- due to this, code that was writing synchronously to logWatcher.Msg
can be and is removed as the code above it handles this case;
- function returns once ConsumerGone is received, freeing all the
resources -- this is the bugfix itself.
While at it,
1. Let's also remove the ctx usage to simplify the code a bit.
It was introduced by commit a69a59ffc7e3d ("Decouple removing the
fileWatcher from reading") in order to fix a bug. The bug was actually
a deadlock in fsnotify, and the fix was just a workaround. Since then
the fsnofify bug has been fixed, and a new fsnotify was vendored in.
For more details, please see
https://github.com/moby/moby/pull/27782#issuecomment-416794490
2. Since `(*filePoller).Close()` is fixed to remove all the files
being watched, there is no need to explicitly call
fileWatcher.Remove(name) anymore, so get rid of the extra code.
Should fix https://github.com/moby/moby/issues/37391
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 916eabd459fe707b5c4a86377d12e2ad1871b353)
Upstream-commit: 84a5b528aede5579861201e869870d10fc98c07c
Component: engine
This test case checks that followLogs() exits once the reader is gone.
Currently it does not (i.e. this test is supposed to fail) due to #37391.
[kolyshkin@: test case Brian Goff, changelog and all bugs are by me]
Source: https://gist.github.com/cpuguy83/e538793de18c762608358ee0eaddc197
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit d37a11bfbab83ab42b1160f116e863daac046192)
Upstream-commit: 511741735e0aa2fe68a66d99384c00d187d1a157
Component: engine
This code has many return statements, for some of them the
"end logs" or "end stream" message was not printed, giving
the impression that this "for" loop never ended.
Make sure that "begin logs" is to be followed by "end logs".
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 2e4c2a6bf9cb47fd07e42f9c043024ed3dbcd04d)
Upstream-commit: 2b8bc86679b7153bb4ace063a858637df0f16a2e
Component: engine
The code in Close() that removes the watches was not working,
because it first sets `w.closed = true` and then calls w.close(),
which starts with
```
if w.closed {
return errPollerClosed
}
```
Fix by setting w.closed only after calling w.remove() for all the
files being watched.
While at it, remove the duplicated `delete(w.watches, name)` code.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit fffa8958d00860b4e3563327a2cc6836a12d4ba9)
Upstream-commit: 4e2dbfa1af48191126b0910b9463bf94d8371886
Component: engine
There is no need to wait for up to 200ms in order to close
the file descriptor once the chClose is received.
This commit might reduce the chances for occasional "The process
cannot access the file because it is being used by another process"
error on Windows, where an opened file can't be removed.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit dfbb64ea7d042d5b2bb0c1c2b88e3682b7069b10)
Upstream-commit: 3a3bfcbf47e98212abfc9cfed860d9e99fc41cdc
Component: engine
In case of errors, the file descriptor is never closed. Fix it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 88bcf1573ca2eaffc15da346a1651a3749567554)
Upstream-commit: 7be43586af6824c1e55cb502d9d2bab45c9b4505
Component: engine
To include https://github.com/vbatts/tar-split/pull/48 which
fixes the issue of creating an image with >8GB file in it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 92e75439037d205c218208945c70cfb3633e87aa)
Upstream-commit: d7085abec2e445630bedd3e79782c5ec33f62682
Component: engine
Add a test case for creating a 8GB file inside a container.
Due to a bug in tar-split this was failing in Docker 18.06.
The file being created is sparse, so there's not much I/O
happening or disk space being used -- meaning the test is
fast and does not require a lot of disk space.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit b3165f5b2d7c1063c2a76a7ed5c2dd18f868276b)
Upstream-commit: fc1d808c44f77e3929c4eeba7066501890aecd4e
Component: engine
And update the associated packages that have also updated from
docker/docker vendor.conf.
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Upstream-commit: 4e6798794d
Component: cli
Fix default case causing the throttling to not be used.
Ensure that nil client condition is handled.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
(cherry picked from commit c3e32938430e03a316311f9e4fbdb743e492a07e)
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Upstream-commit: f121eccf29576ce5d4b8256a71a9d32ee688ff7d
Component: engine
This patch adds --filter, --keep-storage, --all and --force to builder prune.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit c806eb49c9)
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 9d43f1ed48
Component: cli
It's generally bad-form to generate two identically named package files
that are actually different, so we should name our packages based on
the distro and version, not just the distro.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
(cherry picked from commit 13bde85f0067a751f775d31919705645ecc0a6d7)
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 3c757b281bb623d9622c74143340fd892c8655ad
Component: packaging
Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229
(6bf0f408e4)
both the old, and new location are accepted by systemd 229 and up, so using the old location
to make them work for either version of systemd.
StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230
(f0367da7d1)
both the old, and new name are accepted by systemd 230 and up, so using the old name to make
this option work for either version of systemd.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2c2bfea5d009fb884f5c61b62e3c85bd88e7909e)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 5963decd1502e5012692a1f958bfdc5403f02920
Component: packaging
This adds support for reloading the docker daemon
(SIGHIUP) so that changes in '/etc/docker/daemon.json'
can be loaded at runtime by reloading the service
through systemd ('systemctl reload docker')
Before this change, systemd would output an error
that "reloading" is not supported for the docker
service;
systemctl reload docker
Failed to reload docker.service: Job type reload is not applicable for unit docker.service.
After this change, the docker daemon can be reloaded
through 'systemctl reload docker', which reloads
the configuration;
journalctl -f -u docker.service
May 02 03:49:20 testing systemd[1]: Reloading Docker Application Container Engine.
May 02 03:49:20 testing docker[28496]: time="2016-05-02T03:49:20.143964103-04:00" level=info msg="Got signal to reload configuration, reloading from: /etc/docker/daemon.json"
May 02 03:49:20 testing systemd[1]: Reloaded Docker Application Container Engine.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3e1b508e5f70b35869d9c8417d3b65a141820af9)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 127426fc693a882b670339391da5d12f08c1438d
Component: packaging
Change the kill mode to process so that systemd does not kill container
processes when the daemon is shutdown but only the docker daemon
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d736ae9da7401f582469fa7e943adde9f2163024)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 784c5936ec984b5525ac71eca3b7731e49eb6b37
Component: packaging
We need to add delegate yes to docker's service file so that it can
manage the cgroups of the processes that it launches without systemd
interfering with them and moving the processes after it is reloaded.
Delegate=
Turns on delegation of further resource control partitioning to
processes of the unit. For unprivileged services (i.e. those
using the User= setting), this allows processes to create a
subhierarchy beneath its control group path. For privileged
services and scopes, this ensures the processes will have all
control group controllers enabled.
This is the proper fix for issue moby/moby#20152
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e134e666a585b1f13e9e5e371dd93e5ce04a4b34)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 1120496ca0492d29731befd1460a934ac115e7f8
Component: packaging
Systemd sets a default of 512 tasks, which is far
too low to run many containers.
Note that TasksMax is only supported on systemd 226
and above.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 82fe96733f551d36018c3840cf21d813807e9b76)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: c42e4736e055c472284fd09ca05d405d31c4516a
Component: packaging
There is a not-insignificant performance overhead for all containers (if
containerd is a child of Docker, which is the current setup) if systemd
sets rlimits on the main Docker daemon process (because the limits
propogate to all children).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 68e15413dce8d6e8f428ac286641a9482d30aabc)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 1246dedcd55058a2856c7042509fb0badcefbf91
Component: packaging
set LimitCORE=infinity to ensure complete core creation,
allows extraction of as much information as possible.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit da69663b9ccd72d95ec60007ef707d82a35324fa)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: f93f4525173e3a32d315794344d44bae780de00e
Component: packaging
Includes fixes to the go command, linker, and the net/http, mime/multipart,
ld/macho, bytes, and strings packages. See the Go 1.10.4 milestone on the
issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.10.4
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 11b4630f4c73885123336b7917cc139c43cf413a)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 3920ea81ea2cab8e1f5f17fb16e133cfb87d9c98
Component: packaging
[18.09] backport: fix regression when filtering container names using a leading slash
Upstream-commit: b8a4fe5f8f58bccfa3763ca75c6465447c8ccd06
Component: engine
In case a volume is specified via Mounts API, and SELinux is enabled,
the following error happens on container start:
> $ docker volume create testvol
> $ docker run --rm --mount source=testvol,target=/tmp busybox true
> docker: Error response from daemon: error setting label on mount
> source '': no such file or directory.
The functionality to relabel the source of a local mount specified via
Mounts API was introduced in commit 5bbf5cc and later broken by commit
e4b6adc, which removed setting mp.Source field.
With the current data structures, the host dir is already available in
v.Mountpoint, so let's just use it.
Fixes: e4b6adc
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 4032b6778df39f53fda0e6e54f0256c9a3b1d618
Component: engine
Bump our kube dependencies to the latest patch
level for kube 1.11.
Signed-off-by: Marcus Martins <marcus@docker.com>
(cherry picked from commit c67e05796b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0efb62cab1
Component: cli
Commit 5c8da2e96738addd8a175e5b9a23b8c37d4a4dfe updated the filtering behavior
to match container-names without having to specify the leading slash.
This change caused a regression in situations where a regex was provided as
filter, using an explicit leading slash (`--filter name=^/mycontainername`).
This fix changes the filters to match containers both with, and without the
leading slash, effectively making the leading slash optional when filtering.
With this fix, filters with and without a leading slash produce the same result:
$ docker ps --filter name=^a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
21afd6362b0c busybox "sh" 2 minutes ago Up 2 minutes a2
56e53770e316 busybox "sh" 2 minutes ago Up 2 minutes a1
$ docker ps --filter name=^/a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
21afd6362b0c busybox "sh" 2 minutes ago Up 2 minutes a2
56e53770e316 busybox "sh" 3 minutes ago Up 3 minutes a1
$ docker ps --filter name=^b
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b69003b6a6fe busybox "sh" About a minute ago Up About a minute b1
$ docker ps --filter name=^/b
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b69003b6a6fe busybox "sh" 56 seconds ago Up 54 seconds b1
$ docker ps --filter name=/a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
21afd6362b0c busybox "sh" 3 minutes ago Up 3 minutes a2
56e53770e316 busybox "sh" 4 minutes ago Up 4 minutes a1
$ docker ps --filter name=a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
21afd6362b0c busybox "sh" 3 minutes ago Up 3 minutes a2
56e53770e316 busybox "sh" 4 minutes ago Up 4 minutes a1
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6f9b5ba810e9314ab9335490cd41316940a32b5e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5fa80da2d385520f2c3c91e8ce23d181e4f93097
Component: engine
While testing on older ubuntu images we discovered
we do depend on a newer version of libseccomp2.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: f2ceca98823548e401c6cca125b7057df28c4892
Component: packaging
If the system has a containerd-proxy managing the lifecycle
of the daemon, make sure the config is updated with the new image
on update.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: f96ddaedf7
Component: cli
Basic enterprise licenses and platform license keys will be signed with
two different keys in the upcoming release. This adds support for the
CLI to support both variants.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 0fb6bb35a4
Component: cli
During refactoring of the implementation PR progress reporting
was broken. This gets the progress reporting back in action.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 7f4c842e8a
Component: cli
This encodes the platform as "Docker Engine - Community"
for community engines
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 4d9fd7d17f3aab3463baf6030072f5789a003a74
Component: packaging
This should make it easier to change downstream builds
to change the scope.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 72406691244c12d5360889f8658bd4989b588364
Component: packaging
The other way was not working.
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
(cherry picked from commit f00df1bf1efa2ed1ea508ea7b1c291500441571d)
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Upstream-commit: a9b0387bb0f40bd7687d8653a0714724eb66f3b9
Component: packaging
Should rely on list in the daemon
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 605758124d0750c14d24bfb1ebcf77d102591f4a)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 6cc81c2e8c95127baac275750bc544452fec7799
Component: packaging
BuildKit can now be enabled without the daemon having
experimental features enabled.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 60c75fda67
Component: cli
Warnings are now generated by the daemon, and returned as
part of the /info API response.
If warnings are returned by the daemon; use those instead
of generating them locally.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 3c27ce21c9
Component: cli
The remote API allows full privilege escalation and is equivalent to
having root access on the host. Because of this, the API should never
be accessible through an insecure connection (TCP without TLS, or TCP
without TLS verification).
Although a warning is already logged on startup if the daemon uses an
insecure configuration, this warning is not very visible (unless someone
decides to read the logs).
This patch attempts to make insecure configuration more visible by sending
back warnings through the API (which will be printed when using `docker info`).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 547b993e07330f3e74cba935975fce05e8661381
Component: engine
This feature brings new attribute/option for swarm init command.
default-addr-pool will take string input which can be in below format.
"CIDR,CIDR,CIDR...:SUBNET-SIZE".
Signed-off-by: selansen <elango.siva@docker.com>
Upstream-commit: 587a94c935
Component: cli
When requesting information about the daemon's configuration through the `/info`
endpoint, missing features (or non-recommended settings) may have to be presented
to the user.
Detecting these situations, and printing warnings currently is handled by the
cli, which results in some complications:
- duplicated effort: each client has to re-implement detection and warnings.
- it's not possible to generate warnings for reasons outside of the information
returned in the `/info` response.
- cli-side detection has to be updated for new conditions. This means that an
older cli connecting to a new daemon may not print all warnings (due to
it not detecting the new conditions)
- some warnings (in particular, warnings about storage-drivers) depend on
driver-status (`DriverStatus`) information. The format of the information
returned in this field is not part of the API specification and can change
over time, resulting in cli-side detection no longer being functional.
This patch adds a new `Warnings` field to the `/info` response. This field is
to return warnings to be presented by the user.
Existing warnings that are currently handled by the CLI are copied to the daemon
as part of this patch; This change is backward-compatible with existing
clients; old client can continue to use the client-side warnings, whereas new
clients can skip client-side detection, and print warnings that are returned by
the daemon.
Example response with this patch applied;
```bash
curl --unix-socket /var/run/docker.sock http://localhost/info | jq .Warnings
```
```json
[
"WARNING: bridge-nf-call-iptables is disabled",
"WARNING: bridge-nf-call-ip6tables is disabled"
]
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a3d4238b9ce653d2863fbc93057ed4162a83221e
Component: engine
The daemon won't actually start without containerd
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 16037618282dc5076089c7b749cdd07376650f8d
Component: packaging
Blocks the execution of tasks during the Prepare phase until there
exists an IP address for every overlay network in use by the task. This
prevents a task from starting before the NetworkAttachment containing
the IP address has been sent down to the node.
Includes a basic test for the correct use case.
Signed-off-by: Drew Erny <drew.erny@docker.com>
Upstream-commit: 3c81dc3103d9c88cb333c80e0810f80ab80c374e
Component: engine
This feature allows user to specify list of subnets for global
default address pool. User can configure subnet list using
'swarm init' command. Daemon passes the information to swarmkit.
We validate the information in swarmkit, then store it in cluster
object. when IPAM init is called, we pass subnet list to IPAM driver.
Signed-off-by: selansen <elango.siva@docker.com>
Upstream-commit: f7ad95cab9cc7ba8925673a933028d53284c13f5
Component: engine
vndr buildkit to e8c7acc
vndr containerd to v1.2.0-beta.0
vndr console to 4d8a41f
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Upstream-commit: 7a73d112ff
Component: cli
This new collection of commands supports initializing a local
engine using containerd, updating that engine, and activating
the EE product
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: fd2f1b3b66
Component: cli
* Expose license status in Info
This wires up a new field in the Info payload that exposes the license.
For moby this is hardcoded to always report a community edition.
Downstream enterprise dockerd will have additional licensing logic wired
into this function to report details about the current license status.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
* Code review comments
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
* Add windows autogen support
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 896d1b1c61a48e2df1a7b4644ddde6ee97db6111
Component: engine
Makes it so that you can overwrite them better if you include this
makefile in another makefile...
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 446f8d31394ae35fc38187aa5e40627b33a95a27
Component: packaging
Was reporting 404's using the old url for some reason, might have to
refactor this in the future to just use a git clone...
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 763ad6aa7ac24c269dc1dfa223771671acffc8fc
Component: packaging
Allows us to switch out debian files and rpm specs on the fly and also
gives us an out to have ARCH specific dockerfiles if we need them for
RPM's. The same strategy can be used for DEB's if need be
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: b72dc2edb8329151bfe2df6989fb329f41e720fb
Component: packaging
This driver uses protobuf to store log messages and has better defaults
for log file handling (e.g. compression and file rotation enabled by
default).
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: a351b38e7217af059eb2f8fc3dba14dc03836a45
Component: engine
This patch adds a new builder subcommand, allowing to add more builder-related
commands in the future. Unfortunately `build` expects an argument so could not
be used as a subcommand.
This also implements `docker builder prune`, which is needed to prune the builder
cache manually without having to call `docker system prune`.
Today when relying on the legacy builder, users are able to prune dangling images
(used as build cache) by running `docker image prune`. This patch allows the
same usecase with buildkit.
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: f597f2d026
Component: cli
This patch implements `docker build --secret id=mysecret,src=/secret/file`
for buildkit frontends that request the mysecret secret.
It is currently implemented in the tonistiigi/dockerfile:secrets20180808
frontend via RUN --mount=type=secret,id=mysecret
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: c4c4825591
Component: cli
Signed-off-by: John Howard <jhoward@microsoft.com>
Fixes#36764
@johnstep PTAL. @jterry75 FYI.
There are two commits in this PR. The first ensure that errors are actually returned to the caller - it was being thrown away.
The second commit changes the LCOW driver to map, on a per service VM basis, "long" container paths such as `/tmp/c8fa0ae1b348f505df2707060f6a49e63280d71b83b7936935c827e2e9bde16d` to much shorter paths, based on a per-service VM counter, so something more like /tmp/d3. This means that the root cause of the failure where the mount call to create the overlay was failing due to command line length becomes something much shorter such as below.
`mount -t overlay overlay -olowerdir=/tmp/d3:/tmp/d4:/tmp/d5:/tmp/d6:/tmp/d7:/tmp/d8:/tmp/d9:/tmp/d10:/tmp/d11:/tmp/d12:/tmp/d13:/tmp/d14:/tmp/d15:/tmp/d16:/tmp/d17:/tmp/d18:/tmp/d19:/tmp/d20:/tmp/d21:/tmp/d22:/tmp/d23:/tmp/d24:/tmp/d25:/tmp/d26:/tmp/d27:/tmp/d28:/tmp/d29:/tmp/d30:/tmp/d31:/tmp/d32:/tmp/d33:/tmp/d34:/tmp/d35:/tmp/d36:/tmp/d37:/tmp/d38:/tmp/d39:/tmp/d40:/tmp/d41:/tmp/d42:/tmp/d43:/tmp/d44:/tmp/d45:/tmp/d46:/tmp/d47:/tmp/d48:/tmp/d49:/tmp/d50:/tmp/d51:/tmp/d52:/tmp/d53:/tmp/d54:/tmp/d55:/tmp/d56:/tmp/d57:/tmp/d58:/tmp/d59:/tmp/d60:/tmp/d61:/tmp/d62,upperdir=/tmp/d2/upper,workdir=/tmp/d2/work /tmp/c8fa0ae1b348f505df2707060f6a49e63280d71b83b7936935c827e2e9bde16d-mount`
For those worrying about overflow (which I'm sure @thaJeztah will mention...): It's safe to use a counter here as SVMs are disposable in the default configuration. The exception is when running the daemon in unsafe LCOW "global" mode (ie `--storage-opt lcow.globalmode=1`) where the SVMs aren't disposed of, but a single one is reused. However, to overflow the command line length, it would require several hundred-thousand trillion (conservative, I should sit down and work it out accurately if I get -really- bored) of SCSI hot-add operations, and even to hit that would be hard as just running containers normally uses the VPMEM path for the containers UVM, not to the global SVM on SCSI. It gets incremented by one per build step (commit more accurately) as a general rule. Hence it would be necessary to have to be doing automated builds without restarting the daemon for literally years on end in unsafe mode. 😇
Note that in reality, the previous limit of ~47 layers before hitting the command line length limit is close to what is possible in the platform, at least as of RS5/Windows Server 2019 where, in the HCS v1 schema, a single SCSI controller is used, and that can only support 64 disks per controller per the Hyper-V VDEV. And remember we have one slot taken up for the SVMs scratch, and another for the containers scratch when committing a layer. So the best you can architecturally get on the platform is around the following (it's also different by 1 depending on whether in unsafe or default mode)
```
PS E:\docker\build\36764\short> docker build --no-cache .
Sending build context to Docker daemon 2.048kB
Step 1/4 : FROM alpine as first
---> 11cd0b38bc3c
Step 2/4 : RUN echo test > /test
---> Running in 8ddfe20e5bfb
Removing intermediate container 8ddfe20e5bfb
---> b0103a00b1c9
Step 3/4 : FROM alpine
---> 11cd0b38bc3c
Step 4/4 : COPY --from=first /test /test
---> 54bfae391eba
Successfully built 54bfae391eba
PS E:\docker\build\36764\short> cd ..
PS E:\docker\build\36764> docker build --no-cache .
Sending build context to Docker daemon 4.689MB
Step 1/61 : FROM alpine as first
---> 11cd0b38bc3c
Step 2/61 : RUN echo test > /test
---> Running in 02597ff870db
Removing intermediate container 02597ff870db
---> 3096de6fc454
Step 3/61 : RUN echo test > /test
---> Running in 9a8110f4ff19
Removing intermediate container 9a8110f4ff19
---> 7691808cf28e
Step 4/61 : RUN echo test > /test
---> Running in 9afb8f51510b
Removing intermediate container 9afb8f51510b
---> e42a0df2bb1c
Step 5/61 : RUN echo test > /test
---> Running in fe977ed6804e
Removing intermediate container fe977ed6804e
---> 55850c9b0479
Step 6/61 : RUN echo test > /test
---> Running in be65cbfad172
Removing intermediate container be65cbfad172
---> 0cf8acba70f0
Step 7/61 : RUN echo test > /test
---> Running in fd5b0907b6a9
Removing intermediate container fd5b0907b6a9
---> 257a4493d85d
Step 8/61 : RUN echo test > /test
---> Running in f7ca0ffd9076
Removing intermediate container f7ca0ffd9076
---> 3baa6f4fa2d5
Step 9/61 : RUN echo test > /test
---> Running in 5146814d4727
Removing intermediate container 5146814d4727
---> 485b9d5cf228
Step 10/61 : RUN echo test > /test
---> Running in a090eec1b743
Removing intermediate container a090eec1b743
---> a7eb10155b51
Step 11/61 : RUN echo test > /test
---> Running in 942660b288df
Removing intermediate container 942660b288df
---> 9d286a1e2133
Step 12/61 : RUN echo test > /test
---> Running in c3d369aa91df
Removing intermediate container c3d369aa91df
---> f78be4788992
Step 13/61 : RUN echo test > /test
---> Running in a03c3ac6888f
Removing intermediate container a03c3ac6888f
---> 6504363f61ab
Step 14/61 : RUN echo test > /test
---> Running in 0c3c2fca3f90
Removing intermediate container 0c3c2fca3f90
---> fe3448b8bb29
Step 15/61 : RUN echo test > /test
---> Running in 828d51c76d3b
Removing intermediate container 828d51c76d3b
---> 870684e3aea0
Step 16/61 : RUN echo test > /test
---> Running in 59a2f7c5f3ad
Removing intermediate container 59a2f7c5f3ad
---> cf84556ca5c0
Step 17/61 : RUN echo test > /test
---> Running in bfb4e088eeb3
Removing intermediate container bfb4e088eeb3
---> 9c8f9f652cef
Step 18/61 : RUN echo test > /test
---> Running in f1b88bb5a2d7
Removing intermediate container f1b88bb5a2d7
---> a6233ad21648
Step 19/61 : RUN echo test > /test
---> Running in 45f70577d709
Removing intermediate container 45f70577d709
---> 1b5cc52d370d
Step 20/61 : RUN echo test > /test
---> Running in 2ce231d5043d
Removing intermediate container 2ce231d5043d
---> 4a0e17cbebaa
Step 21/61 : RUN echo test > /test
---> Running in 52e4b0928f1f
Removing intermediate container 52e4b0928f1f
---> 99b50e989bcb
Step 22/61 : RUN echo test > /test
---> Running in f7ba3da7460d
Removing intermediate container f7ba3da7460d
---> bfa3cad88285
Step 23/61 : RUN echo test > /test
---> Running in 60180bf60f88
Removing intermediate container 60180bf60f88
---> fe7271988bcb
Step 24/61 : RUN echo test > /test
---> Running in 20324d396531
Removing intermediate container 20324d396531
---> e930bc039128
Step 25/61 : RUN echo test > /test
---> Running in b3ac70fd4404
Removing intermediate container b3ac70fd4404
---> 39d0a11ea6d8
Step 26/61 : RUN echo test > /test
---> Running in 0193267d3787
Removing intermediate container 0193267d3787
---> 8062d7aab0a5
Step 27/61 : RUN echo test > /test
---> Running in f41f45fb7985
Removing intermediate container f41f45fb7985
---> 1f5f18f2315b
Step 28/61 : RUN echo test > /test
---> Running in 90dd09c63d6e
Removing intermediate container 90dd09c63d6e
---> 02f0a1141f11
Step 29/61 : RUN echo test > /test
---> Running in c557e5386e0a
Removing intermediate container c557e5386e0a
---> dbcd6fb1f6f4
Step 30/61 : RUN echo test > /test
---> Running in 65369385d855
Removing intermediate container 65369385d855
---> e6e9058a0650
Step 31/61 : RUN echo test > /test
---> Running in d861fcc388fd
Removing intermediate container d861fcc388fd
---> 6e4c2c0f741f
Step 32/61 : RUN echo test > /test
---> Running in 1483962b7e1c
Removing intermediate container 1483962b7e1c
---> cf8f142aa055
Step 33/61 : RUN echo test > /test
---> Running in 5868934816c1
Removing intermediate container 5868934816c1
---> d5ff87cdc204
Step 34/61 : RUN echo test > /test
---> Running in e057f3201f3a
Removing intermediate container e057f3201f3a
---> b4031b7ab4ac
Step 35/61 : RUN echo test > /test
---> Running in 22b769b9079c
Removing intermediate container 22b769b9079c
---> 019d898510b6
Step 36/61 : RUN echo test > /test
---> Running in f1d364ef4ff8
Removing intermediate container f1d364ef4ff8
---> 9525cafdf04d
Step 37/61 : RUN echo test > /test
---> Running in 5bf505b8bdcc
Removing intermediate container 5bf505b8bdcc
---> cd5002b33bfd
Step 38/61 : RUN echo test > /test
---> Running in be24a921945c
Removing intermediate container be24a921945c
---> 8675db44d1b7
Step 39/61 : RUN echo test > /test
---> Running in 352dc6beef3d
Removing intermediate container 352dc6beef3d
---> 0ab0ece43c71
Step 40/61 : RUN echo test > /test
---> Running in eebde33e5d9b
Removing intermediate container eebde33e5d9b
---> 46ca4b0dfc03
Step 41/61 : RUN echo test > /test
---> Running in f920313a1e85
Removing intermediate container f920313a1e85
---> 7f3888414d58
Step 42/61 : RUN echo test > /test
---> Running in 10e2f4dc1ac7
Removing intermediate container 10e2f4dc1ac7
---> 14db9e15f2dc
Step 43/61 : RUN echo test > /test
---> Running in c849d6e89aa5
Removing intermediate container c849d6e89aa5
---> fdb770494dd6
Step 44/61 : RUN echo test > /test
---> Running in 419d1a8353db
Removing intermediate container 419d1a8353db
---> d12e9cf078be
Step 45/61 : RUN echo test > /test
---> Running in 0f1805263e4c
Removing intermediate container 0f1805263e4c
---> cd005e7b08a4
Step 46/61 : RUN echo test > /test
---> Running in 5bde05b46441
Removing intermediate container 5bde05b46441
---> 05aa426a3d4a
Step 47/61 : RUN echo test > /test
---> Running in 01ebc84bd1bc
Removing intermediate container 01ebc84bd1bc
---> 35d371fa4342
Step 48/61 : RUN echo test > /test
---> Running in 49f6c2f51dd4
Removing intermediate container 49f6c2f51dd4
---> 1090b5dfa130
Step 49/61 : RUN echo test > /test
---> Running in f8a9089cd725
Removing intermediate container f8a9089cd725
---> b2d0eec0716d
Step 50/61 : RUN echo test > /test
---> Running in a1697a0b2db0
Removing intermediate container a1697a0b2db0
---> 10d96ac8f497
Step 51/61 : RUN echo test > /test
---> Running in 33a2332c06eb
Removing intermediate container 33a2332c06eb
---> ba5bf5609c1c
Step 52/61 : RUN echo test > /test
---> Running in e8920392be0d
Removing intermediate container e8920392be0d
---> 5b3a95685c7e
Step 53/61 : RUN echo test > /test
---> Running in 4b9298587c65
Removing intermediate container 4b9298587c65
---> d4961a349141
Step 54/61 : RUN echo test > /test
---> Running in 8a0c960c2ba1
Removing intermediate container 8a0c960c2ba1
---> b413197fcfa2
Step 55/61 : RUN echo test > /test
---> Running in 536ee3b9596b
Removing intermediate container 536ee3b9596b
---> fc16b69b224a
Step 56/61 : RUN echo test > /test
---> Running in 8b817b8d7b59
Removing intermediate container 8b817b8d7b59
---> 2f0896400ff9
Step 57/61 : RUN echo test > /test
---> Running in ab0ed79ec3d4
Removing intermediate container ab0ed79ec3d4
---> b4fb420e736c
Step 58/61 : RUN echo test > /test
---> Running in 8548d7eead1f
Removing intermediate container 8548d7eead1f
---> 745103fd5a38
Step 59/61 : RUN echo test > /test
---> Running in 1980559ad5d6
Removing intermediate container 1980559ad5d6
---> 08c1c74a5618
Step 60/61 : FROM alpine
---> 11cd0b38bc3c
Step 61/61 : COPY --from=first /test /test
---> 67f053c66c27
Successfully built 67f053c66c27
PS E:\docker\build\36764>
```
Note also that subsequent error messages once you go beyond current platform limitations kind of suck (such as insufficient resources with a bunch of spew which is incomprehensible to most) and we could do better to detect this earlier in the daemon. That'll be for a (reasonably low-priority) follow-up though as and when I have time. Theoretically we *may*, if the platform doesn't require additional changes for RS5, be able to have bigger platform limits using the v2 schema with up to 127 VPMem devices, and the possibility to have multiple SCSI controllers per SVM/UVM. However, currently LCOW is using HCS v1 schema calls, and there's no plans to rewrite the graphdriver/libcontainerd components outside of the moving LCOW fully over to the containerd runtime/snapshotter using HCS v2 schema, which is still some time off fruition.
PS OK, while waiting for a full run to complete, I did get bored. Turns out it won't overflow line length as max(uint64) is 18446744073709551616 which would still be short enough at 127 layers, double the current platform limit. And I could always change it to hex or base36 to make it even shorter, or remove the 'd' from /tmp/dN. IOW, pretty sure no-one is going to hit the limit even if we could get the platform to 256 which is the current Hyper-V SCSI limit per VM (4x64), although PMEM at 127 would be the next immediate limit.
Upstream-commit: f586fe5637bbadd919ce4a126c6d95e4f3b1523b
Component: engine
Get's rid of architecture specific dockerfiles (yay manifest lists),
also follows very closely to what the RPM makefile does with the
sources.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: ab1ba336ad1720a166e8ed4469c616851d89a8d1
Component: packaging
Old versions of things on CentOS 7 strike again!
infinity is not a thing for TimeoutSec on systemd < 229
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 86dafe5391e2ef298248536cd038e27230a4df63
Component: packaging
Why didn't we do this the whole time? `¯\_(ツ)_/¯`
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: df2a0d6fc5a59a7a8b352689238eddf5a46d3ed4
Component: packaging
Modify hack/make.ps1 to use the version value used in
"FROM golang" statement.
While at it:
1. Make search expression a bit more strict (use ^ to match at BOL only).
2. Simplify by removing Get-Contents as Select-String can read files.
After this, ENV GO_VERSION can be removed from Dockerfile.
Unfortunately it can't be done in one commit as Windows CI
fails (presumably because Dockerfile is being modified in
place).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 6a2851332d5f86bb61a69d277f663f369377037b
Component: engine
rpmlint checks can be re-added later
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: c245ce12c1e0a092ef21094b287b8cb94a51ee1e
Component: packaging
Also split off containerd.mk into it's own thing since most of this
stuff will be re-used with debian packaging anyways.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 5513629a1b4868d82a41249eac634edbff51e315
Component: packaging
Splits out the docker-ce package and docker-ce-cli package into their
own things.
Still TODO: need to have a cleanup on the dependencies for the
Dockerfiles
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 2a27fb1dad0f91c25acb885c117563a33f6c5e88
Component: packaging
1. After running d.Cmd(), in case an error is returned, it makes sense
to print command output, as its stderr may contain a clue about what
went wrong. This is by no means complete, just as far as I could go.
2. In case the comment in c.Assert is a constant string, it's better
to provide it as a comment which will be printed.
3. An arbitrary string should not be passed on to a function expecting
%-style formatting. Use %s to fix this.
4. Print the output string before transformation, not after.
5. Unify the output format (drop "out:" prefix").
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: ac038eab298eeecab55a62440de302e3bb3f4889
Component: engine
It is wrong to pass an arbitrary string to a function expecting
%-style formatting. One solution would be to replace any % with %%,
but it's easier to just do what this patch does.
Generated with:
for f in $(git grep -l 'check.Commentf(out)'); do \
sed -i -e 's/check\.Commentf(out)/check.Commentf("%s", out)/g' $f; \
done
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 83363fb2d4d97d952a0052d079faa8ae39aa20b6
Component: engine
This implements chown support on Windows. Built-in accounts as well
as accounts included in the SAM database of the container are supported.
NOTE: IDPair is now named Identity and IDMappings is now named
IdentityMapping.
The following are valid examples:
ADD --chown=Guest . <some directory>
COPY --chown=Administrator . <some directory>
COPY --chown=Guests . <some directory>
COPY --chown=ContainerUser . <some directory>
On Windows an owner is only granted the permission to read the security
descriptor and read/write the discretionary access control list. This
fix also grants read/write and execute permissions to the owner.
Signed-off-by: Salahuddin Khan <salah@docker.com>
Upstream-commit: 763d8392612942ff5c32a35f8bdafd7ae93d3321
Component: engine
Also moved package deps into spec file BuildRequires. Had to hard-code
Epoch in spec file in order to realize deps with yum-builddep.
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Upstream-commit: 07f46e8e96692c511c04f29fbc78c2aa9592e479
Component: packaging
Also moved package deps into spec file BuildRequires. Had to hard-code
Epoch in spec file in order to realize deps with yum-builddep.
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Upstream-commit: b09bb012dd220e822aa6e90ef20d9cb83ccada8b
Component: packaging
Also moved package deps into spec file BuildRequires. Had to hard-code
Epoch in spec file in order to realize deps with yum-builddep.
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Upstream-commit: 784a53cec5061e6a718acc3a7104c61215c15078
Component: packaging
Looks like yum-builddep will only work if rpmlint passed on the spec
file and the spec file errors out on Epoch not being hardcoded.
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Upstream-commit: 42b49f6556022ecba47801f4639ddbc7864c9fe0
Component: packaging
This makes it so consumers of `LogFile` should pass in how to get an
io.Reader to the requested number of lines to tail.
This is also much more efficient when tailing a large number of lines.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 94a10150f64a24793216f5324a34e62be3f31a2d
Component: engine
This fixes a bug when the build results in an error and instead of sending
the logs of the container to the client, it signals a cancelation.
The context should not be wired to the status request, as only the
build requests need to be canceled.
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: f1ef16c736b477be17f6bc1bce18d0958a87d840
Component: engine
This changes the experimental --console flag to --progress following
feedback indicating avoidable confusion.
In addition to naming changes, the help output now has an additional
clarification, specifically: container output during builds are only
shown when progress output is set to plain. Not mentioning this was also
a big cause of confusion.
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: faeb8bb571
Component: cli
Relevant changes;
- swarmkit #2593 agent: return error when failing to apply network key
- swarmkit #2645 Replace deprecated grpc functions
- swarmkit #2720 Test if error is nil before to log it
- swarmkit #2712 [orchestrator] Fix task sorting
- swarmkit #2677 [manager/orchestrator/reaper] Fix the condition used for skipping over running tasks
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 660fa129c0ea09c1aa1770b5f2d26ec762213484
Component: engine
This splits out the CLI into a discrete package and removes
the engine from the engine package. Instead the engine
is initialized via a post-inst script using the new CLI UX.
Upstream-commit: 662e248f680eb49a9951a8b34125506b8f82dfed
Component: packaging
… as it is possible to do it when interpolating. It also fixes when
there is 2 variables on the same *value* (in the composefile, on the
same line)
Finaly, renaming the default, used in cli, pattern to `defaultPattern`
to not be shadowed unintentionally.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 4c87725c35
Component: cli
e.g. docker -H ssh://me@server
The `docker` CLI also needs to be installed on the remote host to
provide `docker system dial-stdio`, which proxies the daemon socket to stdio.
Please refer to docs/reference/commandline/dockerd.md .
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Upstream-commit: 6f61cf053a
Component: cli
This fix migrates some ipc container tests from integration-cli
to integration test.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 80c92c9b692b79273200b6e37f14a6d4e984ab8d
Component: engine
Changed unexpected Unicode character 0x1fbf GREEK PSILI that was standing in as an imposter for an apostrophe: an _impostrophe_.
Signed-off-by: Chad Faragher <wyckster@hotmail.com>
Upstream-commit: b59823c784
Component: cli
For the Q3 release the names will be refined slightly, and "ce" will no
longer be used - it's now just the docker engine, with "community" or
"enterprise" at the end if disambiguation is needed.
Upstream-commit: e69e4ea4997e468d20d93247ba19cd5e86daf24f
Component: packaging
It allows to get easily all the variables defined in a
composefile (the `map[string]interface{}` representation that
`loader.ParseYAML` returns at least) and their default value too.
This commit also does some small function extract on substitution
funcs to reduce a tiny bit duplication.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: afb87e42f2
Component: cli
fix the race condition in the integration test TestRunContainerWithBridgeNone
Upstream-commit: 9149ef67be8ac945d68fafb16a1aa4ccb2f72249
Component: engine
PersistentPreRunE needs to be called within the help function to initialize all the flags (notably the orchestrator flag)
Add an e2e test as regression test
Signed-off-by: Silvin Lubecki <silvin.lubecki@docker.com>
Upstream-commit: 21cce52b30
Component: cli
* Add the `help` target to document make targets when building using a
container
* Remove the `watch` target (filewatcher was removed with c0588a9c) from
docker.Makefile and Makefile
Signed-off-by: Marco Vedovati <mvedovati@suse.com>
Upstream-commit: 760ca04709
Component: cli
This should make it easier for people to write custom composefile
parser without duplicating too much code. It takes the default
transformers and any additional number of transformer for any
types. That way it's possible to transform a `cli/compose` map into a
custom type that would use some of `cli/compose` types and its own.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 0246bc1b3b
Component: cli
Make all dynbinary builds be position-independent (this adds both
security benefits and can help with flaky builds on POWER
architectures).
Signed-off-by: Aleksa Sarai <asarai@suse.de>
Upstream-commit: 164e812b7a
Component: cli
This flag was added in Docker 17.06, API version 1.31 through
moby@8dc8cd4719f165c01c98e7d3ce1d6cea6a8f60b8, but didn't add
API-version annotations.
This patch adds the missing annotations to hide this flag if
the CLI is connected to an older version of the daemon that
doesn't support that API.
Before this patch:
DOCKER_API_VERSION=1.30 docker swarm init --help | grep data-path-addr
--data-path-addr string Address or interface to use for data path traffic (format: <ip|interface>)
DOCKER_API_VERSION=1.31 docker swarm init --help | grep data-path-addr
--data-path-addr string Address or interface to use for data path traffic (format: <ip|interface>)
With this patch applied:
DOCKER_API_VERSION=1.30 docker swarm init --help | grep data-path-addr
# (no result)
DOCKER_API_VERSION=1.31 docker swarm init --help | grep data-path-addr
--data-path-addr string Address or interface to use for data path traffic (format: <ip|interface>)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 91b1ad9d2b
Component: cli
Else it is harder to understand where we've failed in StartWithLogFile
We have an errror:
[d42ce729d0b06] waiting for daemon to start
[d42ce729d0b06] waiting for daemon to start
[d42ce729d0b06] waiting for daemon to start
[d42ce729d0b06] waiting for daemon to start
[d42ce729d0b06] waiting for daemon to start
[d42ce729d0b06] waiting for daemon to start
[d42ce729d0b06] waiting for daemon to start
[d42ce729d0b06] waiting for daemon to start
[d42ce729d0b06] waiting for daemon to start
[d42ce729d0b06] waiting for daemon to start
[d42ce729d0b06] waiting for daemon to start
[d42ce729d0b06] waiting for daemon to start
docker_cli_userns_test.go:27:
s.d.StartWithBusybox(c, "--userns-remap", "default")
/go/src/github.com/docker/docker/internal/test/daemon/daemon.go:203:
t.Fatalf("Error starting daemon with arguments: %v", args)
... Error: Error starting daemon with arguments: [--userns-remap default]
[d42ce729d0b06] exiting daemon
Likely it is "[..] Daemon exited during startup" case, but these error
message is lost
Also we lose the error returned by waiting daemon so also add it
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Upstream-commit: 2ed53943f37c26017260759b12d32cfdd75be152
Component: engine
Makes code less confusing.
Otherwise it looks like an error (typo of "==" instead "!=").
Signed-off-by: Iskander Sharipov <quasilyte@gmail.com>
Upstream-commit: cda9d5f7f0c4a155d1ae9e06c203fcb89600ff2a
Component: engine
WithDialContext() allows specifying custom dialer for hijacking and supposed to
replace WithDialer().
WithDialer() is also updated to use WithDialContext().
client.Dialer() returns the dialer configured with WithDialContext().
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Upstream-commit: edac92409a3b1d0cfb7f5c0e2d10b3bb71f27245
Component: engine
- Use `Contains` instead of `Include`
- Use `ToJSON` instead of `ToParam`
- Remove usage of `ParseFlag` as it is deprecated too
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 55edeb497a
Component: cli
The message was mistakenly replaced with a completely different one.
Fixes: cf2ea7613899477
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 2c2834ba4d59bb1bbb3efee07c8d3f09445c66b7
Component: engine
migrate service update integration tests from integration-cli to integration/service package
Upstream-commit: 3f4410163b8bfbfb67fa2730107235f16c6bbe59
Component: engine
Handle the case of systemd-resolved, and if in place
use a different resolv.conf source.
Set appropriately the option on libnetwork.
Move unix specific code to container_operation_unix
Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
Upstream-commit: e353e7e3f0ce8eceeff657393cba2876375403fa
Component: engine
Even though those fields are not supported by `docker stack deploy`
they are defined in versions `3.x` of compose schema, so the `compose`
package should be able to marshal/unmarshal them.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: d13e2df65b
Component: cli
Previously, dockerd would always ask containerd to pass --leave-running
to runc/runsc, ignoring the exit boolean value. Hence, even `docker
checkpoint create --leave-running=false ...` would not stop the
container.
Signed-off-by: Brielle Broder <bbroder@google.com>
Upstream-commit: db621eb7ee6a837f6879959ac56c67fa2b30ae1d
Component: engine
Disable cri plugin by default in containerd and
allows an option to enable the plugin. This only
has an effect on containerd when supervised by
dockerd. When containerd is managed outside of
dockerd, the configuration is not effected.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: 8fb5f4d5c9b4933be31bf5371d65a95edb037261
Component: engine
1. Sets the LinuxMetadata flag by default on Windows LCOW v1
MappedDirectories.
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
Upstream-commit: e93077bcdf77be2e7e3aec9f9686dace9e18c850
Component: engine
based on golang's pseudo-version: https://groups.google.com/forum/#!topic/golang-dev/a5PqQuBljF4
> using a "pseudo-version" of the form v0.0.0-yyyymmddhhmmss-abcdefabcdef,
> where the time is the commit time in UTC and the final suffix is the prefix
> of the commit hash. The time portion ensures that two pseudo-versions can
> be compared to determine which happened later, the commit hash identifes
> the underlying commit, and the v0.0.0- prefix identifies the pseudo-version
> as a pre-release before version v0.0.0, so that the go command prefers any
> tagged release over any pseudo-version.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: d2baf8f4891e82117bce354e0b0761258a0eafc9
Component: packaging
Go 1.11 includes a fix to os/user to be working in a static binary
(fixing https://github.com/golang/go/issues/23265). The fix requires
`osusergo` build tag to be set for static binaries, which is what
this commit adds (also for containerd).
[v2: sort build tags alphabetically]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 70cdb1c66429582ecfdc5abed67189dd90ab7572
Component: engine
Since go-1.11beta1 archive/tar, tar headers with Typeflag == TypeRegA
(numeric 0) (which is the default unless explicitly initialized) are
modified to have Typeflag set to either tar.TypeReg (character value
'0', not numeric 0) or tar.TypeDir (character value '5') [1].
This results in different Typeflag value in the resulting header,
leading to a different Checksum, and causing the following test
case errors:
> 12:09:14 --- FAIL: TestTarSums (0.05s)
> 12:09:14 tarsum_test.go:393: expecting
> [tarsum+sha256:8bf12d7e67c51ee2e8306cba569398b1b9f419969521a12ffb9d8875e8836738],
> but got
> [tarsum+sha256:75258b2c5dcd9adfe24ce71eeca5fc5019c7e669912f15703ede92b1a60cb11f]
> ... (etc.)
All the other code explicitly sets the Typeflag field, but this test
case is not, causing the incompatibility with Go 1.11. Therefore,
the fix is to set TypeReg explicitly, and change the expected checksums
in test cases).
Alternatively, we can vendor archive/tar again (for the 100th time),
but given that the issue is limited to the particular test case it
does not make sense.
This fixes the test for all Go versions.
[1] https://go-review.googlesource.com/c/go/+/85656
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 05cbe23db9836476677e7071d21ad53bf93dc2e7
Component: engine
Set BuildKit's ExportedProduct variable to show useful errors in the future
Upstream-commit: 9ebed53c5431a7c8296107869f1b3feabee9e8f3
Component: engine
Clarified ambiguous error message
Update kubernetes/cli.go
Infromed user of why the error was caused when file is not there
Signed-off-by: Justyn Temme <justyntemme@gmail.com>
Upstream-commit: bdd58a4096
Component: cli
This test occassionally fails on s390x and Power;
03:16:04 --- FAIL: TestExternalGraphDriver/pull (1.08s)
03:16:04 external_test.go:402: assertion failed: error is not nil: Error: No such image: busybox:latest
Most likely these failures are caused due to Docker Hub updating
the busybox:latest image, but not all architectures yet being
available.
Instead of using `:latest`, pull an image by digest, so that
the test doesn't depend on Docker Hub having all architectures
available for `:latest`.
I selected the same digest as is currently used as "frozen image"
in the Dockerfile.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 352db26d5fd896b681046b1119f07e0ce1ed45b8
Component: engine
This introduces a PRODUCT environment variable that is used to set a constant
at dockerversion.ProductName.
That is then used to set BuildKit's ExportedProduct variable in order to show
useful error messages to users when a certain version of the product doesn't
support a BuildKit feature.
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 195919d9d645aa4ab5680a2331c57ff33eb9e5d9
Component: engine
moved integration tests from docker_cli_config_create_test.go to integration/config
Upstream-commit: a5495f289aafafac8a01c62f2a9cb44856658ece
Component: engine
Updates cri version to 1.0.4, to add `max-container-log-line-size`
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9e773a12fb1cc5da7bec13d46fe04673a4593632
Component: engine
* Regex name filters were display undesired behavior due to
names containing the trailing slash when being compared
* Adjusted filterByNameIDMatches and includeContainerInList to
strip the slash prefix before doing name comparisons
* Added test case and helper functions for the test to list_test
* Force failed tests during development to ensure there were
no false positives
Signed-off-by: Chris White <me@cwprogram.com>
Upstream-commit: 5c8da2e96738addd8a175e5b9a23b8c37d4a4dfe
Component: engine
Instead of waiting for the DNS to fail, try to access
a specific external IP and verify that 100% of the pakcets
are being lost.
Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
Upstream-commit: a2bb2144b3e0d49ac615976bfac462a307d85f0e
Component: engine
Should fix
```
api/types/volume/volume_create.go
Line 10: warning: comment on exported type VolumeCreateBody should be of the form "VolumeCreateBody ..." (with optional leading article) (golint)
api/types/volume/volume_list.go
Line 12: warning: comment on exported type VolumeListOKBody should be of the form "VolumeListOKBody ..." (with optional leading article) (golint)
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: bd06a5ea4df919ff323510fba10801b5673a3af6
Component: engine
17.06 has reached EOL a long time ago; let's use a current
version in CI as well :D
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e2a56c47da
Component: cli
- remove some hints that are no longer needed
- added a nolint: unparam for removeSingleSigner() (return bool is only used in tests)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9847e96765
Component: cli
alexkohler/nakedret is now installed by default with gometalinter,
so it's no longer needed to install this manually
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f3811e865e
Component: cli
When go-1.11beta1 is used for building, the following error is
reported:
> 14:56:20 daemon\graphdriver\lcow\lcow.go:236: Debugf format %s reads
> arg #2, but call has 1 arg
While fixing this, let's also fix a few other things in this
very function (startServiceVMIfNotRunning):
1. Do not use fmt.Printf when not required.
2. Use `title` whenever possible.
3. Don't add `id` to messages as `title` already has it.
4. Remove duplicated colons.
5. Try to unify style of messages.
6. s/startservicevmifnotrunning/startServiceVMIfNotRunning/
...
In general, logging/debugging here is a mess and requires much more
love than I can give it at the moment. Areas for improvement:
1. Add a global var logger = logrus.WithField("storage-driver", "lcow")
and use it everywhere else in the code.
2. Use logger.WithField("id", id) whenever possible (same for "context"
and other similar fields).
3. Revise all the errors returned to be uniform.
4. Make use of errors.Wrap[f] whenever possible.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: b7a95a3ce4c731c0fca204435be758ea89d6050f
Component: engine
There are two build errors when using go-1.11beta1:
> daemon/logger/loggerutils/logfile.go:367: Warningf format %q arg f.Name is a func value, not called
> daemon/logger/loggerutils/logfile.go:564: Debug call has possible formatting directive %v
In the first place, the file name is actually not required as error
message already includes it.
While at it, fix a couple of other places for more correct messages, and
make sure to not add a file name if an error already has it.
Fixes: f69f09f44c
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 09ad434f10cff48741322854a3003686b28295b5
Component: engine
Fix the following go-1.11beta1 build error:
> daemon/graphdriver/aufs/aufs.go:376: Wrapf format %s reads arg #1, but call has 0 args
While at it, change '%s' to %q.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2e30e9e6db42043cb2bd67d25a7152488c834f9f
Component: engine
Go 1.11beta1 (rightfully) complains:
> 15:38:37 daemon/cluster/controllers/plugin/controller.go:183:
> Entry.Debugf format %#T has unrecognized flag #
This debug print was added by commit 72c3bcf2a5.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a9a136572dc2a2c2ec5da320c4d0a32b5a15c550
Component: engine
In particular, these two:
> daemon/daemon_unix.go:1129: Wrapf format %v reads arg #1, but call has 0 args
> daemon/kill.go:111: Warn call has possible formatting directive %s
and a few more.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 3737194b9f2875c10f3f2117c1816054ba0ff262
Component: engine
tilde doesn't play nice with docker images so to be uniform let's
replace the tilde with a dash
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: ad988a9a4020eaf44245659a8fe7053a2e9b874b
Component: packaging
mkimage-alpine.sh: Removed the "-i -t" arguments from the smoke test calling printf (the…
Upstream-commit: c04f9ae9d7af14d0fb49bb84e46c03b871f59a5d
Component: engine
The absence of the file /proc/sys/net/ipv6/conf/all/disable_ipv6
doesn't appear to affect functionality, at least at this time.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: d58c4cbe6c781c78db352db2956e9949b8fc8609
Component: engine
dockerd allows the `--log-level` to be specified, but this log-level
was not forwarded to the containerd process.
This patch sets containerd's log-level to the same as dockerd if a
custom level is provided.
Now that `--log-level` is also passed to containerd, the default "info"
is removed, so that containerd's default (or the level configured in containerd.toml)
is still used if no log-level is set.
Before this change:
containerd would always be started without a log-level set (only the level that's configured in `containerd.toml`);
```
root 1014 2.5 2.1 496484 43468 pts/0 Sl+ 12:23 0:00 dockerd
root 1023 1.2 1.1 681768 23832 ? Ssl 12:23 0:00 \_ docker-containerd --config /var/run/docker/containerd/containerd.toml
```
After this change:
when running `dockerd` without options (same as current);
```
root 1014 2.5 2.1 496484 43468 pts/0 Sl+ 12:23 0:00 dockerd
root 1023 1.2 1.1 681768 23832 ? Ssl 12:23 0:00 \_ docker-containerd --config /var/run/docker/containerd/containerd.toml
```
when running `dockerd --debug`:
```
root 600 0.8 2.1 512876 43180 pts/0 Sl+ 12:20 0:00 dockerd --debug
root 608 0.6 1.1 624428 23672 ? Ssl 12:20 0:00 \_ docker-containerd --config /var/run/docker/containerd/containerd.toml --log-level debug
```
when running `dockerd --log-level=panic`
```
root 747 0.6 2.1 496548 43996 pts/0 Sl+ 12:21 0:00 dockerd --log-level=panic
root 755 0.7 1.1 550696 24100 ? Ssl 12:21 0:00 \_ docker-containerd --config /var/run/docker/containerd/containerd.toml --log-level panic
```
combining `--debug` and `--log-level` (`--debug` takes precedence):
```
root 880 2.7 2.1 634692 43336 pts/0 Sl+ 12:23 0:00 dockerd --debug --log-level=panic
root 888 1.0 1.1 616232 23652 ? Ssl 12:23 0:00 \_ docker-containerd --config /var/run/docker/containerd/containerd.toml --log-level debug
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: aaa1392279ffe67a63581d2a43b3a3c7b0fc47b1
Component: engine
It said `TESTFLAGS='-test.run ^TestValidateIPAddress$' make test-unit`
runs `TestBuild` test, but actually runs `TestValidateIPAddress` test.
Signed-off-by: Donghwa Kim <shanytt@gmail.com>
Upstream-commit: 3048c3a7682919297d8d7d26897a59dc91b3be09
Component: engine
Before this commit Healthcheck run if HEALTHCHECK
instruction appears before RUN instruction.
By passing `withoutHealthcheck` to `copyRunConfig`,
always RUN instruction run without Healthcheck.
Fix: https://github.com/moby/moby/issues/37362
Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
Upstream-commit: 44e08d8a7d1249a1956018c6c3d3655642a4f273
Component: engine
Commit c0bc14e8 wrapped the return value of nw.Delete() with some extra
information. However, this breaks the code in
containerAdaptor.removeNetworks() which ignores certain specific
libnetwork error return codes. Said codes actually don't represent
errors, but just regular conditions to be expected in normal operation.
The removeNetworks() call checked for these errors by type assertions
which the errors.Wrap(err...) breaks.
This has a cascading effect, because controller.Remove() invokes
containerAdaptor.removeNetworks() and if the latter returns an error,
then Remove() fails to remove the container itself. This is not
necessarily catastrophic since the container reaper apparently will
purge the container later, but it is clearly not the behavior we want.
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: 6225d1f15c5fd916c3e0ef3afe022f6cc14ac696
Component: engine
Switches dev versioning from being based on a set version like
`18.06.0-dev` to being based purely on the git commit timestamp / git
commit sha.
This should give us a bit more flexibility in terms of nightlies not
being tied to specific versions.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 559c9a323a77fe1215f91d851ce72053ce5dd99b
Component: packaging
Bump libnetwork to 3ac297bc7fd0afec9051bbb47024c9bc1d75bf5b in order to
get fix 0c3d9f00 which addresses a flaw that the scalable load balancing
code revealed. Attempting to print sandbox IDs where the sandbox name
was too short results in a goroutine panic. This can occur with
sandboxes with names of 1 or 2 characters in the previous code. But due
to naming updates in the scalable load balancing code, it could now
occur for networks whose name was 3 characters and at least one of the
integration tests employed such networks (named 'foo', 'bar' and 'baz').
This update also brings in several changes as well:
* 6c7c6017 - Fix error handling about bridgeSetup
* 5ed38221 - Optimize networkDB queue
* cfa9afdb - ndots: produce error on negative numbers
* 5586e226 - improve error message for invalid ndots number
* 449672e5 - Allows to set generic knobs on the Sandbox
* 6b4c4af7 - do not ignore user-provided "ndots:0" option
* 843a0e42 - Adjust corner case for reconnect logic
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: 0e162d992394d9216f590176991d33ff48ef3389
Component: engine
The deafult OCI linux spec in oci/defaults{_linux}.go in Docker/Moby
from 1.11 to current upstream master does not block /proc/acpi pathnames
allowing attackers to modify host's hardware like enabling/disabling
bluetooth or turning up/down keyboard brightness. SELinux prevents all
of this if enabled.
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
Upstream-commit: 569b9702a59804617e1cd3611fbbe953e4247b3e
Component: engine
This fix migrates some ipcmode tests in integration-cli
to integration tests.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: e0403604e26868b1546a766ab0b40b6cb1677ee6
Component: engine
Matching the version that is used in SwarmKit
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 7084487fdcfaf0fdb76a98757d49033c817c5978
Component: engine
Fixes a possible deadlock on closing the watcher on kqueue
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9f016c0fe1cd30381805fdb225854a36a806e961
Component: engine
this should work ( tried on my machine)
Signed-off-by: Ian Chen <ianre657@gmail.com>
Upstream-commit: a7652107189336f243e5c9a89b33a577df34fdd2
Component: engine
This partially reverts https://github.com/moby/moby/pull/37350
Although specs.Platform is desirable in the API, there is more work
to be done on helper functions, namely containerd's platforms.Parse
that assumes the default platform of the Go runtime.
That prevents a client to use the recommended Parse function to
retrieve a specs.Platform object.
With this change, no parsing is expected from the client.
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: facad557440a0c955beb615495b8d0175f25e4e3
Component: engine
This patch is required for the updated version of libnetwork and entails
two minor changes.
First, it uses the new libnetwork.NetworkDeleteOptionRemoveLB option to
the network.Delete() method to automatically remove the load balancing
endpoint for ingress networks. This allows removal of the
deleteLoadBalancerSandbox() function whose functionality is now within
libnetwork.
The second change is to allocate a load balancer endpoint IP address for
all overlay networks rather than just "ingress" and windows overlay
networks. Swarmkit is already performing this allocation, but moby was
not making use of these IP addresses for Linux overlay networks (except
ingress). The current version of libnetwork makes use of these IP
addresses by creating a load balancing sandbox and endpoint similar to
ingress's for all overlay network and putting all load balancing state
for a given node in that sandbox only. This reduces the amount of linux
kernel state required per node.
In the prior scheme, libnetwork would program each container's network
namespace with every piece of load balancing state for every other
container that shared *any* network with the first container. This
meant that the amount of kernel state on a given node scaled with the
square of the number of services in the cluster and with the square of
the number of containers per service. With the new scheme, kernel state
at each node scales linearly with the number of services and the number
of containers per service. This also reduces the number of system calls
required to add or remove tasks and containers. Previously the number
of system calls required grew linearly with the number of other
tasks that shared a network with the container. Now the number of
system calls grows linearly only with the number of networks that the
task/container is attached to. This results in a significant
performance improvement when adding and removing services to a cluster
that already heavily loaded.
The primary disadvantage to this scheme is that it requires the
allocation of an additional IP address per node per subnet for every
node in the cluster that has a task on the given subnet. However, as
mentioned, swarmkit is already allocating these IP addresses for every
node and they are going unused. Future swarmkit modifications should be
examined to only allocate said IP addresses when nodes actually require
them.
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: 8e0f6bc90351525f3e52f3bc357378fcccccdd27
Component: engine
Bump libnetwork to b0186632522c68f4e1222c4f6d7dbe518882024f. This
includes the following changes:
* Dockerize protocol buffer generation and update (78d9390a..e12dd44c)
* Use new plugin interfaces provided by plugin pkg (be94e134)
* Improve linux load-balancing scalability (5111c24e..366b9110)
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: 92335eaef12fb0fb86e5e7503f88b8873a9fa973
Component: engine
in swarm.
Also, fix some CLI command confusions:
1. If the --external-ca flag is provided, require a --ca-cert flag as well, otherwise
the external CA is set but the CA certificate is actually rotated to an internal
cert
2. If a --ca-cert flag is provided, require a --ca-key or --external-ca flag be
provided as well, otherwise either the server will say that the request is
invalid, or if there was previously an external CA corresponding to the cert, it
will succeed. While that works, it's better to require the user to explicitly
set all the parameters of the new desired root CA.
This also changes the `swarm update` function to set the external CA's CACert field,
which while not strictly necessary, makes the CA list more explicit.
Signed-off-by: Ying Li <ying.li@docker.com>
Upstream-commit: 4243440e1f
Component: cli
By 0296797f0f39477d675128c93c1646b3186937ee, `progressReader`
and `remoteURL` were removed from arguments. So developers who
use `Detect` not need to care about when `ProgressReaderFunc`
is used.
Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
Upstream-commit: 0bbd476ceb8da679f818df529cc917ec807a16af
Component: engine
When using the mounts API, bind mounts are not supposed to be
automatically created.
Before this patch there is a race condition between valiating that a
bind path exists and then actually setting up the bind mount where the
bind path may exist during validation but was removed during mountpooint
setup.
This adds a field to the mountpoint struct to ensure that binds created
over the mounts API are not accidentally created.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 1caeb79963d3c9f770b23be2f12c584adf49538d
Component: engine
parsing an environment file should give an error in case a zero-length
variable name (definition w/o a variable name) is encountered.
previously these lines went through unnoticed not informing the user about
a potential configuration error.
Signed-off-by: Tom Klingenberg <tklingenberg@lastflood.net>
Upstream-commit: b91fd12996
Component: cli
previously docker did import environment variables if they were present
but created them if they were not when it was asked via a --env-file
cli option to import but not create them.
fix is to only import the variable into the environment if it is present.
additionally do not import variable names of zero-length (which are lines
w/ a potential variable definition w/o a variable name).
refs:
- https://github.com/docker/for-linux/issues/284
Signed-off-by: Tom Klingenberg <tklingenberg@lastflood.net>
Upstream-commit: 96c026eb30
Component: cli
test to show current behavior is wrong at parsing an environment file
defining an undefined variable - it must not be defined!
NOTE: this test assume the $HOME variable is always set (see POSIX, this
normally is the case, e.g. the test suite remains stable).
Signed-off-by: Tom Klingenberg <tklingenberg@lastflood.net>
Upstream-commit: 1e89745704
Component: cli
The TestDockerNetworkIPAMMultipleNetworks test allocates several
networks simultaneously with overlapping IP addresses. Libnetwork now
forbids this. Adjust the test case to use distinct IP ranges for the
networks it creates.
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: efb7909befa0fe2236148543a6d50e2563bf386c
Component: engine
'CheckInitialized' in the credential-helper library actually invokes
`pass`, which isn't desirable (see #699).
This moves the check to be simpler, and then pass will only be invoked
when it's needed (such as for `docker login` or when pulling from a
private registry).
This logic could also reasonably live in the credential-helper library,
but it's simple enough it seems fine in either location.
Signed-off-by: Euan Kemp <euank@euank.com>
Upstream-commit: 056015c3d8
Component: cli
Bump libnetwork to 430c00a6a6b3dfdd774f21e1abd4ad6b0216c629. This
includes the following moby-affecting changes:
* Update vendoring for go-sockaddr (8df9f31a)
* Fix inconsistent subnet allocation by preventing allocation of
overlapping subnets (8579c5d2)
* Handle IPv6 literals correctly in port bindings (474fcaf4)
* Update vendoring for miekg/dns (8f307ac8)
* Avoid subnet reallocation until required (9756ff7ed)
* Bump libnetwork build to use go version 1.10.2 (603d2c1a)
* Unwrap error type returned by PluginGetter (aacec8e1)
* Update vendored components to match moby (d768021dd)
* Add retry field to cluster-peers probe (dbbd06a7)
* Fix net driver response loss on createEndpoint (1ab6e506)
(fixes https://github.com/docker/for-linux/issues/348)
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: f155f828a26cfb3ee0dd8ca48bd2c702c9032b0b
Component: engine
When a manifest list contains both Linux and Windows images, always
prefer Windows when the platform OS is unspecified. Also, filter out any
Windows images with a higher build than the host, since they cannot run.
Signed-off-by: John Stephens <johnstep@docker.com>
Upstream-commit: ddcdb7255d960a4b2049439245a4f887b26af891
Component: engine
OCI types are backwards compatible with Docker manifest
types, however the media types must be registered.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: c4f0515837b834ec66ec463d92a32015062a0804
Component: engine
Stores complete OCI descriptor instead of digest and platform
fields. This includes the size which was getting lost by not
storing the original manifest bytes.
Attempt to support existing cached files, if not output
the filename with the incorrect content.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: 1fd2d66df8
Component: cli
Fixes named pipe support for hyper-v isolated containers
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 74095588baa0693a2c07bd05c001c42830f82ed3
Component: engine
It's not longer supported since Debian Jessie is now an LTS release as
of June 17, 2018. LTS releases of Debian only support i386, amd64, armel and armhf
Sources:
* https://wiki.debian.org/LTS
* https://wiki.debian.org/LTS/FAQ
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 24f8b422aaed0fba9d0f03bfd5b99c58f0d7f30d
Component: packaging
This is mainly for the `pass` helper ; `pass` shouldn't be called
every docker command anymore ;).
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: d9741fc96b
Component: cli
This stuff doesn't belong here and is causing imports of libnetwork into
the router, which is not what we want.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: c0bc14e8dd5a31f5edc804e9a1347bb3eb44483e
Component: engine
It makes it easier to get the correct stack from a compose config
struct without requiring the client (and thus talking to k8s API)
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: f2e6ee6899
Component: cli
This is a follow-up of commit 408c7ad (PR: #32453)
Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
Upstream-commit: dbd4c290b7029d31fa69c906a8da864589f19e09
Component: engine
Add the OCI spec compatible image support in client side.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 7f334d3acfd7bfde900e16e393662587b9ff74a1
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
Addresses https://github.com/moby/moby/pull/35089#issuecomment-367802698.
This change enables the daemon to automatically select an image under LCOW
that can be used if the API doesn't specify an explicit platform.
For example:
FROM supertest2014/nyan
ADD Dockerfile /
And docker build . will download the linux image (not a multi-manifest image)
And similarly docker pull ubuntu will match linux/amd64
Upstream-commit: 35193c0e7dc301e1d2f6ea96e0ce34ffd2d4b88d
Component: engine
- Handle `bundlefile` directly in the `top-level`
command. `bundlefile` is still experimental and will be deprecated
in future version — this should make be easier to remove it.
- Validate the `stack` name in all cases (i.e. whatever the
orchestrator is used)
- Load the composefile ahead of choosing the orchestrator. This
removes some slight duplication.
- Makes `RunDeploy` easier to use from outside packages (like
`docker/app`) with a preloaded configuration.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 0f9d24f78d
Component: cli
- Add the possibility to skip interpolation
- Add the possibility to skip schema validation
- Allow customizing the substitution function, to add special cases.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 9fdd14f399
Component: cli
- `make test-e2e` runs the e2e tests twice : once against on
non-experimental daemon (as before), once against an experimental
daemon.
- adds `test-e2e-experimental` and `test-e2e-non-experimental` target
to run tests for the specified cases
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: a522a78231
Component: cli
> Run an init inside the container that forwards signals and reaps
processes
This is supported on `run` and now on Swarm services too, so it's also
possible to have in on a composefile :).
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: cc26da94ed
Component: cli
That field is automaticaly populated with any `x-*` field in the yaml.
And marshalling the compose config struct put them back into place.
This make it possible to get those extra fields without re-inventing
the wheel (i.e. reimplementing 80% of the `cli/compose/*` packages.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 80c26f618e
Component: cli
The `-k` shorthand was alreaady removed from other
commands, so best to be consistent.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f0a8598b8d
Component: cli
The output of this information can be confusing,
so removing until we have a better design for this.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a63252ba71
Component: cli
* Renaming DOCKER_ORCHESTRATOR to DOCKER_STACK_ORCHESTRATOR
* Renaming config file option "orchestrator" to "stackOrchestrator"
* "--orchestrator" flag is no more global but local to stack command and subcommands
* Cleaning all global orchestrator code
* Replicating Hidden flags in help and Supported flags from root command to stack command
Signed-off-by: Silvin Lubecki <silvin.lubecki@docker.com>
Upstream-commit: 71272dd203
Component: cli
The archive changes function is not implemented correctly
to handle opaque directories.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: ac5d363e67e01c769b4d9e20ed76a278434ee4de
Component: engine
This test is testing if any "no space left on device" errors
that occur during `docker pull` will not be masked by other
errors. To test for this, a new loopback-device was created,
and used as `--data-dir` ("/var/lib/docker").
However, `/var/lib/docker` is used for storing various
other things, including a `cache.db` database, used by
BuildKit, which is created during startup of the daemon.
Creation of that file failed (due to `--data-dir` path
being on a mount with limited size), which caused daemon
start to fail before the test was able to run.
This patch changes the size-limited mount to be used for
the storage-driver directory only, so that the test is
not affected by other parts of the code attempting to
write files in it.
To have a predictable path; the daemon used in this test
is configured to use the `vfs` storage-driver.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 754aeb28fdb9dd3aafb7705224e57f28281db264
Component: engine
… wherever it's possible. Should make it even easier to test and use
another implementation.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 88068b9dd7
Component: cli
As we did for `Pull`, it will be helpful to expose the push and save
implementation.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: e43c7920ea
Component: cli
Unfortunately, this is for now the only way to see the output of RUN commands when using buildkit.
It is equivalent to `DOCKER_BUILDKIT=1 docker build . 2>&1 | cat`
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: ed75f6202b
Component: cli
With this patch the following become true even with buildkit enabled:
1. `docker build -q .` only outputs the created image's sha256 ID.
2. `docker build -q .` outputs as if no `-q` was specified, if error occurred
3. `docker build . &> out` outputs JSON (instead of TTY characters)
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 640cbb8d2f
Component: cli
This commit brings a more pedantic change in the following ambiguous case:
cat Dockerfile | docker build -f otherDockerfile -
The legacy builder does not error out and prefers the Dockerfile from stdin
while the buildkit-based one errors out.
Note that this is only in the case where stdin is a Dockerfile (not an archive)
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 82f0e1e5d8
Component: cli
This is a follow up of #35168.
Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
Upstream-commit: 7b3966245de27b753c6acfe0347833811ed9eed8
Component: engine
github.com/gotestyourself/gotestyourself moved to gotest.tools with
version 2.0.0. Moving to that one, bumping it to v2.1.0.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: c147e9e1a45b1c66bc5773814b53bcd27132557c
Component: engine
go1.10.3 (released 2018/06/05) includes fixes to the go command, and the
crypto/tls, crypto/x509, and strings packages. In particular, it adds minimal
support to the go command for the vgo transition. See the Go 1.10.3 milestone
on our issue tracker for details;
https://github.com/golang/go/issues?q=milestone%3AGo1.10.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1c71c957cb
Component: cli
go1.10.3 (released 2018/06/05) includes fixes to the go command, and the
crypto/tls, crypto/x509, and strings packages. In particular, it adds minimal
support to the go command for the vgo transition. See the Go 1.10.3 milestone
on our issue tracker for details;
https://github.com/golang/go/issues?q=milestone%3AGo1.10.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 65fc4f1c0cc81739df97605f1fbf91dbf2e7739f
Component: packaging
Mainly to get inline with `docker/cli` version of that dependency
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: f929f15dd6e316402394143eeb60ad9ba95a81cb
Component: engine
Since go1.8, the stdlib TLS net.Conn implementation implements the
`CloseWrite()` interface.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2ac277a56f50d0ad1831603dccbca5c19ff717e9
Component: engine
github.com/gotestyourself/gotestyourself moved to gotest.tools with
version 2.0.0. Moving to that one, bumping it to v2.1.0.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: dd9478a1f7
Component: cli
Errors were not checked while initializing the docker client in the help command
Signed-off-by: Silvin Lubecki <silvin.lubecki@docker.com>
Upstream-commit: f4b0780e20
Component: cli
Scenario:
Daemon is ungracefully shutdown and leaves plugins running (no
live-restore).
Daemon comes back up.
The next time a container tries to use that plugin it will cause a
daemon panic because the plugin client is not set.
This fixes that by ensuring that the plugin does get shutdown.
Note, I do not think there would be any harm in just re-attaching to the
running plugin instead of shutting it down, however historically we shut
down plugins and containers when live-restore is not enabled.
[kir@: consolidate code to deleteTaskAndContainer, a few minor nits]
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: dbeb4329655e91dbe0e6574405937f03fabf3f2f
Component: engine
… discovered recently that our very own Sebastiaan was not even listed
in the maintainers, so fixing this mistake !
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 9fda91bed4
Component: cli
… discovered recently that our very own Sebastiaan was not even listed
in the maintainers, so fixing this mistake !
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 7f9dd507d3f50cd343480cb0776a3ccca83c5764
Component: engine
It's already supported by `swarmkit`, and act the same as
`HostConfig.Init` on container creation.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: e401b88e59e098745744917c555d549f08353e6d
Component: engine
This last assertion shouldn't be there, those cases should be handled
by the for/switch above.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 04207c2db9cd82e47aa82bc2834fe3820086a697
Component: engine
This adds MaskedPaths and ReadOnlyPaths options to HostConfig for containers so
that a user can override the default values.
When the value sent through the API is nil the default is used.
Otherwise the default is overridden.
Adds integration tests for MaskedPaths and ReadonlyPaths.
Signed-off-by: Jess Frazelle <acidburn@microsoft.com>
Upstream-commit: 3694c1e34e40fa2e255a97b5541645cec9c8d1d5
Component: engine
Updates swarmkit, grpc, and all related vendors
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: 52ed3e0896c203013388a3eb603658f345a9d80a
Component: engine
This PR is trying to refactor the `probeAndCreate` and cleanup
related codes based on the refactoring.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 7f280f6f6514598a31d2523d4b9437ca07ee0526
Component: engine
`btrfs-progs` doesn't actually contain the files we need, since we only
need the headers for btrfs, which `libbtrfs-dev` now provides.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 73678285d0671e1b67ec30d21e99cfe1580159fa
Component: packaging
`make DOCKERD_ARGS=--init binary run` should start the daemon with
`--init` as flags (with any other "automagically" added ones).
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: d2af0d96e3fbdfec340cb707b795b863411b00e1
Component: engine
Warns that `-compress` has no effect when used together with the
expremintal `--stream` flag.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 8b3dc39fce
Component: cli
This dependency now uses SemVer, tagged releases.
Also updates dependencies
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 4acddf21b7c11c2c6a71f0f9ba1d329cfbb99e5f
Component: engine
* Kubernetes native filtering (server side) is an exact match, now filtering on name is made client-side to add prefix-matching
Signed-off-by: Silvin Lubecki <silvin.lubecki@docker.com>
Upstream-commit: b1ff199167
Component: cli
Configs and Secrets were note present, some labels on volumes and
networks too.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 55426d9396
Component: cli
This patch adds annotations to mark the checkpoint commands as Linux only, which
hides them if the daemon is running a non-matching operating-system type;
Before:
docker
Usage: docker COMMAND
A self-sufficient runtime for containers
...
Management Commands:
config Manage Docker configs
container Manage containers
image Manage images
After:
docker
Usage: docker COMMAND
A self-sufficient runtime for containers
...
Management Commands:
checkpoint Manage checkpoints
config Manage Docker configs
container Manage containers
image Manage images
This change also prints errors when attempting to use checkpoint commands or
flags if the feature is not supported by the Daemon's operating system;
$ docker checkpoint --help
docker checkpoint is only supported on a Docker daemon running on linux, but the Docker daemon is running on windows
$ docker checkpoint create --help
docker checkpoint create is only supported on a Docker daemon running on linux, but the Docker daemon is running on windows
$ docker checkpoint ls --help
docker checkpoint ls is only supported on a Docker daemon running on linux, but the Docker daemon is running on windows
$ docker checkpoint rm --help
docker checkpoint rm is only supported on a Docker daemon running on linux, but the Docker daemon is running on windows
$ docker container start --checkpoint=foo mycontainer
"--checkpoint" requires the Docker daemon to run on linux, but the Docker daemon is running on windows
$ docker container start --checkpoint-dir=/foo/bar mycontainer
"--checkpoint-dir" requires the Docker daemon to run on linux, but the Docker daemon is running on windows
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1df6b68111
Component: cli
With a full attach, each attach was leaking 4 goroutines.
This updates attach to use errgroup instead of the hodge-podge of
waitgroups and channels.
In addition, the detach event was never being sent.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 0f5147701775a6c5d4980a7b7c0ed2e830688034
Component: engine
As for top-level key, any 3rd-level key which starts with `x-` will be
ignored by compose. This allows for users to:
* include additional metadata in their compose files
* create YAML anchor objects that can be re-used in other parts of the config
This matches a similar feature in the swagger spec definition:
https://swagger.io/specification/#specificationExtensions
This means a composefile like the following is valid
```
verison: "3.7"
services:
foo:
image: foo/bar
x-foo: bar
network:
bar:
x-bar: baz
```
It concerns services, volumes, networks, configs and secrets.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 6bcfa8806c
Component: cli
Includes some refactoring to allow testing the output
in isolation.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: bb71808605
Component: cli
btrfs-tools is being deprecated in favor of btrfs-progs, will only
matter for debian versions >= buster
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 7c9db958146b66ad59727e8395b928a6e9743642
Component: packaging
This makes it a bit simpler to remove this interface for v2 plugins
and not break external projects (libnetwork and swarmkit).
Note that before we remove the `Client()` interface from `CompatPlugin`
libnetwork and swarmkit must be updated to explicitly check for the v1
client interface as is done int his PR.
This is just a minor tweak that I realized is needed after trying to
implement the needed changes on libnetwork.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 7c77df8acc597cd4f540d873de5fe53a3d414ba9
Component: engine
- Build image that contains everything needed to run e2e tests
- Add ability to run e2e tests against an endpoint
Signed-off-by: Christopher Crone <christopher.crone@docker.com>
Upstream-commit: 6b38918ce4
Component: cli
This fix adds `t.Name()` to tests in integration/network
so that services created in those tests have unique names.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 76195f8047eb9a6be710d53ec493ab97bcf86e20
Component: engine
Add validation for stack names to prevent an empty name resulting in _all_
stacks to be returned after filtering, which can result in removal of services
for all stacks if `--prune`, or `docker stack rm` is used.
Before this change;
docker stack deploy -c docker-compose.yml one
docker stack deploy -c docker-compose.yml two
docker stack deploy -c docker-compose.yml three
docker stack deploy -c docker-compose.yml --prune ''
Removing service one_web
Removing service two_web
Removing service three_web
After this change:
docker stack deploy -c docker-compose.yml one
docker stack deploy -c docker-compose.yml two
docker stack deploy -c docker-compose.yml three
docker stack deploy -c docker-compose.yml --prune ''
invalid stack name: ""
Other stack commands were updated as well:
Before this change;
docker stack deploy -c docker-compose.yml ''
Creating network _default
failed to create network _default: Error response from daemon: rpc error: code = InvalidArgument desc = name must be valid as a DNS name component
docker stack ps ''
nothing found in stack:
docker stack rm ''
Removing service one_web
Removing service three_web
Removing service two_web
After this change:
docker stack deploy -c docker-compose.yml ''
invalid stack name: ""
docker stack ps ''
invalid stack name: ""
docker stack rm ''
invalid stack name: ""
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: d38f397da1
Component: cli
- Don't wrap the code to be slightly more readable
- Rename `getServiceFilter()` to `getStackServiceFilter()` to be
consistent with other helper functions.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0c175fc21a
Component: cli
* Add "kubernetes" struct in config file with "allNamespaces" option, to opt-out this behavior when set as "disabled"
Signed-off-by: Mathieu Champlon <mathieu.champlon@docker.com>
Upstream-commit: 1c1300bef6
Component: cli
Begin to copy the data until the command to exit and any coping to
stdin or copy from stdout/stderr has completed.
Also adding defense code to trim the possible '\x00' null value.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 386e0f36c42593ef434517448ccfac5262f958d6
Component: engine
This cleans up some of the package API's used for interacting with
volumes, and simplifies management.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: e4b6adc88e967971de634596654d9bc33e7bd7e0
Component: engine
`${}`, `${:}` and so on are invalid because there's
no parameter within the brackets; fix detection for
this situation and add/update tests.
There were some existing test-cases that were testing
for the wrong behavior, which are now updated.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 334bf3ea76004d0abe02dd1698989f9eaf87a86a
Component: engine
The builder did not detect syntax errors in substitusions in the
Dockerfile, causing those values to be processed incorrectly instead of
producing an error.
Example 1: missing `}`
docker build --no-cache -<<'EOF'
FROM busybox
ARG var=${aaa:-bbb
RUN echo $var
EOF
Before:
Step 3/3 : RUN echo $var
---> Running in f06571e77146
bbb
After:
Step 2/3 : ARG var=${aaa:-bbb
failed to process "${aaa:-bbb": syntax error: missing '}'
Example 2: missing closing `}`, no default value
docker build --no-cache -<<'EOF'
FROM busybox
ARG var=${aaa
RUN echo $var
EOF
Before:
Step 2/3 : ARG var=${aaa
failed to process "${aaa": missing ':' in substitution
After:
Step 2/3 : ARG var=${aaa
failed to process "${aaa": syntax error: missing '}'
Example 3: double opening bracket (`{`)
docker build --no-cache -<<'EOF'
FROM busybox
ARG var=${{aaa:-bbb}
RUN echo $var
EOF
Before:
Step 2/3 : ARG var=${{aaa:-bbb}
failed to process "${{aaa:-bbb}": missing ':' in substitution
After:
Step 2/3 : ARG var=${{aaa:-bbb}
failed to process "${{aaa:-bbb}": syntax error: bad substitution
Example 4: double opening bracket (`{`), no default value
docker build --no-cache -<<'EOF'
FROM busybox
ARG var=${{aaa}
RUN echo $var
EOF
Before:
Step 2/3 : ARG var=${{aaa}
failed to process "${{aaa}": missing ':' in substitution
After:
Step 2/3 : ARG var=${{aaa}
failed to process "${{aaa}": syntax error: bad substitution
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 955a6ad95f7891a45692d975793abf1eeb07cdd5
Component: engine
If "ps" fails, in many cases it prints a meaningful error message
which a user can benefit from. Let's use it.
While at it, let's use errdefs.System to classify the error,
as well as errors.Wrap.
Before:
> $ docker top $CT <any bad ps options>
> Error response from daemon: Error running ps: exit status 1
After:
> $ docker top $CT auxm
> Error response from daemon: ps: error: thread display conflicts with forest display
or
> $ docker top $CT saur
> Error response from daemon: ps: error: conflicting format options
or, if there's no meaningful error on stderr, same as before:
> $ docker top $CT 1234
> Error response from daemon: ps: exit status 1
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: a41328d5704b8d1adbcd099fb4bb0697060df806
Component: engine
Current ContainerTop (a.k.a. docker top) implementation uses "ps"
to get the info about *all* running processes, then parses it, then
filters the results to only contain PIDs used by the container.
Collecting data only to throw most of it away is inefficient,
especially on a system running many containers (or processes).
For example, "docker top" on a container with a single process
can take up to 0.5 seconds to execute (on a mostly idle system)
which is noticeably slow.
Since the containers PIDs are known beforehand, let's use ps's
"-q" option to provide it with a list of PIDs we want info about.
The problem with this approach is, some ps options can't be used
with "-q" (the only one I'm aware of is "f" ("forest view") but
there might be more). As the list of such options is not known,
in case ps fails, it is executed again without "q" (retaining
the old behavior).
Next, the data produced by "ps" is filtered in the same way as before.
The difference here is, in case "-q" worked, the list is much shorter.
I ran some benchmarks on my laptop, with about 8000 "sleep" processes
running to amplify the savings.
The improvement in "docker top" execution times is 5x to 10x (roughly
0.05s vs 0.5s).
The improvement in ContainerTop() execution time is up to 100x
(roughly 3ms vs 300ms).
I haven't measured the memory or the CPU time savings, guess those
are not that critical.
NOTE that busybox ps does not implement -q so the fallback is always
used, but AFAIK it is not usable anyway and Docker expects a normal
ps to be on the system (say the list of fields produced by
"busybox ps -ef" differs from normal "ps -ef" etc.).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: a076badb8b33f1ecdc5d46f0a3701f10c0579f73
Component: engine
`docker stack services --filter=label=foo=bar --filter=label=foo=baz my-stack` with Swarm gets handled as `filter on (a label named foo with value bar) AND (a label named foo with value baz).
This obviously yields an empty result set every time, but if and how this should be changed is out of scope here, so simply align Kubernetes with Swarm for now.
Signed-off-by: Mathieu Champlon <mathieu.champlon@docker.com>
Upstream-commit: 297866ebbe
Component: cli
This fix was not yet included in Docker 17.05, so
API version v1.37 was not the right selector (Docker
18.03, 18.04 and 18.05 all support API v1.37).
We should change these checks for engine versions,
or use a different method to skip tests when running
against older engines.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a061b1e2d8f6117b0524e44de7b6bc391245864e
Component: engine
Before this change:
----------------------------------------------------
Create a service with reservations and limits for memory and cpu:
docker service create --name test \
--limit-memory=100M --limit-cpu=1 \
--reserve-memory=100M --reserve-cpu=1 \
nginx:alpine
Verify the configuration
docker service inspect --format '{{json .Spec.TaskTemplate.Resources}}' test
{
"Limits": {
"NanoCPUs": 1000000000,
"MemoryBytes": 104857600
},
"Reservations": {
"NanoCPUs": 1000000000,
"MemoryBytes": 104857600
}
}
Update just CPU limit and reservation:
docker service update --limit-cpu=2 --reserve-cpu=2 test
Notice that the memory limit and reservation is not preserved:
docker service inspect --format '{{json .Spec.TaskTemplate.Resources}}' test
{
"Limits": {
"NanoCPUs": 2000000000
},
"Reservations": {
"NanoCPUs": 2000000000
}
}
Update just Memory limit and reservation:
docker service update --limit-memory=200M --reserve-memory=200M test
Notice that the CPU limit and reservation is not preserved:
docker service inspect --format '{{json .Spec.TaskTemplate.Resources}}' test
{
"Limits": {
"MemoryBytes": 209715200
},
"Reservations": {
"MemoryBytes": 209715200
}
}
After this change:
----------------------------------------------------
Create a service with reservations and limits for memory and cpu:
docker service create --name test \
--limit-memory=100M --limit-cpu=1 \
--reserve-memory=100M --reserve-cpu=1 \
nginx:alpine
Verify the configuration
docker service inspect --format '{{json .Spec.TaskTemplate.Resources}}' test
{
"Limits": {
"NanoCPUs": 1000000000,
"MemoryBytes": 104857600
},
"Reservations": {
"NanoCPUs": 1000000000,
"MemoryBytes": 104857600
}
}
Update just CPU limit and reservation:
docker service update --limit-cpu=2 --reserve-cpu=2 test
Confirm that the CPU limits/reservations are updated, but memory limit and reservation are preserved:
docker service inspect --format '{{json .Spec.TaskTemplate.Resources}}' test
{
"Limits": {
"NanoCPUs": 2000000000,
"MemoryBytes": 104857600
},
"Reservations": {
"NanoCPUs": 2000000000,
"MemoryBytes": 104857600
}
}
Update just Memory limit and reservation:
docker service update --limit-memory=200M --reserve-memory=200M test
Confirm that the Mempry limits/reservations are updated, but CPU limit and reservation are preserved:
docker service inspect --format '{{json .Spec.TaskTemplate.Resources}}' test
{
"Limits": {
"NanoCPUs": 2000000000,
"MemoryBytes": 209715200
},
"Reservations": {
"NanoCPUs": 2000000000,
"MemoryBytes": 209715200
}
}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: df43eb931e
Component: cli
remove unnescessary import aliases, brackets, and so on.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f23c00d8701e4bd0f2372a586dacbf66a26f9a51
Component: engine
Fixes an issue where `CopyDir` would truncate files larger than 2^32
bytes.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 616fa9446e49502da501a7d0f92b13191f5058dd
Component: engine
Sometimes docker-master CI fails on rhel4+selinux configuration,
like this:
--- FAIL: TestMount (0.12s)
--- FAIL: TestMount/none-remount,size=128k (0.01s)
mounter_linux_test.go:209: unexpected mount option "seclabel" expected "rw,size=128k"
--- FAIL: TestMount/none-remount,ro,size=128k (0.01s)
mounter_linux_test.go:209: unexpected mount option "seclabel" expected "ro,size=128k"
Earlier, commit 8bebd42df2 (PR #34965) fixed this failure,
but not entirely (i.e. the test is now flaky). It looks like
either selinux detection code is not always working (it won't
work in d-in-d), or the kernel might or might not add 'seclabel'
option).
As the subject of this test case is definitely not selinux,
it can just ignore the option added by it.
While at it, fix error messages:
- add missing commas;
- fix a typo;
- allow for clear distinction between mount
and vfs (per-superblock) options.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: d78e885326213e8ef89919c3cc6d16e712e852a8
Component: engine
The following failure is seen in CI from time to time:
> FAIL: docker_api_containers_test.go:435: DockerSuite.TestContainerAPITop
>
> docker_api_containers_test.go:453:
> c.Assert(top.Processes[0][10], checker.Equals, "/bin/sh -c top")
> ... obtained string = "top"
> ... expected string = "/bin/sh -c top"
The test case expects two processes in the output:
1. /bin/sh -c top
2. top
in the given order.
Now, "ps aux" output is sorted by PID*, and so since the "top" is a child
of "/bin/sh -c top" it has a higher PID and will come second as expected
by the test... unless the PIDs on the system are exhausted and PID rollover
happens, in which case PID of "top" will be lower than that of "/bin/sh".
Fix: sort output by process name.
* - in fact it is not sorted, but is being printed in the same order as
the kernel list PID entries in /proc directory, which appears to be
sorted by PID (see ls -1 -U /proc).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 0823ab70990fa4cbf520726013e75c264e6c5231
Component: engine
It does not make sense to keep looking for PID once
we found it, so let's give it a break.
The side effect of this patch is, if there's more than one column
titled "PID", the last (rightmost) column was used before, and now
the first (leftmost) column is used. Should make no practical
difference whatsoever.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 654a7625fc9f0b7b04da0e0e4d151af04a65cc7f
Component: engine
Adds functionality to parse and return network attachment spec
information. Network attachment tasks are phony tasks created in
swarmkit to deal with unmanaged containers attached to swarmkit. Before
this change, attempting `docker inspect` on the task id of a network
attachment task would result in an empty task object. After this change,
a full task object is returned
Fixes#26548 the correct way.
Signed-off-by: Drew Erny <drew.erny@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5b69ff466e61fa168f24869710f2070c742a5565
Component: engine
This prevents the following test case failures "go test" is run
as non-root in the daemon/ directory:
> --- FAIL: TestContainerInitDNS (0.02s)
> daemon_test.go:209: chown /tmp/docker-container-test-054812199/volumes: operation not permitted
>
> --- FAIL: TestDaemonReloadNetworkDiagnosticPort (0.00s)
> reload_test.go:525: mkdir /var/lib/docker/network/files/: permission denied
> --- FAIL: TestRootMountCleanup (0.00s)
> daemon_linux_test.go:240: assertion failed: error is not nil: operation not permitted
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 16670ed4842b1ee4853ba39b6ebf2b771d28db9a
Component: engine
The pigz binary is used to improve performance when
pulling images, but is not available in the default
CentOS package repositories.
Given that RPM's don't have a "recommends" option,
it's better to remove it as a requirement for now,
instead of failing to install for some users.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: abb1f117d3736fdcbbcc15ecef3e36d1f47c63fa
Component: packaging
Arrays expect a type to be set for items in the array.
This patch adds the "string" type, adds a short description,
and some example values.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8b6b55986644c093564497bfef51733a8d598baf
Component: engine
Use a variable in the Makefile to set the GO_VERSION for the builds.
GO_VERSION is defaulted in all Makefiles as `1.10.2`, but can be set when running make: `make GO_VERSION=1.10.1 deb`.
Signed-off-by: corbin-coleman <corbin.coleman@docker.com>
Upstream-commit: 0fe741746287335a5ba4afaf06be301915324e5a
Component: packaging
`arm64` needs get more time duration for the test to finish.
`pty.Start()` opens a file, so the caller should close it explicitly,
else the file I/O can result in unexpected data synchronization issue.
All those changes will not affect the test itself.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 476d7872efb60b1ef1bc7d9d83952f9dbc8f8798
Component: engine
Changes included:
- libnetwork#2147 Adding logs for ipam state
- libnetwork#2143 Fix race conditions in the overlay network driver
- possibly addresses moby#36743 services do not start: ingress-sbox is already present
- possibly addresses moby#30427 Flaky Test: TestSwarmPublishDuplicatePorts on s390
- possibly addresses moby#36501 Flaky tests: Service "port" tests
- libnetwork#2142 Add wait time into xtables lock warning
- libnetwork#2135 filter xtables lock warnings when firewalld is active
- libnetwork#2140 Switch from x/net/context to context
- libnetwork#2134 Adding a recovery mechanism for a split gossip cluster
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 158ab95c17bed9e849e0d935899b75018cdf61a4
Component: engine
`GetTimestamp()` "assumed" values it could not parse
to be a valid unix timestamp, and would use invalid
values ("hello world") as-is (even testing that
it did so).
This patch validates unix timestamp to be a valid
numeric value, and makes other values invalid.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 48cfe3f0872647bf0f52e6ba65022e1859e808e8
Component: engine
In case aufs driver is not supported because supportsAufs() said so,
it is not possible to get a real reason from the logs.
To fix, log the error returned.
Note we're not using WithError here as the error message itself is the
sole message we want to print (i.e. there's nothing to add to it).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 91f85d1c784f3dc9d892b2af2f51d6b6f3b0be69
Component: engine
Simplify the code by using a single logger instance.
While at it, use WithError in Umount.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: c6e2af54256211b8ac757e9b25caa6fb6c9b3c6e
Component: engine
The `deleteAllImages()` cleanup step in the integration suite was printing a
message for each image that was removed.
These messages were not very informative (given that `removeImage()` prints an
error if image removal failed), and made the test output harder to read
(more difficult to see error-conditions when scanning the output).
This patch removes the messages to make the output slightly less noisy.
Before this patch applied:
--- PASS: TestBuildMultiStageOnBuild (7.15s)
main_test.go:32: Removing image sha256:9db3ddbaaadd52804d8a417081f68db41fd4b8f80c85c1b4c4aee2d9b584c074
main_test.go:32: Removing image sha256:7eeb04d90b5e62a99ac6a5b2c10b9ba54b89b176fe2783e41461581c482852b3
main_test.go:32: Removing image sha256:adf42475eefff99b4a611c1a5d8353c4d0a011a7f7b9dc59a75d951cd54fa77f
main_test.go:32: Removing image sha256:c547a770806e0445f5dfc255683ced771a23be6157ba8d0617bb9ab55dcee6d1
main_test.go:32: Removing image sha256:b79659c3e6d34faf2a075f1df1ea2c805833982f112666b25a466177b5d1352d
main_test.go:32: Removing image sha256:29430078cdc927c19c87416cd1fb1ec386f167c5e201ee5dfb1644dcf268a3c2
=== RUN TestBuildUncleanTarFilenames
--- PASS: TestBuildUncleanTarFilenames (2.47s)
main_test.go:32: Removing image sha256:0a6418fb221dc2f25085a1a7e507e01c4a3938cc5c65f1cb85a8c0fb09d6814f
main_test.go:32: Removing image sha256:3d6e4bd0cce01ce5823b40dcb717cd16b3b4b769ff73dd86fa448aac49aa6d7c
main_test.go:32: Removing image sha256:98e3f335e874612668335b3a5f125a1e5cbd0f6c79a7c3f719529b69d0abf2a5
main_test.go:32: Removing image sha256:08919f344b382fd1447da7f3e8ffd2a7125f5f7d191ed7d33242736dbe3c59cf
main_test.go:32: Removing image sha256:d144b3c13838e841ec319a17e1046471d726bb2aa3211e167a6a53f766a2dcdc
main_test.go:32: Removing image sha256:7c768ec742d628020f50c99dc5af32400b78534ca9fc4c01a9f00ec0ab19193a
main_test.go:32: Removing image sha256:4c26c71d142045fbf3448aa1f6363d5a7a803cb438a78a4b20b7c847df03d50a
main_test.go:32: Removing image sha256:08dcd63c964f2dbb17ff2665b6b86993fb14c0d3e169da187ac48f078a560d25
main_test.go:32: Removing image sha256:c1743fab233f36f2d7f83cb13f8c10ff06bdbda8f8a218d25a3796d1bc2f9e84
=== RUN TestBuildMultiStageLayerLeak
--- PASS: TestBuildMultiStageLayerLeak (5.59s)
main_test.go:32: Removing image sha256:5e9974558276c34d7c9aab3fad408d433047b7b15bdae43ab5048adc58a15431
main_test.go:32: Removing image sha256:ac3c613c8c48794237c1e46ac0657ebbb1132910a240086bb2e9df9770fdc017
main_test.go:32: Removing image sha256:a936df268131ad427f7b4b66ce3dbb1e41866d7269a4d383cebcb1c5930d3346
main_test.go:32: Removing image sha256:5e613ea0ce7dbc908b0315c49585ae43ad6c34158e9e0b59a3dc93b00ef0ea41
main_test.go:32: Removing image sha256:e676f4ec41a42823b6d91e05e3290d3827f9175dea6fba5d8b769aa13aa7e082
main_test.go:32: Removing image sha256:93c8daab2703126b23d957d4d6b04f07949356f1cd95d4f8fdbededf4ab5c21e
With this patch applied:
=== RUN TestBuildMultiStageOnBuild
--- PASS: TestBuildMultiStageOnBuild (6.74s)
=== RUN TestBuildUncleanTarFilenames
--- PASS: TestBuildUncleanTarFilenames (2.49s)
=== RUN TestBuildMultiStageLayerLeak
--- PASS: TestBuildMultiStageLayerLeak (5.14s)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5afce21e2efe572be3a256b748bf269afdd645a5
Component: engine
dockerd has no subcommands, so the help command
is not used.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 298a26e0b2d086a32f14ee675bf24322cc92ddd4
Component: engine
> HasAvailableFlags checks if the command contains any flags (local
> plus persistent from the entire structure) which are not hidden or
> deprecated.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9411856536ef49883fb3550fd1ee804ab99c4072
Component: engine
Use a tagged release of Cobra. All relevant PR's were merged, so the fork is
no longer needed.
Relevant changes:
- spf13/cobra#552 Add a field to disable [flags] in UseLine()
- spf13/cobra#567 Add `CalledAs` method to cobra.Command
- spf13/cobra#580 Update error message for missing required flags
- spf13/cobra#584 Add support for --version flag
- spf13/cobra#614 If user has a project in symlink, just use its destination folder and work there
- spf13/cobra#649 terminates the flags when -- is found in commandline
- spf13/cobra#662 Add support for ignoring parse errors
- spf13/cobra#686 doc: hide hidden parent flags
Also various improvements were added for generating Bash
completion scripts (currently not used by us)
Fixes usage output for dockerd;
Before this update:
dockerd --help
Usage: dockerd COMMAND
A self-sufficient runtime for containers.
After this update:
dockerd --help
Usage: dockerd [OPTIONS] [flags]
A self-sufficient runtime for containers.
Bump spf13/pflag to v1.0.1
Relevant changes:
- spf13/pflag#106 allow lookup by shorthand
- spf13/pflag#113 Add SortFlags option
- spf13/pflag#138 Generate flag error output for errors returned from the parseFunc
- spf13/pflag#141 Fixing Count flag usage string
- spf13/pflag#143 add int16 flag
- spf13/pflag#122 DurationSlice: implementation and tests
- spf13/pflag#115 Implement BytesHex type of argument
- spf13/pflag#150 Add uintSlice and boolSlice to name prettifier
- spf13/pflag#155 Add multiline wrapping support
- spf13/pflag#158 doc: clarify difference between string slice vs. array
- spf13/pflag#160 add ability to ignore unknown flags
- spf13/pflag#163 Allow Users To Show Deprecated Flags
Hide [flags] in usage output
Hides the [flags] in the usage output of commands (present in newer
versions of Cobra), using the `.DisableFlagsInUseLine` option.
Before this change:
dockerd --help
Usage: dockerd [OPTIONS] [flags]
A self-sufficient runtime for containers.
After this change:
dockerd --help
Usage: dockerd [OPTIONS]
A self-sufficient runtime for containers.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Â# modified: vendor/github.com/spf13/pflag/string_array.go
§
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ed75c7727bf3e454a5faa6baf686de12152d911c
Component: engine
This patch hides the [flags] in the usage output of commands, using the
new `.DisableFlagsInUseLine` option, instead of the temporary workaround
added in 8e600e10f7
Before this change:
docker run
"docker run" requires at least 1 argument.
See 'docker run --help'.
Usage: docker run [OPTIONS] IMAGE [COMMAND] [ARG...] [flags]
Run a command in a new container
After this change:
docker run
"docker run" requires at least 1 argument.
See 'docker run --help'.
Usage: docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
Run a command in a new container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 00d080269a
Component: cli
Use a tagged release of Cobra
Relevant changes:
- spf13/cobra#567 Add `CalledAs` method to cobra.Command
- spf13/cobra#580 Update error message for missing required flags
- spf13/cobra#584 Add support for --version flag
- spf13/cobra#614 If user has a project in symlink, just use its destination folder and work there
- spf13/cobra#649 terminates the flags when -- is found in commandline
- spf13/cobra#662 Add support for ignoring parse errors
- spf13/cobra#686 doc: hide hidden parent flags
Also various improvements were added for generating Bash
completion scripts (currently not used by us)
Bump spf13/pflag to v1.0.1
Relevant changes:
- spf13/pflag#122 DurationSlice: implementation and tests
- spf13/pflag#115 Implement BytesHex type of argument
- spf13/pflag#150 Add uintSlice and boolSlice to name prettifier
- spf13/pflag#155 Add multiline wrapping support
- spf13/pflag#158 doc: clarify difference between string slice vs. array
- spf13/pflag#160 add ability to ignore unknown flags
- spf13/pflag#163 Allow Users To Show Deprecated Flags
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0acb2522bd
Component: cli
This hasn't been the way to release Docker for the past year so let's
just remove them altogether
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 80bc8b4766cc808186af91930152d78afb22aef8
Component: engine
The `docker version` output now uses a tabwriter, so use single
tabs to print the output.
Before this change:
Server:
Engine:
Version: 18.05.0-ce
API version: 1.37 (minimum version 1.12)
Go version: go1.10.1
Git commit: f150324
Built: Wed May 9 22:20:16 2018
OS/Arch: linux/amd64
Experimental: true
Kubernetes:
Version: v1.9.6
StackAPI: v1beta2
After this change:
Server:
Engine:
Version: 18.05.0-ce
API version: 1.37 (minimum version 1.12)
Go version: go1.10.1
Git commit: f150324
Built: Wed May 9 22:20:16 2018
OS/Arch: linux/amd64
Experimental: true
Kubernetes:
Version: v1.9.6
StackAPI: v1beta2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 26d8122594
Component: cli
ext4 support d_type by default, but filetype feature is a tunable so
there is still a chance to disable it for some reasons. In this case,
print additional message to explicitly tell how to support d_type.
Signed-off-by: Chengguang Xu <cgxu519@gmx.com>
Upstream-commit: 8a21b128d4deb874c05eb81ebbc1265175ad69ba
Component: engine
Debian Wheezy will be EOL on 31-05-2018 per the official documentation,
https://wiki.debian.org/LTS
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 0e45aed95d86bc516b9cd05f3316073e8dda0a0b
Component: packaging
Since we use `NewLabelCommand()` instead of `addNodesForLabelOption()`
to create the 'LABEL' commands from '--label' options, so all the related
functions should be removed.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: c7b543164daed58fbea36471592438b4e53ab748
Component: engine
Add a new test case `TestBuildLabelWithTargets` to cover the Docker builder
with both `--label` and `--target` options.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: f7add4262b69294aa8035c4794236193ce4bc68e
Component: engine
This PR is tring to fix issue #36996.
Currently for multi-stage build, if `--target` specified, the `--label` option
will be ignored. The root cause is the last stage build will remove the `LABEL`
command(s) node created from the `--label` option. In order to address this issue,
we can create `LABEL` command(s) and add it/tem to the last stage.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 9c238ebd55e4105ad7f7edc04231ea61bb278ae8
Component: engine
Version 1.1
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
Signed-off-by: nkwangleiGIT <nkwanglei@126.com>
Upstream-commit: b6def77c55
Component: cli
This issue has been reported by issue #36877.
The purpose of this test case is for the regression test of #12546,
so we only need to make sure the essential of the testing is still
in the way to check that while not disturbed by some testing noises,
which is exactly what this PR want to do.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 96abf9f59ebd350cc9a70034b4d30279e6ae04e4
Component: engine
All other docker stack commands report an error when passed this value.
Signed-off-by: Mathieu Champlon <mathieu.champlon@docker.com>
Upstream-commit: 4d947de292
Component: cli
The hardcoded values make it so that we have to update this everytime we
add/subtract something which is sometimes followed/not followed.
This makes it so that we no longer have to update this.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 04ec3b020edc79830c5a0100790f5bb6686d8efc
Component: packaging
Fedora 26 will be EOL one month after the release of Fedora 28
(2018-05-01)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: c638836c18d26c5f7e4602784d6b1c7d13f39904
Component: packaging
In cases where a logging plugin has crashed when the daemon tries to
copy the container stdio to the logging plugin it returns a broken pipe
error and any log entries that occurr while the plugin is down are lost.
Fix this by opening read+write in the daemon so logs are not lost while
the plugin is down.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: e7479e3ab8128f9e84cc640f0bed4e77b268a6e9
Component: engine
- Check errors.Cause(err) when comparing errors
- Fix bug where oldest log file is not actually removed. This in
particular causes issues when compression is enabled. On rotate it just
overwrites the data in the log file corrupting it.
- Use O_TRUNC to open new gzip files to ensure we don't corrupt log
files as happens without the above fix.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: e87e9e6ad6ba501cc42a2ef47ac18c88a68f258f
Component: engine
Closing the log driver was in a defer meanwhile logs are
collected asyncronously, so the log driver was being closed before reads
were actually finished.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 2c252a48c252749d41079cf8c450d00a5c18296e
Component: engine
This was broken by bf6a790f00ab96bb8857e3c73502909ee5e36217
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: b16b125bb4326e2eec9948fd54ca8c5d83eba36a
Component: engine
In TestDockerNetworkConnectAlias the network and container
names used are unique which are not preferred. This
fix address the issue by appending t.Name() so that
names are randomized.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: e7b5f4500dc77ec55e5ef202065fbc92218ce394
Component: engine
This message was missing the name of the plugin, resulting in
plugin_test.go:92: timeout hit after 30s: plugin %!q(MISSING) exists
On failing tests.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b526964584d213074de7c5573a3373a58699c181
Component: engine
Embedded new lines aren't interpreted correctly in markdown renderers (they are treated as preformatted text instead). I removed the embedded newlines in the docker image spec.
Signed-off-by: Alex Goodman <wagoodman@gmail.com>
Upstream-commit: 4fb59c20a4fb54f944fe170d0ff1d00eb4a24d6f
Component: engine
In version-history.md, the link for `Docker Engine API v1.37`
was pointed to `v1.36`.
This fix fixes the incorrect link.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 6d66743ae3da148adbd29fb7e0c0433e93fec12b
Component: engine
This fix re-orders the CHANGELOG so that `make validate` test
could pass. The validation requires that the date in the changelog
is always ordered.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 333693c37c8a2c14458d2a088829b9a2a3570eed
Component: engine
Switch from x/net/context to context made "go vet" see the previously
unseen errors:
> cli/command/container/start.go:57::error: the cancelFun function is
> not used on all paths (possible context leak) (vet)
> cli/command/container/start.go:63::error: this return statement may be
> reached without using the cancelFun var defined on line 57 (vet)
> cli/command/container/run.go:159::error: the cancelFun function is not
> used on all paths (possible context leak) (vet)
> cli/command/container/run.go:164::error: this return statement may be
> reached without using the cancelFun var defined on line 159 (vet)
Do call the cancel function.
Note we might end up calling it twice which is fine as long as I can see
from the Go 1.10 source code.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 395957155f
Component: cli
Since go 1.7, "context" is a standard package. Since go 1.9,
x/net/context merely provides some types aliased to those in
the standard context package.
The changes were performed by the following script:
for f in $(git ls-files \*.go | grep -v ^vendor/); do
sed -i 's|golang.org/x/net/context|context|' $f
goimports -w $f
for i in 1 2; do
awk '/^$/ {e=1; next;}
/\t"context"$/ {e=0;}
{if (e) {print ""; e=0}; print;}' < $f > $f.new && \
mv $f.new $f
goimports -w $f
done
done
[v2: do awk/goimports fixup twice]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 6f8070deb2
Component: cli
Removing a host by `<host>:<ip>` should only remove occurences of the host with
a matching IP-address, instead of removing all entries for that host.
In addition, combining `--host-rm` and `--host-add` for the same host should
result in the new host being added.
This patch fixes the way the diff is calculated to allow combining
removing/adding, and to support entries having both a canonical, and aliases.
Aliases cannot be added by the CLI, but are supported in the Service spec, thus
should be taken into account:
Entries can be removed by either a specific `<host-name>:<ip-address>`
mapping, or by `<host>` alone:
- If both IP-address and host-name is provided, only remove the hostname
from entries that match the given IP-address.
- If only a host-name is provided, remove the hostname from any entry it
is part of (either as _canonical_ host-name, or as _alias_).
- If, after removing the host-name from an entry, no host-names remain in
the entry, the entry itself should be removed.
For example, the list of host-entries before processing could look like this:
hosts = &[]string{
"127.0.0.2 host3 host1 host2 host4",
"127.0.0.1 host1 host4",
"127.0.0.3 host1",
"127.0.0.1 host1",
}
Removing `host1` removes every occurrence:
hosts = &[]string{
"127.0.0.2 host3 host2 host4",
"127.0.0.1 host4",
}
Whereas removing `host1:127.0.0.1` only remove the host if the IP-address matches:
hosts = &[]string{
"127.0.0.2 host3 host1 host2 host4",
"127.0.0.1 host4",
"127.0.0.3 host1",
}
Before this patch:
$ docker service create --name my-service --host foo:127.0.0.1 --host foo:127.0.0.2 --host foo:127.0.0.3 nginx:alpine
$ docker service update --host-rm foo:127.0.0.1 --host-add foo:127.0.0.4 my-service
$ docker service inspect --format '{{.Spec.TaskTemplate.ContainerSpec.Hosts}}' my-service
[]
After this patch is applied:
$ docker service create --name my-service --host foo:127.0.0.1 --host foo:127.0.0.2 --host foo:127.0.0.3 nginx:alpine
$ docker service update --host-rm foo:127.0.0.1 --host-add foo:127.0.0.5 my-service
$ docker service inspect --format '{{.Spec.TaskTemplate.ContainerSpec.Hosts}}' my-service
[127.0.0.2 foo 127.0.0.3 foo 127.0.0.4 foo]
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 27c0858f43
Component: cli
This function was added in 23e5c94cfb26eb72c097892712d3dbaa93ee9bc0 but never used
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8a8ec00c1fece033b20548b7a5ec2f3a1f340834
Component: engine
The overlay storage driver currently does not support any option, but was silently
ignoring any option that was passed.
This patch verifies that no options are passed, and if they are passed will produce
an error.
Before this change:
dockerd --storage-driver=overlay --storage-opt dm.thinp_percent=95
INFO[2018-05-11T11:40:40.996597152Z] libcontainerd: started new docker-containerd process pid=256
....
INFO[2018-05-11T11:40:41.135392535Z] Daemon has completed initialization
INFO[2018-05-11T11:40:41.141035093Z] API listen on /var/run/docker.sock
After this change:
dockerd --storage-driver=overlay --storage-opt dm.thinp_percent=95
INFO[2018-05-11T11:39:21.632610319Z] libcontainerd: started new docker-containerd process pid=233
....
Error starting daemon: error initializing graphdriver: overlay: unknown option dm.thinp_percent
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 30f15d2bdc551870464d1cd024a92341cf1ae4aa
Component: engine
The "update" and "rollback" configurations were cross-wired, as a result, setting
`--rollback-*` options would override the service's update-options.
Creating a service with both update, and rollback configuration:
docker service create \
--name=test \
--update-failure-action=pause \
--update-max-failure-ratio=0.6 \
--update-monitor=3s \
--update-order=stop-first \
--update-parallelism=3 \
--rollback-failure-action=continue \
--rollback-max-failure-ratio=0.5 \
--rollback-monitor=4s \
--rollback-order=start-first \
--rollback-parallelism=2 \
--tty \
busybox
Before this change:
docker service inspect --format '{{json .Spec.UpdateConfig}}' test \
&& docker service inspect --format '{{json .Spec.RollbackConfig}}' test
Produces:
{"Parallelism":3,"FailureAction":"pause","Monitor":3000000000,"MaxFailureRatio":0.6,"Order":"stop-first"}
{"Parallelism":3,"FailureAction":"pause","Monitor":3000000000,"MaxFailureRatio":0.6,"Order":"stop-first"}
After this change:
{"Parallelism":3,"FailureAction":"pause","Monitor":3000000000,"MaxFailureRatio":0.6,"Order":"stop-first"}
{"Parallelism":2,"FailureAction":"continue","Monitor":4000000000,"MaxFailureRatio":0.5,"Order":"start-first"}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f367aa9330
Component: cli
Really these mounts shouldn't be in the container pacakge.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: d6558ad6a4552fc745e893f93ae190710d1a36bc
Component: engine
These network operations really don't have anything to do with the
container but rather are setting up the networking.
Ideally these wouldn't get shoved into the daemon package, but doing
something else (e.g. extract a network service into a new package) but
there's a lot more work to do in that regard.
In reality, this probably simplifies some of that work as it moves all
the network operations to the same place.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: cc8f358c23d307d19b06cd5e7d039d8fad01a947
Component: engine
A recent optimization in getSourceMount() made it return an error
in case when the found mount point is "/". This prevented bind-mounted
volumes from working in such cases.
A (rather trivial but adeqate) unit test case is added.
Fixes: 871c957242 ("getSourceMount(): simplify")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: d8fd6137a1f6d95a2bcdfeb6e1dfa6b816790c5e
Component: engine
The Partial property of the Logger message
was replaced by PLogMetaData, causing the build to fail.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: daaef83cd24f6eb335b77e9d8d692235eff1b201
Component: engine
Experimental features are no longer a build-time option,
and can be enabled at runtime.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 050b421f4c435c90dc63bb094789b0513689e80c
Component: packaging
Append a warning with which filter the elements will be pruned with if filter is set.
Signed-off-by: Justyn Temme <justyntemme@gmail.com>
Upstream-commit: 7d113de085
Component: cli
This ensures Windows paths are handled correctly as explained in the path package documentation.
Signed-off-by: Mathieu Champlon <mathieu.champlon@docker.com>
Upstream-commit: 698ea75c66
Component: cli
This is an old script using tools that are no longer maintained or
recommended (and don't even work anymore).
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: ddb01ee1e05f1b460267eb40070e2a62a1c33692
Component: engine
These are no longer used and instead users should use the
`container-selinux` package on their distribution. Additionally, these
are unmaintained and untested.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: eeea1e37a116c0e3765e2591a2ced936e48b5abb
Component: engine
This is left-over stuff from building Docker pacakges. These aren't
really maintained outside of bumping the golang version, and are never
tested.
These builders can be found at
https://github.com/docker/docker-ce-packaging where they are kept up to
date.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 1b7fe816e86f21a968410fca7a0cb9302c386f84
Component: engine
It does not make sense to copy a slice element by element, then discard
the source one. Let's do copy in place instead which is way more
efficient.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: d4c94e83ca58b5ffd99840a8e3cbf22f1ab331c0
Component: engine
When this test fails, the error looks like this:
> FAIL: docker_api_attach_test.go:98: DockerSuite.TestPostContainersAttach
> docker_api_attach_test.go:211:
> c.Assert(actualStdout.Bytes(), checker.DeepEquals, []byte("hello\nsuccess"), check.Commentf("Attach didn't return the expected data from stdout"))
> ... obtained []uint8 = []byte{0x73, 0x75, 0x63, 0x63, 0x65, 0x73, 0x73}
> ... expected []uint8 = []byte{0x68, 0x65, 0x6c, 0x6c, 0x6f, 0xa, 0x73, 0x75, 0x63, 0x63, 0x65, 0x73, 0x73}
> ... Attach didn't return the expected data from stdout
Let's use strings for comparisons to make the output more readable.
While at it,
- get the container's stderr as well, and make sure it's empty;
- check that stdcopy.StdCopy() did not return an error, except for
the timeout which is expected;
- move/remove comments, simplify var names.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: ecc54889c9a233d558b578dfd48eb5d84506a62a
Component: engine
1. As daemon.ContainerStop() documentation says,
> If a negative number of seconds is given, ContainerStop
> will wait for a graceful termination.
but since commit cfdf84d5d04c8ee (PR #32237) this is no longer the case.
This happens because `context.WithTimeout(ctx, timeout)` is implemented
as `WithDeadline(ctx, time.Now().Add(timeout))`, resulting in a deadline
which is in the past.
To fix, don't use WithDeadline() if the timeout is negative.
2. Add a test case to validate the correct behavior and
as a means to prevent a similar regression in the future.
3. Fix/improve daemon.ContainerStop() and client.ContainerStop()
description for clarity and completeness.
4. Fix/improve DefaultStopTimeout description.
Fixes: cfdf84d5d04c ("Update Container Wait")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 69b4fe406540c7989bd99b2244ed67ced96e17c7
Component: engine
Add testing code to cover the `--target` name case sensitive
issue reported by issue #36956.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: a95fabc70efacc1ff6c0c62c490cf551215bc503
Component: engine
This PR is trying to fix issue #36956.
The stage name is case-insensitive by design, so we should use
`strings.EqualFold()` as the comparison method to eliminate the
case sensitive noise.
Also we need to return a pre-defined error code order to avoid below
message like:
"FIXME: Got an API for which error does not match any expected type!!!:
failed to reach build target dev in Dockerfile"
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 7c0570473cfa181aeb3278072cc9af4f9298cb98
Component: engine
vendored distribution is quite old, and current distribution contains an
API break, which means it's not possible to vendor a bugfixed
distribution and a docker/docker at the same time.
Signed-off-by: Mike Lundy <mike@fluffypenguin.org>
Upstream-commit: 07ffa1cea9d995a7ab55793b5b08758c38fec3f0
Component: engine
In pkg/term/proxy.go and pkg/term/proxy_test.go, check if escapeKeys is empty and if it is, return the one key read
Signed-off-by: Patrik Cyvoct <patrik@ptrk.io>
Upstream-commit: d339130f307634a0edd6aef29b6934c3934e9ba2
Component: engine
It will be helpful to expose the pull implementation which supports
pulling private images for other CLI commands that rely on helper images.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 812f113685
Component: cli
When then non-blocking mode is specified, awslogs will:
- No longer potentially block calls to logstream.Log(), instead will
return an error if the awslogs buffer is full. This has the effect of
dropping log messages sent to awslogs.Log() that are made while the
buffer is full.
- Wait to initialize the log stream until the first Log() call instead of in
New(). This has the effect of allowing the container to start in
the case where Cloudwatch Logs is unreachable.
Both of these changes require the --log-opt mode=non-blocking to be
explicitly set and do not modify the default behavior.
Signed-off-by: Cody Roseborough <crrosebo@amazon.com>
Upstream-commit: c7e379988c9cd6ec0af528e6f59eea3c51b36738
Component: engine
We really need to run those on the CI too at some point.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: ef2c2040c22eba8ab6ea1033ad64ba9b0095db9b
Component: engine
This bumps the version of tini used to fec3683b971d9c3ef73f284f176672c44b448662 (v0.18.0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b711dd94fdad0ef8e2c9b2ebbe6d903d270752de
Component: engine
- Volume store created dir with wrong permissions
- Local volume driver hardcoded uid/gid 0
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: d15734ec3c10eda667b716f67e18d5d86e708e3e
Component: engine
Since Go 1.7, context is a standard package. Since Go 1.9, everything
that is provided by "x/net/context" is a couple of type aliases to
types in "context".
Many vendored packages still use x/net/context, so vendor entry remains
for now.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 7d62e40f7e4f3c17d229a7687d6fcca5448de813
Component: engine
Fix a version mismatch in the API documentation.
Signed-off-by: Wassim DHIF <wassimdhif@gmail.com>
Upstream-commit: 2058928edc284653b1fb8e719d713ac7edecd091
Component: engine
- Some of them don't make sense anymore
- Some are deprecated and removed from the engine since a few versions
already.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 740d260cd2
Component: cli
Use more gotestyourself for `env.Patch`, and `icmd.RunCommand`
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: ae03dd7f46
Component: cli
It should check `os.Geteuid` with `uid` instead of `os.Getegid`.
On the container (where the tests run), the uid and gid seems to be
the same, thus this doesn't fail.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 5d8b88b114da7c40f04a95909a35287ae1dac37f
Component: engine
This fix consists of some improvement in restart_test.go
by replacing Fatal with assert, so that they are consistent
with other tests in integration/container.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 67535921b32f68863f7a882d2300b59e0bbeffa4
Component: engine
govet complains (when using standard "context" package):
> the cancel function returned by context.WithTimeout should be called,
> not discarded, to avoid a context leak (vet)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 05e2f7e2fafd0fbc818c9f4cda7ac513c785d49c
Component: engine
We are using interface in the api routers to not explicitely depend on
the daemon struct (`daemon.Daemon`), but somehow, we do depend on the
`daemon` package for the cluster functionalities.
This removes this dependency by defining the correct interfaces.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: f4106b46db47524b4f38abeee48137d42e3fe4eb
Component: engine
That way, those lines won't be reported in the failure.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: cb8db44395df70fa3044d2a9683d6d24438cfa74
Component: engine
Updates swarmkit to 33d06bf5189881b4d1e371b5571f4d3acf832816, to bring in
docker/swarmkit#2610 (Don't use wrappers for grpc metadata)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b18f7033b2644c0246345bb5747a5436568b1a71
Component: engine
This is not for the sake of test to run faster of course;
this is to simplify the code as well as have some more
testing for mount.SingleEntryFilter().
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: ce468f0ad0d075c5d0c44c78bd61c489e6d7d70c
Component: engine
There is no need to parse mount table and iterate through the list of
mounts, and then call Unmount() which again parses the mount table and
iterates through the list of mounts.
It is totally OK to call Unmount() unconditionally.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: ac39a95ea618601f78662972c35838d928858904
Component: engine
Now, every Unmount() call takes a burden to parse the whole nine yards
of /proc/self/mountinfo to figure out whether the given mount point is
mounted or not (and returns an error in case parsing fails somehow).
Instead, let's just call umount() and ignore EINVAL, which results
in the same behavior, but much better performance.
Note that EINVAL is returned from umount(2) not only in the case when
`target` is not mounted, but also for invalid flags. As the flags are
hardcoded here, it can't be the case.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: a1d095199ddb9b4811e1417b6adcdfadad7d73f4
Component: engine
The mountinfo parser implemented via `fmt.Sscanf()` is slower than the one
using `strings.Split()` and `strconv.Atoi()`. This rewrite helps to speed it
up to a factor of 8x, here is a result from go bench:
> BenchmarkParsingScanf-4 300 22294112 ns/op
> BenchmarkParsingSplit-4 3000 2780703 ns/op
I tried other approaches, such as using `fmt.Sscanf()` for the first
three (integer) fields and `strings.Split()` for the rest, but it slows
things down considerably:
> BenchmarkParsingMixed-4 1000 8827058 ns/op
Note the old code uses `fmt.Sscanf`, when a linear search for '-' field,
when a split for the last 3 fields. The new code relies on a single
split.
I have also added more comments to aid in future development.
Finally, the test data is fixed to now have white space before the first field.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: c611f18a7f16d8aa878a5a5c7537d23a0937c40a
Component: engine
The flow of getSourceMount was:
1 get all entries from /proc/self/mountinfo
2 do a linear search for the `source` directory
3 if found, return its data
4 get the parent directory of `source`, goto 2
The repeated linear search through the whole mountinfo (which can have
thousands of records) is inefficient. Instead, let's just
1 collect all the relevant records (only those mount points
that can be a parent of `source`)
2 find the record with the longest mountpath, return its data
This was tested manually with something like
```go
func TestGetSourceMount(t *testing.T) {
mnt, flags, err := getSourceMount("/sys/devices/msr/")
assert.NoError(t, err)
t.Logf("mnt: %v, flags: %v", mnt, flags)
}
```
...but it relies on having a specific mount points on the system
being used for testing.
[v2: add unit tests for ParentsFilter]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 871c957242df9f8c74faf751a2f14eb5178d4140
Component: engine
Use mount.SingleEntryFilter as we're only interested in a single entry.
Test case data of TestShouldUnmountRoot is modified accordingly, as
from now on:
1. `info` can't be nil;
2. the mountpoint check is not performed (as SingleEntryFilter
guarantees it to be equal to daemon.root).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: d3ebcde82aae79be8cbecab365367b17adac4b3e
Component: engine
Functions `GetMounts()` and `parseMountTable()` return all the entries
as read and parsed from /proc/self/mountinfo. In many cases the caller
is only interested only one or a few entries, not all of them.
One good example is `Mounted()` function, which looks for a specific
entry only. Another example is `RecursiveUnmount()` which is only
interested in mount under a specific path.
This commit adds `filter` argument to `GetMounts()` to implement
two things:
1. filter out entries a caller is not interested in
2. stop processing if a caller is found what it wanted
`nil` can be passed to get a backward-compatible behavior, i.e. return
all the entries.
A few filters are implemented:
- `PrefixFilter`: filters out all entries not under `prefix`
- `SingleEntryFilter`: looks for a specific entry
Finally, `Mounted()` is modified to use `SingleEntryFilter()`, and
`RecursiveUnmount()` is using `PrefixFilter()`.
Unit tests are added to check filters are working.
[v2: ditch NoFilter, use nil]
[v3: ditch GetMountsFiltered()]
[v4: add unit test for filters]
[v5: switch to gotestyourself]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: bb934c6aca3e77541dd4fd51b9ab2706294dadda
Component: engine
The unit test is checking that setting of non-default StopTimeout
works, but it checked the value of StopSignal instead.
Amazingly, the test was working since the default StopSignal is SIGTERM,
which has the numeric value of 15.
Fixes: commit e66d21089 ("Add config parameter to change ...")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 743f5afcd13659e2b6b76791415876797e2ede52
Component: engine
This moves the platform specific stuff in a separate package and keeps
the `volume` package and the defined interfaces light to import.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 6a70fd222b95643a8a6b88e2634d5f085ae4122a
Component: engine
All `plugins` tests that require an `ExperimentalDaemon` are migrated
to `integration/plugin/*` and start an experimental daemon to test on
it.
The end goal being to remove the `experimental` build.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 69bab3832ce1d72dd1d3ed2f2b53143ec9ff102c
Component: engine
This makes sure that if the daemon root was already a self-binded mount
(thus meaning the daemonc only performed a remount) that the daemon does
not try to unmount.
Example:
```
$ sudo mount --bind /var/lib/docker /var/lib/docker
$ sudo dockerd &
```
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: c403f0036b9945bd58a84e2c93f794ed9861fe99
Component: engine
The primary need for this is x/net/context now is just a type alias to
the stdlib context package.
This makes issues with conflicts between "golang.org/x/net/context" and
the stdib "context" go away (primarily a concern in interface
definitions/implementations).
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 848ef1da8b0d1c01a84fbe215f24f00fa741690d
Component: engine
Also starts to create more "poll/check" function to `internal/test/daemon`.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: aa0db6f9e1d0fd0420271f419d0d225089a95bbd
Component: engine
Instead of using a global store for volume drivers, scope the driver
store to the caller (e.g. the volume store). This makes testing much
simpler.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 977109d808ae94eb3931ae920338b1aa669f627e
Component: engine
Since the volume store already provides this functionality, we should
just use it rather than duplicating it.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 63826e291ba3b88443b64802084bbb3931857b56
Component: engine
This unblocks the client to take other restore requests and makes sure
that a long/stuck request can't block the client forever.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 806700e410a1b30aa1033d71d825e57880c75298
Component: engine
… and change a bit the method signature
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 42f6fdf059807ecbe66cce90676021363a541154
Component: engine
Porting helpers from `integration-cli/daemon.Daemon` to this struct
and use the API instead of the cli.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 9722214c8add2a83f6c4974848139cadc79d9f0e
Component: engine
Update all build stages to output artefacts to /build
Signed-off-by: Jeff Hajewski <jeff.hajewski@gmail.com>
Upstream-commit: 00555f7b261111084b966920da607a504b068953
Component: engine
- Move go package used by both `integration-cli` and `integration` to
`internal/test/fixtures`.
- Remove fixtures that are not used anymore (moved to `docker/cli` a
while ago) : deploy, notary, secrets.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 5f56503f583f21d655394f755f71849381bd58c7
Component: engine
This use case is currently _working correctly_, which is nice, but there is no documentation to be found about it. This PR fixes that.
Signed-off-by: Per Lundberg <perlun@gmail.com>
Upstream-commit: 82c23208ee
Component: cli
This fix converts some `client.ContainerCreate` to `container.Create`,
and removes some unneeded `name` fields when test containers are created.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: ab9bb47b05b1dde445a5e4ba78ae97303208dc8b
Component: engine
This fix migrates image tag tests from integration-cli to api tests.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 9bcb960508a6066811cffcca1e35ca44d7f1cf94
Component: engine
- Move the code from `integration-cli` to `internal/test`.
- Use `testingT` and `assert` when creating the registry.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 66de2e6e3b6d927a3396743cd7c363aa9f7b776e
Component: engine
In Go 1.10.1, the Deps for pkg\tarsum\tarsum_test.go are empty ([]) and
the PowerShell script ends up setting its import list to a string value
of False instead of an empty array. This can be remedied by forcing the
result to be an array, by concatenating to an empty array (@() + ...)
Signed-off-by: John Stephens <johnstep@docker.com>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit ec3f9230d70506c536a24e844da0f0b3af9b43f6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ee508d47c3917b423b6caae0ad2867c4f6ffe696
Component: engine
Add synchronization around adding logs to a plugin
and reading those logs. Without the follow configuration,
a race occurs between go routines to add the logs into
the plugin and read the logs out of the plugin. This
adds a function to synchronize the action to avoid the
race.
Removes use of file for buffering, instead buffering whole
messages so log count can be checked discretely.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c208f1c8a8d57f9a7f48f63345e77146774aa7a6
Component: engine
Prevent changing the tar output by setting the format to
PAX and keeping the times truncated.
Without this change the archiver will produce different tar
archives with different hashes with go 1.10.
The addition of the access and changetime timestamps would
also cause diff comparisons to fail.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: fb170206ba12752214630b269a40ac7be6115ed4
Component: engine
Remove invalid flush commands, flush should only occur when file
has been completely written. This is already handle, remove these calls.
Ensure data gets written after EOF in correct order and before close.
Remove gname and uname from sum for hash compatibility.
Update tarsum tests for gname/uname removal.
Return valid length after eof.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a422774e593b33bd287d9890544ad9e09b380d8c
Component: engine
This fix is now upstream in Go 1.10
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 950a1aa3170cedabd904cdcfcfa70c7d551a5e16
Component: engine
Full diff
5c1218c956...c15b372ef2
Fixes a panic on concurrent read/write to a map.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 248aed5766ba330ab8cb2b10b03b6ce57dc64283
Component: engine
Docker daemon has a 16K buffer for log messages. If a message length
exceeds 16K, it should be split by the logger and merged at the
endpoint.
This change adds `PartialLogMetaData` struct for enhanced partial support
- LastPartial (bool) : indicates if this is the last of all partials.
- ID (string) : unique 32 bit ID. ID is same across all partials.
- Ordinal (int starts at 1) : indicates the position of msg in the series of partials.
Also, the timestamps across partials in the same.
Signed-off-by: Anusha Ragunathan <anusha.ragunathan@docker.com>
Upstream-commit: 0b4b0a7b5d5de8cb575b666312fceaa2cd58e658
Component: engine
When the authz response buffer limit is hit, perform a flush.
This prevents excessive buffer sizes, especially on large responses
(e.g. `/containers/<id>/archive` or `/containers/<id>/export`).
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 74f8e47352e71aad4015d8d9dea8f16e7a055863
Component: engine
This remove the daemon.Swarm construction by make the new test Daemon
struct aware of swarm.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 83d18cf4e3e84055f7034816eed2a10c04e777ca
Component: engine
… making each folder/suites quicker to run
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: a3323d2e4349b7e8d449c6e571ca3d4aa3e53d63
Component: engine
All `Ipvlan` related test on `DockerSuite` and `DockerNetworkSuite`
are migrated to `ipvlan_test.go`.
The end goal being to remove the `experimental` builds.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 24f934751120ea420b7ba4d2e314df805f3eff06
Component: engine
… and do not use the `docker` cli in it. One of the reason of this
move is to not make `integration` package using legacy
`integration-cli` package.
Next move will be to support swarm within this package *and* provide
some helper function using the api (compared to the one using cli in
`integration-cli/daemon` package).
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: f0d277fe84a72b29c0d2d541c20d5a9c4d7e4884
Component: engine
Currently we hardcode the architecture to the `runtime.GOARCH` when
building a docker image, this will result in a confusing info if the
arch in the base image is different from the one on the host.
This PR takes use of the arch data from the base image during the build
process, thus we can get consistent arch info between the base image
and the finally built image.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 92b17b10bad1f1788419b70db5d6ed9cdf8d15ef
Component: engine
It does not make any sense to vary this based on whether the
rootfs is read only. We removed all the other mount dependencies
on read-only eg see #35344.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Upstream-commit: a729853bc712910574a7417f67764ec8c523928b
Component: engine
… and other cases too. Updating mergo fixes the bugs (but introduced a
slight behaviour change that had to be fixed too)
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 0122730faf
Component: cli
go1.9.5 (released 2018/03/28) includes fixes to the compiler, go command, and
net/http/pprof package. See the Go 1.9.5 milestone on the issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.9.5
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c62336593118020462d65d4e9784143e07de2da7
Component: packaging
We have seen a panic when re-joining a node to a swarm cluster. The
cause of the issue is unknown, so we just need to add a test for nil
objects and log when we get the condition. Hopefully this can prevent
the crash and we can recover the config at a later time.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: 454128c6e82cded211c1412e3eb350b1f7533ee2
Component: engine
go1.9.5 (released 2018/03/28) includes fixes to the compiler, go
command, and net/http/pprof package. See the Go 1.9.5 milestone on
the issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.9.5
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0b6f8a7eff325a683b10d64db363da2145aa1c36
Component: engine
* libnetwork#2121: Retry other external DNS servers on ServFail
* libnetwork#2125: Fix README flag and expose orphan network peers
* libnetwork#2126: Adding goreport card
* libnetwork#2130: Modify awk to use cut in check_ip_overlap
* libnetwork#2117: [Carry 1534] Improve scalabiltiy of bridge network isolation rules
Full changes: 2bf63300c5...5c1218c956
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Upstream-commit: b159da19734269c4a162763ebfa28dff07b703f3
Component: engine
Changes included:
- Ingress network should not be attachable
- [manager/state] Add fernet as an option for raft encryption
- Log GRPC server errors
- Log leadership changes at manager level
- [state/raft] Increase raft ElectionTick to 10xHeartbeatTick
- Remove the containerd executor
- agent: backoff session when no remotes are available
- [ca/manager] Remove root CA key encryption support entirely
- Fix agent logging race (fixes https://github.com/docker/swarmkit/issues/2576)
- Adding logic to restore networks in order
Also adds github.com/fernet/fernet-go as a new dependency
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 27749659d5a30999691e401a351221780a483099
Component: engine
Update the docs so that users don't use `echo` when creating
secrets from STDIN. `echo` adds a trailing new line, so users
will probably be confused when their passwords don't work.
Signed-off-by: Joao Fernandes <joao.fernandes@docker.com>
Upstream-commit: 5238f3e93e
Component: cli
This change fixes bug when $CURDIR contains special character e.g. '&&'.
Change-Id: I658eb60d97c2229cf8b4e0a61dbbecf3edc9d573
Signed-off-by: Fangming.Fang <fangming.fang@arm.com>
Upstream-commit: 83062593d7ce759bb96e271e30460afa4a695dba
Component: engine
commit 617c352e9225 "Don't create devices if in a user namespace"
introduced check, which meant to skip mknod operation when run
in user namespace, but instread skipped FIFO and socket files
copy.
Signed-off-by: Maxim Ivanov <ivanov.maxim@gmail.com>
Upstream-commit: 6f084f292932c464a30b56edb9edbe238bdcf0aa
Component: engine
There is no need to perform a separate losetup step; mount (even
the one in busybox!) is smart enough to set up a loopback device
all by itself (even without -o loop present!). More to say, while
doing this, it sets LO_FLAGS_AUTOCLEAR flag for the kernel to
delete the loopback device as soon as its fs is unmounted (this
is supposed to work since kernel 2.6.25).
Also, remove mount options (-t ext4, -o loop,rw) as they are
either defaults (rw) or mount is smart enough to figure out.
Leave -n so it won't do unnecessary write to container's /etc/mtab.
While at it, touch up some comments.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: a978fd22daefcc68d37c46177a545f7e1dfc6d9e
Component: engine
All `docker build` tests that require an `ExperimentalDaemon` are
migrated to `integration/build` package and start an experimental
daemon to test on it.
The end goal being to remove the `experimental` builds.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 183076e89df64928bd2e94ad0da9725b482367cd
Component: engine
A minor nit. `test01` never been created and used in
`TestDockerNetworkInspectCustomSpecified()` function, so correct it.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: f041953d04bffa2be05466173f02dd016c68286d
Component: engine
Makes sure that if the user cancels a request that the daemon stops
trying to traverse a directory.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 9d46c4c138d7b3f7778c13fe84857712bd6c97a9
Component: engine
Using the default PollSettings functions to adjust the timeout
value instead of changing the value each time when needed.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: b8912feeffcdfd489c9fc1212277840adac2719c
Component: engine
Add the default function per resource to override the `pollSettings`
which will be re-used where it's needed.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: ee6959addc5664a5c55765f2c721f84414ea4779
Component: engine
This call was added as part of commit a042e5a20 and at the time was
useful. sandbox.DisableService() basically calls
endpoint.deleteServiceInfoFromCluster() for every endpoint in the
sandbox. However, with the libnetwork change, endpoint.sbLeave()
invokes endpoint.deleteServiceInfoFromCluster(). The releaseNetwork()
call invokes sandbox.Delete() immediately after
sandbox.DisableService(). The sandbox.Delete() in turn ultimately
invokes endpoint.sbLeave() for every endpoint in the sandbox which thus
removes the endpoint's load balancing entry via
endpoint.deleteServiceInfoFromCluster(). So the call to
sandbox.DisableService() is now redundant.
It is noteworthy that, while redundant, the presence of the call would
not cause errors. It would just be sub-optimal. The DisableService()
call would cause libnetwork to down-weight the load balancing entries
while the call to sandbox.Delete() would cause it to remove the entries
immediately afterwards. Aside from the wasted computation, the extra
call would also propagate an extra state change in the networkDB gossip
messages. So, overall, it is much better to just avoid the extra
overhead.
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: c27417aa7de46daa415600b39fc8a9c411c8c493
Component: engine
This patch allows endpoints to complete servicing connections while
being removed from a service. The fix is entirely within libnetwork
and requires no changes to the moby codebase proper. It operates
by initially down-weighting a container endpoint in the load balancer
to 0 while keeping the endpoint present in the load balancer. This
allows traffic to continue to flow to the endpoint while preventing new
connections from going to the endpoint. This allows the container
to complete requests during the "stop_grace_period" and then exit when
finished without interruption of service.
This change requires propagating the status of disabled service
endpoints via the networkDB. Accordingly, the patch includes both code
to generate and handle service update messages. It also augments the
service structure with a ServiceDisabled boolean to convey whether an
endpoint should ultimately be removed or just disabled. This,
naturally, required a rebuild of the protocol buffer code.
The protocol buffer encoding is designed to support additions of fields
to messages in a backwards-compatible manner. Protocol buffer
unmarshalling code automatically skips past any fields that it isn't
aware of. As a result, an older moby daemon without this fix can
receive and will process correctly networkDB messages from newer moby
daemons with this patch.
As it turns out, the additional field is simply a bool that is otherwise
irrelevent on networkDB create and delete events. So its absence in
older moby daemon processing has no impact. However, the fix leverages
the "update" networkDB message which was previously unused in
libnetwork. Although older libnetwork implementations parse the message
cleanly, they will see the message as unexpected and as such issue a log
at error level indicating the receipt of such.
Other than this there should be no other negative impact for use of this
patch in mixed environments. (Although older mobys won't be able to
gracefully downgrade connections on their nodes of course.)
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: 50dbdeff9fd186bb0e9926996436e1f56529a831
Component: engine
This fixes an issue where the containerd client is cached in a container
object in libcontainerd and becomes stale after containerd is restarted.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 2c682d5209d1d1cdd232ce3909fa25b6023f9a5b
Component: engine
This does not bump the containerd binary.
Picks last commit before go1.10 switch, which is not currently supported
in moby.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 30cb23360e21c7914f41e69496abde4780bd7bc1
Component: engine
This was added as part of a53930a04fa81b082aa78e66b342ff19cc63cc5f with
the intent to sort the mounts in the plugin config, but this was sorting
*all* the mounts from the default OCI spec which is problematic.
In reality we don't need to sort this because we are only adding a
self-binded mount to flag it as rshared.
We may want to look at sorting the plugin mounts before they are added
to the OCI spec in the future, but for now I think the existing behavior
is fine since the plugin author has control of the order (except for the
propagated mount).
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: ec90839ca302ca53a7d55e4c7f79e7b4779f5e15
Component: engine
All `Macvlan` related test on `DockerSuite` and `DockerNetworkSuite`
are migrated to `macvlan_test.go`.
Also, as `macvlan` seems to be out of experimental, this removes
the *skip* when the run is not experimental (and doesn't start a
daemon with experimental either).
The end goal being to remove the `experimental` builds.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: ef5bc603266b9fa5df525319d67329ebc14a8ee7
Component: engine
Now all of the storage drivers use the field "storage-driver" in their log
messages, which is set to name of the respective driver.
Storage drivers changed:
- Aufs
- Btrfs
- Devicemapper
- Overlay
- Overlay 2
- Zfs
Signed-off-by: Alejandro GonzÃlez Hevia <alejandrgh11@gmail.com>
Upstream-commit: 9392838150f5495a63f33fed6570ae41f5a6d62e
Component: engine
When updating a service with the `--force` option, the `ForceUpdate`
property of the taskspec is incremented.
Stack deploy did not take this into account, and reset this
field to its default value (0), causing the service to be
re-deployed.
This patch copies the existing value before updating the service.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 76439457d2
Component: cli
receive operations on a nil channel will always block, this lead containerd not started.
Signed-off-by: Shukui Yang <yangshukui@huawei.com>
Upstream-commit: a50d9c0765d9861d226b323ffe190c0c99c3ac17
Component: engine
The list of changes here is too long to be described.
Maybe it's better to just rewrite the page from scratch, but at list this commit
makes some things better than before.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: e4405205ad
Component: cli
Use the proper man typesetting when describing the options
(i.e. use italics for variable parts, bold for literals).
Expand the description. Give better examples. Refer to mknod(2).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 70dea5c144
Component: cli
dm_task_deferred_remove is not supported by all distributions, due to
out-dated versions of devicemapper. However, in the case where the
devicemapper library was updated without rebuilding Docker (which can
happen in some distributions) then we should attempt to dynamically load
the relevant object rather than try to link to it.
This can only be done if Docker was built dynamically, for obvious
reasons.
In order to avoid having issues arise when dlsym(3) was unnecessary,
gate the whole dlsym(3) logic behind a buildflag that we disable by
default (libdm_dlsym_deferred_remove).
Signed-off-by: Aleksa Sarai <asarai@suse.de>
Upstream-commit: 98fe4bd8f1e35f8e498e268f653a43cbfa31e751
Component: engine
- Fix Windows absolute path detection on non-Windows. [docker/cli#1990](https://github.com/docker/cli/pull/1990)
- Fix Docker refusing to load key from delegation.key on Windows. [docker/cli#1968](https://github.com/docker/cli/pull/1968)
- Completion scripts updates for bash and zsh.
### Logging
- Fix for reading journald logs. [moby/moby#37819](https://github.com/moby/moby/pull/37819) [moby/moby#38859](https://github.com/moby/moby/pull/38859)
### Networking
- Prevent panic on network attach to a container with disabled networking. [moby/moby#39589](https://github.com/moby/moby/pull/39589)
- Fix service port for an application becomes unavailable randomly. [docker/libnetwork#2069](https://github.com/docker/libnetwork/pull/2069)
- Fix cleaning up `--config-only` networks after `--config-from` networks have ungracefully exited. [docker/libnetwork#2373](https://github.com/docker/libnetwork/pull/2373)
### Runtime
- Update to Go 1.11.13.
- Fix a potential engine panic when using XFS disk quota for containers. [moby/moby#39644](https://github.com/moby/moby/pull/39644)
### Swarm
- Fix "grpc: received message larger than max" errors. [moby/moby#39306](https://github.com/moby/moby/pull/39306)
- Fix an issue where nodes with lots of tasks could not be removed. [docker/swarmkit#2867](https://github.com/docker/swarmkit/pull/2867)
## 18.09.8 (2019-07-17)
### Client
- Fix Rollback config type interpolation. [docker/cli#1973](https://github.com/docker/cli/pull/1973)
### Runtime
- Fix [CVE-2019-13509](https://nvd.nist.gov/vuln/detail/CVE-2019-13509) in DebugRequestMiddleware: unconditionally scrub data field.
## 18.09.7 (2019-06-27)
### Builder
- Fix panic when building dockerfiles containing only comments. [moby/moby#38487](https://github.com/moby/moby/pull/38487)
- builder: add workaround for gcr auth issue. [moby/moby#38246](https://github.com/moby/moby/pull/38246)
* Performance optimizations in aufs and layer store for massively parallel container creation/removal. [moby/moby#39107](https://github.com/moby/moby/pull/39107)
* Update to containerd 1.2.6. [moby/moby#39016](https://github.com/moby/moby/pull/39016)
- Fix: [CVE-2018-15664](https://nvd.nist.gov/vuln/detail/CVE-2018-15664) symlink-exchange attack with directory traversal. [moby/moby#39357](https://github.com/moby/moby/pull/39357)
- Windows: fix support for `docker service create --limit-cpu`. [moby/moby#39190](https://github.com/moby/moby/pull/39190)
- Stop sorting uid and gid ranges in id maps. [moby/moby#39288](https://github.com/moby/moby/pull/39288)
### Logging
- Fix to allow large log lines for logger plugins. [moby/moby#39038](https://github.com/moby/moby/pull/39038)
## 18.09.6 (2019-05-02)
### Builder
- Fix `COPY` and `ADD` with multiple `<src>` do not invalidate cache if `DOCKER_BUILDKIT=1`. [moby/moby#38964](https://github.com/moby/moby/issues/38964)
### Networking
- Cleanup the cluster provider when the agent is closed. [docker/libnetwork#2354](https://github.com/docker/libnetwork/pull/2354)
- Windows: pick a random host port if the user does not specify a host port. [docker/libnetwork#2369](https://github.com/docker/libnetwork/pull/2369)
- Fix the stack informer's selector used to track deployment. [docker/cli#1794](https://github.com/docker/cli/pull/1794)
### Networking
- Fix `network=host` using wrong `resolv.conf` with `systemd-resolved`. [docker/engine#180](https://github.com/docker/engine/pull/180)
- Fix Windows ARP entries getting corrupted randomly under load. [docker/engine#192](https://github.com/docker/engine/pull/192)
### Runtime
- Fix stopped containers with restart policy showing as `Restarting`. [docker/engine#181](https://github.com/docker/engine/pull/181)
- Fix to use original process spec for execs. [docker/engine#178](https://github.com/docker/engine/pull/178)
### Swarm Mode
- Fix leaking task resources when nodes are deleted. [docker/engine#185](https://github.com/docker/engine/pull/185)
## 18.09.4 (2019-03-27)
### Builder
- Add validation for git ref so it can't be misinterpreted as a flag. [moby/moby#38944](https://github.com/moby/moby/pull/38944)
### Runtime
- Fix `docker cp` error with filenames greater than 100 characters. [moby/moby#38634](https://github.com/moby/moby/pull/38634)
- Fix layer/layer_store: ensure NewInputTarStream resources are released. [moby/moby#38413](https://github.com/moby/moby/pull/38413)
* Increase GRPC limit for GetConfigs. [moby/moby#38800](https://github.com/moby/moby/pull/38800)
* Update to containerd 1.2.5. [docker/engine#173](https://github.com/docker/engine/pull/173)
### Swarm Mode
- Fix nil pointer exception when joining node to swarm. [moby/moby#38618](https://github.com/moby/moby/issues/38618)
## 18.09.3 (2019-02-28)
### Networking
- Windows: avoid regeneration of network ids to prevent broken references to networks. [docker/engine#149](https://github.com/docker/engine/pull/149)
### Runtime
* Update to Go 1.10.8.
* Modify some of the names in the container name generator. [docker/engine#159](https://github.com/docker/engine/pull/159)
- When copying existing folder, ignore xattr set errors when the target filesystem doesn't support xattr. [docker/engine#135](https://github.com/docker/engine/pull/135)
- Graphdriver: fix "device" mode not being detected if "character-device" bit is set. [docker/engine#160](https://github.com/docker/engine/pull/160)
- Fix nil pointer derefence on failure to connect to containerd. [docker/engine#162](https://github.com/docker/engine/pull/162)
- Delete stale containerd object on start failure. [docker/engine#154](https://github.com/docker/engine/pull/154)
## 18.09.2 (2019-02-11)
### Security
- Update `runc` to address a critical vulnerability that allows specially-crafted containers to gain administrative privileges on the host. ([CVE-2019-5736](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-5736))
- Fix docker system prune doesn't accept until filter. [docker/engine#122](https://github.com/docker/engine/pull/122)
- Avoid unset credentials in containerd. [docker/engine#122](https://github.com/docker/engine/pull/122)
* Update to BuildKit 0.3.3. [docker/engine#122](https://github.com/docker/engine/pull/122)
+ Additional warnings for use of deprecated legacy overlay and devicemapper storage dirvers. [docker/engine#85](https://github.com/docker/engine/pull/85)
### Client
+ Add bash completion for experimental CLI commands (manifest). [docker/cli#1542](https://github.com/docker/cli/pull/1542)
- Fix yamldocs outputing `[flags]` in usage output. [docker/cli#1540](https://github.com/docker/cli/pull/1540)
- Fix setting default schema to tcp for docker host. [docker/cli#1454](https://github.com/docker/cli/pull/1454)
- prune: perform image pruning before build cache pruning. [docker/cli#1532](https://github.com/docker/cli/pull/1532)
- Fix bash completion for `service update --force`. [docker/cli#1526](https://github.com/docker/cli/pull/1526)
### Networking
- Fix iptables compatibility on debian. [docker/engine#107](https://github.com/docker/engine/pull/107)
### Packaging
+ Add docker.socket requirement for docker.service. [docker/docker-ce-packaging#276](https://github.com/docker/docker-ce-packaging/pull/276)
+ Add socket activation for RHEL-based distributions. [docker/docker-ce-packaging#274](https://github.com/docker/docker-ce-packaging/pull/274)
- Add libseccomp requirement for RPM packages. [docker/docker-ce-packaging#266](https://github.com/docker/docker-ce-packaging/pull/266)
### Runtime
* Add `/proc/asound` to masked paths. [docker/engine#126](https://github.com/docker/engine/pull/126)
* Update to containerd 1.2.1-rc.0. [docker/engine#121](https://github.com/docker/engine/pull/121)
+ Windows: allow process isolation. [docker/engine#81](https://github.com/docker/engine/pull/81)
- Windows: DetachVhd attempt in cleanup [docker/engine#113](https://github.com/docker/engine/pull/113)
- API: properly handle invalid JSON to return a 400 status. [docker/engine#110](https://github.com/docker/engine/pull/110)
- API: ignore default address-pools on API <1.39. [docker/engine#118](https://github.com/docker/engine/pull/118)
Restart docker with 'systemctl restart docker' to complete the update.`)
returnnil
}
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
