This test is failing on Windows currently:
```
11:59:47 --- FAIL: TestHealthKillContainer (8.12s)
11:59:47 health_test.go:57: assertion failed: error is not nil: Error response from daemon: Invalid signal: SIGUSR1
``
That test was added recently in https://github.com/moby/moby/pull/39454, but
rewritten in a commit in the same PR:
f8aef6a92f
In that rewrite, there were some changes:
- originally it was skipped on Windows, but the rewritten test doesn't have that skip:
```go
testRequires(c, DaemonIsLinux) // busybox doesn't work on Windows
```
- the original test used `SIGINT`, but the new one uses `SIGUSR1`
Analysis:
- The Error bubbles up from: 8e610b2b55/pkg/signal/signal.go (L29-L44)
- Interestingly; `ContainerKill` should validate if a signal is valid for the given platform, but somehow we don't hit that part; f1b5612f20/daemon/kill.go (L40-L48)
- Windows only looks to support 2 signals currently 8e610b2b55/pkg/signal/signal_windows.go (L17-L26)
- Upstream Golang looks to define `SIGINT` as well; 77f9b2728e/src/runtime/defs_windows.go (L44)
- This looks like the current list of Signals upstream in Go; 3b58ed4ad3/windows/types_windows.go (L52-L67)
```go
const (
// More invented values for signals
SIGHUP = Signal(0x1)
SIGINT = Signal(0x2)
SIGQUIT = Signal(0x3)
SIGILL = Signal(0x4)
SIGTRAP = Signal(0x5)
SIGABRT = Signal(0x6)
SIGBUS = Signal(0x7)
SIGFPE = Signal(0x8)
SIGKILL = Signal(0x9)
SIGSEGV = Signal(0xb)
SIGPIPE = Signal(0xd)
SIGALRM = Signal(0xe)
SIGTERM = Signal(0xf)
)
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit eeaa0b30d47e6b9dac8d8ea2ced6d5ce44c24463)
Signed-off-by: Dani Louca <dani.louca@docker.com>
Upstream-commit: f481d4c02366093b337e9aebfbbf23b1ff3968fe
Component: engine
Docker daemon always stops healthcheck before sending signal to a
container now. However, when we use "docker kill" to send signals
other than SIGTERM or SIGKILL to a container, such as SIGINT,
daemon still stops container health check though container process
handles the signal normally and continues to work.
Signed-off-by: Ruilin Li <liruilin4@huawei.com>
(cherry picked from commit da574f93432e600fda561da5e6983e7f69b364a9)
Signed-off-by: Dani Louca <dani.louca@docker.com>
Upstream-commit: afee55c3740644be40882d883d8287782f2bc6c2
Component: engine
For now, just verifying that an error is returned, but not checking the
error message itself, because those are not under our control, and may
change with different Go versions.
```
=== Failed
=== FAIL: opts TestParseDockerDaemonHost (0.00s)
hosts_test.go:87: tcp tcp:a.b.c.d address expected error "Invalid bind address format: tcp:a.b.c.d" return, got "parse tcp://tcp:a.b.c.d: invalid port \":a.b.c.d\" after host" and addr
hosts_test.go:87: tcp tcp:a.b.c.d/path address expected error "Invalid bind address format: tcp:a.b.c.d/path" return, got "parse tcp://tcp:a.b.c.d/path: invalid port \":a.b.c.d\" after host" and addr
=== FAIL: opts TestParseTCP (0.00s)
hosts_test.go:129: tcp tcp:a.b.c.d address expected error Invalid bind address format: tcp:a.b.c.d return, got parse tcp://tcp:a.b.c.d: invalid port ":a.b.c.d" after host and addr
hosts_test.go:129: tcp tcp:a.b.c.d/path address expected error Invalid bind address format: tcp:a.b.c.d/path return, got parse tcp://tcp:a.b.c.d/path: invalid port ":a.b.c.d" after host and addr
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit de1523d221)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: feb68f9055
Component: cli
go1.11.13 (released 2019/08/13) includes security fixes to the net/http and net/url packages.
See the Go 1.11.13 milestone on our issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.13
- net/http: Denial of Service vulnerabilities in the HTTP/2 implementation
net/http and golang.org/x/net/http2 servers that accept direct connections from untrusted
clients could be remotely made to allocate an unlimited amount of memory, until the program
crashes. Servers will now close connections if the send queue accumulates too many control
messages.
The issues are CVE-2019-9512 and CVE-2019-9514, and Go issue golang.org/issue/33606.
Thanks to Jonathan Looney from Netflix for discovering and reporting these issues.
This is also fixed in version v0.0.0-20190813141303-74dc4d7220e7 of golang.org/x/net/http2.
net/url: parsing validation issue
- url.Parse would accept URLs with malformed hosts, such that the Host field could have arbitrary
suffixes that would appear in neither Hostname() nor Port(), allowing authorization bypasses
in certain applications. Note that URLs with invalid, not numeric ports will now return an error
from url.Parse.
The issue is CVE-2019-14809 and Go issue golang.org/issue/29098.
Thanks to Julian Hector and Nikolai Krein from Cure53, and Adi Cohen (adico.me) for discovering
and reporting this issue.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9a7c1a1e5f0858ac54f1c7577c4fca8665f70496
Component: packaging
go1.11.13 (released 2019/08/13) includes security fixes to the net/http and net/url packages.
See the Go 1.11.13 milestone on our issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.13
- net/http: Denial of Service vulnerabilities in the HTTP/2 implementation
net/http and golang.org/x/net/http2 servers that accept direct connections from untrusted
clients could be remotely made to allocate an unlimited amount of memory, until the program
crashes. Servers will now close connections if the send queue accumulates too many control
messages.
The issues are CVE-2019-9512 and CVE-2019-9514, and Go issue golang.org/issue/33606.
Thanks to Jonathan Looney from Netflix for discovering and reporting these issues.
This is also fixed in version v0.0.0-20190813141303-74dc4d7220e7 of golang.org/x/net/http2.
net/url: parsing validation issue
- url.Parse would accept URLs with malformed hosts, such that the Host field could have arbitrary
suffixes that would appear in neither Hostname() nor Port(), allowing authorization bypasses
in certain applications. Note that URLs with invalid, not numeric ports will now return an error
from url.Parse.
The issue is CVE-2019-14809 and Go issue golang.org/issue/29098.
Thanks to Julian Hector and Nikolai Krein from Cure53, and Adi Cohen (adico.me) for discovering
and reporting this issue.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: aed09dc7eb
Component: cli
go1.11.13 (released 2019/08/13) includes security fixes to the net/http and net/url packages.
See the Go 1.11.13 milestone on our issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.13
- net/http: Denial of Service vulnerabilities in the HTTP/2 implementation
net/http and golang.org/x/net/http2 servers that accept direct connections from untrusted
clients could be remotely made to allocate an unlimited amount of memory, until the program
crashes. Servers will now close connections if the send queue accumulates too many control
messages.
The issues are CVE-2019-9512 and CVE-2019-9514, and Go issue golang.org/issue/33606.
Thanks to Jonathan Looney from Netflix for discovering and reporting these issues.
This is also fixed in version v0.0.0-20190813141303-74dc4d7220e7 of golang.org/x/net/http2.
net/url: parsing validation issue
- url.Parse would accept URLs with malformed hosts, such that the Host field could have arbitrary
suffixes that would appear in neither Hostname() nor Port(), allowing authorization bypasses
in certain applications. Note that URLs with invalid, not numeric ports will now return an error
from url.Parse.
The issue is CVE-2019-14809 and Go issue golang.org/issue/29098.
Thanks to Julian Hector and Nikolai Krein from Cure53, and Adi Cohen (adico.me) for discovering
and reporting this issue.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: d122605850fca5df1ff8babe7ee9f1dfed2a335b
Component: engine
1. Use "in-place" variables for if statements to limit their scope to
the respectful `if` block.
2. Report the error returned from sd_journal_* by using CErr().
3. Use errors.New() instead of fmt.Errorf().
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 20a0e58a794cfb9b1a1f757d222248e22555f7f0)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: b32bc11a00a82584138ae78e083930918dcb0b36
Component: engine
From the first glance, `docker logs --tail 0` does not make sense,
as it is supposed to produce no output, but `tail -n 0` from GNU
coreutils is working like that, plus there is even a test case
(`TestLogsTail` in integration-cli/docker_cli_logs_test.go).
Now, something like `docker logs --follow --tail 0` makes total
sense, so let's make it work.
(NOTE if --tail is not used, config.Tail is set to -1)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit dd4bfe30a8ac1b31630310090dc36ae3d9253444)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 4427414624c56c459431d3dbbde81f3396cb02b0
Component: engine
If we take a long time to process log messages, and during that time
journal file rotation occurs, the journald client library will keep
those rotated files open until sd_journal_process() is called.
By periodically calling sd_journal_process() during the processing
loop we shrink the window of time a client instance has open file
descriptors for rotated (deleted) journal files.
This code is modelled after that of journalctl [1]; the above explanation
as well as the value of 1024 is taken from there.
[v2: fix CErr() argument]
[1] https://github.com/systemd/systemd/blob/dc16327c48d/src/journal/journalctl.c#L2676
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit b73fb8fd5d521081c92b5c2cce334c21b2e0ff5f)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 48160719f2c5734b513aec7cedbbca0c72f4a085
Component: engine
TL;DR: simplify the code, fix --follow hanging indefinitely
Do the following to simplify the followJournal() code:
1. Use Go-native select instead of C-native polling.
2. Use Watch{Producer,Consumer}Gone(), eliminating the need
to have journald.closed variable, and an extra goroutine.
3. Use sd_journal_wait(). In the words of its own man page:
> A synchronous alternative for using sd_journal_get_fd(),
> sd_journal_get_events(), sd_journal_get_timeout() and
> sd_journal_process() is sd_journal_wait().
Unfortunately, the logic is still not as simple as it
could be; the reason being, once the container has exited,
journald might still be writing some logs from its internal
buffers onto journal file(s), and there is no way to
figure out whether it's done so we are guaranteed to
read all of it back. This bug can be reproduced with
something like
> $ ID=$(docker run -d busybox seq 1 150000); docker logs --follow $ID
> ...
> 128123
> $
(The last expected output line should be `150000`).
To avoid exiting from followJournal() early, add the
following logic: once the container is gone, keep trying
to drain the journal until there's no new data for at
least `waitTimeout` time period.
Should fix https://github.com/docker/for-linux/issues/575
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit f091febc942859ffbc881f3a3aa327366603ae65)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: be568f93432ed1399a0aacca5fbd2c1046ccbc85
Component: engine
1. The journald client library initializes inotify watch(es)
during the first call to sd_journal_get_fd(), and it make sense
to open it earlier in order to not lose any journal file rotation
events.
2. It only makes sense to call this if we're going to use it
later on -- so add a check for config.Follow.
3. Remove the redundant call to sd_journal_get_fd().
NOTE that any subsequent calls to sd_journal_get_fd() return
the same file descriptor, so there's no real need to save it
for later use in wait_for_data_cancelable().
Based on earlier patch by Nalin Dahyabhai <nalin@redhat.com>.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 981c01665bcb2c9fc5e555c5b976995f31c2a6b4)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 349e199eab5337f03b442f38720293143e1b1fca
Component: engine
In case the LogConsumer is gone, the code that sends the message can
stuck forever. Wrap the code in select case, as all other loggers do.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 79039720c8b7352691350bd56be3cc226d67f205)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 56a8a516127a2630c3a68ab31c416f223b91e9df
Component: engine
In case Tail=N parameter is requested, we need to show N lines.
It does not make sense to walk backwards one by one if we can
do it at once. Now, if Since=T is also provided, make sure we
haven't jumped too far (before T), and if we did, move forward.
The primary motivation for this was to make the code simpler.
This also fixes a tiny bug in the "since" implementation.
Before this commit:
> $ docker logs -t --tail=6000 --since="2019-03-10T03:54:25.00" $ID | head
> 2019-03-10T03:54:24.999821000Z 95981
After:
> $ docker logs -t --tail=6000 --since="2019-03-10T03:54:25.00" $ID | head
> 2019-03-10T03:54:25.000013000Z 95982
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit ff3cd167ea4d089b7695a263ba2fc4caa0a0750c)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 2a124db7da440f1efd4c2957320d8b25d9d9ce36
Component: engine
Minor code simplification.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit e8f6166791c097deb15c39f8dddf6f97be65b224)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 1d336dc53bd7bf5941596ffeb253d102de609a51
Component: engine
Clean up a deferred function call in the journal reading logic.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
(cherry picked from commit 1ada3e85bf89201910c28f2ff6892c00cee0f137)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: e700930ca521d0c004b6a3ed8bdd35a2d538aa15
Component: engine
As in other similar drivers (jsonlog, local), use a set
(i.e. `map[whatever]struct{}`), making the code simpler.
While at it, make sure we remove the reader from the set
after calling `ProducerGone()` on it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit b2b169f13f681cd0d591ccb06d6cfff97933db77)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: fe85c72a2eac4cbf249d2c4c754684bb447eefdd
Component: engine
There are a few more places, apparently, that List operations against
Swarm exist, besides just in the List methods. This increases the max
received message size in those places.
Signed-off-by: Drew Erny <drew.erny@docker.com>
(cherry picked from commit a84a78e9767d82abd4744dad9ce4fb3f64141a8f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 41da428d065ab15fd2d8aba1fbd0d0056136a43b
Component: engine
full diff: 19e791fd6d...142a73731c
included:
- docker/swarmkit#2872 [19.03 backport] Only update non-terminal tasks on node removal
- backport of docker/swarmkit#2867 Only update non-terminal tasks on node removal
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 7fcfdbaab6d8c6a2d55ad7b72a851c5e92b360ac
Component: engine
The Golang base images switch to buster, which causes some breakage
in networking and packages that are no longer available; (`btrfs-tools`
is now an empty package, and `libprotobuf-c0-dev` is gone).
Some of out tests also start faiilng on stretch, and will have to be
investigated further;
```
15:13:06 --- FAIL: TestRenameAnonymousContainer (3.37s)
15:13:06 rename_test.go:168: assertion failed: 0 (int) != 1 (inspect.State.ExitCode int): container a7fe866d588d65f353f42ffc5ea5288e52700384e1d90850e9c3d4dce8657666 exited with the wrong exitcode:
15:13:38 --- FAIL: TestHostnameDnsResolution (2.23s)
15:13:38 run_linux_test.go:128: assertion failed:
15:13:38 --- ←
15:13:38 +++ →
15:13:38 @@ -1 +1,2 @@
15:13:38 +ping: bad address 'foobar'
15:13:38
15:13:38
15:13:38 run_linux_test.go:129: assertion failed: 0 (int) != 1 (res.ExitCode int)
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ed672bb523cb255d0b2b79837d9c45a7c3255000)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 3c1bc29b2c372d0f1e37dc8aac33cdc01b623aa0
Component: engine
This check was used to make sure we don't bump Go versions independently
(Linux/Windows). The Dockerfile switched to using a build-arg to allow
overriding the Go version, which rendered this check non-functional.
It also fails if Linux versions use a specific variant of the image;
08:41:31 ERROR: Failed 'ERROR: Mismatched GO versions between Dockerfile and Dockerfile.windows. Update your PR to ensure that both files are updated and in sync. ${GO_VERSION}-stretch ${GO_VERSION}' at 07/20/2019 08:41:31
08:41:31 At C:\gopath\src\github.com\docker\docker\hack\ci\windows.ps1:448 char:9
08:41:31 + Throw "ERROR: Mismatched GO versions between Dockerfile and D ...
08:41:31 + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This patch fixes the check by looking for the value of `GO_VERSION` instead
of looking at the `FROM` line (which is harder to parse).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4fa57a8191b1d23c6466725b688519f83c0ac5dd)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: a2d32c5654a206954fd0d9a5fd08abd8c03d4523
Component: engine
This was an oversight when changing the Dockerfile to use a build-arg;
the Windows Dockerfile downloads the Go binaries, which never have a
trailing `.0`.
This patch makes sure that the trailing zero (if any) is removed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c5bd6e3dc7680d6c683496f63dafb1f30f87eaa7)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 8f1a27c51f93698b5fe17f0fb043f47794e11cca
Component: engine
This allows overriding the version of Go without making modifications in the
source code, which can be useful to test against multiple versions.
For example:
make GO_VERSION=1.13beta1 shell
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c6281bc4383b7f9eab617fd73601e8594c93365b)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Conflicts:
* Dockerfile, Dockerfile.e2e, Dockerfile.simple, Dockerfile.windows:
(due to Go version difference, missing CROSS etc.)
Upstream-commit: d9ba337adb0c3880acd34ffe9b0a15717990876b
Component: engine
Protect access to q.quotas map, and lock around changing nextProjectID.
Techinically, the lock in findNextProjectID() is not needed as it is
only called during initialization, but one can never be too careful.
Fixes: 52897d1c092 ("projectquota: utility class for project quota controls")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 1ac0a66a64a906911d0708cd0e5fa397a2f0b595)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 7027bb9bedae63879c1e41894739ba0ea2deedc1
Component: engine
The docker-in-docker image now enables TLS by default (added in
docker-library/docker#166), which complicates testing in our
environment, and isn't needed for the tests we're running.
This patch sets the `DOCKER_TLS_CERTDIR` to an empty value to
disable TLS.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b1a3c1aad1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9c8ac0a123
Component: cli
The edge channel is deprecated and no longer updated
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 08fd6dd63c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a88330d9dd
Component: cli
[18.09] Initialize nss libraries in Glibc so that the dynamic libraries are l…
Upstream-commit: 8ff992ae26ecedb7b80ee71be902d0fbc0879861
Component: engine
The fastly cdn mirror we're using also mirrors the debian security
repository;
```
Welcome to deb.debian.org (fastly instance)!
This is deb.debian.org. This service provides mirrors for the following Debian archive repositories:
/debian/
/debian-debug/
/debian-ports/
/debian-security/
The server deb.debian.org does not have packages itself, but the name has SRV records in DNS that let apt in stretch and later find places.
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c8f43b5f6f7c83cfb5570f7f013c7efaa430d285)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Conflicts:
* Dockerfile (GO_VERSION value differs, and CROSS (#39010) is absent)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 6e603e957eec3a46db4d8d69c3a7946c4d2e9c59
Component: engine
As of Alpine Linux 3.3 (or 3.2?) there exists a new --no-cache
option for apk. It allows users to install packages with an index
that is updated and used on-the-fly and not cached locally.
This avoids the need to use --update and remove /var/cache/apk/*
when done installing packages.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 3b44dd66a4b316049df86521c491b50466d9b55d)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: a4ae2af0e59018a309d4bc71ced729b34e906564
Component: engine
This helps merge conflicts in situations where downstream
projects have additional dependencies.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8c5460a2cc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 7b3d023163
Component: cli
To make it better readable, and to encourage pinning
by sha, but "align" to a tagged release.
similar to 6026ce4a8b (#1822)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1764ffafca
Component: cli
I would like to use the regex matcher
Signed-off-by: Ian Campbell <ijc@docker.com>
(cherry picked from commit 986196e3e3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c1c3add698
Component: cli
Path-specific rules were removed, so this is no longer used.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 530e63c1a61b105a6f7fc143c5acb9b5cd87f958)
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit f8a0f26843bc5aff33cf9201b75bd4bdbb48a3ad)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a11c3098a3d5106b2d7c90d971c9300099556a49
Component: engine
Commit 77b8465d7e68ca102d7aae839c7b3fe0ecd28398 added a secret update
endpoint to allow updating labels on existing secrets. However, when
implementing the endpoint, the DebugRequestMiddleware was not updated
to scrub the Data field (as is being done when creating a secret).
When updating a secret (to set labels), the Data field should be either
`nil` (not set), or contain the same value as the existing secret. In
situations where the Data field is set, and the `dockerd` daemon is
running with debugging enabled / log-level debug, the base64-encoded
value of the secret is printed to the daemon logs.
The docker cli does not have a `docker secret update` command, but
when using `docker stack deploy`, the docker cli sends the secret
data both when _creating_ a stack, and when _updating_ a stack, thus
leaking the secret data if the daemon runs with debug enabled:
1. Start the daemon in debug-mode
dockerd --debug
2. Initialize swarm
docker swarm init
3. Create a file containing a secret
echo secret > my_secret.txt
4. Create a docker-compose file using that secret
cat > docker-compose.yml <<'EOF'
version: "3.3"
services:
web:
image: nginx:alpine
secrets:
- my_secret
secrets:
my_secret:
file: ./my_secret.txt
EOF
5. Deploy the stack
docker stack deploy -c docker-compose.yml test
6. Verify that the secret is scrubbed in the daemon logs
DEBU[2019-07-01T22:36:08.170617400Z] Calling POST /v1.30/secrets/create
DEBU[2019-07-01T22:36:08.171364900Z] form data: {"Data":"*****","Labels":{"com.docker.stack.namespace":"test"},"Name":"test_my_secret"}
7. Re-deploy the stack to trigger an "update"
docker stack deploy -c docker-compose.yml test
8. Notice that this time, the Data field is not scrubbed, and the base64-encoded secret is logged
DEBU[2019-07-01T22:37:35.828819400Z] Calling POST /v1.30/secrets/w3hgvwpzl8yooq5ctnyp71v52/update?version=34
DEBU[2019-07-01T22:37:35.829993700Z] form data: {"Data":"c2VjcmV0Cg==","Labels":{"com.docker.stack.namespace":"test"},"Name":"test_my_secret"}
This patch modifies `maskSecretKeys` to unconditionally scrub `Data` fields.
Currently, only the `secrets` and `configs` endpoints use a field with this
name, and no other POST API endpoints use a data field, so scrubbing this
field unconditionally will only scrub requests for those endpoints.
If a new endpoint is added in future where this field should not be scrubbed,
we can re-introduce more fine-grained (path-specific) handling.
This patch introduces some change in behavior:
- In addition to secrets, requests to create or update _configs_ will
now have their `Data` field scrubbed. Generally, the actual data should
not be interesting for debugging, so likely will not be problematic.
In addition, scrubbing this data for configs may actually be desirable,
because (even though they are not explicitely designed for this purpose)
configs may contain sensitive data (credentials inside a configuration
file, e.g.).
- Requests that send key/value pairs as a "map" and that contain a
key named "data", will see the value of that field scrubbed. This
means that (e.g.) setting a `label` named `data` on a config, will
scrub/mask the value of that label.
- Note that this is already the case for any label named `jointoken`,
`password`, `secret`, `signingcakey`, or `unlockkey`.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c7ce4be93ae8edd2da62a588e01c67313a4aba0c)
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 73db8c77bfb2d0cbdf71ce491f3d3e66c9dd5be6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 32b40c53662e733b4627b0b303c71b52484a31f4
Component: engine
1. Adds `docker events` description info on the two scope types of events.
2. Adds `docker events` note in two places about backlog limit of event log.
Further info and background info in Issue 727
Signed-off-by: Bret Fisher <bret@bretfisher.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 988b9a0d96)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 74d51dc13b
Component: cli
When deploying a stack using a relative path as bind-mount
source in the compose file, the CLI converts the relative
path to an absolute path, relative to the location of the
docker-compose file.
This causes a problem when deploying a stack that uses
an absolute Windows path, because a non-Windows client will
fail to detect that the path (e.g. `C:\somedir`) is an absolute
path (and not a relative directory named `C:\`).
The existing code did already take Windows clients deploying
a Linux stack into account (by checking if the path had a leading
slash). This patch adds the reverse, and adds detection for Windows
absolute paths on non-Windows clients.
The code used to detect Windows absolute paths is copied from the
Golang filepath package;
1d0e94b1e1/src/path/filepath/path_windows.go (L12-L65)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d6dd08d568)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a757fad956
Component: cli
go1.11.11 (released 2019/06/11) includes a fix to the crypto/x509 package.
See the Go 1.11.11 milestone on the issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.11
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f33c398df1eabda69ca05ff6c5c45351790a8696
Component: packaging
go1.11.3 (released 2018/12/14)
- crypto/x509: CPU denial of service in chain validation golang/go#29233
- cmd/go: directory traversal in "go get" via curly braces in import paths golang/go#29231
- cmd/go: remote command execution during "go get -u" golang/go#29230
See the Go 1.11.3 milestone on the issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 517a30b48d7b483e27eb3c3328356a8e83272988)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 17565b93d361b1b189962b3e98c431316a7cc628
Component: packaging
[18.09] switch to balenalib/rpi-raspbian because resin/rpi-raspbian is deprecated
Upstream-commit: e8cb60baf6abb7b2cd50c9991056891be462b839
Component: packaging
version 2.9 is really old; this sets the same minimal version
as is used for the containerd.io package
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a246b19b0723d41a8e58ff42d3af52fd57a7e94d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 95dea749d4cb7e474194bbe209f052e94da5df2e
Component: packaging
This code was attempting to check Linux file permissions
to determine if the key was accessible by other users, which
doesn't work, and therefore prevented users on Windows
to load keys.
Skipping this check on Windows (correspinding tests
were already skipped).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 15d361fd77)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 6be8fce6f8
Component: cli
Full diff: https://github.com/golang/go/compare/go1.11.5...go1.11.9
go1.11.6 (released 2019/03/14) includes fixes to cgo, the compiler, linker,
runtime, go command, and the crypto/x509, encoding/json, net, and net/url
packages. See the Go 1.11.6 milestone on our issue tracker for details.
https://github.com/golang/go/issues?q=milestone%3AGo1.11.6
go1.11.7 (released 2019/04/05) includes fixes to the runtime and the net
packages. See the Go 1.11.7 milestone on our issue tracker for details.
https://github.com/golang/go/issues?q=milestone%3AGo1.11.7
go1.11.8 (released 2019/04/08) was accidentally released without its
intended fix. It is identical to go1.11.7, except for its version number.
The intended fix is in go1.11.9.
go1.11.9 (released 2019/04/11) fixes an issue where using the prebuilt
binary releases on older versions of GNU/Linux led to failures when linking
programs that used cgo. Only Linux users who hit this issue need to update.
See golang/go#31293 for details
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2cc1df06ee264ab5d1769bd4aae9e417feda9402
Component: engine
go1.11.4 (released 2018/12/14) includes fixes to cgo, the compiler, linker,
runtime, documentation, go command, and the net/http and go/types packages. It
includes a fix to a bug introduced in Go 1.11.3 that broke go get for import
path patterns containing "...".
See the Go 1.11.4 milestone for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.4+label%3ACherryPickApproved
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3770f386477e5eea4e6ba02516393a1edd5cd28b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e05367a1ba3cbf2cd7f99fef73908b900b57b929
Component: engine
go1.11.13 (released 2018/12/14)
- crypto/x509: CPU denial of service in chain validation golang/go#29233
- cmd/go: directory traversal in "go get" via curly braces in import paths golang/go#29231
- cmd/go: remote command execution during "go get -u" golang/go#29230
See the Go 1.11.3 milestone on the issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6b7c093b0de21d574ce120aee891e60187749174)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 19d37c9a337e82e0e8ce0ff28271739e8ec78e83
Component: engine
Includes fixes to the compiler, linker, documentation, go command, and the
database/sql and go/types packages. See the Go 1.11.2 milestone on the issue
tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e80ee5206e6e62521c8149381d06bbb0bea88169)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f9a71f917f0979742b1d7ee606fb596e873d773c
Component: engine
go1.11.1 (released 2018/10/01) includes fixes to the compiler,
documentation, go command, runtime, and the crypto/x509, encoding/json,
go/types, net, net/http, and reflect packages.
See the Go 1.11.1 milestone on our issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.1
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0347751117513312be72e8b0d03f32319027f145)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 6b397d155bf464a94fa05620a485d969edf42393
Component: engine
As per https://github.com/etcd-io/etcd/blob/fa57f7fbc787b4/Gopkg.lock
List of packages required by subset of etcd used is provided by:
go list -f '{{join .Deps "\n"}}' \
github.com/docker/docker/vendor/github.com/coreos/etcd/... \
| grep -F . | grep -v coreos/etcd | sort | uniq
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 7008ac01fa4dbde9a069cc647a09c982f3648fc3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c4bf0803154de1b26ee3437c7fd9307722e2a5e4
Component: engine
We can do that now as we're no longer carrying archive/tar.
Note that latest vndr removes vendor/ subdir so we don't have to,
thus the change in hack/validate/vendor.
While at it, re-run a new vndr version to make sure everything
that should be there is.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit ce858feb6ac124b3a42400b857e480ef98710164)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2f2588f137f642b0c1520e8a41ec731040d0b336
Component: engine
It's that time of year again! Go 1.11 is released, time to use it.
This commit also
* removes our archive/tar fork, since upstream archive/tar
is fixed for static builds, and osusergo build tag is set.
* removes ENV GO_VERSION from Dockerfile as it's not needed
anymore since PR #37592 is merged.
[v2: switch to beta2]
[v3: switch to beta3]
[v4: rc1]
[v5: remove ENV GO_VERSION as PR #37592 is now merged]
[v6: rc2]
[v7: final!]
[v8: use 1.11.0]
[v9: back to 1.11]
[v8: use 1.11.0]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 10fd0516b9f9f04d0f0e2c0755e704303f1a487f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ebf396050d6e977df2669d5e7d3f38098719f7c2
Component: engine
We would like to use a version with .0 suffix (like 1.11.0) in
Dockerfile, so that once a .1 version is out (like 1.11.1) we
won't accidentally switch to it.
Unfortunately it's not possible to use .0 suffix currently
as it breaks the check in make.ps1. This patch fixes that.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 8ad648b59a2993156586458cc9ff8dcbc89b86e4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: dc5371b4134dd6474a467308da85daaa27914a64
Component: engine
Instead of installing golang from sources, it's easier to use
golang image which is based on Debian Stretch.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 32a23311033dc5bfdfbc80c54a159cc92990efd2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2e82d410bef320096e316ef7a3db8797694f88fe
Component: engine
To include the following backported fix:
https://github.com/kolyshkin/ugorji-go/commit/1cf431c13dec46596
which should fix this:
> 13:40:53 vendor/github.com/ugorji/go/codec/gen-helper.generated.go:1:
> possible malformed +build comment%!(EXTRA []interface {}=[])
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 09921ca33fb1cdb7648143e787a2a6222d844eb0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ca9d80ea6a07f54b6171cd3cb1a5125948fd5fdc
Component: engine
This should eliminate a bunch of new (go-1.11 related) validation
errors telling that the code is not formatted with `gofmt -s`.
No functional change, just whitespace (i.e.
`git show --ignore-space-change` shows nothing).
Patch generated with:
> git ls-files | grep -v ^vendor/ | grep .go$ | xargs gofmt -s -w
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 9b0097a69900009ab5c2480e047952cba60462a7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ee28567c7066368207a947e02c6242db7a4adb16
Component: engine
This is set only if it is not already set.
This should give a little speedup to CI builds.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 1275a001a68722494d090d5beca6749a83710cc2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e64cd6abed308a26d9048d4d0e4c52207c2ad5df
Component: engine
comparing PR commit(s) to HEAD of moby/moby master branch and if founds
new (or renamed) integration tests will run stress tests for them.
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
(cherry picked from commit 8a8fd37f6fb53716cb4b3a7e93e1e3cf385927e2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e5c0923b277055c977994ebfdc689346c3104a16
Component: engine
This patch;
- adds support for multiple newlines
- removes the 1-space indentation of wrapped lines
- allows numerical characters in targets (0-9)
Given these targets:
```Makefile
.PHONY: foobar
foobar: ## runs the foobar lorum ipsum.\nand so pn\nand so on
echo foobar
.PHONY: e2e-tests
e2e-tests: ## runs the end-to-end tests
echo e2e-tests
```
Before this change, the output of `make help` was
```
foobar runs the foobar lorum ipsum.
and so pn\nand so on
```
After this change, the output is:
```
foobar runs the foobar lorum ipsum.
and so pn
and so on
e2e-tests runs the end-to-end tests
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 202c9d8c98614e7cce2017f5c99d3d783fe8b509)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 733b86683f347f922affc714d305d9f03fe9b47e
Component: engine
build the final stage of the Dockerfile (including COPY ...) if no BIND_DIR
is used.
if BIND_DIR is used, build the dev stage, thus skipping the COPY.
Original author: @thaJeztah
Signed-off-by: Mohammad Nasirifar <farnasirim@gmail.com>
(cherry picked from commit e6d7df2e5d313800414b955e10a26d6687e7a1bf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e5a039169f389120be3fbc2a9db45ef8bc99eae5
Component: engine
Through the env variable of the same name.
The idea here is pretty simple: I/O perf on native mounted disks
on non-Linux (notably Mac OS) is just terrible, thus making it
a real pain to develop: one has to choose between re-building
the image after every single change (eg to run a test) or just
work directly inside the same container (eg with vim, but even then
one would have to re-configure their dev container every time
it gets destroyed - containers, after all, are not supposed to
be long-lived).
Allowing to override `DOCKER_MOUNT` makes it easy for everyone
to decide what their volume/syncing strategy is; for example
one can choose to use [docker-sync](https://github.com/EugenMayer/docker-sync)
This patch won't change anything for anyone who doesn't
set the `DOCKER_MOUNT` env variable in their environment.
Signed-off-by: Jean Rouge <jer329@cornell.edu>
(cherry picked from commit aea6fdf3d340835a1b0af208839ce42ace3a5b89)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ecc423df7fde7f0edc963f03d70da133f03bb63c
Component: engine
Looks like -i (together with DOCKER_INCREMENTAL_BINARY etc)
were used to get faster incremental builds.
Nowdays (since Go 1.10) this is no longer the case, as
go build cache is used [1]. Here's a quote:
> You do not have to use "go test -i" or "go build -i" or
> "go install" just to get fast incremental builds. We will
> not have to teach new users those workarounds anymore.
> Everything will just be fast.
To enable go cache between builds, add a volume for /root/.cache.
[1] https://groups.google.com/forum/#!msg/golang-dev/qfa3mHN4ZPA/X2UzjNV1BAAJ
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit bdcd81d3301a053eefc320de16ac842ec47ed459)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0f190f798f7b9f90bd008fe8fb0fc087ec965642
Component: engine
[18.09 backport] Consider WINDOWS_BASE_IMAGE_TAG override when setting Windows base image for tests
Upstream-commit: bb54c5bf2a3ab1c698fda6b3d67773d25755ae45
Component: engine
Previously, getWalkRoot("/", "foo") would return "//foo"
Now it returns "/foo"
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 7410f1a859063d4ed3d8fca44f27bdde4c2cb5a3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8677bbe3f31a8e4516e1ba413ce1063c803ea10c
Component: engine
Before 7a7357da, archive.TarResourceRebase was being used to copy files
and folders from the container. That function splits the source path
into a dirname + basename pair to support copying a file:
if you wanted to tar `dir/file` it would tar from `dir` the file `file`
(as part of the IncludedFiles option).
However, that path splitting logic was kept for folders as well, which
resulted in weird inputs to archive.TarWithOptions:
if you wanted to tar `dir1/dir2` it would tar from `dir1` the directory
`dir2` (as part of IncludedFiles option).
Although it was weird, it worked fine until we started chrooting into
the container rootfs when doing a `docker cp` with container source set
to `/` (cf 3029e765).
The fix is to only do the path splitting logic if the source is a file.
Unfortunately, 7a7357da added support for LCOW by duplicating some of
this subtle logic. Ideally we would need to do more refactoring of the
archive codebase to properly encapsulate these behaviors behind well-
documented APIs.
This fix does not do that. Instead, it fixes the issue inline.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 171538c190ee3a1a8211946ab8fa78cdde54b47a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 70a837138aa1c5c0260a005b470ac0c5717984b4
Component: engine
- moby/buildkit#952 [18.09 backport] Have parser error on dockerfiles without instructions
- backport of moby/buildkit#771 Have parser error on dockerfiles without instructions
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a1d76e0585c498f5167b28b9909fd8c88fd3370d
Component: engine
Make sure adapter.removeNetworks executes during task Remove
adapter.removeNetworks was being skipped for cases when
isUnknownContainer(err) was true after adapter.remove was executed
This fix eliminates the nil return case forcing the function
to continue executing unless there is a true error
Fixes https://github.com/moby/moby/issues/39225
Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
(cherry picked from commit 70fa7b6a3fd9aaada582ae02c50710f218b54d1a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 75887d37e1ddbef579e239ff0b1b7a2508e486fd
Component: engine
make sure the LB sandbox is removed when a service is updated
with a --network-rm option
Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
(cherry picked from commit 680d0ba4abfb00c8ac959638c72003dba0826b46)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 280b8dff7df137dd21fcf27cacf6d12ed531770a
Component: engine
This fix moves multiple places of serviceRunningTasksCount
to one location in integration/internal/swarm, so that
code duplication could be removed.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
(cherry picked from commit e485a60e2bcc59860f387c94f6afaa0130ea7040)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c087f681d40332bd7d158baf615c03f6a72274f1
Component: engine
Tests which will re-deploy containers uses function serviceIsUpdated() to
make sure that service update really reached state UpdateStateCompleted.
Tests which will not re-deploy container uses function
serviceSpecIsUpdated to make sure that service version is increased.
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
(cherry picked from commit b868ada474b5c214ed9bddb792dd36f794dc1fa6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ccc1abea092a54d99dec51345ef9303b4629572d
Component: engine
[18.09 backport] explicitly set filesystem type for mount to avoid 'invalid argument' error on arm
Upstream-commit: 51ebfcbe423c679f06cfb37f7e6be565e0955639
Component: engine
[18.09 backport] bugfix: fetch the right device number which great than 255
Upstream-commit: b236a1e78d35a62b164d00f4e19f59a556da8831
Component: engine
[18.09 backport] Ensure all integration daemon logging happens before test exit
Upstream-commit: 08f6e9c14fd0de408db93914058d75df5b32ac67
Component: engine
go1.11.9 (released 2019/04/11) fixes an issue where using the prebuilt
binary releases on older versions of GNU/Linux led to failures when linking
programs that used cgo. Only Linux users who hit this issue need to update.
See golang/go#31293 for details
Full diff: https://github.com/golang/go/compare/go1.11.8...go1.11.9
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1b3ed84535
Component: cli
go1.11.6 (released 2019/03/14) includes fixes to cgo, the compiler, linker,
runtime, go command, and the crypto/x509, encoding/json, net, and net/url
packages. See the Go 1.11.6 milestone on our issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.6
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1500105975)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9959062d9a
Component: cli
go1.11.4 (released 2018/12/14) includes fixes to cgo, the compiler, linker,
runtime, documentation, go command, and the net/http and go/types packages. It
includes a fix to a bug introduced in Go 1.11.3 that broke go get for import
path patterns containing "...".
See the Go 1.11.4 milestone for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.4+label%3ACherryPickApproved
go1.11.3 (released 2018/12/14)
- crypto/x509: CPU denial of service in chain validation golang/go#29233
- cmd/go: directory traversal in "go get" via curly braces in import paths golang/go#29231
- cmd/go: remote command execution during "go get -u" golang/go#29230
See the Go 1.11.3 milestone on the issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit deaf6e13ab)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a378a00954
Component: cli
go1.11.2 (released 2018/11/02) includes fixes to the compiler, linker,
documentation, go command, and the database/sql and go/types packages.
See the milestone on the issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 58f0bfcf51)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 534c774fab
Component: cli
With a docker build cache already primed with the build image I am seeing
`time make build -f docker.Makefile DOCKER_BUILDKIT=1 GO_BUILD_CACHE=n` takes
more than 1 minute.
By contrast `time make build -f docker.Makefile DOCKER_BUILDKIT=1
GO_BUILD_CACHE=y` takes less than 10s with a hot cache irrespective of whether
the source tree has changed
Signed-off-by: Ian Campbell <ijc@docker.com>
(cherry picked from commit d5de8358f0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c1c4b46f29
Component: cli
When building the Dockerfiles for development, those images are mainly used to
create a reproducible build-environment. The source code is bind-mounted into
the image at runtime; there is no need to create an image with the actual
source code, and copying the source code into the image would lead to a new
image being created for each code-change (possibly leading up to many "dangling"
images for previous code-changes).
However, when building (and using) the development images in CI, bind-mounting
is not an option, because the daemon is running remotely.
To make this work, the circle-ci script patched the Dockerfiles when CI is run;
adding a `COPY` to the respective Dockerfiles.
Patching Dockerfiles is not really a "best practice" and, even though the source
code does not and up in the image, the source would still be _sent_ to the daemon
for each build (unless BuildKit is used).
This patch updates the makefiles, circle-ci script, and Dockerfiles;
- When building the Dockerfiles locally, pipe the Dockerfile through stdin.
Doing so, prevents the build-context from being sent to the daemon. This speeds
up the build, and doesn't fill up the Docker "temp" directory with content that's
not used
- Now that no content is sent, add the COPY instructions to the Dockerfiles, and
remove the code in the circle-ci script to "live patch" the Dockerfiles.
Before this patch is applied (with cache):
```
$ time make -f docker.Makefile build_shell_validate_image
docker build -t docker-cli-shell-validate -f ./dockerfiles/Dockerfile.shellcheck .
Sending build context to Docker daemon 41MB
Step 1/2 : FROM debian:stretch-slim
...
Successfully built 81e14e8ad856
Successfully tagged docker-cli-shell-validate:latest
2.75 real 0.45 user 0.56 sys
```
After this patch is applied (with cache)::
```
$ time make -f docker.Makefile build_shell_validate_image
cat ./dockerfiles/Dockerfile.shellcheck | docker build -t docker-cli-shell-validate -
Sending build context to Docker daemon 2.048kB
Step 1/2 : FROM debian:stretch-slim
...
Successfully built 81e14e8ad856
Successfully tagged docker-cli-shell-validate:latest
0.33 real 0.07 user 0.08 sys
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 166856ab1b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 206ea57da8
Component: cli
gofmt/goimports changed some heuristics in 1.11 and the code is now
formatted slightly differently.
No functional change, just whitespace.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 906c2d161a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: edbc0e0613
Component: cli
In case go build will see a need to call C++ (rather than C)
compiler, CXX env var need to be properly set (to osxcross wrapper).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit ee461303f9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f9a42a4024
Component: cli
Fix this warning from go-1.11
> cli/registry/client/fetcher.go:234: Debugf format %s has arg
> repoEndpoint of wrong type client.repositoryEndpoint
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 51848bf3bb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a13ec91543
Component: cli
Currently the API spec would allow `"443/tcp": [null]`, but what should
be allowed is `"443/tcp": null`
Signed-off-by: Dominic Tubach <dominic.tubach@to.com>
(cherry picked from commit 32b5d296ea5d392c28affe2854b9d4201166bc27)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 3ee1e060fc9a31844064035cc3e8b76fddb8b341
Component: engine
This test runs on a daemon also used by other tests
so make sure we don't get failures if another test
doesn't cleanup or is running in parallel.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 915acffdb4cf95b934dd9872c1f54ea4487819b7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 05599804157a38db2eda84d3e8060a879b755ab7
Component: engine
Running a bundled aufs benchmark sometimes results in this warning:
> WARN[0001] Couldn't run auplink before unmount /tmp/aufs-tests/aufs/mnt/XXXXX error="exit status 22" storage-driver=aufs
If we take a look at what aulink utility produces on stderr, we'll see:
> auplink:proc_mnt.c:96: /tmp/aufs-tests/aufs/mnt/XXXXX: Invalid argument
and auplink exits with exit code of 22 (EINVAL).
Looking into auplink source code, what happens is it tries to find a
record in /proc/self/mounts corresponding to the mount point (by using
setmntent()/getmntent_r() glibc functions), and it fails.
Some manual testing, as well as runtime testing with lots of printf
added on mount/unmount, as well as calls to check the superblock fs
magic on mount point (as in graphdriver.Mounted(graphdriver.FsMagicAufs, target)
confirmed that this record is in fact there, but sometimes auplink
can't find it. I was also able to reproduce the same error (inability
to find a mount in /proc/self/mounts that should definitely be there)
using a small C program, mocking what `auplink` does:
```c
#include <stdio.h>
#include <err.h>
#include <mntent.h>
#include <string.h>
#include <stdlib.h>
int main(int argc, char **argv)
{
FILE *fp;
struct mntent m, *p;
char a[4096];
char buf[4096 + 1024];
int found =0, lines = 0;
if (argc != 2) {
fprintf(stderr, "Usage: %s <mountpoint>\n", argv[0]);
exit(1);
}
fp = setmntent("/proc/self/mounts", "r");
if (!fp) {
err(1, "setmntent");
}
setvbuf(fp, a, _IOLBF, sizeof(a));
while ((p = getmntent_r(fp, &m, buf, sizeof(buf)))) {
lines++;
if (!strcmp(p->mnt_dir, argv[1])) {
found++;
}
}
printf("found %d entries for %s (%d lines seen)\n", found, argv[1], lines);
return !found;
}
```
I have also wrote a few other C proggies -- one that reads
/proc/self/mounts directly, one that reads /proc/self/mountinfo instead.
They are also prone to the same occasional error.
It is not perfectly clear why this happens, but so far my best theory
is when a lot of mounts/unmounts happen in parallel with reading
contents of /proc/self/mounts, sometimes the kernel fails to provide
continuity (i.e. it skips some part of file or mixes it up in some
other way). In other words, this is a kernel bug (which is probably
hard to fix unless some other interface to get a mount entry is added).
Now, there is no real fix, and a workaround I was able to come up
with is to retry when we got EINVAL. It usually works on the second
attempt, although I've once seen it took two attempts to go through.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit ae431b10a9508e2bf3b1782e9d435855e3e13977)
Upstream-commit: c303e63ca6c3d25d29b8992451898734fa6e4e7c
Component: engine
Do not use filepath.Walk() as there's no requirement to recursively
go into every directory under mnt -- a (non-recursive) list of
directories in mnt is sufficient.
With filepath.Walk(), in case some container will fail to unmount,
it'll go through the whole container filesystem which is both
excessive and useless.
This is similar to commit f1a459229724f5e8e440b49f ("devmapper.shutdown:
optimize")
While at it, raise the priority of "unmount error" message from debug
to a warning. Note we don't have to explicitly add `m` as unmount error (from
pkg/mount) will have it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 8fda12c6078ed5c86be0822a7a980c6fbc9220bf)
Upstream-commit: b85d4a4f09badf40201ef7fcbd5b930de216190d
Component: engine
In case there are a big number of layers, so that mount data won't fit
into a single memory page (4096 bytes on most platforms, which is good
enough for about 40 layers, depending on how long graphdriver root path
is), we supply additional layers with O_REMOUNT, as described in aufs
documentation.
Problem is, the current implementation does that one layer at a time
(i.e. there is one mount syscall per each additional layer).
Optimize the code to supply as many layers as we can fit in one page
(basically reusing the same code as for the original mount).
Note, per aufs docs, "[a]t remount-time, the options are interpreted
in the given order, e.g. left to right" so we should be good.
Tested on an image with ~100 layers.
Before (35 syscalls):
> [pid 22756] 1556919088.686955 mount("none", "/mnt/volume_sfo2_09/docker-aufs/aufs/mnt/a86f8c9dd0ec2486293119c20b0ec026e19bbc4d51332c554f7cf05d777c9866", "aufs", 0, "br:/mnt/volume_sfo2_09/docker-au"...) = 0 <0.000504>
> [pid 22756] 1556919088.687643 mount("none", "/mnt/volume_sfo2_09/docker-aufs/aufs/mnt/a86f8c9dd0ec2486293119c20b0ec026e19bbc4d51332c554f7cf05d777c9866", 0xc000c451b0, MS_REMOUNT, "append:/mnt/volume_sfo2_09/docke"...) = 0 <0.000105>
> [pid 22756] 1556919088.687851 mount("none", "/mnt/volume_sfo2_09/docker-aufs/aufs/mnt/a86f8c9dd0ec2486293119c20b0ec026e19bbc4d51332c554f7cf05d777c9866", 0xc000c451ba, MS_REMOUNT, "append:/mnt/volume_sfo2_09/docke"...) = 0 <0.000098>
> ..... (~30 lines skipped for clarity)
> [pid 22756] 1556919088.696182 mount("none", "/mnt/volume_sfo2_09/docker-aufs/aufs/mnt/a86f8c9dd0ec2486293119c20b0ec026e19bbc4d51332c554f7cf05d777c9866", 0xc000c45310, MS_REMOUNT, "append:/mnt/volume_sfo2_09/docke"...) = 0 <0.000266>
After (2 syscalls):
> [pid 24352] 1556919361.799889 mount("none", "/mnt/volume_sfo2_09/docker-aufs/aufs/mnt/8e7ba189e347a834e99eea4ed568f95b86cec809c227516afdc7c70286ff9a20", "aufs", 0, "br:/mnt/volume_sfo2_09/docker-au"...) = 0 <0.001717>
> [pid 24352] 1556919361.801761 mount("none", "/mnt/volume_sfo2_09/docker-aufs/aufs/mnt/8e7ba189e347a834e99eea4ed568f95b86cec809c227516afdc7c70286ff9a20", 0xc000dbecb0, MS_REMOUNT, "append:/mnt/volume_sfo2_09/docke"...) = 0 <0.001358>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit d58c434bffef76e48bff75ede290937874488dd3)
Upstream-commit: 75488521735325e4564e66d9cf9c2b1bc1c2c64b
Component: engine
Apparently there is some kind of race in aufs kernel module code,
which leads to the errors like:
[98221.158606] aufs au_xino_create2:186:dockerd[25801]: aufs.xino create err -17
[98221.162128] aufs au_xino_set:1229:dockerd[25801]: I/O Error, failed creating xino(-17).
[98362.239085] aufs au_xino_create2:186:dockerd[6348]: aufs.xino create err -17
[98362.243860] aufs au_xino_set:1229:dockerd[6348]: I/O Error, failed creating xino(-17).
[98373.775380] aufs au_xino_create:767:dockerd[27435]: open /dev/shm/aufs.xino(-17)
[98389.015640] aufs au_xino_create2:186:dockerd[26753]: aufs.xino create err -17
[98389.018776] aufs au_xino_set:1229:dockerd[26753]: I/O Error, failed creating xino(-17).
[98424.117584] aufs au_xino_create:767:dockerd[27105]: open /dev/shm/aufs.xino(-17)
So, we have to have a lock around mount syscall.
While at it, don't call the whole Unmount() on an error path, as
it leads to bogus error from auplink flush.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 5cd62852fa199f272b542d828c8c5d1db427ea53)
Upstream-commit: bb4b9fe29eba57ed71d8105ec0e5dacc8e72129e
Component: engine
1. Use mount.Unmount() which ignores EINVAL ("not mounted") error,
and provides better error diagnostics (so we don't have to explicitly
add target to error messages).
2. Since we're ignoring "not mounted" error, we can call
multiple unmounts without any locking -- but since "auplink flush"
is still involved and can produce an error in logs, let's keep
the check for fs being mounted (it's just a statfs so should be fast).
2. While at it, improve the "can't unmount" error message in Put().
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 4beee98026feabe4f4f0468215b8fd9b56f90d5e)
Upstream-commit: 5b68d00abc4cad8b707a9a01dde9a420ca6eb995
Component: engine
Both mount and unmount calls are already protected by fine-grained
(per id) locks in Get()/Put() introduced in commit fc1cf1911bb
("Add more locking to storage drivers"), so there's no point in
having a global lock in mount/unmount.
The only place from which unmount is called without any locking
is Cleanup() -- this is to be addressed in the next patch.
This reverts commit 824c24e6802ad3ed7e26b4f16e5ae81869b98185.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit f93750b2c4d5f6144f0790ffa89291da3c097b80)
Upstream-commit: 701939112efc71a0c867a626f0a56df9c56030db
Component: engine
The function is not needed as it's just a shallow wrapper around
unix.Mount().
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 2f98b5f51fb0a21e1bd930c0e660c4a7c4918aa5)
Upstream-commit: 023b63a0f276b79277c1c961d8c98d4d5b39525d
Component: engine
The errors returned from Mount and Unmount functions are raw
syscall.Errno errors (like EPERM or EINVAL), which provides
no context about what has happened and why.
Similar to os.PathError type, introduce mount.Error type
with some context. The error messages will now look like this:
> mount /tmp/mount-tests/source:/tmp/mount-tests/target, flags: 0x1001: operation not permitted
or
> mount tmpfs:/tmp/mount-test-source-516297835: operation not permitted
Before this patch, it was just
> operation not permitted
[v2: add Cause()]
[v3: rename MountError to Error, document Cause()]
[v4: fixes; audited all users]
[v5: make Error type private; changes after @cpuguy83 reviews]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 65331369617e89ce54cc9be080dba70f3a883d1c)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 7f1c6bf5a745c8faeba695d3556dff4c4ff5f473
Component: engine
Moby currently sorts uid and gid ranges in id maps. This causes subuid
and subgid files to be interpreted wrongly.
The subuid file
```
> cat /etc/subuid
jonas:100000:1000
jonas:1000:1
```
configures that the container uids 0-999 are mapped to the host uids
100000-100999 and uid 1000 in the container is mapped to uid 1000 on the
host. The expected uid_map is:
```
> docker run ubuntu cat /proc/self/uid_map
0 100000 1000
1000 1000 1
```
Moby currently sorts the ranges by the first id in the range. Therefore
with the subuid file above the uid 0 in the container is mapped to uid
100000 on host and the uids 1-1000 in container are mapped to the uids
1-1000 on the host. The resulting uid_map is:
```
> docker run ubuntu cat /proc/self/uid_map
0 1000 1
1 100000 1000
```
The ordering was implemented to work around a limitation in Linux 3.8.
This is fixed since Linux 3.9 as stated on the user namespaces manpage
[1]:
> In the initial implementation (Linux 3.8), this requirement was
> satisfied by a simplistic implementation that imposed the further
> requirement that the values in both field 1 and field 2 of successive
> lines must be in ascending numerical order, which prevented some
> otherwise valid maps from being created. Linux 3.9 and later fix this
> limitation, allowing any valid set of nonoverlapping maps.
This fix changes the interpretation of subuid and subgid files which do
not have the ids of in the numerical order for each individual user.
This breaks users that rely on the current behaviour.
The desired mapping above - map low user ids in the container to high
user ids on the host and some higher user ids in the container to lower
user on host - can unfortunately not archived with the current
behaviour.
[1] http://man7.org/linux/man-pages/man7/user_namespaces.7.html
Signed-off-by: Jonas Dohse <jonas@dohse.ch>
(cherry picked from commit c4628d79d26c47bfbac9a3b22d684ee5fd78973c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 601f11300ed9995ae1d9117a52fd38b3e86563d8
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
(cherry picked from commit 293c74ba79f0008f48073985507b34af59b45fa6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 91f5be57af815df372371e1e989d00963ce4d02f
Component: engine
It has been pointed out that we're ignoring EINVAL from umount(2)
everywhere, so let's move it to a lower-level function. Also, its
implementation should be the same for any UNIX incarnation, so
let's consolidate it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 90be078fe59a8cfeff2bcc5dc2f34a00309837b6)
Upstream-commit: 47c51447e1b6dacf92b40574f6f929958ca9d621
Component: engine
As standard mount.Unmount does what we need, let's use it.
In addition, this adds ignoring "not mounted" condition, which
was previously implemented (see PR#33329, commit cfa2591d3f26)
via a very expensive call to mount.Mounted().
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 77bc327e24a60791fe7e87980faf704cf7273cf9)
Upstream-commit: 893b24b80db170279d5c9532ed508a81c328de5e
Component: engine
[18.09 backport] Pass root to chroot to for chroot Tar/Untar (CVE-2018-15664)
Upstream-commit: 72797601723f6a8847027a1abbd1a3cea2667718
Component: engine
Increases the max recieved gRPC message size for Node and Secret list
operations. This has already been done for the other swarm types, but
was not done for these.
Signed-off-by: Drew Erny <drew.erny@docker.com>
(cherry picked from commit a0903e1fa3eca32065c7dbfda8d1e0879cbfbb8f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a987f31fbc42526bf2cccf1381066f075b18dfe0
Component: engine
Previously only unpack operations were supported with chroot.
This adds chroot support for packing operations.
This prevents potential breakouts when copying data from a container.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 3029e765e241ea2b5249868705dbf9095bc4d529)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 61e0459053c359e322b8d5c017e855f616fd34c0
Component: engine
This is useful for preventing CVE-2018-15664 where a malicious container
process can take advantage of a race on symlink resolution/sanitization.
Before this change chrootarchive would chroot to the destination
directory which is attacker controlled. With this patch we always chroot
to the container's root which is not attacker controlled.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit d089b639372a8f9301747ea56eaf0a42df24016a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 155939994f453559676656bc4b05635e83ebef56
Component: engine
[18.09] Backport Forcing a nil IP specified in PortBindings to IPv4 zero (0.0.0.0).
Upstream-commit: c882c0011f0857da5e8cef1ea48e0aa80019abf1
Component: engine
As pointed out by Tonis, there's a race between ReleaseRWLayer()
and GetRWLayer():
```
----- goroutine 1 ----- ----- goroutine 2 -----
ReleaseRWLayer()
m := ls.mounts[l.Name()]
...
m.deleteReference(l)
m.hasReferences()
... GetRWLayer()
... mount := ls.mounts[id]
ls.driver.Remove(m.mountID)
ls.store.RemoveMount(m.name) return mount.getReference()
delete(ls.mounts, m.Name())
----------------------- -----------------------
```
When something like this happens, GetRWLayer will return
an RWLayer without a storage. Oops.
There might be more races like this, and it seems the best
solution is to lock by layer id/name by using pkg/locker.
With this in place, name collision could not happen, so remove
the part of previous commit that protected against it in
CreateRWLayer (temporary nil assigmment and associated rollback).
So, now we have
* layerStore.mountL sync.Mutex to protect layerStore.mount map[]
(against concurrent access);
* mountedLayer's embedded `sync.Mutex` to protect its references map[];
* layerStore.layerL (which I haven't touched);
* per-id locker, to avoid name conflicts and concurrent operations
on the same rw layer.
The whole rig seems to look more readable now (mutexes use is
straightforward, no nested locks).
Reported-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit af433dd200f8287305b1531d5058780be36b7e2e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1ebe324c6a481de86a5b84494057a639773624f1
Component: engine
This is an additon to commit 1fea38856a ("Remove v1.10 migrator")
aka PR #38265. Since that one, CreateRWLayerByGraphID() is not
used anywhere, so let's drop it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit b4e9b507655e7dbdfb44d4ee284dcd658b859b3f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1576eaba3357b7aab5121b51b9ebc34e731082fe
Component: engine
Goroutine stack analisys shown some lock contention
while doing massively (100 instances of `docker rm`)
parallel image removal, with many goroutines waiting
for the mountL mutex. Optimize it.
With this commit, the above operation is about 3x
faster, with no noticeable change to container
creation times (tested on aufs and overlay2).
kolyshkin@:
- squashed commits
- added description
- protected CreateRWLayer against name collisions by
temporary assiging nil to ls.mounts[name], and treating
nil as "non-existent" in all the other functions.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 05250a4f0094e6802dd7d338d632ea632d0c7e34)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 78cbf4d1388a4eef155f08da0d2422e3a2cad11f
Component: engine
Add a mutex to protect concurrent access to mountedLayer.references map.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit f73b5cb4e8b9a23ad6700577840582d66017a733)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: bb80a60be2a9236b0db27222310f568e1d3fba11
Component: engine
As of Go 1.12, the `testing` package panics if a goroutine logs to a
`testing.T` after the relevant test has completed. This was not
documented as a change at all; see the commit
95d06ab6c982f58b127b14a52c3325acf0bd3926 in the Go repository for the
relevant change.
At any point in the integration tests, tests could panic with the
message "Log in goroutine after TEST_FUNCTION has completed". This was
exacerbated by less direct logging I/O, e.g. running `make test` with
its output piped instead of attached to a TTY.
The most common cause of panics was that there was a race condition
between an exit logging goroutine and the `StopWithError` method:
`StopWithError` could return, causing the calling test method to return,
causing the `testing.T` to be marked as finished, before the goroutine
could log that the test daemon had exited. The fix is simple: capture
the result of `cmd.Wait()`, _then_ log, _then_ send the captured
result over the `Wait` channel. This ensures that the message is
logged before `StopWithError` can return, blocking the test method
so that the target `testing.T` is not marked as finished.
Signed-off-by: Daniel Sweet <danieljsweet@icloud.com>
(cherry picked from commit 7546322e994c4a23ea3cae0cf0a2a8019de12c03)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b2168eec8b37619a1d28d0bb44cf052d4f9883c7
Component: engine
Checks for environment variable VERSION if it exists then it sets dockerVersion to VERSION
Signed-off-by: corbin-coleman <corbin.coleman@docker.com>
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit edc639e99f21d0dd814581858d02787c23ba1599)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 238951ac3a78a8517d7764de0f48b8ded28441e2
Component: engine
append(newRoot.DiffIDs) without element does nothing,
so it's probably not what was intended. Changed code
to perform a slice copying instead.
Fixes#38834.
Signed-off-by: Iskander Sharipov <quasilyte@gmail.com>
(cherry picked from commit 3429e9993085520ac902fef2ef6aabd57080bd36)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1c755a73bfd63f63d6fe3363d8237f9db634633d
Component: engine
This test starts a new daemon, which will fail when testing
against a remote daemon;
--- FAIL: TestInfoAPIWarnings (0.00s)
info_test.go:53: failed to start daemon with arguments [-H=0.0.0.0:23756 -H=unix:///tmp/docker-integration/d5153ebcf89ef.sock] : [d5153ebcf89ef] could not find docker binary in $PATH: exec: "dockerd": executable file not found in $PATH
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 056840c2a644cc37f91392103f5c3ae05d14a695)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5e024a2fcd10585611ab1ea792d0256c6477c87c
Component: engine
This test is dependent on the search results returned by Docker Hub, which
can change at any moment, and causes this test to be unpredictable.
Removing this test instead of trying to catch up with Docker Hub any time
the results change, because it's effectively testing Docker Hub, and not
the daemon.
Unit tests are already in place to test the core functionality of the daemon,
so it should be safe to remove this test.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 21e662c77468d1e3fc4efaa049021e2ad0e49b4d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0d052bc40e908f1f596b0f16840b4e93fc6a9ec2
Component: engine
Errors were being ignored and always telling the user that the path
doesn't exist even if it was some other problem, such as a permission
error.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit ebcef288343698dd86ff307f5b9c58aa52ce9fdd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0ac09de395d61d8cecdd6496aa3fb9ceb174f295
Component: engine
`chmod` is a legacy syscall, and not present on arm64, which
caused this test to fail.
Add `fchmodat` to the profile so that this test can run both
on x64 and arm64.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4bd8964b23f616a1aef63ea0acebbdc4b6ea90e6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1b642b8c45a734d4c2d9af2f6849289db556d003
Component: engine
Sometimes this test fails (allegedly due to problems with Docker Hub),
but it fails later than it should, for example:
> 01:20:34.845 assertion failed: expression is false: strings.Count(outSearchCmdStars, "[OK]") <= strings.Count(outSearchCmd, "[OK]"): The quantity of images with stars should be less than that of all images: <...>
This, with non-empty list of images following, means that the initial
`docker search busybox` command returned not enough results. So, add
a check that `docker search busybox` returns something.
While at it,
* raise the number of stars to 10;
* simplify check for number of lines (no need to count [OK]'s);
* improve error message.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 4f80a1953d66265d9a1cbf4e172b8fa58e3e403a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8a369c735b2bf07da53275fe9fe61d429320f233
Component: engine
Today `$ docker service create --limit-cpu` configures a containers
`CpuPeriod` and `CpuQuota` variables, this commit switches this to
configure a containers `NanoCpu` variable instead.
Signed-off-by: Olly Pomeroy <olly@docker.com>
(cherry picked from commit 8a60a1e14a5e98b7ed0fbea57369615a766a1ae9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 675f4256ae2b85556b4243da44ff627900660463
Component: engine
GCR does not currently support 401 response from blob endpoints.
This detects the case where no manifest requests have been
performed for the current resolver and does a dummy request
to enable authorization.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit bcd8298c35f53f79b42c6e089e8da114ddb5c57e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 21cce932b05a4b979359624668f7f09e83dc90cd
Component: engine
full diff: c9029898e3...872f0a83c9
- docker/libnetwork#2354 [18.09 backport] Cleanup the cluster provider when the agent is closed
- backport of docker/libnetwork#2307 Fix for problem where agent is stopped and does not restart
- fixesdocker/for-linux#495 Docker swarm overlay networking not working after --force-new-cluster
- docker/libnetwork#2369 [18.09 BACKPORT] Pick a random host port if the user does not specify a host port
- backport of docker/libnetwork#2368 (windows) Pick a random host port if the user does not specify a host port
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5354408039681020f9ad6afe4bf696fc90f9ce69
Component: engine
[18.09 backport] test-fixes and updates, and fix images filter when use multi reference filter
Upstream-commit: 82a2b8882e5a9dc2ba4389f589ad28dc9cee0d57
Component: engine
In case of failure, stale out was printed.
Fixes: 6212ea669b4e92b3
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 1921753b4b30dcca4fe772e7c1b0bc3f7bb7cd62)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b60ecd32a1d3ba7debf02d15fe346ea5ca607bbd
Component: engine
Move the test case from integration-cli to integration.
The test logic itself has not changed, except these
two things:
* the new test sets default-ipc-mode via command line
rather than via daemon.json (less code);
* the new test uses current API version.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 9fd765f07cc08ccc2ea991d21835bf50ece9318b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b228a498d55fcd47c741d51a510a7f5eab0794f7
Component: engine
NOTE TestUpdateRestartPolicy is left as is as otherwise
it will decrease its readability.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit f664df01d1836686c7a8a917e9da81d56c758d74)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2c546aace40872169b03e18f07294d3fbfa3d452
Component: engine
Since container.Create() already initializes HostConfig
to be non-nil, there is no need for this code. Remove it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 17022b3ad2d95863e5acd17ebaf943b7761623cc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ed48a92a580b7ccc5c5837c7d802f787000146eb
Component: engine
This test sometimes failed because the number of events received did not
match the expected number:
FAIL: docker_cli_events_test.go:316: DockerSuite.TestEventsFilterLabels
docker_cli_events_test.go:334:
c.Assert(len(events), checker.Equals, 3)
... obtained int = 2
... expected int = 3
This patch makes the test more stable, by:
- use a wider range between `--since` and `--until`. These options were set
so that the client detaches after events were received, but the actual
range should not matter. Changing the range will cause more events to be
returned, but we're specifically looking for the container ID's, so this
should not make a difference for the actual test.
- use `docker create` instead of `docker run` for the containers. the
containers don't have to be running to trigger an event; using `create`
speeds up the test.
- check the exit code of the `docker create` to verify the containers were
succesfully created.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0e15c02465c87c82908bcc45b0c1d6bd38f27c32)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 26f7e0a8b08b2890d079d4083735c05377770ca8
Component: engine
I noticed that this test failed, because the node was in status "pending".
The test checks for the node's status immediately after it was restarted, so
possibly it needs some time to unlock.
14:07:10 FAIL: docker_cli_swarm_test.go:1128: DockerSwarmSuite.TestSwarmLockUnlockCluster
...
14:07:10 docker_cli_swarm_test.go:1168:
14:07:10 checkSwarmLockedToUnlocked(c, d)
14:07:10 docker_cli_swarm_test.go:1017:
14:07:10 c.Assert(getNodeStatus(c, d), checker.Equals, swarm.LocalNodeStateActive)
14:07:10 ... obtained swarm.LocalNodeState = "pending"
14:07:10 ... expected swarm.LocalNodeState = "active"
This patch adds a `waitAndAssert` for the node's status, with a 1 second timeout.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 973ca00d60712ef644b5b37abf7fa01078bb4ade)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 954425f8bdb5db95355cb20be465d248f7c3b1f1
Component: engine
When starting docker daemons for swarm testing, we disable iptables
and use lo for communication (in order to avoid network conflicts).
The problem is, these options are lost on restart, that can lead
to any sorts of network conflicts and thus connectivity issues
between swarm nodes.
Fix this. This does not fix issues with swarm test failures, but
it seems they appear are less often after this one.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 2ed512c7faea938b0b07e69187b8a132e2ecb66a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 553b09684cb68962403ebdc5495fb548364f778f
Component: engine
This is repeated 6 times in different tests, with slight
minor variations. Let's factor it out, for clarity.
While at it, simplify the code: instead of more complex
parsing of "docker swarm init|update --autolock" output (1)
and checking if the key is also present in
"docker swarm unlock-key" output (2), get the key
from (2) and check it is present in (1).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 24cbb9897193894f4716583d1861091ab2fa1ae2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 6ec991ec8371bdba94480c587eea6ea24f3e9d43
Component: engine
A timer is leaking on every daemon start and stop.
Probably nothing major, but given the amount of
daemon starts/stops during tests, it's better to
be accurate about it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 6016520162fdcb19f50d08c4f0b54b06a7a6eac0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 784a89354b118ef68f1f20fb4ff383c6282ea03c
Component: engine
Since commit 17173efbe00 checkSwarmLockedToUnlocked() no longer
require its third argument, so remove it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 66cb1222d6559e120d9d1a29932aa778aa517894)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2a02a2e9dae0045d550e98fa849aceafc0c41a0e
Component: engine
1. Using MNT_FORCE flag does not make sense for nsfs. Using MNT_DETACH
though might help.
2. When -check.vv is added to TESTFLAGS, there are a lot of messages
like this one:
> unmount of /tmp/dxr/d847fd103a4ba/netns failed: invalid argument
and some like
> unmount of /tmp/dxr/dd245af642d94/netns failed: no such file or directory
The first one means directory is not a mount point, the second one
means it's gone. Do ignore both of these.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 73baee2dcf546b2561bdd9a500b0af08cb62b1be)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 90cb9b6545fbf7f91cf3eb427f627a7353ea684f
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
(cherry picked from commit b1b9937bc75f0db9c804838ecce9bb6792a42525)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 533b0f602d10af8fe325c3544a2411f5060a47b8
Component: engine
It looks like the logic of the test became wrong after commit
ae0883c ("Move TestAttachDetach to integration-cli").
The original logic was:
* (a few first steps skipped for clarity)
* send escape sequence to "attach";
* check "attach" is exiting (i.e. escape sequence works);
* check the container is still alive;
* kill the container.
Also, timeouts were big at that time, in the order of seconds.
The logic after the above mentioned commit and until now is:
* ...
* send escape sequence to "attach";
* check the container is running (why shouldn't it?);
* kill the container;
* checks that the "attach" has exited.
So, from the "let's check detach using escape sequence is working"
the test became something like "let's check that attach is gone
once we kill the container".
Let's fix the above test, also increasing the timeout waiting
for attach to exit (which fails from time to time on power CI).
Now, the second test, TestAttachDetachTruncatedID, does the exact
same thing, except it uses a truncated container ID. It does not
seem to be of much value, so let's remove it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 9f3a343a5101ab661a6a97c9e149a0b11ccc320a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 17ae129697c10e9f866ffc253bc211f8912588df
Component: engine
This introduces `NoTasksForService` and `NoTasks` poller checks, that
can be used to check if no tasks are left in general, or for a specific
service.
Some redundant checks were also removed from some tests.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 56a68c15f8a093b1761e77a74d8b7acdfbcb30a2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8f6421457d3be6280236aa466e533fb32cb4509b
Component: engine
Clean up and refactor this test;
- make `serviceRunningTasksCount` to use a `desired-state` filter
- use subtests, and inline the `validNetworkVerbose` checks; also use
asserts for the individual checks, so that any failure will log exactly
what failed
- remove helper functions that are no longer needed
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 60d93aab2e68b13b3f22c43add4762d3e8108227)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 046352162f1a23f10a05b59e4fac9d932bcd4f35
Component: engine
This fix removes code duplication and consolidates networkIsRemoved
into one place.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
(cherry picked from commit 28b7824caa9c8f1acc0471136a8a4fd80e51f491)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: dd20556e037b262473f3f8163dfc7c21e9161c8e
Component: engine
These tests are run on a local Linux daemon only, so no need
to do a platform-check.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 69c0b7e47682a2a7a850122a9a2f711259fbb25a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: bb6db57acc8a63f7f85fafe4be0cfe7fa929e3a1
Component: engine
These tests can only be run on a local Linux daemon, so there's
no need to build them on Windows
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 84224935ea78d93e21a21c0e1e0aa3e83a5c7853)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b4c0a7efd46d5fb13df7214aface1f291ba356d2
Component: engine
A client is already created in testenv.New(), so we can just
as well use that one, instead of creating a new client.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0de62d9bbcb92e9b7c73ee4cdef51c2229878e05)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e438d4799d6e233fe631d332973dcac1d977fef6
Component: engine
`testEnv` is a package-level variable, so protecting / restoring
`testEnv` in parallel will result in "concurrent map write" errors.
This patch removes `t.Parallel()` from tests that use this
functionality (through `defer setupTest(t)()`).
Note that _subtests_ can still be run in parallel, as the defer
will be called after all subtests have completed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4d88a95d6730383624570f8730aa203a56caadc3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 05ecb140c4f3c4c7cbb860baa425104fa6f132ea
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
(cherry picked from commit bc80e8df3e6300354305f81a4c0c48b855a1776b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: d5a0ffa172a012e388d5b31edf332acc264b39b6
Component: engine
I noticed this error in CI:
```
20:37:25 INFO: Non-base image count on control daemon to delete is 9
20:37:25 "docker rmi" requires at least 1 argument.
20:37:25 See 'docker rmi --help'.
20:37:25
20:37:25 Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
20:37:25
20:37:25 Remove one or more images
```
Which indicated that the PowerShell script managed to find images to delete, but
not actually passing the images to `docker rmi`.
The reason for this failing was that the script attempted to convert the
collection/array to a string, which produces;
```powershell
Write-Output $(docker images --format "{{.Repository}}:{{.ID}}" | `
>> select-string -NotMatch "windowsservercore" | `
>> select-string -NotMatch "nanoserver" | `
>> select-string -NotMatch "docker" `
>> ).ToString()
System.Object[]
```
Which, when trying to split by the chosen separator (`:`), will return the same;
```powershell
Write-Output "System.Object[]".Split(":")[0]
```
This patch:
- Adds an intermediate variable (`$allImages`) to make the code better readable
- Switches the separator to `#`, to prevent breaking on images pulled from a
repository with a port in its name (`myregistry:5000/my/image`)
- Switches to use a comma-separated list for `-NotMatch` (for readability)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5580b79813d3a5d2a7b83bcf1607839a5b0f468d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 74fe2c044f56e58356e9f598e5b08d94e676bbd4
Component: engine
This patch replaces PowerShell aliases for their real commands, see https://blogs.technet.microsoft.com/heyscriptingguy/2012/04/21/when-you-should-use-powershell-aliases/
For example;
- use `Get-Location` instead of `pwd`
- use `Set-Location` instead of `cd`
- use `ForEach-Object` instead of the `%` shorthand
- use `Write-Output` instead of `echo`
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6130c89ccec5262c25b91b3226a6ef2240bfde3b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 4f0911d9122bf73d7efb90a50444cd6e52a09a96
Component: engine
Fixed some mixed/tabs spaces for indentation, and used
tabs for auto-generated Go code.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0f8b616c0c8f959ac4b29f29a8716b8cb7ea28fd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 73e5951b2e3c13ce84781d28b08cdc1958fb6224
Component: engine
Old selector was wrong (it watched for the label we applied to child
resources when reconciling the stack, instead of the stack itself)
This should be back-ported to older version of the CLI
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
(cherry picked from commit 8cd74eb33a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f89d05edcb
Component: cli
Vendoring can take some time, depending on network-speed, so
reduce flakiness by increasing the default timeout, to prevent:
make[1]: Entering directory '/go/src/github.com/docker/cli'
rm -rf vendor
bash -c 'vndr |& grep -v -i clone'
2019/03/18 11:38:26 Collecting initial packages
Too long with no output (exceeded 10m0s)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit dba90e4999)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 90fa621791
Component: cli
This patch switches the shellcheck image to use the official image
from Docker Hub.
Note that this does not yet update shellcheck to the latest version (v0.5.x);
Shellcheck v0.4.7 added some new checks, which makes CI currently fail, so will
be done in a follow-up PR. Instead, the v0.4.6 version is used in this PR, which
is closest to the same version as was installed in the image before this change;
```
docker run --rm docker-cli-shell-validate shellcheck --version
ShellCheck - shell script analysis tool
version: 0.4.4
license: GNU General Public License, version 3
website: http://www.shellcheck.net
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 388646eab0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b59752479b
Component: cli
When building the Dockerfiles for development, those images are mainly used to
create a reproducible build-environment. The source code is bind-mounted into
the image at runtime; there is no need to create an image with the actual
source code, and copying the source code into the image would lead to a new
image being created for each code-change (possibly leading up to many "dangling"
images for previous code-changes).
However, when building (and using) the development images in CI, bind-mounting
is not an option, because the daemon is running remotely.
To make this work, the circle-ci script patched the Dockerfiles when CI is run;
adding a `COPY` to the respective Dockerfiles.
Patching Dockerfiles is not really a "best practice" and, even though the source
code does not and up in the image, the source would still be _sent_ to the daemon
for each build (unless BuildKit is used).
This patch updates the makefiles, circle-ci script, and Dockerfiles;
- When building the Dockerfiles locally, pipe the Dockerfile through stdin.
Doing so, prevents the build-context from being sent to the daemon. This speeds
up the build, and doesn't fill up the Docker "temp" directory with content that's
not used
- Now that no content is sent, add the COPY instructions to the Dockerfiles, and
remove the code in the circle-ci script to "live patch" the Dockerfiles.
Before this patch is applied (with cache):
```
$ time make -f docker.Makefile build_shell_validate_image
docker build -t docker-cli-shell-validate -f ./dockerfiles/Dockerfile.shellcheck .
Sending build context to Docker daemon 41MB
Step 1/2 : FROM debian:stretch-slim
...
Successfully built 81e14e8ad856
Successfully tagged docker-cli-shell-validate:latest
2.75 real 0.45 user 0.56 sys
```
After this patch is applied (with cache)::
```
$ time make -f docker.Makefile build_shell_validate_image
cat ./dockerfiles/Dockerfile.shellcheck | docker build -t docker-cli-shell-validate -
Sending build context to Docker daemon 2.048kB
Step 1/2 : FROM debian:stretch-slim
...
Successfully built 81e14e8ad856
Successfully tagged docker-cli-shell-validate:latest
0.33 real 0.07 user 0.08 sys
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 166856ab1b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8997667aa2
Component: cli
When manually stopping a container with a restart-policy, the container
would show as "restarting" in `docker ps` whereas its actual state
is "exited".
Stopping a container with a restart policy shows the container as "restarting"
docker run -d --name test --restart unless-stopped busybox false
docker stop test
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7e07409fa1d3 busybox "false" 5 minutes ago Restarting (1) 4 minutes ago test
However, inspecting the same container shows that it's exited:
docker inspect test --format '{{ json .State }}'
{
"Status": "exited",
"Running": false,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 0,
"ExitCode": 1,
"Error": "",
"StartedAt": "2019-02-14T13:26:27.6091648Z",
"FinishedAt": "2019-02-14T13:26:27.689427Z"
}
And killing the container confirms this;
docker kill test
Error response from daemon: Cannot kill container: test: Container 7e07409fa1d36dc8d8cb8f25cf12ee1168ad9040183b85fafa73ee2c1fcf9361 is not running
docker run -d --name test --restart unless-stopped busybox false
docker stop test
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d0595237054a busybox "false" 5 minutes ago Restarting (1) 4 minutes ago exit
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8c0ecb638705e89746a81fa1320aafaa7ff701b2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 00f0b9df0dde80b2c40023b12a29dd476f2a77e2
Component: engine
When running a container in the host's network namespace, the container
gets a copy of the host's resolv.conf (copied to `/etc/resolv.conf` inside
the container).
The current code always used the default (`/etc/resolv.conf`) path on the
host, irregardless if `systemd-resolved` was used or not.
This patch uses the correct file if `systemd-resolved` was detected
to be running.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8364d1c9d590d4266871cd820b76ef12e2b934ed)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 04ae160eca4a8bc134ca9abafa044191b893080b
Component: engine
The integration test TestExportContainerWithOutputAndImportImage in moby/moby is the same as TestExportContainerAndImportImage,
except for the output file option. Adding a unit test to cover the output file option of the export command here allows
the removal of the redundant integration test TestExportContainerWithOutputAndImportImage.
Signed-off-by: Arash Deshmeh <adeshmeh@ca.ibm.com>
(cherry picked from commit fc1e11d46a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 079adf3f23
Component: cli
Tagger was not called for BuildKit-mode.
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
(cherry picked from commit 7fc0f820ea1e9036a1466ee8ef7a7395b792623f)
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Upstream-commit: 8d87a2a4bc88b235f06e3995bb33f978d7c28cdf
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 9d2e97ac6e20b17477947fc63e70299938606a38)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 91703956dffcf1d9c997aea83f9489a8c768412d
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
(cherry picked from commit 80fce6d747c5208b42e94ac9e3f22cef28dd8afe)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: fd4670161de8f9947c524138c477177f935c98f2
Component: engine
There's little way of knowing what each exit status means at present
because it's not documented. I'm assuming they are the same as docker
run.
Signed-off-by: Eric Curtin <ericcurtin17@gmail.com>
(cherry picked from commit 23670968cc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 27ab7cc3d6
Component: cli
Noticed a typo in this markdown file: "instead" instead of "in stead"
Signed-off-by: Ryan Wilson-Perkin <ryanwilsonperkin@gmail.com>
(cherry picked from commit 7a9fc782c5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1d314f2227
Component: cli
Copies the improved description from the man page
to the online reference docs.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 89bc5fbbae)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 275ab1f063
Component: cli
The `docker image rm` command can be used not only
to remove images but also remove tags.
This update improves the documentation to make
this clear.
Signed-off-by: Filip Jareš <filipjares@gmail.com>
(cherry picked from commit 2ba9601ef1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0717f261ed
Component: cli
By default, exec uses the environment of the current process, however,
if `exec.Env` is not `nil`, the environment is discarded:
e73f489494/src/os/exec/exec.go (L57-L60)
> If Env is nil, the new process uses the current process's environment.
When adding a new environment variable, prepend the current environment,
to make sure it is not discarded.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6c4fbb7738)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: fc8717799f
Component: cli
Relevant changes:
- containerd/containerd#51 Fix empty device type
- containerd/containerd#52 Remove call to unitName
- Calling unitName incorrectly appends -slice onto the end of the slice cgroup we are looking for
- addresses containerd/containerd#47 cgroups: cgroup deleted
- containerd/containerd#53 systemd-239+ no longer allows delegate slice
- containerd/containerd#54 Bugfix: can't write to cpuset cgroup
- containerd/containerd#63 Makes Load function more lenient on subsystems' checking
- addresses containerd/containerd#58 Very strict checking of subsystems' existence while loading cgroup
- containerd/containerd#67 Add functionality for retrieving all tasks of a cgroup
- containerd/containerd#68 Fix net_prio typo
- containerd/containerd#69 Blkio weight/leafWeight pointer value
- containerd/containerd#77 Check for non-active/supported cgroups
- addresses containerd/containerd#76 unable to find * in controller set: unknown
- addresses docker/for-linux#545 Raspbian: Error response from daemon: unable to find "net_prio" in controller set: unknown
- addresses docker/for-linux#552 Error response from daemon: unable to find "cpuacct" in controller set: unknown
- addresses docker/for-linux#545 Raspbian: Error response from daemon: unable to find "net_prio" in controller set: unknown
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 386b06eacd689bb98fb4117087614466c6e130f3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a36f6bfc36d2425c693097102b91152fa8f80012
Component: engine
Notable Updates
- Fix an issue that non-existent parent directory in image layers is created with permission 0700. containerd#3017
- Fix an issue that snapshots of the base image can be deleted by mistake, when images built on top of it are deleted. containerd#3087
- Support for GC references to content from snapshot and container objects. containerd#3080
- cgroups updated to dbea6f2bd41658b84b00417ceefa416b97 to fix issues for systemd 420 and non-existent cgroups. containerd#3079
- runc updated to 2b18fe1d885ee5083ef9f0838fee39b62d653e30 to include the improved fix for CVE-2019-5736. containerd#3082
- cri: Fix a bug that pod can't get started when the same volume is defined differently in the image and the pod spec. cri#1059
- cri: Fix a bug that causes container start failure after in-place upgrade containerd to 1.2.4+ or 1.1.6+. cri#1082
- cri updated to a92c40017473cbe0239ce180125f12669757e44f. containerd#3084
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 25cdae293fedb18f3538788d629d6e4475acaa4b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 6c715008627a9ff2073e932d5c792dc7bc1f2f01
Component: engine
After switching to Go 1.12, the format-string causes an error;
```
=== Errors
cli/config/config_test.go:154:3: Fatalf format %q has arg config of wrong type *github.com/docker/cli/cli/config/configfile.ConfigFile
cli/config/config_test.go:217:3: Fatalf format %q has arg config of wrong type *github.com/docker/cli/cli/config/configfile.ConfigFile
cli/config/config_test.go:253:3: Fatalf format %q has arg config of wrong type *github.com/docker/cli/cli/config/configfile.ConfigFile
cli/config/config_test.go:288:3: Fatalf format %q has arg config of wrong type *github.com/docker/cli/cli/config/configfile.ConfigFile
cli/config/config_test.go:435:3: Fatalf format %q has arg config of wrong type *github.com/docker/cli/cli/config/configfile.ConfigFile
cli/config/config_test.go:448:3: Fatalf format %q has arg config of wrong type *github.com/docker/cli/cli/config/configfile.ConfigFile
DONE 1115 tests, 2 skipped, 6 errors in 215.984s
make: *** [Makefile:22: test-coverage] Error 2
Exited with code 2
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d4877fb225)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 7ea48a16e3
Component: cli
This was leaking the fd.
Signed-off-by: Ian Campbell <ijc@docker.com>
(cherry picked from commit 186e7456ac)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 75e9075591
Component: cli
This happens on Windows when dialing a named pipe (a path which is used by CLI
plugins), in that case some debugging shows:
DEBU[0000] conn is a *winio.win32MessageBytePipe
DEBU[0000] conn is a halfReadCloser: false
DEBU[0000] conn is a halfWriteCloser: true
the raw stream connection does not implement halfCloser
In such cases we can simply wrap with a nop function since closing for read
isn't too critical.
Signed-off-by: Ian Campbell <ijc@docker.com>
(cherry picked from commit 8919bbf04d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 69e1094f5a
Component: cli
... in preference to `chsh`, since in recent alpine 3.9.2 images that can fail
with:
Password: chsh: PAM: Authentication token manipulation error
Which seems to relate to the use of `!` as the password for `root` in `/etc/shadow`gq
Signed-off-by: Ian Campbell <ijc@docker.com>
(cherry picked from commit 5de2d9e8a9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b666e9a090
Component: cli
[18.09 backport] Fix TestBuildWithSession, TestBuildSquashParent using wrong daemon during test
Upstream-commit: c97602ef9903fd18e65b341a7359287e684cde84
Component: engine
These tests were spinning up a new daemon, but after the daemon was spun up,
the default test-daemon was used by the client.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9a3911ced83331b6efe698528a4480bd5ff3d7a6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 605bc1ffc5adc5a3968ebe7a58060d2542cc28f3
Component: engine
relevant changes:
- swarmkit#2826 [18.09 backport] use a custom grpc dialer when managers are joining (backport of swarmkit#2802)
- swarmkit#2801 [18.09 backport] Include old error-message for backward compatibility (backport of swarmkit#2797)
- swarmkit#2788 [18.09 backport] Return correct error-codes on conflicting names (backport of swarmkit#2779)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1a60fdbdcefbecf762e063fbb5b3593e53e877bb
Component: engine
As reported in docker/for-linux/issues/484, since Docker 18.06
docker cp with a destination file name fails with the following error:
> archive/tar: cannot encode header: Format specifies USTAR; and USTAR cannot encode Name="a_very_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_long_filename_that_is_101_characters"
The problem is caused by changes in Go 1.10 archive/tar, which
mis-guesses the tar stream format as USTAR (rather than PAX),
which, in turn, leads to inability to specify file names
longer than 100 characters.
This tar stream is sent by TarWithOptions() (which, since we switched to
Go 1.10, explicitly sets format=PAX for every file, see FileInfoHeader(),
and before Go 1.10 it was PAX by default). Unfortunately, the receiving
side, RebaseArchiveEntries(), which calls tar.Next(), mistakenly guesses
header format as USTAR, which leads to the above error.
The fix is easy: set the format to PAX in RebaseArchiveEntries()
where we read the tar stream and change the file name.
A unit test is added to prevent future regressions.
NOTE this code is not used by dockerd, but rather but docker cli
(also possibly other clients), so this needs to be re-vendored
to cli in order to take effect.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit f55a4176febbd0dffd6e5eb65beb70bc32912d0b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 989e7f5d3a3f40ebb936376245b770f766ea42e9
Component: engine
In applyTar, if the driver's ApplyDiff returns an error, the function
returns early without calling io.Copy.
As a consequence, the resources (a goroutine and some buffers holding
the uncompressed image, the digest, etc...) allocated or referenced by
NewInputTarStream above aren't released, as the worker goroutine only
finishes when it finds EOF or a closed pipe.
Signed-off-by: Sergio Lopez <slp@redhat.com>
(cherry picked from commit 5846db10af9fb37061ab92a07c3d82fbea92b2e0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f660ef2c25590f21fdd10357c08a542a4876d6c3
Component: engine
This commit ensures that we test the correct branches of Docker
when making packaging changes.
Signed-off-by: Dave Tucker <dt@docker.com>
(cherry picked from commit 1ef9fb3f0b2b8016b50c6c71f2d14bea16faff35)
Upstream-commit: 5e59ff8a5e8ea44444aa4f22a8baf12e091bcc76
Component: packaging
Signed-off-by: John Howard <jhoward@microsoft.com>
This is a follow-on from https://github.com/moby/moby/pull/38277
but had to be done in a couple of stages to ensure that CI didn't
break. v1.1 of the busybox image is now based on a CMD of "sh"
rather than using an entrypoint. And it also uses the bin directory
rather than `c:\busybox`. This makes it look a lot closer to the
Linux busybox image, and means that a couple of Windows-isms in
CI tests can be reverted back to be identical to their Linux
equivalents.
(cherry picked from commit 561e0f6b7fc256c160292b32695cf1d6150741db)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 613c2f27ed2e7d65474c2f3e786d9e24e757d99d
Component: engine
- Fixes a vulnerability in runc that allows a container escape (CVE-2019-5736)
6635b4f0c6,
- Includes security fix for `runc run --no-pivot` (`DOCKER_RAMDISK=1`):
28a697cce3
(NOTE: the vuln is attackable only when `DOCKER_RAMDISK=1` is set && seccomp is disabled)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f03698b69a7777b8d30b9c5897504f8704b87676)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c7fca75c035ba0b750f46a9676a376f8e4409f15
Component: engine
This reverts commit eb137ff1765faeb29c2d99025bfd8ed41836dd06, reversing
changes made to a79fabbfe84117696a19671f4aa88b82d0f64fc1.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 88330c9aac5556d0abc7a5afcb4d906604a07fa2
Component: engine
[18.09 backport] Graphdriver: fix "device" mode not being detected if "character-device" bit is set
Upstream-commit: 2e4c5c57c30522dc4b33b5cd5371f294ae3fee82
Component: engine
[18.09 backport] Allow overriding repository and branch in validate scripts, and no need to git fetch in CI
Upstream-commit: 02c953cf36fa4f017be402451106d9c273023b4f
Component: engine
- TestAPISwarmLeaderElection
- TestAPISwarmRaftQuorum
- TestSwarmClusterRotateUnlockKey
because they are known to be flaky.
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
(cherry picked from commit 02157c638ba0c325d8fd1debc1678e7e99eacfc1)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 278f1a130b66de23f07e472792b70b640f777666
Component: engine
Due to a bug in Golang (github.com/golang#27640), the "character device"
bit was omitted when checking file-modes with `os.ModeType`.
This bug was resolved in Go 1.12, but as a result, graphdrivers
would no longer recognize "device" files, causing pulling of
images that have a file with this filemode to fail;
failed to register layer:
unknown file type for /var/lib/docker/vfs/dir/.../dev/console
The current code checked for an exact match of Modes to be set. The
`os.ModeCharDevice` and `os.ModeDevice` bits will always be set in
tandem, however, because the code was only looking for an exact
match, this detection broke now that `os.ModeCharDevice` was added.
This patch changes the code to be more defensive, and instead
check if the `os.ModeDevice` bit is set (either with, or without
the `os.ModeCharDevice` bit).
In addition, some information was added to the error-message if
no type was matched, to assist debugging in case additional types
are added in future.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c7a38c2c06f7ab844a48c6c447942913131b83d6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 3744b45ba8ad93f1a21cbc80420856b04efc4593
Component: engine
* Replaces `cocks` with `cerf` as the former might be perceived as
offensive by some people (as pointed out by @jeking3
[here](https://github.com/moby/moby/pull/37157#commitcomment-31758059))
* Removes a duplicate entry for `burnell`
* Re-arranges the entry for `sutherland` to ensure that the names are in
sorted order
* Adds entries for `shamir` and `wilbur`
Signed-off-by: Debayan De <debayande@users.noreply.github.com>
(cherry picked from commit e50f791d42d1167a5ef757b1aa179e84f0f81bba)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a818442de73b16d8ad756c74e5e660d132e97848
Component: engine
By default, exec uses the environment of the current process, however,
if `exec.Env` is not `nil`, the environment is discarded:
e73f489494/src/os/exec/exec.go (L57-L60)
> If Env is nil, the new process uses the current process's environment.
When adding a new environment variable, prepend the current environment,
to make sure it is not discarded.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b84bff7f8ad1562a7d05f21bd84179d3306f2b4b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 19e733f89f7652f58b567b5178bacc10ef2940b5
Component: engine
Here's what happens:
1. One runs `make binary` once
2. Days go by...
3. One makes changes to dockerd sources
4. One runs `make test-integration` to test the changes
5. One spends a long time figuring out why on Earth
those changes in step 3 are ignored by step 4.
6. One writes this patch
7. ...
8. PROFIT!!
OK, so `make test-integration` builds a dockerd binary
in bundles/dynbinary-daemon/, when starts a daemon instance
for testing. The problem is, the script that starts the
daemon sets PATH to try `bundles/binary-daemon/` first,
and `bundles/dynbinary-daemon/` second.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 228bc35e824d09a6f40d60619808462124f0af31)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e9ecd5e486c591979e34782025bb849f7faf8eba
Component: engine
CIs are assumed to do a git fetch and git merge before running tests.
Therefore, no need for a git fetch inside our validate scripts in CI.
If VALIDATE_ORIGIN_BRANCH is set, then git fetch is skipped and
VALIDATE_ORIGIN_BRANCH is used in validate scripts.
Otherwise, behavior is unchanged.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit feb70fd5c9e2fb3f300e953dd83053f0830f3895)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 7b9ec00eec7ffe745ebd2f807daa50d84b3e10e7
Component: engine
When running CI in other repositories (e.g. Docker's downstream
docker/engine repository), or other branches, the validation
scripts were calculating the list of changes based on the wrong
information.
This lead to weird failures in CI in a branch where these values
were not updated ':-) (CI on a pull request failed because it detected
that new tests were added to the deprecated `integration-cli` test-suite,
but the pull request did not actually make changes in that area).
This patch allows overriding the target repository (and branch)
to compare to (without having to edit the scripts).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2a08f33166247da9d4c09d4c6c72cbb8119bf8df)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 748f37022df465c39a76461c4970f4c678f629e3
Component: engine
containerd has two objects with regard to containers.
There is a "container" object which is metadata and a "task" which is
manging the actual runtime state.
When docker starts a container, it creartes both the container metadata
and the task at the same time. So when a container exits, docker deletes
both of these objects as well.
This ensures that if, on start, when we go to create the container metadata object
in containerd, if there is an error due to a name conflict that we go
ahead and clean that up and try again.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 5ba30cd1dc6000ee53b34f628cbff91d7f6d7231)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1d0353548a21b0b60595ad44859a7072f90ea6c6
Component: engine
for windows all networks are re-populated in the store during network controller initialization. In current version it also regenerate network Ids which may be referenced by other components and it may cause broken references to a networks. This commit avoids regeneration of network ids.
Signed-off-by: Andrey Kolomentsev <andrey.kolomentsev@docker.com>
(cherry picked from commit e017717d96540dd263d95f90fdb2457928909924)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 317e0acc4e602f978e4d9c0130a113d179026c8e
Component: engine
containerd is now running as a separate service, and should
no longer be started as a managed child-process of dockerd.
The dockerd service already specifies that it should be started
`After` the containerd.service, but there is still a race
condition, where containerd is started, but its socket is not yet
created.
In that situation, `dockerd` detects that the containerd socket
is missing, and will start a new instance of containerd (as a
managed child-process), which causes live-restore to fail.
This patch explicitly sets the `--containerd` daemon option.
If this option is set, `dockerd` will not start a new instance
of containerd.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1985463b1337e55c0dfcad2fb985fb5af5a10c78)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 15653df497646518ec5af5cab55ebb4461ee5b80
Component: packaging
A git bundle allows us keep the same SHA, giving us the ability to
validate our patch against a known entity and allowing us to push
directly from our private forks to public forks without having to
re-apply any patches.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 03dfb0ba53cc5f64b746a25aa5ed8a48763ea223
Component: engine
- Fix a bug that a container can't be stopped or inspected when its corresponding image is deleted
- Fix a bug that the cri plugin handles containerd events outside of k8s.io namespace
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 27cc170d2891f25ab066620e8fe94d62f81b9aa3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f80c6d7ae15443f15c014ddbd17e30cfac47b906
Component: engine
RHEL/CentOS 3.10 kernels report that kernel-memory accounting is supported,
but it actually does not work.
Runc (when compiled for those kernels) will be compiled without kernel-memory
support, so even though the daemon may be reporting that it's supported,
it actually is not.
This cause tests to fail when testing against a daemon that's using a runc
version without kmem support.
For now, skip these tests based on the kernel version reported by the daemon.
This should fix failures such as:
```
FAIL: /go/src/github.com/docker/docker/integration-cli/docker_cli_run_unix_test.go:499: DockerSuite.TestRunWithKernelMemory
assertion failed:
Command: /usr/bin/docker run --kernel-memory 50M --name test1 busybox cat /sys/fs/cgroup/memory/memory.kmem.limit_in_bytes
ExitCode: 0
Error: <nil>
Stdout: 9223372036854771712
Stderr: WARNING: You specified a kernel memory limit on a kernel older than 4.0. Kernel memory limits are experimental on older kernels, it won't work as expected and can cause your system to be unstable.
Failures:
Expected stdout to contain "52428800"
FAIL: /go/src/github.com/docker/docker/integration-cli/docker_cli_update_unix_test.go:125: DockerSuite.TestUpdateKernelMemory
/go/src/github.com/docker/docker/integration-cli/docker_cli_update_unix_test.go:136:
...open /go/src/github.com/docker/docker/integration-cli/docker_cli_update_unix_test.go: no such file or directory
... obtained string = "9223372036854771712"
... expected string = "104857600"
----------------------------------------------------------------------
FAIL: /go/src/github.com/docker/docker/integration-cli/docker_cli_update_unix_test.go:139: DockerSuite.TestUpdateKernelMemoryUninitialized
/go/src/github.com/docker/docker/integration-cli/docker_cli_update_unix_test.go:149:
...open /go/src/github.com/docker/docker/integration-cli/docker_cli_update_unix_test.go: no such file or directory
... value = nil
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1e1156cf67233cf8eaee2da9c17465ff0d9c2aa0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e042692db1316a60be35bfdca10d7e08d20f50ad
Component: engine
Revert "[18.09 backport] API: fix status code on conflicting service names"
Upstream-commit: ce8b8f1cf33bb33811077be3b0b972dd2dbc4c79
Component: engine
This was removed in 662e248f680eb49a9951a8b34125506b8f82dfed. The
removal breaks usage on Debian derivatives that do not use systemd
(see https://github.com/docker/for-linux/issues/482).
Upstream-commit: 196c4028288ad241f52201d0f171c251ed13b98f
Component: packaging
The CLI package does not provide the functionality of the
"selinux" and "engine" packages (it does _conflict_ with
older engine packages though).
This removes the "obsoletes" from the CLI package, as the
Engine package already obsoletes the other ones.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d4e1ddb963a43ff9f0a06e4ed1b37307c8f26a17)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9e6f124b6b5deba3878ad242661d1991881f33b9
Component: packaging
Currently running the e2e tests produces a warning/error:
$ make -f docker.Makefile test-e2e
«...»
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock docker-cli-e2e
./scripts/test/e2e/run: line 20: test: : integer expression expected
This is from:
test "${DOCKERD_EXPERIMENTAL:-}" -eq "1" && «...»
Where `${DOCKERD_EXPERIMENTAL:-}` expands to the empty string, resulting in
`test "" -eq "1"` which produces the warning. This error is enough to trigger
the short-circuiting behaviour of `&&` so the result is as expected, but fix
the issue nonetheless by provdiing a default `0`.
Signed-off-by: Ian Campbell <ijc@docker.com>
(cherry picked from commit 4f483276cf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c462e06fcd
Component: cli
This includes a patch on top of containerd 1.2.1 to handle fifo
timeouts.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
(cherry picked from commit e5d9d721626958a37dccfa0b234d9fc96d8c2bfb)
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Upstream-commit: d161dfe1a36929a03ee3dfa916d296abfd4ccef1
Component: engine
go1.10.6 (released 2018/12/14)
- crypto/x509: CPU denial of service in chain validation golang/go#29233
- cmd/go: directory traversal in "go get" via curly braces in import paths golang/go#29231
- cmd/go: remote command execution during "go get -u" golang/go#29230
See the Go 1.10.6 milestone on the issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.10.6
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8afe9f422dc0183ce48e1db09189ccbde634080a
Component: engine
[18.09 backport] fixes display text in Multiple IDs found with provided prefix
Upstream-commit: a8572d3e8e1e3f738d6e9bc3bf0f137d64bab616
Component: engine
This avoids the corner case where `via` is not nil, but has a length of 0,
so the updated code does not panic in that situation.
Signed-off-by: Iskander Sharipov <quasilyte@gmail.com>
(cherry picked from commit a5c185b99404ea3fbab47ff9d7ba143392566bc1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 3482a3b14a6414977bd9860c513abf06dedd6bf7
Component: engine
This updates the swarmkit vendoring to the latest version in the bump_v18.09
branch
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ad7105260f3c2ff32a375ff78dce9a96e01d87cb
Component: engine
We don't use these spec files to actually build from anymore so we
should just go through and remove them.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit b98d7aad1e153c18da5b488afc40ba751f90cb36)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9d68fe72f3649fad5e3f902602c8ae9ab4a6abe5
Component: packaging
Without this the docker.socket would not start by default when starting
the docker.service leading to failures to start.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 88885d18b1bb0ef91eab4ad3311773f9c40838b7)
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Upstream-commit: 93f125a2602b7eb72128cbc16d9d35c9d92ed18a
Component: packaging
go1.10.5 (released 2018/11/02) includes fixes to the go command, linker,
runtime and the database/sql package. See the milestone on the issue
tracker for details:
List of changes; https://github.com/golang/go/issues?q=milestone%3AGo1.10.5
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 871d24d3fc
Component: cli
Removes the systemd drop-in unit file for socket activation and instead
prefers socket activation by default for both RHEL based and DEBIAN
based distributions.
Socket activation for RHEL based distributions was tested on CentOS 7 and Fedora 28.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 91c85cd38197b9d92d5b3e8a1b577d3178d73fcc)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 6a7e302c260a2ff7e4d8b7a6d01249a09c140618
Component: packaging
go1.10.5 (released 2018/11/02) includes fixes to the go command, linker,
runtime and the database/sql package. See the milestone on the issue
tracker for details:
List of changes; https://github.com/golang/go/issues?q=milestone%3AGo1.10.5
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 00ad8e7c5730f3c50ae2e548b47d1340202f72b2
Component: engine
@sw-pschmied originally post this in #38285
While looking through the Moby source code was found /proc/asound to be
shared with containers as read-only (as defined in
https://github.com/moby/moby/blob/master/oci/defaults.go#L128).
This can lead to two information leaks.
---
**Leak of media playback status of the host**
Steps to reproduce the issue:
- Listen to music/Play a YouTube video/Do anything else that involves
sound output
- Execute docker run --rm ubuntu:latest bash -c "sleep 7; cat
/proc/asound/card*/pcm*p/sub*/status | grep state | cut -d ' ' -f2 |
grep RUNNING || echo 'not running'"
- See that the containerized process is able to check whether someone
on the host is playing music as it prints RUNNING
- Stop the music output
- Execute the command again (The sleep is delaying the output because
information regarding playback status isn't propagated instantly)
- See that it outputs not running
**Describe the results you received:**
A containerized process is able to gather information on the playback
status of an audio device governed by the host. Therefore a process of a
container is able to check whether and what kind of user activity is
present on the host system. Also, this may indicate whether a container
runs on a desktop system or a server as media playback rarely happens on
server systems.
The description above is in regard to media playback - when examining
`/proc/asound/card*/pcm*c/sub*/status` (`pcm*c` instead of `pcm*p`) this
can also leak information regarding capturing sound, as in recording
audio or making calls on the host system.
Signed-off-by: Jonathan A. Schweder <jonathanschweder@gmail.com>
(cherry picked from commit 64e52ff3dbdb31adc0a9930b3ea74b04b0df8d86)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5fffdb32261145b1178f571e25fbd71572769d58
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
(cherry picked from commit 14c8b67e51a8340284dfd8919843ce9edd1e5c71)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9c93de59da8eaa0e0e0377578a33b465b9465bb2
Component: engine
This is needed for implementing bash completion for the `docker manifest`
command family.
Signed-off-by: Harald Albers <github@albersweb.de>
(cherry picked from commit a183c952c6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2a36695037
Component: cli
This preapares bash completion for more context sensitivity:
- experimental cli features
- orchestrator specific features
Also renames _daemon_ to _server_ where used in context of `docker version`
because the fields there are grouped unter _Server_.
Signed-off-by: Harald Albers <github@albersweb.de>
(cherry picked from commit 564d4da06e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: dc74fc81f2
Component: cli
A similar change was made in the CLI itself, but is not
inherited by the code that generates the YAML docs.
Before this patch is applied;
```
usage: docker container exec [OPTIONS] CONTAINER COMMAND [ARG...] [flags]
```
With this patch applied:
```
usage: docker container exec [OPTIONS] CONTAINER COMMAND [ARG...]
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 44d96e9120)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 7e90635652
Component: cli
The RPM packages list containerd as a hard dependency. While
having containerd installed allows certain features (e.g., allow
you to run `docker engine activate`), this should not be a requirement
for installing the Docker CLI, as it limits the use of this package
for situations where the CLI is installed to connect to a remote
daemon.
This patch removes the containerd dependency from the RPM packages
(the deb packages don't have this dependency, so no change is needed
in those packages)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 037349c5dc3f18d3bea3be0217e249f716a8be25)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: bc3867a8f6e59c56c967720c33c6ce9a65b64b7b
Component: packaging
During the migration to "image based builds", some dependencies
were removed.
This patch brings back those dependencies.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a6ff66f1a51af62c9fd17a8b931f8a9b0283247a
Component: packaging
This requirement was originally added in 86f76496ce33bd6eff1737348bc44add4723ddd2,
but got removed in the migration to the new image-based packaging.
Commit f2ceca98823548e401c6cca125b7057df28c4892 added this requirement
back for `.deb` packages, but did not include the same changes for
RPMs.
This patch adds back the requirement for RPM packages as well.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 86ed6eb2cef4e7386979fc73397c1844531eb3a9
Component: packaging
Nightly builds wouldn't install correctly since our versioning scheme
for nightly builds is 0.0.0~ and 0 < 18.
Should be backwards compatible with 18.09 builds so there's no need to
actually backport this to 18.09.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit de312224beff9c6bc7fd6def46459d531cf2ce4a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 3e383bee967aa07d35ea717544bc18aafbbad4f7
Component: packaging
This is to keep the UX consistent. `unused-for` is still accepted and a synonym.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 369da264bac15769ae944cc880e66abaf158612b)
Upstream-commit: 0b2d88d328ca88c8732dc11c72873b53be3bd2f8
Component: engine
This is cleaner because running system prune twice in a row
now results in a no-op the second time.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 6c10abb247)
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 4a4a1f3615
Component: cli
These options were added in API 1.39, so should be ignored
when using an older version of the API.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7632ccbc66c82179547b96524672bd5082ae7481)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 4cc45d91eb44cbaddcfd75335c3f72ede035c440
Component: engine
In newer kernels, AppArmor will reject attempts to send signals to a
container because the signal originated from outside of that AppArmor
profile. Correct this by allowing all unconfined signals to be received.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Signed-off-by: Aleksa Sarai <asarai@suse.de>
(cherry picked from commit 4822fb1e2423d88cdf0ad5d039b8fd3274b05401)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 67c602c3fe3d5d817fb7210e50d7ed1688b28801
Component: engine
The previous update used a commit from master. Now that
all the fixes are backported to the containerd 1.2 release
branch, we can switch back to that branch.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2fb5de68a9bd05b1dbf3ae3f7ae82bcd5e64dc5c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: db7f375d6a2aaf6d79f5c690e2f302c640bdde04
Component: engine
release notes: https://github.com/containerd/containerd/releases/tag/v1.2.0
- New V2 Runtime with a stable gRPC interface for managing containers through
external shims.
- Updated CRI Plugin, validated against Kubernetes v1.11 and v1.12, but it is
also compatible with Kubernetes v1.10.
- Support for Kubernetes Runtime Class, introduced in Kubernetes 1.12
- A new proxy plugin configuration has been added to allow external
snapshotters be connected to containerd using gRPC.-
- A new Install method on the containerd client allows users to publish host
level binaries using standard container build tooling and container
distribution tooling to download containerd related binaries on their systems.
- Add support for cleaning up leases and content ingests to garbage collections.
- Improved multi-arch image support using more precise matching and ranking
- Added a runtime `options` field for shim v2 runtime. Use the `options` field to
config runtime specific options, e.g. `NoPivotRoot` and `SystemdCgroup` for
runtime type `io.containerd.runc.v1`.
- Some Minor API additions
- Add `ListStream` method to containers API. This allows listing a larger
number of containers without hitting message size limts.
- Add `Sync` flag to `Delete` in leases API. Setting this option will ensure
a garbage collection completes before the removal call is returned. This can
be used to guarantee unreferenced objects are removed from disk after a lease.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8674930c84140c990451adb148165422d008b661)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e137337fe6083da91fd6d83d699cff3a857f636e
Component: engine
Containerd should be "leading" when specifying which version of runc to use.
From the RUNC.MD document in the containerd repository
(https://github.com/containerd/containerd/blob/b1e202c32724e82779544365528a1a082
b335553/RUNC.md);
> We depend on a specific runc version when dealing with advanced features. You
> should have a specific runc build for development. The current supported runc
> commit is described in vendor.conf. Please refer to the line that starts with
> github.com/opencontainers/runc.
This patch adds a note to vendor.conf and runc.installer to describe the order
in which runc should be updated.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit da3810d235bc0bd0197243d0128f258394554704)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c9c87d76d651d57d72e52c575a2c9600170b5212
Component: engine
log-opts are passed to logging-drivers as-is, so the daemon is not
aware what value-type each option takes.
For this reason, all options must be provided as a string, even if
they are used as numeric values by the logging driver.
For example, to pass the "max-file" option to the default (json-file)
logging driver, this value has to be passed as a string;
```json
{
"log-driver": "json-file",
"log-opts": {
"max-size": "10m",
"max-file": "3"
}
}
```
When passed as a _number_ (`"max-file": 3`), the daemon will invalidate
the configuration file, and fail to start;
unable to configure the Docker daemon with file /etc/docker/daemon.json: json: cannot unmarshal number into Go value of type string
This patch adds an example to the daemon.json to show these values
have to be passed as strings.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit fd33e0d933)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9b3eea87ee
Component: cli
- `--force` is not available in `service create`
- `--force` is a boolean option
Signed-off-by: Harald Albers <github@albersweb.de>
(cherry picked from commit 5fa5eb1da6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 046ffa4e87
Component: cli
On kernels without this options set, publishing ports for swarm
services does not work, making the published port not accessible
("connection refused")
Thanks to Wenbo Wang for reporting, and Tianon for finding this.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 44e1c6ce81ff62c76ba630caf16a73922950abc3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 25bec4665b6f011e52f7b2765ba1579c7430481d
Component: engine
Re-adds the docker.socket file for debian based distributions.
TAR-144 +review
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 69bd2eea398e0396c24f97d0697f9befd0f2464e)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: aaf035970219d6ae19e22bb94adaf51e61a9ff82
Component: packaging
In case we're running on RHEL7 kernel, which has non-working
and broken kernel memory controller, add 'nokmem' build tag
so that runc never enables kmem accounting.
For more info, see the following runc commit:
https://github.com/opencontainers/runc/commit/6a2c1559684
This behavior can be overriden by having `RUNC_NOKMEM` environment
variable set (e.g. to empty value to disable setting nokmem).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 8972aa9350d52e4a7e58242447b7a9d2f0c27f37)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8486ea11ae800a1e6d634b741dfb007ba29f6003
Component: engine
client.checkResponseErr() was hanging and consuming infinite memory
when the serverResp.Body io.Reader returns infinite stream.
This commit prohibits reading more than 1MiB.
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
(cherry picked from commit 1db4be0c32267b7a8a27998089ce96440bed492e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 49556e047059d81b64f6cd12f4c602c9a9c471c7
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
This is a fix for a few related scenarios where it's impossible to remove layers or containers
until the host is rebooted. Generally (or at least easiest to repro) through a forced daemon kill
while a container is running.
Possibly slightly worse than that, as following a host reboot, the scratch layer would possibly be leaked and
left on disk under the dataroot\windowsfilter directory after the container is removed.
One such example of a failure:
1. run a long running container with the --rm flag
docker run --rm -d --name test microsoft/windowsservercore powershell sleep 30
2. Force kill the daemon not allowing it to cleanup. Simulates a crash or a host power-cycle.
3. (re-)Start daemon
4. docker ps -a
PS C:\control> docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7aff773d782b malloc "powershell start-sl…" 11 seconds ago Removal In Progress malloc
5. Try to remove
PS C:\control> docker rm 7aff
Error response from daemon: container 7aff773d782bbf35d95095369ffcb170b7b8f0e6f8f65d5aff42abf61234855d: driver "windowsfilter" failed to remove root filesystem: rename C:\control\windowsfilter\7aff773d782bbf35d95095369ffcb170b7b8f0e6f8f65d5aff42abf61234855d C:\control\windowsfilter\7aff773d782bbf35d95095369ffcb170b7b8f0e6f8f65d5aff42abf61234855d-removing: Access is denied.
PS C:\control>
Step 5 fails.
(cherry picked from commit efdad5374465a2a889d7572834a2dcca147af4fb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 02fe71843e0e45ddc986a6c8182370b042349a27
Component: engine
The CloudWatch Logs API defines its limits in terms of bytes, but its
inputs in terms of UTF-8 encoded strings. Byte-sequences which are not
valid UTF-8 encodings are normalized to the Unicode replacement
character U+FFFD, which is a 3-byte sequence in UTF-8. This replacement
can cause the input to grow, exceeding the API limit and causing failed
API calls.
This commit adds logic for counting the effective byte length after
normalization and splitting input without splitting valid UTF-8
byte-sequences into two invalid byte-sequences.
Fixes https://github.com/moby/moby/issues/37747
Signed-off-by: Samuel Karp <skarp@amazon.com>
(cherry picked from commit 1e8ef386279e2e28aff199047e798fad660efbdd)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 757650e8dcca87f95ba083a80639769a0b6ca1cc
Component: engine
These tests don't seem to have anything Linux-specific,
so enable them on Windows
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b334198e652b503d432832ec6f4faab7707d0eb4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e8eb3ca4eef1c913563195787eb1f2527c9febf3
Component: engine
Some improvements in this test;
- use the volume-information that's returned by VolumeCreate as "expected"
- don't use an explict name for the volume, as it was only used to reference
the volume for inspection
- improve the test-output on failure, so that "expected" and "actual" values
are printed
Without this patch applied;
=== RUN TestVolumesInspect
--- FAIL: TestVolumesInspect (0.02s)
volume_test.go:108: assertion failed: false (bool) != true (true bool): Time Volume is CreatedAt not equal to current time
FAIL
With this patch applied;
=== RUN TestVolumesInspect
--- FAIL: TestVolumesInspect (0.02s)
volume_test.go:95: assertion failed: expression is false: createdAt.Truncate(time.Minute).Equal(now.Truncate(time.Minute)): CreatedAt (2018-11-01 16:15:20 +0000 UTC) not equal to creation time (2018-11-01 16:15:20.2421166 +0000 UTC m=+13.733512701)
FAIL
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8e8cac82637952d49f8df42a3164478f13978c90)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 65bf95f3df84de5901479091685715f227c333ce
Component: engine
- Add windows CI entrypoint script.
Signed-off-by: John Howard <jhoward@microsoft.com>
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
(cherry picked from commit d3cc071bb98669545d4e3043c9bd85879292b815)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 59be98043a02f44b63b26f159461fed08292e027
Component: engine
Fixes a compatibility issue on recent debian versions, where iptables now uses
nft by default.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e1783a72d1b84bc3e32470c468d14445e5fba8db
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
(cherry picked from commit d03ab106624ebc30b69080d0092702ad1fe1285c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 41f3cea42f2f45244051b7f829c0ef9c27383c26
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
This removes the need for an SVM in the LCOW driver to ApplyDiff.
This change relates to a fix for https://github.com/moby/moby/issues/36353
However, it found another issue, tracked by https://github.com/moby/moby/issues/37955
(cherry picked from commit bde99960657818efabf313991867c42921882275)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9cf6464b639aef09df9b16999cc340a1276d0bf8
Component: engine
Only the filters the user specified should be added as cache filters to buildkit.
Make an AND operation of the provided filters.
ID filter now does prefix-matching.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit b6137bebb83e886aef906b7ff277778b69616991)
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 52a3c39506b883f713694ce39d1a4fd9f5638800
Component: engine
`d.Sock()` already returns the socket-path including the
`unix://` scheme.
Also removed `--iptables=false`, as it didn't really seem
nescessary for this test.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 14342046477b7ed87a9c5c451bb4520c7f6cabcc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c40a7d393bca990e07973024e71034b4b6fc05e5
Component: engine
4MB client side limit was introduced in vendoring go-grpc#1165 (v1.4.0)
making these requests likely to produce errors
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 489b8eda6674523df8b82a210399b7d2954427d0)
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Upstream-commit: 6ca0546f2571cf4acdc1f541bccfac23a78cb8d2
Component: engine
Hopefully this removes some confusion as to what this version number
should be.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 5bdfa19b8646839f9d704307aa6589c7d686db44)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 64b0c76151ceb7b26f9c7477f3044dac747d227b
Component: engine
Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
(cherry picked from commit f962bd06ed8824d1f75d8546b428965cd61bdf7f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5591f0b1ee7dec101b490228258613cd7caf64ee
Component: engine
- remove features/fixes that were backported to 18.06, so already shipped
- re-organised some change to be in a more suitable section
- add information about deprecated features, and distros
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Packages are no longer built for Ubuntu 17.10 since
commit eb635bdb3044b008344c8318c8d727ac94662526, so
remove it from the README as well
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9cab4c43c4dd30e86efed1dfb9199ed91d766eca)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 89705eb29d9f8782cf218a6d4692dd8a34a95857
Component: packaging
Docker no longer ships packages for Ubuntu 14.04,
so remove the build scripts as well.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3a8fa7fc0acc5c3fb468ec5ec41e236e741c804e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 90fadd7b66e8696cc0d3479293bc62ea730ec51e
Component: packaging
Docker is no longer packaged for Debian Jessie,
so remove the scripts.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 49524c88c2b2ab1751941992913adcff36487f7a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 146e29aa88e8598d5a89ae29a091ff303fd4f3e5
Component: packaging
Packages for Wheezy are no longer built, so remove it
from the list of packages in the README.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c8935f360bc083ed61984a691bcfe19e9cfa1749)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9e724cc033c392764c1bf3b37d40ccb4d2e66bcd
Component: packaging
Includes fixes to the go command, linker, and the net/http, mime/multipart,
ld/macho, bytes, and strings packages. See the Go 1.10.4 milestone on the
issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.10.4
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit fe1fb7417c360f185ce608e0bfda41c0ef9fe364)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 72368177254811e5816f03a4773deaafb9df5202
Component: engine
I am attempting to push a tag to a private repository. The documentation for `docker tag` has an explicit example to for how ["To push an image to a private registry"](https://docs.docker.com/engine/reference/commandline/tag/#tag-an-image-referenced-by-name). My colleague clarified that this command does not in fact push anything, so I thought this PR might save some future novice the same confusion.
Signed-off-by: Jake Lambert <jake.lambert@volusion.com>
(cherry picked from commit 4ed484bac4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1df47ffb4d
Component: cli
ddadd3db49 mass standardized the
formatting, with some errors.
This commit fixes errors on `login.md`:
- revert wrong `Logging out` headline
- restore correct level for some headlines (relative to parent
headline level change)
- re-add `Usage` headlines, with better name
Also add `related commands` headline on `login` and `logout`.
Signed-off-by: Thomas Riccardi <thomas@deepomatic.com>
(cherry picked from commit a0e3ec8790)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2e7e529a18
Component: cli
add regression case for the issue#37870
Signed-off-by: Wei Fu <fuweid89@gmail.com>
(cherry picked from commit 8e25f4ff6d89888a1bcd578f3f8f7aab89dce24d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ae6284a623bac86ac6ab718fa4a369dd8c0a3cfc
Component: engine
Avoid testing for session support in non-buildkit builder to support
servers that falsely report as `1.39` compatible
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 3e8c41beb0)
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 4c68a9666f
Component: cli
[18.09 backport] move the syslog syscall to be gated by CAP_SYS_ADMIN or CAP_SYSLOG
Upstream-commit: 6f1145e740f35a0b805a11d5d29daa89d2a27ed0
Component: engine
Removes the billing profile flow which is now handled on the back-end.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 8e565d0399
Component: cli
Bump libnetwork to 6da50d1978302f04c3e2089e29112ea24812f05b which
is the current tip of libnetwork's bump_18.09 branch to get the DSR load
balancing mode option changes for the 18.09 branch of Docker CE.
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: fd1fe0b702571865cc77d66937e4ca570b5b9cc3
Component: engine
The `devicemapper` storage driver is deprecated in favor of `overlay2`, and will
be removed in a future release. Users of the `devicemapper` storage driver are
recommended to migrate to a different storage driver, such as `overlay2`, which
is now the default storage driver.
The `devicemapper` storage driver facilitates running Docker on older (3.x) kernels
that have no support for other storage drivers (such as overlay2, or AUFS).
Now that support for `overlay2` is added to all supported distros (as they are
either on kernel 4.x, or have support for multiple lowerdirs backported), there
is no reason to continue maintenance of the `devicemapper` storage driver.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 662441ba31)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: fde819236b
Component: cli
The `overlay` storage driver is deprecated in favor of the `overlay2` storage
driver, which has all the benefits of `overlay`, without its limitations (excessive
inode consumption). The legacy `overlay` storage driver will be removed in a future
release. Users of the `overlay` storage driver should migrate to the `overlay2`
storage driver.
The legacy `overlay` storage driver allowed using overlayFS-backed filesystems
on pre 4.x kernels. Now that all supported distributions are able to run `overlay2`
(as they are either on kernel 4.x, or have support for multiple lowerdirs
backported), there is no reason to keep maintaining the `overlay` storage driver.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8bc2aa45a6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: aa6314c663
Component: cli
For some reason, shared mount propagation between the host
and a container does not work for btrfs, unless container
root directory (i.e. graphdriver home) is a bind mount.
The above issue was reproduced on SLES 12sp3 + btrfs using
the following script:
#!/bin/bash
set -eux -o pipefail
# DIR should not be under a subvolume
DIR=${DIR:-/lib}
MNT=$DIR/my-mnt
FILE=$MNT/file
ID=$(docker run -d --privileged -v $DIR:$DIR:rshared ubuntu sleep 24h)
docker exec $ID mkdir -p $MNT
docker exec $ID mount -t tmpfs tmpfs $MNT
docker exec $ID touch $FILE
ls -l $FILE
umount $MNT
docker rm -f $ID
which fails this way:
+ ls -l /lib/my-mnt/file
ls: cannot access '/lib/my-mnt/file': No such file or directory
meaning the mount performed inside a priviledged container is not
propagated back to the host (even if all the mounts have "shared"
propagation mode).
The remedy to the above is to make graphdriver home a bind mount.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 16d822bba8ac5ab22c8697750f700403bca3dbf3)
Upstream-commit: fa8ac946165b8004a15e85744e774ed6ba99fd38
Component: engine
This function ensures the argument is the mount point
(i.e. if it's not, it bind mounts it to itself).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 8abadb36fa8149cd44e76b0e7fdedd6f1f2eccd0)
Upstream-commit: 2199ada691dc635cac5cdd065d909a539dd0b793
Component: engine
1. There is no need to specify rw argument -- bind mounts are
read-write by default.
2. There is no point in parsing /proc/self/mountinfo after performing
a mount, especially if we don't check whether the fs is mounted or
not -- the only outcome from it could be an error from our mountinfo
parser, which makes no sense in this context.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit f01297d1ae352bc2bf01ebf62e879c1c83cdbee4)
Upstream-commit: fd7611ff1f1d61d5b4b45b2c0bd83976cbccf174
Component: engine
The `overlay` storage driver is deprecated in favor of the `overlay2` storage
driver, which has all the benefits of `overlay`, without its limitations (excessive
inode consumption). The legacy `overlay` storage driver will be removed in a future
release. Users of the `overlay` storage driver should migrate to the `overlay2`
storage driver.
The legacy `overlay` storage driver allowed using overlayFS-backed filesystems
on pre 4.x kernels. Now that all supported distributions are able to run `overlay2`
(as they are either on kernel 4.x, or have support for multiple lowerdirs
backported), there is no reason to keep maintaining the `overlay` storage driver.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 31be4e0ba11532e5d6df8401f5d459e292c58f36)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c20e8dffbbb4dd328e4c00d18e04eecb80cf4d4e
Component: engine
The `devicemapper` storage driver is deprecated in favor of `overlay2`, and will
be removed in a future release. Users of the `devicemapper` storage driver are
recommended to migrate to a different storage driver, such as `overlay2`, which
is now the default storage driver.
The `devicemapper` storage driver facilitates running Docker on older (3.x) kernels
that have no support for other storage drivers (such as overlay2, or AUFS).
Now that support for `overlay2` is added to all supported distros (as they are
either on kernel 4.x, or have support for multiple lowerdirs backported), there
is no reason to continue maintenance of the `devicemapper` storage driver.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 06fcabbaa0d4f25db0580d3604bbed708223fb06)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 734e7a8e55e41b064025273776dc5a76e45bf2e1
Component: engine
Unlike `docker build --secret`, `docker build --ssh` allows the build container to
use SSH keys with passphrases.
$ eval $(ssh-agent)
$ ssh-add ~/.ssh/id_rsa
(Input your passphrase here)
$ docker build --ssh default=$SSH_AUTH_SOCK ...
This feature requires the daemon with `CapExecMountSSH` build capability (moby/moby#37973) .
Currently, the official Dockerfile frontend does not provide the syntax for using the SSH forwarder.
However, the experimental `RUN --mount=type=ssh` syntax can be enabled by using
the Dockerfile frontend image built with the `BUILDTAGS="dfrunmount dfssh"`, via the `# syntax =` "shebang".
The Dockerfile for the Dockerfile frontend is available at github.com/moby/buildkit/frontend/dockerfile/cmd/dockerfile-frontend)
The pre-built image is also available as `tonistiigi/dockerfile:ssh20181002` .
An example Dockerfile with `RUN --mount=type=ssh`:
# syntax = tonistiigi/dockerfile:ssh20181002
FROM alpine
RUN apk add --no-cache openssh-client
RUN mkdir -p -m 0700 ~/.ssh && ssh-keyscan gitlab.com >> ~/.ssh/known_hosts
RUN --mount=type=ssh ssh git@gitlab.com | tee /hello
# "Welcome to GitLab, @GITLAB_USERNAME_ASSOCIATED_WITH_SSHKEY" should be printed here
More info available at moby/buildkit#608, moby/buildkit#655
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
(cherry picked from commit db7399a016)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e942084530
Component: cli
[18.09] backport fix denial of service with large numbers in cpuset-cpus and cpuset-mems
Upstream-commit: 4b8336f7cf091fd5c4742286bda1e34c45667d78
Component: engine
As pointed out in https://github.com/moby/moby/issues/37970,
Docker overlay driver can't work with index=on feature of
the Linux kernel "overlay" filesystem. In case the global
default is set to "yes", Docker will fail with EBUSY when
trying to mount, like this:
> error creating overlay mount to ...../merged: device or resource busy
and the kernel log should contain something like:
> overlayfs: upperdir is in-use by another mount, mount with
> '-o index=off' to override exclusive upperdir protection.
A workaround is to set index=off in overlay kernel module
parameters, or even recompile the kernel with
CONFIG_OVERLAY_FS_INDEX=n in .config. Surely this is not
always practical or even possible.
The solution, as pointed out my Amir Goldstein (as well as
the above kernel message:) is to use 'index=off' option
when mounting.
NOTE since older (< 4.13rc1) kernels do not support "index="
overlayfs parameter, try to figure out whether the option
is supported. In case it's not possible to figure out,
assume it is not.
NOTE the default can be changed anytime (by writing to
/sys/module/overlay/parameters/index) so we need to always
use index=off.
[v2: move the detection code to Init()]
[v3: don't set index=off if stat() failed]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 8422d85087bfa770b62ef4e1daaca95ee6783d86)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 690e097fedd7362f3b2781c32ca872ad966d286e
Component: engine
This simplifies the code a lot.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit a55d32546a8556f9e6cabbc99836b573b9944f0c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: dc0a4db7c9dc593a8568a8e30e4e21e118c2839d
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
(cherry picked from commit c907c2486c0850030f8ac1819ac8c87631472c68)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 7184074c0880c656be00645007588a00ec2266cd
Component: engine
The stdin fifo of exec process is created in containerd side after
client calls Start. If the client calls CloseIO before Start call, the
stdin of exec process is still opened and wait for close.
For this case, client closes stdinCloseSync channel after Start.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
(cherry picked from commit c7890f25a9eaae8d07614bd85b2b3231b03e54ec)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 6679a5faeb724f1ad060f2fdf6d189f1005924b9
Component: engine
We'll be using a redirect from this URL to the back-end docs system for
hosting release notes. Final location confirmed with Docs team and PM.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: f93908213a
Component: cli
Using a value such as `--cpuset-mems=1-9223372036854775807` would cause
`dockerd` to run out of memory allocating a map of the values in the
validation code. Set limits to the normal limit of the number of CPUs,
and improve the error handling.
Reported by Huawei PSIRT.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f8e876d7616469d07b8b049ecb48967eeb8fa7a5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0922d32bce74657266aff213f83dfa638e8077f4
Component: engine
This updates the containerd dependencies to match
the versions used by the vendored containerd version
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 31a9c9e79101cdf38d383104afbc1b48ede75291)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5070e418b806cc96ad0f5b3ac32c8d416ff8449a
Component: engine
With containerd reaching 1.0, the runtime now
has a stable API, so there's no need to do a check
if the installed version matches the expected version.
Current versions of Docker now also package containerd
and runc separately, and can be _updated_ separately.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c65f0bd13c85d29087419fa555281311091825e7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 054c3c2931cec5dca8bb84af97f1457c343ec02f
Component: engine
We check for max value for -default-addr-pool-mask-length param as 32.
But There won't be enough addresses on the overlay network. Hence we are
keeping it 29 so that we would be having atleast 8 addresses in /29 network.
Signed-off-by: selansen <elango.siva@docker.com>
(cherry picked from commit d25c5df80e60cdbdc23fe3d0e2a6808123643dc7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9406f3622d18a0d9b6c438190e8fdd8be53d3b22
Component: engine
Addressing few review comments as part of code refactoring.
Also moved validation logic from CLI to Moby.
Signed-off-by: selansen <elango.siva@docker.com>
(cherry picked from commit 148ff00a0a800fad99de11ee3021d4c5d4869157)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9816bfcaf58a609d64d648043c10817c27dcfa36
Component: engine
[18.09] Fix long startup on windows, with non-hns governed Hyper-V networks
Upstream-commit: 6e5ed2ccce0923950dae7a6cb9bd1dd851ed5af8
Component: engine
Similar to a related issue where previously, private Hyper-V networks
would each add 15 secs to the daemon startup, non-hns governed internal
networks are reported by hns as network type "internal" which is not
mapped to any network plugin (and thus we get the same plugin load retry
loop as before).
This issue hits Docker for Desktop because we setup such a network for
the Linux VM communication.
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
(cherry picked from commit 6a1a4f97217b0a8635bc21fc86628f48bf8824d1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 54bd14a3fe1d4925c6fa88b24949063d99067c07
Component: engine
The parameter name was wrong, which may mislead a user.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit c378fb774e413ba8bf5cadf655d2b67e9c94245a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c9ddc6effc444c54def41d498b359a9a986ad79d
Component: engine
This call is what is used to implement `dmesg` to get kernel messages
about the host. This can leak substantial information about the host.
It is normally available to unprivileged users on the host, unless
the sysctl `kernel.dmesg_restrict = 1` is set, but this is not set
by standard on the majority of distributions. Blocking this to restrict
leaks about the configuration seems correct.
Fix#37897
See also https://googleprojectzero.blogspot.com/2018/09/a-cache-invalidation-bug-in-linux.html
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
(cherry picked from commit ccd22ffcc8b564dfc21e7067b5248819d68c56c6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 16836e60bc87abb3e9ab16f33c2038931c1d473b
Component: engine
This should ensure we don't produce builds without a valid "Built:"
field in the the version payload.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 9a071a3fbe316ea6d70bc1a5111cbe14ae99aec9
Component: packaging
During the refactoring for 18.09 the activate/update flows no longer
restart the engine explicitly but let the user do that when they're ready,
so the health check logic is no longer required.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: f2b2061cc3
Component: cli
This is a follow up PR to #1381 to address some of the review comments
we didn't get to.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: c12e23a4c1
Component: cli
Help the user understand which license they're about
to load in case they have multiple licenses they need to
figure out.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 5a97a93ae1
Component: cli
These messages were enhanced to include the path that was
missing (in df6af282b9048dfedcd7b7a9a89126aca887f4e1), but
also changed the first part of the message.
This change complicates running e2e tests with mixed versions
of the engine.
Looking at the full error message, "mount" is a bit redundant
as well, because the error message already indicates this is
about a "mount";
docker run --rm --mount type=bind,source=/no-such-thing,target=/foo busybox
docker: Error response from daemon: invalid mount config for type "bind": bind mount source path does not exist: /no-such-thing.
Removing the "mount" part from the error message, because
it was redundant, and makes cross-version testing easier :)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 574db7a53782c57554089c9606505af1c108df0b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b499acc0e834e11882909269238407c65f68f034
Component: engine
People blow away `/var/lib/docker` all the time so we probably shouldn't
store important data there.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 9391057c9472ba24049b8645c251a4c63894522f)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 865140fc4155481b20f694a4528e04e48eb76de4
Component: packaging
Epoch needs to be included in the Replaces / Breaks sections
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit f6a3e274b4892f8f501e1191a73d8b8935a56aa5)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: bebe7028c6b41ecee80278cdfc46de1e7704d199
Component: packaging
For static builds.
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 43ebdc0845840c13c2f92c7b564ddb56eb38d428
Component: packaging
Due to a typo, substitution would not work if the given
environment-variable was set.
Given the following docker compose file;
```yaml
version: "3.7"
services:
app:
image: nginx:${version:-latest}
```
Deploying a stack with `$version` set would ignore the `$version`
environment variable, and use the default value instead;
```bash
version=alpine docker stack deploy -c docker-compose.yml foobar
Creating network foobar_default
Creating service foobar_app
docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
rskkjxe6sm0w foobar_app replicated 1/1 nginx:latest
```
This patch also fixes "soft default" not detecting empty environment variables,
only non-set environment variables.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ec3daea021)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b8702b8a9a
Component: cli
This allows to run the daemon in environments that have upstream containerd installed.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 34eede0296bce6a9c335cb429f10728ae3f4252d)
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: b3bb2aabb8ed5a8af0a9f48fb5aba3f39af38e0d
Component: engine
This allows to provide more information for build cache disk usage.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit a90b99edfc)
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 9de1318e36
Component: cli
[18.09] backport fixing daemon won't start when "runtimes" option defined in both config file and cli
Upstream-commit: 90a90ae2e1fdb1f9782a51e41fe2b7dc80d48952
Component: engine
The packages will deliver this as a link so lets make sure we don't
write through the link to the underlying packaged file.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 0f22d7e295
Component: cli
If, for some reason, the certs directory has permissions that are
inaccessible by docker, we should still be able to fetch manifests using
the `insecure` flag.
Since the cli doesn't access the engine's list of insecure registries,
the registry client should make a singleton list of the registry being queried with the
`insecure` flag.
Closes#1358
Signed-off-by: Christy Norman <christy@linux.vnet.ibm.com>
(cherry picked from commit d57adbc034)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f9d666b057
Component: cli
These files are not used, and having them in repo is misleading.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: fca673ad8051d1b377435821ec6c76c0de24b663
Component: packaging
Set the PATH to what appears to be the standard on latest Ubuntu (18.04)
and Debian (9), fixing the following two issues:
1. PATH did not contain /bin (leading to ContainerTop/ps not working
on newer distros, among the other things).
2. $PATH can't be specified in Environment directives in .service files.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 62d9a0d1253049a2fab10f938542a8e09dd0804a
Component: packaging
Adapt the CLI to the host install model for 18.09.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 342afe44fb
Component: cli
The `/etc/docker` directory is used both by the dockerd daemon
and the docker cli (if installed on the saem host as the daemon).
In situations where the `/etc/docker` directory does not exist,
and an initial `key.json` (legacy trust key) is generated (at the
default location), the `/etc/docker/` directory was created with
0700 permissions, making the directory only accessible by `root`.
Given that the `0600` permissions on the key itself already protect
it from being used by other users, the permissions of `/etc/docker`
can be less restrictive.
This patch changes the permissions for the directory to `0755`, so
that the CLI (if executed as non-root) can also access this directory.
> **NOTE**: "strictly", this patch is only needed for situations where no _custom_
> location for the trustkey is specified (not overridden with `--deprecated-key-path`),
> but setting the permissions only for the "default" case would make
> this more complicated.
```bash
make binary shell
make install
ls -la /etc/ | grep docker
dockerd
^C
ls -la /etc/ | grep docker
drwxr-xr-x 2 root root 4096 Sep 14 12:11 docker
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cecd9817177093be99c1c9bb0dcf43ccec14ad1d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: fc576226b24e8b5db6e95e48967d56c5808f9fe9
Component: engine
devicemapper cannot be enabled on a statically compiled binary so we
dynamically compile it on a rhel based distribution in order to enable
devicemapper usage.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 7cb8d358947974abae3e3c346b09523904030609)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: c80ac56dfa393c405c2cd33af94ad6ac1366ece2
Component: packaging
iptables is sometimes placed in `/usr/sbin`
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit c86a958d6d344d6bf568523c28d6643d7b49ba43)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 36a08784a0b7bc49487005d323bf4e41fd50d36d
Component: packaging
[18.09] backport always hornor client side to choose which builder to use with DOCKER_…
Upstream-commit: 5fb0a7ced7bcdb9e86e048fe3b24d0d4297d6155
Component: engine
I could not reproduce the panic in #37735, so here's a bandaid.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 7dac70324d0ce6acd23458b0bef06f099837d648)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2c26eac56628527ed64c79ce9145ed97583cbeca
Component: engine
Saw this in moby ci:
> 00:22:07.582 === RUN TestServiceWithDefaultAddressPoolInit
> 00:22:08.887 --- FAIL: TestServiceWithDefaultAddressPoolInit (1.30s)
> 00:22:08.887 daemon.go:290: [d905878b35bb9] waiting for daemon to start
> 00:22:08.887 daemon.go:322: [d905878b35bb9] daemon started
> 00:22:08.888 panic: runtime error: index out of range [recovered]
> 00:22:08.889 panic: runtime error: index out of range
> 00:22:08.889
> 00:22:08.889 goroutine 360 [running]:
> 00:22:08.889 testing.tRunner.func1(0xc42069d770)
> 00:22:08.889 /usr/local/go/src/testing/testing.go:742 +0x29d
> 00:22:08.890 panic(0x85d680, 0xb615f0)
> 00:22:08.890 /usr/local/go/src/runtime/panic.go:502 +0x229
> 00:22:08.890 github.com/docker/docker/integration/network.TestServiceWithDefaultAddressPoolInit(0xc42069d770)
> 00:22:08.891 /go/src/github.com/docker/docker/integration/network/service_test.go:348 +0xb53
> .....
Apparently `out.IPAM.Config[0]` is not there, so to avoid panic, let's
check the size of `out.IPAM.Config` first.
Fixes: f7ad95cab9c
[v2: add logging of data returned by NetworkInspect()]
[v3: use assert.Assert to fail immediately]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 69d3a8936b8d91e43290aadb9f7ac2cd2929dc61)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f43fc6650cf5a452157fe081086098c124a426fd
Component: engine
Prevents looping of startup errors such as containerd
not being found on the path.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
(cherry picked from commit ce0b0b72bcc58de82ba1e0e7127499d07f678122)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 85361af1f749517c8bdfd3d36b0df94a92e29b2b
Component: engine
… so moving it in test files for now.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
(cherry picked from commit 37ca5d6813)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 62aed95bc1
Component: cli
osusergo build tag is needed for better chances to have
a proper static binary when Go >= 1.11 is used, and is
harmless otherwise.
seccomp build tag is needed so the resulting dockerd binary
has seccomp support.
[v2: add libseccomp-dev installation]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 97596405f4300a7ae6eb7664f6c58785955dd8d3
Component: packaging
Should generate correct bits whether or not the suffix is present.
Working example:
❯ ./gen-deb-ver . 18.09.0-beta1
18.09.0~1.1.beta1 18.09.0-beta1
❯ ./gen-deb-ver . 18.09.0-ce-beta1
18.09.0~ce~1.1.beta1 18.09.0-ce-beta1
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 9de3f6fb573806abc2e546447c41552ff90731a4)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: cc336d0804076b31d6d059b164e9b4e3b3cec886
Component: packaging
Includes fixes related to upgrade cycles
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 83a20d53f14adcd2bcc9e932435aa66b6e056afa)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: befa3c61954d12f2413df56a384989d7ae2cb9b3
Component: packaging
A bit overdue
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 11643a30e0bdde4016afe42ab5f1f85e0fc83ecf)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: f38206fa477c59ec39b06b8ddde40be1f65c7cf0
Component: packaging
docker load doesn't support OCI based tar files so we have to make 2
different ones.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 696cdc009ca73b070d8877ee5cdd13c4b0e4a39b)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: ea811c67ee1e558ecec1b394fc591458b57f7ed1
Component: packaging
The official access point for the Q3 engine images will
be prefixed by store.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 3597d75281
Component: cli
This is set in a top-level makefile, but apparently
isn't getting passed all the way through as expected.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: a5ad5471419cb2a0935148f891c5ada8b13b4779
Component: packaging
Also cleans up the target in rpm and deb Makfiles because
the tar target already output a tar.
Signed-off-by: Jose Bigio <jose.bigio@docker.com>
(cherry picked from commit 8a4744f48aa13bea0b4e9174165be4add0a5a42f)
Signed-off-by: Jose Bigio <jose.bigio@docker.com>
Upstream-commit: 8e4a92dc667e2143b6d5707e19c5564b7da46ba1
Component: packaging
… this removes a whole lot of dependencies from people depending on docker/cli…
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
(cherry picked from commit 2d344b2f61)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: fc3dc8f058
Component: cli
… this is, for now, the only platform that is supported
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
(cherry picked from commit a3a955f204)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2a46a3d46c
Component: cli
This will make it easier for anybody to build with a specific version of a distro they want to build on.
They could lock into a specific sha of a docker image if they wanted, as an example.
Signed-off-by: corbin-coleman <corbin.coleman@docker.com>
Upstream-commit: 7afc35b95ac46bfef383788341ddb0b157af14eb
Component: packaging
This should accomodate the new versioning fix
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: be00095e1d0d5674665a63b6e5932178632feb0e
Component: packaging
Removes the need for the offline installer to install the shim process
and instead installs the shim process as part of the packaging.
May be easier in the future to just package the shim process on it's own
but that'll come after this 18.09 release
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit f8bd366d58f8bdf8a82b9a033353ca5bf4eda948)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 31d0cb047b98ab43f661bd026bdd63deef62543d
Component: packaging
This should test that
- all the messages produced are delivered (i.e. not lost)
- followLogs() exits
Loosely based on the test having the same name by Brian Goff, see
https://gist.github.com/cpuguy83/e538793de18c762608358ee0eaddc197
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit f845d76d047760c91dc0c7076aea840291fbdbc5)
Upstream-commit: 2a82480df9ad91593d59be4b5283917dbea2da39
Component: engine
When daemon.ContainerLogs() is called with options.follow=true
(as in "docker logs --follow"), the "loggerutils.followLogs()"
function never returns (even then the logs consumer is gone).
As a result, all the resources associated with it (including
an opened file descriptor for the log file being read, two FDs
for a pipe, and two FDs for inotify watch) are never released.
If this is repeated (such as by running "docker logs --follow"
and pressing Ctrl-C a few times), this results in DoS caused by
either hitting the limit of inotify watches, or the limit of
opened files. The only cure is daemon restart.
Apparently, what happens is:
1. logs producer (a container) is gone, calling (*LogWatcher).Close()
for all its readers (daemon/logger/jsonfilelog/jsonfilelog.go:175).
2. WatchClose() is properly handled by a dedicated goroutine in
followLogs(), cancelling the context.
3. Upon receiving the ctx.Done(), the code in followLogs()
(daemon/logger/loggerutils/logfile.go#L626-L638) keeps to
send messages _synchronously_ (which is OK for now).
4. Logs consumer is gone (Ctrl-C is pressed on a terminal running
"docker logs --follow"). Method (*LogWatcher).Close() is properly
called (see daemon/logs.go:114). Since it was called before and
due to to once.Do(), nothing happens (which is kinda good, as
otherwise it will panic on closing a closed channel).
5. A goroutine (see item 3 above) keeps sending log messages
synchronously to the logWatcher.Msg channel. Since the
channel reader is gone, the channel send operation blocks forever,
and resource cleanup set up in defer statements at the beginning
of followLogs() never happens.
Alas, the fix is somewhat complicated:
1. Distinguish between close from logs producer and logs consumer.
To that effect,
- yet another channel is added to LogWatcher();
- {Watch,}Close() are renamed to {Watch,}ProducerGone();
- {Watch,}ConsumerGone() are added;
*NOTE* that ProducerGone()/WatchProducerGone() pair is ONLY needed
in order to stop ConsumerLogs(follow=true) when a container is stopped;
otherwise we're not interested in it. In other words, we're only
using it in followLogs().
2. Code that was doing (logWatcher*).Close() is modified to either call
ProducerGone() or ConsumerGone(), depending on the context.
3. Code that was waiting for WatchClose() is modified to wait for
either ConsumerGone() or ProducerGone(), or both, depending on the
context.
4. followLogs() are modified accordingly:
- context cancellation is happening on WatchProducerGone(),
and once it's received the FileWatcher is closed and waitRead()
returns errDone on EOF (i.e. log rotation handling logic is disabled);
- due to this, code that was writing synchronously to logWatcher.Msg
can be and is removed as the code above it handles this case;
- function returns once ConsumerGone is received, freeing all the
resources -- this is the bugfix itself.
While at it,
1. Let's also remove the ctx usage to simplify the code a bit.
It was introduced by commit a69a59ffc7e3d ("Decouple removing the
fileWatcher from reading") in order to fix a bug. The bug was actually
a deadlock in fsnotify, and the fix was just a workaround. Since then
the fsnofify bug has been fixed, and a new fsnotify was vendored in.
For more details, please see
https://github.com/moby/moby/pull/27782#issuecomment-416794490
2. Since `(*filePoller).Close()` is fixed to remove all the files
being watched, there is no need to explicitly call
fileWatcher.Remove(name) anymore, so get rid of the extra code.
Should fix https://github.com/moby/moby/issues/37391
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 916eabd459fe707b5c4a86377d12e2ad1871b353)
Upstream-commit: 84a5b528aede5579861201e869870d10fc98c07c
Component: engine
This test case checks that followLogs() exits once the reader is gone.
Currently it does not (i.e. this test is supposed to fail) due to #37391.
[kolyshkin@: test case Brian Goff, changelog and all bugs are by me]
Source: https://gist.github.com/cpuguy83/e538793de18c762608358ee0eaddc197
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit d37a11bfbab83ab42b1160f116e863daac046192)
Upstream-commit: 511741735e0aa2fe68a66d99384c00d187d1a157
Component: engine
This code has many return statements, for some of them the
"end logs" or "end stream" message was not printed, giving
the impression that this "for" loop never ended.
Make sure that "begin logs" is to be followed by "end logs".
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 2e4c2a6bf9cb47fd07e42f9c043024ed3dbcd04d)
Upstream-commit: 2b8bc86679b7153bb4ace063a858637df0f16a2e
Component: engine
The code in Close() that removes the watches was not working,
because it first sets `w.closed = true` and then calls w.close(),
which starts with
```
if w.closed {
return errPollerClosed
}
```
Fix by setting w.closed only after calling w.remove() for all the
files being watched.
While at it, remove the duplicated `delete(w.watches, name)` code.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit fffa8958d00860b4e3563327a2cc6836a12d4ba9)
Upstream-commit: 4e2dbfa1af48191126b0910b9463bf94d8371886
Component: engine
There is no need to wait for up to 200ms in order to close
the file descriptor once the chClose is received.
This commit might reduce the chances for occasional "The process
cannot access the file because it is being used by another process"
error on Windows, where an opened file can't be removed.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit dfbb64ea7d042d5b2bb0c1c2b88e3682b7069b10)
Upstream-commit: 3a3bfcbf47e98212abfc9cfed860d9e99fc41cdc
Component: engine
In case of errors, the file descriptor is never closed. Fix it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 88bcf1573ca2eaffc15da346a1651a3749567554)
Upstream-commit: 7be43586af6824c1e55cb502d9d2bab45c9b4505
Component: engine
To include https://github.com/vbatts/tar-split/pull/48 which
fixes the issue of creating an image with >8GB file in it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 92e75439037d205c218208945c70cfb3633e87aa)
Upstream-commit: d7085abec2e445630bedd3e79782c5ec33f62682
Component: engine
Add a test case for creating a 8GB file inside a container.
Due to a bug in tar-split this was failing in Docker 18.06.
The file being created is sparse, so there's not much I/O
happening or disk space being used -- meaning the test is
fast and does not require a lot of disk space.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit b3165f5b2d7c1063c2a76a7ed5c2dd18f868276b)
Upstream-commit: fc1d808c44f77e3929c4eeba7066501890aecd4e
Component: engine
And update the associated packages that have also updated from
docker/docker vendor.conf.
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Upstream-commit: 4e6798794d
Component: cli
Fix default case causing the throttling to not be used.
Ensure that nil client condition is handled.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
(cherry picked from commit c3e32938430e03a316311f9e4fbdb743e492a07e)
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Upstream-commit: f121eccf29576ce5d4b8256a71a9d32ee688ff7d
Component: engine
This patch adds --filter, --keep-storage, --all and --force to builder prune.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit c806eb49c9)
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 9d43f1ed48
Component: cli
It's generally bad-form to generate two identically named package files
that are actually different, so we should name our packages based on
the distro and version, not just the distro.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
(cherry picked from commit 13bde85f0067a751f775d31919705645ecc0a6d7)
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 3c757b281bb623d9622c74143340fd892c8655ad
Component: packaging
Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229
(6bf0f408e4)
both the old, and new location are accepted by systemd 229 and up, so using the old location
to make them work for either version of systemd.
StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230
(f0367da7d1)
both the old, and new name are accepted by systemd 230 and up, so using the old name to make
this option work for either version of systemd.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2c2bfea5d009fb884f5c61b62e3c85bd88e7909e)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 5963decd1502e5012692a1f958bfdc5403f02920
Component: packaging
This adds support for reloading the docker daemon
(SIGHIUP) so that changes in '/etc/docker/daemon.json'
can be loaded at runtime by reloading the service
through systemd ('systemctl reload docker')
Before this change, systemd would output an error
that "reloading" is not supported for the docker
service;
systemctl reload docker
Failed to reload docker.service: Job type reload is not applicable for unit docker.service.
After this change, the docker daemon can be reloaded
through 'systemctl reload docker', which reloads
the configuration;
journalctl -f -u docker.service
May 02 03:49:20 testing systemd[1]: Reloading Docker Application Container Engine.
May 02 03:49:20 testing docker[28496]: time="2016-05-02T03:49:20.143964103-04:00" level=info msg="Got signal to reload configuration, reloading from: /etc/docker/daemon.json"
May 02 03:49:20 testing systemd[1]: Reloaded Docker Application Container Engine.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3e1b508e5f70b35869d9c8417d3b65a141820af9)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 127426fc693a882b670339391da5d12f08c1438d
Component: packaging
Change the kill mode to process so that systemd does not kill container
processes when the daemon is shutdown but only the docker daemon
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d736ae9da7401f582469fa7e943adde9f2163024)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 784c5936ec984b5525ac71eca3b7731e49eb6b37
Component: packaging
We need to add delegate yes to docker's service file so that it can
manage the cgroups of the processes that it launches without systemd
interfering with them and moving the processes after it is reloaded.
Delegate=
Turns on delegation of further resource control partitioning to
processes of the unit. For unprivileged services (i.e. those
using the User= setting), this allows processes to create a
subhierarchy beneath its control group path. For privileged
services and scopes, this ensures the processes will have all
control group controllers enabled.
This is the proper fix for issue moby/moby#20152
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e134e666a585b1f13e9e5e371dd93e5ce04a4b34)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 1120496ca0492d29731befd1460a934ac115e7f8
Component: packaging
Systemd sets a default of 512 tasks, which is far
too low to run many containers.
Note that TasksMax is only supported on systemd 226
and above.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 82fe96733f551d36018c3840cf21d813807e9b76)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: c42e4736e055c472284fd09ca05d405d31c4516a
Component: packaging
There is a not-insignificant performance overhead for all containers (if
containerd is a child of Docker, which is the current setup) if systemd
sets rlimits on the main Docker daemon process (because the limits
propogate to all children).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 68e15413dce8d6e8f428ac286641a9482d30aabc)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 1246dedcd55058a2856c7042509fb0badcefbf91
Component: packaging
set LimitCORE=infinity to ensure complete core creation,
allows extraction of as much information as possible.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit da69663b9ccd72d95ec60007ef707d82a35324fa)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: f93f4525173e3a32d315794344d44bae780de00e
Component: packaging
Includes fixes to the go command, linker, and the net/http, mime/multipart,
ld/macho, bytes, and strings packages. See the Go 1.10.4 milestone on the
issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.10.4
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 11b4630f4c73885123336b7917cc139c43cf413a)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 3920ea81ea2cab8e1f5f17fb16e133cfb87d9c98
Component: packaging
[18.09] backport: fix regression when filtering container names using a leading slash
Upstream-commit: b8a4fe5f8f58bccfa3763ca75c6465447c8ccd06
Component: engine
In case a volume is specified via Mounts API, and SELinux is enabled,
the following error happens on container start:
> $ docker volume create testvol
> $ docker run --rm --mount source=testvol,target=/tmp busybox true
> docker: Error response from daemon: error setting label on mount
> source '': no such file or directory.
The functionality to relabel the source of a local mount specified via
Mounts API was introduced in commit 5bbf5cc and later broken by commit
e4b6adc, which removed setting mp.Source field.
With the current data structures, the host dir is already available in
v.Mountpoint, so let's just use it.
Fixes: e4b6adc
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 4032b6778df39f53fda0e6e54f0256c9a3b1d618
Component: engine
Bump our kube dependencies to the latest patch
level for kube 1.11.
Signed-off-by: Marcus Martins <marcus@docker.com>
(cherry picked from commit c67e05796b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0efb62cab1
Component: cli
Commit 5c8da2e96738addd8a175e5b9a23b8c37d4a4dfe updated the filtering behavior
to match container-names without having to specify the leading slash.
This change caused a regression in situations where a regex was provided as
filter, using an explicit leading slash (`--filter name=^/mycontainername`).
This fix changes the filters to match containers both with, and without the
leading slash, effectively making the leading slash optional when filtering.
With this fix, filters with and without a leading slash produce the same result:
$ docker ps --filter name=^a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
21afd6362b0c busybox "sh" 2 minutes ago Up 2 minutes a2
56e53770e316 busybox "sh" 2 minutes ago Up 2 minutes a1
$ docker ps --filter name=^/a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
21afd6362b0c busybox "sh" 2 minutes ago Up 2 minutes a2
56e53770e316 busybox "sh" 3 minutes ago Up 3 minutes a1
$ docker ps --filter name=^b
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b69003b6a6fe busybox "sh" About a minute ago Up About a minute b1
$ docker ps --filter name=^/b
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b69003b6a6fe busybox "sh" 56 seconds ago Up 54 seconds b1
$ docker ps --filter name=/a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
21afd6362b0c busybox "sh" 3 minutes ago Up 3 minutes a2
56e53770e316 busybox "sh" 4 minutes ago Up 4 minutes a1
$ docker ps --filter name=a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
21afd6362b0c busybox "sh" 3 minutes ago Up 3 minutes a2
56e53770e316 busybox "sh" 4 minutes ago Up 4 minutes a1
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6f9b5ba810e9314ab9335490cd41316940a32b5e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5fa80da2d385520f2c3c91e8ce23d181e4f93097
Component: engine
While testing on older ubuntu images we discovered
we do depend on a newer version of libseccomp2.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: f2ceca98823548e401c6cca125b7057df28c4892
Component: packaging
If the system has a containerd-proxy managing the lifecycle
of the daemon, make sure the config is updated with the new image
on update.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: f96ddaedf7
Component: cli
Basic enterprise licenses and platform license keys will be signed with
two different keys in the upcoming release. This adds support for the
CLI to support both variants.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 0fb6bb35a4
Component: cli
During refactoring of the implementation PR progress reporting
was broken. This gets the progress reporting back in action.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 7f4c842e8a
Component: cli
This encodes the platform as "Docker Engine - Community"
for community engines
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 4d9fd7d17f3aab3463baf6030072f5789a003a74
Component: packaging
This should make it easier to change downstream builds
to change the scope.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 72406691244c12d5360889f8658bd4989b588364
Component: packaging
The other way was not working.
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
(cherry picked from commit f00df1bf1efa2ed1ea508ea7b1c291500441571d)
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Upstream-commit: a9b0387bb0f40bd7687d8653a0714724eb66f3b9
Component: packaging
Should rely on list in the daemon
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 605758124d0750c14d24bfb1ebcf77d102591f4a)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 6cc81c2e8c95127baac275750bc544452fec7799
Component: packaging
BuildKit can now be enabled without the daemon having
experimental features enabled.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 60c75fda67
Component: cli
Warnings are now generated by the daemon, and returned as
part of the /info API response.
If warnings are returned by the daemon; use those instead
of generating them locally.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 3c27ce21c9
Component: cli
The remote API allows full privilege escalation and is equivalent to
having root access on the host. Because of this, the API should never
be accessible through an insecure connection (TCP without TLS, or TCP
without TLS verification).
Although a warning is already logged on startup if the daemon uses an
insecure configuration, this warning is not very visible (unless someone
decides to read the logs).
This patch attempts to make insecure configuration more visible by sending
back warnings through the API (which will be printed when using `docker info`).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 547b993e07330f3e74cba935975fce05e8661381
Component: engine
This feature brings new attribute/option for swarm init command.
default-addr-pool will take string input which can be in below format.
"CIDR,CIDR,CIDR...:SUBNET-SIZE".
Signed-off-by: selansen <elango.siva@docker.com>
Upstream-commit: 587a94c935
Component: cli
When requesting information about the daemon's configuration through the `/info`
endpoint, missing features (or non-recommended settings) may have to be presented
to the user.
Detecting these situations, and printing warnings currently is handled by the
cli, which results in some complications:
- duplicated effort: each client has to re-implement detection and warnings.
- it's not possible to generate warnings for reasons outside of the information
returned in the `/info` response.
- cli-side detection has to be updated for new conditions. This means that an
older cli connecting to a new daemon may not print all warnings (due to
it not detecting the new conditions)
- some warnings (in particular, warnings about storage-drivers) depend on
driver-status (`DriverStatus`) information. The format of the information
returned in this field is not part of the API specification and can change
over time, resulting in cli-side detection no longer being functional.
This patch adds a new `Warnings` field to the `/info` response. This field is
to return warnings to be presented by the user.
Existing warnings that are currently handled by the CLI are copied to the daemon
as part of this patch; This change is backward-compatible with existing
clients; old client can continue to use the client-side warnings, whereas new
clients can skip client-side detection, and print warnings that are returned by
the daemon.
Example response with this patch applied;
```bash
curl --unix-socket /var/run/docker.sock http://localhost/info | jq .Warnings
```
```json
[
"WARNING: bridge-nf-call-iptables is disabled",
"WARNING: bridge-nf-call-ip6tables is disabled"
]
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a3d4238b9ce653d2863fbc93057ed4162a83221e
Component: engine
The daemon won't actually start without containerd
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 16037618282dc5076089c7b749cdd07376650f8d
Component: packaging
Blocks the execution of tasks during the Prepare phase until there
exists an IP address for every overlay network in use by the task. This
prevents a task from starting before the NetworkAttachment containing
the IP address has been sent down to the node.
Includes a basic test for the correct use case.
Signed-off-by: Drew Erny <drew.erny@docker.com>
Upstream-commit: 3c81dc3103d9c88cb333c80e0810f80ab80c374e
Component: engine
This feature allows user to specify list of subnets for global
default address pool. User can configure subnet list using
'swarm init' command. Daemon passes the information to swarmkit.
We validate the information in swarmkit, then store it in cluster
object. when IPAM init is called, we pass subnet list to IPAM driver.
Signed-off-by: selansen <elango.siva@docker.com>
Upstream-commit: f7ad95cab9cc7ba8925673a933028d53284c13f5
Component: engine
vndr buildkit to e8c7acc
vndr containerd to v1.2.0-beta.0
vndr console to 4d8a41f
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Upstream-commit: 7a73d112ff
Component: cli
This new collection of commands supports initializing a local
engine using containerd, updating that engine, and activating
the EE product
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: fd2f1b3b66
Component: cli
* Expose license status in Info
This wires up a new field in the Info payload that exposes the license.
For moby this is hardcoded to always report a community edition.
Downstream enterprise dockerd will have additional licensing logic wired
into this function to report details about the current license status.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
* Code review comments
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
* Add windows autogen support
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 896d1b1c61a48e2df1a7b4644ddde6ee97db6111
Component: engine
Makes it so that you can overwrite them better if you include this
makefile in another makefile...
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 446f8d31394ae35fc38187aa5e40627b33a95a27
Component: packaging
Was reporting 404's using the old url for some reason, might have to
refactor this in the future to just use a git clone...
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 763ad6aa7ac24c269dc1dfa223771671acffc8fc
Component: packaging
Allows us to switch out debian files and rpm specs on the fly and also
gives us an out to have ARCH specific dockerfiles if we need them for
RPM's. The same strategy can be used for DEB's if need be
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: b72dc2edb8329151bfe2df6989fb329f41e720fb
Component: packaging
This driver uses protobuf to store log messages and has better defaults
for log file handling (e.g. compression and file rotation enabled by
default).
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: a351b38e7217af059eb2f8fc3dba14dc03836a45
Component: engine
This patch adds a new builder subcommand, allowing to add more builder-related
commands in the future. Unfortunately `build` expects an argument so could not
be used as a subcommand.
This also implements `docker builder prune`, which is needed to prune the builder
cache manually without having to call `docker system prune`.
Today when relying on the legacy builder, users are able to prune dangling images
(used as build cache) by running `docker image prune`. This patch allows the
same usecase with buildkit.
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: f597f2d026
Component: cli
This patch implements `docker build --secret id=mysecret,src=/secret/file`
for buildkit frontends that request the mysecret secret.
It is currently implemented in the tonistiigi/dockerfile:secrets20180808
frontend via RUN --mount=type=secret,id=mysecret
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: c4c4825591
Component: cli
Signed-off-by: John Howard <jhoward@microsoft.com>
Fixes#36764
@johnstep PTAL. @jterry75 FYI.
There are two commits in this PR. The first ensure that errors are actually returned to the caller - it was being thrown away.
The second commit changes the LCOW driver to map, on a per service VM basis, "long" container paths such as `/tmp/c8fa0ae1b348f505df2707060f6a49e63280d71b83b7936935c827e2e9bde16d` to much shorter paths, based on a per-service VM counter, so something more like /tmp/d3. This means that the root cause of the failure where the mount call to create the overlay was failing due to command line length becomes something much shorter such as below.
`mount -t overlay overlay -olowerdir=/tmp/d3:/tmp/d4:/tmp/d5:/tmp/d6:/tmp/d7:/tmp/d8:/tmp/d9:/tmp/d10:/tmp/d11:/tmp/d12:/tmp/d13:/tmp/d14:/tmp/d15:/tmp/d16:/tmp/d17:/tmp/d18:/tmp/d19:/tmp/d20:/tmp/d21:/tmp/d22:/tmp/d23:/tmp/d24:/tmp/d25:/tmp/d26:/tmp/d27:/tmp/d28:/tmp/d29:/tmp/d30:/tmp/d31:/tmp/d32:/tmp/d33:/tmp/d34:/tmp/d35:/tmp/d36:/tmp/d37:/tmp/d38:/tmp/d39:/tmp/d40:/tmp/d41:/tmp/d42:/tmp/d43:/tmp/d44:/tmp/d45:/tmp/d46:/tmp/d47:/tmp/d48:/tmp/d49:/tmp/d50:/tmp/d51:/tmp/d52:/tmp/d53:/tmp/d54:/tmp/d55:/tmp/d56:/tmp/d57:/tmp/d58:/tmp/d59:/tmp/d60:/tmp/d61:/tmp/d62,upperdir=/tmp/d2/upper,workdir=/tmp/d2/work /tmp/c8fa0ae1b348f505df2707060f6a49e63280d71b83b7936935c827e2e9bde16d-mount`
For those worrying about overflow (which I'm sure @thaJeztah will mention...): It's safe to use a counter here as SVMs are disposable in the default configuration. The exception is when running the daemon in unsafe LCOW "global" mode (ie `--storage-opt lcow.globalmode=1`) where the SVMs aren't disposed of, but a single one is reused. However, to overflow the command line length, it would require several hundred-thousand trillion (conservative, I should sit down and work it out accurately if I get -really- bored) of SCSI hot-add operations, and even to hit that would be hard as just running containers normally uses the VPMEM path for the containers UVM, not to the global SVM on SCSI. It gets incremented by one per build step (commit more accurately) as a general rule. Hence it would be necessary to have to be doing automated builds without restarting the daemon for literally years on end in unsafe mode. 😇
Note that in reality, the previous limit of ~47 layers before hitting the command line length limit is close to what is possible in the platform, at least as of RS5/Windows Server 2019 where, in the HCS v1 schema, a single SCSI controller is used, and that can only support 64 disks per controller per the Hyper-V VDEV. And remember we have one slot taken up for the SVMs scratch, and another for the containers scratch when committing a layer. So the best you can architecturally get on the platform is around the following (it's also different by 1 depending on whether in unsafe or default mode)
```
PS E:\docker\build\36764\short> docker build --no-cache .
Sending build context to Docker daemon 2.048kB
Step 1/4 : FROM alpine as first
---> 11cd0b38bc3c
Step 2/4 : RUN echo test > /test
---> Running in 8ddfe20e5bfb
Removing intermediate container 8ddfe20e5bfb
---> b0103a00b1c9
Step 3/4 : FROM alpine
---> 11cd0b38bc3c
Step 4/4 : COPY --from=first /test /test
---> 54bfae391eba
Successfully built 54bfae391eba
PS E:\docker\build\36764\short> cd ..
PS E:\docker\build\36764> docker build --no-cache .
Sending build context to Docker daemon 4.689MB
Step 1/61 : FROM alpine as first
---> 11cd0b38bc3c
Step 2/61 : RUN echo test > /test
---> Running in 02597ff870db
Removing intermediate container 02597ff870db
---> 3096de6fc454
Step 3/61 : RUN echo test > /test
---> Running in 9a8110f4ff19
Removing intermediate container 9a8110f4ff19
---> 7691808cf28e
Step 4/61 : RUN echo test > /test
---> Running in 9afb8f51510b
Removing intermediate container 9afb8f51510b
---> e42a0df2bb1c
Step 5/61 : RUN echo test > /test
---> Running in fe977ed6804e
Removing intermediate container fe977ed6804e
---> 55850c9b0479
Step 6/61 : RUN echo test > /test
---> Running in be65cbfad172
Removing intermediate container be65cbfad172
---> 0cf8acba70f0
Step 7/61 : RUN echo test > /test
---> Running in fd5b0907b6a9
Removing intermediate container fd5b0907b6a9
---> 257a4493d85d
Step 8/61 : RUN echo test > /test
---> Running in f7ca0ffd9076
Removing intermediate container f7ca0ffd9076
---> 3baa6f4fa2d5
Step 9/61 : RUN echo test > /test
---> Running in 5146814d4727
Removing intermediate container 5146814d4727
---> 485b9d5cf228
Step 10/61 : RUN echo test > /test
---> Running in a090eec1b743
Removing intermediate container a090eec1b743
---> a7eb10155b51
Step 11/61 : RUN echo test > /test
---> Running in 942660b288df
Removing intermediate container 942660b288df
---> 9d286a1e2133
Step 12/61 : RUN echo test > /test
---> Running in c3d369aa91df
Removing intermediate container c3d369aa91df
---> f78be4788992
Step 13/61 : RUN echo test > /test
---> Running in a03c3ac6888f
Removing intermediate container a03c3ac6888f
---> 6504363f61ab
Step 14/61 : RUN echo test > /test
---> Running in 0c3c2fca3f90
Removing intermediate container 0c3c2fca3f90
---> fe3448b8bb29
Step 15/61 : RUN echo test > /test
---> Running in 828d51c76d3b
Removing intermediate container 828d51c76d3b
---> 870684e3aea0
Step 16/61 : RUN echo test > /test
---> Running in 59a2f7c5f3ad
Removing intermediate container 59a2f7c5f3ad
---> cf84556ca5c0
Step 17/61 : RUN echo test > /test
---> Running in bfb4e088eeb3
Removing intermediate container bfb4e088eeb3
---> 9c8f9f652cef
Step 18/61 : RUN echo test > /test
---> Running in f1b88bb5a2d7
Removing intermediate container f1b88bb5a2d7
---> a6233ad21648
Step 19/61 : RUN echo test > /test
---> Running in 45f70577d709
Removing intermediate container 45f70577d709
---> 1b5cc52d370d
Step 20/61 : RUN echo test > /test
---> Running in 2ce231d5043d
Removing intermediate container 2ce231d5043d
---> 4a0e17cbebaa
Step 21/61 : RUN echo test > /test
---> Running in 52e4b0928f1f
Removing intermediate container 52e4b0928f1f
---> 99b50e989bcb
Step 22/61 : RUN echo test > /test
---> Running in f7ba3da7460d
Removing intermediate container f7ba3da7460d
---> bfa3cad88285
Step 23/61 : RUN echo test > /test
---> Running in 60180bf60f88
Removing intermediate container 60180bf60f88
---> fe7271988bcb
Step 24/61 : RUN echo test > /test
---> Running in 20324d396531
Removing intermediate container 20324d396531
---> e930bc039128
Step 25/61 : RUN echo test > /test
---> Running in b3ac70fd4404
Removing intermediate container b3ac70fd4404
---> 39d0a11ea6d8
Step 26/61 : RUN echo test > /test
---> Running in 0193267d3787
Removing intermediate container 0193267d3787
---> 8062d7aab0a5
Step 27/61 : RUN echo test > /test
---> Running in f41f45fb7985
Removing intermediate container f41f45fb7985
---> 1f5f18f2315b
Step 28/61 : RUN echo test > /test
---> Running in 90dd09c63d6e
Removing intermediate container 90dd09c63d6e
---> 02f0a1141f11
Step 29/61 : RUN echo test > /test
---> Running in c557e5386e0a
Removing intermediate container c557e5386e0a
---> dbcd6fb1f6f4
Step 30/61 : RUN echo test > /test
---> Running in 65369385d855
Removing intermediate container 65369385d855
---> e6e9058a0650
Step 31/61 : RUN echo test > /test
---> Running in d861fcc388fd
Removing intermediate container d861fcc388fd
---> 6e4c2c0f741f
Step 32/61 : RUN echo test > /test
---> Running in 1483962b7e1c
Removing intermediate container 1483962b7e1c
---> cf8f142aa055
Step 33/61 : RUN echo test > /test
---> Running in 5868934816c1
Removing intermediate container 5868934816c1
---> d5ff87cdc204
Step 34/61 : RUN echo test > /test
---> Running in e057f3201f3a
Removing intermediate container e057f3201f3a
---> b4031b7ab4ac
Step 35/61 : RUN echo test > /test
---> Running in 22b769b9079c
Removing intermediate container 22b769b9079c
---> 019d898510b6
Step 36/61 : RUN echo test > /test
---> Running in f1d364ef4ff8
Removing intermediate container f1d364ef4ff8
---> 9525cafdf04d
Step 37/61 : RUN echo test > /test
---> Running in 5bf505b8bdcc
Removing intermediate container 5bf505b8bdcc
---> cd5002b33bfd
Step 38/61 : RUN echo test > /test
---> Running in be24a921945c
Removing intermediate container be24a921945c
---> 8675db44d1b7
Step 39/61 : RUN echo test > /test
---> Running in 352dc6beef3d
Removing intermediate container 352dc6beef3d
---> 0ab0ece43c71
Step 40/61 : RUN echo test > /test
---> Running in eebde33e5d9b
Removing intermediate container eebde33e5d9b
---> 46ca4b0dfc03
Step 41/61 : RUN echo test > /test
---> Running in f920313a1e85
Removing intermediate container f920313a1e85
---> 7f3888414d58
Step 42/61 : RUN echo test > /test
---> Running in 10e2f4dc1ac7
Removing intermediate container 10e2f4dc1ac7
---> 14db9e15f2dc
Step 43/61 : RUN echo test > /test
---> Running in c849d6e89aa5
Removing intermediate container c849d6e89aa5
---> fdb770494dd6
Step 44/61 : RUN echo test > /test
---> Running in 419d1a8353db
Removing intermediate container 419d1a8353db
---> d12e9cf078be
Step 45/61 : RUN echo test > /test
---> Running in 0f1805263e4c
Removing intermediate container 0f1805263e4c
---> cd005e7b08a4
Step 46/61 : RUN echo test > /test
---> Running in 5bde05b46441
Removing intermediate container 5bde05b46441
---> 05aa426a3d4a
Step 47/61 : RUN echo test > /test
---> Running in 01ebc84bd1bc
Removing intermediate container 01ebc84bd1bc
---> 35d371fa4342
Step 48/61 : RUN echo test > /test
---> Running in 49f6c2f51dd4
Removing intermediate container 49f6c2f51dd4
---> 1090b5dfa130
Step 49/61 : RUN echo test > /test
---> Running in f8a9089cd725
Removing intermediate container f8a9089cd725
---> b2d0eec0716d
Step 50/61 : RUN echo test > /test
---> Running in a1697a0b2db0
Removing intermediate container a1697a0b2db0
---> 10d96ac8f497
Step 51/61 : RUN echo test > /test
---> Running in 33a2332c06eb
Removing intermediate container 33a2332c06eb
---> ba5bf5609c1c
Step 52/61 : RUN echo test > /test
---> Running in e8920392be0d
Removing intermediate container e8920392be0d
---> 5b3a95685c7e
Step 53/61 : RUN echo test > /test
---> Running in 4b9298587c65
Removing intermediate container 4b9298587c65
---> d4961a349141
Step 54/61 : RUN echo test > /test
---> Running in 8a0c960c2ba1
Removing intermediate container 8a0c960c2ba1
---> b413197fcfa2
Step 55/61 : RUN echo test > /test
---> Running in 536ee3b9596b
Removing intermediate container 536ee3b9596b
---> fc16b69b224a
Step 56/61 : RUN echo test > /test
---> Running in 8b817b8d7b59
Removing intermediate container 8b817b8d7b59
---> 2f0896400ff9
Step 57/61 : RUN echo test > /test
---> Running in ab0ed79ec3d4
Removing intermediate container ab0ed79ec3d4
---> b4fb420e736c
Step 58/61 : RUN echo test > /test
---> Running in 8548d7eead1f
Removing intermediate container 8548d7eead1f
---> 745103fd5a38
Step 59/61 : RUN echo test > /test
---> Running in 1980559ad5d6
Removing intermediate container 1980559ad5d6
---> 08c1c74a5618
Step 60/61 : FROM alpine
---> 11cd0b38bc3c
Step 61/61 : COPY --from=first /test /test
---> 67f053c66c27
Successfully built 67f053c66c27
PS E:\docker\build\36764>
```
Note also that subsequent error messages once you go beyond current platform limitations kind of suck (such as insufficient resources with a bunch of spew which is incomprehensible to most) and we could do better to detect this earlier in the daemon. That'll be for a (reasonably low-priority) follow-up though as and when I have time. Theoretically we *may*, if the platform doesn't require additional changes for RS5, be able to have bigger platform limits using the v2 schema with up to 127 VPMem devices, and the possibility to have multiple SCSI controllers per SVM/UVM. However, currently LCOW is using HCS v1 schema calls, and there's no plans to rewrite the graphdriver/libcontainerd components outside of the moving LCOW fully over to the containerd runtime/snapshotter using HCS v2 schema, which is still some time off fruition.
PS OK, while waiting for a full run to complete, I did get bored. Turns out it won't overflow line length as max(uint64) is 18446744073709551616 which would still be short enough at 127 layers, double the current platform limit. And I could always change it to hex or base36 to make it even shorter, or remove the 'd' from /tmp/dN. IOW, pretty sure no-one is going to hit the limit even if we could get the platform to 256 which is the current Hyper-V SCSI limit per VM (4x64), although PMEM at 127 would be the next immediate limit.
Upstream-commit: f586fe5637bbadd919ce4a126c6d95e4f3b1523b
Component: engine
Get's rid of architecture specific dockerfiles (yay manifest lists),
also follows very closely to what the RPM makefile does with the
sources.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: ab1ba336ad1720a166e8ed4469c616851d89a8d1
Component: packaging
Old versions of things on CentOS 7 strike again!
infinity is not a thing for TimeoutSec on systemd < 229
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 86dafe5391e2ef298248536cd038e27230a4df63
Component: packaging
Why didn't we do this the whole time? `¯\_(ツ)_/¯`
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: df2a0d6fc5a59a7a8b352689238eddf5a46d3ed4
Component: packaging
Modify hack/make.ps1 to use the version value used in
"FROM golang" statement.
While at it:
1. Make search expression a bit more strict (use ^ to match at BOL only).
2. Simplify by removing Get-Contents as Select-String can read files.
After this, ENV GO_VERSION can be removed from Dockerfile.
Unfortunately it can't be done in one commit as Windows CI
fails (presumably because Dockerfile is being modified in
place).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 6a2851332d5f86bb61a69d277f663f369377037b
Component: engine
rpmlint checks can be re-added later
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: c245ce12c1e0a092ef21094b287b8cb94a51ee1e
Component: packaging
Also split off containerd.mk into it's own thing since most of this
stuff will be re-used with debian packaging anyways.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 5513629a1b4868d82a41249eac634edbff51e315
Component: packaging
Splits out the docker-ce package and docker-ce-cli package into their
own things.
Still TODO: need to have a cleanup on the dependencies for the
Dockerfiles
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 2a27fb1dad0f91c25acb885c117563a33f6c5e88
Component: packaging
1. After running d.Cmd(), in case an error is returned, it makes sense
to print command output, as its stderr may contain a clue about what
went wrong. This is by no means complete, just as far as I could go.
2. In case the comment in c.Assert is a constant string, it's better
to provide it as a comment which will be printed.
3. An arbitrary string should not be passed on to a function expecting
%-style formatting. Use %s to fix this.
4. Print the output string before transformation, not after.
5. Unify the output format (drop "out:" prefix").
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: ac038eab298eeecab55a62440de302e3bb3f4889
Component: engine
It is wrong to pass an arbitrary string to a function expecting
%-style formatting. One solution would be to replace any % with %%,
but it's easier to just do what this patch does.
Generated with:
for f in $(git grep -l 'check.Commentf(out)'); do \
sed -i -e 's/check\.Commentf(out)/check.Commentf("%s", out)/g' $f; \
done
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 83363fb2d4d97d952a0052d079faa8ae39aa20b6
Component: engine
This implements chown support on Windows. Built-in accounts as well
as accounts included in the SAM database of the container are supported.
NOTE: IDPair is now named Identity and IDMappings is now named
IdentityMapping.
The following are valid examples:
ADD --chown=Guest . <some directory>
COPY --chown=Administrator . <some directory>
COPY --chown=Guests . <some directory>
COPY --chown=ContainerUser . <some directory>
On Windows an owner is only granted the permission to read the security
descriptor and read/write the discretionary access control list. This
fix also grants read/write and execute permissions to the owner.
Signed-off-by: Salahuddin Khan <salah@docker.com>
Upstream-commit: 763d8392612942ff5c32a35f8bdafd7ae93d3321
Component: engine
Also moved package deps into spec file BuildRequires. Had to hard-code
Epoch in spec file in order to realize deps with yum-builddep.
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Upstream-commit: 07f46e8e96692c511c04f29fbc78c2aa9592e479
Component: packaging
Also moved package deps into spec file BuildRequires. Had to hard-code
Epoch in spec file in order to realize deps with yum-builddep.
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Upstream-commit: b09bb012dd220e822aa6e90ef20d9cb83ccada8b
Component: packaging
Also moved package deps into spec file BuildRequires. Had to hard-code
Epoch in spec file in order to realize deps with yum-builddep.
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Upstream-commit: 784a53cec5061e6a718acc3a7104c61215c15078
Component: packaging
Looks like yum-builddep will only work if rpmlint passed on the spec
file and the spec file errors out on Epoch not being hardcoded.
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Upstream-commit: 42b49f6556022ecba47801f4639ddbc7864c9fe0
Component: packaging
This makes it so consumers of `LogFile` should pass in how to get an
io.Reader to the requested number of lines to tail.
This is also much more efficient when tailing a large number of lines.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 94a10150f64a24793216f5324a34e62be3f31a2d
Component: engine
This fixes a bug when the build results in an error and instead of sending
the logs of the container to the client, it signals a cancelation.
The context should not be wired to the status request, as only the
build requests need to be canceled.
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: f1ef16c736b477be17f6bc1bce18d0958a87d840
Component: engine
This changes the experimental --console flag to --progress following
feedback indicating avoidable confusion.
In addition to naming changes, the help output now has an additional
clarification, specifically: container output during builds are only
shown when progress output is set to plain. Not mentioning this was also
a big cause of confusion.
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: faeb8bb571
Component: cli
Relevant changes;
- swarmkit #2593 agent: return error when failing to apply network key
- swarmkit #2645 Replace deprecated grpc functions
- swarmkit #2720 Test if error is nil before to log it
- swarmkit #2712 [orchestrator] Fix task sorting
- swarmkit #2677 [manager/orchestrator/reaper] Fix the condition used for skipping over running tasks
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 660fa129c0ea09c1aa1770b5f2d26ec762213484
Component: engine
This splits out the CLI into a discrete package and removes
the engine from the engine package. Instead the engine
is initialized via a post-inst script using the new CLI UX.
Upstream-commit: 662e248f680eb49a9951a8b34125506b8f82dfed
Component: packaging
… as it is possible to do it when interpolating. It also fixes when
there is 2 variables on the same *value* (in the composefile, on the
same line)
Finaly, renaming the default, used in cli, pattern to `defaultPattern`
to not be shadowed unintentionally.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 4c87725c35
Component: cli
e.g. docker -H ssh://me@server
The `docker` CLI also needs to be installed on the remote host to
provide `docker system dial-stdio`, which proxies the daemon socket to stdio.
Please refer to docs/reference/commandline/dockerd.md .
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Upstream-commit: 6f61cf053a
Component: cli
This fix migrates some ipc container tests from integration-cli
to integration test.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 80c92c9b692b79273200b6e37f14a6d4e984ab8d
Component: engine
Changed unexpected Unicode character 0x1fbf GREEK PSILI that was standing in as an imposter for an apostrophe: an _impostrophe_.
Signed-off-by: Chad Faragher <wyckster@hotmail.com>
Upstream-commit: b59823c784
Component: cli
For the Q3 release the names will be refined slightly, and "ce" will no
longer be used - it's now just the docker engine, with "community" or
"enterprise" at the end if disambiguation is needed.
Upstream-commit: e69e4ea4997e468d20d93247ba19cd5e86daf24f
Component: packaging
It allows to get easily all the variables defined in a
composefile (the `map[string]interface{}` representation that
`loader.ParseYAML` returns at least) and their default value too.
This commit also does some small function extract on substitution
funcs to reduce a tiny bit duplication.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: afb87e42f2
Component: cli
fix the race condition in the integration test TestRunContainerWithBridgeNone
Upstream-commit: 9149ef67be8ac945d68fafb16a1aa4ccb2f72249
Component: engine
PersistentPreRunE needs to be called within the help function to initialize all the flags (notably the orchestrator flag)
Add an e2e test as regression test
Signed-off-by: Silvin Lubecki <silvin.lubecki@docker.com>
Upstream-commit: 21cce52b30
Component: cli
* Add the `help` target to document make targets when building using a
container
* Remove the `watch` target (filewatcher was removed with c0588a9c) from
docker.Makefile and Makefile
Signed-off-by: Marco Vedovati <mvedovati@suse.com>
Upstream-commit: 760ca04709
Component: cli
This should make it easier for people to write custom composefile
parser without duplicating too much code. It takes the default
transformers and any additional number of transformer for any
types. That way it's possible to transform a `cli/compose` map into a
custom type that would use some of `cli/compose` types and its own.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 0246bc1b3b
Component: cli
Make all dynbinary builds be position-independent (this adds both
security benefits and can help with flaky builds on POWER
architectures).
Signed-off-by: Aleksa Sarai <asarai@suse.de>
Upstream-commit: 164e812b7a
Component: cli
This flag was added in Docker 17.06, API version 1.31 through
moby@8dc8cd4719f165c01c98e7d3ce1d6cea6a8f60b8, but didn't add
API-version annotations.
This patch adds the missing annotations to hide this flag if
the CLI is connected to an older version of the daemon that
doesn't support that API.
Before this patch:
DOCKER_API_VERSION=1.30 docker swarm init --help | grep data-path-addr
--data-path-addr string Address or interface to use for data path traffic (format: <ip|interface>)
DOCKER_API_VERSION=1.31 docker swarm init --help | grep data-path-addr
--data-path-addr string Address or interface to use for data path traffic (format: <ip|interface>)
With this patch applied:
DOCKER_API_VERSION=1.30 docker swarm init --help | grep data-path-addr
# (no result)
DOCKER_API_VERSION=1.31 docker swarm init --help | grep data-path-addr
--data-path-addr string Address or interface to use for data path traffic (format: <ip|interface>)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 91b1ad9d2b
Component: cli
Else it is harder to understand where we've failed in StartWithLogFile
We have an errror:
[d42ce729d0b06] waiting for daemon to start
[d42ce729d0b06] waiting for daemon to start
[d42ce729d0b06] waiting for daemon to start
[d42ce729d0b06] waiting for daemon to start
[d42ce729d0b06] waiting for daemon to start
[d42ce729d0b06] waiting for daemon to start
[d42ce729d0b06] waiting for daemon to start
[d42ce729d0b06] waiting for daemon to start
[d42ce729d0b06] waiting for daemon to start
[d42ce729d0b06] waiting for daemon to start
[d42ce729d0b06] waiting for daemon to start
[d42ce729d0b06] waiting for daemon to start
docker_cli_userns_test.go:27:
s.d.StartWithBusybox(c, "--userns-remap", "default")
/go/src/github.com/docker/docker/internal/test/daemon/daemon.go:203:
t.Fatalf("Error starting daemon with arguments: %v", args)
... Error: Error starting daemon with arguments: [--userns-remap default]
[d42ce729d0b06] exiting daemon
Likely it is "[..] Daemon exited during startup" case, but these error
message is lost
Also we lose the error returned by waiting daemon so also add it
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Upstream-commit: 2ed53943f37c26017260759b12d32cfdd75be152
Component: engine
Makes code less confusing.
Otherwise it looks like an error (typo of "==" instead "!=").
Signed-off-by: Iskander Sharipov <quasilyte@gmail.com>
Upstream-commit: cda9d5f7f0c4a155d1ae9e06c203fcb89600ff2a
Component: engine
WithDialContext() allows specifying custom dialer for hijacking and supposed to
replace WithDialer().
WithDialer() is also updated to use WithDialContext().
client.Dialer() returns the dialer configured with WithDialContext().
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Upstream-commit: edac92409a3b1d0cfb7f5c0e2d10b3bb71f27245
Component: engine
- Use `Contains` instead of `Include`
- Use `ToJSON` instead of `ToParam`
- Remove usage of `ParseFlag` as it is deprecated too
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 55edeb497a
Component: cli
The message was mistakenly replaced with a completely different one.
Fixes: cf2ea7613899477
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 2c2834ba4d59bb1bbb3efee07c8d3f09445c66b7
Component: engine
migrate service update integration tests from integration-cli to integration/service package
Upstream-commit: 3f4410163b8bfbfb67fa2730107235f16c6bbe59
Component: engine
Handle the case of systemd-resolved, and if in place
use a different resolv.conf source.
Set appropriately the option on libnetwork.
Move unix specific code to container_operation_unix
Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
Upstream-commit: e353e7e3f0ce8eceeff657393cba2876375403fa
Component: engine
Even though those fields are not supported by `docker stack deploy`
they are defined in versions `3.x` of compose schema, so the `compose`
package should be able to marshal/unmarshal them.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: d13e2df65b
Component: cli
Previously, dockerd would always ask containerd to pass --leave-running
to runc/runsc, ignoring the exit boolean value. Hence, even `docker
checkpoint create --leave-running=false ...` would not stop the
container.
Signed-off-by: Brielle Broder <bbroder@google.com>
Upstream-commit: db621eb7ee6a837f6879959ac56c67fa2b30ae1d
Component: engine
Disable cri plugin by default in containerd and
allows an option to enable the plugin. This only
has an effect on containerd when supervised by
dockerd. When containerd is managed outside of
dockerd, the configuration is not effected.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: 8fb5f4d5c9b4933be31bf5371d65a95edb037261
Component: engine
1. Sets the LinuxMetadata flag by default on Windows LCOW v1
MappedDirectories.
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
Upstream-commit: e93077bcdf77be2e7e3aec9f9686dace9e18c850
Component: engine
based on golang's pseudo-version: https://groups.google.com/forum/#!topic/golang-dev/a5PqQuBljF4
> using a "pseudo-version" of the form v0.0.0-yyyymmddhhmmss-abcdefabcdef,
> where the time is the commit time in UTC and the final suffix is the prefix
> of the commit hash. The time portion ensures that two pseudo-versions can
> be compared to determine which happened later, the commit hash identifes
> the underlying commit, and the v0.0.0- prefix identifies the pseudo-version
> as a pre-release before version v0.0.0, so that the go command prefers any
> tagged release over any pseudo-version.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: d2baf8f4891e82117bce354e0b0761258a0eafc9
Component: packaging
Go 1.11 includes a fix to os/user to be working in a static binary
(fixing https://github.com/golang/go/issues/23265). The fix requires
`osusergo` build tag to be set for static binaries, which is what
this commit adds (also for containerd).
[v2: sort build tags alphabetically]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 70cdb1c66429582ecfdc5abed67189dd90ab7572
Component: engine
Since go-1.11beta1 archive/tar, tar headers with Typeflag == TypeRegA
(numeric 0) (which is the default unless explicitly initialized) are
modified to have Typeflag set to either tar.TypeReg (character value
'0', not numeric 0) or tar.TypeDir (character value '5') [1].
This results in different Typeflag value in the resulting header,
leading to a different Checksum, and causing the following test
case errors:
> 12:09:14 --- FAIL: TestTarSums (0.05s)
> 12:09:14 tarsum_test.go:393: expecting
> [tarsum+sha256:8bf12d7e67c51ee2e8306cba569398b1b9f419969521a12ffb9d8875e8836738],
> but got
> [tarsum+sha256:75258b2c5dcd9adfe24ce71eeca5fc5019c7e669912f15703ede92b1a60cb11f]
> ... (etc.)
All the other code explicitly sets the Typeflag field, but this test
case is not, causing the incompatibility with Go 1.11. Therefore,
the fix is to set TypeReg explicitly, and change the expected checksums
in test cases).
Alternatively, we can vendor archive/tar again (for the 100th time),
but given that the issue is limited to the particular test case it
does not make sense.
This fixes the test for all Go versions.
[1] https://go-review.googlesource.com/c/go/+/85656
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 05cbe23db9836476677e7071d21ad53bf93dc2e7
Component: engine
Set BuildKit's ExportedProduct variable to show useful errors in the future
Upstream-commit: 9ebed53c5431a7c8296107869f1b3feabee9e8f3
Component: engine
Clarified ambiguous error message
Update kubernetes/cli.go
Infromed user of why the error was caused when file is not there
Signed-off-by: Justyn Temme <justyntemme@gmail.com>
Upstream-commit: bdd58a4096
Component: cli
This test occassionally fails on s390x and Power;
03:16:04 --- FAIL: TestExternalGraphDriver/pull (1.08s)
03:16:04 external_test.go:402: assertion failed: error is not nil: Error: No such image: busybox:latest
Most likely these failures are caused due to Docker Hub updating
the busybox:latest image, but not all architectures yet being
available.
Instead of using `:latest`, pull an image by digest, so that
the test doesn't depend on Docker Hub having all architectures
available for `:latest`.
I selected the same digest as is currently used as "frozen image"
in the Dockerfile.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 352db26d5fd896b681046b1119f07e0ce1ed45b8
Component: engine
This introduces a PRODUCT environment variable that is used to set a constant
at dockerversion.ProductName.
That is then used to set BuildKit's ExportedProduct variable in order to show
useful error messages to users when a certain version of the product doesn't
support a BuildKit feature.
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 195919d9d645aa4ab5680a2331c57ff33eb9e5d9
Component: engine
moved integration tests from docker_cli_config_create_test.go to integration/config
Upstream-commit: a5495f289aafafac8a01c62f2a9cb44856658ece
Component: engine
Updates cri version to 1.0.4, to add `max-container-log-line-size`
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9e773a12fb1cc5da7bec13d46fe04673a4593632
Component: engine
* Regex name filters were display undesired behavior due to
names containing the trailing slash when being compared
* Adjusted filterByNameIDMatches and includeContainerInList to
strip the slash prefix before doing name comparisons
* Added test case and helper functions for the test to list_test
* Force failed tests during development to ensure there were
no false positives
Signed-off-by: Chris White <me@cwprogram.com>
Upstream-commit: 5c8da2e96738addd8a175e5b9a23b8c37d4a4dfe
Component: engine
Instead of waiting for the DNS to fail, try to access
a specific external IP and verify that 100% of the pakcets
are being lost.
Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
Upstream-commit: a2bb2144b3e0d49ac615976bfac462a307d85f0e
Component: engine
Should fix
```
api/types/volume/volume_create.go
Line 10: warning: comment on exported type VolumeCreateBody should be of the form "VolumeCreateBody ..." (with optional leading article) (golint)
api/types/volume/volume_list.go
Line 12: warning: comment on exported type VolumeListOKBody should be of the form "VolumeListOKBody ..." (with optional leading article) (golint)
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: bd06a5ea4df919ff323510fba10801b5673a3af6
Component: engine
17.06 has reached EOL a long time ago; let's use a current
version in CI as well :D
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e2a56c47da
Component: cli
- remove some hints that are no longer needed
- added a nolint: unparam for removeSingleSigner() (return bool is only used in tests)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9847e96765
Component: cli
alexkohler/nakedret is now installed by default with gometalinter,
so it's no longer needed to install this manually
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f3811e865e
Component: cli
When go-1.11beta1 is used for building, the following error is
reported:
> 14:56:20 daemon\graphdriver\lcow\lcow.go:236: Debugf format %s reads
> arg #2, but call has 1 arg
While fixing this, let's also fix a few other things in this
very function (startServiceVMIfNotRunning):
1. Do not use fmt.Printf when not required.
2. Use `title` whenever possible.
3. Don't add `id` to messages as `title` already has it.
4. Remove duplicated colons.
5. Try to unify style of messages.
6. s/startservicevmifnotrunning/startServiceVMIfNotRunning/
...
In general, logging/debugging here is a mess and requires much more
love than I can give it at the moment. Areas for improvement:
1. Add a global var logger = logrus.WithField("storage-driver", "lcow")
and use it everywhere else in the code.
2. Use logger.WithField("id", id) whenever possible (same for "context"
and other similar fields).
3. Revise all the errors returned to be uniform.
4. Make use of errors.Wrap[f] whenever possible.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: b7a95a3ce4c731c0fca204435be758ea89d6050f
Component: engine
There are two build errors when using go-1.11beta1:
> daemon/logger/loggerutils/logfile.go:367: Warningf format %q arg f.Name is a func value, not called
> daemon/logger/loggerutils/logfile.go:564: Debug call has possible formatting directive %v
In the first place, the file name is actually not required as error
message already includes it.
While at it, fix a couple of other places for more correct messages, and
make sure to not add a file name if an error already has it.
Fixes: f69f09f44c
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 09ad434f10cff48741322854a3003686b28295b5
Component: engine
Fix the following go-1.11beta1 build error:
> daemon/graphdriver/aufs/aufs.go:376: Wrapf format %s reads arg #1, but call has 0 args
While at it, change '%s' to %q.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2e30e9e6db42043cb2bd67d25a7152488c834f9f
Component: engine
Go 1.11beta1 (rightfully) complains:
> 15:38:37 daemon/cluster/controllers/plugin/controller.go:183:
> Entry.Debugf format %#T has unrecognized flag #
This debug print was added by commit 72c3bcf2a5.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a9a136572dc2a2c2ec5da320c4d0a32b5a15c550
Component: engine
In particular, these two:
> daemon/daemon_unix.go:1129: Wrapf format %v reads arg #1, but call has 0 args
> daemon/kill.go:111: Warn call has possible formatting directive %s
and a few more.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 3737194b9f2875c10f3f2117c1816054ba0ff262
Component: engine
tilde doesn't play nice with docker images so to be uniform let's
replace the tilde with a dash
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: ad988a9a4020eaf44245659a8fe7053a2e9b874b
Component: packaging
mkimage-alpine.sh: Removed the "-i -t" arguments from the smoke test calling printf (the…
Upstream-commit: c04f9ae9d7af14d0fb49bb84e46c03b871f59a5d
Component: engine
The absence of the file /proc/sys/net/ipv6/conf/all/disable_ipv6
doesn't appear to affect functionality, at least at this time.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: d58c4cbe6c781c78db352db2956e9949b8fc8609
Component: engine
dockerd allows the `--log-level` to be specified, but this log-level
was not forwarded to the containerd process.
This patch sets containerd's log-level to the same as dockerd if a
custom level is provided.
Now that `--log-level` is also passed to containerd, the default "info"
is removed, so that containerd's default (or the level configured in containerd.toml)
is still used if no log-level is set.
Before this change:
containerd would always be started without a log-level set (only the level that's configured in `containerd.toml`);
```
root 1014 2.5 2.1 496484 43468 pts/0 Sl+ 12:23 0:00 dockerd
root 1023 1.2 1.1 681768 23832 ? Ssl 12:23 0:00 \_ docker-containerd --config /var/run/docker/containerd/containerd.toml
```
After this change:
when running `dockerd` without options (same as current);
```
root 1014 2.5 2.1 496484 43468 pts/0 Sl+ 12:23 0:00 dockerd
root 1023 1.2 1.1 681768 23832 ? Ssl 12:23 0:00 \_ docker-containerd --config /var/run/docker/containerd/containerd.toml
```
when running `dockerd --debug`:
```
root 600 0.8 2.1 512876 43180 pts/0 Sl+ 12:20 0:00 dockerd --debug
root 608 0.6 1.1 624428 23672 ? Ssl 12:20 0:00 \_ docker-containerd --config /var/run/docker/containerd/containerd.toml --log-level debug
```
when running `dockerd --log-level=panic`
```
root 747 0.6 2.1 496548 43996 pts/0 Sl+ 12:21 0:00 dockerd --log-level=panic
root 755 0.7 1.1 550696 24100 ? Ssl 12:21 0:00 \_ docker-containerd --config /var/run/docker/containerd/containerd.toml --log-level panic
```
combining `--debug` and `--log-level` (`--debug` takes precedence):
```
root 880 2.7 2.1 634692 43336 pts/0 Sl+ 12:23 0:00 dockerd --debug --log-level=panic
root 888 1.0 1.1 616232 23652 ? Ssl 12:23 0:00 \_ docker-containerd --config /var/run/docker/containerd/containerd.toml --log-level debug
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: aaa1392279ffe67a63581d2a43b3a3c7b0fc47b1
Component: engine
It said `TESTFLAGS='-test.run ^TestValidateIPAddress$' make test-unit`
runs `TestBuild` test, but actually runs `TestValidateIPAddress` test.
Signed-off-by: Donghwa Kim <shanytt@gmail.com>
Upstream-commit: 3048c3a7682919297d8d7d26897a59dc91b3be09
Component: engine
Before this commit Healthcheck run if HEALTHCHECK
instruction appears before RUN instruction.
By passing `withoutHealthcheck` to `copyRunConfig`,
always RUN instruction run without Healthcheck.
Fix: https://github.com/moby/moby/issues/37362
Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
Upstream-commit: 44e08d8a7d1249a1956018c6c3d3655642a4f273
Component: engine
Commit c0bc14e8 wrapped the return value of nw.Delete() with some extra
information. However, this breaks the code in
containerAdaptor.removeNetworks() which ignores certain specific
libnetwork error return codes. Said codes actually don't represent
errors, but just regular conditions to be expected in normal operation.
The removeNetworks() call checked for these errors by type assertions
which the errors.Wrap(err...) breaks.
This has a cascading effect, because controller.Remove() invokes
containerAdaptor.removeNetworks() and if the latter returns an error,
then Remove() fails to remove the container itself. This is not
necessarily catastrophic since the container reaper apparently will
purge the container later, but it is clearly not the behavior we want.
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: 6225d1f15c5fd916c3e0ef3afe022f6cc14ac696
Component: engine
Switches dev versioning from being based on a set version like
`18.06.0-dev` to being based purely on the git commit timestamp / git
commit sha.
This should give us a bit more flexibility in terms of nightlies not
being tied to specific versions.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 559c9a323a77fe1215f91d851ce72053ce5dd99b
Component: packaging
Bump libnetwork to 3ac297bc7fd0afec9051bbb47024c9bc1d75bf5b in order to
get fix 0c3d9f00 which addresses a flaw that the scalable load balancing
code revealed. Attempting to print sandbox IDs where the sandbox name
was too short results in a goroutine panic. This can occur with
sandboxes with names of 1 or 2 characters in the previous code. But due
to naming updates in the scalable load balancing code, it could now
occur for networks whose name was 3 characters and at least one of the
integration tests employed such networks (named 'foo', 'bar' and 'baz').
This update also brings in several changes as well:
* 6c7c6017 - Fix error handling about bridgeSetup
* 5ed38221 - Optimize networkDB queue
* cfa9afdb - ndots: produce error on negative numbers
* 5586e226 - improve error message for invalid ndots number
* 449672e5 - Allows to set generic knobs on the Sandbox
* 6b4c4af7 - do not ignore user-provided "ndots:0" option
* 843a0e42 - Adjust corner case for reconnect logic
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: 0e162d992394d9216f590176991d33ff48ef3389
Component: engine
The deafult OCI linux spec in oci/defaults{_linux}.go in Docker/Moby
from 1.11 to current upstream master does not block /proc/acpi pathnames
allowing attackers to modify host's hardware like enabling/disabling
bluetooth or turning up/down keyboard brightness. SELinux prevents all
of this if enabled.
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
Upstream-commit: 569b9702a59804617e1cd3611fbbe953e4247b3e
Component: engine
This fix migrates some ipcmode tests in integration-cli
to integration tests.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: e0403604e26868b1546a766ab0b40b6cb1677ee6
Component: engine
Matching the version that is used in SwarmKit
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 7084487fdcfaf0fdb76a98757d49033c817c5978
Component: engine
Fixes a possible deadlock on closing the watcher on kqueue
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9f016c0fe1cd30381805fdb225854a36a806e961
Component: engine
this should work ( tried on my machine)
Signed-off-by: Ian Chen <ianre657@gmail.com>
Upstream-commit: a7652107189336f243e5c9a89b33a577df34fdd2
Component: engine
This partially reverts https://github.com/moby/moby/pull/37350
Although specs.Platform is desirable in the API, there is more work
to be done on helper functions, namely containerd's platforms.Parse
that assumes the default platform of the Go runtime.
That prevents a client to use the recommended Parse function to
retrieve a specs.Platform object.
With this change, no parsing is expected from the client.
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: facad557440a0c955beb615495b8d0175f25e4e3
Component: engine
This patch is required for the updated version of libnetwork and entails
two minor changes.
First, it uses the new libnetwork.NetworkDeleteOptionRemoveLB option to
the network.Delete() method to automatically remove the load balancing
endpoint for ingress networks. This allows removal of the
deleteLoadBalancerSandbox() function whose functionality is now within
libnetwork.
The second change is to allocate a load balancer endpoint IP address for
all overlay networks rather than just "ingress" and windows overlay
networks. Swarmkit is already performing this allocation, but moby was
not making use of these IP addresses for Linux overlay networks (except
ingress). The current version of libnetwork makes use of these IP
addresses by creating a load balancing sandbox and endpoint similar to
ingress's for all overlay network and putting all load balancing state
for a given node in that sandbox only. This reduces the amount of linux
kernel state required per node.
In the prior scheme, libnetwork would program each container's network
namespace with every piece of load balancing state for every other
container that shared *any* network with the first container. This
meant that the amount of kernel state on a given node scaled with the
square of the number of services in the cluster and with the square of
the number of containers per service. With the new scheme, kernel state
at each node scales linearly with the number of services and the number
of containers per service. This also reduces the number of system calls
required to add or remove tasks and containers. Previously the number
of system calls required grew linearly with the number of other
tasks that shared a network with the container. Now the number of
system calls grows linearly only with the number of networks that the
task/container is attached to. This results in a significant
performance improvement when adding and removing services to a cluster
that already heavily loaded.
The primary disadvantage to this scheme is that it requires the
allocation of an additional IP address per node per subnet for every
node in the cluster that has a task on the given subnet. However, as
mentioned, swarmkit is already allocating these IP addresses for every
node and they are going unused. Future swarmkit modifications should be
examined to only allocate said IP addresses when nodes actually require
them.
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: 8e0f6bc90351525f3e52f3bc357378fcccccdd27
Component: engine
Bump libnetwork to b0186632522c68f4e1222c4f6d7dbe518882024f. This
includes the following changes:
* Dockerize protocol buffer generation and update (78d9390a..e12dd44c)
* Use new plugin interfaces provided by plugin pkg (be94e134)
* Improve linux load-balancing scalability (5111c24e..366b9110)
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: 92335eaef12fb0fb86e5e7503f88b8873a9fa973
Component: engine
in swarm.
Also, fix some CLI command confusions:
1. If the --external-ca flag is provided, require a --ca-cert flag as well, otherwise
the external CA is set but the CA certificate is actually rotated to an internal
cert
2. If a --ca-cert flag is provided, require a --ca-key or --external-ca flag be
provided as well, otherwise either the server will say that the request is
invalid, or if there was previously an external CA corresponding to the cert, it
will succeed. While that works, it's better to require the user to explicitly
set all the parameters of the new desired root CA.
This also changes the `swarm update` function to set the external CA's CACert field,
which while not strictly necessary, makes the CA list more explicit.
Signed-off-by: Ying Li <ying.li@docker.com>
Upstream-commit: 4243440e1f
Component: cli
By 0296797f0f39477d675128c93c1646b3186937ee, `progressReader`
and `remoteURL` were removed from arguments. So developers who
use `Detect` not need to care about when `ProgressReaderFunc`
is used.
Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
Upstream-commit: 0bbd476ceb8da679f818df529cc917ec807a16af
Component: engine
When using the mounts API, bind mounts are not supposed to be
automatically created.
Before this patch there is a race condition between valiating that a
bind path exists and then actually setting up the bind mount where the
bind path may exist during validation but was removed during mountpooint
setup.
This adds a field to the mountpoint struct to ensure that binds created
over the mounts API are not accidentally created.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 1caeb79963d3c9f770b23be2f12c584adf49538d
Component: engine
parsing an environment file should give an error in case a zero-length
variable name (definition w/o a variable name) is encountered.
previously these lines went through unnoticed not informing the user about
a potential configuration error.
Signed-off-by: Tom Klingenberg <tklingenberg@lastflood.net>
Upstream-commit: b91fd12996
Component: cli
previously docker did import environment variables if they were present
but created them if they were not when it was asked via a --env-file
cli option to import but not create them.
fix is to only import the variable into the environment if it is present.
additionally do not import variable names of zero-length (which are lines
w/ a potential variable definition w/o a variable name).
refs:
- https://github.com/docker/for-linux/issues/284
Signed-off-by: Tom Klingenberg <tklingenberg@lastflood.net>
Upstream-commit: 96c026eb30
Component: cli
test to show current behavior is wrong at parsing an environment file
defining an undefined variable - it must not be defined!
NOTE: this test assume the $HOME variable is always set (see POSIX, this
normally is the case, e.g. the test suite remains stable).
Signed-off-by: Tom Klingenberg <tklingenberg@lastflood.net>
Upstream-commit: 1e89745704
Component: cli
The TestDockerNetworkIPAMMultipleNetworks test allocates several
networks simultaneously with overlapping IP addresses. Libnetwork now
forbids this. Adjust the test case to use distinct IP ranges for the
networks it creates.
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: efb7909befa0fe2236148543a6d50e2563bf386c
Component: engine
'CheckInitialized' in the credential-helper library actually invokes
`pass`, which isn't desirable (see #699).
This moves the check to be simpler, and then pass will only be invoked
when it's needed (such as for `docker login` or when pulling from a
private registry).
This logic could also reasonably live in the credential-helper library,
but it's simple enough it seems fine in either location.
Signed-off-by: Euan Kemp <euank@euank.com>
Upstream-commit: 056015c3d8
Component: cli
Bump libnetwork to 430c00a6a6b3dfdd774f21e1abd4ad6b0216c629. This
includes the following moby-affecting changes:
* Update vendoring for go-sockaddr (8df9f31a)
* Fix inconsistent subnet allocation by preventing allocation of
overlapping subnets (8579c5d2)
* Handle IPv6 literals correctly in port bindings (474fcaf4)
* Update vendoring for miekg/dns (8f307ac8)
* Avoid subnet reallocation until required (9756ff7ed)
* Bump libnetwork build to use go version 1.10.2 (603d2c1a)
* Unwrap error type returned by PluginGetter (aacec8e1)
* Update vendored components to match moby (d768021dd)
* Add retry field to cluster-peers probe (dbbd06a7)
* Fix net driver response loss on createEndpoint (1ab6e506)
(fixes https://github.com/docker/for-linux/issues/348)
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: f155f828a26cfb3ee0dd8ca48bd2c702c9032b0b
Component: engine
When a manifest list contains both Linux and Windows images, always
prefer Windows when the platform OS is unspecified. Also, filter out any
Windows images with a higher build than the host, since they cannot run.
Signed-off-by: John Stephens <johnstep@docker.com>
Upstream-commit: ddcdb7255d960a4b2049439245a4f887b26af891
Component: engine
OCI types are backwards compatible with Docker manifest
types, however the media types must be registered.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: c4f0515837b834ec66ec463d92a32015062a0804
Component: engine
Stores complete OCI descriptor instead of digest and platform
fields. This includes the size which was getting lost by not
storing the original manifest bytes.
Attempt to support existing cached files, if not output
the filename with the incorrect content.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: 1fd2d66df8
Component: cli
Fixes named pipe support for hyper-v isolated containers
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 74095588baa0693a2c07bd05c001c42830f82ed3
Component: engine
It's not longer supported since Debian Jessie is now an LTS release as
of June 17, 2018. LTS releases of Debian only support i386, amd64, armel and armhf
Sources:
* https://wiki.debian.org/LTS
* https://wiki.debian.org/LTS/FAQ
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 24f8b422aaed0fba9d0f03bfd5b99c58f0d7f30d
Component: packaging
This is mainly for the `pass` helper ; `pass` shouldn't be called
every docker command anymore ;).
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: d9741fc96b
Component: cli
This stuff doesn't belong here and is causing imports of libnetwork into
the router, which is not what we want.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: c0bc14e8dd5a31f5edc804e9a1347bb3eb44483e
Component: engine
It makes it easier to get the correct stack from a compose config
struct without requiring the client (and thus talking to k8s API)
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: f2e6ee6899
Component: cli
This is a follow-up of commit 408c7ad (PR: #32453)
Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
Upstream-commit: dbd4c290b7029d31fa69c906a8da864589f19e09
Component: engine
Add the OCI spec compatible image support in client side.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 7f334d3acfd7bfde900e16e393662587b9ff74a1
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
Addresses https://github.com/moby/moby/pull/35089#issuecomment-367802698.
This change enables the daemon to automatically select an image under LCOW
that can be used if the API doesn't specify an explicit platform.
For example:
FROM supertest2014/nyan
ADD Dockerfile /
And docker build . will download the linux image (not a multi-manifest image)
And similarly docker pull ubuntu will match linux/amd64
Upstream-commit: 35193c0e7dc301e1d2f6ea96e0ce34ffd2d4b88d
Component: engine
- Handle `bundlefile` directly in the `top-level`
command. `bundlefile` is still experimental and will be deprecated
in future version — this should make be easier to remove it.
- Validate the `stack` name in all cases (i.e. whatever the
orchestrator is used)
- Load the composefile ahead of choosing the orchestrator. This
removes some slight duplication.
- Makes `RunDeploy` easier to use from outside packages (like
`docker/app`) with a preloaded configuration.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 0f9d24f78d
Component: cli
- Add the possibility to skip interpolation
- Add the possibility to skip schema validation
- Allow customizing the substitution function, to add special cases.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 9fdd14f399
Component: cli
- `make test-e2e` runs the e2e tests twice : once against on
non-experimental daemon (as before), once against an experimental
daemon.
- adds `test-e2e-experimental` and `test-e2e-non-experimental` target
to run tests for the specified cases
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: a522a78231
Component: cli
> Run an init inside the container that forwards signals and reaps
processes
This is supported on `run` and now on Swarm services too, so it's also
possible to have in on a composefile :).
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: cc26da94ed
Component: cli
That field is automaticaly populated with any `x-*` field in the yaml.
And marshalling the compose config struct put them back into place.
This make it possible to get those extra fields without re-inventing
the wheel (i.e. reimplementing 80% of the `cli/compose/*` packages.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 80c26f618e
Component: cli
The `-k` shorthand was alreaady removed from other
commands, so best to be consistent.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f0a8598b8d
Component: cli
The output of this information can be confusing,
so removing until we have a better design for this.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a63252ba71
Component: cli
* Renaming DOCKER_ORCHESTRATOR to DOCKER_STACK_ORCHESTRATOR
* Renaming config file option "orchestrator" to "stackOrchestrator"
* "--orchestrator" flag is no more global but local to stack command and subcommands
* Cleaning all global orchestrator code
* Replicating Hidden flags in help and Supported flags from root command to stack command
Signed-off-by: Silvin Lubecki <silvin.lubecki@docker.com>
Upstream-commit: 71272dd203
Component: cli
The archive changes function is not implemented correctly
to handle opaque directories.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: ac5d363e67e01c769b4d9e20ed76a278434ee4de
Component: engine
This test is testing if any "no space left on device" errors
that occur during `docker pull` will not be masked by other
errors. To test for this, a new loopback-device was created,
and used as `--data-dir` ("/var/lib/docker").
However, `/var/lib/docker` is used for storing various
other things, including a `cache.db` database, used by
BuildKit, which is created during startup of the daemon.
Creation of that file failed (due to `--data-dir` path
being on a mount with limited size), which caused daemon
start to fail before the test was able to run.
This patch changes the size-limited mount to be used for
the storage-driver directory only, so that the test is
not affected by other parts of the code attempting to
write files in it.
To have a predictable path; the daemon used in this test
is configured to use the `vfs` storage-driver.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 754aeb28fdb9dd3aafb7705224e57f28281db264
Component: engine
… wherever it's possible. Should make it even easier to test and use
another implementation.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 88068b9dd7
Component: cli
As we did for `Pull`, it will be helpful to expose the push and save
implementation.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: e43c7920ea
Component: cli
Unfortunately, this is for now the only way to see the output of RUN commands when using buildkit.
It is equivalent to `DOCKER_BUILDKIT=1 docker build . 2>&1 | cat`
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: ed75f6202b
Component: cli
With this patch the following become true even with buildkit enabled:
1. `docker build -q .` only outputs the created image's sha256 ID.
2. `docker build -q .` outputs as if no `-q` was specified, if error occurred
3. `docker build . &> out` outputs JSON (instead of TTY characters)
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 640cbb8d2f
Component: cli
This commit brings a more pedantic change in the following ambiguous case:
cat Dockerfile | docker build -f otherDockerfile -
The legacy builder does not error out and prefers the Dockerfile from stdin
while the buildkit-based one errors out.
Note that this is only in the case where stdin is a Dockerfile (not an archive)
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 82f0e1e5d8
Component: cli
This is a follow up of #35168.
Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
Upstream-commit: 7b3966245de27b753c6acfe0347833811ed9eed8
Component: engine
github.com/gotestyourself/gotestyourself moved to gotest.tools with
version 2.0.0. Moving to that one, bumping it to v2.1.0.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: c147e9e1a45b1c66bc5773814b53bcd27132557c
Component: engine
go1.10.3 (released 2018/06/05) includes fixes to the go command, and the
crypto/tls, crypto/x509, and strings packages. In particular, it adds minimal
support to the go command for the vgo transition. See the Go 1.10.3 milestone
on our issue tracker for details;
https://github.com/golang/go/issues?q=milestone%3AGo1.10.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1c71c957cb
Component: cli
go1.10.3 (released 2018/06/05) includes fixes to the go command, and the
crypto/tls, crypto/x509, and strings packages. In particular, it adds minimal
support to the go command for the vgo transition. See the Go 1.10.3 milestone
on our issue tracker for details;
https://github.com/golang/go/issues?q=milestone%3AGo1.10.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 65fc4f1c0cc81739df97605f1fbf91dbf2e7739f
Component: packaging
Mainly to get inline with `docker/cli` version of that dependency
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: f929f15dd6e316402394143eeb60ad9ba95a81cb
Component: engine
Since go1.8, the stdlib TLS net.Conn implementation implements the
`CloseWrite()` interface.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2ac277a56f50d0ad1831603dccbca5c19ff717e9
Component: engine
github.com/gotestyourself/gotestyourself moved to gotest.tools with
version 2.0.0. Moving to that one, bumping it to v2.1.0.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: dd9478a1f7
Component: cli
Errors were not checked while initializing the docker client in the help command
Signed-off-by: Silvin Lubecki <silvin.lubecki@docker.com>
Upstream-commit: f4b0780e20
Component: cli
Scenario:
Daemon is ungracefully shutdown and leaves plugins running (no
live-restore).
Daemon comes back up.
The next time a container tries to use that plugin it will cause a
daemon panic because the plugin client is not set.
This fixes that by ensuring that the plugin does get shutdown.
Note, I do not think there would be any harm in just re-attaching to the
running plugin instead of shutting it down, however historically we shut
down plugins and containers when live-restore is not enabled.
[kir@: consolidate code to deleteTaskAndContainer, a few minor nits]
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: dbeb4329655e91dbe0e6574405937f03fabf3f2f
Component: engine
… discovered recently that our very own Sebastiaan was not even listed
in the maintainers, so fixing this mistake !
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 9fda91bed4
Component: cli
… discovered recently that our very own Sebastiaan was not even listed
in the maintainers, so fixing this mistake !
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 7f9dd507d3f50cd343480cb0776a3ccca83c5764
Component: engine
It's already supported by `swarmkit`, and act the same as
`HostConfig.Init` on container creation.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: e401b88e59e098745744917c555d549f08353e6d
Component: engine
This last assertion shouldn't be there, those cases should be handled
by the for/switch above.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 04207c2db9cd82e47aa82bc2834fe3820086a697
Component: engine
This adds MaskedPaths and ReadOnlyPaths options to HostConfig for containers so
that a user can override the default values.
When the value sent through the API is nil the default is used.
Otherwise the default is overridden.
Adds integration tests for MaskedPaths and ReadonlyPaths.
Signed-off-by: Jess Frazelle <acidburn@microsoft.com>
Upstream-commit: 3694c1e34e40fa2e255a97b5541645cec9c8d1d5
Component: engine
Updates swarmkit, grpc, and all related vendors
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: 52ed3e0896c203013388a3eb603658f345a9d80a
Component: engine
This PR is trying to refactor the `probeAndCreate` and cleanup
related codes based on the refactoring.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 7f280f6f6514598a31d2523d4b9437ca07ee0526
Component: engine
`btrfs-progs` doesn't actually contain the files we need, since we only
need the headers for btrfs, which `libbtrfs-dev` now provides.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 73678285d0671e1b67ec30d21e99cfe1580159fa
Component: packaging
`make DOCKERD_ARGS=--init binary run` should start the daemon with
`--init` as flags (with any other "automagically" added ones).
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: d2af0d96e3fbdfec340cb707b795b863411b00e1
Component: engine
Warns that `-compress` has no effect when used together with the
expremintal `--stream` flag.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 8b3dc39fce
Component: cli
This dependency now uses SemVer, tagged releases.
Also updates dependencies
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 4acddf21b7c11c2c6a71f0f9ba1d329cfbb99e5f
Component: engine
* Kubernetes native filtering (server side) is an exact match, now filtering on name is made client-side to add prefix-matching
Signed-off-by: Silvin Lubecki <silvin.lubecki@docker.com>
Upstream-commit: b1ff199167
Component: cli
Configs and Secrets were note present, some labels on volumes and
networks too.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 55426d9396
Component: cli
This patch adds annotations to mark the checkpoint commands as Linux only, which
hides them if the daemon is running a non-matching operating-system type;
Before:
docker
Usage: docker COMMAND
A self-sufficient runtime for containers
...
Management Commands:
config Manage Docker configs
container Manage containers
image Manage images
After:
docker
Usage: docker COMMAND
A self-sufficient runtime for containers
...
Management Commands:
checkpoint Manage checkpoints
config Manage Docker configs
container Manage containers
image Manage images
This change also prints errors when attempting to use checkpoint commands or
flags if the feature is not supported by the Daemon's operating system;
$ docker checkpoint --help
docker checkpoint is only supported on a Docker daemon running on linux, but the Docker daemon is running on windows
$ docker checkpoint create --help
docker checkpoint create is only supported on a Docker daemon running on linux, but the Docker daemon is running on windows
$ docker checkpoint ls --help
docker checkpoint ls is only supported on a Docker daemon running on linux, but the Docker daemon is running on windows
$ docker checkpoint rm --help
docker checkpoint rm is only supported on a Docker daemon running on linux, but the Docker daemon is running on windows
$ docker container start --checkpoint=foo mycontainer
"--checkpoint" requires the Docker daemon to run on linux, but the Docker daemon is running on windows
$ docker container start --checkpoint-dir=/foo/bar mycontainer
"--checkpoint-dir" requires the Docker daemon to run on linux, but the Docker daemon is running on windows
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1df6b68111
Component: cli
With a full attach, each attach was leaking 4 goroutines.
This updates attach to use errgroup instead of the hodge-podge of
waitgroups and channels.
In addition, the detach event was never being sent.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 0f5147701775a6c5d4980a7b7c0ed2e830688034
Component: engine
As for top-level key, any 3rd-level key which starts with `x-` will be
ignored by compose. This allows for users to:
* include additional metadata in their compose files
* create YAML anchor objects that can be re-used in other parts of the config
This matches a similar feature in the swagger spec definition:
https://swagger.io/specification/#specificationExtensions
This means a composefile like the following is valid
```
verison: "3.7"
services:
foo:
image: foo/bar
x-foo: bar
network:
bar:
x-bar: baz
```
It concerns services, volumes, networks, configs and secrets.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 6bcfa8806c
Component: cli
Includes some refactoring to allow testing the output
in isolation.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: bb71808605
Component: cli
btrfs-tools is being deprecated in favor of btrfs-progs, will only
matter for debian versions >= buster
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 7c9db958146b66ad59727e8395b928a6e9743642
Component: packaging
This makes it a bit simpler to remove this interface for v2 plugins
and not break external projects (libnetwork and swarmkit).
Note that before we remove the `Client()` interface from `CompatPlugin`
libnetwork and swarmkit must be updated to explicitly check for the v1
client interface as is done int his PR.
This is just a minor tweak that I realized is needed after trying to
implement the needed changes on libnetwork.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 7c77df8acc597cd4f540d873de5fe53a3d414ba9
Component: engine
- Build image that contains everything needed to run e2e tests
- Add ability to run e2e tests against an endpoint
Signed-off-by: Christopher Crone <christopher.crone@docker.com>
Upstream-commit: 6b38918ce4
Component: cli
This fix adds `t.Name()` to tests in integration/network
so that services created in those tests have unique names.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 76195f8047eb9a6be710d53ec493ab97bcf86e20
Component: engine
Add validation for stack names to prevent an empty name resulting in _all_
stacks to be returned after filtering, which can result in removal of services
for all stacks if `--prune`, or `docker stack rm` is used.
Before this change;
docker stack deploy -c docker-compose.yml one
docker stack deploy -c docker-compose.yml two
docker stack deploy -c docker-compose.yml three
docker stack deploy -c docker-compose.yml --prune ''
Removing service one_web
Removing service two_web
Removing service three_web
After this change:
docker stack deploy -c docker-compose.yml one
docker stack deploy -c docker-compose.yml two
docker stack deploy -c docker-compose.yml three
docker stack deploy -c docker-compose.yml --prune ''
invalid stack name: ""
Other stack commands were updated as well:
Before this change;
docker stack deploy -c docker-compose.yml ''
Creating network _default
failed to create network _default: Error response from daemon: rpc error: code = InvalidArgument desc = name must be valid as a DNS name component
docker stack ps ''
nothing found in stack:
docker stack rm ''
Removing service one_web
Removing service three_web
Removing service two_web
After this change:
docker stack deploy -c docker-compose.yml ''
invalid stack name: ""
docker stack ps ''
invalid stack name: ""
docker stack rm ''
invalid stack name: ""
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: d38f397da1
Component: cli
- Don't wrap the code to be slightly more readable
- Rename `getServiceFilter()` to `getStackServiceFilter()` to be
consistent with other helper functions.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0c175fc21a
Component: cli
* Add "kubernetes" struct in config file with "allNamespaces" option, to opt-out this behavior when set as "disabled"
Signed-off-by: Mathieu Champlon <mathieu.champlon@docker.com>
Upstream-commit: 1c1300bef6
Component: cli
Begin to copy the data until the command to exit and any coping to
stdin or copy from stdout/stderr has completed.
Also adding defense code to trim the possible '\x00' null value.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 386e0f36c42593ef434517448ccfac5262f958d6
Component: engine
This cleans up some of the package API's used for interacting with
volumes, and simplifies management.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: e4b6adc88e967971de634596654d9bc33e7bd7e0
Component: engine
`${}`, `${:}` and so on are invalid because there's
no parameter within the brackets; fix detection for
this situation and add/update tests.
There were some existing test-cases that were testing
for the wrong behavior, which are now updated.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 334bf3ea76004d0abe02dd1698989f9eaf87a86a
Component: engine
The builder did not detect syntax errors in substitusions in the
Dockerfile, causing those values to be processed incorrectly instead of
producing an error.
Example 1: missing `}`
docker build --no-cache -<<'EOF'
FROM busybox
ARG var=${aaa:-bbb
RUN echo $var
EOF
Before:
Step 3/3 : RUN echo $var
---> Running in f06571e77146
bbb
After:
Step 2/3 : ARG var=${aaa:-bbb
failed to process "${aaa:-bbb": syntax error: missing '}'
Example 2: missing closing `}`, no default value
docker build --no-cache -<<'EOF'
FROM busybox
ARG var=${aaa
RUN echo $var
EOF
Before:
Step 2/3 : ARG var=${aaa
failed to process "${aaa": missing ':' in substitution
After:
Step 2/3 : ARG var=${aaa
failed to process "${aaa": syntax error: missing '}'
Example 3: double opening bracket (`{`)
docker build --no-cache -<<'EOF'
FROM busybox
ARG var=${{aaa:-bbb}
RUN echo $var
EOF
Before:
Step 2/3 : ARG var=${{aaa:-bbb}
failed to process "${{aaa:-bbb}": missing ':' in substitution
After:
Step 2/3 : ARG var=${{aaa:-bbb}
failed to process "${{aaa:-bbb}": syntax error: bad substitution
Example 4: double opening bracket (`{`), no default value
docker build --no-cache -<<'EOF'
FROM busybox
ARG var=${{aaa}
RUN echo $var
EOF
Before:
Step 2/3 : ARG var=${{aaa}
failed to process "${{aaa}": missing ':' in substitution
After:
Step 2/3 : ARG var=${{aaa}
failed to process "${{aaa}": syntax error: bad substitution
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 955a6ad95f7891a45692d975793abf1eeb07cdd5
Component: engine
If "ps" fails, in many cases it prints a meaningful error message
which a user can benefit from. Let's use it.
While at it, let's use errdefs.System to classify the error,
as well as errors.Wrap.
Before:
> $ docker top $CT <any bad ps options>
> Error response from daemon: Error running ps: exit status 1
After:
> $ docker top $CT auxm
> Error response from daemon: ps: error: thread display conflicts with forest display
or
> $ docker top $CT saur
> Error response from daemon: ps: error: conflicting format options
or, if there's no meaningful error on stderr, same as before:
> $ docker top $CT 1234
> Error response from daemon: ps: exit status 1
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: a41328d5704b8d1adbcd099fb4bb0697060df806
Component: engine
Current ContainerTop (a.k.a. docker top) implementation uses "ps"
to get the info about *all* running processes, then parses it, then
filters the results to only contain PIDs used by the container.
Collecting data only to throw most of it away is inefficient,
especially on a system running many containers (or processes).
For example, "docker top" on a container with a single process
can take up to 0.5 seconds to execute (on a mostly idle system)
which is noticeably slow.
Since the containers PIDs are known beforehand, let's use ps's
"-q" option to provide it with a list of PIDs we want info about.
The problem with this approach is, some ps options can't be used
with "-q" (the only one I'm aware of is "f" ("forest view") but
there might be more). As the list of such options is not known,
in case ps fails, it is executed again without "q" (retaining
the old behavior).
Next, the data produced by "ps" is filtered in the same way as before.
The difference here is, in case "-q" worked, the list is much shorter.
I ran some benchmarks on my laptop, with about 8000 "sleep" processes
running to amplify the savings.
The improvement in "docker top" execution times is 5x to 10x (roughly
0.05s vs 0.5s).
The improvement in ContainerTop() execution time is up to 100x
(roughly 3ms vs 300ms).
I haven't measured the memory or the CPU time savings, guess those
are not that critical.
NOTE that busybox ps does not implement -q so the fallback is always
used, but AFAIK it is not usable anyway and Docker expects a normal
ps to be on the system (say the list of fields produced by
"busybox ps -ef" differs from normal "ps -ef" etc.).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: a076badb8b33f1ecdc5d46f0a3701f10c0579f73
Component: engine
`docker stack services --filter=label=foo=bar --filter=label=foo=baz my-stack` with Swarm gets handled as `filter on (a label named foo with value bar) AND (a label named foo with value baz).
This obviously yields an empty result set every time, but if and how this should be changed is out of scope here, so simply align Kubernetes with Swarm for now.
Signed-off-by: Mathieu Champlon <mathieu.champlon@docker.com>
Upstream-commit: 297866ebbe
Component: cli
This fix was not yet included in Docker 17.05, so
API version v1.37 was not the right selector (Docker
18.03, 18.04 and 18.05 all support API v1.37).
We should change these checks for engine versions,
or use a different method to skip tests when running
against older engines.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a061b1e2d8f6117b0524e44de7b6bc391245864e
Component: engine
Before this change:
----------------------------------------------------
Create a service with reservations and limits for memory and cpu:
docker service create --name test \
--limit-memory=100M --limit-cpu=1 \
--reserve-memory=100M --reserve-cpu=1 \
nginx:alpine
Verify the configuration
docker service inspect --format '{{json .Spec.TaskTemplate.Resources}}' test
{
"Limits": {
"NanoCPUs": 1000000000,
"MemoryBytes": 104857600
},
"Reservations": {
"NanoCPUs": 1000000000,
"MemoryBytes": 104857600
}
}
Update just CPU limit and reservation:
docker service update --limit-cpu=2 --reserve-cpu=2 test
Notice that the memory limit and reservation is not preserved:
docker service inspect --format '{{json .Spec.TaskTemplate.Resources}}' test
{
"Limits": {
"NanoCPUs": 2000000000
},
"Reservations": {
"NanoCPUs": 2000000000
}
}
Update just Memory limit and reservation:
docker service update --limit-memory=200M --reserve-memory=200M test
Notice that the CPU limit and reservation is not preserved:
docker service inspect --format '{{json .Spec.TaskTemplate.Resources}}' test
{
"Limits": {
"MemoryBytes": 209715200
},
"Reservations": {
"MemoryBytes": 209715200
}
}
After this change:
----------------------------------------------------
Create a service with reservations and limits for memory and cpu:
docker service create --name test \
--limit-memory=100M --limit-cpu=1 \
--reserve-memory=100M --reserve-cpu=1 \
nginx:alpine
Verify the configuration
docker service inspect --format '{{json .Spec.TaskTemplate.Resources}}' test
{
"Limits": {
"NanoCPUs": 1000000000,
"MemoryBytes": 104857600
},
"Reservations": {
"NanoCPUs": 1000000000,
"MemoryBytes": 104857600
}
}
Update just CPU limit and reservation:
docker service update --limit-cpu=2 --reserve-cpu=2 test
Confirm that the CPU limits/reservations are updated, but memory limit and reservation are preserved:
docker service inspect --format '{{json .Spec.TaskTemplate.Resources}}' test
{
"Limits": {
"NanoCPUs": 2000000000,
"MemoryBytes": 104857600
},
"Reservations": {
"NanoCPUs": 2000000000,
"MemoryBytes": 104857600
}
}
Update just Memory limit and reservation:
docker service update --limit-memory=200M --reserve-memory=200M test
Confirm that the Mempry limits/reservations are updated, but CPU limit and reservation are preserved:
docker service inspect --format '{{json .Spec.TaskTemplate.Resources}}' test
{
"Limits": {
"NanoCPUs": 2000000000,
"MemoryBytes": 209715200
},
"Reservations": {
"NanoCPUs": 2000000000,
"MemoryBytes": 209715200
}
}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: df43eb931e
Component: cli
remove unnescessary import aliases, brackets, and so on.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f23c00d8701e4bd0f2372a586dacbf66a26f9a51
Component: engine
Fixes an issue where `CopyDir` would truncate files larger than 2^32
bytes.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 616fa9446e49502da501a7d0f92b13191f5058dd
Component: engine
Sometimes docker-master CI fails on rhel4+selinux configuration,
like this:
--- FAIL: TestMount (0.12s)
--- FAIL: TestMount/none-remount,size=128k (0.01s)
mounter_linux_test.go:209: unexpected mount option "seclabel" expected "rw,size=128k"
--- FAIL: TestMount/none-remount,ro,size=128k (0.01s)
mounter_linux_test.go:209: unexpected mount option "seclabel" expected "ro,size=128k"
Earlier, commit 8bebd42df2 (PR #34965) fixed this failure,
but not entirely (i.e. the test is now flaky). It looks like
either selinux detection code is not always working (it won't
work in d-in-d), or the kernel might or might not add 'seclabel'
option).
As the subject of this test case is definitely not selinux,
it can just ignore the option added by it.
While at it, fix error messages:
- add missing commas;
- fix a typo;
- allow for clear distinction between mount
and vfs (per-superblock) options.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: d78e885326213e8ef89919c3cc6d16e712e852a8
Component: engine
The following failure is seen in CI from time to time:
> FAIL: docker_api_containers_test.go:435: DockerSuite.TestContainerAPITop
>
> docker_api_containers_test.go:453:
> c.Assert(top.Processes[0][10], checker.Equals, "/bin/sh -c top")
> ... obtained string = "top"
> ... expected string = "/bin/sh -c top"
The test case expects two processes in the output:
1. /bin/sh -c top
2. top
in the given order.
Now, "ps aux" output is sorted by PID*, and so since the "top" is a child
of "/bin/sh -c top" it has a higher PID and will come second as expected
by the test... unless the PIDs on the system are exhausted and PID rollover
happens, in which case PID of "top" will be lower than that of "/bin/sh".
Fix: sort output by process name.
* - in fact it is not sorted, but is being printed in the same order as
the kernel list PID entries in /proc directory, which appears to be
sorted by PID (see ls -1 -U /proc).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 0823ab70990fa4cbf520726013e75c264e6c5231
Component: engine
It does not make sense to keep looking for PID once
we found it, so let's give it a break.
The side effect of this patch is, if there's more than one column
titled "PID", the last (rightmost) column was used before, and now
the first (leftmost) column is used. Should make no practical
difference whatsoever.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 654a7625fc9f0b7b04da0e0e4d151af04a65cc7f
Component: engine
Adds functionality to parse and return network attachment spec
information. Network attachment tasks are phony tasks created in
swarmkit to deal with unmanaged containers attached to swarmkit. Before
this change, attempting `docker inspect` on the task id of a network
attachment task would result in an empty task object. After this change,
a full task object is returned
Fixes#26548 the correct way.
Signed-off-by: Drew Erny <drew.erny@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5b69ff466e61fa168f24869710f2070c742a5565
Component: engine
This prevents the following test case failures "go test" is run
as non-root in the daemon/ directory:
> --- FAIL: TestContainerInitDNS (0.02s)
> daemon_test.go:209: chown /tmp/docker-container-test-054812199/volumes: operation not permitted
>
> --- FAIL: TestDaemonReloadNetworkDiagnosticPort (0.00s)
> reload_test.go:525: mkdir /var/lib/docker/network/files/: permission denied
> --- FAIL: TestRootMountCleanup (0.00s)
> daemon_linux_test.go:240: assertion failed: error is not nil: operation not permitted
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 16670ed4842b1ee4853ba39b6ebf2b771d28db9a
Component: engine
The pigz binary is used to improve performance when
pulling images, but is not available in the default
CentOS package repositories.
Given that RPM's don't have a "recommends" option,
it's better to remove it as a requirement for now,
instead of failing to install for some users.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: abb1f117d3736fdcbbcc15ecef3e36d1f47c63fa
Component: packaging
Arrays expect a type to be set for items in the array.
This patch adds the "string" type, adds a short description,
and some example values.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8b6b55986644c093564497bfef51733a8d598baf
Component: engine
Use a variable in the Makefile to set the GO_VERSION for the builds.
GO_VERSION is defaulted in all Makefiles as `1.10.2`, but can be set when running make: `make GO_VERSION=1.10.1 deb`.
Signed-off-by: corbin-coleman <corbin.coleman@docker.com>
Upstream-commit: 0fe741746287335a5ba4afaf06be301915324e5a
Component: packaging
`arm64` needs get more time duration for the test to finish.
`pty.Start()` opens a file, so the caller should close it explicitly,
else the file I/O can result in unexpected data synchronization issue.
All those changes will not affect the test itself.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 476d7872efb60b1ef1bc7d9d83952f9dbc8f8798
Component: engine
Changes included:
- libnetwork#2147 Adding logs for ipam state
- libnetwork#2143 Fix race conditions in the overlay network driver
- possibly addresses moby#36743 services do not start: ingress-sbox is already present
- possibly addresses moby#30427 Flaky Test: TestSwarmPublishDuplicatePorts on s390
- possibly addresses moby#36501 Flaky tests: Service "port" tests
- libnetwork#2142 Add wait time into xtables lock warning
- libnetwork#2135 filter xtables lock warnings when firewalld is active
- libnetwork#2140 Switch from x/net/context to context
- libnetwork#2134 Adding a recovery mechanism for a split gossip cluster
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 158ab95c17bed9e849e0d935899b75018cdf61a4
Component: engine
`GetTimestamp()` "assumed" values it could not parse
to be a valid unix timestamp, and would use invalid
values ("hello world") as-is (even testing that
it did so).
This patch validates unix timestamp to be a valid
numeric value, and makes other values invalid.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 48cfe3f0872647bf0f52e6ba65022e1859e808e8
Component: engine
In case aufs driver is not supported because supportsAufs() said so,
it is not possible to get a real reason from the logs.
To fix, log the error returned.
Note we're not using WithError here as the error message itself is the
sole message we want to print (i.e. there's nothing to add to it).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 91f85d1c784f3dc9d892b2af2f51d6b6f3b0be69
Component: engine
Simplify the code by using a single logger instance.
While at it, use WithError in Umount.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: c6e2af54256211b8ac757e9b25caa6fb6c9b3c6e
Component: engine
The `deleteAllImages()` cleanup step in the integration suite was printing a
message for each image that was removed.
These messages were not very informative (given that `removeImage()` prints an
error if image removal failed), and made the test output harder to read
(more difficult to see error-conditions when scanning the output).
This patch removes the messages to make the output slightly less noisy.
Before this patch applied:
--- PASS: TestBuildMultiStageOnBuild (7.15s)
main_test.go:32: Removing image sha256:9db3ddbaaadd52804d8a417081f68db41fd4b8f80c85c1b4c4aee2d9b584c074
main_test.go:32: Removing image sha256:7eeb04d90b5e62a99ac6a5b2c10b9ba54b89b176fe2783e41461581c482852b3
main_test.go:32: Removing image sha256:adf42475eefff99b4a611c1a5d8353c4d0a011a7f7b9dc59a75d951cd54fa77f
main_test.go:32: Removing image sha256:c547a770806e0445f5dfc255683ced771a23be6157ba8d0617bb9ab55dcee6d1
main_test.go:32: Removing image sha256:b79659c3e6d34faf2a075f1df1ea2c805833982f112666b25a466177b5d1352d
main_test.go:32: Removing image sha256:29430078cdc927c19c87416cd1fb1ec386f167c5e201ee5dfb1644dcf268a3c2
=== RUN TestBuildUncleanTarFilenames
--- PASS: TestBuildUncleanTarFilenames (2.47s)
main_test.go:32: Removing image sha256:0a6418fb221dc2f25085a1a7e507e01c4a3938cc5c65f1cb85a8c0fb09d6814f
main_test.go:32: Removing image sha256:3d6e4bd0cce01ce5823b40dcb717cd16b3b4b769ff73dd86fa448aac49aa6d7c
main_test.go:32: Removing image sha256:98e3f335e874612668335b3a5f125a1e5cbd0f6c79a7c3f719529b69d0abf2a5
main_test.go:32: Removing image sha256:08919f344b382fd1447da7f3e8ffd2a7125f5f7d191ed7d33242736dbe3c59cf
main_test.go:32: Removing image sha256:d144b3c13838e841ec319a17e1046471d726bb2aa3211e167a6a53f766a2dcdc
main_test.go:32: Removing image sha256:7c768ec742d628020f50c99dc5af32400b78534ca9fc4c01a9f00ec0ab19193a
main_test.go:32: Removing image sha256:4c26c71d142045fbf3448aa1f6363d5a7a803cb438a78a4b20b7c847df03d50a
main_test.go:32: Removing image sha256:08dcd63c964f2dbb17ff2665b6b86993fb14c0d3e169da187ac48f078a560d25
main_test.go:32: Removing image sha256:c1743fab233f36f2d7f83cb13f8c10ff06bdbda8f8a218d25a3796d1bc2f9e84
=== RUN TestBuildMultiStageLayerLeak
--- PASS: TestBuildMultiStageLayerLeak (5.59s)
main_test.go:32: Removing image sha256:5e9974558276c34d7c9aab3fad408d433047b7b15bdae43ab5048adc58a15431
main_test.go:32: Removing image sha256:ac3c613c8c48794237c1e46ac0657ebbb1132910a240086bb2e9df9770fdc017
main_test.go:32: Removing image sha256:a936df268131ad427f7b4b66ce3dbb1e41866d7269a4d383cebcb1c5930d3346
main_test.go:32: Removing image sha256:5e613ea0ce7dbc908b0315c49585ae43ad6c34158e9e0b59a3dc93b00ef0ea41
main_test.go:32: Removing image sha256:e676f4ec41a42823b6d91e05e3290d3827f9175dea6fba5d8b769aa13aa7e082
main_test.go:32: Removing image sha256:93c8daab2703126b23d957d4d6b04f07949356f1cd95d4f8fdbededf4ab5c21e
With this patch applied:
=== RUN TestBuildMultiStageOnBuild
--- PASS: TestBuildMultiStageOnBuild (6.74s)
=== RUN TestBuildUncleanTarFilenames
--- PASS: TestBuildUncleanTarFilenames (2.49s)
=== RUN TestBuildMultiStageLayerLeak
--- PASS: TestBuildMultiStageLayerLeak (5.14s)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5afce21e2efe572be3a256b748bf269afdd645a5
Component: engine
dockerd has no subcommands, so the help command
is not used.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 298a26e0b2d086a32f14ee675bf24322cc92ddd4
Component: engine
> HasAvailableFlags checks if the command contains any flags (local
> plus persistent from the entire structure) which are not hidden or
> deprecated.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 9411856536ef49883fb3550fd1ee804ab99c4072
Component: engine
Use a tagged release of Cobra. All relevant PR's were merged, so the fork is
no longer needed.
Relevant changes:
- spf13/cobra#552 Add a field to disable [flags] in UseLine()
- spf13/cobra#567 Add `CalledAs` method to cobra.Command
- spf13/cobra#580 Update error message for missing required flags
- spf13/cobra#584 Add support for --version flag
- spf13/cobra#614 If user has a project in symlink, just use its destination folder and work there
- spf13/cobra#649 terminates the flags when -- is found in commandline
- spf13/cobra#662 Add support for ignoring parse errors
- spf13/cobra#686 doc: hide hidden parent flags
Also various improvements were added for generating Bash
completion scripts (currently not used by us)
Fixes usage output for dockerd;
Before this update:
dockerd --help
Usage: dockerd COMMAND
A self-sufficient runtime for containers.
After this update:
dockerd --help
Usage: dockerd [OPTIONS] [flags]
A self-sufficient runtime for containers.
Bump spf13/pflag to v1.0.1
Relevant changes:
- spf13/pflag#106 allow lookup by shorthand
- spf13/pflag#113 Add SortFlags option
- spf13/pflag#138 Generate flag error output for errors returned from the parseFunc
- spf13/pflag#141 Fixing Count flag usage string
- spf13/pflag#143 add int16 flag
- spf13/pflag#122 DurationSlice: implementation and tests
- spf13/pflag#115 Implement BytesHex type of argument
- spf13/pflag#150 Add uintSlice and boolSlice to name prettifier
- spf13/pflag#155 Add multiline wrapping support
- spf13/pflag#158 doc: clarify difference between string slice vs. array
- spf13/pflag#160 add ability to ignore unknown flags
- spf13/pflag#163 Allow Users To Show Deprecated Flags
Hide [flags] in usage output
Hides the [flags] in the usage output of commands (present in newer
versions of Cobra), using the `.DisableFlagsInUseLine` option.
Before this change:
dockerd --help
Usage: dockerd [OPTIONS] [flags]
A self-sufficient runtime for containers.
After this change:
dockerd --help
Usage: dockerd [OPTIONS]
A self-sufficient runtime for containers.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Â# modified: vendor/github.com/spf13/pflag/string_array.go
§
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ed75c7727bf3e454a5faa6baf686de12152d911c
Component: engine
This patch hides the [flags] in the usage output of commands, using the
new `.DisableFlagsInUseLine` option, instead of the temporary workaround
added in 8e600e10f7
Before this change:
docker run
"docker run" requires at least 1 argument.
See 'docker run --help'.
Usage: docker run [OPTIONS] IMAGE [COMMAND] [ARG...] [flags]
Run a command in a new container
After this change:
docker run
"docker run" requires at least 1 argument.
See 'docker run --help'.
Usage: docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
Run a command in a new container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 00d080269a
Component: cli
Use a tagged release of Cobra
Relevant changes:
- spf13/cobra#567 Add `CalledAs` method to cobra.Command
- spf13/cobra#580 Update error message for missing required flags
- spf13/cobra#584 Add support for --version flag
- spf13/cobra#614 If user has a project in symlink, just use its destination folder and work there
- spf13/cobra#649 terminates the flags when -- is found in commandline
- spf13/cobra#662 Add support for ignoring parse errors
- spf13/cobra#686 doc: hide hidden parent flags
Also various improvements were added for generating Bash
completion scripts (currently not used by us)
Bump spf13/pflag to v1.0.1
Relevant changes:
- spf13/pflag#122 DurationSlice: implementation and tests
- spf13/pflag#115 Implement BytesHex type of argument
- spf13/pflag#150 Add uintSlice and boolSlice to name prettifier
- spf13/pflag#155 Add multiline wrapping support
- spf13/pflag#158 doc: clarify difference between string slice vs. array
- spf13/pflag#160 add ability to ignore unknown flags
- spf13/pflag#163 Allow Users To Show Deprecated Flags
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0acb2522bd
Component: cli
This hasn't been the way to release Docker for the past year so let's
just remove them altogether
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 80bc8b4766cc808186af91930152d78afb22aef8
Component: engine
The `docker version` output now uses a tabwriter, so use single
tabs to print the output.
Before this change:
Server:
Engine:
Version: 18.05.0-ce
API version: 1.37 (minimum version 1.12)
Go version: go1.10.1
Git commit: f150324
Built: Wed May 9 22:20:16 2018
OS/Arch: linux/amd64
Experimental: true
Kubernetes:
Version: v1.9.6
StackAPI: v1beta2
After this change:
Server:
Engine:
Version: 18.05.0-ce
API version: 1.37 (minimum version 1.12)
Go version: go1.10.1
Git commit: f150324
Built: Wed May 9 22:20:16 2018
OS/Arch: linux/amd64
Experimental: true
Kubernetes:
Version: v1.9.6
StackAPI: v1beta2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 26d8122594
Component: cli
ext4 support d_type by default, but filetype feature is a tunable so
there is still a chance to disable it for some reasons. In this case,
print additional message to explicitly tell how to support d_type.
Signed-off-by: Chengguang Xu <cgxu519@gmx.com>
Upstream-commit: 8a21b128d4deb874c05eb81ebbc1265175ad69ba
Component: engine
Debian Wheezy will be EOL on 31-05-2018 per the official documentation,
https://wiki.debian.org/LTS
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 0e45aed95d86bc516b9cd05f3316073e8dda0a0b
Component: packaging
Since we use `NewLabelCommand()` instead of `addNodesForLabelOption()`
to create the 'LABEL' commands from '--label' options, so all the related
functions should be removed.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: c7b543164daed58fbea36471592438b4e53ab748
Component: engine
Add a new test case `TestBuildLabelWithTargets` to cover the Docker builder
with both `--label` and `--target` options.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: f7add4262b69294aa8035c4794236193ce4bc68e
Component: engine
This PR is tring to fix issue #36996.
Currently for multi-stage build, if `--target` specified, the `--label` option
will be ignored. The root cause is the last stage build will remove the `LABEL`
command(s) node created from the `--label` option. In order to address this issue,
we can create `LABEL` command(s) and add it/tem to the last stage.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 9c238ebd55e4105ad7f7edc04231ea61bb278ae8
Component: engine
Version 1.1
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
Signed-off-by: nkwangleiGIT <nkwanglei@126.com>
Upstream-commit: b6def77c55
Component: cli
This issue has been reported by issue #36877.
The purpose of this test case is for the regression test of #12546,
so we only need to make sure the essential of the testing is still
in the way to check that while not disturbed by some testing noises,
which is exactly what this PR want to do.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 96abf9f59ebd350cc9a70034b4d30279e6ae04e4
Component: engine
All other docker stack commands report an error when passed this value.
Signed-off-by: Mathieu Champlon <mathieu.champlon@docker.com>
Upstream-commit: 4d947de292
Component: cli
The hardcoded values make it so that we have to update this everytime we
add/subtract something which is sometimes followed/not followed.
This makes it so that we no longer have to update this.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 04ec3b020edc79830c5a0100790f5bb6686d8efc
Component: packaging
Fedora 26 will be EOL one month after the release of Fedora 28
(2018-05-01)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: c638836c18d26c5f7e4602784d6b1c7d13f39904
Component: packaging
In cases where a logging plugin has crashed when the daemon tries to
copy the container stdio to the logging plugin it returns a broken pipe
error and any log entries that occurr while the plugin is down are lost.
Fix this by opening read+write in the daemon so logs are not lost while
the plugin is down.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: e7479e3ab8128f9e84cc640f0bed4e77b268a6e9
Component: engine
- Check errors.Cause(err) when comparing errors
- Fix bug where oldest log file is not actually removed. This in
particular causes issues when compression is enabled. On rotate it just
overwrites the data in the log file corrupting it.
- Use O_TRUNC to open new gzip files to ensure we don't corrupt log
files as happens without the above fix.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: e87e9e6ad6ba501cc42a2ef47ac18c88a68f258f
Component: engine
Closing the log driver was in a defer meanwhile logs are
collected asyncronously, so the log driver was being closed before reads
were actually finished.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 2c252a48c252749d41079cf8c450d00a5c18296e
Component: engine
This was broken by bf6a790f00ab96bb8857e3c73502909ee5e36217
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: b16b125bb4326e2eec9948fd54ca8c5d83eba36a
Component: engine
In TestDockerNetworkConnectAlias the network and container
names used are unique which are not preferred. This
fix address the issue by appending t.Name() so that
names are randomized.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: e7b5f4500dc77ec55e5ef202065fbc92218ce394
Component: engine
This message was missing the name of the plugin, resulting in
plugin_test.go:92: timeout hit after 30s: plugin %!q(MISSING) exists
On failing tests.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b526964584d213074de7c5573a3373a58699c181
Component: engine
Embedded new lines aren't interpreted correctly in markdown renderers (they are treated as preformatted text instead). I removed the embedded newlines in the docker image spec.
Signed-off-by: Alex Goodman <wagoodman@gmail.com>
Upstream-commit: 4fb59c20a4fb54f944fe170d0ff1d00eb4a24d6f
Component: engine
In version-history.md, the link for `Docker Engine API v1.37`
was pointed to `v1.36`.
This fix fixes the incorrect link.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 6d66743ae3da148adbd29fb7e0c0433e93fec12b
Component: engine
This fix re-orders the CHANGELOG so that `make validate` test
could pass. The validation requires that the date in the changelog
is always ordered.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 333693c37c8a2c14458d2a088829b9a2a3570eed
Component: engine
Switch from x/net/context to context made "go vet" see the previously
unseen errors:
> cli/command/container/start.go:57::error: the cancelFun function is
> not used on all paths (possible context leak) (vet)
> cli/command/container/start.go:63::error: this return statement may be
> reached without using the cancelFun var defined on line 57 (vet)
> cli/command/container/run.go:159::error: the cancelFun function is not
> used on all paths (possible context leak) (vet)
> cli/command/container/run.go:164::error: this return statement may be
> reached without using the cancelFun var defined on line 159 (vet)
Do call the cancel function.
Note we might end up calling it twice which is fine as long as I can see
from the Go 1.10 source code.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 395957155f
Component: cli
Since go 1.7, "context" is a standard package. Since go 1.9,
x/net/context merely provides some types aliased to those in
the standard context package.
The changes were performed by the following script:
for f in $(git ls-files \*.go | grep -v ^vendor/); do
sed -i 's|golang.org/x/net/context|context|' $f
goimports -w $f
for i in 1 2; do
awk '/^$/ {e=1; next;}
/\t"context"$/ {e=0;}
{if (e) {print ""; e=0}; print;}' < $f > $f.new && \
mv $f.new $f
goimports -w $f
done
done
[v2: do awk/goimports fixup twice]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 6f8070deb2
Component: cli
Removing a host by `<host>:<ip>` should only remove occurences of the host with
a matching IP-address, instead of removing all entries for that host.
In addition, combining `--host-rm` and `--host-add` for the same host should
result in the new host being added.
This patch fixes the way the diff is calculated to allow combining
removing/adding, and to support entries having both a canonical, and aliases.
Aliases cannot be added by the CLI, but are supported in the Service spec, thus
should be taken into account:
Entries can be removed by either a specific `<host-name>:<ip-address>`
mapping, or by `<host>` alone:
- If both IP-address and host-name is provided, only remove the hostname
from entries that match the given IP-address.
- If only a host-name is provided, remove the hostname from any entry it
is part of (either as _canonical_ host-name, or as _alias_).
- If, after removing the host-name from an entry, no host-names remain in
the entry, the entry itself should be removed.
For example, the list of host-entries before processing could look like this:
hosts = &[]string{
"127.0.0.2 host3 host1 host2 host4",
"127.0.0.1 host1 host4",
"127.0.0.3 host1",
"127.0.0.1 host1",
}
Removing `host1` removes every occurrence:
hosts = &[]string{
"127.0.0.2 host3 host2 host4",
"127.0.0.1 host4",
}
Whereas removing `host1:127.0.0.1` only remove the host if the IP-address matches:
hosts = &[]string{
"127.0.0.2 host3 host1 host2 host4",
"127.0.0.1 host4",
"127.0.0.3 host1",
}
Before this patch:
$ docker service create --name my-service --host foo:127.0.0.1 --host foo:127.0.0.2 --host foo:127.0.0.3 nginx:alpine
$ docker service update --host-rm foo:127.0.0.1 --host-add foo:127.0.0.4 my-service
$ docker service inspect --format '{{.Spec.TaskTemplate.ContainerSpec.Hosts}}' my-service
[]
After this patch is applied:
$ docker service create --name my-service --host foo:127.0.0.1 --host foo:127.0.0.2 --host foo:127.0.0.3 nginx:alpine
$ docker service update --host-rm foo:127.0.0.1 --host-add foo:127.0.0.5 my-service
$ docker service inspect --format '{{.Spec.TaskTemplate.ContainerSpec.Hosts}}' my-service
[127.0.0.2 foo 127.0.0.3 foo 127.0.0.4 foo]
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 27c0858f43
Component: cli
This function was added in 23e5c94cfb26eb72c097892712d3dbaa93ee9bc0 but never used
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8a8ec00c1fece033b20548b7a5ec2f3a1f340834
Component: engine
The overlay storage driver currently does not support any option, but was silently
ignoring any option that was passed.
This patch verifies that no options are passed, and if they are passed will produce
an error.
Before this change:
dockerd --storage-driver=overlay --storage-opt dm.thinp_percent=95
INFO[2018-05-11T11:40:40.996597152Z] libcontainerd: started new docker-containerd process pid=256
....
INFO[2018-05-11T11:40:41.135392535Z] Daemon has completed initialization
INFO[2018-05-11T11:40:41.141035093Z] API listen on /var/run/docker.sock
After this change:
dockerd --storage-driver=overlay --storage-opt dm.thinp_percent=95
INFO[2018-05-11T11:39:21.632610319Z] libcontainerd: started new docker-containerd process pid=233
....
Error starting daemon: error initializing graphdriver: overlay: unknown option dm.thinp_percent
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 30f15d2bdc551870464d1cd024a92341cf1ae4aa
Component: engine
The "update" and "rollback" configurations were cross-wired, as a result, setting
`--rollback-*` options would override the service's update-options.
Creating a service with both update, and rollback configuration:
docker service create \
--name=test \
--update-failure-action=pause \
--update-max-failure-ratio=0.6 \
--update-monitor=3s \
--update-order=stop-first \
--update-parallelism=3 \
--rollback-failure-action=continue \
--rollback-max-failure-ratio=0.5 \
--rollback-monitor=4s \
--rollback-order=start-first \
--rollback-parallelism=2 \
--tty \
busybox
Before this change:
docker service inspect --format '{{json .Spec.UpdateConfig}}' test \
&& docker service inspect --format '{{json .Spec.RollbackConfig}}' test
Produces:
{"Parallelism":3,"FailureAction":"pause","Monitor":3000000000,"MaxFailureRatio":0.6,"Order":"stop-first"}
{"Parallelism":3,"FailureAction":"pause","Monitor":3000000000,"MaxFailureRatio":0.6,"Order":"stop-first"}
After this change:
{"Parallelism":3,"FailureAction":"pause","Monitor":3000000000,"MaxFailureRatio":0.6,"Order":"stop-first"}
{"Parallelism":2,"FailureAction":"continue","Monitor":4000000000,"MaxFailureRatio":0.5,"Order":"start-first"}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f367aa9330
Component: cli
Really these mounts shouldn't be in the container pacakge.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: d6558ad6a4552fc745e893f93ae190710d1a36bc
Component: engine
These network operations really don't have anything to do with the
container but rather are setting up the networking.
Ideally these wouldn't get shoved into the daemon package, but doing
something else (e.g. extract a network service into a new package) but
there's a lot more work to do in that regard.
In reality, this probably simplifies some of that work as it moves all
the network operations to the same place.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: cc8f358c23d307d19b06cd5e7d039d8fad01a947
Component: engine
A recent optimization in getSourceMount() made it return an error
in case when the found mount point is "/". This prevented bind-mounted
volumes from working in such cases.
A (rather trivial but adeqate) unit test case is added.
Fixes: 871c957242 ("getSourceMount(): simplify")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: d8fd6137a1f6d95a2bcdfeb6e1dfa6b816790c5e
Component: engine
The Partial property of the Logger message
was replaced by PLogMetaData, causing the build to fail.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: daaef83cd24f6eb335b77e9d8d692235eff1b201
Component: engine
Experimental features are no longer a build-time option,
and can be enabled at runtime.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 050b421f4c435c90dc63bb094789b0513689e80c
Component: packaging
Append a warning with which filter the elements will be pruned with if filter is set.
Signed-off-by: Justyn Temme <justyntemme@gmail.com>
Upstream-commit: 7d113de085
Component: cli
This ensures Windows paths are handled correctly as explained in the path package documentation.
Signed-off-by: Mathieu Champlon <mathieu.champlon@docker.com>
Upstream-commit: 698ea75c66
Component: cli
This is an old script using tools that are no longer maintained or
recommended (and don't even work anymore).
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: ddb01ee1e05f1b460267eb40070e2a62a1c33692
Component: engine
These are no longer used and instead users should use the
`container-selinux` package on their distribution. Additionally, these
are unmaintained and untested.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: eeea1e37a116c0e3765e2591a2ced936e48b5abb
Component: engine
This is left-over stuff from building Docker pacakges. These aren't
really maintained outside of bumping the golang version, and are never
tested.
These builders can be found at
https://github.com/docker/docker-ce-packaging where they are kept up to
date.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 1b7fe816e86f21a968410fca7a0cb9302c386f84
Component: engine
It does not make sense to copy a slice element by element, then discard
the source one. Let's do copy in place instead which is way more
efficient.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: d4c94e83ca58b5ffd99840a8e3cbf22f1ab331c0
Component: engine
When this test fails, the error looks like this:
> FAIL: docker_api_attach_test.go:98: DockerSuite.TestPostContainersAttach
> docker_api_attach_test.go:211:
> c.Assert(actualStdout.Bytes(), checker.DeepEquals, []byte("hello\nsuccess"), check.Commentf("Attach didn't return the expected data from stdout"))
> ... obtained []uint8 = []byte{0x73, 0x75, 0x63, 0x63, 0x65, 0x73, 0x73}
> ... expected []uint8 = []byte{0x68, 0x65, 0x6c, 0x6c, 0x6f, 0xa, 0x73, 0x75, 0x63, 0x63, 0x65, 0x73, 0x73}
> ... Attach didn't return the expected data from stdout
Let's use strings for comparisons to make the output more readable.
While at it,
- get the container's stderr as well, and make sure it's empty;
- check that stdcopy.StdCopy() did not return an error, except for
the timeout which is expected;
- move/remove comments, simplify var names.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: ecc54889c9a233d558b578dfd48eb5d84506a62a
Component: engine
1. As daemon.ContainerStop() documentation says,
> If a negative number of seconds is given, ContainerStop
> will wait for a graceful termination.
but since commit cfdf84d5d04c8ee (PR #32237) this is no longer the case.
This happens because `context.WithTimeout(ctx, timeout)` is implemented
as `WithDeadline(ctx, time.Now().Add(timeout))`, resulting in a deadline
which is in the past.
To fix, don't use WithDeadline() if the timeout is negative.
2. Add a test case to validate the correct behavior and
as a means to prevent a similar regression in the future.
3. Fix/improve daemon.ContainerStop() and client.ContainerStop()
description for clarity and completeness.
4. Fix/improve DefaultStopTimeout description.
Fixes: cfdf84d5d04c ("Update Container Wait")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 69b4fe406540c7989bd99b2244ed67ced96e17c7
Component: engine
Add testing code to cover the `--target` name case sensitive
issue reported by issue #36956.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: a95fabc70efacc1ff6c0c62c490cf551215bc503
Component: engine
This PR is trying to fix issue #36956.
The stage name is case-insensitive by design, so we should use
`strings.EqualFold()` as the comparison method to eliminate the
case sensitive noise.
Also we need to return a pre-defined error code order to avoid below
message like:
"FIXME: Got an API for which error does not match any expected type!!!:
failed to reach build target dev in Dockerfile"
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 7c0570473cfa181aeb3278072cc9af4f9298cb98
Component: engine
vendored distribution is quite old, and current distribution contains an
API break, which means it's not possible to vendor a bugfixed
distribution and a docker/docker at the same time.
Signed-off-by: Mike Lundy <mike@fluffypenguin.org>
Upstream-commit: 07ffa1cea9d995a7ab55793b5b08758c38fec3f0
Component: engine
In pkg/term/proxy.go and pkg/term/proxy_test.go, check if escapeKeys is empty and if it is, return the one key read
Signed-off-by: Patrik Cyvoct <patrik@ptrk.io>
Upstream-commit: d339130f307634a0edd6aef29b6934c3934e9ba2
Component: engine
It will be helpful to expose the pull implementation which supports
pulling private images for other CLI commands that rely on helper images.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 812f113685
Component: cli
When then non-blocking mode is specified, awslogs will:
- No longer potentially block calls to logstream.Log(), instead will
return an error if the awslogs buffer is full. This has the effect of
dropping log messages sent to awslogs.Log() that are made while the
buffer is full.
- Wait to initialize the log stream until the first Log() call instead of in
New(). This has the effect of allowing the container to start in
the case where Cloudwatch Logs is unreachable.
Both of these changes require the --log-opt mode=non-blocking to be
explicitly set and do not modify the default behavior.
Signed-off-by: Cody Roseborough <crrosebo@amazon.com>
Upstream-commit: c7e379988c9cd6ec0af528e6f59eea3c51b36738
Component: engine
We really need to run those on the CI too at some point.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: ef2c2040c22eba8ab6ea1033ad64ba9b0095db9b
Component: engine
This bumps the version of tini used to fec3683b971d9c3ef73f284f176672c44b448662 (v0.18.0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b711dd94fdad0ef8e2c9b2ebbe6d903d270752de
Component: engine
- Volume store created dir with wrong permissions
- Local volume driver hardcoded uid/gid 0
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: d15734ec3c10eda667b716f67e18d5d86e708e3e
Component: engine
Since Go 1.7, context is a standard package. Since Go 1.9, everything
that is provided by "x/net/context" is a couple of type aliases to
types in "context".
Many vendored packages still use x/net/context, so vendor entry remains
for now.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 7d62e40f7e4f3c17d229a7687d6fcca5448de813
Component: engine
Fix a version mismatch in the API documentation.
Signed-off-by: Wassim DHIF <wassimdhif@gmail.com>
Upstream-commit: 2058928edc284653b1fb8e719d713ac7edecd091
Component: engine
- Some of them don't make sense anymore
- Some are deprecated and removed from the engine since a few versions
already.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 740d260cd2
Component: cli
Use more gotestyourself for `env.Patch`, and `icmd.RunCommand`
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: ae03dd7f46
Component: cli
It should check `os.Geteuid` with `uid` instead of `os.Getegid`.
On the container (where the tests run), the uid and gid seems to be
the same, thus this doesn't fail.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 5d8b88b114da7c40f04a95909a35287ae1dac37f
Component: engine
This fix consists of some improvement in restart_test.go
by replacing Fatal with assert, so that they are consistent
with other tests in integration/container.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 67535921b32f68863f7a882d2300b59e0bbeffa4
Component: engine
govet complains (when using standard "context" package):
> the cancel function returned by context.WithTimeout should be called,
> not discarded, to avoid a context leak (vet)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 05e2f7e2fafd0fbc818c9f4cda7ac513c785d49c
Component: engine
We are using interface in the api routers to not explicitely depend on
the daemon struct (`daemon.Daemon`), but somehow, we do depend on the
`daemon` package for the cluster functionalities.
This removes this dependency by defining the correct interfaces.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: f4106b46db47524b4f38abeee48137d42e3fe4eb
Component: engine
That way, those lines won't be reported in the failure.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: cb8db44395df70fa3044d2a9683d6d24438cfa74
Component: engine
Updates swarmkit to 33d06bf5189881b4d1e371b5571f4d3acf832816, to bring in
docker/swarmkit#2610 (Don't use wrappers for grpc metadata)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b18f7033b2644c0246345bb5747a5436568b1a71
Component: engine
This is not for the sake of test to run faster of course;
this is to simplify the code as well as have some more
testing for mount.SingleEntryFilter().
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: ce468f0ad0d075c5d0c44c78bd61c489e6d7d70c
Component: engine
There is no need to parse mount table and iterate through the list of
mounts, and then call Unmount() which again parses the mount table and
iterates through the list of mounts.
It is totally OK to call Unmount() unconditionally.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: ac39a95ea618601f78662972c35838d928858904
Component: engine
Now, every Unmount() call takes a burden to parse the whole nine yards
of /proc/self/mountinfo to figure out whether the given mount point is
mounted or not (and returns an error in case parsing fails somehow).
Instead, let's just call umount() and ignore EINVAL, which results
in the same behavior, but much better performance.
Note that EINVAL is returned from umount(2) not only in the case when
`target` is not mounted, but also for invalid flags. As the flags are
hardcoded here, it can't be the case.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: a1d095199ddb9b4811e1417b6adcdfadad7d73f4
Component: engine
The mountinfo parser implemented via `fmt.Sscanf()` is slower than the one
using `strings.Split()` and `strconv.Atoi()`. This rewrite helps to speed it
up to a factor of 8x, here is a result from go bench:
> BenchmarkParsingScanf-4 300 22294112 ns/op
> BenchmarkParsingSplit-4 3000 2780703 ns/op
I tried other approaches, such as using `fmt.Sscanf()` for the first
three (integer) fields and `strings.Split()` for the rest, but it slows
things down considerably:
> BenchmarkParsingMixed-4 1000 8827058 ns/op
Note the old code uses `fmt.Sscanf`, when a linear search for '-' field,
when a split for the last 3 fields. The new code relies on a single
split.
I have also added more comments to aid in future development.
Finally, the test data is fixed to now have white space before the first field.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: c611f18a7f16d8aa878a5a5c7537d23a0937c40a
Component: engine
The flow of getSourceMount was:
1 get all entries from /proc/self/mountinfo
2 do a linear search for the `source` directory
3 if found, return its data
4 get the parent directory of `source`, goto 2
The repeated linear search through the whole mountinfo (which can have
thousands of records) is inefficient. Instead, let's just
1 collect all the relevant records (only those mount points
that can be a parent of `source`)
2 find the record with the longest mountpath, return its data
This was tested manually with something like
```go
func TestGetSourceMount(t *testing.T) {
mnt, flags, err := getSourceMount("/sys/devices/msr/")
assert.NoError(t, err)
t.Logf("mnt: %v, flags: %v", mnt, flags)
}
```
...but it relies on having a specific mount points on the system
being used for testing.
[v2: add unit tests for ParentsFilter]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 871c957242df9f8c74faf751a2f14eb5178d4140
Component: engine
Use mount.SingleEntryFilter as we're only interested in a single entry.
Test case data of TestShouldUnmountRoot is modified accordingly, as
from now on:
1. `info` can't be nil;
2. the mountpoint check is not performed (as SingleEntryFilter
guarantees it to be equal to daemon.root).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: d3ebcde82aae79be8cbecab365367b17adac4b3e
Component: engine
Functions `GetMounts()` and `parseMountTable()` return all the entries
as read and parsed from /proc/self/mountinfo. In many cases the caller
is only interested only one or a few entries, not all of them.
One good example is `Mounted()` function, which looks for a specific
entry only. Another example is `RecursiveUnmount()` which is only
interested in mount under a specific path.
This commit adds `filter` argument to `GetMounts()` to implement
two things:
1. filter out entries a caller is not interested in
2. stop processing if a caller is found what it wanted
`nil` can be passed to get a backward-compatible behavior, i.e. return
all the entries.
A few filters are implemented:
- `PrefixFilter`: filters out all entries not under `prefix`
- `SingleEntryFilter`: looks for a specific entry
Finally, `Mounted()` is modified to use `SingleEntryFilter()`, and
`RecursiveUnmount()` is using `PrefixFilter()`.
Unit tests are added to check filters are working.
[v2: ditch NoFilter, use nil]
[v3: ditch GetMountsFiltered()]
[v4: add unit test for filters]
[v5: switch to gotestyourself]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: bb934c6aca3e77541dd4fd51b9ab2706294dadda
Component: engine
The unit test is checking that setting of non-default StopTimeout
works, but it checked the value of StopSignal instead.
Amazingly, the test was working since the default StopSignal is SIGTERM,
which has the numeric value of 15.
Fixes: commit e66d21089 ("Add config parameter to change ...")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 743f5afcd13659e2b6b76791415876797e2ede52
Component: engine
This moves the platform specific stuff in a separate package and keeps
the `volume` package and the defined interfaces light to import.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 6a70fd222b95643a8a6b88e2634d5f085ae4122a
Component: engine
All `plugins` tests that require an `ExperimentalDaemon` are migrated
to `integration/plugin/*` and start an experimental daemon to test on
it.
The end goal being to remove the `experimental` build.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 69bab3832ce1d72dd1d3ed2f2b53143ec9ff102c
Component: engine
This makes sure that if the daemon root was already a self-binded mount
(thus meaning the daemonc only performed a remount) that the daemon does
not try to unmount.
Example:
```
$ sudo mount --bind /var/lib/docker /var/lib/docker
$ sudo dockerd &
```
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: c403f0036b9945bd58a84e2c93f794ed9861fe99
Component: engine
The primary need for this is x/net/context now is just a type alias to
the stdlib context package.
This makes issues with conflicts between "golang.org/x/net/context" and
the stdib "context" go away (primarily a concern in interface
definitions/implementations).
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 848ef1da8b0d1c01a84fbe215f24f00fa741690d
Component: engine
Also starts to create more "poll/check" function to `internal/test/daemon`.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: aa0db6f9e1d0fd0420271f419d0d225089a95bbd
Component: engine
Instead of using a global store for volume drivers, scope the driver
store to the caller (e.g. the volume store). This makes testing much
simpler.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 977109d808ae94eb3931ae920338b1aa669f627e
Component: engine
Since the volume store already provides this functionality, we should
just use it rather than duplicating it.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 63826e291ba3b88443b64802084bbb3931857b56
Component: engine
This unblocks the client to take other restore requests and makes sure
that a long/stuck request can't block the client forever.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 806700e410a1b30aa1033d71d825e57880c75298
Component: engine
… and change a bit the method signature
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 42f6fdf059807ecbe66cce90676021363a541154
Component: engine
Porting helpers from `integration-cli/daemon.Daemon` to this struct
and use the API instead of the cli.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 9722214c8add2a83f6c4974848139cadc79d9f0e
Component: engine
Update all build stages to output artefacts to /build
Signed-off-by: Jeff Hajewski <jeff.hajewski@gmail.com>
Upstream-commit: 00555f7b261111084b966920da607a504b068953
Component: engine
- Move go package used by both `integration-cli` and `integration` to
`internal/test/fixtures`.
- Remove fixtures that are not used anymore (moved to `docker/cli` a
while ago) : deploy, notary, secrets.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 5f56503f583f21d655394f755f71849381bd58c7
Component: engine
This use case is currently _working correctly_, which is nice, but there is no documentation to be found about it. This PR fixes that.
Signed-off-by: Per Lundberg <perlun@gmail.com>
Upstream-commit: 82c23208ee
Component: cli
This fix converts some `client.ContainerCreate` to `container.Create`,
and removes some unneeded `name` fields when test containers are created.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: ab9bb47b05b1dde445a5e4ba78ae97303208dc8b
Component: engine
This fix migrates image tag tests from integration-cli to api tests.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 9bcb960508a6066811cffcca1e35ca44d7f1cf94
Component: engine
- Move the code from `integration-cli` to `internal/test`.
- Use `testingT` and `assert` when creating the registry.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 66de2e6e3b6d927a3396743cd7c363aa9f7b776e
Component: engine
In Go 1.10.1, the Deps for pkg\tarsum\tarsum_test.go are empty ([]) and
the PowerShell script ends up setting its import list to a string value
of False instead of an empty array. This can be remedied by forcing the
result to be an array, by concatenating to an empty array (@() + ...)
Signed-off-by: John Stephens <johnstep@docker.com>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit ec3f9230d70506c536a24e844da0f0b3af9b43f6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ee508d47c3917b423b6caae0ad2867c4f6ffe696
Component: engine
Add synchronization around adding logs to a plugin
and reading those logs. Without the follow configuration,
a race occurs between go routines to add the logs into
the plugin and read the logs out of the plugin. This
adds a function to synchronize the action to avoid the
race.
Removes use of file for buffering, instead buffering whole
messages so log count can be checked discretely.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c208f1c8a8d57f9a7f48f63345e77146774aa7a6
Component: engine
Prevent changing the tar output by setting the format to
PAX and keeping the times truncated.
Without this change the archiver will produce different tar
archives with different hashes with go 1.10.
The addition of the access and changetime timestamps would
also cause diff comparisons to fail.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: fb170206ba12752214630b269a40ac7be6115ed4
Component: engine
Remove invalid flush commands, flush should only occur when file
has been completely written. This is already handle, remove these calls.
Ensure data gets written after EOF in correct order and before close.
Remove gname and uname from sum for hash compatibility.
Update tarsum tests for gname/uname removal.
Return valid length after eof.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a422774e593b33bd287d9890544ad9e09b380d8c
Component: engine
This fix is now upstream in Go 1.10
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 950a1aa3170cedabd904cdcfcfa70c7d551a5e16
Component: engine
Full diff
5c1218c956...c15b372ef2
Fixes a panic on concurrent read/write to a map.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 248aed5766ba330ab8cb2b10b03b6ce57dc64283
Component: engine
Docker daemon has a 16K buffer for log messages. If a message length
exceeds 16K, it should be split by the logger and merged at the
endpoint.
This change adds `PartialLogMetaData` struct for enhanced partial support
- LastPartial (bool) : indicates if this is the last of all partials.
- ID (string) : unique 32 bit ID. ID is same across all partials.
- Ordinal (int starts at 1) : indicates the position of msg in the series of partials.
Also, the timestamps across partials in the same.
Signed-off-by: Anusha Ragunathan <anusha.ragunathan@docker.com>
Upstream-commit: 0b4b0a7b5d5de8cb575b666312fceaa2cd58e658
Component: engine
When the authz response buffer limit is hit, perform a flush.
This prevents excessive buffer sizes, especially on large responses
(e.g. `/containers/<id>/archive` or `/containers/<id>/export`).
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 74f8e47352e71aad4015d8d9dea8f16e7a055863
Component: engine
This remove the daemon.Swarm construction by make the new test Daemon
struct aware of swarm.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 83d18cf4e3e84055f7034816eed2a10c04e777ca
Component: engine
… making each folder/suites quicker to run
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: a3323d2e4349b7e8d449c6e571ca3d4aa3e53d63
Component: engine
All `Ipvlan` related test on `DockerSuite` and `DockerNetworkSuite`
are migrated to `ipvlan_test.go`.
The end goal being to remove the `experimental` builds.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 24f934751120ea420b7ba4d2e314df805f3eff06
Component: engine
… and do not use the `docker` cli in it. One of the reason of this
move is to not make `integration` package using legacy
`integration-cli` package.
Next move will be to support swarm within this package *and* provide
some helper function using the api (compared to the one using cli in
`integration-cli/daemon` package).
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: f0d277fe84a72b29c0d2d541c20d5a9c4d7e4884
Component: engine
Currently we hardcode the architecture to the `runtime.GOARCH` when
building a docker image, this will result in a confusing info if the
arch in the base image is different from the one on the host.
This PR takes use of the arch data from the base image during the build
process, thus we can get consistent arch info between the base image
and the finally built image.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 92b17b10bad1f1788419b70db5d6ed9cdf8d15ef
Component: engine
It does not make any sense to vary this based on whether the
rootfs is read only. We removed all the other mount dependencies
on read-only eg see #35344.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Upstream-commit: a729853bc712910574a7417f67764ec8c523928b
Component: engine
… and other cases too. Updating mergo fixes the bugs (but introduced a
slight behaviour change that had to be fixed too)
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 0122730faf
Component: cli
go1.9.5 (released 2018/03/28) includes fixes to the compiler, go command, and
net/http/pprof package. See the Go 1.9.5 milestone on the issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.9.5
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c62336593118020462d65d4e9784143e07de2da7
Component: packaging
We have seen a panic when re-joining a node to a swarm cluster. The
cause of the issue is unknown, so we just need to add a test for nil
objects and log when we get the condition. Hopefully this can prevent
the crash and we can recover the config at a later time.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: 454128c6e82cded211c1412e3eb350b1f7533ee2
Component: engine
go1.9.5 (released 2018/03/28) includes fixes to the compiler, go
command, and net/http/pprof package. See the Go 1.9.5 milestone on
the issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.9.5
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0b6f8a7eff325a683b10d64db363da2145aa1c36
Component: engine
* libnetwork#2121: Retry other external DNS servers on ServFail
* libnetwork#2125: Fix README flag and expose orphan network peers
* libnetwork#2126: Adding goreport card
* libnetwork#2130: Modify awk to use cut in check_ip_overlap
* libnetwork#2117: [Carry 1534] Improve scalabiltiy of bridge network isolation rules
Full changes: 2bf63300c5...5c1218c956
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Upstream-commit: b159da19734269c4a162763ebfa28dff07b703f3
Component: engine
Changes included:
- Ingress network should not be attachable
- [manager/state] Add fernet as an option for raft encryption
- Log GRPC server errors
- Log leadership changes at manager level
- [state/raft] Increase raft ElectionTick to 10xHeartbeatTick
- Remove the containerd executor
- agent: backoff session when no remotes are available
- [ca/manager] Remove root CA key encryption support entirely
- Fix agent logging race (fixes https://github.com/docker/swarmkit/issues/2576)
- Adding logic to restore networks in order
Also adds github.com/fernet/fernet-go as a new dependency
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 27749659d5a30999691e401a351221780a483099
Component: engine
Update the docs so that users don't use `echo` when creating
secrets from STDIN. `echo` adds a trailing new line, so users
will probably be confused when their passwords don't work.
Signed-off-by: Joao Fernandes <joao.fernandes@docker.com>
Upstream-commit: 5238f3e93e
Component: cli
This change fixes bug when $CURDIR contains special character e.g. '&&'.
Change-Id: I658eb60d97c2229cf8b4e0a61dbbecf3edc9d573
Signed-off-by: Fangming.Fang <fangming.fang@arm.com>
Upstream-commit: 83062593d7ce759bb96e271e30460afa4a695dba
Component: engine
commit 617c352e9225 "Don't create devices if in a user namespace"
introduced check, which meant to skip mknod operation when run
in user namespace, but instread skipped FIFO and socket files
copy.
Signed-off-by: Maxim Ivanov <ivanov.maxim@gmail.com>
Upstream-commit: 6f084f292932c464a30b56edb9edbe238bdcf0aa
Component: engine
There is no need to perform a separate losetup step; mount (even
the one in busybox!) is smart enough to set up a loopback device
all by itself (even without -o loop present!). More to say, while
doing this, it sets LO_FLAGS_AUTOCLEAR flag for the kernel to
delete the loopback device as soon as its fs is unmounted (this
is supposed to work since kernel 2.6.25).
Also, remove mount options (-t ext4, -o loop,rw) as they are
either defaults (rw) or mount is smart enough to figure out.
Leave -n so it won't do unnecessary write to container's /etc/mtab.
While at it, touch up some comments.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: a978fd22daefcc68d37c46177a545f7e1dfc6d9e
Component: engine
All `docker build` tests that require an `ExperimentalDaemon` are
migrated to `integration/build` package and start an experimental
daemon to test on it.
The end goal being to remove the `experimental` builds.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 183076e89df64928bd2e94ad0da9725b482367cd
Component: engine
A minor nit. `test01` never been created and used in
`TestDockerNetworkInspectCustomSpecified()` function, so correct it.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: f041953d04bffa2be05466173f02dd016c68286d
Component: engine
Makes sure that if the user cancels a request that the daemon stops
trying to traverse a directory.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 9d46c4c138d7b3f7778c13fe84857712bd6c97a9
Component: engine
Using the default PollSettings functions to adjust the timeout
value instead of changing the value each time when needed.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: b8912feeffcdfd489c9fc1212277840adac2719c
Component: engine
Add the default function per resource to override the `pollSettings`
which will be re-used where it's needed.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: ee6959addc5664a5c55765f2c721f84414ea4779
Component: engine
This call was added as part of commit a042e5a20 and at the time was
useful. sandbox.DisableService() basically calls
endpoint.deleteServiceInfoFromCluster() for every endpoint in the
sandbox. However, with the libnetwork change, endpoint.sbLeave()
invokes endpoint.deleteServiceInfoFromCluster(). The releaseNetwork()
call invokes sandbox.Delete() immediately after
sandbox.DisableService(). The sandbox.Delete() in turn ultimately
invokes endpoint.sbLeave() for every endpoint in the sandbox which thus
removes the endpoint's load balancing entry via
endpoint.deleteServiceInfoFromCluster(). So the call to
sandbox.DisableService() is now redundant.
It is noteworthy that, while redundant, the presence of the call would
not cause errors. It would just be sub-optimal. The DisableService()
call would cause libnetwork to down-weight the load balancing entries
while the call to sandbox.Delete() would cause it to remove the entries
immediately afterwards. Aside from the wasted computation, the extra
call would also propagate an extra state change in the networkDB gossip
messages. So, overall, it is much better to just avoid the extra
overhead.
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: c27417aa7de46daa415600b39fc8a9c411c8c493
Component: engine
This patch allows endpoints to complete servicing connections while
being removed from a service. The fix is entirely within libnetwork
and requires no changes to the moby codebase proper. It operates
by initially down-weighting a container endpoint in the load balancer
to 0 while keeping the endpoint present in the load balancer. This
allows traffic to continue to flow to the endpoint while preventing new
connections from going to the endpoint. This allows the container
to complete requests during the "stop_grace_period" and then exit when
finished without interruption of service.
This change requires propagating the status of disabled service
endpoints via the networkDB. Accordingly, the patch includes both code
to generate and handle service update messages. It also augments the
service structure with a ServiceDisabled boolean to convey whether an
endpoint should ultimately be removed or just disabled. This,
naturally, required a rebuild of the protocol buffer code.
The protocol buffer encoding is designed to support additions of fields
to messages in a backwards-compatible manner. Protocol buffer
unmarshalling code automatically skips past any fields that it isn't
aware of. As a result, an older moby daemon without this fix can
receive and will process correctly networkDB messages from newer moby
daemons with this patch.
As it turns out, the additional field is simply a bool that is otherwise
irrelevent on networkDB create and delete events. So its absence in
older moby daemon processing has no impact. However, the fix leverages
the "update" networkDB message which was previously unused in
libnetwork. Although older libnetwork implementations parse the message
cleanly, they will see the message as unexpected and as such issue a log
at error level indicating the receipt of such.
Other than this there should be no other negative impact for use of this
patch in mixed environments. (Although older mobys won't be able to
gracefully downgrade connections on their nodes of course.)
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: 50dbdeff9fd186bb0e9926996436e1f56529a831
Component: engine
This fixes an issue where the containerd client is cached in a container
object in libcontainerd and becomes stale after containerd is restarted.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 2c682d5209d1d1cdd232ce3909fa25b6023f9a5b
Component: engine
This does not bump the containerd binary.
Picks last commit before go1.10 switch, which is not currently supported
in moby.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 30cb23360e21c7914f41e69496abde4780bd7bc1
Component: engine
This was added as part of a53930a04fa81b082aa78e66b342ff19cc63cc5f with
the intent to sort the mounts in the plugin config, but this was sorting
*all* the mounts from the default OCI spec which is problematic.
In reality we don't need to sort this because we are only adding a
self-binded mount to flag it as rshared.
We may want to look at sorting the plugin mounts before they are added
to the OCI spec in the future, but for now I think the existing behavior
is fine since the plugin author has control of the order (except for the
propagated mount).
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: ec90839ca302ca53a7d55e4c7f79e7b4779f5e15
Component: engine
All `Macvlan` related test on `DockerSuite` and `DockerNetworkSuite`
are migrated to `macvlan_test.go`.
Also, as `macvlan` seems to be out of experimental, this removes
the *skip* when the run is not experimental (and doesn't start a
daemon with experimental either).
The end goal being to remove the `experimental` builds.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: ef5bc603266b9fa5df525319d67329ebc14a8ee7
Component: engine
Now all of the storage drivers use the field "storage-driver" in their log
messages, which is set to name of the respective driver.
Storage drivers changed:
- Aufs
- Btrfs
- Devicemapper
- Overlay
- Overlay 2
- Zfs
Signed-off-by: Alejandro GonzÃlez Hevia <alejandrgh11@gmail.com>
Upstream-commit: 9392838150f5495a63f33fed6570ae41f5a6d62e
Component: engine
When updating a service with the `--force` option, the `ForceUpdate`
property of the taskspec is incremented.
Stack deploy did not take this into account, and reset this
field to its default value (0), causing the service to be
re-deployed.
This patch copies the existing value before updating the service.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 76439457d2
Component: cli
This clarifies that the behaviour of `unless-stopped` will restart the container on daemon start. This was implied before, but now the restart-on-daemon-start behaviour is mentioned directly.
Signed-off-by: David Beitey <david@davidjb.com>
Upstream-commit: 8ca237054f
Component: cli
Using `dpkg --print-architecture` instead of the `uname -m` to abstract
the architecture value from the container images, which the build process
is running inside, to match exactly the behavior specified by the following
Docker file while not 'passthru' to the host.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 803a756941f5e4b68429a3642d52585c8ea6dbaa
Component: engine
Since the recent release of CRIU has already supported other
arches such as AArch64, ppc64le, and s390x, so we can enable
it now.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 7fd54a7a48f9f1bb2b28144e38c886d1fe813a04
Component: engine
Use unique names for test resources of container/inspect, container/ps, container/stop tests
Upstream-commit: 57c504799d979caa706142e6f5dc4be4a5b1e3e2
Component: engine
> HasAvailableFlags checks if the command contains any flags (local
> plus persistent from the entire structure) which are not hidden or
> deprecated.
This fix the `--help` display when the `Options` is empty (but
showing), like on `docker trust key`
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 66fdd085a5
Component: cli
If container will run as non root user, drop permitted, effective caps early
Upstream-commit: b67c1e078c7eeb20199dce301e95fa8999c98109
Component: engine
`--rm` moved to the daemon, so is now also supported
when combined with `-d`.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e9ce688d6e
Component: cli
TestServiceWithPredefinedNetwork test case was failing
at times. To fix the issue, added new API to check
for services after we clean up all services. Tested
multiple times and this sould fix flaky issue.
Signed-off-by: selansen <elango.siva@docker.com>
Upstream-commit: dabffd806c98ab13dbc25e57bee21c5291b9a50c
Component: engine
This moves some of the code that was conditionally
executed on Windows to a separate, windows-only file.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: cd3e84c6b38e74d03ab31db804bc9b49dcab8243
Component: engine
receive operations on a nil channel will always block, this lead containerd not started.
Signed-off-by: Shukui Yang <yangshukui@huawei.com>
Upstream-commit: a50d9c0765d9861d226b323ffe190c0c99c3ac17
Component: engine
Found a bug where a number was being appended to the nightly dev builds
package name causing the latest packages not to be considered as the
latest packages by the package manager.
This makes it so that the nubmer is not included on dev builds.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 63be84fc6ed88847a003dfb32e0dc8bc9126411e
Component: packaging
As soon as the initial executable in the container is executed as a non root user,
permitted and effective capabilities are dropped. Drop them earlier than this, so
that they are dropped before executing the file. The main effect of this is that
if `CAP_DAC_OVERRIDE` is set (the default) the user will not be able to execute
files they do not have permission to execute, which previously they could.
The old behaviour was somewhat surprising and the new one is definitely correct,
but it is not in any meaningful way exploitable, and I do not think it is
necessary to backport this fix. It is unlikely to have any negative effects as
almost all executables have world execute permission anyway.
Use the bounding set not the effective set as the canonical set of capabilities, as
effective will now vary.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Upstream-commit: 15ff09395c001bcb0f284461abbc404a1d8bab4d
Component: engine
This is a test case for issue https://github.com/moby/moby/issues/35123,
making sure we can set container's memory limit when using
`native.cgroupdriver=systemd`.
[v2: skip if no systemd present]
[v3: add --iptables=false to avoid flaky tests with t.Parallel()]
[v4: rebase after PR#36507 merge]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 4ca5c5361059e29ed31074ca5b96f8b2030b5f99
Component: engine
… mainly by skipping if daemon is remote.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 6016e79d2552b21643f4bfd093ce76d8ef956d79
Component: engine
error
local digest cache will be removed when error occured on push image
but it should not be removed if it is an auth error while on auth was
provided
https://github.com/moby/moby/issues/36309
Signed-off-by: 慕陶 <jihui.xjh@alibaba-inc.com>
Upstream-commit: 8b387b165ab2eaab3f9fdac25caa186d05d236a0
Component: engine
There was a lot of repeated parts in this Makefile, I moved them to a
common variable so that if / when we decide to change how things are run
then it's easy to do so.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: c869c9fcf510d1a6c4e25c1a14ea6cb0bb3e6aa4
Component: packaging
Enables the usage of a new versioning scheme for test builds:
* tp -> Technical Previews
* beta -> Beta Releases
* rc -> Release Candidates
* ga -> General Availability Releases
This PR fixes the versioning order for both `deb` and `rpm` packages
when it relates to the new versioning scheme (which may or may not be
used).
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 9ba8e36e8588ea75209d813558c8065844c953a0
Component: packaging
Commit fd0e24b7189374e0fe7c55b6d26ee916d3ee1655 changed
the stats collection loop to use a `sleep()` instead
of `time.Tick()` in the for-loop.
This change caused a regression in situations where
no stats are being collected, or an error is hit
in the loop (in which case the loop would `continue`,
and the `sleep()` is not hit).
This patch puts the sleep at the start of the loop
to guarantee it's always hit.
This will delay the sampling, which is similar to the
behavior before fd0e24b7189374e0fe7c55b6d26ee916d3ee1655.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 481b8e54b45955e40075f49a9af321afce439320
Component: engine
This PR adds support for compressibility of log file.
I added a new option conpression for the jsonfile log driver,
this option allows the user to specify compression algorithm to
compress the log files. By default, the log files will be
not compressed. At present, only support 'gzip'.
Signed-off-by: Yanqiang Miao <miao.yanqiang@zte.com.cn>
'docker logs' can read from compressed files
Signed-off-by: Yanqiang Miao <miao.yanqiang@zte.com.cn>
Add Metadata to the gzip header, optmize 'readlog'
Signed-off-by: Yanqiang Miao <miao.yanqiang@zte.com.cn>
Upstream-commit: f69f09f44ce9fedbc9d70f11980c1fc8d7f77cec
Component: engine
This test case checks that a container created before start
of the currently running dockerd can be exported (as reported
in #36561). To satisfy this condition, either a pre-existing
container is required, or a daemon restart after container
creation.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 6e7141c7a2c0de6fa3d6c9dcc56978a81f9d835e
Component: engine
Bring in:
- moby 36517 ensure hijackedConn implements CloseWrite function
- moby 36489 (fixes `errdefs.ErrSystem` interface)
- moby 36506 pkg/mount: use sort.Slice
- moby 36451 Windows: Report Version and UBR
Also update moby dependencies to keep them in sync
- golang.org/x/sync (no code changes)
- Bump runc to 4fc53a81fb7c994640722ac585fa9ca548971871
- Bump swarmkit to 49a9d7f6ba3c1925262641e694c18eb43575f74b
no local code changes
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b17215c456
Component: cli
Update libnetwork to 1b91bc94094ecfdae41daa465cc0c8df37dfb3dd to bring in a fix
for stale HNS endpoints on Windows:
When Windows Server 2016 is restarted with the Docker service running, it is
possible for endpoints to be deleted from the libnetwork store without being
deleted from HNS. This does not occur if the Docker service is stopped cleanly
first, or forcibly terminated (since the endpoints still exist in both). This
change works around the issue by removing any stale HNS endpoints for a network
when creating it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: fb364f07468e94226250a1e77579ee6117c64be2
Component: engine
- `replaceDockerfileForContentTrust` is only used when content trust is
enabled, so remove the boolean.
- rename `isContentTrustEnabled` to `contentTrustEnabled`
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 63ebcae382
Component: cli
This updates libnetwork to 8892d7537c67232591f1f3af60587e3e77e61d41 to bring in
IPAM fixes for duplicate IP addresses.
- IPAM tests (libnetwork PR 2104) (no changes in vendored files)
- Fix for Duplicate IP issues (libnetwork PR 2105)
Also bump golang/x/sync to match libnetwork (no code-changes, other
than the README being updated)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 55e0fe24db68b16edccb2fa49c3b1b9d3a9ce58c
Component: engine
connection does. If this isn't done, then a container listening on stdin won't
receive an EOF when the client closes the stream at their end.
Signed-off-by: Jim Minter <jminter@redhat.com>
Upstream-commit: 37983921c90b468cafd3ba2ca2574fb81cafe5a7
Component: engine
* added tests on Kubernetes service conversion to swarm service
Signed-off-by: Silvin Lubecki <silvin.lubecki@docker.com>
Upstream-commit: b816bde6cc
Component: cli
Commit 7a7357dae1bccc ("LCOW: Implemented support for docker cp + build")
changed `container.BaseFS` from being a string (that could be empty but
can't lead to nil pointer dereference) to containerfs.ContainerFS,
which could be be `nil` and so nil dereference is at least theoretically
possible, which leads to panic (i.e. engine crashes).
Such a panic can be avoided by carefully analysing the source code in all
the places that dereference a variable, to make the variable can't be nil.
Practically, this analisys are impossible as code is constantly
evolving.
Still, we need to avoid panics and crashes. A good way to do so is to
explicitly check that a variable is non-nil, returning an error
otherwise. Even in case such a check looks absolutely redundant,
further changes to the code might make it useful, and having an
extra check is not a big price to pay to avoid a panic.
This commit adds such checks for all the places where it is not obvious
that container.BaseFS is not nil (which in this case means we do not
call daemon.Mount() a few lines earlier).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: d6ea46cedaca0098c15843c5254a337d087f5cd6
Component: engine
In case ContainerExport() is called for an unmounted container, it leads
to a daemon panic as container.BaseFS, which is dereferenced here, is
nil.
To fix, do not rely on container.BaseFS; use the one returned from
rwlayer.Mount().
Fixes: 7a7357dae1bccc ("LCOW: Implemented support for docker cp + build")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 81f6307eda44ab3a91de6e29304810a976161d74
Component: engine
The `docker trust` commands were moved out of experimental,
and the `docker trust view` command was changed to
`docker trust inspect --pretty`.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2a6808db87
Component: cli
With the ticker this could end up just doing back-to-back checks, which
isn't really what we want here.
Instead use a sleep to ensure we actually sleep for the desired
interval.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 04a0d6b863ed50cfffa79936cf9cdab7a3a9e7df
Component: engine
In info, we only need the number of images, but `CountImages` was
getting the whole map of images and then grabbing the length from that.
This causes a lot of unnecessary CPU usage and memory allocations, which
increases with O(n) on the number of images.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: f6a7763b6f3256bed9a7352021745189d0ca8dc9
Component: engine
I noticed this test failed on Windows:
> 17:46:24 docker_cli_run_test.go:4361:
> 17:46:24 c.Fatal("running container timed out") // cleanup in teardown
I also noticed that in general tests are running slower on Windows,
for example TestStartAttachSilent (which runs a container with
`busybox echo test` and then starts it again) took 29.763s.
This means a simple container start can easily take 15s, which
explains the above failure.
Double the timeout from 15s to 30s.
Fixes: 4e262f6387 ("Fix race on sending stdin close event")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 5043639645123f2728c81c9a55fea525475ec324
Component: engine
I am not quite sure why but this test is sometimes failing like this:
> 15:21:41 --- FAIL: TestLinksEtcHostsContentMatch (0.53s)
> 15:21:41 assertions.go:226:
>
> Error Trace: links_linux_test.go:46
> 15:21:41
> Error: Not equal:
> 15:21:41
> expected: "127.0.0.1\tlocalhost\n::1\tlocalhost
> ip6-localhost
> ip6-loopback\nfe00::0\tip6-localnet\nff00::0\tip6-mcastprefix\nff02::1\tip6-allnodes\nff02::2\tip6-allrouters\n172.17.0.2\tf53feb6df161\n"
> 15:21:41
> received: ""
To eliminate some possible failures (like ignoring stderr from `cat` or
its exit code), let's use container.Exec() to read a file from a container.
Fixes: e6bd20edcbf ("Migrate some integration-cli test to api tests")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: ad2f88d8ccbd9dd0a8d9c4f96ece3956f60489df
Component: engine
As mentioned in commit 9e31938, test cases that use t.Parallel()
and start a docker daemon might step on each other toes as they
try to configure iptables during startup, resulting in flaky tests.
To avoid this, --iptables=false should be used while starting daemon.
Fixes: eaa5192856c1 ("Make container resource mounts unbindable")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: c125e10a0486623ba3badebf974ea6e582373151
Component: engine
Ingress networks will no longer automatically remove their
load-balancing endpoint (and sandbox) automatically when the network is
otherwise upopulated. This is to prevent automatic removal of the
ingress networks when all the containers leave them. Therefore
explicit removal of an ingress network also requires explicit removal
of its load-balancing endpoint.
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: 3da4ebf355d3494d1403b2878a1ae6958b2724e9
Component: engine
This PR prevents automatic removal of the load balancing sandbox
endpoint when the endpoint is the last one in the network but
the network is marked as ingress.
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: bebad150c9c3bc6eb63758c10ef24b9298ecf6e2
Component: engine
The commit https://github.com/moby/moby/pull/35422 had the result of
accidentally causing the removal of the ingress network when the
last member of a service left the network. This did not appear
in swarm instances because the swarm manager would still maintain
and return cluster state about the network even though it had
removed its sandbox and endpoint. This test verifies that after a
service gets added and removed that the ingress sandbox remains
in a functional state.
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: 805b6a7f749a6c7cbb237e21ee7260d536621808
Component: engine
Since now we have only one Dockerfile, so the arch-specific suffix
of the Dockerfile is not needed anymore.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 8eb7ed673b687ae17e2c7df5dd40f8081c299bc2
Component: engine
Removing all the existing arch-specific Dockerfiles since we already
have a new multi-arch supported one as the replacement.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 162f9aee476bd204c2b0146c0128949182e8bd5e
Component: engine
This PR consolidates the existing arch-specific Dockerfiles into only
one file `Dockefile` to ease the code maintenance effort.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: f1701a741d77a92d28138944193e23aabfc74abe
Component: engine
These tests were enabled by changing a config option on the ci
machines, instead of from a patch, so let me disable them
for now on ppc64le and open up another patch to enable them, where I can find
out what the issues are with them.
Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
Upstream-commit: 620ddc78a1437feaa42f40853ef586d268991620
Component: engine
It has been pointed out that if --read-only flag is given, /dev/shm
also becomes read-only in case of --ipc private.
This happens because in this case the mount comes from OCI spec
(since commit 7120976d74195), and is a regression caused by that
commit.
The meaning of --read-only flag is to only have a "main" container
filesystem read-only, not the auxiliary stuff (that includes /dev/shm,
other mounts and volumes, --tmpfs, /proc, /dev and so on).
So, let's make sure /dev/shm that comes from OCI spec is not made
read-only.
Fixes: 7120976d74195 ("Implement none, private, and shareable ipc modes")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: cad74056c09f6276b0f4a996a1511553177cd3d7
Component: engine
The test case checks that in case of IpcMode: private and
ReadonlyRootfs: true (as in "docker run --ipc private --read-only")
the resulting /dev/shm mount is NOT made read-only.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 33dd562e3acff71ee18a2543d14fcbecf9bf0e62
Component: engine
Remove the global variable used. Allows easier unit testing.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 6e21829af4
Component: cli
Scripts were changed around to do static by default, this changes so
that we have "dynamic" inserted where it needs to be inserted
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 130f74155e39ddc36b59d7c47867230284739710
Component: packaging
There was a typo with the buildmode flag for containerd
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 5e4885b9afb1de30133627ce751af2c0e7b72a4e
Component: engine
These were originally static binaries in the first place, this changes
them back to that.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 63c7bb24637fdbfd905096ecc75b435ecefd31e9
Component: engine
To avoid noise in sampling CPU usage metrics, we now sample the system
usage closer to the actual response from the underlying runtime. Because
the response from the runtime may be delayed, this makes the sampling
more resilient in loaded conditions. In addition to this, we also
replace the tick with a sleep to avoid situations where ticks can backup
under loaded conditions.
The trade off here is slightly more load reading the system CPU usage
for each container. There may be an optimization required for large
amounts of containers but the cost is on the order of 15 ms per 1000
containers. If this becomes a problem, we can time slot the sampling,
but the complexity may not be worth it unless we can test further.
Unfortunately, there aren't really any good tests for this condition.
Triggering this behavior is highly system dependent. As a matter of
course, we should qualify the fix with the users that are affected.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: fd0e24b7189374e0fe7c55b6d26ee916d3ee1655
Component: engine
Some of them are skipped for now (because the feature is not supported
or needs more work), some of them are fixed.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 0cf2e6353a
Component: cli
Adds a `make.ps1` powershell script to make it easy to compile and test.
```
.\scripts\make.ps1 -Binary
INFO: make.ps1 starting at 03/01/2018 14:37:28
INFO: Building...
________ ____ __.
\_____ \ | |/ _|
/ | \| <
/ | \ | \
\_______ /____|__ \
\/ \/
INFO: make.ps1 ended at 03/01/2018 14:37:30
.\scripts\make.ps1 -TestUnit
```
The next step is to run e2e tests on windows too.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: facb22573d
Component: cli
The list of changes here is too long to be described.
Maybe it's better to just rewrite the page from scratch, but at list this commit
makes some things better than before.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: e4405205ad
Component: cli
Use the proper man typesetting when describing the options
(i.e. use italics for variable parts, bold for literals).
Expand the description. Give better examples. Refer to mknod(2).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 70dea5c144
Component: cli
Sorting by mount point length can be implemented in a more
straightforward fashion since Go 1.8 introduced sort.Slice()
with an ability to provide a less() function in place.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: a00310b54c0cdcafb402aeea92feca865da9fdf3
Component: engine
This makes `go test .` to pass if run as non-root user, skipping
those tests that require superuser privileges (for `mount`).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 4aae77602a7540b4f977572f3fbdc0891ac57cab
Component: engine
Layer metadata storage has not been implemented outside of the layer
store and will be deprecated by containerd metadata storage. To prepare
for this and freeze the current metadata storage, remove the exported
interface and make it internal to the layer store.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: 0aebcbc32dc6a7b35982a2bfc44335562cbb68ec
Component: engine
With a default filename tests will leave a file in the working directory
that is never cleaned up.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 7c8b5708eb
Component: cli
The TestAPIServiceUpdatePort test performs exactly
the same steps.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 36e1646e4f010ea033643c6df3d9c3dccc166ed2
Component: engine
Fix tests that failed when using cmp.Compare()
internal/test/testutil/assert
InDelta
Fix DeepEqual with kube metav1.Time
Convert some ErrorContains to assert
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 5155cda716
Component: cli
With the inclusion of PR 30897, creating service for host network
fails in 18.02. Modified IsPreDefinedNetwork check and return
NetworkNameError instead of errdefs.Forbidden to address this issue
Signed-off-by: selansen <elango.siva@docker.com>
Upstream-commit: 7cf8b20762cc9491f52ff3f3d94c880378183696
Component: engine
While a `types.go` file is handly when there are a lot of record types,
it is completely obnoxious when used for concrete, utility types with a
struct, new function and method set in the same file. This change
removes the `types.go` file in favor of the simpler approach.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: 244e59e94f153af82e6c3bd8a6c200a48d3cea60
Component: engine
The types defined in the errdefs package do not
satisfy the `error` interface, because they do not
implement `Error()`.
Instead of returning the matched interface, return
the original error.
When matching _multiple_ interfaces/types, Golang doesn't complain:
func getImplementer(err error) error {
switch e := err.(type) {
case
ErrNotFound,
ErrInvalidParameter:
return e
default:
return err
}
}
But matching a single interface/type:
func getImplementer(err error) error {
switch e := err.(type) {
case
ErrNotFound:
return e
default:
return err
}
}
Produces an error:
cannot use e (type ErrNotFound) as type error in return argument: ErrNotFound does not implement error (missing Error method)
Return the original `err` instead of the matched interface/type instead.
Also added some additional tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 59854e407d1541d644a7d814e8e0096dc477dd82
Component: engine
This fix addresses `expected` vs `actual` in integration tests
so that they match `assert.Equal(t, expected, actual)`
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 8a854e933b3dbb26cfce28b920cff61909412c6f
Component: engine
This fix migrates docker rm test in integration-cli
to api tests.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: ed58ba99fb28ceac56063b7f003f38b597ddef80
Component: engine
Move the "unmount and deactivate" code into a separate method, and
optimize it a bit:
1. Do not use filepath.Walk() as there's no requirement to recursively
go into every directory under home/mnt; a list of directories in mnt
is sufficient. With filepath.Walk(), in case some container will fail
to unmount, it'll go through the whole container filesystem which is
excessive and useless.
2. Do not use GetMounts() and check if a directory is mounted; just
unmount it and ignore "not mounted" error. Note the same error
is returned in case of wrong flags set, but as flags are hardcoded
we can safely ignore such case.
While at it, promote "can't unmount" log level from debug to warning.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: f1a459229724f5e8e440b49f058167c2eeeb2dc6
Component: engine
1. Make sure it's clear the error is from unmount.
2. Simplify the code a bit to make it more readable.
[v2: use errors.Wrap]
[v3: use errors.Wrapf]
[v4: lowercase the error message]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 9d00aedebc25507042c5afd4ab8fc6b333ca7c53
Component: engine
This fix migrates events tests in integration-cli to api tests.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 3a749157d2c2b320fea49f7aa4d4eb634f52662f
Component: engine
Log the error returned from logdriver.Log() instead of the logdriver
itself.
Signed-off-by: Cody Roseborough <crrosebo@amazon.com>
Upstream-commit: a1956b5623fad186ad39ae8aca998284003b0cd3
Component: engine
1. Replace EnsureRemoveAll() with Rmdir(), as here we are removing
the container's mount point, which is already properly unmounted
and is therefore an empty directory.
2. Ignore the Rmdir() error (but log it unless it's ENOENT). This
is a mount point, currently unmounted (i.e. an empty directory),
and an older kernel can return EBUSY if e.g. the mount was
leaked to other mount namespaces.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 732dd9b848bec70a2ecb5b4998918886a0cec497
Component: engine
This fix adds several improvement:
1. No need for explicit ContainerRemove as it has been handled in setupTest()
2. Added `container.WithImage` helper function and used it in commit tests.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 6ab465804b0b8cec6c5ac278a21151d49e34885d
Component: engine
We were on a slightly earlier random commit.
We should try to stay on releases where possible...
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Upstream-commit: 9da428da2e
Component: cli
Exec processes do not automatically inherit AppArmor
profiles from the container.
This patch sets the AppArmor profile for the exec
process.
Before this change:
apparmor_parser -q -r <<EOF
#include <tunables/global>
profile deny-write flags=(attach_disconnected) {
#include <abstractions/base>
file,
network,
deny /tmp/** w,
capability,
}
EOF
docker run -dit --security-opt "apparmor=deny-write" --name aa busybox
docker exec aa sh -c 'mkdir /tmp/test'
(no error)
With this change applied:
docker exec aa sh -c 'mkdir /tmp/test'
mkdir: can't create directory '/tmp/test': Permission denied
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8f3308ae10ec9ad0dd4edfb46fde53a0e1e19b34
Component: engine
This fix migrates export tests in integration-cli to api tests.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 4e702cf70d50ee5b0737270f27d9973fd3084c66
Component: engine
The canonical import comment was added some time ago, though several
newly added files do not have the comment. This fix adds the missing
canonical import comment to files in integration tests
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 9045406144413920da49629a181bb787c670197e
Component: engine
The Docker 18.03 branch was cut from
CLI: 84c7dd6057
Engine: fb005971c2bd82de482d1cfe924f2f4ec60c60ae
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 06dd62ab1e)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
This fix is a minor enhancement to replace several ContainerCreate with
helper funcs of `container.Create` in tests.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 6ad4720c78d6ac61a60a3e7ed1d0c0119c5d103e
Component: engine
Fix runc exec on big-endian, causing:
container_linux.go:265: starting container process caused "open /dev/pts/4294967296: no such file or directory"
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: aab5eaddccb8cb196fdb1e285890dfa94a071b14
Component: engine
Release notes: https://github.com/opencontainers/runc/releases/tag/v1.0.0-rc5
Possibly relevant changes included:
- chroot when no mount namespaces is provided
- fix systemd slice expansion so that it could be consumed by cAdvisor
- libcontainer/capabilities_linux: Drop os.Getpid() call
- Update console dependency to fix runc exec on BE (causing: `container_linux.go:265: starting container process caused "open /dev/pts/4294967296: no such file or directory"`)
- libcontainer: setupUserNamespace is always called (fixes: Devices are mounted with wrong uid/gid)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a2f5a1a5b2d77d694c5bd47798be15b3c0bcdf70
Component: engine
Overriding is the incorrect part of speech for this sentence. It is more common to state that the values are overridden instead.
Other options would include:
```golang
// ldflags is overriding these values
```
```golang
// These values will be overridden by ldflags
```
etc.
Signed-off-by: Kevin Kirsche <kevin.kirsche@verizon.com>
Upstream-commit: 0e2fdc92ce
Component: cli
It looks like no one uses this function.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 0450f61cb995c8fc2f41a6909526be6ed4093565
Component: engine
Remove temp directories and close file loggers in container unit tests.
Signed-off-by: mnussbaum <michael.nussbaum@getbraintree.com>
Upstream-commit: 07d5446fe27cb92d881df48be6e8a6510d9608b0
Component: engine
This fix migrates config inspect test in integration-cli
to api test.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 4b99d782079dc390c2d8fb78f6973bbeee7d8a47
Component: engine
Changes Details:
--------------
Fixes: #36395
Refactoring the code to do the following:
1. Add the method `errBindSourceDoesNotExist` inside `validate.go` to be in-line with the rest of error message
2. Utilised the new method inside `linux_parser.go`, `windows_parser.go` and `validate_test.go`
3. Change the format from `bind mount source path: '%s' does not exist` to `bind mount source path does not exist: %s`
4. Reflected the format change into the 2 unit tests, namely: `volume_test.go` and `validate_test.go`
5. Reflected the format change into `docker_api_containers_test.go` integration test
Signed-off-by: Amr Gawish <amr.gawish@gmail.com>
Upstream-commit: df6af282b9048dfedcd7b7a9a89126aca887f4e1
Component: engine
This fix migrates several docker rm tests to api tests
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 6bd4f4801b244555213f0040b9885033e99d4ae8
Component: engine
Remove TestBuildRenamedDockerfile and TestBuildDockerfileOutsideContext
that are cli-only tests (and already tested in the docker/cli
repository).
Also adds some comments on few tests that could be migrate to
docker/cli.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 894c213b3bd6f4d8f344837b5b5084360a013680
Component: engine
Commit 2b17f4c8a8 fixed the way empty labels
are taken into account (i.e. not interpolated from environment variable),
but it created a regression.
`ValidateLabel` functions doesn't allow empty label value, but it has
always been possible to pass an empty label via the cli (`docker run --label foo`).
This fixes that by not validating the label flag.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 31dc5c0a9a
Component: cli
Using the `busybox:glibc` instead of `busybox:latest` to the
network related test cases (`ping` issue).
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 0d31dee5ec724731607e277a415b1ca4ecb7b2c4
Component: engine
Adding `busybox:latest` and `busybox:glibc` as the frozen images
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 3ae45c5f173d88ba621116f9e1b5611fe687e050
Component: engine
This fix migrates volumes tests in integration-cli to api tests
in integration/
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: d896f87c0595134fa2f0787dad30b237815f233f
Component: engine
imageService provides the backend for the image API and handles the
imageStore, and referenceStore.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 0dab53ff3cb0aae91aae068a3f1f2fd32339e23b
Component: engine
The Docker 18.03 branch was cut from
CLI: 84c7dd6057
Engine: fb005971c2bd82de482d1cfe924f2f4ec60c60ae
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 06dd62ab1e
Component: cli
This fix migrates some secret create tests to api tests,
and remove redundant TestConfigCreate.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 99e28188507bbcb925b0c09df6b53cdd882d24c5
Component: engine
This fix update docker-py so that containers from the tests run
could be cleaned up during teardown:
```diff
-ENV DOCKER_PY_COMMIT 5e28dcaace5f7b70cbe44c313b7a3b288fa38916
+ENV DOCKER_PY_COMMIT 8b246db271a85d6541dc458838627e89c683e42f
```
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 66935a0f64f0a72162fb3919c759f4f500b6c372
Component: engine
This fix moves helper functions containerIsStopped and
containerIsInState to integration/internal/container,
so that they could be used outside of integration/container.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: eda311c18f388ed4541dc44dcfba08cd4347a685
Component: engine
Binary installation was broken after the
hack/dockerfile/install-binaries script was removed.
This remedies that.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 59164bedeab571029805a107e8e5a32fc9cd56b3
Component: packaging
We need to clean the resources created in some test cases, else
in some cases we'll get below error for other tests:
> FAIL: docker_experimental_network_test.go:37: DockerNetworkSuite.TestDockerNetworkMacvlanPersistance
> docker_experimental_network_test.go:44:
> ...
> Command: ip link add dm-dummy0 type dummy
> ExitCode: 2
> Error: exit status 2
> Stdout:
> Stderr: RTNETLINK answers: File exists
> ...
Logically, each test case should be independent, the failure of previous
test case should not have side-effect for the test cases followed.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 57d85e7e54f7d074af8c496cba43ee18d3815207
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
While debugging #32838, it was found (https://github.com/moby/moby/issues/32838#issuecomment-356005845) that the utility VM in some circumstances was crashing. Unfortunately, this was silently thrown away, and as far as the build step (also applies to docker run) was concerned, the exit code was zero and the error was thrown away. Windows containers operate differently to containers on Linux, and there can be legitimate system errors during container shutdown after the init process exits. This PR handles this and passes the error all the way back to the client, and correctly causes a build step running a container which hits a system error to fail, rather than blindly trying to keep going, assuming all is good, and get a subsequent failure on a commit.
With this change, assuming an error occurs, here's an example of a failure which previous was reported as a commit error:
```
The command 'powershell -Command $ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue'; Install-WindowsFeature -Name Web-App-Dev ; Install-WindowsFeature -Name ADLDS; Install-WindowsFeature -Name Web-Mgmt-Compat; Install-WindowsFeature -Name Web-Mgmt-Service; Install-WindowsFeature -Name Web-Metabase; Install-WindowsFeature -Name Web-Lgcy-Scripting; Install-WindowsFeature -Name Web-WMI; Install-WindowsFeature -Name Web-WHC; Install-WindowsFeature -Name Web-Scripting-Tools; Install-WindowsFeature -Name Web-Net-Ext45; Install-WindowsFeature -Name Web-ASP; Install-WindowsFeature -Name Web-ISAPI-Ext; Install-WindowsFeature -Name Web-ISAPI-Filter; Install-WindowsFeature -Name Web-Default-Doc; Install-WindowsFeature -Name Web-Dir-Browsing; Install-WindowsFeature -Name Web-Http-Errors; Install-WindowsFeature -Name Web-Static-Content; Install-WindowsFeature -Name Web-Http-Redirect; Install-WindowsFeature -Name Web-DAV-Publishing; Install-WindowsFeature -Name Web-Health; Install-WindowsFeature -Name Web-Http-Logging; Install-WindowsFeature -Name Web-Custom-Logging; Install-WindowsFeature -Name Web-Log-Libraries; Install-WindowsFeature -Name Web-Request-Monitor; Install-WindowsFeature -Name Web-Http-Tracing; Install-WindowsFeature -Name Web-Stat-Compression; Install-WindowsFeature -Name Web-Dyn-Compression; Install-WindowsFeature -Name Web-Security; Install-WindowsFeature -Name Web-Windows-Auth; Install-WindowsFeature -Name Web-Basic-Auth; Install-WindowsFeature -Name Web-Url-Auth; Install-WindowsFeature -Name Web-WebSockets; Install-WindowsFeature -Name Web-AppInit; Install-WindowsFeature -Name NET-WCF-HTTP-Activation45; Install-WindowsFeature -Name NET-WCF-Pipe-Activation45; Install-WindowsFeature -Name NET-WCF-TCP-Activation45;' returned a non-zero code: 4294967295: container shutdown failed: container ba9c65054d42d4830fb25ef55e4ab3287550345aa1a2bb265df4e5bfcd79c78a encountered an error during WaitTimeout: failure in a Windows system call: The compute system exited unexpectedly. (0xc0370106)
```
Without this change, it would be incorrectly reported such as in this comment: https://github.com/moby/moby/issues/32838#issuecomment-309621097
```
Step 3/8 : ADD buildtools C:/buildtools
re-exec error: exit status 1: output: time="2017-06-20T11:37:38+10:00" level=error msg="hcsshim::ImportLayer failed in Win32: The system cannot find the path specified. (0x3) layerId=\\\\?\\C:\\ProgramData\\docker\\windowsfilter\\b41d28c95f98368b73fc192cb9205700e21
6691495c1f9ac79b9b04ec4923ea2 flavour=1 folder=C:\\Windows\\TEMP\\hcs232661915"
hcsshim::ImportLayer failed in Win32: The system cannot find the path specified. (0x3) layerId=\\?\C:\ProgramData\docker\windowsfilter\b41d28c95f98368b73fc192cb9205700e216691495c1f9ac79b9b04ec4923ea2 flavour=1 folder=C:\Windows\TEMP\hcs232661915
```
Upstream-commit: 8c52560ea4593935322c1d056124be44e234b934
Component: engine
This images is used to run integration and integration-cli tests on
anything that implements the docker api :). The image wasn't building
anywore :D
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 9c46b587eec84f6d9b525ecca1d8b66a51055c12
Component: engine
Originally I worked on this for the multi-stage build Dockerfile
changes. Decided to split this out as we are still waiting for
multi-stage to be available on CI and rebasing these is pretty annoying.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: b529d1b0936b90ae14d584c73f7332919f8d76b7
Component: engine
Includes:
- [client] Remove duplicate NewClient functions
- Add API support for templated secrets and configs
- Adjust minimum API version for templated configs/secrets
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 60930d309c
Component: cli
- Add `Version` to `types.Config`
- Add a new `Services` types (that is just `[]ServiceConfig`) and add
`MarshalYAML` method on it.
- Clean other top-level custom marshaling as `Services` is the only one
required.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: cf86a4d922
Component: cli
This leaks information about keyrings on the host. Keyrings are
not namespaced.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Upstream-commit: de23cb939858a66829d5b75057c7ac664c5acda5
Component: engine
… otherwise the k8s controller might fail to parse the file as it will
think it's version 1.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 9f9f1c8515
Component: cli
Also adds a note to the API version history
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a3efeaad529b945ce5af78c4b08a6ed47399f8d5
Component: engine
This fixes an issue where the container LogPath was empty when the
non-blocking logging mode was enabled. This change sets the LogPath on
the container as soon as the path is generated, instead of setting the
LogPath on a logger struct and then attempting to pull it off that
logger at a later point. That attempt to pull the LogPath off the logger
was error prone since it assumed that the logger would only ever be a
single type.
Prior to this change docker inspect returned an empty string for
LogPath. This caused issues with tools that rely on docker inspect
output to discover container logs, e.g. Kubernetes.
This commit also removes some LogPath methods that are now unnecessary
and are never invoked.
Signed-off-by: junzhe and mnussbaum <code@getbraintree.com>
Upstream-commit: 20ca612a59c45c0bd58c71c199a7ebd2a6bf1a9e
Component: engine
Use the default version because it is used by the client package
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: e73d742cd7deee396eac3c97664b40264ee358cb
Component: engine
Test made some bad assumptions about on-disk state of volume data.
This updates the test to only test based on what the volume API is
designed to provide.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 0df654f3d61d8691ee113a8429bcc8ef65786bc7
Component: engine
The function name should be TestUpdateCPUQuota and not TestUpdateCPUQUota.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 31825081d4c5e643a150b515547cb2a2ea223de4
Component: engine
Use client instead of helpers for TLS in integration test
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: a68ae4a2d95b1ff143025a435195af0f1ab30ace
Component: engine
Zesty is EOL and doesn't even have a subdirectory in the `deb` directory of this repo. There's no need to have it as a default.
Signed-off-by: Corbin <corbin.coleman@docker.com>
Upstream-commit: 9b3e8f85f039dfe7391512de6143ef213574973a
Component: packaging
- Add API support for SCTP port mapping
- Add canonical import path
- Add `REMOVE` and `ORPHANED` to TaskState
- Fix TLS from environment variables in client
- Introduce NewClientWithOpts func to build custom client easily
- Wrap response errors for container copy methodsto fix error detection using
`IsErrNotFound` and `IsErrNotImplemented` for `ContainerStatPath`,
`CopyFromContainer`, and `CopyToContainer` methods.
- Produce errors when empty ids are passed into inspect calls
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: aaa7a7cb95
Component: cli
Historically, the Dockerfile had to be insde the build-context, because it was
sent as part of the build-context.
3f6dc81e10
added support for passing the Dockerfile through stdin, in which case the
contents of the Dockerfile is injected into the build-context.
This patch uses the same mechanism for situations where the location of the
Dockerfile is passed, and its path is outside of the build-context.
Before this change:
$ mkdir -p myproject/context myproject/dockerfiles && cd myproject
$ echo "hello" > context/hello
$ echo -e "FROM busybox\nCOPY /hello /\nRUN cat /hello" > dockerfiles/Dockerfile
$ docker build --no-cache -f $PWD/dockerfiles/Dockerfile $PWD/context
unable to prepare context: the Dockerfile (/Users/sebastiaan/projects/test/dockerfile-outside/myproject/dockerfiles/Dockerfile) must be within the build context
After this change:
$ mkdir -p myproject/context myproject/dockerfiles && cd myproject
$ echo "hello" > context/hello
$ echo -e "FROM busybox\nCOPY /hello /\nRUN cat /hello" > dockerfiles/Dockerfile
$ docker build --no-cache -f $PWD/dockerfiles/Dockerfile $PWD/context
Sending build context to Docker daemon 2.607kB
Step 1/3 : FROM busybox
---> 6ad733544a63
Step 2/3 : COPY /hello /
---> 9a5ae1c7be9e
Step 3/3 : RUN cat /hello
---> Running in 20dfef2d180f
hello
Removing intermediate container 20dfef2d180f
---> ce1748f91bb2
Successfully built ce1748f91bb2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a1048523d2
Component: cli
- explain the either "name" or "id" can be used to reference a container
- explain that signals can be sent by name or number
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0c085c2a77
Component: cli
1. Use integration/internal/exec, removing the getContainerSysFSValue().
2. Avoid repeating magic numbers, use a variable for those.
3. Fix order of arguments to assert.Equal (first "expected", then "actual").
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 0f9da07b569f0d9cbe574db3af3951b4d5c968c0
Component: engine
An implementation of exec in TestUpdateCPUQUota had a few issues,
including resource leaking and calling both ContainerExecAttach and
ContainerExecRun. The last one makes the test flaky:
update_linux_test.go:136: expected cgroup value 20000, got: Error: Exec
command f923baf709525f6b38f6511126addc5d9bb88fb477eeca1c22440551090fa2bb
is already running
Fix by using the integration/internal/exec package.
While at it, use require/assert to further improve code readability.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 8a7d6143fca69623e2f5d409328c97603843ccb6
Component: engine
Some test cases might need an ability to execute a command inside a
container (in order to analyse its output and/or exit code). It is a bit
complicated operation to do so using engine API. The function provided
aims to hide this complexity, making exec almost as simple as 'docker
exec'.
NOTE that the exec is synchronous, and command's stdin is closed.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 01143afe54f1be7308c5663a0cc110740626c62b
Component: engine
When a container is created if "--network" is set to "host" all the
ports in the container are bound to the host.
Thus, adding "-p" or "--publish" to the command-line is meaningless.
Unlike "docker run" and "docker create", "docker service create" sends
an error message when network mode is host and port bindings are given
This patch however suggests to send a warning message to the client when
such a case occurs.
The warning message is added to "warnings" which are returned from
"verifyPlatformContainerSettings".
Signed-off-by: Boaz Shuster <ripcurld.github@gmail.com>
Upstream-commit: 6e78fdb790d2e1dbf95a1733cab9395b1b936622
Component: engine
This adds the Engine version to `docker node ls`, and `.EngineVersion` as a
template option.
With this patch applied:
docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
wp9231itoqsh4rqceojqo01vp * linuxkit-025000000001 Ready Active Leader 18.01.0-ce
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e888dd711f
Component: cli
This updates the containerd/continuity package to d8fb8589b0e8e85b8c8bbaa8840226d0dfeb7371
which fixes builds failing on ARM 32-bit, after this dependency was added in
b3aab5e31faf04d8a29f17be55562e4d0c0cb364
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f0947a541866ca05b030afe07dd659887a655e3e
Component: engine
backend.
Currently, the timeout is set to 0, which means no timeout. Set it to a
sane default timeout of 30 seconds.
Signed-off-by: Anusha Ragunathan <anusha.ragunathan@docker.com>
Upstream-commit: f1e2030ce8
Component: cli
Makes sure if the user specifies an older API version that we don't pass
through templating options for versions that templating was not
supported.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: a407761e483d9c5ea425a6fd5e55fec03a90485c
Component: engine
On unix, merge secrets/configs handling. This is important because
configs can contain secrets (via templating) and potentially a config
could just simply have secret information "by accident" from the user.
This just make sure that configs are as secure as secrets and de-dups a
lot of code.
Generally this makes everything simpler and configs more secure.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: c02171802b788fb2d4d48bebcee2a57c8eabeeaa
Component: engine
This makes configs and secrets behavior identical.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 8e8f5f4457d8e1b02031576dbc18c903be4bcfb6
Component: engine
The BurntSushi/toml code is now re-licensed as MIT. While
the vendored package was already updated, the tomlv binary
used was still using the old license type.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 508d5a0bc00400fb41395223cb4c8be3d9b74ade
Component: engine
dm_task_deferred_remove is not supported by all distributions, due to
out-dated versions of devicemapper. However, in the case where the
devicemapper library was updated without rebuilding Docker (which can
happen in some distributions) then we should attempt to dynamically load
the relevant object rather than try to link to it.
This can only be done if Docker was built dynamically, for obvious
reasons.
In order to avoid having issues arise when dlsym(3) was unnecessary,
gate the whole dlsym(3) logic behind a buildflag that we disable by
default (libdm_dlsym_deferred_remove).
Signed-off-by: Aleksa Sarai <asarai@suse.de>
Upstream-commit: 98fe4bd8f1e35f8e498e268f653a43cbfa31e751
Component: engine
This is only for the case when dockerd has had to re-mount the daemon
root as shared.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 487c6c7e73dbb7871e80d75f176dd2a3539a2947
Component: engine
This ensures that all log plugins are registered when the log validator
is run.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: b0b9a25e7e60abbe143e149ccaaf4dfb62044016
Component: engine
The utils package was moved to "internal" in commit
af306d149e76b100e08972cda364647bd7bcfe1e
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ce35439015e4d2190bd82a3b6dfec98f7a12ac90
Component: engine
This fix migreates container diff tests in integration-cli
to api tests.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 9537498cedc4e28ee0c8c26ba3d9e59ebb59fcad
Component: engine
This fix updates docker-py:
```
-ENV DOCKER_PY_COMMIT 1d6b5b203222ba5df7dedfcd1ee061a452f99c8a
+ENV DOCKER_PY_COMMIT 5e28dcaace5f7b70cbe44c313b7a3b288fa38916
```
The updated docker-py includes https://github.com/docker/docker-py/pull/1909
which is required to have #36292 pass the tests.
Full diff is in 1d6b5b2032...5e28dcaace.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 9d9af83b0fd70ff6a7faa15cf8746669f0f3b588
Component: engine
To ensure we are loading the composefile the same wether we are pointing
to swarm or kubernetes, we need to share the loading code between both.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 570ee9cb54
Component: cli
These are now more in-line with wiki instructions. Also removes
broken/deprecated make target test-unit.
Signed-off-by: Brett Randall <javabrett@gmail.com>
Upstream-commit: ba49e8c49830b69c833edff3c393716da20f897a
Component: engine
Update and unify the `busybox` images on all arches to the `glibc` multi-arch
version and remove the temp workaround on amd64 which uses the old version
busybox (v1.26) before this PR to bypass the failure of those network related
test cases. Also, this PR will fix all the network related issues with `glibc`
version `busybox` image.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 3a971009763387856bb7f162accdf6714100e39b
Component: engine
When tailing a container log, if the log file is empty it will cause the
log stream to abort with an unexpected `EOF`.
Note that this only applies to the "current" log file as rotated files
cannot be empty.
This fix just skips adding the "current" file the log tail if it is
empty.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: f40860c5f3d3575629d4a932207e866c1fea625d
Component: engine
This fix is related to 36219
This fix updates runc to:
```
-RUNC_COMMIT=9f9c96235cc97674e935002fc3d78361b696a69e
+RUNC_COMMIT=6c55f98695e902427906eed2c799e566e3d3dfb5
-github.com/opencontainers/runc 9f9c96235cc97674e935002fc3d78361b696a69e
+github.com/opencontainers/runc 6c55f98695e902427906eed2c799e566e3d3dfb5
```
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: d644050db2a2e341726df49b7a43fc37c05d554a
Component: engine
This fix is a sync up with 36266 so that relevant api tests
use the newly added container.Run/Create in helper package
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 9fcd2a05106af98e6ffd6efb9f124d64426956e4
Component: engine
The PKCS8 changes updated the encryption on the keys so that the
`x509.IsEncryptedPEMBlock` may no longer return true because it cannot
parse the PEM block. The `keyutils` module in SwarmKit can tell whether
it is encrypted either way.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ed7d7b9b0577895e8305e87816cb52fc4e2a0b92
Component: engine
Describe the possible values for `--restart`, `--ipc`, and `--network`
options. While at it, improve formatting for `--name` options arguments.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 0d9bd33bab
Component: cli
The biggest motivation for this is proper table rendering; in the
old version it was broken so tables were not rendered at al
(i.e. anything that was put into table was lost, for example,
description of LOG_* log levels in dockerd(8) page).
This also fixes lists, including nested lists. This fixes the
description of behavior in docker-cp(1) which is rendered as a tree:
BEFORE:
```
Assuming a path separator of /, a first argument of SRC_PATH and second
argument of DEST_PATH, the behavior is as follows:
· SRC_PATH specifies a file
· DEST_PATH does not exist
· the file is saved to a file created at DEST_PATH
· DEST_PATH does not exist and ends with /
· Error condition: the destination directory must exist.
...
```
AFTER:
```
Assuming a path separator of /, a first argument of SRC_PATH and second
argument of DEST_PATH, the behavior is as follows:
· SRC_PATH specifies a file
· DEST_PATH does not exist
· the file is saved to a file created at DEST_PATH
· DEST_PATH does not exist and ends with /
· Error condition: the destination directory must exist.
...
```
Manually checking the diff between the man pages generated by the old
and the new version, there are no changes other than the indentation
(.RS/.RE) for lists, and proper formatting for tables. Formatted man
pages also look decent, nothing seems broken.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 6566f5ff2c
Component: cli
This fix migrates several swarm configs tests from integration-cli to api tests
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 63bd2425fd5ec0dd17aad830652c1b0c5515a31c
Component: engine
This fix migrates some update restart tests in
integration-cli to api tests in integration.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 5b65cee9150b343c97606adc525cf680fb330642
Component: engine
This fix migrates container pause tests from integration-cli
to api tests in integration/.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: ea2f076ca986038c48af37feaba6524f32761406
Component: engine
This fix migrates session tests in integration-cli to
api tests in integration.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 1d40c6a8999bdddbb57c199c2cf6e5d103153e69
Component: engine
This fix migrates several swarm secrets from integration-cli to api tests
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: f955d2e2042cabaeac8a451b83a507b76e606d33
Component: engine
This fix migrates tests in integration-cli/docker_cli_oom_killed_test.go
to api tests.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 3c21274b76323883e8ffb146fb989850c4de221a
Component: engine
This fix migrates config list tests from integration-cli to api tests
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 246f450ac4fcaff15f36874475f15f435ab03a15
Component: engine
This fix is a follow up to 36266 to update some api tests
to use the newly added container helper package.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: e9f19df6a9d8ba682f3c9dcdaffed2ac4e0c6189
Component: engine
To help creating/running/… containers using the client for test integration.
This should make test more readable and reduce duplication a bit.
Usage example
```
// Create a default container named foo
id1 := container.Create(t, ctx, client, container.WithName("foo"))
// Run a default container with a custom command
id2 := container.Run(t, ctx, client, container.WithCmd("echo", "hello world"))
```
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 0bb7d42b03bfb125cd50ab50dabe99726e66ab71
Component: engine
Both names have no real sense, but one allows to make sure these packages
aren't used outside of `integration`.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: af306d149e76b100e08972cda364647bd7bcfe1e
Component: engine
- Replace EC Private Key with PKCS#8 PEMs
- Fix IP overlap with empty EndpointSpec
- Add support for Support SCTP port mapping (depends on changes in libnetwork)
- [orchestrator/updater] Do not reschedule tasks if only placement constraints change and are satisfied by the assigned node
- Ensure task reaper stopChan is closed no more than once
- [manager/dispatcher] Synchronization fixes
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 191324b8f1e1a2b2dbe692f869bfbf525d150ade
Component: engine
A recent change accidently caused any TLS configuration in FromEnv to be
ignored. This change alters WithHost to create a new http client only if
one doesn't already exist, and otherwise applies the logic to the
transport on the existing client. This preserves the TLS configuration
that might already be on the client.
Signed-off-by: Drew Erny <drew.erny@docker.com>
Upstream-commit: 80904e9571e7724328160c97ede6a71864f3c06a
Component: engine
When the daemon restores containers on daemon restart, it syncs up with
containerd to determine the existing state. For stopped containers it
then removes the container metadata from containerd.
In some cases this is not handled properly and causes an error when
someone attempts to start that container again.
In particular, this case is just a bad error check.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: c0d56ab71701ba47ca6066c7952e724f4f5977c0
Component: engine
Since commit e9b9e4ace294230c6b8eb has landed, there is a chance that
container.RWLayer is nil (due to some half-removed container). Let's
check the pointer before use to avoid any potential nil pointer
dereferences, resulting in a daemon crash.
Note that even without the abovementioned commit, it's better to perform
an extra check (even it's totally redundant) rather than to have a
possibility of a daemon crash. In other words, better be safe than
sorry.
[v2: add a test case for daemon.getInspectData]
[v3: add a check for container.Dead and a special error for the case]
Fixes: e9b9e4ace294230c6b8eb
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 195893d38160c0893e326b8674e05ef6714aeaa4
Component: engine
This fix migrates TestKillDifferentUserContainer to api test
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 0855922cd3cd1e9d846fd85ef968653ff8649a44
Component: engine
`docker stack deploy` now support multiple composefil. This updates the
reference doc to take that fact into account.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 4c50007496
Component: cli
This fix migrates docker_api_inspect_unix_test.go to integration api test
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 8197529ca2fabc95b9fc4a7e3ff643740b1f4388
Component: engine
Service of inspect network can't be finished within 10s on AArch64 platform,
so we need to adjust the timeout value avoid to paper cover the real issue, plus
to make the integreation test can continue while not terminate with below error:
> === RUN TestInspectNetwork
> --- FAIL: TestInspectNetwork (27.65s)
> daemon.go:285: [de79880f4ed4a] waiting for daemon to start
> daemon.go:317: [de79880f4ed4a] daemon started
> inspect_test.go:57: timeout hit after 10s: waiting for tasks to enter run state
> daemon.go:275: [de79880f4ed4a] exiting daemon
> FAIL
> ---> Making bundle: .integration-daemon-stop (in bundles/test-integration)
> Removing test suite binaries
> Makefile:171: recipe for target 'test-integration' failed
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 8f5c1841a8bffb4a7e33f51174177b7ad0182969
Component: engine
This fix migrates secret list tests from integration-cli to api tests
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 9349c035831e09057bd039b8f677fa19d3354f73
Component: engine
This fix migrates rename tests in integration-cli to api tests
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: be24a6b11e36b542d37cf168f6d198d5445803a5
Component: engine
The imageRefs map was being popualted with containerID, and accessed
with an imageID which would never match.
Remove this broken code because: 1) it hasn't ever worked so isn't
necessary, and 2) because at best it would be racy
ImageDelete() should already handle preventing of removal of used
images.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 3aa4f7f0d71f04c5cc93d5e80cbdd47b0b5fdb7f
Component: engine
This fix migrates TestCreateTmpfsMountsTarget test to api test,
and removed integration-cli/docker_cli_create_unix_test.go
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: f601bc16d555238c86dad1cc0a67bcfcf36a3301
Component: engine
This allows to create a client with default values and override those
using functors. As an example, `NewEnvClient()` becomes
`NewClientWithOpts(FromEnv)` ; and if you want a different api version
for this client : `NewClientWithOpts(FromEnv, WithVersion("1.35"))`
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 772edd020cc784913973387b00c4d0c526a10a26
Component: engine
This fixes a vulnerability in `go get` (CVE-2018-6574, http://golang.org/issue/23672),
but shouldn't really affect our code, but it's good to keep in sync.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b32599761f
Component: cli
This workaround for golang/go#15286 was added for Nano server TP5 in
fa82c0aa10cfac8c6d5e2446876dc79b2b0c1bf9, and should no longer be
needed
Due to a security fix in Go 1.9.4/1.8.7, loading the .dll is no longer
allowed, and produces an error:
.\docker_windows.go:9:3: //go:cgo_import_dynamic main.dummy CommandLineToArgvW%2 "shell32.dll" only allowed in cgo-generated code
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 250193387c98a4ad69a6591d5fe5a39c1409ffba
Component: engine
volume prune command removes only local volumes
not used by at least one container.
Signed-off-by: Sungwon Han <sungwon.han@navercorp.com>
Upstream-commit: 34504d0a1e
Component: cli
This fixes a vulnerability in `go get` (CVE-2018-6574, http://golang.org/issue/23672),
but shouldn't really affect our code, but it's good to keep in sync.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 6263b1254b179af81ff4ef97563fe2e1a053993a
Component: packaging
This fixes a vulnerability in `go get` (CVE-2018-6574, http://golang.org/issue/23672),
but shouldn't really affect our code, but it's good to keep in sync.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: caeab268430a033fedd27c53be16758ac1a0f71e
Component: engine
This reverts commit 0c2821d6f2de692d105e50a399daa65169697cca.
Due to other changes this is no longer needed and resolves some other
issues with plugins.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 37d7b7ddc332541f516d0c41d9ad30b28f2d3e22
Component: engine
Before this change, volume management was relying on the fact that
everything the plugin mounts is visible on the host within the plugin's
rootfs. In practice this caused some issues with mount leaks, so we
changed the behavior such that mounts are not visible on the plugin's
rootfs, but available outside of it, which breaks volume management.
To fix the issue, allow the plugin to scope the path correctly rather
than assuming that everything is visible in `p.Rootfs`.
In practice this is just scoping the `PropagatedMount` paths to the
correct host path.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 0e5eaf8ee32662182147f5f62c1bfebef66f5c47
Component: engine
Setting up the mounts on the host increases chances of mount leakage and
makes for more cleanup after the plugin has stopped.
With this change all mounts for the plugin are performed by the
container runtime and automatically cleaned up when the container exits.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: a53930a04fa81b082aa78e66b342ff19cc63cc5f
Component: engine
Currently the metrics plugin uses a really hackish host mount with
propagated mounts to get the metrics socket into a plugin after the
plugin is alreay running.
This approach ends up leaking mounts which requires setting the plugin
manager root to private, which causes some other issues.
With this change, plugin subsystems can register a set of modifiers to
apply to the plugin's runtime spec before the plugin is ever started.
This will help to generalize some of the customization work that needs
to happen for various plugin subsystems (and future ones).
Specifically it lets the metrics plugin subsystem append a mount to the
runtime spec to mount the metrics socket in the plugin's mount namespace
rather than the host's and prevetns any leaking due to this mount.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 426e610e43179d58b29c496bc79a53f410a4b1e1
Component: engine
The goal of this refactor is to make it easier to integrate buildkit
and containerd snapshotters.
Commit is used from two places (api and build), each calls it
with distinct arguments. Refactored to pull out the common commit
logic and provide different interfaces for each consumer.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: daff039049aea6e19a4bda1df2834d14b4198bc0
Component: engine
This was added in #36047 just as a way to make sure the tree is fully
unmounted on shutdown.
For ZFS this could be a breaking change since there was no unmount before.
Someone could have setup the zfs tree themselves. It would be better, if
we really do want the cleanup to actually the unpacked layers checking
for mounts rather than a blind recursive unmount of the root.
BTRFS does not use mounts and does not need to unmount anyway.
These was only an unmount to begin with because for some reason the
btrfs tree was being moutned with `private` propagation.
For the other graphdrivers that still have a recursive unmount here...
these were already being unmounted and performing the recursive unmount
shouldn't break anything. If anyone had anything mounted at the
graphdriver location it would have been unmounted on shutdown anyway.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 2fe4f888bee52b1f256d6fa5e20f9b061d30221c
Component: engine
By default, if a user requests a bind mount it uses private propagation.
When the source path is a path within the daemon root this, along with
some other propagation values that the user can use, causes issues when
the daemon tries to remove a mountpoint because a container will then
have a private reference to that mount which prevents removal.
Unmouting with MNT_DETATCH can help this scenario on newer kernels, but
ultimately this is just covering up the problem and doesn't actually
free up the underlying resources until all references are destroyed.
This change does essentially 2 things:
1. Change the default propagation when unspecified to `rslave` when the
source path is within the daemon root path or a parent of the daemon
root (because everything is using rbinds).
2. Creates a validation error on create when the user tries to specify
an unacceptable propagation mode for these paths...
basically the only two acceptable modes are `rslave` and `rshared`.
In cases where we have used the new default propagation but the
underlying filesystem is not setup to handle it (fs must hvae at least
rshared propagation) instead of erroring out like we normally would,
this falls back to the old default mode of `private`, which preserves
backwards compatibility.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 589a0afa8cbe39b6512662fd1705873e2d236dd0
Component: engine
This fix migrates docker info tests in integration-cli
to integration tests.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 68d9beedbe0994af5c91022874daa271b657be8c
Component: engine
Before this patch, when containerd is restarted (due to a crash, or
kill, whatever), the daemon would keep trying to process the event
stream against the old socket handles. This would lead to a CPU spin due
to the error handling when the client can't connect to containerd.
This change makes sure the containerd remote client is updated for all
registered libcontainerd clients.
This is not neccessarily the ideal fix which would likely require a
major refactor, but at least gets things to a working state with a
minimal patch.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 400126f8698233099259da967378c0a76bc3ea31
Component: engine
If a blank nodeID was previously passed in it resulted in a node list
request. The response would then fail to umarshal into a `Node`
type returning a JSON error.
This adds an extra validation to all inspect calls to check that the ID
that is required is provided and if not return an error.
Signed-off-by: Emil Davtyan <emil2k@gmail.com>
Upstream-commit: 3e6bbefd268f51755be5af0644995297a71a05d7
Component: engine
This fix is a follow up to 36198 by adding description
to TestContainerNetworkMountsNoChown so that it is clear
about the purpose of the test for ownership.
This fix is related to comment in 36198.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 28a2187ea7f0484ce515e7ae1662d778dcf94720
Component: engine
As there is already a runSimpleContainer, I think it makes
sense to combine runSimpleContainer with runContainer for rename test
to reduce code duplication.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 203d871658104b00099d818425b25f4cd1eff55b
Component: engine
This fix migrates TestContainersAPINetworkMountsNoChown from
integration-cli to api tests in integration.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: c028da3557cc0e9f80aee9b08118e9947e1fa57a
Component: engine
Current implementaion of docke daemon doesn't pass down the
`--oom-kill-disable` option specified by the end user to the containerd
when spawning a new docker instance with help from `runc` component, which
results in the `--oom-kill-disable` doesn't work no matter the flag is `true`
or `false`.
This PR will fix this issue reported by #36090
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Upstream-commit: 44b074d199de84b9af8cc94005fbed4f76bd9ab8
Component: engine
This fix migrates nat tests in docker_cli_nat_test.go
to api tests.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 8fb933a30b7cb2f1726c1d73001266d520cf121d
Component: engine
Dev builds for debian packages previously had system time inserted into
their packaging names while both static and rpm builds instead had the
git commit time.
This commit remedies that by pushing the generation of the debian
package version into a separate script to mirror what is being done in
static and rpm builds.
To give you an idea of what it looks like take the following examples:
```
❯ ./gen-deb-ver ~/work/docker-ce/components/engine/ $(cat ~/work/docker-ce/VERSION)
18.03.0~ce~dev~git20180202.170651.0.e232737
```
```
❯ ./gen-rpm-ver ~/work/docker-ce/components/engine $(cat ~/work/docker-ce/VERSION)
18.03.0.ce 0.0.dev.git20180202.170651.0.e232737 e232737
```
```
❯ ./gen-static-ver ~/work/docker-ce/components/engine/ $(cat ~/work/docker-ce/VERSION)
18.03.0-ce-dev-20180202.170651-e232737
```
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 23e3e42df6cb4e5cafd8648344ad37de25002971
Component: packaging
This fix migrates TestSecretInspect from integration-cli to api tests
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 71c794d9126dc4b4626cf6ab90267a990c8923f6
Component: engine
The plugin spec says that plugins can live in one of:
- /var/run/docker/plugins/<name>.sock
- /var/run/docker/plugins/<name>/<name>.sock
- /etc/docker/plugins/<name>.[json,spec]
- /etc/docker/plugins/<name>/<name>.<json,spec>
- /usr/lib/docker/plugins/<name>.<json,spec>
- /usr/lib/docker/plugins/<name>/<name>.<json,spec>
However, the plugin scanner which is used by the volume list API was
doing `filepath.Walk`, which will walk the entire tree for each of the
supported paths.
This means that even v2 plugins in
`/var/run/docker/plugins/<id>/<name>.sock` were being detected as a v1
plugin.
When the v1 plugin loader tried to load such a plugin it would log an
error that it couldn't find it because it doesn't match one of the
supported patterns... e.g. when in a subdir, the subdir name must match
the plugin name for the socket.
There is no behavior change as the error is only on the `Scan()` call,
which is passing names to the plugin registry when someone calls the
volume list API.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: b27f70d45a0fbb744c17dda02f597ffa6a47d4d9
Component: engine
Sometimes this check fails, but git status doesn't give us enough information
to debug the failure.
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
Upstream-commit: d80f25d079b84d8aa1e3233464c415e3b5d4480a
Component: engine
vendor/ must be removed first, otherwise files added to vendor/ but not vendor.conf
will not cause the validation to fail
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
Upstream-commit: a6dedd1a12
Component: cli
Previously adding files to vendor/ without adding to vendor.conf would not fail the
validation.
Also be consistent with indentation and use tabs.
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
Upstream-commit: 075fd7a9bee630f62f0fa0d7a0a26c660ed504ea
Component: engine
Attachable networks are networks created on the cluster which can then
be attached to by non-swarm containers. These networks are lazily
created on the node that wants to attach to that network.
When no container is currently attached to one of these networks on a
node, and then multiple containers which want that network are started
concurrently, this can cause a race condition in the network attachment
where essentially we try to attach the same network to the node twice.
To easily reproduce this issue you must use a multi-node cluster with a
worker node that has lots of CPUs (I used a 36 CPU node).
Repro steps:
1. On manager, `docker network create -d overlay --attachable test`
2. On worker, `docker create --restart=always --network test busybox
top`, many times... 200 is a good number (but not much more due to
subnet size restrictions)
3. Restart the daemon
When the daemon restarts, it will attempt to start all those containers
simultaneously. Note that you could try to do this yourself over the API,
but it's harder to trigger due to the added latency from going over
the API.
The error produced happens when the daemon tries to start the container
upon allocating the network resources:
```
attaching to network failed, make sure your network options are correct and check manager logs: context deadline exceeded
```
What happens here is the worker makes a network attachment request to
the manager. This is an async call which in the happy case would cause a
task to be placed on the node, which the worker is waiting for to get
the network configuration.
In the case of this race, the error ocurrs on the manager like this:
```
task allocation failure" error="failed during network allocation for task n7bwwwbymj2o2h9asqkza8gom: failed to allocate network IP for task n7bwwwbymj2o2h9asqkza8gom network rj4szie2zfauqnpgh4eri1yue: could not find an available IP" module=node node.id=u3489c490fx1df8onlyfo1v6e
```
The task is not created and the worker times out waiting for the task.
---
The mitigation for this is to make sure that only one attachment reuest
is in flight for a given network at a time *when the network doesn't
already exist on the node*. If the network already exists on the node
there is no need for synchronization because the network is already
allocated and on the node so there is no need to request it from the
manager.
This basically comes down to a race with `Find(network) ||
Create(network)` without any sort of syncronization.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: c379d2681ffe8495a888fb1d0f14973fbdbdc969
Component: engine
This fix migrates several resize tests from integration-cli to api tests.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 8f800c941570ffcd087c920c37d3a368a5a19e6d
Component: engine
Migrate TestAPIStatsContainerGetMemoryLimit from integration-cli to api tests
Upstream-commit: fd87db3769f111cdaf4ed09f0f36a254bf46fbbd
Component: engine
This fix migrates TestAPIStatsContainerGetMemoryLimit from
integration-cli to api test.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: d5cbde514f4887f5655096bce05faa7b91068038
Component: engine
This fix migrates TestAPIUpdateContainer from integration-cli to api tests
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 490edd35829a7dd1144da50105fc377d41a52fc0
Component: engine
This fix tries to address the issue raised in 36139 where
ExitCode and PID does not show up in Task.Status.ContainerStatus
The issue was caused by `json:",omitempty"` in PID and ExitCode
which interprate 0 as null.
This is confusion as ExitCode 0 does have a meaning.
This fix removes `json:",omitempty"` in ExitCode and PID,
but changes ContainerStatus to pointer so that ContainerStatus
does not show up at all if no content. If ContainerStatus
does have a content, then ExitCode and PID will show up (even if
they are 0).
This fix fixes 36139.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 9247e09944a4c7f3c2f3f20f180c047a19fb6bae
Component: engine
ReleaseRWLayer can and should only be called once (unless it returns
an error), but might be called twice in case of a failure from
`system.EnsureRemoveAll(container.Root)`. This results in the
following error:
> Error response from daemon: driver "XXX" failed to remove root filesystem for YYY: layer not retained
The obvious fix is to set container.RWLayer to nil as soon as
ReleaseRWLayer() succeeds.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: e9b9e4ace294230c6b8eb010eda564a2541c4564
Component: engine
Moves builder/shell_parser and into its own subpackage at builder/shell since it
has no dependencies other than the standard library. This will make it
much easier to vendor for downstream libraries, without pulling all the
dependencies of builder/.
Fixes#36154
Signed-off-by: Matt Rickard <mrick@google.com>
Upstream-commit: a634526d14639c6b98c509a069ef29e2b69c0ef0
Component: engine
This fix migrate TestLinksEtcHostsContentMatch
to api test.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: e6bd20edcbf3b7a6f87a356ab0943714936c70e1
Component: engine
This fix tries to address the issue raised in 36142 where
there are discrepancies between Swarm API and swagger.yaml.
This fix adds two recently added state `REMOVE` and `ORPHANED` to TaskState.
This fix fixes 36142.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: a40687f5ac7df27bc6c6c3a6f69513a397a1a05a
Component: engine
This fix migrate docker_cli_stop_test.go to api test
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 4f378124ff649b844de88c93f6ca70e6b3f5d7d7
Component: engine
`--label-file` has the exact same behavior as `--env-file`, meaning any
placeholder (i.e. a simple key, no `=` sign, no value), it will get the
value from the environment variable.
For `--label-file` it should just add an empty label.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 2b17f4c8a8
Component: cli
This fix tries to address the issue raised in 36042
where secret and config are not configured with the
specified file mode.
This fix update the file mode so that it is not impacted
with umask.
Additional tests have been added.
This fix fixes 36042.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 3305221eefd18ba7712a308c1fb05d4eeeac2cc6
Component: engine
Fix issue where network inspect does not show Created time for networks in swarm scope
Upstream-commit: 924fb0e843930ca444e0f3a6632d7cb67a3da479
Component: engine
Daemon flags that can be specified multiple times use
singlar names for flags, but plural names for the configuration
file.
To make the daemon configuration know how to correlate
the flag with the corresponding configuration option,
`opt.NewNamedListOptsRef()` should be used instead of
`opt.NewListOptsRef()`.
Commit 6702ac590e6148cb3f606388dde93a011cb14931 attempted
to fix the daemon not corresponding the flag with the configuration
file option, but did so by changing the name of the flag
to plural.
This patch reverts that change, and uses `opt.NewNamedListOptsRef()`
instead.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 6e7715d65ba892a47d355e16bf9ad87fb537a2d0
Component: engine
You don't need to resolve the symlink for the exec as long as the
process is to keep running during execution.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Upstream-commit: 59ec65cd8cec942cee6cbf2b8327ec57eb5078f0
Component: engine
Signed-off-by: Allen Sun <shlallen1990@gmail.com>
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: de68ac8393d32d2c2028dd11c5816430ad0d8d8b
Component: engine
During a plugin remove, docker performs an `os.Rename` to move the
plugin data dir to a new location before removing to acheive an atomic
removal.
`os.Rename` can return either a `NotExist` error if the source path
doesn't exist, or an `Exist` error if the target path already exists.
Both these cases can happen when there is an error on the final
`os.Remove` call, which is common on older kernels (`device or resource
busy`).
When calling rename, we can safely ignore these error types and proceed
to try and remove the plugin.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 93027b1ff2475b66b6321b5722009fad4def8187
Component: engine
A small golint fix with ro_layer.go.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 19918b88b7b9fe6fc668847f31c94068c78ef2d1
Component: engine
Adds git date plus git commit sha to static builds if the version being
built for is a development version
Output is similar to: `docker-18.02.0-ce-dev-20180120.170357-fa4fb35.tgz`
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 8a86c3a6a54760b19cd67854bf68e408bd2a17fb
Component: packaging
This change sets an explicit mount propagation for the daemon root.
This is useful for people who need to bind mount the docker daemon root
into a container.
Since bind mounting the daemon root should only ever happen with at
least `rlsave` propagation (to prevent the container from holding
references to mounts making it impossible for the daemon to clean up its
resources), we should make sure the user is actually able to this.
Most modern systems have shared root (`/`) propagation by default
already, however there are some cases where this may not be so
(e.g. potentially docker-in-docker scenarios, but also other cases).
So this just gives the daemon a little more control here and provides
a more uniform experience across different systems.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: a510192b86e7eb1e1112f3f625d80687fdec6578
Component: engine
This fix tries to address the issue raised in 36083 where
`network inspect` does not show Created time if the network is
created in swarm scope.
The issue was that Created was not converted from swarm api.
This fix addresses the issue.
An unit test has been added.
This fix fixes 36083.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 090c439fb8a863731cc80fcb9932ce5958d8166d
Component: engine
Since systemd version 228, a new setting, `TasksMax`, has appeared,
which limits the number of tasks used by a service (via pids cgroup
controller). Unfortunately, a default for this setting, `DefaultTaskMax`,
is set to 512. In systemd version 231 it is changed to 15% which
practically is 4195, as the value from /proc/sys/kernel/pid_max is
treated like 100%).
Either 512 or 4195 is severily limited value for Docker Engine,
as it can run thousands of containers with thousands of tasks in each,
and the number of tasks limit should be set on a per-container basis
by the Docker user. So, the most reasonable setting for `TasksMax`
is `unlimited`.
Unfortunately, older versions of systemd warn about unknown `TasksMax`
parameter in `docker.service` file, and the warning is rather annoying,
therefore this setting is commented out by default, and is supposed
to be uncommented by the user.
The problem with that is, once the limit is hit, all sorts of bad things
happen and it's not really clear even to an advanced user that this
setting is the source of issues.
Now, `rules` file already contain a hack to check for the systemd
version (during build time) and in case the version is greater than 227,
uncomment the `TasksMax=unlimited` line. Alas, it does not work
during normal builds, the reason being systemd is not installed
into build environments.
An obvious fix would be to add systemd to the list of installed
packages in all Dockerfiles used to build debs. Fortunately,
there is a simpler way, as libsystemd-dev is installed, and
it's a subpackage of systemd built from the same source and
carrying the same version, so it can also be checked.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: d80738e4b4459816c64757a2a63e5d8058d0ccf4
Component: packaging
Since systemd version 228, a new setting, `TasksMax`, has appeared, which
limits the number of tasks used by a service (via pids cgroup
controller). Unfortunately, a default for this setting, `DefaultTaskMax`,
is set to 512. In systemd version 231 it is changed to 15% which
practically is 4195, as the value from /proc/sys/kernel/pid_max is
treated like 100%).
Either 512 or 4195 is severily limited value for Docker Engine, as it
can run thousands of containers with thousands of tasks in each, and
the number of tasks limit should be set on a per-container basis by the
Docker user. So, the most reasonable setting for `TasksMax` is `unlimited`.
Unfortunately, older versions of systemd warn about unknown `TasksMax`
parameter in `docker.service` file, and the warning is rather annoying,
therefore this setting is commented out by default, and is supposed to
be uncommented by the user.
The problem with that is, once the limit is hit, all sorts of bad things
happen and it's not really clear even to an advanced user that this
setting is the source of issues.
As Fedora 25 ships systemd 231, it (and later Fedora releases) support
TasksMax, so it makes total sense to uncomment the setting, this is what
this commit does.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 9055832bb0725f05d518c3ebc9b7cc93a69420c7
Component: packaging
This fix tries to address the issue raised in 33661 where
network alias does not work when connect to a network the second time.
This fix address the issue.
This fix fixes 33661.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: d63a5a1ff593f14957f3e0a9678633e8237defc9
Component: engine
-dev was being removed, per legacy code, but we'd like to add it back
into the naming so that deb / rpm packages will look mostly the same
when compiled with a `-dev` version.
RPMS end up looking like:
docker-ce-18.02.0.ce-0.0.dev.git20180120.170357.0.fa4fb35.el7.centos.x86_64.rpm
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 65e2c3cf9ff8c3130e628a455e3d02921876cdfc
Component: packaging
This fix tries to address the issue raised in 35752
where container start will trigger a crash if EndpointSettings is nil.
This fix adds the validation to make sure EndpointSettings != nil
This fix fixes 35752.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 8d2f4cb24129d87674a13319ca48ce8636ee527a
Component: engine
When succesfully reloading the daemon configuration, print a message
in the logs with the active configuration:
INFO[2018-01-15T15:36:20.901688317Z] Got signal to reload configuration, reloading from: /etc/docker/daemon.json
INFO[2018-01-14T02:23:48.782769942Z] Reloaded configuration: {"mtu":1500,"pidfile":"/var/run/docker.pid","data-root":"/var/lib/docker","exec-root":"/var/run/docker","group":"docker","deprecated-key-path":"/etc/docker/key.json","max-concurrent-downloads":3,"max-concurrent-uploads":5,"shutdown-timeout":15,"debug":true,"hosts":["unix:///var/run/docker.sock"],"log-level":"info","swarm-default-advertise-addr":"","metrics-addr":"","log-driver":"json-file","ip":"0.0.0.0","icc":true,"iptables":true,"ip-forward":true,"ip-masq":true,"userland-proxy":true,"disable-legacy-registry":true,"experimental":false,"network-control-plane-mtu":1500,"runtimes":{"runc":{"path":"docker-runc"}},"default-runtime":"runc","oom-score-adjust":-500,"default-shm-size":67108864,"default-ipc-mode":"shareable"}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8378dcf46d017c70df97d6f851e0196b113b422e
Component: engine
This fix updates aws-sdk-go and go-ini to recent versions.
The aws-sdk-go used to be `v1.4.22` which was more than a
year old, and go-ini used to be pre-1.0 release.
This fix updates aws-sdk-go to v1.12.66 and go-ini to v1.25.4:
```
github.com/aws/aws-sdk-go v1.12.66
github.com/go-ini/ini v1.25.4
```
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 79bedc4f46e7fe5870f7d53f27f05d05d0821697
Component: engine
This fix carries PR 34248: Added tag log option to json-logger
This fix changes to use RawAttrs based on review feedback.
This fix fixes 19803, this fix closes 34248.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: e77267c5a682e2c5aaa32469f2c83c2479d57566
Component: engine
Fixes#19803
Updated the json-logger to utilize the common log option
'tag' that can define container/image information to include
as part of logging.
When the 'tag' log option is not included, there is no change
to the log content via the json-logger. When the 'tag' log option
is included, the tag will be parsed as a template and the result
will be stored within each log entry as the attribute 'tag'.
Update: Removing test added to integration_cli as those have been deprecated.
Update: Using proper test calls (require and assert) in jsonfilelog_test.go based on review.
Update: Added new unit test configs for logs with tag. Updated unit test error checking.
Update: Cleanup check in jsonlogbytes_test.go to match pending changes in PR #34946.
Update: Merging to correct conflicts from PR #34946.
Signed-off-by: bonczj <josh.bonczkowski@gmail.com>
Upstream-commit: 5f50f4f511cd84e79bf005817af346b1764df27f
Component: engine
This change is in response to https://github.com/moby/moby/pull/35697
It adds pigz to the recommended binaries that should be installed with
docker-ce.
Signed-off-by: Sargun Dhillon <sargun@sargun.me>
Upstream-commit: 1ca014b9440a92b46e2e03d879c03ddc5c51d4c0
Component: packaging
Signed-off-by: John Howard <jhoward@microsoft.com>
The re-coalesces the daemon stores which were split as part of the
original LCOW implementation.
This is part of the work discussed in https://github.com/moby/moby/issues/34617,
in particular see the document linked to in that issue.
Upstream-commit: ce8e529e182bde057cdfafded62c210b7293b8ba
Component: engine
The idea behind making the graphdrivers private is to prevent leaking
mounts into other namespaces.
Unfortunately this is not really what happens.
There is one case where this does work, and that is when the namespace
was created before the daemon's namespace.
However with systemd each system servie winds up with it's own mount
namespace. This causes a race betwen daemon startup and other system
services as to if the mount is actually private.
This also means there is a negative impact when other system services
are started while the daemon is running.
Basically there are too many things that the daemon does not have
control over (nor should it) to be able to protect against these kinds
of leakages. One thing is certain, setting the graphdriver roots to
private disconnects the mount ns heirarchy preventing propagation of
unmounts... new mounts are of course not propagated either, but the
behavior is racey (or just bad in the case of restarting services)... so
it's better to just be able to keep mount propagation in tact.
It also does not protect situations like `-v
/var/lib/docker:/var/lib/docker` where all mounts are recursively bound
into the container anyway.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 9803272f2db84df7955b16c0d847ad72cdc494d1
Component: engine
Libcontainer no longer provides placeholders for
unsupported platforms, which cause the Windows
builds to fail.
This patch moves features that are not supported
to platform-specific files.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: d1c34831e930c1f6b3de28cab3f4a358845a79d5
Component: engine
matching the version that's used by containerd 1.0.1
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f58aa31075bf74ab8d2369dafb591ae43ed36ee6
Component: engine
This PR contains a fix for moby/moby#30321. There was a moby/moby#31142
PR intending to fix the issue by adding a delay between disabling the
service in the cluster and the shutdown of the tasks. However
disabling the service was not deleting the service info in the cluster.
Added a fix to delete service info from cluster and verified using siege
to ensure there is zero downtime on rolling update of a service.In order
to support it and ensure consitency of enabling and disable service knob
from the daemon, we need to ensure we disable service when we release
the network from the container. This helps in making the enable and
disable service less racy. The corresponding part of libnetwork fix is
part of docker/libnetwork#1824
Signed-off-by: abhi <abhi@docker.com>
Upstream-commit: a042e5a20a7801efc936daf7a639487bb37ca966
Component: engine
This protects the daemon from volume plugins that are slow or
deadlocked.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: b15f8d2d4f054a87052a7065c50441f7e8479fa9
Component: engine
A linter (vet) found the following bug in the code:
> daemon/metrics.go:124::error: range variable p captured by func literal (vet)
Here a variable p is used in an async fashion by goroutine, and most
probably by the time of use it is set to the last element of a range.
For example, the following code
```go
for _, c := range []string{"here ", "we ", "go"} {
go func() {
fmt.Print(c)
}()
}
```
will print `gogogo` rather than `here we go` as one would expect.
Fixes: 0e8e8f0f31 ("Add support for metrics plugins")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 9db2c62488734a44a4f1bb9a0252c520b787acfe
Component: engine
pickup changes which use t.Helper()
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 4ac4b690f78a645cc50030b81077fd5319b53501
Component: engine
1. Add = between the option and the argument, otherwise the argument
appears to be passed on to the linters directly, as in:
> DEBUG: [golint.8]: executing /home/kir/go/bin/golint
> -min_confidence 0.800000 ./10m ./api ./api/errdefs <...>
2. Fix setting the default for GOMETALINTER_OPTS -- the default
was -deadline (rather than --deadline).
Fixes: b96093fa56a9 ("gometalinter: add per-platform configurable options")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: c11508a5f405084da13c35ee7ab62f1670e4da39
Component: engine
Because we merge master into the branch before running tests, so the
actual git sha does not exist on any git remote.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: b2faf24925ea0f820db62e1f3d96ee347972f12a
Component: engine
It's a common scenario for admins and/or monitoring applications to
mount in the daemon root dir into a container. When doing so all mounts
get coppied into the container, often with private references.
This can prevent removal of a container due to the various mounts that
must be configured before a container is started (for example, for
shared /dev/shm, or secrets) being leaked into another namespace,
usually with private references.
This is particularly problematic on older kernels (e.g. RHEL < 7.4)
where a mount may be active in another namespace and attempting to
remove a mountpoint which is active in another namespace fails.
This change moves all container resource mounts into a common directory
so that the directory can be made unbindable.
What this does is prevents sub-mounts of this new directory from leaking
into other namespaces when mounted with `rbind`... which is how all
binds are handled for containers.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: eaa5192856c1ad09614318e88030554b96bb6e81
Component: engine
The `POST /volumes/create` expects a request body to be provided.
If no body was provided, a 500 status was returned. A 500 status
is incorrect, because the request is invalid (it's not a server
error).
Before this change:
$ curl --unix-socket /var/run/docker.sock -v -X POST http://localhost/volumes/create
* Trying /var/run/docker.sock...
* Connected to localhost (/Users/sebastiaan/Library/Containers/com.dock) port 80 (#0)
> POST /volumes/create HTTP/1.1
> Host: localhost
> User-Agent: curl/7.51.0
> Accept: */*
>
< HTTP/1.1 500 Internal Server Error
< Api-Version: 1.30
< Content-Length: 18
< Content-Type: application/json
< Date: Wed, 19 Jul 2017 11:29:26 GMT
< Docker-Experimental: true
< Ostype: linux
< Server: Docker/17.06.0-ce (linux)
<
{"message":"EOF"}
* Curl_http_done: called premature == 0
* Connection #0 to host localhost left intact
After this change:
$ curl --unix-socket /var/run/docker.sock -v -X POST http://localhost/volumes/create
* Trying /var/run/docker.sock...
* Connected to localhost (/var/run/docker.sock) port 80 (#0)
> POST /volumes/create HTTP/1.1
> Host: localhost
> User-Agent: curl/7.52.1
> Accept: */*
>
< HTTP/1.1 400 Bad Request
< Api-Version: 1.36
< Content-Type: application/json
< Docker-Experimental: false
< Ostype: linux
< Server: Docker/dev (linux)
< Date: Tue, 09 Jan 2018 15:00:13 GMT
< Content-Length: 42
<
{"message":"no body provided in request"}
* Curl_http_done: called premature == 0
* Connection #0 to host localhost left intact
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5ad1e4be6bd2fd1033bc2adb5be2fe821787b59c
Component: engine
The Golang built-in gzip library is serialized, and fairly slow
at decompressing. It also only decompresses on demand, versus
pipelining decompression.
This change switches to using the pigz external command
for gzip decompression, as opposed to using the built-in
golang one. This code is not vendored, but will be used
if it autodetected as part of the OS.
This also switches to using context, versus a manually
managed channel to manage cancellations, and synchronization.
There is a little bit of weirdness around manually having
to cancel in the error cases.
Signed-off-by: Sargun Dhillon <sargun@sargun.me>
Upstream-commit: fd35494a251a497c359f706f61f33e689e2af678
Component: engine
Fixes an issue where if cpu quota/period is sent via the update API, the
values are updated in the stored container data but not actually sent to
the running container.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 86ba63db82e87c943c9dee52559118e46f23defc
Component: engine
PR #36011 fixed almost all of the golint issues though
there is still one golint error:
https://goreportcard.com/report/github.com/docker/docker#golint
```
Golint is a linter for Go source code.
docker/daemon/reload.go
Line 64: warning: redundant if ...; err != nil check, just return error instead. (golint)
```
This fix fixes the last one.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: e02a3d9f5ba3bd9fa7f21596a6ee784bb58053f9
Component: engine
This fix is a follow up to 30397, with `FindUniqueNetwork`
changed to `FindNetwork` based on the review feedback.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: ccc2ed01894a1950eaf47db2ad0860ad87cd78d1
Component: engine
Some improvements to the test;
- Combine tests to reduce duplicated code
- Add test-cases for empty version in request using the default version
- Add test for valid versions in request actually setting the version
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 63906d8fae8164c8ef00e1edc957a90b22908bcf
Component: engine
Both lcow_parser.go and linux_parser.go are duplicating the error:
"invalid specification: destination can't be '/'"
This commit creates a new error called "ErrVolumeTargetIsRoot"
that is used by both linux_parser and lcow_parser and remove
the duplication in the code.
Signed-off-by: Boaz Shuster <ripcurld.github@gmail.com>
Upstream-commit: 62143af5437a29d4b95f971d1905cfef763b0847
Component: engine
The pattern `echo str | grep -qE pattern` likes to fail on the z CI here for
an unknown reason. Use `grep -qE pattern <<< str` instead.
Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
Upstream-commit: 24da8a0ed415c019179ca2c4f7496cbdffb7e4d0
Component: engine
Now we only adjust the timeout value for `arm` while not `arm64`,
actually the avarage duration for this test is about 25s to crate
multiple services on arm64, else the integration test will terminate
with below error:
> --- FAIL: TestCreateServiceMultipleTimes (24.11s)
> daemon.go:285: [ddc3c7c1476c2] waiting for daemon to start
> daemon.go:317: [ddc3c7c1476c2] daemon started
> poll.go:121: timeout hit after 10s: task count at 4 waiting for 0
> daemon.go:275: [ddc3c7c1476c2] exiting daemon
> clean.go:108: Removing image sha256:e6a8d12d58602a19277ee5632b7ff9fa56a4ea52ba00eedf1d3f6f5a495fe761
> clean.go:108: Removing image sha256:876244cc2ecb8fe1b0b2e817e3b78709a2a735edb093bc6849f99aa6c18f3a01
This PR adjusts the timeout value for both `arm64` and `arm` to mitigate
this issue on those 2 platforms.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 4542016cbe985d2af60c25f6a5b24df50bb50aba
Component: engine
Instead of having to create a bunch of custom error types that are doing
nothing but wrapping another error in sub-packages, use a common helper
to create errors of the requested type.
e.g. instead of re-implementing this over and over:
```go
type notFoundError struct {
cause error
}
func(e notFoundError) Error() string {
return e.cause.Error()
}
func(e notFoundError) NotFound() {}
func(e notFoundError) Cause() error {
return e.cause
}
```
Packages can instead just do:
```
errdefs.NotFound(err)
```
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 87a12421a94faac294079bebc97c8abb4180dde5
Component: engine
When a recursive unmount fails, don't bother parsing the mount table to check
if what we expected to be a mountpoint is still mounted. `EINVAL` is
returned when you try to unmount something that is not a mountpoint, the
other cases of `EINVAL` would not apply here unless everything is just
wrong. Parsing the mount table over and over is relatively expensive,
especially in the code path that it's in.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: dd2108766017c13a19bdbfd1a56cd1358580e0bb
Component: engine
Prefer "strict" values for orchestrator, as it's
easier to add aliases (if we think it's needed) than
to remove them later.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a812995f98
Component: cli
Due to a copy/paste error, commands annotated with "swarm"
were incorrectly setting the "kubernetes" property.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 6be06a3db2
Component: cli
Using parallel tests is nice, however it can cause an issue with
multiple daemons trying to make changes to iptables at the same time
which causes flakey tests.
This just disables iptables for the set of tests since it is not
required.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 9e3193810da91d81f6b2dba3171443557f756794
Component: engine
This allows IsErrNotFound and IsErrNotImplemented to work as intended.
Signed-off-by: Emil Davtyan <emil2k@gmail.com>
Upstream-commit: 44369bdd6507006108f39b85b5950f933bf1380a
Component: engine
Use `cat -v` command instead of `catv` for the latest version of
busybox(V1.28.0) with multi-arch
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: ec6659a1216fbfe3fead759b0220501847d12e28
Component: engine
BusyBox v1.26.2 (2017-03-09 00:04:38 UTC) supports `-le` option
to get the full date and time information, while BusyBox v1.27.2
(2017-11-01 23:22:25 UTC, which is used by the official multi-arch
image, uses `--full-time` instead of `-e` to get the same data. As
a result, we will get below error for the `DockerSuite.TestBuildLastModified`
test case in case of multi-arch image used:
> docker_cli_build_test.go:446:
> out2 = cli.DockerCmd(c, "run", name, "ls", "-le", "/file").Combined()
> o/src/github.com/docker/docker/vendor/github.com/gotestyourself/gotestyourself/icmd/command.go:61:
> t.Fatalf("at %s:%d - %s\n", filepath.Base(file), line, err.Error())
> ... Error: at cli.go:33 -
> Command: /usr/local/bin/docker run testbuildlastmodified ls -le /file
> ExitCode: 1
> Error: exit status 1
> Stdout:
> Stderr: ls: invalid option -- e
> BusyBox v1.27.2 (2017-11-01 23:22:25 UTC) multi-call binary.
This PR tries to fix the above compatible issue for busybox image.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: f88c2c04ef4dbae20eaddeaf69e605878f498041
Component: engine
We don't need the test image namespace anymore since we've already
upgrade those images to the latest multi-arch ones.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 662bdb4a5638e56d78e66bb2cb5d7a4a751135c9
Component: engine
Upgrade the frozen images to the multi-arch ones.
Since issue #35963 is not fixed yet on linux/amd64, so we keep the busybox
image on amd64 untouched.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: eaae7750efbc80314c5e028c4d43d3cd9e104edd
Component: engine
This fix adds an integration test for `before` and `since` filter for `docker ps`
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 52b44b98161872f704f7e3eea16e0f3177ca4e42
Component: engine
When adding a network using `docker service update --network-add`,
the new network was added by _name_.
Existing entries in a service spec are listed by network ID, which
resulted in the CLI not detecting duplicate entries for the same
network.
This patch changes the behavior to always use the network-ID,
so that duplicate entries are correctly caught.
Before this change;
$ docker network create -d overlay foo
$ docker service create --name=test --network=foo nginx:alpine
$ docker service update --network-add foo test
$ docker service inspect --format '{{ json .Spec.TaskTemplate.Networks}}' test
[
{
"Target": "9ot0ieagg5xv1gxd85m7y33eq"
},
{
"Target": "9ot0ieagg5xv1gxd85m7y33eq"
}
]
After this change:
$ docker network create -d overlay foo
$ docker service create --name=test --network=foo nginx:alpine
$ docker service update --network-add foo test
service is already attached to network foo
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e6ebaf55dd
Component: cli
Make the behvious of cleaning up DirectIO more obvious
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 9d20d5eb3fd744088e700292b15d56de29a3361d
Component: engine
This fix updates doc for CopyFromContainer to explicitly
mention that the content received from the reader is
a TAR archive.
This fix closes 35965.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: e330e7a5ce77736c76c36ba77983accfae01c405
Component: engine
Enable inspection (aka "shallow pull") of images' manifest info, and
also the creation of manifest lists (aka "fat manifests").
The workflow for creating a manifest list will be:
`docker manifest create new-list-ref-name image-ref [image-ref...]`
`docker manifest annotate new-list-ref-name image-ref --os linux --arch
arm`
`docker manifest push new-list-ref-name`
The annotate step is optional. Most architectures are fine by default.
There is also a `manifest inspect` command to allow for a "shallow pull"
of an image's manifest: `docker manifest inspect
manifest-or-manifest_list`.
To be more in line with the existing external manifest tool, there is
also a `-v` option for inspect that will show information depending on
what the reference maps to (list or single manifest).
Signed-off-by: Christy Perez <christy@linux.vnet.ibm.com>
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 02719bdbb5
Component: cli
Actually the specification was expecting a 'additionalProperties' for the Volumes data, where in fact it's expecting
a map of string pointing to empty object.
Signed-off-by: Joel Wurtz <joel.wurtz@gmail.com>
Upstream-commit: dc883c0486b398eb5ad99f35aef3ff02a5a7dd29
Component: engine
This fix is part of the effort to address 30242 where
issue arise because of the fact that multiple networks
may share the same name (within or across local/swarm scopes).
The focus of this fix is to allow creation of service
when a network in local scope has the same name as the
service network.
An integration test has been added.
This fix fixes 30242.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: cafed80cd019a8b40025eaa5e5b37459362607fb
Component: engine
This fix tries to address the issue raised in 35931 where
`before` and `since` filter for `docker ps` does not work
and returns an error
```
Error response from daemon: no such container <container_name>
```
The issue was that `before` and `since` filter are matched
with `view.Get()` which does not take into considerations
of name match.
This fix fixes the issue by adding additional logic for name
match.
This fix fixes 35931.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 9833332dba5cba3709c5d78c28d3dbc52e49bfa9
Component: engine
This fix tries to address the issue raised in 35920 where the filter
of `docker ps` with `health=starting` always returns nothing.
The issue was that in container view, the human readable string (`HealthString()` => `Health.String()`)
of health status was used. In case of starting it is `"health: starting"`.
However, the filter still uses `starting` so no match returned.
This fix fixes the issue by using `container.Health.Status()` instead so that it matches
the string (`starting`) passed by filter.
This fix fixes 35920.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 97b16aecf9275f4103c2737b79d0c5e81583aa58
Component: engine
This code kept being modified by my IDE, so fixing
the formatting.
This is auto-generated code, but the code to generate
is not in this repository, so this is a temporary fix
until the code-generation upstream is fixed :)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: bb01064691
Component: cli
This fix adds a test case for 35333: Devicemapper: ignore Nodata errors when delete thin device
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 7c6ef28042c20fdad23cd461ab49b9cfa0c757df
Component: engine
- More strict on orchestrator flag
- Make orchestrator flag more explicit as experimental
- Add experimentalCLI annotation on kubernetes flags
- Better kubeconfig error message
- Prefix service name with stackname in ps and services stack subcommands
- Fix yaml documentation
- Fix code coverage ignoring generated code
Signed-off-by: Silvin Lubecki <silvin.lubecki@docker.com>
Upstream-commit: f1b116179f
Component: cli
The "-f" flag is an alias for --force, not --filter (as correctly stated at the top of each documents). The system_prune.md didn't have this error.
Signed-off-by: Mateusz Major <apkd@users.noreply.github.com>
Upstream-commit: 8b07518458
Component: cli
if thin device is deteled and the metadata exists, you can not
delete related containers. This patch ignore Nodata errors for
thin device deletion
Signed-off-by: Liu Hua <sdu.liu@huawei.com>
Upstream-commit: 8451d03d8ef7457f82112179cd3e300c05a08d3d
Component: engine
The event filter used two separate filter-conditions for
"namespace" and "topic". As a result, both events matching
"topic" and events matching "namespace" were subscribed to,
causing events to be handled both by the "plugin" client, and
"container" client.
This patch rewrites the filter to match only if both namespace
and topic match.
Thanks to Stephen Day for providing the correct filter :)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 295bb09184fe473933498bb0efb59b8acb124f55
Component: engine
Further to 355cf9483c1b8ede5ae3ed50add4de2a69d62645 which caught some
of these. This should fix the remainder in the contributing docs.
Signed-off-by: Brett Randall <javabrett@gmail.com>
Upstream-commit: e96b33665e58f73a16923b89a9bcc6fe6fcdb6c6
Component: engine
According to https://github.com/golang/go/issues/5373, go recognizes
(and optimizes for) the following syntax:
```go
for i := range b {
b[i] = 0
}
```
so let's use it. Limited testing shows ~7.5x speed increase,
compared to the previously used syntax.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: f0cab0e28512de5eecc0412212425cc74d62af71
Component: engine
I got the following test failure on power:
10:00:56
10:00:56
----------------------------------------------------------------------
10:00:56 FAIL: docker_cli_build_test.go:3521:
DockerSuite.TestBuildNotVerboseFailureRemote
10:00:56
10:00:56 docker_cli_build_test.go:3536:
10:00:56 c.Fatal(fmt.Errorf("Test[%s] expected that quiet stderr and
verbose stdout are equal; quiet [%v], verbose [%v]", name,
quietResult.Stderr(), result.Combined()))
10:00:56 ... Error: Test[quiet_build_wrong_remote] expected that quiet
stderr and verbose stdout are equal; quiet [
10:00:56 unable to prepare context: unable to download remote context
http://something.invalid: Get http://something.invalid: dial tcp: lookup
something.invalid on 172.29.128.11:53: no such host
10:00:56 ], verbose [unable to prepare context: unable to download
remote context http://something.invalid: Get http://something.invalid:
dial tcp: lookup something.invalid on 8.8.8.8:53: no such host
10:00:56 ]
10:00:56
10:00:56
10:00:56
----------------------------------------------------------------------
The reason is, either more than one name server is configured, or
nameserver was reconfigured in the middle of the test run. In any case,
different nameserver IP in an error messages should not be treated
as a failure, so let's strip those out.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 3676bd8569f4df28a4f850cd4814e3558d8c03f6
Component: engine
`TestCleanupMountsAfterDaemonAndContainerKill` was supposedly written
when the container mounts were visible from the host. Currently they
all live in their own mount namespace and the only visible mount is
the tmpfs one for shareable /dev/shm inside the container (i.e.
/var/lib/docker/containers/<ID>/shm), which will no longer be there
in case of `--default-ipc-mode private` is used, and so the test will
fail. Add a check if any container mounts are visible from the host,
and skip the test if there are none, as there's nothing to check.
`TestCleanupMountsAfterDaemonCrash`: fix in a similar way, keeping
all the other checks it does, and skipping the "mounts gone" check
if there were no mounts visible from the host.
While at it, also fix the tests to use `d.Kill()` in order to not
leave behind a stale `docker.pid` files.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: f5e01452d2c2a07bab48b4e05306ef9446770c4a
Component: engine
1. The functionality of this test is superceded by
`TestAPIIpcModeShareableAndContainer` (see
integration-cli/docker_api_ipcmode_test.go).
2. This test won't work with --default-ipc-mode private.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 519c06607ca7e8a544afddbd61ad57afe63a98b4
Component: engine
Was getting annoying to have to write it into one spot so moved them to
above the targets that they specify.
Signed-off-by: Eli Uriegas <seemethere101@gmail.com>
Upstream-commit: a8e527df5d6579cc53372c26ef86fdcf449aa9ae
Component: packaging
* Refactor tests on version and kubernetes switch
* Fix rebase errors
* Refactor for gocyclo linter
Signed-off-by: Silvin Lubecki <silvin.lubecki@docker.com>
Upstream-commit: ad409767bf
Component: cli
Services do not support custom "pid"-modes (e.g. `--pid=host`), but this
option was ignored silently when deploying a stack.
This patch adds `pid` to the list of unsupported options so that a warning
is printed;
With this patch applied:
$ docker stack deploy -c docker-compose.yml foobar
Ignoring unsupported options: pid
Creating network foobar_default
Creating service foobar_test
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 70a29b492d
Component: cli
- Define command and subcommands only once
- Use annotations for k8s or swarm specific flags or subcommands
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: dedd0db51a
Component: cli
- Add support for kubernetes for docker stack command
- Update to go 1.9
- Add kubernetes to vendors
- Print orchestrator in docker version command
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Signed-off-by: Silvin Lubecki <silvin.lubecki@docker.com>
Upstream-commit: 8417e49792
Component: cli
Allow to mark some commands and flags experimental on cli (i.e. not
depending to the state of the daemon). This will allow more flexibility
on experimentation with the cli.
Marking `docker trust` as cli experimental as it is documented so.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 84fe1a1b5b
Component: cli
The description for capabilities are mismatched for MAC_ADMIN and MAC_OVERRIDE.
Signed-off-by: T K Sourabh <sourabhtk37@gmail.com>
Upstream-commit: afcc78aae3
Component: cli
The building machinery was being handed an uninitialized container
Config. This changes it to use the target container's Config.
Resolves#30538
Signed-off-by: Anthony Sottile <asottile@umich.edu>
Upstream-commit: 0785836c4b440a8d4a5dfdb8df82e50f9f4d23a1
Component: engine
The `Status` field was deprecated in favor of `Action`.
This patch updates the test to use the `Action` field,
but adds a check that both are set to the same value.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b7d204ef6b1b2d6a3bafb42f844cdc146976e68f
Component: engine
Commit 59d45c384a2de7bca73296ce1471646db14cb0c8 changed
the `eventsLimit` from 64 to 256, but did not update
the GoDoc accordingly.
This patch updates the GoDoc for `Subscribe` and `SubscribeTopic`
to match the actual limit.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: fb3935022dbc160fc1531fa43f0ca2db69184800
Component: engine
This test was added a long time ago, and over the years has proven to be flaky,
and slow. To address those issues, it was modified to;
- cleanup containers afterwards
- take clock-skew into account
- improve performance by parallelizing the container runs
- _reduce_ parallelization to address platform issues on Windows (twice..)
- adjust the test to take new limits into account
- adjust the test to account for more events being generated by containers
The last change to this test (made in ddae20c032058a0fd42c34c2e9750ee8f62) actually
broke the test, as it's now testing that all events sent by containers
(`numContainers*eventPerContainer`) are received, but the number of events that
is generated (17 containers * 7 events = 119) is less than the limit (256 events).
The limit is already covered by the `TestLogEvents` unit-test, that was added in
8d056423f8c433927089bd7eb6bc97abbc1ed502, and tests that the number of events
is limited to `eventsLimit`.
This patch removes the test, because it's not needed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b7ad3e7ea10e285226a0a5b1665e8205b3264128
Component: engine
When compiling containerd binaries statically, linker rightfully
complains:
+ make BUILDTAGS=static_build 'EXTRA_FLAGS=-buildmode pie' 'EXTRA_LDFLAGS=-extldflags "-fno-PIC -static"'
🇩 bin/ctr
# github.com/containerd/containerd/cmd/ctr
/tmp/go-link-343047789/000000.o: In function `_cgo_b0c710f30cfd_C2func_getaddrinfo':
/tmp/go-build/net/_obj/cgo-gcc-prolog:46: warning: Using 'getaddrinfo'
in statically linked applications requires at runtime the shared
libraries from the glibc version used for linking
The same error appears for ctr, containerd, and containerd-stress
binaries.
The fix is to use Go's own DNS resolver functions, rather than
glibc's getaddrinfo() -- this option is turned on by `netgo` build tag.
See https://golang.org/pkg/net/ (look for "Name Resolution") for more
details.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 7368ef96c89fd4f6879addf5214c4a09889a05be
Component: engine
This is a fix to regression in vfs graph driver introduced by
commit 7a1618ced359a3ac92 ("add quota support to VFS graphdriver").
On some filesystems, vfs fails to init with the following error:
> Error starting daemon: error initializing graphdriver: Failed to mknod
> /go/src/github.com/docker/docker/bundles/test-integration/d6bcf6de610e9/root/vfs/backingFsBlockDev:
> function not implemented
As quota is not essential for vfs, let's ignore (but log as a warning) any error
from quota init.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 1e8a087850aa9f96c5000a3ad90757d2e9c0499f
Component: engine
Revendor swarmkit to 713d79dc8799b33465c58ed120b870c52eb5eb4f to include
https://github.com/docker/swarmkit/pull/2473.
Signed-off-by: Marcus Martins <marcus@docker.com>
Upstream-commit: af73d31e60fd5c26d58bb8275785e628c3febdc0
Component: engine
Validation of Mounts was only performed on container _creation_, not on
container _start_. As a result, if the host-path no longer existed
when the container was started, a directory was created in the given
location.
This is the wrong behavior, because when using the `Mounts` API, host paths
should never be created, and an error should be produced instead.
This patch adds a validation step on container start, and produces an
error if the host path is not found.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 7cb96ba308dc53824d2203fd343a4a297d17976e
Component: engine
When the containerd 1.0 runtime changes were made, we inadvertantly
removed the functionality where any running containers are killed on
startup when not using live-restore.
This change restores that behavior.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: e69127bd5ba4dcf8ae1f248db93a95795eb75b93
Component: engine
If mknod() returns ENOSYS, it most probably means quota is not supported
here, so return the appropriate error.
This is a conservative* fix to regression in vfs graph driver introduced
by commit 7a1618ced359a3ac92 ("add quota support to VFS graphdriver").
On some filesystems, vfs fails to init with the following error:
> Error starting daemon: error initializing graphdriver: Failed to mknod
> /go/src/github.com/docker/docker/bundles/test-integration/d6bcf6de610e9/root/vfs/backingFsBlockDev:
> function not implemented
Reported-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 2dd39b7841bdb9968884bbedc5db97ff77d4fe3e
Component: engine
Files that are suffixed with `_linux.go` or `_windows.go` are
already only built on Linux / Windows, so these build-tags
were redundant.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 6ed1163c98703f8dd0693cecbadc84d2cda811c3
Component: engine
Solaris is no longer being worked on, so these files
are now just dead code.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1589cc0a85396e2768bfe9e558c7c2100dc3bc87
Component: engine
Follow the conventions for namespace naming set out by other projects,
such as linuxkit and cri-containerd. Typically, they are some sort of
host name, with a subdomain describing functionality of the namespace.
In the case of linuxkit, services are launched in `services.linuxkit`.
In cri-containerd, pods are launched in `k8s.io`, making it clear that
these are from kubernetes.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: 521e7eba86df25857647b93f13e5366c554e9d63
Component: engine
With the contianerd 1.0 migration we now have strongly typed errors that
we can check for process not found.
We also had some bad error checks looking for `ESRCH` which would only
be returned from `unix.Kill` and never from containerd even though we
were checking containerd responses for it.
Fixes some race conditions around process handling and our error checks
that could lead to errors that propagate up to the user that should not.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: e55bead518e4c72cdecf7de2e49db6c477cb58eb
Component: engine
Previously, the completions for `--xxx` and the corresponding
`-xxx-add` and `-xxx-rm` options were defined in separate blocks.
This caused a lot of duplicated code.
This PR removes duplication for xxx=config|group|host|placement-pref|secret.
Now the blocks for `create` and `update` only contain completions for
options that either only exist for the particular command or are specific
to it (completions for `--env-rm` and `--env|env-add` differ).
Signed-off-by: Harald Albers <github@albersweb.de>
Upstream-commit: f2b42bb6a8
Component: cli
`service create` and `service update` both used to have `--env`, `--label`
and `--mount` options.
These options now are only valid for `service create`.
`service update` got corresponding `--xxx-add|rm` options instead.
Signed-off-by: Harald Albers <github@albersweb.de>
Upstream-commit: d149c93c0c
Component: cli
Seen failing on Windows:
22:27:52 ----------------------------------------------------------------------
22:27:52 PANIC: docker_api_logs_test.go:152: DockerSuite.TestLogsAPIUntil
22:27:52
22:27:52 ... Panic: runtime error: index out of range (PC=0x45AC01)
22:27:52
22:27:52 d:/CI/CI-7caa30e89/go/src/runtime/asm_amd64.s:509
22:27:52 in call32
22:27:52 d:/CI/CI-7caa30e89/go/src/runtime/panic.go:491
22:27:52 in gopanic
22:27:52 d:/CI/CI-7caa30e89/go/src/runtime/panic.go:28
22:27:52 in panicindex
22:27:52 docker_api_logs_test.go:175
22:27:52 in DockerSuite.TestLogsAPIUntil
22:27:52 d:/CI/CI-7caa30e89/go/src/runtime/asm_amd64.s:509
22:27:52 in call32
22:27:52 d:/CI/CI-7caa30e89/go/src/reflect/value.go:434
22:27:52 in Value.call
22:27:52 d:/CI/CI-7caa30e89/go/src/reflect/value.go:302
22:27:52 in Value.Call
22:27:52 c:/gopath/src/github.com/docker/docker/vendor/github.com/go-check/check/check.go:816
22:27:52 in suiteRunner.forkTest.func1
22:27:52 c:/gopath/src/github.com/docker/docker/vendor/github.com/go-check/check/check.go:672
22:27:52 in suiteRunner.forkCall.func1
22:27:52 d:/CI/CI-7caa30e89/go/src/runtime/asm_amd64.s:2337
22:27:52 in goexit
22:27:54
22:27:54 ----------------------------------------------------------------------
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e77de7856bf212c764555ab8c2f346f2c529467c
Component: engine
The `repository:shortid` syntax for referencing images is very little used,
collides with with tag references can be confused with digest references.
The `repository:shortid` notation was deprecated in Docker 1.13 through
5fc71599a0b77189f0fedf629ed43c7f7067956c, and scheduled for removal
in Docker 17.12.
This patch removes the support for this notation.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a942c92dd77aff229680c7ae2a6de27687527b8a
Component: engine
The `repository:shortid` syntax for referencing images is very little used,
collides with with tag references can be confused with digest references.
The `repository:shortid` notation was deprecated in Docker 1.13, and scheduled
for removal in Docker 17.12.
This patch updates the deprecation status for this feature.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1a21ca12a6
Component: cli
Commit ddadd3db49 refactored
the markdown documentation, but accidentally changed
`on-failure` to `failure`.
This patch corrects this change.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 43217d7332
Component: cli
Currently we only support 'application/vnd.docker.distribution.manifest.v2+json'
manifest images download, with more multi-arch images used, we need to support
download images with 'application/vnd.docker.distribution.manifest.list.v2+json'
format(aka "fat manifest"), else we will fail to download those multi-arch ones.
This PR adds 'application/vnd.docker.distribution.manifest.list.v2+json' manifest
support, thus we can download both multi-arch and legacy images.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 0af5db511ed1ed5beab0feb09a2a96347a263410
Component: engine
The overlay2 driver was not setting up the archive.TarOptions field
properly like other storage backend routes to "applyTarLayer"
functionality. The InUserNS field is populated now for overlay2 using
the same query function used by the other storage drivers.
Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
Upstream-commit: 05b8d59015f8a5ce26c8bbaa8053b5bc7cb1a77b
Component: engine
The previous bytes counter was moved out of scope was not counting the
total number of bytes in the batch. This type encapsulates the counter
and the batch for consideration and code ergonomics.
Signed-off-by: Jacob Vallejo <jakeev@amazon.com>
Upstream-commit: ad14dbf1346742f0607d7c28a8ef3d4064f5f9fd
Component: engine
Also added back some of the maintainer processes that were in
MAINTAINERS but moved to docker/opensource repo. I believe this
project's governance should be disconnected from docker/opensource as
project's remaining under docker/opensource will not use the Moby TSC.
Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
Upstream-commit: 449c870afbd21563a6df04445fbb136d3230629b
Component: engine
align naming convention with x-go-name
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Upstream-commit: 8e7f9afa471c2be079f2c8897cf3b87e7f60544e
Component: engine
We are planning to remove supports for non-Linux platform in
runc (https://github.com/opencontainers/runc/pull/1654). The current
import here is the only thing that i found in docker that is windows-related
so fixing this would remove the rest of windows code in runc.
This changes some functions in daemon_windows to be the same as
daemon_unix to use runtime-spec public API instead of runc.
Signed-off-by: Daniel Dao <dqminh89@gmail.com>
Upstream-commit: 4d1d486202a7c3977e51275c2efdba922375b0cd
Component: engine
- the `--disable-legacy-registry` daemon flag was removed
- duplicate keys with conflicting values for engine labels
now produce an error instead of a warning.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a119e39f0c
Component: cli
The `docker info` command compares the installed version
of containerd using a Git-sha. We currently use a tag for
this, but that tag is not returned by the version-API of
containerd, resulting in the `docker info` output to show:
containerd version: 89623f28b87a6004d4b785663257362d1658a729 (expected: v1.0.0)
This patch changes the `v1.0.0` tag to the commit that
corresponds with the tag, so that the `docker info` output
does not show the `expected:` string.
This should be considered a temporary workaround; the check
for the exact version of containerd that's installed was needed
when we still used the 0.2.x branch, because it did not have
stable releases yet.
With containerd reaching 1.0, and using SemVer, we can likely
do a comparison for "Major" version, or make this a "packaging"
issue, and remove the check entirely (we can still _print_ the
version that's installed if we think it's usefule).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2c8018f4bd7f48bf8f35770dea68f81b9591bb58
Component: engine
Replace those legacy docker images prefixed by 'aarch64/' and 'arm64v8/'
with official multi-arch ones.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 7fc697eb0b62557b4abaa3b9fc0cd44b5c1652ac
Component: engine
Commit ee594dcb7d42f95048c9047d86c61447243db3cd removed the
`sleep 0.5` from this test, because sleep has a full-second
precision. However, in some cases, all three log-entries
are output at the same time, causing the `--until` filter
to fail.
This patch adds back a `sleep`, but uses 1 second instead.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1360f0dc9a5460bccf7974f7718d031435b883f3
Component: engine
VERSION file does not exist anymore for moby/moby so we need to
compensate
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: c411321b8870101c227b03e64930260984c862fd
Component: packaging
Right now we only log source and destination (and demsg) if mount operation
fails. fstype and mount options are available easily. It probably is a good
idea to log these as well. Especially sometimes failures can happen due to
mount options.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Upstream-commit: f728d74ac5d185adaa5f1a88eadc71217806859f
Component: engine
Interacting with v1 registries was deprecated in Docker 1.8.3, disabled by default
in Docker 17.06, and scheduled for removal in Docker 17.12.
This patch disallows enabling V1 registry through the `--disable-legacy-registry`
option, and the `"disable-legacy-registry": false` option in the daemon configuration
file. The actual V1 registry code is still in place, and will be removed separately.
With this patch applied:
$ dockerd --disable-legacy-registry=false
ERROR: The '--disable-legacy-registry' flag has been removed. Interacting with legacy (v1) registries is no longer supported
Or, when setting through the `daemon.json` configuration file
$ mkdir -p /etc/docker/
$ echo '{"disable-legacy-registry":false}' > /etc/docker/daemon.json
$ dockerd
ERROR: The 'disable-legacy-registry' configuration option has been removed. Interacting with legacy (v1) registries is no longer supported
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8d6df8a0addc9a37b48c5a1827dd3f65f2ed57cf
Component: engine
The installation steps for go-swagger was a bit noisy, and not consistent with
other installation steps.
This patch makes it similar to other steps, which makes it less noisy, and
makes the image slightly smaller.
Before:
b53d7aac3200 14 minutes ago |1 APT_MIRROR=deb.debian.org /bin/sh -c git … 107MB
fa74acf32f99 2 hours ago /bin/sh -c #(nop) ENV GO_SWAGGER_COMMIT=c28… 0B
After:
6b2454f1a9a5 10 minutes ago |1 APT_MIRROR=deb.debian.org /bin/sh -c set … 35.2MB
fa74acf32f99 2 hours ago /bin/sh -c #(nop) ENV GO_SWAGGER_COMMIT=c28… 0B
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 29d77acaf8bfb234ee4f0b3db9e28d7410b99d4e
Component: engine
This sleep probably doesn't work because sleep typically takes only
whole numbers, to do < 1s you need to use usleep. It also is not really
needed as the loop has a very low bound that will not eat up too much
CPU.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: ee594dcb7d42f95048c9047d86c61447243db3cd
Component: engine
Relates to an upgrade in `rpm` to `14.4.0`, where missing build-ids now
cause builds to self-terminate. May be needed in other distros if/when
the `rpm` package is updated in those repos.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: dcd681da54f2c2b2286cc09d053dd6cb865d3f98
Component: packaging
When building we should default to a dummy version unless otherwise
specified so we don't get ourselves confused over what is official and
what is not.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: ffcd040b5c3bd4f81e1d1f11c32f9bb81e235ed1
Component: packaging
The 17.12.0-rc1 release was cut-off from commit
ace5417954
This patch bumps the version to 18.01-dev to
indicate that any change is now for the upcoming
18.01 release.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c4875264a3
Component: cli
Before this patch, if a splunk endpoint returned a non-OK status, the
splunk driver would read the entire response body. This could lead to
excessive memory allocations. Instead put a cap on the amount the driver
will read from the response body.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 7f14542ddd1401de734be3ac0331c0ada941c959
Component: engine
If there is a package-rename forthcoming, it isn't currently evident on master, but seems to show in doc.
Changed doc to show actual docker/docker paths as they occur now instead of moby/moby.
Signed-off-by: Brett Randall <javabrett@gmail.com>
Upstream-commit: 355cf9483c1b8ede5ae3ed50add4de2a69d62645
Component: engine
Close pipe in chrootarchive.invokeUnpack when cmd.Start()/json.NewEncoder failed
Upstream-commit: cefb33700cf0931a6e3f9ea2cb0a122148d10e49
Component: engine
The Server section of version output is now composed of an Engine
component and potentially more, based on what the /version endpoint
returns.
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 5f4c5f8bb6
Component: cli
This change adds a Platform struct with a Name field and a general
Components field to the Version API type. This will allow API
consumers to show version information for the whole platform and
it will allow API providers to set the versions for the various
components of the platform.
All changes here are backwards compatible.
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 9152e63290e4a4e586b811cce39082efc649b912
Component: engine
Add a new configuration option to allow the enabling
of the networkDB debug. The option is only parsed using the
reload event. This will protect the daemon on start or restart
if the option is left behind in the config file
Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
Upstream-commit: a97e45794ea8318a08daf763a5b63b04184a886b
Component: engine
When the client closes websocket connections that sends container
output through websocket, an error message is displayed:
"Error attaching websocket: %!s(<nil>)"
This message is misleading. Thus, this change suggests to check
if error is nil and print the correct message accordingly.
Signed-off-by: Boaz Shuster <ripcurld.github@gmail.com>
Upstream-commit: 8f65bb6d90e3a95420bb634e415c3cce36d86201
Component: engine
Permit a broader range of errors from mirror endpoints when determining whether to fall back
Upstream-commit: 4055bfb3d3031de7077406ddf0f7e529c47852a9
Component: engine
Ensures that when a plugin is removed that it doesn't interfere with
other plugins mounts and also ensures its own mounts are cleaned up.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 5017b5bef55c31db4c04c8058ef7db8597b11341
Component: engine
Even though it's highly discouraged, there are existing
installs that are running overlay/overlay2 on filesystems
without d_type support.
This patch allows the daemon to start in such cases, instead of
refusing to start without an option to override.
For fresh installs, backing filesystems without d_type support
will still cause the overlay/overlay2 drivers to be marked as
"unsupported", and skipped during the automatic selection.
This feature is only to keep backward compatibility, but
will be removed at some point.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0a4e793a3da9ba6d20bccfb83f7c48e20a76d895
Component: engine
Support for running overlay/overlay2 on a backing filesystem
without d_type support (most likely: xfs, as ext4 supports
this by default), was deprecated for some time.
Running without d_type support is problematic, and can
lead to difficult to debug issues ("invalid argument" errors,
or unable to remove files from the container's filesystem).
This patch turns the warning that was previously printed
into an "unsupported" error, so that the overlay/overlay2
drivers are not automatically selected when detecting supported
storage drivers.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0abb8dec3f730f3ad2cc9a161c97968a6bfd0631
Component: engine
The fsmagic check was always performed on "data-root" (`/var/lib/docker`),
not on the storage-driver's home directory (e.g. `/var/lib/docker/<somedriver>`).
This caused detection to be done on the wrong filesystem in situations
where `/var/lib/docker/<somedriver>` was a mount, and a different
filesystem than `/var/lib/docker` itself.
This patch checks if the storage-driver's home directory exists, and only
falls back to `/var/lib/docker` if it doesn't exist.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f9c8fa305e1501d8056f8744cb193a720aab0e13
Component: engine
Failed out otherwise:
```
---> Making bundle: dynbinary (in bundles/dynbinary)
Building: bundles/dynbinary-daemon/dockerd-17.09.0-dev
Package libudev was not found in the pkg-config search path.
```
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 4d8e8855d1008f4705ea6db17e5cd5bc90bd2d81
Component: packaging
Seems `docker-edge` was removed as label on jenkins.dockerproject.org.
This removes that label and just puts the x86_64 label
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 6765382587e31a93b1e65d2d169b00a42d183ee2
Component: packaging
* add aarch64 support
* use go 1.9 for arm64 support
* fixtypo
* add backports on jessie
* fix path
* delet yakkety
* rm golang-go
* use arm64v8/ images
* fix error made in merge
* remove double architecture
* use arm64v8/
* use go 1.8.5
* Update aarch64 builds to 1.9.2
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 03eac425534002445e666ce0352f1c666151dc89
Component: packaging
Setting to something that will not be confused with a version number
that will ever be released. Also, this is required for the deb packages
to build properly
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Upstream-commit: 0de78bd1251958a9d8916c3944b8f51d0ffb49b7
Component: packaging
They don't function correctly and they're not worth it
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 3f5a6f288ad7e134abb4ac99e56cbbca17bfae88
Component: packaging
This commit is a set of fixes and improvement for zfs graph driver,
in particular:
1. Remove mount point after umount in `Get()` error path, as well
as in `Put()` (with `MNT_DETACH` flag). This should solve "failed
to remove root filesystem for <ID> .... dataset is busy" error
reported in Moby issue 35642.
To reproduce the issue:
- start dockerd with zfs
- docker run -d --name c1 --rm busybox top
- docker run -d --name c2 --rm busybox top
- docker stop c1
- docker rm c1
Output when the bug is present:
```
Error response from daemon: driver "zfs" failed to remove root
filesystem for XXX : exit status 1: "/sbin/zfs zfs destroy -r
scratch/docker/YYY" => cannot destroy 'scratch/docker/YYY':
dataset is busy
```
Output when the bug is fixed:
```
Error: No such container: c1
```
(as the container has been successfully autoremoved on stop)
2. Fix/improve error handling in `Get()` -- do not try to umount
if `refcount` > 0
3. Simplifies unmount in `Get()`. Specifically, remove call to
`graphdriver.Mounted()` (which checks if fs is mounted using
`statfs()` and check for fs type) and `mount.Unmount()` (which
parses `/proc/self/mountinfo`). Calling `unix.Unmount()` is
simple and sufficient.
4. Add unmounting of driver's home to `Cleanup()`.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: a450a575a672b90f02a4b1d3d9300ce1d70a6311
Component: engine
Previously, the code would set the mtime on the directories before
creating files in the directory itself. This was problematic
because it resulted in the mtimes on the directories being
incorrectly set. This change makes it so that the mtime is
set only _after_ all of the files have been created.
Signed-off-by: Sargun Dhillon <sargun@sargun.me>
Upstream-commit: 77a2bc3e5bbc9be3fe166ed8321b7cd04e7bd097
Component: engine
Changed logic to ignore empty value
Fixes#35626
Signed-off-by: Igor Karpovich <igor@karpovich.me>
Upstream-commit: 27a5b878c149fd70ca1e0beebda58edcc19abc73
Component: engine
The error type libnetwork.ErrNoSuchNetwork is used in the controller
to retry the network creation as a managed network though the manager.
The change of the type was breaking the logic causing the network to
not being created anymore so that no new container on that network
was able to be launched
Added unit test
Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
Upstream-commit: 51cea0a53c2fd36832277402e9faac81bfb4abd4
Component: engine
There was a small issue here, where it copied the data using
traditional mechanisms, even when copy_file_range was successful.
Signed-off-by: Sargun Dhillon <sargun@sargun.me>
Upstream-commit: 0eac562281782257e6f69d58bcbc13fa889f1759
Component: engine
This change makes the VFS graphdriver use the kernel-accelerated
(copy_file_range) mechanism of copying files, which is able to
leverage reflinks.
Signed-off-by: Sargun Dhillon <sargun@sargun.me>
Upstream-commit: d2b71b26604370620630d8d3f35aba75ae474f3f
Component: engine
Previously, graphdriver/copy would improperly copy hardlinks as just regular
files. This patch changes that behaviour, and instead the code now keeps
track of inode numbers, and if it sees the same inode number again
during the copy loop, it hardlinks it, instead of copying it.
Signed-off-by: Sargun Dhillon <sargun@sargun.me>
Upstream-commit: b467f8b2ef21dc2239dcd136a29283ea6c3a0aee
Component: engine
The overlay2 storage-driver requires multiple lower dir
support for overlayFs. Support for this feature was added
in kernel 4.x, but some distros (RHEL 7.4, CentOS 7.4) ship with
an older kernel with this feature backported.
This patch adds feature-detection for multiple lower dirs,
and will perform this feature-detection on pre-4.x kernels
with overlayFS support.
With this patch applied, daemons running on a kernel
with multiple lower dir support will now select "overlay2"
as storage-driver, instead of falling back to "overlay".
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 955c1f881ac94af19c99f0f7d5635e6a574789f2
Component: engine
Plugin config can have Mounts without a 'Source' field. In such cases,
performing a 'plugin set' on the mount source will panic the daemon. Its
the same case for device paths as well. This detects the case and
returns error.
Signed-off-by: Anusha Ragunathan <anusha.ragunathan@docker.com>
Upstream-commit: 6572e27df7f3483cfed7a8294c1f6d9cf157809a
Component: engine
ClusterHQ shutted down at the end of 2016. It seems that ScatterHQ is a group of ClusterHQ former employees that have forked Flocker. It seems that no code has been commited in the last months but if somebody comes to this Docker page it should be referenced to ScatterHQ.
Signed-off-by: Miguel Angel Alvarez Cabrerizo <doncicuto@gmail.com>
Upstream-commit: 3e6e54a90f
Component: cli
The commit '0a13f827a10d3bf61744d9b3f7165c5885a39c5d' introduces an
import test for CVE-2017-14992, it uses a 8GB image to make sure we
don't revert CVE-2017-14992, but unfortunately this test can't finish
in 5-min on AArch64, as a fact, in most cases we have to crate a very
big image to make the test effective on AArch64, but this will result
in a test panic, so now we skip it order to avoid termination of others
tests followed.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 6395b8b3dcc43be6750e0d90d9bab0a83e4eb20b
Component: engine
This subtle bug keeps lurking in because error checking for `Mkdir()`
and `MkdirAll()` is slightly different wrt to `EEXIST`/`IsExist`:
- for `Mkdir()`, `IsExist` error should (usually) be ignored
(unless you want to make sure directory was not there before)
as it means "the destination directory was already there"
- for `MkdirAll()`, `IsExist` error should NEVER be ignored.
Mostly, this commit just removes ignoring the IsExist error, as it
should not be ignored.
Also, there are a couple of cases then IsExist is handled as
"directory already exist" which is wrong. As a result, some code
that never worked as intended is now removed.
NOTE that `idtools.MkdirAndChown()` behaves like `os.MkdirAll()`
rather than `os.Mkdir()` -- so its description is amended accordingly,
and its usage is handled as such (i.e. IsExist error is not ignored).
For more details, a quote from my runc commit 6f82d4b (July 2015):
TL;DR: check for IsExist(err) after a failed MkdirAll() is both
redundant and wrong -- so two reasons to remove it.
Quoting MkdirAll documentation:
> MkdirAll creates a directory named path, along with any necessary
> parents, and returns nil, or else returns an error. If path
> is already a directory, MkdirAll does nothing and returns nil.
This means two things:
1. If a directory to be created already exists, no error is
returned.
2. If the error returned is IsExist (EEXIST), it means there exists
a non-directory with the same name as MkdirAll need to use for
directory. Example: we want to MkdirAll("a/b"), but file "a"
(or "a/b") already exists, so MkdirAll fails.
The above is a theory, based on quoted documentation and my UNIX
knowledge.
3. In practice, though, current MkdirAll implementation [1] returns
ENOTDIR in most of cases described in #2, with the exception when
there is a race between MkdirAll and someone else creating the
last component of MkdirAll argument as a file. In this very case
MkdirAll() will indeed return EEXIST.
Because of #1, IsExist check after MkdirAll is not needed.
Because of #2 and #3, ignoring IsExist error is just plain wrong,
as directory we require is not created. It's cleaner to report
the error now.
Note this error is all over the tree, I guess due to copy-paste,
or trying to follow the same usage pattern as for Mkdir(),
or some not quite correct examples on the Internet.
[1] https://github.com/golang/go/blob/f9ed2f75/src/os/path.go
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 516010e92d56cfcd6d1e343bdc02b6f04bc43039
Component: engine
- removed support for Solaris
- networkdb fixed race on node management
Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
Upstream-commit: 2e5d5c1d325662ac56de4772e4e1fe3a9c7a2d19
Component: engine
Standard golang's `os.MkdirAll()` function returns "not a directory" error
in case a directory to be created already exists but is not a directory
(e.g. a file). Our own `idtools.MkdirAs*()` functions do not replicate
the behavior.
This is a bug since all `Mkdir()`-like functions are expected to ensure
the required directory exists and is indeed a directory, and return an
error otherwise.
As the code is using our in-house `system.Stat()` call returning a type
which is incompatible with that of golang's `os.Stat()`, I had to amend
the `system` package with `IsDir()`.
A test case is also provided.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 2aa13f86f0c9cf3ed58a648a7b1506d4b06f3589
Component: engine
example demonstrates they are expected
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Upstream-commit: 74cb739766e517115c5af9cee2202613661238a0
Component: engine
This removes and recreates the merged dir with each umount/mount
respectively.
This is done to make the impact of leaking mountpoints have less
user-visible impact.
It's fairly easy to accidentally leak mountpoints (even if moby doesn't,
other tools on linux like 'unshare' are quite able to incidentally do
so).
As of recently, overlayfs reacts to these mounts being leaked (see
One trick to force an unmount is to remove the mounted directory and
recreate it. Devicemapper now does this, overlay can follow suit.
Signed-off-by: Euan Kemp <euan.kemp@coreos.com>
Upstream-commit: af0d589623eff9f8cefced8b527dbd7cf221ce61
Component: engine
Adds a main_test for the image integration test, so we can download
frozen images, and clean up after the image test is ran
Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
Upstream-commit: be83f42612e3be42fcd60726d48d7346befc9449
Component: engine
- Fix OOM event updating healthchecks and persisting container state
without locks
- Fix healthchecks being updated without locks on container stop
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 972cb4978795029131697bd3b3746e321eec5c13
Component: engine
When starting the daemon, the `/var/lib/docker` directory
is scanned for existing directories, so that the previously
selected graphdriver will automatically be used.
In some situations, empty directories are present (those
directories can be created during feature detection of
graph-drivers), in which case the daemon refuses to start.
This patch improves detection, and skips empty directories,
so that leftover directories don't cause the daemon to
fail.
Before this change:
$ mkdir /var/lib/docker /var/lib/docker/aufs /var/lib/docker/overlay2
$ dockerd
...
Error starting daemon: error initializing graphdriver: /var/lib/docker contains several valid graphdrivers: overlay2, aufs; Please cleanup or explicitly choose storage driver (-s <DRIVER>)
With this patch applied:
$ mkdir /var/lib/docker /var/lib/docker/aufs /var/lib/docker/overlay2
$ dockerd
...
INFO[2017-11-16T17:26:43.207739140Z] Docker daemon commit=ab90bc296 graphdriver(s)=overlay2 version=dev
INFO[2017-11-16T17:26:43.208033095Z] Daemon has completed initialization
And on restart (prior graphdriver is still picked up):
$ dockerd
...
INFO[2017-11-16T17:27:52.260361465Z] [graphdriver] using prior storage driver: overlay2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1262c57714e694193be6bbcbed83e859dc246c2f
Component: engine
This is a fix to the following issue:
$ docker run --tmpfs /dev/shm busybox sh
docker: Error response from daemon: linux mounts: Duplicate mount point '/dev/shm'.
In current code (daemon.createSpec()), tmpfs mount from --tmpfs is added
to list of mounts (`ms`), when the mount from IpcMounts() is added.
While IpcMounts() is checking for existing mounts first, it does that
by using container.HasMountFor() function which only checks container.Mounts
but not container.Tmpfs.
Ultimately, the solution is to get rid of container.Tmpfs (moving its
data to container.Mounts). Current workaround is to add checking
of container.Tmpfs into container.HasMountFor().
A unit test case is included.
Unfortunately we can't call daemon.createSpec() from a unit test,
as the code relies a lot on various daemon structures to be initialized
properly, and it is hard to achieve. Therefore, we minimally mimick
the code flow of daemon.createSpec() -- barely enough to reproduce
the issue.
https://github.com/moby/moby/issues/35455
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 1861abdc4a31efad202a5c3d89a895bb7a62799a
Component: engine
The code in question looks up mounts two times: first by using
HasMountFor(), and then directly by looking in container.MountPoints.
There is no need to do it twice.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: eab3ac3e70a510b97f9399efd13e3dc01a07c413
Component: engine
Commit dbf580be57a4bb854d7ce20d313e3a22ea337be5 removed
this helper script because it's no longer used in CI.
However, the "make test" target in the Makefile still
called this helper, resulting it to fail.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a17071e88f15625a6be19f80c697ab1c3471381b
Component: engine
There is a typo in the `plugins_volume.md#volumedriverpath` section.
The `/VolumeDriver.Path` response (v1) should be `Mountpoint`and not `Mountpoin`.
Signed-off-by: scipio3000 <gunther@gameslabs.net>
Signed-off-by: Günther Jungbluth <gunther@gameslabs.net>
Upstream-commit: e575b8e756
Component: cli
This route expects `application/json`. Sending a content type header of `application/octet-stream` results in an error.
Signed-off-by: Asad Saeeduddin <masaeedu@gmail.com>
Upstream-commit: 876b32861789a0424557c640622bde47eedd2d98
Component: engine
Commit 7a1618ced359a3ac921d8a05903d62f544ff17d0 regresses running Docker
in user namespaces. The new check for whether quota are supported calls
NewControl() which in turn calls makeBackingFsDev() which tries to
mknod(). Skip quota tests when we detect that we are running in a user
namespace and return ErrQuotaNotSupported to the caller. This just
restores the status quo.
Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com>
Upstream-commit: 7e35df0e0484118740dbf01e7db9b482a1827ef1
Component: engine
Change-Id: I9ebcf49e9e8ac76beb037779ad02ac6020169849
Signed-off-by: Li Yi <denverdino@gmail.com>
Upstream-commit: e987c554c9ff0740b9945e68228b141031bb31c6
Component: engine
Add a way to specify a custom graphdriver priority list
during build. This can be done with something like
go build -ldflags "-X github.com/docker/docker/daemon/graphdriver.priority=overlay2,devicemapper"
As ldflags are already used by the engine build process, and it seems
that only one (last) `-ldflags` argument is taken into account by go,
an envoronment variable `DOCKER_LDFLAGS` is introduced in order to
be able to append some text to `-ldflags`. With this in place,
using the feature becomes
make DOCKER_LDFLAGS="-X github.com/docker/docker/daemon/graphdriver.priority=overlay2,devicemapper" dynbinary
The idea behind this is, the priority list might be different
for different distros, so vendors are now able to change it
without patching the source code.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 17708e72a7ef29fb1d4b03fbded1c5e4c08105fd
Component: engine
Adds a mutex to protect the status, as well. When running the race
detector with the unit test, we can see that the Status field is written
without holding this lock. Adding a mutex to read and set status
addresses the issue.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: 7db30ab0cdf072956d2ceda833b7de22fe17655c
Component: engine
Make it possible to disable overlay and overlay2 separately.
With this commit, we now have `exclude_graphdriver_overlay` and
`exclude_graphdriver_overlay2` build tags for the engine, which
is in line with any other graph driver.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: d014be5426c869d429c1a11cad9e76321dd7a326
Component: engine
Before this change, if the splunk endpoint is blocked it will cause a
deadlock on `Close()`.
This sets a reasonable timeout for the http request to send a log batch.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 24087399d95d60be4184b9ed3eba56466878b4e1
Component: engine
This test case is checking that the built-in default size for /dev/shm
(which is used for `--ipcmode` being `private` or `shareable`)
is not overriding the size of user-defined tmpfs mount for /dev/shm.
In other words, this is a regression test case for issue #35271,
https://github.com/moby/moby/issues/35271
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 2e0a98b605fa278ee1f348c68fe7e07aed57b834
Component: engine
From https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt:
> The lower filesystem can be any filesystem supported by Linux and does
> not need to be writable. The lower filesystem can even be another
> overlayfs. The upper filesystem will normally be writable and if it
> is it must support the creation of trusted.* extended attributes, and
> must provide valid d_type in readdir responses, so NFS is not suitable.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 90dfb1d0cc59d79ccb272997d735864615010785
Component: engine
While this code was likely called from a single thread before, we have
now seen panics, indicating that it could be called in parallel. This
change adds a mutex to protect opening and closing of the channel. There
may be another root cause associated with this panic, such as something
that led to the calling of this in parallel, as this code is old and we
had seen this condition until recently.
This fix is by no means a permanent fix. Typically, bugs like this
indicate misplaced channel ownership. In idiomatic uses, the channel
should have a particular "owner" that coordinates sending and closure.
In this case, the owner of the channel is unclear, so it gets opened
lazily. Synchronizing this access is a decent solution, but a refactor
may yield better results.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: 5b55747a523671fa6e626848060460a48d058451
Component: engine
The `docker daemon` subcommand was only present for
backward compatibility, but deprecated in v1.13,
and scheduled for removal in v17.12
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c6a3199236
Component: cli
Updates runc to b2567b37d7b75eb4cf325b77297b140ea686ce8f which removes
some cross-repo dependencies.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: af248a3fe087805907e4b69ab017ef60d44ce093
Component: engine
Commit 7120976d74195 ("Implement none, private, and shareable ipc
modes") introduces a bug: if a user-specified mount for /dev/shm
is provided, its size is overriden by value of ShmSize.
A reproducer is simple:
docker run --rm
--mount type=tmpfs,dst=/dev/shm,tmpfs-size=100K \
alpine df /dev/shm
This commit is an attempt to fix the bug, as well as optimize things
a but and make the code easier to read.
https://github.com/moby/moby/issues/35271
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 31d30a985d99a0eef92116a22159727f5c332784
Component: engine
Support for duplicate labels (but different values) was
deprecated in commit e4c9079d091a2eeac8a74a0356e3f348db873b87
(Docker 1.13), and scheduled for removal in 17.12
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8c6322338c91cdb88b1fef4def393d9a7b670366
Component: engine
Go has supported PIC builds for a while now, and given the security
benefits of using PIC binaries we should really enable them. There also
appears to be some indication that non-PIC builds have been interacting
oddly on ppc64le (the linker cannot load some shared libraries), and
using PIC builds appears to solve this problem.
Signed-off-by: Aleksa Sarai <asarai@suse.de>
Upstream-commit: 1f4e37cf4bd2f73dc5257d791cc4dba294ddd156
Component: engine
Before this, if a volume exists in a driver but not in the local cache,
the store would just return a bare volume. This means that if a user
supplied options or labels, they will not get stored.
Instead only return early if we have the volume stored locally. Note
this could still have an issue with labels/opts passed in by the user
differing from what is stored, however this isn't really a new problem.
This fixes a problem where if there is a shared storage backend between
two docker nodes, a create on one node will have labels stored and a
create on the other node will not.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 4d8598ad0506b29c12632c1b8ed92eb58fc2f0e2
Component: engine
The test was passing previously because the preamble was already buffered. After
the change to return Scanner.Err() the final read error on the buffer was no
longer being ignored.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: a74cc833450dfc48cc95b2b109cbcb24feff4929
Component: engine
`scope`=<string> local or swarm had special character, which was breaking the Swagger UI
Signed-off-by: Jeeva S. Chelladhurai <sjeeva@gmail.com>
Upstream-commit: a21654c34b741656582b09e09127821311909dcd
Component: engine
In order to avoid reverting our fix for mount leakage in devicemapper,
add a test which checks that devicemapper's Get() and Put() cycle can
survive having a command running in an rprivate mount propagation setup
in-between. While this is quite rudimentary, it should be sufficient.
We have to skip this test for pre-3.18 kernels.
Signed-off-by: Aleksa Sarai <asarai@suse.de>
Upstream-commit: 1af8ea681fba1935c60c11edbbe19b894c9b286f
Component: engine
To ensure that we don't revert CVE-2017-14992, add a test that is quite
similar to that upstream tar-split test (create an empty archive with
lots of junk and make sure the daemon doesn't crash).
Signed-off-by: Aleksa Sarai <asarai@suse.de>
Upstream-commit: 0a13f827a10d3bf61744d9b3f7165c5885a39c5d
Component: engine
This helper acts like /dev/zero (outputs \x00 indefinitely) in an
OS-independent fashion. This ensures we don't need to special-case
around Windows in tests that want to open /dev/zero.
Signed-off-by: Aleksa Sarai <asarai@suse.de>
Upstream-commit: 2f8d3e1c33f77187c68893803018756d43daff15
Component: engine
Update to the latest version of tar-split, which includes a change to
fix a memory exhaustion issue where a malformed image could cause the
Docker daemon to crash.
* tar: asm: store padding in chunks to avoid memory exhaustion
Fixes: CVE-2017-14992
Signed-off-by: Aleksa Sarai <asarai@suse.de>
Upstream-commit: e0ff7cccc3cac73da41ec9ef007b0e4e97c55d01
Component: engine
This patch adds the capability for the VFS graphdriver to use
XFS project quotas. It reuses the existing quota management
code that was created by overlay2 on XFS.
It doesn't rely on a filesystem whitelist, but instead
the quota-capability detection code.
Signed-off-by: Sargun Dhillon <sargun@sargun.me>
Upstream-commit: 7a1618ced359a3ac921d8a05903d62f544ff17d0
Component: engine
This allows much of the read logic to be shared for other things,
especially for the new log driver proposed in
https://github.com/moby/moby/issues/33475
The only logic for reads in the json logger is around decoding log
messages, which gets passed into the log file object.
This also helps with implementing compression as it allows us to
simplify locking strategies.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 16f7cd674902b69b97692de2a83915a1a6be2cdb
Component: engine
Make the `*RotateFileWriter` specifically about writing
`logger.Message`'s, which is what it's used for.
This allows for future changes where the log writer can cache details
about log entries such as (e.g.) the timestamps included in a particular
log file, which can be used to optimize reads.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 52d82b4fbc9f0fe00f63e2df9a3d2a49d4095bda
Component: engine
This is writeable, and can be used to remove devices. Containers do
not need to know about scsi devices.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Upstream-commit: a21ecdf3c8a343a7c94e4c4d01b178c87ca7aaa1
Component: engine
This largely removes references to Docker where possible.
The HOWTO guides are still Docker docs and refer to Docker not
moby, so the next step is to rework these as Moby docs and put
them in `docs/` in this repo.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Upstream-commit: cecd0de3a9f783b34f75ca5e6394b9a0c123999e
Component: engine
The `docker info` code was shelling out to obtain the
version of containerd (using the `--version` flag).
Parsing the output of this version string is error-prone,
and not needed, as the containerd API can return the
version.
This patch adds a `Version()` method to the containerd Client
interface, and uses this to get the containerd version.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: fec2b144feaaa18998ec2ed34c9bc843c4c29abd
Component: engine
/dev is mounted on a tmpfs inside of a container. Processes inside of containers
some times need to create devices nodes, or to setup a socket that listens on /dev/log
Allowing these containers to run with the --readonly flag makes sense. Making a tmpfs
readonly does not add any security to the container, since there is plenty of places
where the container can write tmpfs content.
I have no idea why /dev was excluded.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Upstream-commit: 5f3bd2473ee2a1b9f37ba0130e934133d0e01f89
Component: engine
Commit 3ba1dda1914fa7d380d9d3220c3b158a41f90cba bumped
the API version, but forgot to actually bump the version
in code.
This patch fixes the version to match those changes :-)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 7f8dbe4a86650b02ffb490098b9f3bf0912c575d
Component: engine
The Swagger file contained a version matrix to
find which API version is used by which version
of Docker.
Given that Docker is a downstream of the Moby project,
we should not be maintaining such a matrix in this
repository.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 3fb228ba50ccb921148e0174e5e5e45ab650c3e5
Component: engine
Commit d91c5f42eb37c6f88cec4021c10c0a1ded1785c3 added
support for "Isolation" mode for services, but didn't
get merged before API 1.34.
This patch moves the description in the API version
history to the correct API version (1.35), and does
a slight rewording of the functionality.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 4533a16fa98785c8c18bf9b9a32da5390fc74757
Component: engine
Remove Dockerfile.solaris reference in `hack/make/.detect-daemon-osarch`
as `Dockerfile.solaris` has been removed.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: d157d98a5fbfc98f4adec20ce510b99c795b0d9c
Component: engine
If the user specifies a mountpath from the host, we should not be
attempting to chown files outside the daemon's metadata directory
(represented by `daemon.repository` at init time).
This forces users who want to use user namespaces to handle the
ownership needs of any external files mounted as network files
(/etc/resolv.conf, /etc/hosts, /etc/hostname) separately from the
daemon. In all other volume/bind mount situations we have taken this
same line--we don't chown host file content.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
Upstream-commit: 42716dcf5c986e4cbb51f480f2782c05e5bd0b41
Component: engine
The `.integration-test-helpers` script was sourced by `/etc/bash/bash.rc`.
However, the `$SCRIPTDIR` environment variable is set through `hack/make.sh`,
so will not be set when calling the `.integration-test-helpers` script directly.
Before this patch;
make BIND_DIR=. shell
...
bash: /make/.go-autogen: No such file or directory
After this patch, the warning is no longer printed
Also removed sourcing `.go-autogen` from test-integration and build-integration-test-binary,
as they already sourced `.integration-test-helpers` (which sources
`.go-autogen`).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 6d5e468db1f07c12140af0cf36ba0d1a09bb4ca9
Component: engine
Instead of duplicating the same if condition per plugin manager directory,
use one if condition and a for-loop.
Signed-off-by: Boaz Shuster <ripcurld.github@gmail.com>
Upstream-commit: d75f1d848721f04fdc703a2e88a98600f29933a6
Component: engine
These scripts have not been used for a while now, and should not be used again
because they are for releasing docker, not moby
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: eef85648e43fef2ec73462fd3d734947384cb16e
Component: engine
Signed-off-by: John Howard <jhoward@microsoft.com>
This is the CLI updates for the document discussed in https://github.com/moby/moby/issues/34617
to support Linux Containers on Windows. It adds --platform= as CLI flags to the four
commands listed above. Import still to be completed (needs daemon changes).
Upstream-commit: d8b782560e
Component: cli
This vendoring fixes two issues.
1. When a user specifies an SELinux MCS Label (level) to override moby picking
an unigue MCS label, the code currently picks a label then overrides with the
user selected. This works fine, except the unique MCS Label is leaked and will
not be used until the daemon is restarted.
2. The override label, is not reserved. This could potentially cause an issue
where the daemon could pick the same MCS Label again for a different container.
(~ 1/500,000 Chance).
The updated selinux go bindings, now release the overriden unigue label, and reserve
the one specified by the user.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 73c82386148fe14a47cc515c622bd23b9b7d99b9
Component: engine
Fix 19 typos, grammatical errors and duplicated words.
These fixes have minimal impact on the code as these are either in the
doc files or in comments inside the code files.
Signed-off-by: Abdur Rehman <abdur_rehman@mentor.com>
Upstream-commit: c3fe9d85ef
Component: cli
Update golang.org/x/sys to 95c6576299259db960f6c5b9b69ea52422860fce in
order to get the unix.Utsname with byte array instead of int8/uint8
members.
This allows to use simple byte slice to string conversions instead of
using charsToString or its open-coded version.
Also see golang/go#20753 for details.
Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
Upstream-commit: 6d068bc25b35a420e63e295ea4ab4ac4a6e6665b
Component: engine
In some circumstances we were not properly releasing plugin references,
leading to failures in removing a plugin with no way to recover other
than restarting the daemon.
1. If volume create fails (in the driver)
2. If a driver validation fails (should be rare)
3. If trying to get a plugin that does not match the passed in capability
Ideally the test for 1 and 2 would just be a unit test, however the
plugin interfaces are too complicated as `plugingetter` relies on
github.com/pkg/plugin/Client (a concrete type), which will require
spinning up services from within the unit test... it just wouldn't be a
unit test at this point.
I attempted to refactor this a bit, but since both libnetwork and
swarmkit are reliant on `plugingetter` as well, this would not work.
This really requires a re-write of the lower-level plugin management to
decouple these pieces.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 3816b514387efd24394f0b8e61d55502aa6ac9ac
Component: engine
Document the new `--chown` flag added to the ADD and COPY commands in
the Dockerfile format.
Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
Upstream-commit: a83b9f102b
Component: cli
Port mappings in `docker service ls` are quite verbose, and occupy a lot of
space when ranges of ports are published.
This patch improves the output by reconstructing ranges of ports.
Given the following service;
$ docker service create \
-p 60-61:60-61 \
-p 62:61 \
-p 80:80 \
-p 81:80 \
-p 90-95:90-95 \
-p 90-92:90-92/udp \
-p 93-96:93-96/udp \
--name foo \
nginx:alpine
Before this patch is applied:
$ docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
u1kwguv841qg foo replicated 1/1 nginx:alpine *:60->60/tcp,*:61->61/tcp,*:62->61/tcp,*:80->80/tcp,*:81->80/tcp,*:90->90/tcp,*:91->91/tcp,*:92->92/tcp,*:93->93/tcp,*:94->94/tcp,*:95->95/tcp,*:90->90/udp,*:91->91/udp,*:92->92/udp,*:93->93/udp,*:94->94/udp,*:95->95/udp,*:96->96/udp
After this patch is applied:
$ docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
u1kwguv841qg foo replicated 1/1 nginx:alpine *:60-62->60-61/tcp,*:80-81->80/tcp,*:90-95->90-95/tcp,*:90-96->90-96/udp
Additional enhancements can still be made, and marked as TODO in this change;
- combine non-consecutive ports mapped to a single port (`80->80`, `81->80`,
`84->80`, `86->80`, `87->80`); to be printed as `*:80-81,84,86-87->80`.
- combine `tcp` and `udp` mappings if their port-mapping is the same;
print `*:80-81->80-81/tcp+udp` instead of `*:80-81->80-81/tcp, *:80-81->80-81/udp`
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e98e95e7bc
Component: cli
`strings.Trim()` strips any character listed in the `cutset` argument,
so any example section having `E`, `x`, `a`, `m`, `p`, `l`, `e`, or `s`
in the first word, had these characters missing in the generated
YAML.
Also trim superfluent whitespace characters to consistently use `|-` ("strip")
as block chomping indicator (see http://www.yaml.org/spec/1.2/spec.html#id2794534)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1f605d43ca
Component: cli
libdm currently has a fairly substantial DoS bug that makes certain
operations fail on a libdm device if the device has active references
through mountpoints. This is a significant problem with the advent of
mount namespaces and MS_PRIVATE, and can cause certain --volume mounts
to cause libdm to no longer be able to remove containers:
% docker run -d --name testA busybox top
% docker run -d --name testB -v /var/lib/docker:/docker busybox top
% docker rm -f testA
[fails on libdm with dm_task_run errors.]
This also solves the problem of unprivileged users being able to DoS
docker by using unprivileged mount namespaces to preseve mounts that
Docker has dropped.
Signed-off-by: Aleksa Sarai <asarai@suse.de>
Upstream-commit: 92e45b81e0a8b68d9567a2068247460a1ba59600
Component: engine
you must stop all containers before upgrading to Docker CE 17.11.
If you don't, any containers started by Docker versions that predate
17.11 won't be recognized by Docker after the upgrade and will keep
running, un-managed, on the system.
### Builder
* Test & Fix build with rm/force-rm matrix [moby/moby#35139](https://github.com/moby/moby/pull/35139)
- Fix build with `--stream` with a large context [moby/moby#35404](https://github.com/moby/moby/pull/35404)
## 18.09.9 (2019-08-28)
### Client
*Hide help flag from help output [docker/cli#645](https://github.com/docker/cli/pull/645)
*Support parsing of named pipes for compose volumes [docker/cli#560](https://github.com/docker/cli/pull/560)
*[Compose] Cast values to expected type after interpolating values [docker/cli#601](https://github.com/docker/cli/pull/601)
+ Add output for "secrets" and "configs" on `docker stack deploy` [docker/cli#593](https://github.com/docker/cli/pull/593)
- Fix flag description for `--host-add` [docker/cli#648](https://github.com/docker/cli/pull/648)
* Do not truncate ID on docker service ps --quiet [docker/cli#579](https://github.com/docker/cli/pull/579)
### Deprecation
* Update bash completion and deprecation for synchronous service updates [docker/cli#610](https://github.com/docker/cli/pull/610)
-Fix Windows absolute path detection on non-Windows. [docker/cli#1990](https://github.com/docker/cli/pull/1990)
-Fix Docker refusing to load key from delegation.key on Windows. [docker/cli#1968](https://github.com/docker/cli/pull/1968)
- Completion scripts updates for bash and zsh.
### Logging
*copy to log driver's bufsize, fixes #34887 [moby/moby#34888](https://github.com/moby/moby/pull/34888)
+ Add TCP support for GELF log driver [moby/moby#34758](https://github.com/moby/moby/pull/34758)
+ Add credentials endpoint option for awslogs driver [moby/moby#35055](https://github.com/moby/moby/pull/35055)
-Fix for reading journald logs. [moby/moby#37819](https://github.com/moby/moby/pull/37819) [moby/moby#38859](https://github.com/moby/moby/pull/38859)
### Networking
-Fix network name masking network ID on delete [moby/moby#34509](https://github.com/moby/moby/pull/34509)
- Fix returned error code for network creation from 500 to 409 [moby/moby#35030](https://github.com/moby/moby/pull/35030)
- Fix tasks fail with error "Unable to complete atomic operation, key modified" [docker/libnetwork#2004](https://github.com/docker/libnetwork/pull/2004)
-Prevent panic on network attach to a container with disabled networking. [moby/moby#39589](https://github.com/moby/moby/pull/39589)
- Fix service port for an application becomes unavailable randomly. [docker/libnetwork#2069](https://github.com/docker/libnetwork/pull/2069)
- Fix cleaning up `--config-only` networks after `--config-from` networks have ungracefully exited. [docker/libnetwork#2373](https://github.com/docker/libnetwork/pull/2373)
### Runtime
*Switch to Containerd 1.0 client [moby/moby#34895](https://github.com/moby/moby/pull/34895)
*Increase container default shutdown timeout on Windows [moby/moby#35184](https://github.com/moby/moby/pull/35184)
* LCOW: API: Add `platform` to /images/create and /build [moby/moby#34642](https://github.com/moby/moby/pull/34642)
* Stop filtering Windows manifest lists by version [moby/moby#35117](https://github.com/moby/moby/pull/35117)
* Use windows console mode constants from Azure/go-ansiterm [moby/moby#35056](https://github.com/moby/moby/pull/35056)
*Windows Daemon should respect DOCKER_TMPDIR [moby/moby#35077](https://github.com/moby/moby/pull/35077)
* Performance optimizations in aufs and layer store for massively parallel container creation/removal. [moby/moby#39107](https://github.com/moby/moby/pull/39107)
* Update to containerd 1.2.6. [moby/moby#39016](https://github.com/moby/moby/pull/39016)
- Fix: [CVE-2018-15664](https://nvd.nist.gov/vuln/detail/CVE-2018-15664) symlink-exchange attack with directory traversal. [moby/moby#39357](https://github.com/moby/moby/pull/39357)
- Windows: fix support for `docker service create --limit-cpu`. [moby/moby#39190](https://github.com/moby/moby/pull/39190)
- Stop sorting uid and gid ranges in id maps. [moby/moby#39288](https://github.com/moby/moby/pull/39288)
### Logging
- Fix to allow large log lines for logger plugins. [moby/moby#39038](https://github.com/moby/moby/pull/39038)
## 18.09.6 (2019-05-02)
### Builder
- Fix `COPY` and `ADD` with multiple `<src>` do not invalidate cache if `DOCKER_BUILDKIT=1`. [moby/moby#38964](https://github.com/moby/moby/issues/38964)
### Networking
- Cleanup the cluster provider when the agent is closed. [docker/libnetwork#2354](https://github.com/docker/libnetwork/pull/2354)
- Windows: pick a random host port if the user does not specify a host port. [docker/libnetwork#2369](https://github.com/docker/libnetwork/pull/2369)
- Fix the stack informer's selector used to track deployment. [docker/cli#1794](https://github.com/docker/cli/pull/1794)
### Networking
- Fix `network=host` using wrong `resolv.conf` with `systemd-resolved`. [docker/engine#180](https://github.com/docker/engine/pull/180)
- Fix Windows ARP entries getting corrupted randomly under load. [docker/engine#192](https://github.com/docker/engine/pull/192)
### Runtime
- Fix stopped containers with restart policy showing as `Restarting`. [docker/engine#181](https://github.com/docker/engine/pull/181)
- Fix to use original process spec for execs. [docker/engine#178](https://github.com/docker/engine/pull/178)
### Swarm Mode
*Modifying integration test due to new ipam options in swarmkit [moby/moby#35103](https://github.com/moby/moby/pull/35103)
- Fix deadlock on getting swarm info [moby/moby#35388](https://github.com/moby/moby/pull/35388)
+ Expand the scope of the `Err` field in `TaskStatus` to also cover non-terminal errors that block the task from progressing [docker/swarmkit#2287](https://github.com/docker/swarmkit/pull/2287)
-Fix leaking task resources when nodes are deleted. [docker/engine#185](https://github.com/docker/engine/pull/185)
## 18.09.4 (2019-03-27)
### Builder
- Add validation for git ref so it can't be misinterpreted as a flag. [moby/moby#38944](https://github.com/moby/moby/pull/38944)
### Runtime
- Fix `docker cp` error with filenames greater than 100 characters. [moby/moby#38634](https://github.com/moby/moby/pull/38634)
- Fix layer/layer_store: ensure NewInputTarStream resources are released. [moby/moby#38413](https://github.com/moby/moby/pull/38413)
* Increase GRPC limit for GetConfigs. [moby/moby#38800](https://github.com/moby/moby/pull/38800)
* Update to containerd 1.2.5. [docker/engine#173](https://github.com/docker/engine/pull/173)
### Swarm Mode
- Fix nil pointer exception when joining node to swarm. [moby/moby#38618](https://github.com/moby/moby/issues/38618)
## 18.09.3 (2019-02-28)
### Networking
- Windows: avoid regeneration of network ids to prevent broken references to networks. [docker/engine#149](https://github.com/docker/engine/pull/149)
### Runtime
* Update to Go 1.10.8.
* Modify some of the names in the container name generator. [docker/engine#159](https://github.com/docker/engine/pull/159)
- When copying existing folder, ignore xattr set errors when the target filesystem doesn't support xattr. [docker/engine#135](https://github.com/docker/engine/pull/135)
- Graphdriver: fix "device" mode not being detected if "character-device" bit is set. [docker/engine#160](https://github.com/docker/engine/pull/160)
- Fix nil pointer derefence on failure to connect to containerd. [docker/engine#162](https://github.com/docker/engine/pull/162)
- Delete stale containerd object on start failure. [docker/engine#154](https://github.com/docker/engine/pull/154)
## 18.09.2 (2019-02-11)
### Security
- Update `runc` to address a critical vulnerability that allows specially-crafted containers to gain administrative privileges on the host. ([CVE-2019-5736](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-5736))
- Fix docker system prune doesn't accept until filter. [docker/engine#122](https://github.com/docker/engine/pull/122)
- Avoid unset credentials in containerd. [docker/engine#122](https://github.com/docker/engine/pull/122)
* Update to BuildKit 0.3.3. [docker/engine#122](https://github.com/docker/engine/pull/122)
+ Additional warnings for use of deprecated legacy overlay and devicemapper storage dirvers. [docker/engine#85](https://github.com/docker/engine/pull/85)
### Client
+ Add bash completion for experimental CLI commands (manifest). [docker/cli#1542](https://github.com/docker/cli/pull/1542)
- Fix yamldocs outputing `[flags]` in usage output. [docker/cli#1540](https://github.com/docker/cli/pull/1540)
- Fix setting default schema to tcp for docker host. [docker/cli#1454](https://github.com/docker/cli/pull/1454)
- prune: perform image pruning before build cache pruning. [docker/cli#1532](https://github.com/docker/cli/pull/1532)
- Fix bash completion for `service update --force`. [docker/cli#1526](https://github.com/docker/cli/pull/1526)
### Networking
- Fix iptables compatibility on debian. [docker/engine#107](https://github.com/docker/engine/pull/107)
### Packaging
+ Build packages for Debian 10 (Buster) [docker/docker-ce-packaging#50](https://github.com/docker/docker-ce-packaging/pull/50)
+ Build packages for Ubuntu 17.10 (Artful) [docker/docker-ce-packaging#55](https://github.com/docker/docker-ce-packaging/pull/55)
+ Add docker.socket requirement for docker.service. [docker/docker-ce-packaging#276](https://github.com/docker/docker-ce-packaging/pull/276)
+ Add socket activation for RHEL-based distributions. [docker/docker-ce-packaging#274](https://github.com/docker/docker-ce-packaging/pull/274)
- Add libseccomp requirement for RPM packages. [docker/docker-ce-packaging#266](https://github.com/docker/docker-ce-packaging/pull/266)
### Runtime
* Add `/proc/asound` to masked paths. [docker/engine#126](https://github.com/docker/engine/pull/126)
* Update to containerd 1.2.1-rc.0. [docker/engine#121](https://github.com/docker/engine/pull/121)
+ Windows: allow process isolation. [docker/engine#81](https://github.com/docker/engine/pull/81)
- Windows: DetachVhd attempt in cleanup [docker/engine#113](https://github.com/docker/engine/pull/113)
- API: properly handle invalid JSON to return a 400 status. [docker/engine#110](https://github.com/docker/engine/pull/110)
- API: ignore default address-pools on API <1.39. [docker/engine#118](https://github.com/docker/engine/pull/118)
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
