In case Tail=N parameter is requested, we need to show N lines.
It does not make sense to walk backwards one by one if we can
do it at once. Now, if Since=T is also provided, make sure we
haven't jumped too far (before T), and if we did, move forward.
The primary motivation for this was to make the code simpler.
This also fixes a tiny bug in the "since" implementation.
Before this commit:
> $ docker logs -t --tail=6000 --since="2019-03-10T03:54:25.00" $ID | head
> 2019-03-10T03:54:24.999821000Z 95981
After:
> $ docker logs -t --tail=6000 --since="2019-03-10T03:54:25.00" $ID | head
> 2019-03-10T03:54:25.000013000Z 95982
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit ff3cd167ea4d089b7695a263ba2fc4caa0a0750c)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 2a124db7da440f1efd4c2957320d8b25d9d9ce36
Component: engine
Minor code simplification.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit e8f6166791c097deb15c39f8dddf6f97be65b224)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 1d336dc53bd7bf5941596ffeb253d102de609a51
Component: engine
Clean up a deferred function call in the journal reading logic.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
(cherry picked from commit 1ada3e85bf89201910c28f2ff6892c00cee0f137)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: e700930ca521d0c004b6a3ed8bdd35a2d538aa15
Component: engine
As in other similar drivers (jsonlog, local), use a set
(i.e. `map[whatever]struct{}`), making the code simpler.
While at it, make sure we remove the reader from the set
after calling `ProducerGone()` on it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit b2b169f13f681cd0d591ccb06d6cfff97933db77)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: fe85c72a2eac4cbf249d2c4c754684bb447eefdd
Component: engine
There are a few more places, apparently, that List operations against
Swarm exist, besides just in the List methods. This increases the max
received message size in those places.
Signed-off-by: Drew Erny <drew.erny@docker.com>
(cherry picked from commit a84a78e9767d82abd4744dad9ce4fb3f64141a8f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 41da428d065ab15fd2d8aba1fbd0d0056136a43b
Component: engine
full diff: 19e791fd6d...142a73731c
included:
- docker/swarmkit#2872 [19.03 backport] Only update non-terminal tasks on node removal
- backport of docker/swarmkit#2867 Only update non-terminal tasks on node removal
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 7fcfdbaab6d8c6a2d55ad7b72a851c5e92b360ac
Component: engine
The Golang base images switch to buster, which causes some breakage
in networking and packages that are no longer available; (`btrfs-tools`
is now an empty package, and `libprotobuf-c0-dev` is gone).
Some of out tests also start faiilng on stretch, and will have to be
investigated further;
```
15:13:06 --- FAIL: TestRenameAnonymousContainer (3.37s)
15:13:06 rename_test.go:168: assertion failed: 0 (int) != 1 (inspect.State.ExitCode int): container a7fe866d588d65f353f42ffc5ea5288e52700384e1d90850e9c3d4dce8657666 exited with the wrong exitcode:
15:13:38 --- FAIL: TestHostnameDnsResolution (2.23s)
15:13:38 run_linux_test.go:128: assertion failed:
15:13:38 --- ←
15:13:38 +++ →
15:13:38 @@ -1 +1,2 @@
15:13:38 +ping: bad address 'foobar'
15:13:38
15:13:38
15:13:38 run_linux_test.go:129: assertion failed: 0 (int) != 1 (res.ExitCode int)
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ed672bb523cb255d0b2b79837d9c45a7c3255000)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 3c1bc29b2c372d0f1e37dc8aac33cdc01b623aa0
Component: engine
This check was used to make sure we don't bump Go versions independently
(Linux/Windows). The Dockerfile switched to using a build-arg to allow
overriding the Go version, which rendered this check non-functional.
It also fails if Linux versions use a specific variant of the image;
08:41:31 ERROR: Failed 'ERROR: Mismatched GO versions between Dockerfile and Dockerfile.windows. Update your PR to ensure that both files are updated and in sync. ${GO_VERSION}-stretch ${GO_VERSION}' at 07/20/2019 08:41:31
08:41:31 At C:\gopath\src\github.com\docker\docker\hack\ci\windows.ps1:448 char:9
08:41:31 + Throw "ERROR: Mismatched GO versions between Dockerfile and D ...
08:41:31 + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This patch fixes the check by looking for the value of `GO_VERSION` instead
of looking at the `FROM` line (which is harder to parse).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4fa57a8191b1d23c6466725b688519f83c0ac5dd)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: a2d32c5654a206954fd0d9a5fd08abd8c03d4523
Component: engine
This was an oversight when changing the Dockerfile to use a build-arg;
the Windows Dockerfile downloads the Go binaries, which never have a
trailing `.0`.
This patch makes sure that the trailing zero (if any) is removed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c5bd6e3dc7680d6c683496f63dafb1f30f87eaa7)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 8f1a27c51f93698b5fe17f0fb043f47794e11cca
Component: engine
This allows overriding the version of Go without making modifications in the
source code, which can be useful to test against multiple versions.
For example:
make GO_VERSION=1.13beta1 shell
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c6281bc4383b7f9eab617fd73601e8594c93365b)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Conflicts:
* Dockerfile, Dockerfile.e2e, Dockerfile.simple, Dockerfile.windows:
(due to Go version difference, missing CROSS etc.)
Upstream-commit: d9ba337adb0c3880acd34ffe9b0a15717990876b
Component: engine
Protect access to q.quotas map, and lock around changing nextProjectID.
Techinically, the lock in findNextProjectID() is not needed as it is
only called during initialization, but one can never be too careful.
Fixes: 52897d1c092 ("projectquota: utility class for project quota controls")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 1ac0a66a64a906911d0708cd0e5fa397a2f0b595)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 7027bb9bedae63879c1e41894739ba0ea2deedc1
Component: engine
[18.09] Initialize nss libraries in Glibc so that the dynamic libraries are l…
Upstream-commit: 8ff992ae26ecedb7b80ee71be902d0fbc0879861
Component: engine
The fastly cdn mirror we're using also mirrors the debian security
repository;
```
Welcome to deb.debian.org (fastly instance)!
This is deb.debian.org. This service provides mirrors for the following Debian archive repositories:
/debian/
/debian-debug/
/debian-ports/
/debian-security/
The server deb.debian.org does not have packages itself, but the name has SRV records in DNS that let apt in stretch and later find places.
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c8f43b5f6f7c83cfb5570f7f013c7efaa430d285)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Conflicts:
* Dockerfile (GO_VERSION value differs, and CROSS (#39010) is absent)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 6e603e957eec3a46db4d8d69c3a7946c4d2e9c59
Component: engine
As of Alpine Linux 3.3 (or 3.2?) there exists a new --no-cache
option for apk. It allows users to install packages with an index
that is updated and used on-the-fly and not cached locally.
This avoids the need to use --update and remove /var/cache/apk/*
when done installing packages.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 3b44dd66a4b316049df86521c491b50466d9b55d)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: a4ae2af0e59018a309d4bc71ced729b34e906564
Component: engine
Path-specific rules were removed, so this is no longer used.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 530e63c1a61b105a6f7fc143c5acb9b5cd87f958)
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit f8a0f26843bc5aff33cf9201b75bd4bdbb48a3ad)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a11c3098a3d5106b2d7c90d971c9300099556a49
Component: engine
Commit 77b8465d7e68ca102d7aae839c7b3fe0ecd28398 added a secret update
endpoint to allow updating labels on existing secrets. However, when
implementing the endpoint, the DebugRequestMiddleware was not updated
to scrub the Data field (as is being done when creating a secret).
When updating a secret (to set labels), the Data field should be either
`nil` (not set), or contain the same value as the existing secret. In
situations where the Data field is set, and the `dockerd` daemon is
running with debugging enabled / log-level debug, the base64-encoded
value of the secret is printed to the daemon logs.
The docker cli does not have a `docker secret update` command, but
when using `docker stack deploy`, the docker cli sends the secret
data both when _creating_ a stack, and when _updating_ a stack, thus
leaking the secret data if the daemon runs with debug enabled:
1. Start the daemon in debug-mode
dockerd --debug
2. Initialize swarm
docker swarm init
3. Create a file containing a secret
echo secret > my_secret.txt
4. Create a docker-compose file using that secret
cat > docker-compose.yml <<'EOF'
version: "3.3"
services:
web:
image: nginx:alpine
secrets:
- my_secret
secrets:
my_secret:
file: ./my_secret.txt
EOF
5. Deploy the stack
docker stack deploy -c docker-compose.yml test
6. Verify that the secret is scrubbed in the daemon logs
DEBU[2019-07-01T22:36:08.170617400Z] Calling POST /v1.30/secrets/create
DEBU[2019-07-01T22:36:08.171364900Z] form data: {"Data":"*****","Labels":{"com.docker.stack.namespace":"test"},"Name":"test_my_secret"}
7. Re-deploy the stack to trigger an "update"
docker stack deploy -c docker-compose.yml test
8. Notice that this time, the Data field is not scrubbed, and the base64-encoded secret is logged
DEBU[2019-07-01T22:37:35.828819400Z] Calling POST /v1.30/secrets/w3hgvwpzl8yooq5ctnyp71v52/update?version=34
DEBU[2019-07-01T22:37:35.829993700Z] form data: {"Data":"c2VjcmV0Cg==","Labels":{"com.docker.stack.namespace":"test"},"Name":"test_my_secret"}
This patch modifies `maskSecretKeys` to unconditionally scrub `Data` fields.
Currently, only the `secrets` and `configs` endpoints use a field with this
name, and no other POST API endpoints use a data field, so scrubbing this
field unconditionally will only scrub requests for those endpoints.
If a new endpoint is added in future where this field should not be scrubbed,
we can re-introduce more fine-grained (path-specific) handling.
This patch introduces some change in behavior:
- In addition to secrets, requests to create or update _configs_ will
now have their `Data` field scrubbed. Generally, the actual data should
not be interesting for debugging, so likely will not be problematic.
In addition, scrubbing this data for configs may actually be desirable,
because (even though they are not explicitely designed for this purpose)
configs may contain sensitive data (credentials inside a configuration
file, e.g.).
- Requests that send key/value pairs as a "map" and that contain a
key named "data", will see the value of that field scrubbed. This
means that (e.g.) setting a `label` named `data` on a config, will
scrub/mask the value of that label.
- Note that this is already the case for any label named `jointoken`,
`password`, `secret`, `signingcakey`, or `unlockkey`.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c7ce4be93ae8edd2da62a588e01c67313a4aba0c)
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 73db8c77bfb2d0cbdf71ce491f3d3e66c9dd5be6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 32b40c53662e733b4627b0b303c71b52484a31f4
Component: engine
Full diff: https://github.com/golang/go/compare/go1.11.5...go1.11.9
go1.11.6 (released 2019/03/14) includes fixes to cgo, the compiler, linker,
runtime, go command, and the crypto/x509, encoding/json, net, and net/url
packages. See the Go 1.11.6 milestone on our issue tracker for details.
https://github.com/golang/go/issues?q=milestone%3AGo1.11.6
go1.11.7 (released 2019/04/05) includes fixes to the runtime and the net
packages. See the Go 1.11.7 milestone on our issue tracker for details.
https://github.com/golang/go/issues?q=milestone%3AGo1.11.7
go1.11.8 (released 2019/04/08) was accidentally released without its
intended fix. It is identical to go1.11.7, except for its version number.
The intended fix is in go1.11.9.
go1.11.9 (released 2019/04/11) fixes an issue where using the prebuilt
binary releases on older versions of GNU/Linux led to failures when linking
programs that used cgo. Only Linux users who hit this issue need to update.
See golang/go#31293 for details
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2cc1df06ee264ab5d1769bd4aae9e417feda9402
Component: engine
go1.11.4 (released 2018/12/14) includes fixes to cgo, the compiler, linker,
runtime, documentation, go command, and the net/http and go/types packages. It
includes a fix to a bug introduced in Go 1.11.3 that broke go get for import
path patterns containing "...".
See the Go 1.11.4 milestone for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.4+label%3ACherryPickApproved
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3770f386477e5eea4e6ba02516393a1edd5cd28b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e05367a1ba3cbf2cd7f99fef73908b900b57b929
Component: engine
go1.11.13 (released 2018/12/14)
- crypto/x509: CPU denial of service in chain validation golang/go#29233
- cmd/go: directory traversal in "go get" via curly braces in import paths golang/go#29231
- cmd/go: remote command execution during "go get -u" golang/go#29230
See the Go 1.11.3 milestone on the issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6b7c093b0de21d574ce120aee891e60187749174)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 19d37c9a337e82e0e8ce0ff28271739e8ec78e83
Component: engine
Includes fixes to the compiler, linker, documentation, go command, and the
database/sql and go/types packages. See the Go 1.11.2 milestone on the issue
tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e80ee5206e6e62521c8149381d06bbb0bea88169)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f9a71f917f0979742b1d7ee606fb596e873d773c
Component: engine
go1.11.1 (released 2018/10/01) includes fixes to the compiler,
documentation, go command, runtime, and the crypto/x509, encoding/json,
go/types, net, net/http, and reflect packages.
See the Go 1.11.1 milestone on our issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.1
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0347751117513312be72e8b0d03f32319027f145)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 6b397d155bf464a94fa05620a485d969edf42393
Component: engine
As per https://github.com/etcd-io/etcd/blob/fa57f7fbc787b4/Gopkg.lock
List of packages required by subset of etcd used is provided by:
go list -f '{{join .Deps "\n"}}' \
github.com/docker/docker/vendor/github.com/coreos/etcd/... \
| grep -F . | grep -v coreos/etcd | sort | uniq
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 7008ac01fa4dbde9a069cc647a09c982f3648fc3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c4bf0803154de1b26ee3437c7fd9307722e2a5e4
Component: engine
We can do that now as we're no longer carrying archive/tar.
Note that latest vndr removes vendor/ subdir so we don't have to,
thus the change in hack/validate/vendor.
While at it, re-run a new vndr version to make sure everything
that should be there is.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit ce858feb6ac124b3a42400b857e480ef98710164)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2f2588f137f642b0c1520e8a41ec731040d0b336
Component: engine
It's that time of year again! Go 1.11 is released, time to use it.
This commit also
* removes our archive/tar fork, since upstream archive/tar
is fixed for static builds, and osusergo build tag is set.
* removes ENV GO_VERSION from Dockerfile as it's not needed
anymore since PR #37592 is merged.
[v2: switch to beta2]
[v3: switch to beta3]
[v4: rc1]
[v5: remove ENV GO_VERSION as PR #37592 is now merged]
[v6: rc2]
[v7: final!]
[v8: use 1.11.0]
[v9: back to 1.11]
[v8: use 1.11.0]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 10fd0516b9f9f04d0f0e2c0755e704303f1a487f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ebf396050d6e977df2669d5e7d3f38098719f7c2
Component: engine
We would like to use a version with .0 suffix (like 1.11.0) in
Dockerfile, so that once a .1 version is out (like 1.11.1) we
won't accidentally switch to it.
Unfortunately it's not possible to use .0 suffix currently
as it breaks the check in make.ps1. This patch fixes that.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 8ad648b59a2993156586458cc9ff8dcbc89b86e4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: dc5371b4134dd6474a467308da85daaa27914a64
Component: engine
