Commit Graph

43087 Commits

Author SHA1 Message Date
4d26fa95ad Merge component 'engine' from git@github.com:docker/engine 18.09 2019-07-17 17:01:58 +00:00
67476cf36f Merge pull request #298 from thaJeztah/18.09_backport_scrub
[18.09 backport] DebugRequestMiddleware: unconditionally scrub data field
Upstream-commit: 456712c5b8d9d92c047f6a7d7cff270527ecac28
Component: engine
2019-07-17 09:09:51 -07:00
b48ef8cdca DebugRequestMiddleware: Remove path handling
Path-specific rules were removed, so this is no longer used.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 530e63c1a61b105a6f7fc143c5acb9b5cd87f958)
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit f8a0f26843bc5aff33cf9201b75bd4bdbb48a3ad)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a11c3098a3d5106b2d7c90d971c9300099556a49
Component: engine
2019-07-17 17:24:44 +02:00
3b1e458374 DebugRequestMiddleware: unconditionally scrub data field
Commit 77b8465d7e68ca102d7aae839c7b3fe0ecd28398 added a secret update
endpoint to allow updating labels on existing secrets. However, when
implementing the endpoint, the DebugRequestMiddleware was not updated
to scrub the Data field (as is being done when creating a secret).

When updating a secret (to set labels), the Data field should be either
`nil` (not set), or contain the same value as the existing secret. In
situations where the Data field is set, and the `dockerd` daemon is
running with debugging enabled / log-level debug, the base64-encoded
value of the secret is printed to the daemon logs.

The docker cli does not have a `docker secret update` command, but
when using `docker stack deploy`, the docker cli sends the secret
data both when _creating_ a stack, and when _updating_ a stack, thus
leaking the secret data if the daemon runs with debug enabled:

1. Start the daemon in debug-mode

        dockerd --debug

2. Initialize swarm

        docker swarm init

3. Create a file containing a secret

        echo secret > my_secret.txt

4. Create a docker-compose file using that secret

        cat > docker-compose.yml <<'EOF'
        version: "3.3"
        services:
          web:
            image: nginx:alpine
            secrets:
              - my_secret
        secrets:
          my_secret:
            file: ./my_secret.txt
        EOF

5. Deploy the stack

        docker stack deploy -c docker-compose.yml test

6. Verify that the secret is scrubbed in the daemon logs

        DEBU[2019-07-01T22:36:08.170617400Z] Calling POST /v1.30/secrets/create
        DEBU[2019-07-01T22:36:08.171364900Z] form data: {"Data":"*****","Labels":{"com.docker.stack.namespace":"test"},"Name":"test_my_secret"}

7. Re-deploy the stack to trigger an "update"

        docker stack deploy -c docker-compose.yml test

8. Notice that this time, the Data field is not scrubbed, and the base64-encoded secret is logged

        DEBU[2019-07-01T22:37:35.828819400Z] Calling POST /v1.30/secrets/w3hgvwpzl8yooq5ctnyp71v52/update?version=34
        DEBU[2019-07-01T22:37:35.829993700Z] form data: {"Data":"c2VjcmV0Cg==","Labels":{"com.docker.stack.namespace":"test"},"Name":"test_my_secret"}

This patch modifies `maskSecretKeys` to unconditionally scrub `Data` fields.
Currently, only the `secrets` and `configs` endpoints use a field with this
name, and no other POST API endpoints use a data field, so scrubbing this
field unconditionally will only scrub requests for those endpoints.

If a new endpoint is added in future where this field should not be scrubbed,
we can re-introduce more fine-grained (path-specific) handling.

This patch introduces some change in behavior:

- In addition to secrets, requests to create or update _configs_ will
  now have their `Data` field scrubbed. Generally, the actual data should
  not be interesting for debugging, so likely will not be problematic.
  In addition, scrubbing this data for configs may actually be desirable,
  because (even though they are not explicitely designed for this purpose)
  configs may contain sensitive data (credentials inside a configuration
  file, e.g.).
- Requests that send key/value pairs as a "map" and that contain a
  key named "data", will see the value of that field scrubbed. This
  means that (e.g.) setting a `label` named `data` on a config, will
  scrub/mask the value of that label.
- Note that this is already the case for any label named `jointoken`,
  `password`, `secret`, `signingcakey`, or `unlockkey`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c7ce4be93ae8edd2da62a588e01c67313a4aba0c)
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 73db8c77bfb2d0cbdf71ce491f3d3e66c9dd5be6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 32b40c53662e733b4627b0b303c71b52484a31f4
Component: engine
2019-07-17 17:24:35 +02:00
2294bf630c TestMaskSecretKeys: use subtests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 32d70c7e21631224674cd60021d3ec908c2d888c)
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit ebb542b3f88d7f5551f6b6e1d8d2774a2c166409)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1371b11749854515289abe6bcc0c1b0759ea7a5b
Component: engine
2019-07-17 17:24:23 +02:00
16da52903d TestMaskSecretKeys: add more test-cases
Add tests for

- case-insensitive matching of fields
- recursive masking

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit db5f811216e70bcb4a10e477c1558d6c68f618c5)
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 18dac2cf32faeaada3bd4e8e2bffa576ad4329fe)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 310770b6deae3ff2f244654b8e84c14576e38493
Component: engine
2019-07-17 17:24:14 +02:00
6b835ba41a Merge component 'packaging' from git@github.com:docker/docker-ce-packaging 18.09 2019-07-10 01:01:54 +00:00
531574846b [18.09 backport] require container-selinux >= 2.74 (#352)
[18.09 backport] require container-selinux >= 2.74
Upstream-commit: 2dfffcfb076dccf33ac345fb3475bc349d68459d
Component: packaging
2019-07-09 12:40:17 -10:00
d9a98a0f86 Merge component 'packaging' from git@github.com:docker/docker-ce-packaging 18.09 2019-07-06 21:01:21 +00:00
3c597067b9 [18.09] switch to balenalib/rpi-raspbian because resin/rpi-raspb… (#359)
[18.09] switch to balenalib/rpi-raspbian because resin/rpi-raspbian is deprecated
Upstream-commit: e8cb60baf6abb7b2cd50c9991056891be462b839
Component: packaging
2019-07-06 07:48:51 -10:00
b69692140e Merge component 'cli' from git@github.com:docker/cli 18.09 2019-07-04 00:46:01 +00:00
2f1c550ef2 Merge pull request #1978 from thaJeztah/18.09_backport_fix_rollback_config_interpolation
[18.09 backport] Fix Rollback config type interpolation
Upstream-commit: ec2ecc1c8b
Component: cli
2019-07-03 23:09:20 +02:00
5f9904bac8 Rollback config type interpolation on fields "parallelism" and "max_failure_ratio" were missing, as it uses the same type as update_config.
Signed-off-by: Silvin Lubecki <silvin.lubecki@docker.com>
(cherry picked from commit efdf36fa81)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 23c88a8311
Component: cli
2019-07-03 19:20:08 +02:00
88031e7c4e switch to balenalib/rpi-raspbian because resin/rpi-raspbian is deprecated
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 7ab3b5c11a0426117580b14e1b170b192945af1d
Component: packaging
2019-07-03 00:33:52 +02:00
2d0083d657 Merge pull request #611 from andrewhsu/1809ga
[18.09] bump ver 18.09.7
2019-06-27 10:13:33 -07:00
f53f261a1d bump ver 18.09.7
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-06-27 16:55:22 +00:00
52d814f953 Merge component 'packaging' from git@github.com:docker/docker-ce-packaging 18.09 2019-06-27 01:02:59 +00:00
ba3d612d8c Merge pull request #355 from thaJeztah/18.09_backport_remove_pkcs11
[18.09 backport] Remove the pkcs11 tag, was a holdover from cli
Upstream-commit: cc243930f1600d0a1a5621afe4bd3abe9c299c11
Component: packaging
2019-06-26 12:32:09 -10:00
df56d66f46 Remove the pkcs11 tag, was a holdover from cli
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 76f7249ca4695f58cee9773cb8b8036e698c576b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 095a4c109d5ba3444ace84e53f363eb4855e57d2
Component: packaging
2019-06-26 01:01:03 +02:00
0ac05a6a78 require container-selinux >= 2.74
version 2.9 is really old; this sets the same minimal version
as is used for the containerd.io package

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a246b19b0723d41a8e58ff42d3af52fd57a7e94d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 95dea749d4cb7e474194bbe209f052e94da5df2e
Component: packaging
2019-06-26 00:48:50 +02:00
e97b22c15d Merge pull request #610 from andrewhsu/1809rc1
[18.09] bump ver 18.09.7-rc1
2019-06-20 17:29:08 -07:00
28ef26b589 bump ver 18.09.7-rc1
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-06-20 22:48:21 +00:00
d70605fb20 Merge branch '18.09' of github.com:docker/docker-ce into 18.09 2019-06-20 05:00:56 +00:00
8aad3d72e2 Merge component 'engine' from git@github.com:docker/engine 18.09 2019-06-20 05:00:41 +00:00
3bc33de395 Merge component 'engine' from git@github.com:docker/engine 18.09 2019-06-20 04:59:41 +00:00
31e95a598d Merge pull request #286 from thaJeztah/18.09_backport_cp_slash_fix
[18.09 backport] Fix docker cp when container source path is /
Upstream-commit: c513a4c6c2989445efb75f84b5f62440d1a7bf0c
Component: engine
2019-06-19 18:40:54 -07:00
a3dc56fc6f Merge component 'engine' from git@github.com:docker/engine 18.09 2019-06-18 20:59:34 +00:00
819288aa31 Merge component 'cli' from git@github.com:docker/cli 18.09 2019-06-18 20:36:57 +00:00
6e8fdfff42 Merge pull request #195 from thaJeztah/18.09_backport_makefile_and_test_changes
[18.09 backport] Makefile and test changes
Upstream-commit: bad2185b9909c5083cdaf673d2d84a707d1927b9
Component: engine
2019-06-18 10:33:53 -07:00
83a77ecc35 Merge pull request #196 from thaJeztah/18.09_backport_plugin_partial
[18.09 backport] Adds PartialLogMetadata to encode protobuf for logger plugins
Upstream-commit: 10b63ee8ba1b51d4740f1bbb92be7bfe375d9ed3
Component: engine
2019-06-18 10:27:44 -07:00
2f921c9ec1 Merge pull request #203 from thaJeztah/18.09_backport_gcr_workaround
[18.09 backport] builder: add workaround for gcr auth issue
Upstream-commit: 6b2d2eb653e7e634852fad4fbf4ea0df86868109
Component: engine
2019-06-18 10:20:39 -07:00
ab19839d16 Merge pull request #221 from thaJeztah/18.09_backport_swarmnanocpu
[18.09 backport] Switch swarmmode services to NanoCpu
Upstream-commit: d1a30309de3f9e13d78779976e28ce52dfebdf2f
Component: engine
2019-06-18 10:09:46 -07:00
909fa7d737 Merge component 'engine' from git@github.com:docker/engine 18.09 2019-06-18 17:08:14 +00:00
c211b6a944 Merge pull request #186 from thaJeztah/18.09_backport_bump_cgroups
[18.09 backport] update containerd/cgroups 4994991857f9b0ae8dc439551e8bebdbb4bf66c1
Upstream-commit: af585fc188292c56d6df8f6cd557ba0e59947054
Component: engine
2019-06-18 10:02:21 -07:00
1e7ba22582 Merge pull request #190 from thaJeztah/18.09_backport_mirrors_validation
[18.09 backport] daemon: fix mirrors validation
Upstream-commit: a43a4ab30ec789a31ee3a14118bf02b7cc54cff0
Component: engine
2019-06-18 10:01:17 -07:00
728c223f90 Merge pull request #273 from thaJeztah/18.09_backport_entropy_cannot_be_saved
[18.09 backport] Entropy cannot be saved
Upstream-commit: 0ab832439e8ae2810e7a7ab2e2d7860bb6f45789
Component: engine
2019-06-18 10:00:05 -07:00
afc04c47ed Merge pull request #265 from thaJeztah/18.09_backport_do_not_order_uid_gid_mappings
[18.09 backport] Stop sorting uid and gid ranges in id maps
Upstream-commit: 047143abc38294668b1680c46e8ea0868838beb0
Component: engine
2019-06-18 09:56:53 -07:00
138981a207 Enable buildkit for Makefile build target
This is set only if it is not already set.
This should give a little speedup to CI builds.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 1275a001a68722494d090d5beca6749a83710cc2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e64cd6abed308a26d9048d4d0e4c52207c2ad5df
Component: engine
2019-06-18 17:53:32 +01:00
1247206ab1 hack: Have TIMEOUT take -test.count into account when testing for flakiness
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 42dcfc894a64e9b4c1751b21db1537b3b68a36d8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a9c1bfc1b1cd94968b399ffdddef4b908a722bb3
Component: engine
2019-06-18 17:53:24 +01:00
00d14a4862 CI: Introduce flaky test finder
comparing PR commit(s) to HEAD of moby/moby master branch and if founds
new (or renamed) integration tests will run stress tests for them.

Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
(cherry picked from commit 8a8fd37f6fb53716cb4b3a7e93e1e3cf385927e2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e5c0923b277055c977994ebfdc689346c3104a16
Component: engine
2019-06-18 17:53:21 +01:00
fbaec0bb9e Makefile: make help: fix newline wrapping, and missing targets
This patch;

- adds support for multiple newlines
- removes the 1-space indentation of wrapped lines
- allows numerical characters in targets (0-9)

Given these targets:

```Makefile
.PHONY: foobar
foobar: ## runs the foobar lorum ipsum.\nand so pn\nand so on
	echo foobar

.PHONY: e2e-tests
e2e-tests: ## runs the end-to-end tests
	echo e2e-tests
```

Before this change, the output of `make help` was

```
foobar               runs the foobar lorum ipsum.
                      and so pn\nand so on
```

After this change, the output is:

```
foobar               runs the foobar lorum ipsum.
                     and so pn
                     and so on
e2e-tests            runs the end-to-end tests
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 202c9d8c98614e7cce2017f5c99d3d783fe8b509)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 733b86683f347f922affc714d305d9f03fe9b47e
Component: engine
2019-06-18 17:53:17 +01:00
e6d06f7160 Use BuildKit to skip source code COPY if BIND_DIR set
build the final stage of the Dockerfile (including COPY ...) if no BIND_DIR
is used.
if BIND_DIR is used, build the dev stage, thus skipping the COPY.

Original author: @thaJeztah

Signed-off-by: Mohammad Nasirifar <farnasirim@gmail.com>
(cherry picked from commit e6d7df2e5d313800414b955e10a26d6687e7a1bf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e5a039169f389120be3fbc2a9db45ef8bc99eae5
Component: engine
2019-06-18 17:53:12 +01:00
7e0220b6d6 Allow to override the Makefile's DOCKER_MOUNT variable
Through the env variable of the same name.

The idea here is pretty simple: I/O perf on native mounted disks
on non-Linux (notably Mac OS) is just terrible, thus making it
a real pain to develop: one has to choose between re-building
the image after every single change (eg to run a test) or just
work directly inside the same container (eg with vim, but even then
one would have to re-configure their dev container every time
it gets destroyed - containers, after all, are not supposed to
be long-lived).

Allowing to override `DOCKER_MOUNT` makes it easy for everyone
to decide what their volume/syncing strategy is; for example
one can choose to use [docker-sync](https://github.com/EugenMayer/docker-sync)

This patch won't change anything for anyone who doesn't
set the `DOCKER_MOUNT` env variable in their environment.

Signed-off-by: Jean Rouge <jer329@cornell.edu>
(cherry picked from commit aea6fdf3d340835a1b0af208839ce42ace3a5b89)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ecc423df7fde7f0edc963f03d70da133f03bb63c
Component: engine
2019-06-18 17:53:08 +01:00
36954599b4 go {build,test}: rm -i option, add go cache volume
Looks like -i (together with DOCKER_INCREMENTAL_BINARY etc)
were used to get faster incremental builds.

Nowdays (since Go 1.10) this is no longer the case, as
go build cache is used [1]. Here's a quote:

> You do not have to use "go test -i" or "go build -i" or
> "go install" just to get fast incremental builds. We will
> not have to teach new users those workarounds anymore.
> Everything will just be fast.

To enable go cache between builds, add a volume for /root/.cache.

[1] https://groups.google.com/forum/#!msg/golang-dev/qfa3mHN4ZPA/X2UzjNV1BAAJ

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit bdcd81d3301a053eefc320de16ac842ec47ed459)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0f190f798f7b9f90bd008fe8fb0fc087ec965642
Component: engine
2019-06-18 17:53:03 +01:00
b083df35c8 Merge pull request #1842 from thaJeztah/18.09_bump_buildkit_18.09
[18.09 backport] bump buildkit 05766c5c21a1e528eeb1c3522b2f05493fe9ac47 (docker-18.09 branch)
Upstream-commit: 3a749342a3
Component: cli
2019-06-18 09:49:42 -07:00
800f827460 Merge pull request #199 from thaJeztah/18.09_backport_fix_panic_on_empty_dockerfile
[18.09 backport] fix panic on empty dockerfile
Upstream-commit: 70399c41d321651121a4eb2775c4e7f63796426f
Component: engine
2019-06-18 09:49:05 -07:00
6cdb81566e Merge pull request #202 from thaJeztah/18.09_backport_update_docker_py
[18.09 backport] Update docker-py to 3.7.0
Upstream-commit: 5749d5ae7967038eecd974767142d27652bbbc2f
Component: engine
2019-06-18 09:46:14 -07:00
14405a174e Merge pull request #230 from thaJeztah/18.09_backport_windows_tag
[18.09 backport] Consider WINDOWS_BASE_IMAGE_TAG override when setting Windows base image for tests
Upstream-commit: bb54c5bf2a3ab1c698fda6b3d67773d25755ae45
Component: engine
2019-06-18 09:44:44 -07:00
aa8e14244e Merge pull request #236 from thaJeztah/18.09_backport_thanks_brian_now_im_hungry
[18.09 backport] Fix error handling for bind mount spec parser.
Upstream-commit: 41fbd152739faa225db8c9913e34c663b9202101
Component: engine
2019-06-18 09:43:19 -07:00
28df654a77 Merge pull request #194 from thaJeztah/18.09_backport_bump_containerd_v1.2.6
[18.09 backport] Bump containerd v1.2.6, runc v1.0.0-rc8
Upstream-commit: 3bacaaded9e086ec06fc335146eda614e1110570
Component: engine
2019-06-18 09:41:42 -07:00