Commit Graph

33989 Commits

Author SHA1 Message Date
31e95a598d Merge pull request #286 from thaJeztah/18.09_backport_cp_slash_fix
[18.09 backport] Fix docker cp when container source path is /
Upstream-commit: c513a4c6c2989445efb75f84b5f62440d1a7bf0c
Component: engine
2019-06-19 18:40:54 -07:00
6e8fdfff42 Merge pull request #195 from thaJeztah/18.09_backport_makefile_and_test_changes
[18.09 backport] Makefile and test changes
Upstream-commit: bad2185b9909c5083cdaf673d2d84a707d1927b9
Component: engine
2019-06-18 10:33:53 -07:00
83a77ecc35 Merge pull request #196 from thaJeztah/18.09_backport_plugin_partial
[18.09 backport] Adds PartialLogMetadata to encode protobuf for logger plugins
Upstream-commit: 10b63ee8ba1b51d4740f1bbb92be7bfe375d9ed3
Component: engine
2019-06-18 10:27:44 -07:00
2f921c9ec1 Merge pull request #203 from thaJeztah/18.09_backport_gcr_workaround
[18.09 backport] builder: add workaround for gcr auth issue
Upstream-commit: 6b2d2eb653e7e634852fad4fbf4ea0df86868109
Component: engine
2019-06-18 10:20:39 -07:00
ab19839d16 Merge pull request #221 from thaJeztah/18.09_backport_swarmnanocpu
[18.09 backport] Switch swarmmode services to NanoCpu
Upstream-commit: d1a30309de3f9e13d78779976e28ce52dfebdf2f
Component: engine
2019-06-18 10:09:46 -07:00
c211b6a944 Merge pull request #186 from thaJeztah/18.09_backport_bump_cgroups
[18.09 backport] update containerd/cgroups 4994991857f9b0ae8dc439551e8bebdbb4bf66c1
Upstream-commit: af585fc188292c56d6df8f6cd557ba0e59947054
Component: engine
2019-06-18 10:02:21 -07:00
1e7ba22582 Merge pull request #190 from thaJeztah/18.09_backport_mirrors_validation
[18.09 backport] daemon: fix mirrors validation
Upstream-commit: a43a4ab30ec789a31ee3a14118bf02b7cc54cff0
Component: engine
2019-06-18 10:01:17 -07:00
728c223f90 Merge pull request #273 from thaJeztah/18.09_backport_entropy_cannot_be_saved
[18.09 backport] Entropy cannot be saved
Upstream-commit: 0ab832439e8ae2810e7a7ab2e2d7860bb6f45789
Component: engine
2019-06-18 10:00:05 -07:00
afc04c47ed Merge pull request #265 from thaJeztah/18.09_backport_do_not_order_uid_gid_mappings
[18.09 backport] Stop sorting uid and gid ranges in id maps
Upstream-commit: 047143abc38294668b1680c46e8ea0868838beb0
Component: engine
2019-06-18 09:56:53 -07:00
138981a207 Enable buildkit for Makefile build target
This is set only if it is not already set.
This should give a little speedup to CI builds.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 1275a001a68722494d090d5beca6749a83710cc2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e64cd6abed308a26d9048d4d0e4c52207c2ad5df
Component: engine
2019-06-18 17:53:32 +01:00
1247206ab1 hack: Have TIMEOUT take -test.count into account when testing for flakiness
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 42dcfc894a64e9b4c1751b21db1537b3b68a36d8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a9c1bfc1b1cd94968b399ffdddef4b908a722bb3
Component: engine
2019-06-18 17:53:24 +01:00
00d14a4862 CI: Introduce flaky test finder
comparing PR commit(s) to HEAD of moby/moby master branch and if founds
new (or renamed) integration tests will run stress tests for them.

Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
(cherry picked from commit 8a8fd37f6fb53716cb4b3a7e93e1e3cf385927e2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e5c0923b277055c977994ebfdc689346c3104a16
Component: engine
2019-06-18 17:53:21 +01:00
fbaec0bb9e Makefile: make help: fix newline wrapping, and missing targets
This patch;

- adds support for multiple newlines
- removes the 1-space indentation of wrapped lines
- allows numerical characters in targets (0-9)

Given these targets:

```Makefile
.PHONY: foobar
foobar: ## runs the foobar lorum ipsum.\nand so pn\nand so on
	echo foobar

.PHONY: e2e-tests
e2e-tests: ## runs the end-to-end tests
	echo e2e-tests
```

Before this change, the output of `make help` was

```
foobar               runs the foobar lorum ipsum.
                      and so pn\nand so on
```

After this change, the output is:

```
foobar               runs the foobar lorum ipsum.
                     and so pn
                     and so on
e2e-tests            runs the end-to-end tests
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 202c9d8c98614e7cce2017f5c99d3d783fe8b509)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 733b86683f347f922affc714d305d9f03fe9b47e
Component: engine
2019-06-18 17:53:17 +01:00
e6d06f7160 Use BuildKit to skip source code COPY if BIND_DIR set
build the final stage of the Dockerfile (including COPY ...) if no BIND_DIR
is used.
if BIND_DIR is used, build the dev stage, thus skipping the COPY.

Original author: @thaJeztah

Signed-off-by: Mohammad Nasirifar <farnasirim@gmail.com>
(cherry picked from commit e6d7df2e5d313800414b955e10a26d6687e7a1bf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e5a039169f389120be3fbc2a9db45ef8bc99eae5
Component: engine
2019-06-18 17:53:12 +01:00
7e0220b6d6 Allow to override the Makefile's DOCKER_MOUNT variable
Through the env variable of the same name.

The idea here is pretty simple: I/O perf on native mounted disks
on non-Linux (notably Mac OS) is just terrible, thus making it
a real pain to develop: one has to choose between re-building
the image after every single change (eg to run a test) or just
work directly inside the same container (eg with vim, but even then
one would have to re-configure their dev container every time
it gets destroyed - containers, after all, are not supposed to
be long-lived).

Allowing to override `DOCKER_MOUNT` makes it easy for everyone
to decide what their volume/syncing strategy is; for example
one can choose to use [docker-sync](https://github.com/EugenMayer/docker-sync)

This patch won't change anything for anyone who doesn't
set the `DOCKER_MOUNT` env variable in their environment.

Signed-off-by: Jean Rouge <jer329@cornell.edu>
(cherry picked from commit aea6fdf3d340835a1b0af208839ce42ace3a5b89)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ecc423df7fde7f0edc963f03d70da133f03bb63c
Component: engine
2019-06-18 17:53:08 +01:00
36954599b4 go {build,test}: rm -i option, add go cache volume
Looks like -i (together with DOCKER_INCREMENTAL_BINARY etc)
were used to get faster incremental builds.

Nowdays (since Go 1.10) this is no longer the case, as
go build cache is used [1]. Here's a quote:

> You do not have to use "go test -i" or "go build -i" or
> "go install" just to get fast incremental builds. We will
> not have to teach new users those workarounds anymore.
> Everything will just be fast.

To enable go cache between builds, add a volume for /root/.cache.

[1] https://groups.google.com/forum/#!msg/golang-dev/qfa3mHN4ZPA/X2UzjNV1BAAJ

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit bdcd81d3301a053eefc320de16ac842ec47ed459)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 0f190f798f7b9f90bd008fe8fb0fc087ec965642
Component: engine
2019-06-18 17:53:03 +01:00
800f827460 Merge pull request #199 from thaJeztah/18.09_backport_fix_panic_on_empty_dockerfile
[18.09 backport] fix panic on empty dockerfile
Upstream-commit: 70399c41d321651121a4eb2775c4e7f63796426f
Component: engine
2019-06-18 09:49:05 -07:00
6cdb81566e Merge pull request #202 from thaJeztah/18.09_backport_update_docker_py
[18.09 backport] Update docker-py to 3.7.0
Upstream-commit: 5749d5ae7967038eecd974767142d27652bbbc2f
Component: engine
2019-06-18 09:46:14 -07:00
14405a174e Merge pull request #230 from thaJeztah/18.09_backport_windows_tag
[18.09 backport] Consider WINDOWS_BASE_IMAGE_TAG override when setting Windows base image for tests
Upstream-commit: bb54c5bf2a3ab1c698fda6b3d67773d25755ae45
Component: engine
2019-06-18 09:44:44 -07:00
aa8e14244e Merge pull request #236 from thaJeztah/18.09_backport_thanks_brian_now_im_hungry
[18.09 backport] Fix error handling for bind mount spec parser.
Upstream-commit: 41fbd152739faa225db8c9913e34c663b9202101
Component: engine
2019-06-18 09:43:19 -07:00
28df654a77 Merge pull request #194 from thaJeztah/18.09_backport_bump_containerd_v1.2.6
[18.09 backport] Bump containerd v1.2.6, runc v1.0.0-rc8
Upstream-commit: 3bacaaded9e086ec06fc335146eda614e1110570
Component: engine
2019-06-18 09:41:42 -07:00
b0ff7b744e integration: have container.Create call compile
For reference on why this is needed:
https://github.com/docker/engine/pull/280#issuecomment-502056661

Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 8f4b96f19e64b96df9d8c43208cefb113715ccbf)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 584c0857ab21895e62feac686448085113c6c977
Component: engine
2019-06-18 17:28:54 +01:00
6532144af4 pkg/archive: keep walkRoot clean if source is /
Previously, getWalkRoot("/", "foo") would return "//foo"
Now it returns "/foo"

Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 7410f1a859063d4ed3d8fca44f27bdde4c2cb5a3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8677bbe3f31a8e4516e1ba413ce1063c803ea10c
Component: engine
2019-06-18 14:44:26 +01:00
36f18a1f78 daemon: fix docker cp when container source is /
Before 7a7357da, archive.TarResourceRebase was being used to copy files
and folders from the container. That function splits the source path
into a dirname + basename pair to support copying a file:
if you wanted to tar `dir/file` it would tar from `dir` the file `file`
(as part of the IncludedFiles option).

However, that path splitting logic was kept for folders as well, which
resulted in weird inputs to archive.TarWithOptions:
if you wanted to tar `dir1/dir2` it would tar from `dir1` the directory
`dir2` (as part of IncludedFiles option).

Although it was weird, it worked fine until we started chrooting into
the container rootfs when doing a `docker cp` with container source set
to `/` (cf 3029e765).

The fix is to only do the path splitting logic if the source is a file.

Unfortunately, 7a7357da added support for LCOW by duplicating some of
this subtle logic. Ideally we would need to do more refactoring of the
archive codebase to properly encapsulate these behaviors behind well-
documented APIs.

This fix does not do that. Instead, it fixes the issue inline.

Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 171538c190ee3a1a8211946ab8fa78cdde54b47a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 70a837138aa1c5c0260a005b470ac0c5717984b4
Component: engine
2019-06-18 14:44:15 +01:00
9f18b94c1b add more tests
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 02f1eb89a42b4a9e91a8c80c213f7dc3193c75b9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c820334ff3e048d1207a63ba9d5b9d11e38f649d
Component: engine
2019-06-18 14:44:04 +01:00
2bf8bffc5b Add test for copying entire container rootfs
CID=$(docker create alpine)
docker cp $CID:/ out

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 6db9f1c3d6e9ad634554cacaf197a435efcf8833)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8fd0d71d7e4b03a815f2fb86b19ef2a71a1b7ce0
Component: engine
2019-06-18 14:43:49 +01:00
a3041ba7dd Test: dockerfiles with no instructions are detected
Signed-off-by: Natasha Jarus <linuxmercedes@gmail.com>
(cherry picked from commit 18c7e8b927928654a67ce5d6637e42932fc3e7df)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: cd084b23f7c3ca38c5d9c15ec16e119d25e5d7af
Component: engine
2019-06-18 13:44:08 +01:00
a439056531 Convert parse errors to more informative format
- Wrap parse errors in errdefs.InvalidParameters
- Include dockerfile in error names

Signed-off-by: Natasha Jarus <linuxmercedes@gmail.com>
(cherry picked from commit 64466b0cd9ff36dc3f123a3a68eae438ac9e8e60)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 7eb0e6a095ac71bfe3e3419cef7d5867fef55f20
Component: engine
2019-06-18 13:44:04 +01:00
cbbe578ba2 bump buildkit 05766c5c21a1e528eeb1c3522b2f05493fe9ac47 (docker-18.09 branch)
- moby/buildkit#952 [18.09 backport] Have parser error on dockerfiles without instructions
  - backport of moby/buildkit#771 Have parser error on dockerfiles without instructions

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a1d76e0585c498f5167b28b9909fd8c88fd3370d
Component: engine
2019-06-18 13:43:59 +01:00
0addc7f6c5 Update docker-py to 3.7.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c0c05affc79b199248b457af16fff61c305b7623)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b92e9e9da9519efa38d8a5927af668413cf81cd7
Component: engine
2019-06-18 13:42:20 +01:00
932cc247c5 Entropy cannot be saved
Remove non cryptographic randomness.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
(cherry picked from commit 2df693e533e904f432c59279c07b2b8cbeece4f0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 292b43b15b68cd4b64bfc7b89452dc19ddf2cf48
Component: engine
2019-06-18 13:38:50 +01:00
8642bba6c4 Network not deleted after stack is removed
Make sure adapter.removeNetworks executes during task Remove
adapter.removeNetworks was being skipped for cases when
isUnknownContainer(err) was true after adapter.remove was executed

This fix eliminates the nil return case forcing the function
to continue executing unless there is a true error

Fixes https://github.com/moby/moby/issues/39225

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
(cherry picked from commit 70fa7b6a3fd9aaada582ae02c50710f218b54d1a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 75887d37e1ddbef579e239ff0b1b7a2508e486fd
Component: engine
2019-06-18 13:34:02 +01:00
ac643ab134 Remove a network during task SHUTDOWN instead of REMOVE to
make sure the LB sandbox is removed when a service is updated
with a --network-rm option

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
(cherry picked from commit 680d0ba4abfb00c8ac959638c72003dba0826b46)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 280b8dff7df137dd21fcf27cacf6d12ed531770a
Component: engine
2019-06-18 13:33:58 +01:00
e00a39ba2b Move serviceRunningTasksCount to integration/internal/swarm
This fix moves multiple places of serviceRunningTasksCount
to one location in integration/internal/swarm, so that
code duplication could be removed.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
(cherry picked from commit e485a60e2bcc59860f387c94f6afaa0130ea7040)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c087f681d40332bd7d158baf615c03f6a72274f1
Component: engine
2019-06-18 13:33:14 +01:00
e91395081e integration: Corrected service update tests logic
Tests which will re-deploy containers uses function serviceIsUpdated() to
make sure that service update really reached state UpdateStateCompleted.

Tests which will not re-deploy container uses function
serviceSpecIsUpdated to make sure that service version is increased.

Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
(cherry picked from commit b868ada474b5c214ed9bddb792dd36f794dc1fa6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: ccc1abea092a54d99dec51345ef9303b4629572d
Component: engine
2019-06-18 13:33:06 +01:00
9d41ea2d82 integration: wait for service update to be completed
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8edcd4c3cd294bf276ffbe29bc58afbb006593b8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2ae0365c99627343d001a24e171fba2aa0fea791
Component: engine
2019-06-18 13:33:02 +01:00
29d8b5118f migrated service integration tests from integration-cli/docker_cli_service_update_test.go to integration/service
Signed-off-by: Arash Deshmeh <adeshmeh@ca.ibm.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit be151a73f0c0f362ff9775d2ff4d32329c01b8ba)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 4fe4e891723569fde7703747903289883db1dec0
Component: engine
2019-06-18 13:32:59 +01:00
630cd7d11b Merge pull request #257 from thaJeztah/18.09_backport_increase_swarmkit_grpc
[18.09 backport] Increase max recv gRPC message size for nodes and secrets
Upstream-commit: ce2e2b672a7a3510419cb2706edab3fd7f12fd6c
Component: engine
2019-06-17 12:31:27 -07:00
d25b6ea6fd Merge pull request #244 from thaJeztah/18.09_backport_fix_copying
[18.09 backport] image: do actual RootFS.DiffIDs copying in Clone()
Upstream-commit: 423f39ab05d121e1aeb6e59732386fb50c17f49b
Component: engine
2019-06-17 12:30:32 -07:00
2bfd8549de Merge pull request #218 from thaJeztah/18.09_backport_EDGE374_TestDaemonNoSpaceLeftOnDeviceError
[18.09 backport] explicitly set filesystem type for mount to avoid 'invalid argument' error on arm
Upstream-commit: 51ebfcbe423c679f06cfb37f7e6be565e0955639
Component: engine
2019-06-17 12:28:35 -07:00
eae44d6a4c Merge pull request #223 from thaJeztah/18.09_backport_devno
[18.09 backport] bugfix: fetch the right device number which great than 255
Upstream-commit: b236a1e78d35a62b164d00f4e19f59a556da8831
Component: engine
2019-06-17 12:27:17 -07:00
1b706a11a2 Merge pull request #276 from thaJeztah/18.09_backport_enable_new_integration_tests_for_win
[18.09 backport] Enable integrations API tests for Windows CI
Upstream-commit: 49a4899c79f740e5a8e19df62ea341be9d5200be
Component: engine
2019-06-17 12:26:34 -07:00
75aeb78955 Merge pull request #242 from thaJeztah/18.09_swagger_fixes
[18.09 backport] Swagger fixes
Upstream-commit: 556456701d1bd69a20f62d68d22257b8a6ad71c0
Component: engine
2019-06-17 12:24:38 -07:00
6a5e7a9ed6 Merge pull request #263 from thaJeztah/18.09_backport_39290alternate
[18.09 backport] Windows: Don't attempt detach VHD for R/O layers
Upstream-commit: ab9db72ae809597b5fb7a9e245f18fa3f99857ec
Component: engine
2019-06-17 12:23:41 -07:00
7551e66a79 Merge pull request #250 from thaJeztah/18.09_backport_fix_fix_win_tmp
[18.09 backport] Windows CI - Corrected LOCALAPPDATA location
Upstream-commit: d86fe18fc243c9d505403d50264a6db90e68e721
Component: engine
2019-06-17 12:23:09 -07:00
3317a78c8e Merge pull request #237 from thaJeztah/18.09_backport_remove_TestSearchCmdOptions
[18.09 backport] Remove TestSearchCmdOptions test
Upstream-commit: 87c3750877aec0288945231af20362edc26c7a70
Component: engine
2019-06-17 12:16:19 -07:00
b488dc3c6e Merge pull request #246 from thaJeztah/18.09_backport_log-daemon-exit-before-tests-finish
[18.09 backport] Ensure all integration daemon logging happens before test exit
Upstream-commit: 08f6e9c14fd0de408db93914058d75df5b32ac67
Component: engine
2019-06-17 12:15:43 -07:00
a99bdf7fc2 Merge pull request #269 from thaJeztah/18.09_backport_test_fixes
[18.09 backport] Harden TestPsListContainersFilterExited
Upstream-commit: 6799eea8c80b506aeae54c1331b4b2ad834920bd
Component: engine
2019-06-17 12:15:02 -07:00
20e962774c Merge pull request #233 from thaJeztah/18.09_backport_update_seccomp_test_for_aarch64
[18.09 backport] Update TestRunWithDaemonDefaultSeccompProfile for ARM64
Upstream-commit: 6809464a6395699b687fa0f0dfa190aa1d6d6547
Component: engine
2019-06-17 12:13:46 -07:00
0fafb25ad4 Merge pull request #245 from thaJeztah/18.09_backport_allow_version_overwrite
[18.09 backport] Add ability to override the version in make.ps1
Upstream-commit: f07e4693278434c112870d30d80ad984bc79e527
Component: engine
2019-06-17 12:11:50 -07:00