Commit Graph

32664 Commits

Author SHA1 Message Date
196d64b294 Merge pull request #35967 from Microsoft/jjh/32838-pass-container-shutdown-error-back
Windows: Pass back system errors on container exit
Upstream-commit: 66e6beeb249948634e2815ef5cac97984d5c0d56
Component: engine
2018-02-22 19:12:10 -08:00
d662aea133 Merge pull request #36376 from vdemeester/fix-e2e-dockerfile
Fix Dockerfile.e2e build
Upstream-commit: 8b8a0e2c523eb886094fad7e5342d93407a19c8e
Component: engine
2018-02-22 13:17:22 -05:00
96898bda21 Windows: Pass back system errors on container exit
Signed-off-by: John Howard <jhoward@microsoft.com>

While debugging #32838, it was found (https://github.com/moby/moby/issues/32838#issuecomment-356005845) that the utility VM in some circumstances was crashing. Unfortunately, this was silently thrown away, and as far as the build step (also applies to docker run) was concerned, the exit code was zero and the error was thrown away. Windows containers operate differently to containers on Linux, and there can be legitimate system errors during container shutdown after the init process exits. This PR handles this and passes the error all the way back to the client, and correctly causes a build step running a container which hits a system error to fail, rather than blindly trying to keep going, assuming all is good, and get a subsequent failure on a commit.

With this change, assuming an error occurs, here's an example of a failure which previous was reported as a commit error:

```
The command 'powershell -Command $ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue'; Install-WindowsFeature -Name Web-App-Dev ;   Install-WindowsFeature -Name ADLDS;   Install-WindowsFeature -Name Web-Mgmt-Compat;   Install-WindowsFeature -Name Web-Mgmt-Service;   Install-WindowsFeature -Name Web-Metabase;   Install-WindowsFeature -Name Web-Lgcy-Scripting;   Install-WindowsFeature -Name Web-WMI;   Install-WindowsFeature -Name Web-WHC;   Install-WindowsFeature -Name Web-Scripting-Tools;   Install-WindowsFeature -Name Web-Net-Ext45;   Install-WindowsFeature -Name Web-ASP;   Install-WindowsFeature -Name Web-ISAPI-Ext;   Install-WindowsFeature -Name Web-ISAPI-Filter;   Install-WindowsFeature -Name Web-Default-Doc;   Install-WindowsFeature -Name Web-Dir-Browsing;   Install-WindowsFeature -Name Web-Http-Errors;   Install-WindowsFeature -Name Web-Static-Content;   Install-WindowsFeature -Name Web-Http-Redirect;   Install-WindowsFeature -Name Web-DAV-Publishing;   Install-WindowsFeature -Name Web-Health;   Install-WindowsFeature -Name Web-Http-Logging;   Install-WindowsFeature -Name Web-Custom-Logging;   Install-WindowsFeature -Name Web-Log-Libraries;   Install-WindowsFeature -Name Web-Request-Monitor;   Install-WindowsFeature -Name Web-Http-Tracing;   Install-WindowsFeature -Name Web-Stat-Compression;   Install-WindowsFeature -Name Web-Dyn-Compression;   Install-WindowsFeature -Name Web-Security;   Install-WindowsFeature -Name Web-Windows-Auth;   Install-WindowsFeature -Name Web-Basic-Auth;   Install-WindowsFeature -Name Web-Url-Auth;   Install-WindowsFeature -Name Web-WebSockets;   Install-WindowsFeature -Name Web-AppInit;   Install-WindowsFeature -Name NET-WCF-HTTP-Activation45;   Install-WindowsFeature -Name NET-WCF-Pipe-Activation45;   Install-WindowsFeature -Name NET-WCF-TCP-Activation45;' returned a non-zero code: 4294967295: container shutdown failed: container ba9c65054d42d4830fb25ef55e4ab3287550345aa1a2bb265df4e5bfcd79c78a encountered an error during WaitTimeout: failure in a Windows system call: The compute system exited unexpectedly. (0xc0370106)
```

Without this change, it would be incorrectly reported such as in this comment: https://github.com/moby/moby/issues/32838#issuecomment-309621097

```
Step 3/8 : ADD buildtools C:/buildtools
re-exec error: exit status 1: output: time="2017-06-20T11:37:38+10:00" level=error msg="hcsshim::ImportLayer failed in Win32: The system cannot find the path specified. (0x3) layerId=\\\\?\\C:\\ProgramData\\docker\\windowsfilter\\b41d28c95f98368b73fc192cb9205700e21
6691495c1f9ac79b9b04ec4923ea2 flavour=1 folder=C:\\Windows\\TEMP\\hcs232661915"
hcsshim::ImportLayer failed in Win32: The system cannot find the path specified. (0x3) layerId=\\?\C:\ProgramData\docker\windowsfilter\b41d28c95f98368b73fc192cb9205700e216691495c1f9ac79b9b04ec4923ea2 flavour=1 folder=C:\Windows\TEMP\hcs232661915
```
Upstream-commit: 8c52560ea4593935322c1d056124be44e234b934
Component: engine
2018-02-22 08:53:43 -08:00
34d6002b9b Merge pull request #36373 from dnephin/prepare-for-image-service
Move daemon image code in preparation for ImageService
Upstream-commit: 867a10aadeb56a508fb7ceeee21ab1ef45a506a6
Component: engine
2018-02-22 17:32:43 +01:00
79cad3892d Merge pull request #36363 from dnephin/fix-attach-detach-test
Fix TestAttachAfterDetach to work with latest client
Upstream-commit: c6513bd05170ad3b77def3ae50541ef50d2ad27d
Component: engine
2018-02-22 11:25:02 -05:00
118b6d65e2 Merge pull request #36371 from cpuguy83/bump_swarmkit
Bump swarmkit
Upstream-commit: ffbb130873ad1aba579de7817b348b0ceacc31c2
Component: engine
2018-02-22 12:20:42 +01:00
2c4f0e56a8 Fix Dockerfile.e2e build
This images is used to run integration and integration-cli tests on
anything that implements the docker api :). The image wasn't building
anywore :D

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 9c46b587eec84f6d9b525ecca1d8b66a51055c12
Component: engine
2018-02-22 11:30:47 +01:00
e6062e18f5 Merge pull request #36336 from cpuguy83/split_installers
Split binary installers/commit scripts
Upstream-commit: 49ad3d69a7780e54ace50290d0a1a55c1182a947
Component: engine
2018-02-21 19:20:29 -08:00
112db723e1 Image commit
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: bad33bbd02b9f2754d3a8c64648d46baba509680
Component: engine
2018-02-21 18:26:16 -05:00
718daa9c98 Image events
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 0ac4ad0580c48749fb7d07f60a77a0014b1c3a99
Component: engine
2018-02-21 18:26:16 -05:00
192d8829e0 GetLayerFolders
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: f6639cb46d83cb9bb92a30578c10fdb372a58f04
Component: engine
2018-02-21 18:26:16 -05:00
4811cada93 Move ImagePrune
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 9c25df0fa201279f78d555c91ae000cc4f9b3036
Component: engine
2018-02-21 18:26:16 -05:00
02638eae1b Merge pull request #36209 from dnephin/fix-image-prune-mapping
Remove broken container check from image prune
Upstream-commit: 05c751b1be6785b4f8a42c412e858508b137c10e
Component: engine
2018-02-21 18:22:51 -05:00
c5a1aa150e Bump swarmkit
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: be713ec486090db312f9a83481a2c4663d473f71
Component: engine
2018-02-21 16:08:32 -05:00
f11213a800 Split binary installers/commit scripts
Originally I worked on this for the multi-stage build Dockerfile
changes. Decided to split this out as we are still waiting for
multi-stage to be available on CI and rebasing these is pretty annoying.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: b529d1b0936b90ae14d584c73f7332919f8d76b7
Component: engine
2018-02-21 15:39:24 -05:00
c3a24a0fbd Update authors
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: fb005971c2bd82de482d1cfe924f2f4ec60c60ae
Component: engine
2018-02-21 21:12:31 +01:00
ff7653434c Merge pull request #36368 from justincormack/maskkeys
Add /proc/keys to masked paths
Upstream-commit: 8f6a40a3f126ae1dc0a74dc612a3f7b14de4e7f6
Component: engine
2018-02-21 13:48:19 -05:00
87cd2bf7ea Add /proc/keys to masked paths
This leaks information about keyrings on the host. Keyrings are
not namespaced.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Upstream-commit: de23cb939858a66829d5b75057c7ac664c5acda5
Component: engine
2018-02-21 16:23:34 +00:00
e09ff74ed5 Adjust minimum API version for templated configs/secrets
Also adds a note to the API version history

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a3efeaad529b945ce5af78c4b08a6ed47399f8d5
Component: engine
2018-02-21 15:23:00 +01:00
40a7a5b6d7 Bump default API version to 1.37
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 453f2b8b40b923862ce1c08c11531ff5042770f1
Component: engine
2018-02-21 15:22:40 +01:00
a196815f55 Merge pull request #33702 from aaronlehmann/templated-secrets-and-configs
Templated secrets and configs
Upstream-commit: 0076343b29f508a5deb06861c0d85748659f8881
Component: engine
2018-02-21 13:39:10 +01:00
0308ec9c17 Merge pull request #34899 from dnephin/fix-duplicate-new-client
[client] Remove duplicate NewClient functions
Upstream-commit: 466cc981433e2e1815a64ee714b1c8c083765785
Component: engine
2018-02-21 12:59:42 +01:00
ad01430349 Merge pull request #35829 from cpuguy83/no_private_mount_for_plugins
Perform plugin mounts in the runtime
Upstream-commit: 20028325daab4fcbee9c8e28f43dbfb2b1c5d568
Component: engine
2018-02-21 12:28:13 +01:00
692373dbbd Merge pull request #35898 from javabrett/docs-contributing-test
test.md improvements and corrections
Upstream-commit: 9f68f20faecfc084343dc1a6cbd7de1db616e7e0
Component: engine
2018-02-21 09:32:15 +01:00
143dde6783 Fix TestAttachAfterDetach to work with latest client
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 847b610620a8b8294d61c717d3c4aa13cb7a8b33
Component: engine
2018-02-20 18:34:25 -05:00
10bf273bff Remove explicit DOCKER_API_VERSION from integration env setup
Use the default version because it is used by the client package

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: e73d742cd7deee396eac3c97664b40264ee358cb
Component: engine
2018-02-20 17:27:28 -05:00
49adb54d71 Remove duplicate calls for getting an APIClient
Remove request.SockRequest
Remove request.SockRequestHijack
Remove request.SockRequestRaw()
Remove deprecated ParseHost
Deprecate and unexport more helpers

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 0a91ba2d8cfe16df0ba37c1e283c8e3dbbb086d4
Component: engine
2018-02-20 17:27:24 -05:00
e0869be245 Cleanup volume plugin test with bad assumptions
Test made some bad assumptions about on-disk state of volume data.
This updates the test to only test based on what the volume API is
designed to provide.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 0df654f3d61d8691ee113a8429bcc8ef65786bc7
Component: engine
2018-02-20 16:57:20 -05:00
088ad71eb5 integration/testUpdateCPUQuota: fix name
The function name should be TestUpdateCPUQuota and not TestUpdateCPUQUota.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 31825081d4c5e643a150b515547cb2a2ea223de4
Component: engine
2018-02-20 13:36:27 -08:00
133cf88cac integration/TestUpdateMemory: fix false failure
This fixes the following test failure:

> --- FAIL: TestUpdateMemory (0.53s)
>  	assertions.go:226:
>	Error Trace:	update_linux_test.go:52
>	Error:      	Not equal:
>	            	expected: int(524288000)
>	            	received: int64(524288000)

Fixes: 0f9da07b569f0d9
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: cc866470981a1e6a839004f24eb30bb708078068
Component: engine
2018-02-20 13:31:03 -08:00
180ce35066 Improve docstrings and small cleanup in client
Use client instead of helpers for TLS in integration test

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: a68ae4a2d95b1ff143025a435195af0f1ab30ace
Component: engine
2018-02-20 15:15:02 -05:00
9a83d9fd53 Use gotestyourself env patching
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 2b445a53c17ae6526c11a729cb6d1d1dc57490ff
Component: engine
2018-02-20 15:15:02 -05:00
dbebd52548 Merge pull request #36326 from kolyshkin/integration-exec
Add/use container.Exec() to integration
Upstream-commit: ee9abc212032353e19e5b0f5e6410ad67a0cc9b1
Component: engine
2018-02-20 17:16:09 +01:00
6899375641 Merge pull request #33922 from ishidawataru/sctp
Support SCTP port mapping (bump up API to v1.37)
Upstream-commit: 079ed017b61eb819b8184b90013ce89465d3aaba
Component: engine
2018-02-20 17:00:13 +01:00
ded970f96e Merge pull request #36107 from cpuguy83/cleanup_daemon_root_mount
Ensure daemon root is unmounted on shutdown
Upstream-commit: eb033c11753872c6269304d8127dbfe7315e476f
Component: engine
2018-02-20 10:04:41 +01:00
0c0ee07b87 Merge pull request #36330 from vdemeester/migrate-container-list-tests
test: clean/migrate some docker ps cli-only integration tests 
Upstream-commit: 7060a40addd71605862b668c1ec2b889e3540233
Component: engine
2018-02-20 08:44:17 +01:00
ed5d3f9cce Support SCTP port mapping (bump up API to v1.37)
Signed-off-by: Wataru Ishida <ishida.wataru@lab.ntt.co.jp>
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Upstream-commit: 8e435b8279f2af3e0cebd73fa9e25ca1bb26004e
Component: engine
2018-02-20 11:15:36 +09:00
df67d5ea13 Removed root@... PS1 from in-container root prompts, retaining #.
Signed-off-by: Brett Randall <javabrett@gmail.com>
Upstream-commit: bef0cd70a62eff156d99ddd15933b658a1af8893
Component: engine
2018-02-20 11:29:02 +11:00
f4580247c7 test.md improvements and corrections:
- Mentioned integration-cli test-suite deprecation.
- Removed mentions of removed in-container hack/make.sh
  target test-unit, replaced with hack/test/unit.

Signed-off-by: Brett Randall <javabrett@gmail.com>
Upstream-commit: acaa53bc35ab8fa97d75e90da393d39204a86a15
Component: engine
2018-02-20 11:21:03 +11:00
57f5f76274 integration/TestUpdateMemory: simplify
1. Use integration/internal/exec, removing the getContainerSysFSValue().

2. Avoid repeating magic numbers, use a variable for those.

3. Fix order of arguments to assert.Equal (first "expected", then "actual").

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 0f9da07b569f0d9cbe574db3af3951b4d5c968c0
Component: engine
2018-02-19 11:25:27 -08:00
0582099de6 integration/TestUpdateCPUQUota: use exec
An implementation of exec in TestUpdateCPUQUota had a few issues,
including resource leaking and calling both ContainerExecAttach and
ContainerExecRun. The last one makes the test flaky:

	update_linux_test.go:136: expected cgroup value 20000, got: Error: Exec
	command f923baf709525f6b38f6511126addc5d9bb88fb477eeca1c22440551090fa2bb
	is already running

Fix by using the integration/internal/exec package.

While at it, use require/assert to further improve code readability.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 8a7d6143fca69623e2f5d409328c97603843ccb6
Component: engine
2018-02-19 11:25:27 -08:00
b08746e15a integration: add container.Exec()
Some test cases might need an ability to execute a command inside a
container (in order to analyse its output and/or exit code). It is a bit
complicated operation to do so using engine API. The function provided
aims to hide this complexity, making exec almost as simple as 'docker
exec'.

NOTE that the exec is synchronous, and command's stdin is closed.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 01143afe54f1be7308c5663a0cc110740626c62b
Component: engine
2018-02-19 11:25:11 -08:00
39cad2aa10 Merge pull request #36256 from wcwxyz/fix-refcounter-memory-leak
graphdriver: Fix RefCounter memory leak
Upstream-commit: 733ed2ddd3c621dadafbb74feb7b80d20fd3fd6f
Component: engine
2018-02-19 10:32:14 -08:00
36a89bcfdd Merge pull request #36329 from tonistiigi/tar-cache-fix
builder: fix wrong cache hits building from tars
Upstream-commit: 8cf42d3ad805467be407962f902156a3834062ff
Component: engine
2018-02-19 12:01:20 +01:00
c2407fa324 Clean some cli-only integration tests
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 641c73d211d7efe15255ae36ab2362cef3584260
Component: engine
2018-02-19 11:19:19 +01:00
e46642afe1 Merge pull request #35510 from ripcurld0/fix_35500
Display a warn message when there is binding ports and net mode is host
Upstream-commit: 35d69f10a9ac7479095f39358abb78ee6d3ab65f
Component: engine
2018-02-19 08:57:36 +01:00
da22cbc58d Display a warn message when there is binding ports and net mode is host
When a container is created if "--network" is set to "host" all the
ports in the container are bound to the host.
Thus, adding "-p" or "--publish" to the command-line is meaningless.

Unlike "docker run" and "docker create", "docker service create" sends
an error message when network mode is host and port bindings are given

This patch however suggests to send a warning message to the client when
such a case occurs.

The warning message is added to "warnings" which are returned from
"verifyPlatformContainerSettings".

Signed-off-by: Boaz Shuster <ripcurld.github@gmail.com>
Upstream-commit: 6e78fdb790d2e1dbf95a1733cab9395b1b936622
Component: engine
2018-02-18 13:28:44 +00:00
75d038d03f Update containerd/continuity to fix ARM 32-bit builds
This updates the containerd/continuity package to d8fb8589b0e8e85b8c8bbaa8840226d0dfeb7371
which fixes builds failing on ARM 32-bit, after this dependency was added in
b3aab5e31faf04d8a29f17be55562e4d0c0cb364

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f0947a541866ca05b030afe07dd659887a655e3e
Component: engine
2018-02-17 00:42:10 +01:00
dcede7a976 Merge pull request #36303 from dnephin/cleanup-in-daemon-unix
Cleanup unnecessary and duplicate functions in `daemon_unix.go`
Upstream-commit: 747c163a65365933c5d6d7f0740f2ac8e3775287
Component: engine
2018-02-16 14:55:18 -08:00
fbcb172e80 Merge pull request #36144 from emil2k/node-id-required
Produce errors when empty ids are passed into inspect calls. 
Upstream-commit: db360995df97f9d9bcb4eb6a34a5c35a2711795b
Component: engine
2018-02-16 10:18:47 -08:00