Commit Graph

297 Commits

Author SHA1 Message Date
77116c8094 Update containerd daemon
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: a0009345f51d7a58a2f188d7ffc7e0e837a5238d
Component: engine
2018-06-04 15:37:03 -07:00
60c800804e Update containerd to 1.1
Updates swarmkit, grpc, and all related vendors

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: 52ed3e0896c203013388a3eb603658f345a9d80a
Component: engine
2018-06-04 15:37:03 -07:00
0f96e98e12 Various code-cleanup
remove unnescessary import aliases, brackets, and so on.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f23c00d8701e4bd0f2372a586dacbf66a26f9a51
Component: engine
2018-05-23 17:50:54 +02:00
7f78f7fe15 Fix some linting issues
These showed locally when running `make validate`. CI doesn't seem to have the
same (possibly it's disabled in the configuration)

    builder/fscache/fscache.go:618::error: github.com/docker/docker/vendor/github.com/tonistiigi/fsutil.StatInfo composite literal uses unkeyed fields (vet)
    client/swarm_unlock_test.go:44::error: github.com/docker/docker/api/types/swarm.UnlockRequest composite literal uses unkeyed fields (vet)
    client/swarm_unlock_test.go:20::error: github.com/docker/docker/api/types/swarm.UnlockRequest composite literal uses unkeyed fields (vet)
    cmd/dockerd/daemon_unix.go:113::error: github.com/docker/docker/cmd/dockerd/hack.MalformedHostHeaderOverride composite literal uses unkeyed fields (vet)
    cmd/dockerd/daemon_unix.go:110::error: github.com/docker/docker/cmd/dockerd/hack.MalformedHostHeaderOverride composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay/overlay.go:171::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay/overlay.go:413::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay2/overlay.go:203::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay2/overlay.go:584::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/zfs/zfs.go:109::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/zfs/zfs.go:388::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/volumes_windows.go:27::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    integration/service/network_test.go:31::error: github.com/docker/docker/api/types/network.NetworkingConfig composite literal uses unkeyed fields (vet)
    api/server/server.go:129:10⚠️ should not use basic type string as key in context.WithValue (golint)
    integration/service/network_test.go:54::error: github.com/docker/docker/api/types/network.NetworkingConfig composite literal uses unkeyed fields (vet)
    libcontainerd/client_daemon_linux.go:61::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    libcontainerd/client_daemon_linux.go:74::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    pkg/archive/archive_windows.go:76::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    plugin/manager_linux.go:56::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 4f8c870d623c63b98f8ef8002448f07f4dda4aa9
Component: engine
2018-05-17 19:28:27 +02:00
f766069f3c Merge pull request #36684 from cpuguy83/bump_containerd_client
Bump containerd client
Upstream-commit: 8bb5a28eed5eba5651c6e48eb401c03be938b4c1
Component: engine
2018-04-19 10:38:43 +02:00
c9ce394894 Relax global client lock on containerd restore.
This unblocks the client to take other restore requests and makes sure
that a long/stuck request can't block the client forever.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 806700e410a1b30aa1033d71d825e57880c75298
Component: engine
2018-04-17 12:07:06 -04:00
57682a5381 Merge pull request #36715 from cpuguy83/plugin_exec_fixes
Make sure plugin container is removed on failure
Upstream-commit: 859e43e64c4358c9aa48e045c0e5b048361a4c9b
Component: engine
2018-03-30 13:17:31 +02:00
92debf8e45 Use containerd client Reconnect() API.
This fixes an issue where the containerd client is cached in a container
object in libcontainerd and becomes stale after containerd is restarted.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 2c682d5209d1d1cdd232ce3909fa25b6023f9a5b
Component: engine
2018-03-28 09:43:08 -04:00
a2df8e9502 Make sure plugin container is removed on failure
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: f81172b9031160218e51fb2a7dbeee19962a60a9
Component: engine
2018-03-28 09:07:24 -04:00
287db00d6b Add if judgment before receiving operations on daemonWaitCh
receive operations on a nil channel will always block, this lead containerd not started.

Signed-off-by: Shukui Yang <yangshukui@huawei.com>
Upstream-commit: a50d9c0765d9861d226b323ffe190c0c99c3ac17
Component: engine
2018-03-21 05:53:55 -04:00
60daf5fa97 Automated migration using
gty-migrate-from-testify --ignore-build-tags

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 6be0f709830113966f295401327b027ec2f0bbca
Component: engine
2018-03-16 11:03:43 -04:00
2c6fe9c524 Change containerd monitor ticker to sleep
With the ticker this could end up just doing back-to-back checks, which
isn't really what we want here.
Instead use a sleep to ensure we actually sleep for the desired
interval.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 04a0d6b863ed50cfffa79936cf9cdab7a3a9e7df
Component: engine
2018-03-13 13:20:19 -04:00
a0d675b388 Windows: Remove servicing mode
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: d4f37c08858b90e8603741ba92de8e0d39b88eb3
Component: engine
2018-02-27 08:48:31 -08:00
196d64b294 Merge pull request #35967 from Microsoft/jjh/32838-pass-container-shutdown-error-back
Windows: Pass back system errors on container exit
Upstream-commit: 66e6beeb249948634e2815ef5cac97984d5c0d56
Component: engine
2018-02-22 19:12:10 -08:00
96898bda21 Windows: Pass back system errors on container exit
Signed-off-by: John Howard <jhoward@microsoft.com>

While debugging #32838, it was found (https://github.com/moby/moby/issues/32838#issuecomment-356005845) that the utility VM in some circumstances was crashing. Unfortunately, this was silently thrown away, and as far as the build step (also applies to docker run) was concerned, the exit code was zero and the error was thrown away. Windows containers operate differently to containers on Linux, and there can be legitimate system errors during container shutdown after the init process exits. This PR handles this and passes the error all the way back to the client, and correctly causes a build step running a container which hits a system error to fail, rather than blindly trying to keep going, assuming all is good, and get a subsequent failure on a commit.

With this change, assuming an error occurs, here's an example of a failure which previous was reported as a commit error:

```
The command 'powershell -Command $ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue'; Install-WindowsFeature -Name Web-App-Dev ;   Install-WindowsFeature -Name ADLDS;   Install-WindowsFeature -Name Web-Mgmt-Compat;   Install-WindowsFeature -Name Web-Mgmt-Service;   Install-WindowsFeature -Name Web-Metabase;   Install-WindowsFeature -Name Web-Lgcy-Scripting;   Install-WindowsFeature -Name Web-WMI;   Install-WindowsFeature -Name Web-WHC;   Install-WindowsFeature -Name Web-Scripting-Tools;   Install-WindowsFeature -Name Web-Net-Ext45;   Install-WindowsFeature -Name Web-ASP;   Install-WindowsFeature -Name Web-ISAPI-Ext;   Install-WindowsFeature -Name Web-ISAPI-Filter;   Install-WindowsFeature -Name Web-Default-Doc;   Install-WindowsFeature -Name Web-Dir-Browsing;   Install-WindowsFeature -Name Web-Http-Errors;   Install-WindowsFeature -Name Web-Static-Content;   Install-WindowsFeature -Name Web-Http-Redirect;   Install-WindowsFeature -Name Web-DAV-Publishing;   Install-WindowsFeature -Name Web-Health;   Install-WindowsFeature -Name Web-Http-Logging;   Install-WindowsFeature -Name Web-Custom-Logging;   Install-WindowsFeature -Name Web-Log-Libraries;   Install-WindowsFeature -Name Web-Request-Monitor;   Install-WindowsFeature -Name Web-Http-Tracing;   Install-WindowsFeature -Name Web-Stat-Compression;   Install-WindowsFeature -Name Web-Dyn-Compression;   Install-WindowsFeature -Name Web-Security;   Install-WindowsFeature -Name Web-Windows-Auth;   Install-WindowsFeature -Name Web-Basic-Auth;   Install-WindowsFeature -Name Web-Url-Auth;   Install-WindowsFeature -Name Web-WebSockets;   Install-WindowsFeature -Name Web-AppInit;   Install-WindowsFeature -Name NET-WCF-HTTP-Activation45;   Install-WindowsFeature -Name NET-WCF-Pipe-Activation45;   Install-WindowsFeature -Name NET-WCF-TCP-Activation45;' returned a non-zero code: 4294967295: container shutdown failed: container ba9c65054d42d4830fb25ef55e4ab3287550345aa1a2bb265df4e5bfcd79c78a encountered an error during WaitTimeout: failure in a Windows system call: The compute system exited unexpectedly. (0xc0370106)
```

Without this change, it would be incorrectly reported such as in this comment: https://github.com/moby/moby/issues/32838#issuecomment-309621097

```
Step 3/8 : ADD buildtools C:/buildtools
re-exec error: exit status 1: output: time="2017-06-20T11:37:38+10:00" level=error msg="hcsshim::ImportLayer failed in Win32: The system cannot find the path specified. (0x3) layerId=\\\\?\\C:\\ProgramData\\docker\\windowsfilter\\b41d28c95f98368b73fc192cb9205700e21
6691495c1f9ac79b9b04ec4923ea2 flavour=1 folder=C:\\Windows\\TEMP\\hcs232661915"
hcsshim::ImportLayer failed in Win32: The system cannot find the path specified. (0x3) layerId=\\?\C:\ProgramData\docker\windowsfilter\b41d28c95f98368b73fc192cb9205700e216691495c1f9ac79b9b04ec4923ea2 flavour=1 folder=C:\Windows\TEMP\hcs232661915
```
Upstream-commit: 8c52560ea4593935322c1d056124be44e234b934
Component: engine
2018-02-22 08:53:43 -08:00
5f5ee4bd87 Merge pull request #36268 from Microsoft/jjh/rs3-bump
Windows: Bump to final RS3 build number
Upstream-commit: bf1345d0b6d91f24e06d05e741897bc83cf8bab4
Component: engine
2018-02-12 14:49:33 -08:00
0a75e5196f Windows: Bump to final RS3 build number
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: c04504383a913c1af9868b23880c6401651c71b1
Component: engine
2018-02-09 11:39:57 -08:00
5d3102854b Fix container cleanup on daemon restart
When the daemon restores containers on daemon restart, it syncs up with
containerd to determine the existing state. For stopped containers it
then removes the container metadata from containerd.

In some cases this is not handled properly and causes an error when
someone attempts to start that container again.
In particular, this case is just a bad error check.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: c0d56ab71701ba47ca6066c7952e724f4f5977c0
Component: engine
2018-02-09 14:36:36 -05:00
83ccee96c3 Merge pull request #36173 from cpuguy83/fix_containerd_crash_spin
Refresh containerd remotes on containerd restarted
Upstream-commit: 384ff69f2f97c0cf0ee2b863bd4d90a82e1cc1a4
Component: engine
2018-02-08 06:19:29 -08:00
1e3931f923 Refresh containerd remotes on containerd restarted
Before this patch, when containerd is restarted (due to a crash, or
kill, whatever), the daemon would keep trying to process the event
stream against the old socket handles. This would lead to a CPU spin due
to the error handling when the client can't connect to containerd.

This change makes sure the containerd remote client is updated for all
registered libcontainerd clients.

This is not neccessarily the ideal fix which would likely require a
major refactor, but at least gets things to a working state with a
minimal patch.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 400126f8698233099259da967378c0a76bc3ea31
Component: engine
2018-02-07 11:53:00 -05:00
be83c11fb0 Add canonical import comment
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 4f0d95fa6ee7f865597c03b9e63702cdcb0f7067
Component: engine
2018-02-05 16:51:57 -05:00
d4d0b5c268 Move api/errdefs to errdefs
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: d453fe35b9b8b52d0677fe0c3cc8373f2f5d30d0
Component: engine
2018-01-11 21:21:43 -05:00
952c29f8da Add helpers to create errdef errors
Instead of having to create a bunch of custom error types that are doing
nothing but wrapping another error in sub-packages, use a common helper
to create errors of the requested type.

e.g. instead of re-implementing this over and over:

```go
type notFoundError struct {
  cause error
}

func(e notFoundError) Error() string {
  return e.cause.Error()
}

func(e notFoundError) NotFound() {}

func(e notFoundError) Cause() error {
  return e.cause
}
```

Packages can instead just do:

```
  errdefs.NotFound(err)
```

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 87a12421a94faac294079bebc97c8abb4180dde5
Component: engine
2018-01-11 21:21:43 -05:00
6f88b66c36 Fix libcontainerd/client.Restore() handling of io cleanup
Make the behvious of cleaning up DirectIO more obvious

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 9d20d5eb3fd744088e700292b15d56de29a3361d
Component: engine
2018-01-09 12:03:02 -05:00
cf94b87820 Use cio.FIFOSet.Close() to cleanup fifos
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: d72dfbfa8db9a99be34c2d7963717a08ab7358c5
Component: engine
2018-01-09 12:00:28 -05:00
097704d16a Remove libcontainerd.IOPipe
replaced with cio.DirectIO

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 3fec7c0858a0a3dee5423e6bffc3a3a1b238c30f
Component: engine
2018-01-09 12:00:28 -05:00
bcd054dac5 Merge pull request #35957 from crosbymichael/ramdisk
Honor DOCKER_RAMDISK with containerd 1.0
Upstream-commit: 94b8a116fbf1cd90e68d8f5361b520d326a66f9b
Component: engine
2018-01-08 17:38:17 -08:00
88fc146546 Use errdefs instead of string contains for checking not found
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 9b62d4ffa39f7c042f94f9a3670cae2a816535da
Component: engine
2018-01-08 14:32:39 -05:00
1f027b4d1b Honor DOCKER_RAMDISK with containerd 1.0
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Upstream-commit: 54051e9e64185e442e034c7e49a5707459a9eed2
Component: engine
2018-01-08 12:01:03 -05:00
4b57d7f1d3 Fix event filter filtering on "or"
The event filter used two separate filter-conditions for
"namespace" and "topic". As a result, both events matching
"topic" and events matching "namespace" were subscribed to,
causing events to be handled both by the "plugin" client, and
"container" client.

This patch rewrites the filter to match only if both namespace
and topic match.

Thanks to Stephen Day for providing the correct filter :)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 295bb09184fe473933498bb0efb59b8acb124f55
Component: engine
2017-12-29 02:47:56 +01:00
fbc087df98 Fix some missing synchronization in libcontainerd
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 647cec4324186faa3183bd6a7bc72a032a86c8c9
Component: engine
2017-12-15 12:54:38 -05:00
72249a1f2e Fix error handling for kill/process not found
With the contianerd 1.0 migration we now have strongly typed errors that
we can check for process not found.
We also had some bad error checks looking for `ESRCH` which would only
be returned from `unix.Kill` and never from containerd even though we
were checking containerd responses for it.

Fixes some race conditions around process handling and our error checks
that could lead to errors that propagate up to the user that should not.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: e55bead518e4c72cdecf7de2e49db6c477cb58eb
Component: engine
2017-12-15 10:09:55 -05:00
eae25b27ea Fix typo in log-message
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5c3418e38b9603e8ff582d53c2face57f0f01cce
Component: engine
2017-12-11 10:59:51 -08:00
d27e345310 Fix event filters
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Upstream-commit: a27abc619cc18ead61fa89f652871fac1b318658
Component: engine
2017-12-05 09:55:33 -05:00
ad76847e4b Update daemon code for containerd API changes
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Upstream-commit: aa3ce07c41e0da9331f0659f28fed7f35846556c
Component: engine
2017-11-30 09:55:03 -05:00
6025db4712 Windows: Fix width/height swap
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: f11f351ae77e4090f0d80e9cccfd77287b1f6b17
Component: engine
2017-11-28 12:19:46 -08:00
ae3d074947 Cancelation errors should not be logged
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: b86746d60d4c43dcadd3b95fc4b2da7c03323d84
Component: engine
2017-11-14 20:32:20 -05:00
15db98f6fa Merge pull request #35465 from cpuguy83/update_containerd
Update containerd to v1 beta3
Upstream-commit: b4fbcd80c796f384eba800d1dc89892ab77c58e5
Component: engine
2017-11-14 14:01:48 +01:00
ce5c429be8 libcontainerd: fix leaking container/exec state
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: 6c4ce7cb6c62fb82ed2db1d4ee3a02bc5148cdee
Component: engine
2017-11-13 15:48:21 -08:00
9379e66070 Update containerd to v1 beta3
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: d3f934e304d78296e606d6e3a2ca523d249e954c
Component: engine
2017-11-10 15:44:10 -05:00
4a715bb927 Use containerd API to get version
The `docker info` code was shelling out to obtain the
version of containerd (using the `--version` flag).

Parsing the output of this version string is error-prone,
and not needed, as the containerd API can return the
version.

This patch adds a `Version()` method to the containerd Client
interface, and uses this to get the containerd version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: fec2b144feaaa18998ec2ed34c9bc843c4c29abd
Component: engine
2017-11-03 01:46:58 +01:00
eefbd135ae Remove solaris build tag and `contrib/mkimage/solaris
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 4785f1a7ab7ec857dc3ca849ee6ecadf519ef30e
Component: engine
2017-11-02 00:01:46 +00:00
67f37f6e34 Fixes LCOW after containerd 1.0 introduced regressions
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: 71651e0b801ae874b4a899e3c47add9e3fbc2400
Component: engine
2017-10-27 09:55:43 -07:00
044d7f995b Update libcontainerd to use containerd 1.0
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
Upstream-commit: ddae20c032058a0fd42c34c2e9750ee8f6296ac8
Component: engine
2017-10-20 07:11:37 -07:00
bb1c0c5aa8 Merge pull request #34846 from Microsoft/jjh/debuggcs
LCOW: Add UVM debugability by grabbing logs before tear-down
Upstream-commit: 7cbbbb95097f065757d38bcccdb1bbef81d10ddb
Component: engine
2017-09-19 22:33:46 +02:00
ddb0ee3757 Revendor Microsoft/opengcs @ v0.3.4
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: 2798576b37aa99643a06366f00072b6026c0b77e
Component: engine
2017-09-19 09:56:23 -07:00
aeb89eb179 Volume refactoring for LCOW
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
Upstream-commit: e89b6e8c2d2c36c43f22aeaf2a885646c2994051
Component: engine
2017-09-14 12:33:31 -07:00
d011c4127f LCOW: Add GCS debugging
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: 5a0e2beac330d49c2b7436bf29e87d52dab4f557
Component: engine
2017-09-13 14:54:34 -07:00
18c29c5c5f Add gosimple linter
Update gometalinter

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: f7f101d57ef8cbf2d8723a18b7d723c5c5dd04b6
Component: engine
2017-09-12 12:09:59 -04:00
8db4d9c803 LCOW: VHDX boot to read-only
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: 17a24034bb4281e2b63949beaa5d3846c0891726
Component: engine
2017-09-06 10:55:19 -07:00