Commit Graph

5264 Commits

Author SHA1 Message Date
ed052d78fd pluggable secret backend
This commit extends SwarmKit secret management with pluggable secret
backends support.

Updating the work in
[swarmkit](docker/swarmkit@eebac27434) for
pluggable secret backend and adding the
driver parameter to `SecretSpec`.

Remaining work:
- [ ] CLI support (docker/cli)
- [ ] api in [plugin helpers](docker/go-plugins-helpers))
- [ ] Reference plugin
- [ ] Documenation (after cli work)

Signed-off-by: Liron Levin <liron@twistlock.com>
Upstream-commit: 7d45cafd5746e847e58078aa2fbdde57b5f49fa4
Component: engine
2017-07-15 16:50:02 +03:00
240df15e94 Merge pull request #34076 from yastij/33679-renaming-id-meth
renaming id() to state clearly that it's related to the networkAttachment
Upstream-commit: ce0ef9568d945b3d16442906a428b6f80fedc4dd
Component: engine
2017-07-14 00:29:09 -07:00
b9deee44fd Merge pull request #33969 from Microsoft/jjh/lifetime
LCOW: Service VM lifetime changes
Upstream-commit: f22cecf9297a467f20bf49bc1c7aef5d2ee3a59c
Component: engine
2017-07-13 15:51:21 -07:00
9ed5ded98d LCOW: Service VM lifetime changes
Signed-off-by: John Howard <jhoward@microsoft.com>

This changes the LCOW driver to support both global SVM lifetime and
per-instance lifetime. It also corrects the scratch implementation.
Upstream-commit: 76b0d4819a50c50edd79890db0b92eb96c8d46e6
Component: engine
2017-07-13 11:09:43 -07:00
f68d310a1b Merge pull request #32881 from coolljt0725/fix_exec_faild
Enable inspect exec if container is pause/restarting/non-running
Upstream-commit: 53a75ee05072cf5e59e9fb5b9e5cc5fb91fdb5be
Component: engine
2017-07-12 19:29:27 -07:00
2015772003 Merge pull request #34027 from cpuguy83/15853_allow_stopping_paused_container
Allow stopping of paused container
Upstream-commit: 6fdb2fb069b985f244eb227828b00a940ee9de84
Component: engine
2017-07-13 10:16:46 +08:00
a6bf694e20 Merge pull request #33845 from cyphar/devicemapper-show-me-your-logs
devicemapper: rework logging and add --storage-opt dm.libdm_log_level
Upstream-commit: 00b218216d8a2075bf7200df74aa07b749f72f34
Component: engine
2017-07-12 17:46:14 -07:00
8e0ce0ecdb Merge pull request #33877 from rhvgoyal/sync-removal
devicemapper: Wait for device removal if deferredRemoval=true and deferredDeletion=…
Upstream-commit: e04dbe5ac287c2a856b7c96972d931ee5f0e288f
Component: engine
2017-07-12 17:35:45 -07:00
7aed75f09c Allow stopping of paused container
When a container is paused, signals are sent once the container has been
unpaused.
Instead of forcing the user to unpause a container before they can ever
send a signal, allow the user to send the signals, and in the case of a
stop signal, automatically unpause the container afterwards.

This is much safer than unpausing the container first then sending a
signal (what a user is currently forced to do), as the container may be
paused for very good reasons and should not be unpaused except for
stopping.
Note that not even SIGKILL is possible while a process is paused,
but it is killed the instant it is unpaused.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: c3feb046b9b1eb606cc7e853d020799eb3800e0e
Component: engine
2017-07-12 10:35:48 -04:00
165659798d renaming id() to state clearly that it's related to the networkAttachement container target
Signed-off-by: Yassine TIJANI <yasstij11@gmail.com>
Upstream-commit: 27c0131a43b1f4956446467273e82826264d6c35
Component: engine
2017-07-12 15:58:31 +02:00
bd835f3c9e Merge pull request #34061 from cyli/re-vendor-swarmkit
Re-vendor swarmkit. 
Upstream-commit: c0e6da7637491886904725c2fcec9db6ec261af4
Component: engine
2017-07-11 20:30:56 -07:00
c30b5760e3 Merge pull request #34032 from allencloud/support-config-event-in-cluster
make engine support cluster config event
Upstream-commit: 334702ab1fa8d3d6c64f3c41d80752c7b4196357
Component: engine
2017-07-11 20:25:19 -07:00
52c2abd279 Merge pull request #34004 from yummypeng/fix-docker-stats-hang
Return an empty stats if "container not found"
Upstream-commit: c8a2596d676a815cc17162f178869be4aeb4b301
Component: engine
2017-07-11 20:10:17 -07:00
57481ff30b Re-vendor swarmkit. This includes the following fixes:
- https://github.com/docker/swarmkit/pull/2266 (support for templating Node.Hostname in docker executor)
- https://github.com/docker/swarmkit/pull/2281 (change restore action on objects to be update, not delete/create)
- https://github.com/docker/swarmkit/pull/2285 (extend watch queue with timeout and size limit)
- https://github.com/docker/swarmkit/pull/2253 (version-aware failure tracking in the scheduler)
- https://github.com/docker/swarmkit/pull/2275 (update containerd and port executor to container client library)
- https://github.com/docker/swarmkit/pull/2292 (rename some generic resources)
- https://github.com/docker/swarmkit/pull/2300 (limit the size of the external CA response)
- https://github.com/docker/swarmkit/pull/2301 (delete global tasks when the node running them is deleted)

Minor cleanups, dependency bumps, and vendoring:
- https://github.com/docker/swarmkit/pull/2271
- https://github.com/docker/swarmkit/pull/2279
- https://github.com/docker/swarmkit/pull/2283
- https://github.com/docker/swarmkit/pull/2282
- https://github.com/docker/swarmkit/pull/2274
- https://github.com/docker/swarmkit/pull/2296 (dependency bump of etcd, go-winio)

Signed-off-by: Ying Li <ying.li@docker.com>
Upstream-commit: 4509a001dffff626d2d8708fd2eb1c6317b201f3
Component: engine
2017-07-11 13:43:43 -07:00
e95f4619cd [project] change syscall to /x/sys/unix|windows
Changes most references of syscall to golang.org/x/sys/
Ones aren't changes include, Errno, Signal and SysProcAttr
as they haven't been implemented in /x/sys/.

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>

[s390x] switch utsname from unsigned to signed

per 33267e036f
char in s390x in the /x/sys/unix package is now signed, so
change the buildtags

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
Upstream-commit: 069fdc8a083cb1663e4f86fe3fd9b9a1aebc3e54
Component: engine
2017-07-11 08:00:32 -04:00
94a5aefc94 make engine support cluster config event
Signed-off-by: allencloud <allen.sun@daocloud.io>
Upstream-commit: ed916a233c577693afc8dc3122311c15246d00cc
Component: engine
2017-07-11 17:21:15 +08:00
3ac74eb8fd Merge pull request #33979 from allencloud/return-prune-data-when-context-canceled
return prune data when context canceled
Upstream-commit: 73e8f56d6aed223c0f9997e8bd225e446664e647
Component: engine
2017-07-10 21:17:18 -04:00
b53529452f Merge pull request #33377 from yastij/33371-matchevent-filtering
matching events using ID and name
Upstream-commit: e56e78976eefdd17d333c3aa63382b2e4bfd889d
Component: engine
2017-07-10 16:11:14 +02:00
9e646d8386 Return an empty stats if "container not found"
If we get "container not found" error from containerd, it's possibly
because that this container has already been stopped. It will be ok to
ignore this error and just return an empty stats.

Signed-off-by: Yuanhong Peng <pengyuanhong@huawei.com>
Upstream-commit: 4a6cbf9bcb78d38c48ef963f585f0fadf733e101
Component: engine
2017-07-10 16:30:48 +08:00
f4101eb2fb return prune data when context canceled
Signed-off-by: allencloud <allen.sun@daocloud.io>
Upstream-commit: 87b4dc2002d50e245f432bbd97e9dee2db25cf96
Component: engine
2017-07-10 10:06:24 +08:00
23b4fdf3bc removing TODO, and explain why we can't filter by id on the matchevent method.
Signed-off-by: Yassine TIJANI <yasstij11@gmail.com>
Upstream-commit: be9aa09246534a1fdf7c08cf287dbe1e6de375c9
Component: engine
2017-07-07 23:32:37 +02:00
0887a18a3a Merge pull request #33575 from cpuguy83/plugins_on_swarm
Plugins on swarm
Upstream-commit: 05c7c311390911daebcf5d9519dee813fc02a887
Component: engine
2017-07-07 02:23:07 -07:00
0371882bb8 Merge pull request #33662 from madhanrm/sharedendpoint
Include Endpoint List for Shared Endpoints
Upstream-commit: 9aecbbf9bf50dd5c3d250e8dc1c74360a9f30d8e
Component: engine
2017-07-06 23:44:23 -07:00
cb27d2c59c Merge pull request #33940 from yaocaigen/fix_container_multi_names
Fix run `docker rename <container-id> new_name` concurrently, the container will have multi names
Upstream-commit: 8e3555f8539d91f530f895e19a51db14f1ca93fb
Component: engine
2017-07-07 10:58:01 +09:00
abfe031053 Merge pull request #33713 from x1022as/error_fd
fd leak and error handling
Upstream-commit: 3be2273d03e4b06dff4006b1a600c578c7aa09fc
Component: engine
2017-07-06 16:47:34 -07:00
455cc50b83 Include Endpoint List for Shared Endpoints
Do not allow sharing of container network with hyperv containers

Signed-off-by: Madhan Raj Mookkandy <madhanm@microsoft.com>
Upstream-commit: 349913ce9fde34d8acd08fad5ce866401f4d135e
Component: engine
2017-07-06 12:19:17 -07:00
c782cb6c44 Make plugin emit strongly typed, consumable events
Enables other subsystems to watch actions for a plugin(s).

This will be used specifically for implementing plugins on swarm where a
swarm controller needs to watch the state of a plugin.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 72c3bcf2a533a827402945e3a55872e2db4fb024
Component: engine
2017-07-06 14:26:06 -04:00
74d8ac0d96 Merge pull request #33897 from cpuguy83/jsonfile_log_blocking
Fix log readers can block writes indefinitely
Upstream-commit: db8c2651b84c681579389bb57d309a3b210a7e78
Component: engine
2017-07-05 18:21:14 -07:00
096ae0539f Merge pull request #33898 from crosbymichael/swappiness
Do not set -1 for swappiness
Upstream-commit: 490b55e8fa73be4c78de0118fe63a7ce5dc0f383
Component: engine
2017-07-05 17:04:08 -07:00
2aedd8b160 Fix log readers can block writes indefinitely
Before this patch, a log reader is able to block all log writes
indefinitely (and other operations) by simply opening the log stream and
not consuming all the messages.

The reason for this is we protect the read stream from corruption by
ensuring there are no new writes while the log stream is consumed (and
caught up with the live entries).

We can get around this issue because log files are append only, so we
can limit reads to only the section of the file that was written to when
the log stream was first requested.

Now logs are only blocked until all files are opened, rather than
streamed to the client.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: e2209185ed1c959131d4068ec7fc93e194dc0802
Component: engine
2017-07-05 14:04:52 -04:00
14554f495c Fix run docker rename <container-id> new_name concurrently, the container will have multi names
When run `docker rename <container-id> new_name` concurrently, every operation will release
container's old name. So container will have multi new names reserve in nameIndex.

Signed-off-by: Yang Pengfei <yangpengfei4@huawei.com>
Upstream-commit: cc2340689c431504689d01c9516b3a340b2e8dbc
Component: engine
2017-07-05 16:56:31 +08:00
ec1ee882b3 Merge pull request #33322 from jsoref/spelling
Spelling
Upstream-commit: ff4f700f74450018f36d014f3cde0ff1b9c17fb3
Component: engine
2017-07-04 15:46:34 +02:00
63c7ce310a Merge pull request #33213 from gkze/master
GCP Logs: send log text as jsonPayload.message instead of jsonPayload.data
Upstream-commit: c2ad549533cf2ffdb8b93ce8eb0bad5c695b828c
Component: engine
2017-07-03 16:18:18 -04:00
643654c2f0 Spelling fixes
* additional
* ambiguous
* anonymous
* anything
* application
* because
* before
* building
* capabilities
* circumstances
* commit
* committer
* compresses
* concatenated
* config
* container
* container's
* current
* definition
* delimiter
* disassociates
* discovery
* distributed
* doesnotexist
* downloads
* duplicates
* either
* enhancing
* enumerate
* escapable
* exactly
* expect
* expectations
* expected
* explicitly
* false
* filesystem
* following
* forbidden
* git with
* healthcheck
* ignore
* independent
* inheritance
* investigating
* irrelevant
* it
* logging
* looking
* membership
* mimic
* minimum
* modify
* mountpoint
* multiline
* notifier
* outputting
* outside
* overridden
* override
* parsable
* plugins
* precedence
* propagation
* provided
* provides
* registries
* repositories
* returning
* settings
* should
* signals
* someone
* something
* specifically
* successfully
* synchronize
* they've
* thinking
* uninitialized
* unintentionally
* unmarshaling
* unnamed
* unreferenced
* verify

Signed-off-by: Josh Soref <jsoref@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 39bcaee47b8a284a46b761afe218ba7deda0d482
Component: engine
2017-07-03 13:13:09 -07:00
654d76c922 Do not set -1 for swappiness
Do not set a default value for swappiness as the default value should be
`nil`

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Upstream-commit: 9d87e6e0fb799d6ef3bb9a97bc523f8d343b5fb3
Component: engine
2017-07-03 11:23:15 -07:00
666b2cc734 devicemapper: add --storage-opt dm.libdm_log_level=X option
Because we use our own logging callbacks in order to use libdm
effectively, it is quite difficult to debug complicated devicemapper
issues (because any warnings or notices from libdm are muted by our own
callback function). e07d3cd9a ("devmapper: Fix libdm logging") further
reduced the ability of this debugging by only allowing _LOG_FATAL errors
to be passed to the output.

Unfortunately libdm is very chatty, so in order to avoid making the logs
even more crowded, add a dm.libdm_log_level storage option that allows
people who are debugging the lovely world of libdm to be able to dive in
without recompiling binaries.

The valid values of dm.libdm_log_level map directly to the libdm logging
levels, and are in the range [2,7] as of the time of writing with 7
being _LOG_DEBUG and 2 being _LOG_FATAL. The default is _LOG_FATAL.

Signed-off-by: Aleksa Sarai <asarai@suse.de>
Upstream-commit: 198f83bba120c6c9bda679d33a55acab6cc9f33d
Component: engine
2017-07-04 02:04:26 +10:00
00264ebf6b devicemapper: change LogInit and move all callbacks to pkg
LogInit used to act as a manual way of registering the *necessary*
pkg/devicemapper logging callbacks. In addition, it was used to split up
the logic of pkg/devicemapper into daemon/graphdriver/devmapper (such
that some things were logged from libdm).

The manual aspect of this API was completely non-sensical and was just
begging for incorrect usage of pkg/devicemapper, so remove that semantic
and always register our own libdm callbacks.

In addition, recombine the split out logging callbacks into
pkg/devicemapper so that the default logger is local to the library and
also shown to be the recommended logger. This makes the code
substantially easier to read. Also the new DefaultLogger now has
configurable upper-bound for the log level, which allows for dynamically
changing the logging level.

Signed-off-by: Aleksa Sarai <asarai@suse.de>
Upstream-commit: cfd39e8d6d79eee8a71fbe6820dd67babf05d951
Component: engine
2017-07-04 02:04:26 +10:00
955a71e569 Merge pull request #33926 from coolljt0725/fix_wrong_file_mode
Fix wrong filemode for rotate log files
Upstream-commit: d061ad4a1b3b029e4afa042c1eaf707037562e8c
Component: engine
2017-07-03 17:45:14 +02:00
3a59ac4d9f Fix wrong filemode for rotate log files
the filemode should be 0640 but not 06400

Signed-off-by: Lei Jitang <leijitang@huawei.com>
Upstream-commit: 96ea8eaa1592856e0829c3465e64f93911d3e389
Component: engine
2017-07-03 03:49:22 -04:00
c5ade3e304 daemon: Ignore nonexistent containers when listing containers
The name/ID relationships are maintained separately from the memdb and
can be out of sync from any particular memdb snapshot. If a container
does not exist in the memdb, we must accept this as normal and not fail
the listing. This is consistent with what the code used to do before
memdb was introduced.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: d257a63fb6a2ad0c2369981914544add389917bf
Component: engine
2017-06-30 15:19:47 -07:00
56378645d0 Wait for device removal if deferredRemoval=true and deferredDeletion=false
There have been some cases where umount, a device can be busy for a very
short duration. Maybe its udev rules, or maybe it is runc related races
or probably it is something else. We don't know yet.

If deferred removal is enabled but deferred deletion is not, then for the
case of "docker run -ti --rm fedora bash", a container will exit, device
will be deferred removed and then immediately a call will come to delete
the device. It is possible that deletion will fail if device was busy
at that time.

A device can't be deleted if it can't be removed/deactivated first. There
is only one exception and that is when deferred deletion is on. In that
case graph driver will keep track of deleted device and try to delete it
later and return success to caller.

Always make sure that device deactivation is synchronous when device is
being deleted (except the case when deferred deletion is enabled).

This should also take care of small races when device is busy for a short
duration and it is being deleted.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Upstream-commit: 36cb6efebc599900b691e206fb9e99d3aa2fb9a3
Component: engine
2017-06-30 14:27:26 -04:00
400dd10c7d Merge pull request #33809 from Microsoft/jjh/svmdedicatedscratch
LCOW: Dedicated scratch space for service VM utilities
Upstream-commit: 8b29ba59ccdd4de45fa25137ec5d5a3845f5f5cc
Component: engine
2017-06-28 11:06:46 -07:00
08b4e324e5 Merge pull request #33826 from Microsoft/jjh/lcownits
LCOW: Fix nits from 33241
Upstream-commit: 950d472c9ce4180b80e28030f7de68ffc3254f97
Component: engine
2017-06-27 22:56:38 -07:00
674cb1c136 Merge pull request #32977 from imkin/30491-default-container-size
Add overlay2.size daemon storage-opt
Upstream-commit: acf855bf10e16b4282cb14f6cc34e378831a22c8
Component: engine
2017-06-27 20:00:02 -07:00
9a5747b3f9 LCOW: Fix nits from 33241
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: 4ec9766a27ffb964cec6b8b2745725965b2644aa
Component: engine
2017-06-27 11:59:49 -07:00
92d764e7fb LCOW: Dedicated scratch space for global service VM
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: d1d649c891c8b06567afba7542ea9bf518cdaa02
Component: engine
2017-06-27 10:55:16 -07:00
59255c95cd Merge pull request #33832 from glasser/glasser/msg-source
Fix stderr logging for journald and syslog
Upstream-commit: 7843bfde36d9ded6fada32f469fe1790316137ae
Component: engine
2017-06-27 13:30:41 -04:00
359031100e Add overlay2.size daemon storage-opt
This commit adds the overlay2.size option to the daemon daemon
storage opts.

The user can override this option by the "docker run --storage-opt"
options.

Signed-off-by: Dhawal Yogesh Bhanushali <dbhanushali@vmware.com>
Upstream-commit: a63d5bc03513755015827d0fe93563240429f1e0
Component: engine
2017-06-27 01:07:48 -07:00
e2c85453e0 Merge pull request #33815 from jstarks/lcow_command_args
LCOW: pass command arguments without extra quoting
Upstream-commit: bcc8b2bc474ef2536bd4bac93a7b94935efaf57a
Component: engine
2017-06-26 21:46:29 -07:00
442b559ba0 Merge pull request #31497 from dnephin/engine-local-image-data
Add a LastTagTime for images
Upstream-commit: 8f3c526e7851e047f5dde31bd2ce69f960e1d115
Component: engine
2017-06-26 18:51:09 -07:00