Commit Graph

5720 Commits

Author SHA1 Message Date
a196815f55 Merge pull request #33702 from aaronlehmann/templated-secrets-and-configs
Templated secrets and configs
Upstream-commit: 0076343b29f508a5deb06861c0d85748659f8881
Component: engine
2018-02-21 13:39:10 +01:00
ad01430349 Merge pull request #35829 from cpuguy83/no_private_mount_for_plugins
Perform plugin mounts in the runtime
Upstream-commit: 20028325daab4fcbee9c8e28f43dbfb2b1c5d568
Component: engine
2018-02-21 12:28:13 +01:00
6899375641 Merge pull request #33922 from ishidawataru/sctp
Support SCTP port mapping (bump up API to v1.37)
Upstream-commit: 079ed017b61eb819b8184b90013ce89465d3aaba
Component: engine
2018-02-20 17:00:13 +01:00
ded970f96e Merge pull request #36107 from cpuguy83/cleanup_daemon_root_mount
Ensure daemon root is unmounted on shutdown
Upstream-commit: eb033c11753872c6269304d8127dbfe7315e476f
Component: engine
2018-02-20 10:04:41 +01:00
0c0ee07b87 Merge pull request #36330 from vdemeester/migrate-container-list-tests
test: clean/migrate some docker ps cli-only integration tests 
Upstream-commit: 7060a40addd71605862b668c1ec2b889e3540233
Component: engine
2018-02-20 08:44:17 +01:00
ed5d3f9cce Support SCTP port mapping (bump up API to v1.37)
Signed-off-by: Wataru Ishida <ishida.wataru@lab.ntt.co.jp>
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Upstream-commit: 8e435b8279f2af3e0cebd73fa9e25ca1bb26004e
Component: engine
2018-02-20 11:15:36 +09:00
39cad2aa10 Merge pull request #36256 from wcwxyz/fix-refcounter-memory-leak
graphdriver: Fix RefCounter memory leak
Upstream-commit: 733ed2ddd3c621dadafbb74feb7b80d20fd3fd6f
Component: engine
2018-02-19 10:32:14 -08:00
c2407fa324 Clean some cli-only integration tests
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 641c73d211d7efe15255ae36ab2362cef3584260
Component: engine
2018-02-19 11:19:19 +01:00
e46642afe1 Merge pull request #35510 from ripcurld0/fix_35500
Display a warn message when there is binding ports and net mode is host
Upstream-commit: 35d69f10a9ac7479095f39358abb78ee6d3ab65f
Component: engine
2018-02-19 08:57:36 +01:00
da22cbc58d Display a warn message when there is binding ports and net mode is host
When a container is created if "--network" is set to "host" all the
ports in the container are bound to the host.
Thus, adding "-p" or "--publish" to the command-line is meaningless.

Unlike "docker run" and "docker create", "docker service create" sends
an error message when network mode is host and port bindings are given

This patch however suggests to send a warning message to the client when
such a case occurs.

The warning message is added to "warnings" which are returned from
"verifyPlatformContainerSettings".

Signed-off-by: Boaz Shuster <ripcurld.github@gmail.com>
Upstream-commit: 6e78fdb790d2e1dbf95a1733cab9395b1b936622
Component: engine
2018-02-18 13:28:44 +00:00
dcede7a976 Merge pull request #36303 from dnephin/cleanup-in-daemon-unix
Cleanup unnecessary and duplicate functions in `daemon_unix.go`
Upstream-commit: 747c163a65365933c5d6d7f0740f2ac8e3775287
Component: engine
2018-02-16 14:55:18 -08:00
f68c84b9a0 Merge configs/secrets in unix implementation
On unix, merge secrets/configs handling. This is important because
configs can contain secrets (via templating) and potentially a config
could just simply have secret information "by accident" from the user.
This just make sure that configs are as secure as secrets and de-dups a
lot of code.
Generally this makes everything simpler and configs more secure.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: c02171802b788fb2d4d48bebcee2a57c8eabeeaa
Component: engine
2018-02-16 11:25:14 -05:00
850e2bff8c Always mount configs with tmpfs
This makes configs and secrets behavior identical.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 8e8f5f4457d8e1b02031576dbc18c903be4bcfb6
Component: engine
2018-02-16 11:25:14 -05:00
40e1524cb3 daemon: Check return value of createSecretDir
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 426f4e48e3e53b2445835585d7957043a5fe6ab3
Component: engine
2018-02-16 11:25:14 -05:00
599f92e497 Store configs that contain secrets on tmpfs
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: cd3d0486a6f62afac50f2cf74e2b9d8728848c97
Component: engine
2018-02-16 11:25:14 -05:00
fc6a93f926 api: Add Templating parameter to SecretSpec and ConfigSpec
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: c5df7235f6a4811f26b37441db401f6b04858504
Component: engine
2018-02-16 11:25:13 -05:00
a6e6cffaed executor: Use a TemplatedDependencyGetter to support template expansion
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 56da5fd7d31c9a627fc6a3c482cb0bf0ffb2d26e
Component: engine
2018-02-16 11:25:13 -05:00
b38d6149be Merge pull request #36306 from cpuguy83/fix_logopt_validator_plugins
Move log validator logic after plugins are loaded
Upstream-commit: 04d97267b6b57987030862c0d355c9a49eb0e445
Component: engine
2018-02-16 11:52:23 +01:00
963b00a076 Ensure daemon root is unmounted on shutdown
This is only for the case when dockerd has had to re-mount the daemon
root as shared.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 487c6c7e73dbb7871e80d75f176dd2a3539a2947
Component: engine
2018-02-15 15:58:20 -05:00
b4446f4926 Move log validator logic after plugins are loaded
This ensures that all log plugins are registered when the log validator
is run.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: b0b9a25e7e60abbe143e149ccaaf4dfb62044016
Component: engine
2018-02-15 11:53:11 -05:00
78ec305ac2 Remove unused method from multireader package
Signed-off-by: Weerasak Chongnguluam <singpor@gmail.com>
Upstream-commit: 6e5fba98a53a832dc1654d87637df14c5b9ab2f1
Component: engine
2018-02-15 23:10:56 +07:00
e8ddf74a02 Test invalid filter and move validation on top
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 05e7f2cf58bdd6c1aaf3da9a92ebcd54cdde6d09
Component: engine
2018-02-15 16:24:26 +01:00
66a715574f Merge pull request #36055 from cpuguy83/slave_mounts_for_root
Use rslave propagation for mounts from daemon root
Upstream-commit: ea34f827112b3837e5349827f6309a37217854cb
Component: engine
2018-02-15 12:57:25 +01:00
a592702237 Remove duplicate rootFSToAPIType
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 4ceea53b5e6a86c39122e99f6ffbc1142d28a174
Component: engine
2018-02-14 11:59:18 -05:00
9e1c0d7187 Remove unnecessary getLayerInit
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: c502bcff33e10be55f15366e123b25574016a9af
Component: engine
2018-02-14 11:59:10 -05:00
8bb8847f9c Merge pull request #36237 from cpuguy83/zfs_do_not_unmount
Do not recursive unmount on cleanup of zfs/btrfs
Upstream-commit: 68c3201626439d5be5c24d14d4fe7e27fe93954d
Component: engine
2018-02-14 09:49:17 -05:00
9ab4ce343a Merge pull request #36305 from cpuguy83/35370_fix_logs_eof
Fix log tail with empty logs
Upstream-commit: e698b6e098ac696daa1a7a80d4bdac08b305b328
Component: engine
2018-02-14 16:41:22 +09:00
0fb8610c54 Fix log tail with empty logs
When tailing a container log, if the log file is empty it will cause the
log stream to abort with an unexpected `EOF`.
Note that this only applies to the "current" log file as rotated files
cannot be empty.

This fix just skips adding the "current" file the log tail if it is
empty.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: f40860c5f3d3575629d4a932207e866c1fea625d
Component: engine
2018-02-13 21:33:05 -05:00
f914512cd5 Merge pull request #36271 from dnephin/use-tag-image-in-commit
Use TagImage in Commit
Upstream-commit: 060893ab5d97d6582770224db3623ad1b3bdb6e1
Component: engine
2018-02-13 16:40:17 -05:00
4adc380b90 Fix typos in daemon
Signed-off-by: bin liu <liubin0329@gmail.com>
Upstream-commit: b00a67be6e3d3f241879110bd342abaa8e23cbac
Component: engine
2018-02-10 19:42:54 +08:00
81a9d51f50 Merge pull request #36242 from kolyshkin/rwlayer-nil-deref
c.RWLayer: check for nil before use
Upstream-commit: ab3ea81376441e93311c94910be78ac42292c496
Component: engine
2018-02-09 18:13:03 -08:00
8382b77c1c Use TagImage in Commit
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: afb3eda697efebf18d8ac3bbcfd911a5968081e3
Component: engine
2018-02-09 20:53:39 -05:00
89a07f6033 c.RWLayer: check for nil before use
Since commit e9b9e4ace294230c6b8eb has landed, there is a chance that
container.RWLayer is nil (due to some half-removed container). Let's
check the pointer before use to avoid any potential nil pointer
dereferences, resulting in a daemon crash.

Note that even without the abovementioned commit, it's better to perform
an extra check (even it's totally redundant) rather than to have a
possibility of a daemon crash. In other words, better be safe than
sorry.

[v2: add a test case for daemon.getInspectData]
[v3: add a check for container.Dead and a special error for the case]

Fixes: e9b9e4ace294230c6b8eb
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 195893d38160c0893e326b8674e05ef6714aeaa4
Component: engine
2018-02-09 11:24:09 -08:00
f74751654b graphdriver: Fix RefCounter memory leak
Signed-off-by: WANG Chao <chao.wang@ucloud.cn>
Upstream-commit: 9015a05606a9bb80f0d8d2e3d43b0b682ca53db4
Component: engine
2018-02-09 10:26:06 +08:00
8991a8a5bd Merge pull request #36220 from dnephin/support-proxy-in-splunk-driver
Support a proxy in splunk log driver
Upstream-commit: f653485e57a36518085868dcb48b4d924a520877
Component: engine
2018-02-08 15:32:19 +01:00
e09b72af21 Merge pull request #36224 from dnephin/refactor-commit
Refactor Daemon.Commit()
Upstream-commit: 9769ef333f2af24b30fed0dd7b00384b2df3b953
Component: engine
2018-02-08 21:02:30 +09:00
fd90b16ff5 Merge pull request #36194 from dnephin/add-canonical-import
Add canonical import path
Upstream-commit: 3a633a712c8bbb863fe7e57ec132dd87a9c4eff7
Component: engine
2018-02-07 13:06:45 -08:00
90450b2044 Ensure plugin returns correctly scoped paths
Before this change, volume management was relying on the fact that
everything the plugin mounts is visible on the host within the plugin's
rootfs. In practice this caused some issues with mount leaks, so we
changed the behavior such that mounts are not visible on the plugin's
rootfs, but available outside of it, which breaks volume management.

To fix the issue, allow the plugin to scope the path correctly rather
than assuming that everything is visible in `p.Rootfs`.
In practice this is just scoping the `PropagatedMount` paths to the
correct host path.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 0e5eaf8ee32662182147f5f62c1bfebef66f5c47
Component: engine
2018-02-07 15:48:27 -05:00
3928c278e5 Plugins perform propagated mount in runtime spec
Setting up the mounts on the host increases chances of mount leakage and
makes for more cleanup after the plugin has stopped.
With this change all mounts for the plugin are performed by the
container runtime and automatically cleaned up when the container exits.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: a53930a04fa81b082aa78e66b342ff19cc63cc5f
Component: engine
2018-02-07 15:48:27 -05:00
c932e5d2a5 Use runtime spec modifier for metrics plugin hook
Currently the metrics plugin uses a really hackish host mount with
propagated mounts to get the metrics socket into a plugin after the
plugin is alreay running.
This approach ends up leaking mounts which requires setting the plugin
manager root to private, which causes some other issues.

With this change, plugin subsystems can register a set of modifiers to
apply to the plugin's runtime spec before the plugin is ever started.
This will help to generalize some of the customization work that needs
to happen for various plugin subsystems (and future ones).

Specifically it lets the metrics plugin subsystem append a mount to the
runtime spec to mount the metrics socket in the plugin's mount namespace
rather than the host's and prevetns any leaking due to this mount.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 426e610e43179d58b29c496bc79a53f410a4b1e1
Component: engine
2018-02-07 15:48:26 -05:00
4ec1fd107e Refactor commit
The goal of this refactor is to make it easier to integrate buildkit
and containerd snapshotters.

Commit is used from two places (api and build), each calls it
with distinct arguments. Refactored to pull out the common commit
logic and provide different interfaces for each consumer.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: daff039049aea6e19a4bda1df2834d14b4198bc0
Component: engine
2018-02-07 15:09:06 -05:00
33ddc6d172 Do not recursive unmount on cleanup of zfs/btrfs
This was added in #36047 just as a way to make sure the tree is fully
unmounted on shutdown.

For ZFS this could be a breaking change since there was no unmount before.
Someone could have setup the zfs tree themselves. It would be better, if
we really do want the cleanup to actually the unpacked layers checking
for mounts rather than a blind recursive unmount of the root.

BTRFS does not use mounts and does not need to unmount anyway.
These was only an unmount to begin with because for some reason the
btrfs tree was being moutned with `private` propagation.

For the other graphdrivers that still have a recursive unmount here...
these were already being unmounted and performing the recursive unmount
shouldn't break anything. If anyone had anything mounted at the
graphdriver location it would have been unmounted on shutdown anyway.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 2fe4f888bee52b1f256d6fa5e20f9b061d30221c
Component: engine
2018-02-07 15:08:17 -05:00
f68b87fd47 Support a proxy in splunk log driver
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 3c4537d5b33d951237ea5e4cc123953eda7a37e7
Component: engine
2018-02-07 14:52:32 -05:00
49c61840e2 Use rslave propagation for mounts from daemon root
By default, if a user requests a bind mount it uses private propagation.
When the source path is a path within the daemon root this, along with
some other propagation values that the user can use, causes issues when
the daemon tries to remove a mountpoint because a container will then
have a private reference to that mount which prevents removal.

Unmouting with MNT_DETATCH can help this scenario on newer kernels, but
ultimately this is just covering up the problem and doesn't actually
free up the underlying resources until all references are destroyed.

This change does essentially 2 things:

1. Change the default propagation when unspecified to `rslave` when the
source path is within the daemon root path or a parent of the daemon
root (because everything is using rbinds).
2. Creates a validation error on create when the user tries to specify
an unacceptable propagation mode for these paths...
basically the only two acceptable modes are `rslave` and `rshared`.

In cases where we have used the new default propagation but the
underlying filesystem is not setup to handle it (fs must hvae at least
rshared propagation) instead of erroring out like we normally would,
this falls back to the old default mode of `private`, which preserves
backwards compatibility.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 589a0afa8cbe39b6512662fd1705873e2d236dd0
Component: engine
2018-02-07 14:27:09 -05:00
0919d8f13e Merge pull request #36201 from arm64b/oom-kill-disable-fixing
Daemon: passdown the `--oom-kill-disable` option to containerd
Upstream-commit: 2e8ccbb49e21b8696ad4c39185036a8b7e50402f
Component: engine
2018-02-06 21:39:43 -08:00
d3b2389f2d Merge pull request #35414 from madhanrm/hotadd1
Enable HotAdd for Windows
Upstream-commit: e62d36bcad6ef1b1beafb15411f927b25f52c3d1
Component: engine
2018-02-06 10:40:39 -08:00
be83c11fb0 Add canonical import comment
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 4f0d95fa6ee7f865597c03b9e63702cdcb0f7067
Component: engine
2018-02-05 16:51:57 -05:00
44aff4f98f Merge pull request #36191 from cpuguy83/fix_attachable_network_race
Fix race in attachable network attachment
Upstream-commit: 6987557e0cef9bd139128e62d86586a40cda6036
Component: engine
2018-02-05 09:41:35 -08:00
fa06c65a30 Daemon: passdown the --oom-kill-disable option to containerd
Current implementaion of docke daemon doesn't pass down the
`--oom-kill-disable` option specified by the end user to the containerd
when spawning a new docker instance with help from `runc` component, which
results in the `--oom-kill-disable` doesn't work no matter the flag is `true`
or `false`.

This PR will fix this issue reported by #36090

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Upstream-commit: 44b074d199de84b9af8cc94005fbed4f76bd9ab8
Component: engine
2018-02-05 03:25:59 +00:00
e73d8c24d7 Libnetwork revendoring
Diff:
5ab4ab8300...20dd462e0a

- Memberlist revendor (fix for deadlock on exit)
- Network diagnostic client
- Fix for ndots configuration

Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
Upstream-commit: ec86547244fa329148a096db56f9ade77a7ce7eb
Component: engine
2018-02-02 14:36:32 -08:00