Commit Graph

5891 Commits

Author SHA1 Message Date
43d60463c2 Add support for init on services
It's already supported by `swarmkit`, and act the same as
`HostConfig.Init` on container creation.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: e401b88e59e098745744917c555d549f08353e6d
Component: engine
2018-06-07 14:17:55 +02:00
67fe100ad5 api: add configurable MaskedPaths and ReadOnlyPaths to the API
This adds MaskedPaths and ReadOnlyPaths options to HostConfig for containers so
that a user can override the default values.

When the value sent through the API is nil the default is used.
Otherwise the default is overridden.

Adds integration tests for MaskedPaths and ReadonlyPaths.

Signed-off-by: Jess Frazelle <acidburn@microsoft.com>
Upstream-commit: 3694c1e34e40fa2e255a97b5541645cec9c8d1d5
Component: engine
2018-06-05 12:33:14 -04:00
0e7017a3ae Merge pull request #36688 from cpuguy83/volumes_service
Extract volume interaction to a volumes service
Upstream-commit: 5037c5a8ce762b46638378b7a7d1081572eadba1
Component: engine
2018-06-05 02:16:20 +02:00
285424548c Move plugin client to separate interface
This makes it a bit simpler to remove this interface for v2 plugins
and not break external projects (libnetwork and swarmkit).

Note that before we remove the `Client()` interface from `CompatPlugin`
libnetwork and swarmkit must be updated to explicitly check for the v1
client interface as is done int his PR.

This is just a minor tweak that I realized is needed after trying to
implement the needed changes on libnetwork.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 7c77df8acc597cd4f540d873de5fe53a3d414ba9
Component: engine
2018-05-30 15:22:10 -04:00
75e47b0eb6 Merge pull request #36874 from kolyshkin/stop-timeout
daemon.ContainerStop(): fix for a negative timeout
Upstream-commit: b85799b63fb25423620ed16e717a99401cd3a39b
Component: engine
2018-05-30 13:38:42 -04:00
231bc963aa Merge pull request #37017 from cpuguy83/plugins_addr
Move plugin client creation to the extension point
Upstream-commit: cbbe0ff5119c7e3480827f57a2210f03152644f7
Component: engine
2018-05-30 09:54:46 -07:00
f21c0ccd84 Merge pull request #37131 from kolyshkin/top-opt
Optimize ContainerTop() aka docker top
Upstream-commit: 65bd038fc5e47ed37d2702cbdd6ce484d320380b
Component: engine
2018-05-30 03:28:07 +02:00
4c7a840b18 Move plugin client creation to the extension point
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: f51a96c0165fdcbbe11f62b66b582b7e202f211b
Component: engine
2018-05-25 15:18:53 -04:00
feb51dbad1 Extract volume interaction to a volumes service
This cleans up some of the package API's used for interacting with
volumes, and simplifies management.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: e4b6adc88e967971de634596654d9bc33e7bd7e0
Component: engine
2018-05-25 14:21:07 -04:00
034a285949 ContainerTop: improve error message
If "ps" fails, in many cases it prints a meaningful error message
which a user can benefit from. Let's use it.

While at it, let's use errdefs.System to classify the error,
as well as errors.Wrap.

Before:

> $ docker top $CT <any bad ps options>
> Error response from daemon: Error running ps: exit status 1

After:

> $ docker top $CT auxm
> Error response from daemon: ps: error: thread display conflicts with forest display

or

> $ docker top $CT saur
> Error response from daemon: ps: error: conflicting format options

or, if there's no meaningful error on stderr, same as before:

> $ docker top $CT 1234
> Error response from daemon: ps: exit status 1

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: a41328d5704b8d1adbcd099fb4bb0697060df806
Component: engine
2018-05-24 18:24:36 -07:00
b7dcd71938 ContainerTop: speed up
Current ContainerTop (a.k.a. docker top) implementation uses "ps"
to get the info about *all* running processes, then parses it, then
filters the results to only contain PIDs used by the container.
Collecting data only to throw most of it away is inefficient,
especially on a system running many containers (or processes).
For example, "docker top" on a container with a single process
can take up to 0.5 seconds to execute (on a mostly idle system)
which is noticeably slow.

Since the containers PIDs are known beforehand, let's use ps's
"-q" option to provide it with a list of PIDs we want info about.

The problem with this approach is, some ps options can't be used
with "-q" (the only one I'm aware of is "f" ("forest view") but
there might be more). As the list of such options is not known,
in case ps fails, it is executed again without "q" (retaining
the old behavior).

Next, the data produced by "ps" is filtered in the same way as before.
The difference here is, in case "-q" worked, the list is much shorter.

I ran some benchmarks on my laptop, with about 8000 "sleep" processes
running to amplify the savings.

The improvement in "docker top" execution times is 5x to 10x (roughly
0.05s vs 0.5s).

The improvement in ContainerTop() execution time is up to 100x
(roughly 3ms vs 300ms).

I haven't measured the memory or the CPU time savings, guess those
are not that critical.

NOTE that busybox ps does not implement -q so the fallback is always
used, but AFAIK it is not usable anyway and Docker expects a normal
ps to be on the system (say the list of fields produced by
"busybox ps -ef" differs from normal "ps -ef" etc.).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: a076badb8b33f1ecdc5d46f0a3701f10c0579f73
Component: engine
2018-05-24 18:24:27 -07:00
8c68c67c26 Merge pull request #37136 from muesli/sdnotify-api-const
Use go-systemd const instead of magic string in Linux version of dockerd
Upstream-commit: 80bfcc3a7a870c5b3683489ebc6fec9d67a3dfc4
Component: engine
2018-05-24 14:30:28 +02:00
1a3da70ee0 Merge pull request #37108 from thaJeztah/remove_unneeded_aliases
Various code-cleanup
Upstream-commit: 8e2f9203065987116aec9e2d2a1d5c7039e1a5d4
Component: engine
2018-05-24 01:38:56 +02:00
75bf6c61a8 Adapt listeners to upstream API changes in go-systemd
Signed-off-by: Christian Muehlhaeuser <muesli@gmail.com>
Upstream-commit: 703c3c573599e20e1059d18f74cb4a05647587ae
Component: engine
2018-05-23 21:57:30 +02:00
0f96e98e12 Various code-cleanup
remove unnescessary import aliases, brackets, and so on.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f23c00d8701e4bd0f2372a586dacbf66a26f9a51
Component: engine
2018-05-23 17:50:54 +02:00
843828ae85 daemon/parsePsOutput: minor optimisation
It does not make sense to keep looking for PID once
we found it, so let's give it a break.

The side effect of this patch is, if there's more than one column
titled "PID", the last (rightmost) column was used before, and now
the first (leftmost) column is used. Should make no practical
difference whatsoever.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 654a7625fc9f0b7b04da0e0e4d151af04a65cc7f
Component: engine
2018-05-22 16:24:13 -07:00
c0a75ea18d Output network attachment task information
Adds functionality to parse and return network attachment spec
information. Network attachment tasks are phony tasks created in
swarmkit to deal with unmanaged containers attached to swarmkit. Before
this change, attempting `docker inspect` on the task id of a network
attachment task would result in an empty task object. After this change,
a full task object is returned

Fixes #26548 the correct way.

Signed-off-by: Drew Erny <drew.erny@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5b69ff466e61fa168f24869710f2070c742a5565
Component: engine
2018-05-22 23:36:30 +02:00
60c2eac8bf daemon unit tests: skip some if non-root
This prevents the following test case failures "go test" is run
as non-root in the daemon/ directory:

> --- FAIL: TestContainerInitDNS (0.02s)
> 	daemon_test.go:209: chown /tmp/docker-container-test-054812199/volumes: operation not permitted
>
> --- FAIL: TestDaemonReloadNetworkDiagnosticPort (0.00s)
>	reload_test.go:525: mkdir /var/lib/docker/network/files/: permission denied
> --- FAIL: TestRootMountCleanup (0.00s)
> 	daemon_linux_test.go:240: assertion failed: error is not nil: operation not permitted

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 16670ed4842b1ee4853ba39b6ebf2b771d28db9a
Component: engine
2018-05-22 13:12:29 -07:00
b21cab8bb2 Merge pull request #37110 from thaJeztah/remove-testutil-errorcontains
Replace deprecated testutil.ErrorContains()
Upstream-commit: 9bd5d9912f0cb4fe22b994ac1e7e10038e3be65a
Component: engine
2018-05-21 11:04:45 -07:00
fac8e1d52b Replace deprecated testutil.ErrorContains()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 55bebbaecf5e40db9d83b28080ce08dc8642f407
Component: engine
2018-05-21 00:13:04 +02:00
91ebfe260a aufs: log reason why aufs is not supported.
In case aufs driver is not supported because supportsAufs() said so,
it is not possible to get a real reason from the logs.

To fix, log the error returned.

Note we're not using WithError here as the error message itself is the
sole message we want to print (i.e. there's nothing to add to it).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 91f85d1c784f3dc9d892b2af2f51d6b6f3b0be69
Component: engine
2018-05-20 12:02:41 +02:00
30e8f43743 aufs: use a single logger
Simplify the code by using a single logger instance.

While at it, use WithError in Umount.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: c6e2af54256211b8ac757e9b25caa6fb6c9b3c6e
Component: engine
2018-05-20 12:02:12 +02:00
c03259c23d Merge pull request #37022 from cgxu519/overlayutils-addmsg
Add additional message when backendfs is extfs without d_type support
Upstream-commit: f9dd74deee364dd513615ea112e78008a946b3e5
Component: engine
2018-05-19 22:38:45 +02:00
3b0918309f Merge pull request #37088 from ohbarye/fix-typos-duplicated-the
Fix typos: remove duplicated "the"
Upstream-commit: 3646562a5cfbf93f987d0ccffe7ca1c3487036e2
Component: engine
2018-05-19 20:57:47 +02:00
1ed95ff31a Merge pull request #37091 from thaJeztah/fix-linting-errors
Fix some linting issues
Upstream-commit: 1b75e175842e5ef59284a1c8ca7d64e4d6b62384
Component: engine
2018-05-19 08:39:39 -10:00
c3521e7da6 Add additional message when backendfs is extfs without d_type support
ext4 support d_type by default, but filetype feature is a tunable so
there is still a chance to disable it for some reasons. In this case,
print additional message to explicitly tell how to support d_type.

Signed-off-by: Chengguang Xu <cgxu519@gmx.com>
Upstream-commit: 8a21b128d4deb874c05eb81ebbc1265175ad69ba
Component: engine
2018-05-18 10:32:47 +08:00
e4bec6da7c Merge pull request #37084 from nwneisen/update-ring-buffer-documents
Update documentation for RingLogger's ring buffer
Upstream-commit: 59698778fe9301225c456886f7b67d2e43703a05
Component: engine
2018-05-17 12:10:44 -07:00
7f78f7fe15 Fix some linting issues
These showed locally when running `make validate`. CI doesn't seem to have the
same (possibly it's disabled in the configuration)

    builder/fscache/fscache.go:618::error: github.com/docker/docker/vendor/github.com/tonistiigi/fsutil.StatInfo composite literal uses unkeyed fields (vet)
    client/swarm_unlock_test.go:44::error: github.com/docker/docker/api/types/swarm.UnlockRequest composite literal uses unkeyed fields (vet)
    client/swarm_unlock_test.go:20::error: github.com/docker/docker/api/types/swarm.UnlockRequest composite literal uses unkeyed fields (vet)
    cmd/dockerd/daemon_unix.go:113::error: github.com/docker/docker/cmd/dockerd/hack.MalformedHostHeaderOverride composite literal uses unkeyed fields (vet)
    cmd/dockerd/daemon_unix.go:110::error: github.com/docker/docker/cmd/dockerd/hack.MalformedHostHeaderOverride composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay/overlay.go:171::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay/overlay.go:413::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay2/overlay.go:203::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay2/overlay.go:584::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/zfs/zfs.go:109::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/zfs/zfs.go:388::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/volumes_windows.go:27::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    integration/service/network_test.go:31::error: github.com/docker/docker/api/types/network.NetworkingConfig composite literal uses unkeyed fields (vet)
    api/server/server.go:129:10⚠️ should not use basic type string as key in context.WithValue (golint)
    integration/service/network_test.go:54::error: github.com/docker/docker/api/types/network.NetworkingConfig composite literal uses unkeyed fields (vet)
    libcontainerd/client_daemon_linux.go:61::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    libcontainerd/client_daemon_linux.go:74::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    pkg/archive/archive_windows.go:76::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    plugin/manager_linux.go:56::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 4f8c870d623c63b98f8ef8002448f07f4dda4aa9
Component: engine
2018-05-17 19:28:27 +02:00
3d5979d637 Merge pull request #37075 from tuscloud/fix-typos
Fix typos
Upstream-commit: 9aac187ce1592249ab5f1556d362f2b48ff9c671
Component: engine
2018-05-17 16:00:43 +02:00
111037b920 Fix typos: remove duplicated "the"
Signed-off-by: Masato Ohba <over.rye@gmail.com>
Upstream-commit: 0f95b23d98384a3ae3769b75292cd5b14ba38437
Component: engine
2018-05-17 21:49:51 +09:00
7321d16dc7 Update documentation for RingLogger's ring buffer
Signed-off-by: Nick Neisen <nwneisen@gmail.com>
Upstream-commit: 3521d534e5c9338ae5605d1228554862e922a8a9
Component: engine
2018-05-16 18:52:50 -06:00
9e6796b652 Fix typos
Signed-off-by: weipeng <weipeng@tuscloud.io>
Upstream-commit: f781ec45b5c1745279f0429c2b7f19c793efcb85
Component: engine
2018-05-16 09:15:43 +08:00
131639fdf7 Merge pull request #37028 from cpuguy83/log_plugin_broken_pipe
Fix logging plugin crash unrecoverable
Upstream-commit: 6821ffd1a11f188d6b1c3e8da01885768e027586
Component: engine
2018-05-15 13:57:01 +02:00
9ebdda8c43 Merge pull request #37033 from cpuguy83/some_container_network_cleanups
Some container/network cleanups
Upstream-commit: 6f68db8c7a94489b5e66bb828f748d0baa2e5c22
Component: engine
2018-05-15 13:23:41 +02:00
14395e1de7 Merge pull request #37062 from cpuguy83/fix_logs_daemon_race
Fix race conditions in logs API
Upstream-commit: 07dc3aa685ffd500ada1c41f8a9b7eaf74cf09eb
Component: engine
2018-05-15 11:59:40 +02:00
f2fea2ebf4 Merge pull request #37063 from cpuguy83/fix_log_rotation_issue
Fix some issues in logfile reader and rotation
Upstream-commit: 26da99e326bf7cbfad820395f5379bba7a66c4a5
Component: engine
2018-05-15 11:46:29 +02:00
4347082deb Merge pull request #37060 from cpuguy83/fix_swagger_gen
Fix swagger volume type generation
Upstream-commit: bcd86fc2acd3ef22a7fc6c662c7ae8a67b0bb4f4
Component: engine
2018-05-15 11:46:02 +02:00
9488201604 Fix logging plugin crash unrecoverable
In cases where a logging plugin has crashed when the daemon tries to
copy the container stdio to the logging plugin it returns a broken pipe
error and any log entries that occurr while the plugin is down are lost.

Fix this by opening read+write in the daemon so logs are not lost while
the plugin is down.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: e7479e3ab8128f9e84cc640f0bed4e77b268a6e9
Component: engine
2018-05-14 16:51:56 -04:00
b600301f4e Fix some issues in logfile reader and rotation
- Check errors.Cause(err) when comparing errors
- Fix bug where oldest log file is not actually removed. This in
particular causes issues when compression is enabled. On rotate it just
overwrites the data in the log file corrupting it.
- Use O_TRUNC to open new gzip files to ensure we don't corrupt log
files as happens without the above fix.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: e87e9e6ad6ba501cc42a2ef47ac18c88a68f258f
Component: engine
2018-05-14 15:52:18 -04:00
5e354ed533 Fix race conditions in logs API
Closing the log driver was in a defer meanwhile logs are
collected asyncronously, so the log driver was being closed before reads
were actually finished.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 2c252a48c252749d41079cf8c450d00a5c18296e
Component: engine
2018-05-14 15:48:32 -04:00
1cb0dc30a7 Fix swagger volume type generation
This was broken by bf6a790f00ab96bb8857e3c73502909ee5e36217

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: b16b125bb4326e2eec9948fd54ca8c5d83eba36a
Component: engine
2018-05-14 13:46:20 -04:00
de8a0bba09 Merge pull request #37034 from cpuguy83/moar_metrics
Add metrics for log failures/partials
Upstream-commit: ab0dccf80174fa5eef4583a0235b0bf57bdff8df
Component: engine
2018-05-14 18:05:30 +02:00
70cd643c8a Merge pull request #37040 from thaJeztah/error_on_unsupported_options
overlay: do not ignore invalid storage-driver options
Upstream-commit: a79d04ae55196f73d67eeb8a40cfbeb67fed95e9
Component: engine
2018-05-14 15:37:33 +02:00
7fb4806d90 Merge pull request #36991 from kolyshkin/slice-in-place
daemon.setMounts(): copy slice in place
Upstream-commit: 31aca4bef4d976ae4e7fb559da52745a3a295100
Component: engine
2018-05-14 13:49:47 +02:00
e72e7fdac6 Merge pull request #37045 from abhi/inspect
Fixing network inspect for swarm
Upstream-commit: 8baea3a1bb9a10e4148e320d8933b0a73b7e0b07
Component: engine
2018-05-14 12:51:12 +02:00
4ca879c241 Fixing network inspect for swarm
Signed-off-by: Abhinandan Prativadi <abhi@docker.com>
Upstream-commit: a059d6f4f5eb5fdb319b6f715a9bb50eb5d6173d
Component: engine
2018-05-13 07:31:14 -07:00
3d482c77f6 Merge pull request #37041 from thaJeztah/remove_unused_function_in_test
overlay2: remove unused cdMountFrom() helper function
Upstream-commit: d0a516e2567a67fed3ff3f23ff6c220f1707ad4b
Component: engine
2018-05-11 13:56:56 -04:00
0178ff5bea Merge pull request #37031 from kolyshkin/getmnt
Fix daemon.getSourceMount() for /
Upstream-commit: 379845ec20ae651831dcfb0587b95c38a1631a53
Component: engine
2018-05-11 16:39:57 +02:00
b80e078a6d Merge pull request #36815 from allencloud/simplify-ode
refactor: simplify code to make function getExecConfig  more readable
Upstream-commit: 64336838878281549d87b34b19cf2e4b3756e724
Component: engine
2018-05-11 10:06:33 -04:00
0d708bdf61 overlay2: remove unused cdMountFrom() helper function
This function was added in 23e5c94cfb26eb72c097892712d3dbaa93ee9bc0 but never used

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8a8ec00c1fece033b20548b7a5ec2f3a1f340834
Component: engine
2018-05-11 13:51:50 +02:00