Commit Graph

5821 Commits

Author SHA1 Message Date
e3af8e0774 Switch from x/net/context -> context
Since Go 1.7, context is a standard package. Since Go 1.9, everything
that is provided by "x/net/context" is a couple of type aliases to
types in "context".

Many vendored packages still use x/net/context, so vendor entry remains
for now.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 7d62e40f7e4f3c17d229a7687d6fcca5448de813
Component: engine
2018-04-23 13:52:44 -07:00
5a3297f67b Merge pull request #36913 from vdemeester/test-skip-non-root
Skip some tests requires root uid when run as user…
Upstream-commit: 5c233cf4315747785b27d4b0f0631d220556205f
Component: engine
2018-04-23 11:42:42 -07:00
4521f88522 Merge pull request #36920 from kolyshkin/cancel-func
context.WithTimeout: do call the cancel func
Upstream-commit: 20b524bf2e91466e00cd5fadd87e9d49f8cb8cb3
Component: engine
2018-04-23 20:14:25 +02:00
7f23cb8c8b Skip some tests requires root uid when run as user
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: a7999aaa536818085ec8a23901ef64637587b7e4
Component: engine
2018-04-23 10:14:39 +02:00
554b5427fa Merge pull request #36091 from kolyshkin/mount
pkg/mount improvements
Upstream-commit: 53982e3fc1642f2605189cea48bc6de7b9318611
Component: engine
2018-04-21 11:03:54 +02:00
73e14f67f8 Merge pull request #36879 from cpuguy83/extra_unmount_check
Extra check before unmounting on shutdown
Upstream-commit: cf9c48bb3ecf47ed56a4b7b466eda577913f838e
Component: engine
2018-04-20 17:08:11 -07:00
d80d707e96 context.WithTimeout: do call the cancel func
govet complains (when using standard "context" package):

> the cancel function returned by context.WithTimeout should be called,
> not discarded, to avoid a context leak (vet)

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 05e2f7e2fafd0fbc818c9f4cda7ac513c785d49c
Component: engine
2018-04-20 12:27:13 -07:00
57e3475a77 Merge pull request #36492 from alejgh/36478-log-standardization
36478 storage-drivers log format standardization
Upstream-commit: c4ba5be6af4aa960d2f6d2071b2218cf0b62bb20
Component: engine
2018-04-20 12:11:14 +02:00
cd146e5f0d getSourceMount(): simplify
The flow of getSourceMount was:
 1 get all entries from /proc/self/mountinfo
 2 do a linear search for the `source` directory
 3 if found, return its data
 4 get the parent directory of `source`, goto 2

The repeated linear search through the whole mountinfo (which can have
thousands of records) is inefficient. Instead, let's just

 1 collect all the relevant records (only those mount points
   that can be a parent of `source`)
 2 find the record with the longest mountpath, return its data

This was tested manually with something like

```go
func TestGetSourceMount(t *testing.T) {
	mnt, flags, err := getSourceMount("/sys/devices/msr/")
	assert.NoError(t, err)
	t.Logf("mnt: %v, flags: %v", mnt, flags)
}
```

...but it relies on having a specific mount points on the system
being used for testing.

[v2: add unit tests for ParentsFilter]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 871c957242df9f8c74faf751a2f14eb5178d4140
Component: engine
2018-04-19 14:49:17 -07:00
7e50bdbeb9 daemon.cleanupMounts(): use mount.SingleEntryFilter
Use mount.SingleEntryFilter as we're only interested in a single entry.

Test case data of TestShouldUnmountRoot is modified accordingly, as
from now on:

1. `info` can't be nil;

2. the mountpoint check is not performed (as SingleEntryFilter
   guarantees it to be equal to daemon.root).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: d3ebcde82aae79be8cbecab365367b17adac4b3e
Component: engine
2018-04-19 14:48:25 -07:00
a97e7a41c8 pkg/mount: implement/use filter for mountinfo parsing
Functions `GetMounts()` and `parseMountTable()` return all the entries
as read and parsed from /proc/self/mountinfo. In many cases the caller
is only interested only one or a few entries, not all of them.

One good example is `Mounted()` function, which looks for a specific
entry only. Another example is `RecursiveUnmount()` which is only
interested in mount under a specific path.

This commit adds `filter` argument to `GetMounts()` to implement
two things:
 1. filter out entries a caller is not interested in
 2. stop processing if a caller is found what it wanted

`nil` can be passed to get a backward-compatible behavior, i.e. return
all the entries.

A few filters are implemented:
 - `PrefixFilter`: filters out all entries not under `prefix`
 - `SingleEntryFilter`: looks for a specific entry

Finally, `Mounted()` is modified to use `SingleEntryFilter()`, and
`RecursiveUnmount()` is using `PrefixFilter()`.

Unit tests are added to check filters are working.

[v2: ditch NoFilter, use nil]
[v3: ditch GetMountsFiltered()]
[v4: add unit test for filters]
[v5: switch to gotestyourself]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: bb934c6aca3e77541dd4fd51b9ab2706294dadda
Component: engine
2018-04-19 14:48:09 -07:00
3f0393a442 Move mount parsing to separate package.
This moves the platform specific stuff in a separate package and keeps
the `volume` package and the defined interfaces light to import.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 6a70fd222b95643a8a6b88e2634d5f085ae4122a
Component: engine
2018-04-19 06:35:54 -04:00
068e200711 Extra check before unmounting on shutdown
This makes sure that if the daemon root was already a self-binded mount
(thus meaning the daemonc only performed a remount) that the daemon does
not try to unmount.

Example:

```
$ sudo mount --bind /var/lib/docker /var/lib/docker
$ sudo dockerd &
```

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: c403f0036b9945bd58a84e2c93f794ed9861fe99
Component: engine
2018-04-18 20:43:42 -04:00
856b3acadf Remove use of global volume driver store
Instead of using a global store for volume drivers, scope the driver
store to the caller (e.g. the volume store). This makes testing much
simpler.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 977109d808ae94eb3931ae920338b1aa669f627e
Component: engine
2018-04-17 14:07:08 -04:00
8f56510165 Remove old/uneeded volume migration from vers 1.7
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 0023abbad34282762d5bd17302776d2a8521fffc
Component: engine
2018-04-17 14:06:53 -04:00
27a6dfb147 Move direct volume driver interaction to store
Since the volume store already provides this functionality, we should
just use it rather than duplicating it.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 63826e291ba3b88443b64802084bbb3931857b56
Component: engine
2018-04-17 14:06:53 -04:00
fe0deb4aa6 Merge pull request #36738 from Microsoft/jjh/getlayerpath
Windows: Add GetLayerPath implementation in graphdriver
Upstream-commit: 5c2e6064a1c5cb816081958e5a7079bc6b474940
Component: engine
2018-04-14 09:59:38 -07:00
566b2857cf Merge pull request #35739 from thaJeztah/bump-go-1.10
Bump Golang to 1.10.1
Upstream-commit: 61138fb5fc0f5fc1799cabfa86d28fb88a0f472c
Component: engine
2018-04-13 13:56:38 -10:00
fce8015aac Merge pull request #36845 from medakk/refactor-daemon
Refactor code in cmd/dockerd/daemon.go
Upstream-commit: 3881f1a5219ba5d0a1ed54121ba20cd1700fec52
Component: engine
2018-04-13 15:26:48 -07:00
32a14263cf Merge pull request #35831 from anusha-ragunathan/splunk_partial
Improve partial message support in logger
Upstream-commit: 9689059b5636f23d77cc3ac29ee41c6fabff81ec
Component: engine
2018-04-12 17:38:27 -07:00
c7b8d02f5e Update logger adapter test to avoid race
Add synchronization around adding logs to a plugin
and reading those logs. Without the follow configuration,
a race occurs between go routines to add the logs into
the plugin and read the logs out of the plugin. This
adds a function to synchronize the action to avoid the
race.
Removes use of file for buffering, instead buffering whole
messages so log count can be checked discretely.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c208f1c8a8d57f9a7f48f63345e77146774aa7a6
Component: engine
2018-04-12 13:35:15 -07:00
ee6e3cbd82 Refactor code in cmd/dockerd/daemon.go
Signed-off-by: Karthik Karanth <karanth.karthik@gmail.com>
Upstream-commit: 05346355dbd4300a21bfd64cda93ea6c5aef0cf0
Component: engine
2018-04-13 00:14:20 +05:30
e844cef5f8 Improve partial message support in logger
Docker daemon has a 16K buffer for log messages. If a message length
exceeds 16K, it should be split by the logger and merged at the
endpoint.

This change adds `PartialLogMetaData` struct for enhanced partial support
- LastPartial (bool) : indicates if this is the last of all partials.
- ID (string)        : unique 32 bit ID. ID is same across all partials.
- Ordinal (int starts at 1) : indicates the position of msg in the series of partials.
Also, the timestamps across partials in the same.

Signed-off-by: Anusha Ragunathan <anusha.ragunathan@docker.com>
Upstream-commit: 0b4b0a7b5d5de8cb575b666312fceaa2cd58e658
Component: engine
2018-04-11 13:26:28 -07:00
c4fdce16ba Don't make container mount unbindable
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Upstream-commit: 4c000662feb3c8e3d63cbcb044a47f627cd9bb45
Component: engine
2018-04-10 15:14:34 -04:00
2cf9f7eb88 Always make sysfs read-write with privileged
It does not make any sense to vary this based on whether the
rootfs is read only. We removed all the other mount dependencies
on read-only eg see #35344.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Upstream-commit: a729853bc712910574a7417f67764ec8c523928b
Component: engine
2018-04-06 16:17:18 +01:00
d97a287d94 Merge pull request #36769 from stevvooe/defensive-attachment-processing
daemon/cluster: handle partial attachment entries during configure
Upstream-commit: 8a9e1808cfdef414e60307b4ccd96719731be304
Component: engine
2018-04-05 09:35:11 +02:00
7e2f77eef8 Merge pull request #36728 from darrenstahlmsft/LayerLeak
Fix Windows layer leak when write fails
Upstream-commit: a8260058e39f7014f811394f86ae2c0bc899e11e
Component: engine
2018-04-04 19:05:59 -07:00
95a3a5cb78 daemon/cluster: handle partial attachment entries during configure
We have seen a panic when re-joining a node to a swarm cluster. The
cause of the issue is unknown, so we just need to add a test for nil
objects and log when we get the condition. Hopefully this can prevent
the crash and we can recover the config at a later time.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: 454128c6e82cded211c1412e3eb350b1f7533ee2
Component: engine
2018-04-04 15:53:14 -07:00
28820945d1 Fix Windows layer leak when write fails
Signed-off-by: Darren Stahl <darst@microsoft.com>
Upstream-commit: 1f28844d7869609f371ab2a7881e4488a79a7e27
Component: engine
2018-04-04 12:31:23 -07:00
94036de956 Fix FIFO, sockets and device files when run in user NS
commit 617c352e9225 "Don't create devices if in a user namespace"

introduced check, which meant to skip mknod operation when run
in user namespace, but instread skipped FIFO and socket files
copy.

Signed-off-by: Maxim Ivanov <ivanov.maxim@gmail.com>
Upstream-commit: 6f084f292932c464a30b56edb9edbe238bdcf0aa
Component: engine
2018-04-01 12:33:59 +01:00
187be19082 Merge pull request #36736 from tonistiigi/remove-test-kernel-version
copy: remove kernel version test
Upstream-commit: 6c948b562d826fd95da5f972305d22c47a479494
Component: engine
2018-03-30 10:53:40 +02:00
fa2b7ebcd9 Merge pull request #36734 from cpuguy83/context_directory_size
Support cancellation in `directory.Size()`
Upstream-commit: f0b9eb8627b0bbf3cbd95edd78cc552cb3626520
Component: engine
2018-03-30 10:38:33 +02:00
85555b0bc6 Merge pull request #36726 from chungers/raft-params
Expose swarmkit's Raft tuning parameters in engine config
Upstream-commit: 8b6a8271a62eec40827bf77f32b794f650556c24
Component: engine
2018-03-29 17:19:04 -07:00
9b41292c7f Support cancellation in directory.Size()
Makes sure that if the user cancels a request that the daemon stops
trying to traverse a directory.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 9d46c4c138d7b3f7778c13fe84857712bd6c97a9
Component: engine
2018-03-29 15:49:15 -04:00
60950b5248 Windows: GetLayerPath in graphdriver
Signed-off-by: John Howard (VM) <jhoward@ntdev.microsoft.com>
Upstream-commit: 3aab84803dec4a28484b319531685276435263cd
Component: engine
2018-03-29 11:45:45 -07:00
c26d6b4944 copy: remove kernel version test
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: 4d6b8f73a89abf715fa33b3f553884c575a3b136
Component: engine
2018-03-29 10:45:43 -07:00
9a7c928884 Expose swarm raft tuning parameters in engine config
Signed-off-by: David Chung <david.chung@docker.com>
Upstream-commit: 275a1ca7c8ec1d556c37bb50ac574beffbb16e5a
Component: engine
2018-03-28 16:54:43 -07:00
e260e8f94d Remove (now) extra call to sb.DisableService()
This call was added as part of commit a042e5a20 and at the time was
useful.  sandbox.DisableService() basically calls
endpoint.deleteServiceInfoFromCluster() for every endpoint in the
sandbox.  However, with the libnetwork change, endpoint.sbLeave()
invokes endpoint.deleteServiceInfoFromCluster(). The releaseNetwork()
call invokes sandbox.Delete() immediately after
sandbox.DisableService().  The sandbox.Delete() in turn ultimately
invokes endpoint.sbLeave() for every endpoint in the sandbox which thus
removes the endpoint's load balancing entry via
endpoint.deleteServiceInfoFromCluster().  So the call to
sandbox.DisableService() is now redundant.

It is noteworthy that, while redundant, the presence of the call would
not cause errors.  It would just be sub-optimal.  The DisableService()
call would cause libnetwork to down-weight the load balancing entries
while the call to sandbox.Delete() would cause it to remove the entries
immediately afterwards.  Aside from the wasted computation, the extra
call would also propagate an extra state change in the networkDB gossip
messages.  So, overall, it is much better to just avoid the extra
overhead.

Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: c27417aa7de46daa415600b39fc8a9c411c8c493
Component: engine
2018-03-28 14:16:31 -04:00
252f993dde Standardized log messages accross the different storage drivers.
Now all of the storage drivers use the field "storage-driver" in their log
messages, which is set to name of the respective driver.
Storage drivers changed:
- Aufs
- Btrfs
- Devicemapper
- Overlay
- Overlay 2
- Zfs

Signed-off-by: Alejandro GonzÃlez Hevia <alejandrgh11@gmail.com>
Upstream-commit: 9392838150f5495a63f33fed6570ae41f5a6d62e
Component: engine
2018-03-27 14:37:30 +02:00
113504a7f3 Merge pull request #35089 from Microsoft/jjh/fromplatformbuilder
LCOW - Change platform parser directive to FROM statement flag
Upstream-commit: 29fc64b590badcb1c3f5beff7563ffd31eb58974
Component: engine
2018-03-26 14:17:49 -07:00
de2245153e Increase raft ElectionTick to 10xHeartbeatTick
Signed-off-by: Anshul Pundir <anshul.pundir@docker.com>
Upstream-commit: 6abee2008b314a65553202b15d9a333d171e3433
Component: engine
2018-03-22 15:04:10 -07:00
f358bd4b47 daemon: use context error rather than inventing new one
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: d84da75f01e0a0d20fbddb8b051a325e3b21eded
Component: engine
2018-03-22 09:38:59 -07:00
1f699969f9 Merge pull request #36587 from justincormack/unpriv-caps
If container will run as non root user, drop permitted, effective caps early
Upstream-commit: b67c1e078c7eeb20199dce301e95fa8999c98109
Component: engine
2018-03-22 14:51:29 +01:00
1c15bdeddb Merge pull request #36466 from thaJeztah/fix-exec-apparmor
Fix AppArmor not being applied to Exec processes
Upstream-commit: 0c1006f1abc1af7aa6b9847754370d054dfa6c68
Component: engine
2018-03-19 15:12:41 -07:00
3d783d5dbf If container will run as non root user, drop permitted, effective caps early
As soon as the initial executable in the container is executed as a non root user,
permitted and effective capabilities are dropped. Drop them earlier than this, so
that they are dropped before executing the file. The main effect of this is that
if `CAP_DAC_OVERRIDE` is set (the default) the user will not be able to execute
files they do not have permission to execute, which previously they could.

The old behaviour was somewhat surprising and the new one is definitely correct,
but it is not in any meaningful way exploitable, and I do not think it is
necessary to backport this fix. It is unlikely to have any negative effects as
almost all executables have world execute permission anyway.

Use the bounding set not the effective set as the canonical set of capabilities, as
effective will now vary.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Upstream-commit: 15ff09395c001bcb0f284461abbc404a1d8bab4d
Component: engine
2018-03-19 14:45:27 -07:00
60e6ee9365 Builder: Fix CI issues
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: 317513d6984c0ba5df41dc578b22eb32fec55b55
Component: engine
2018-03-19 14:29:36 -07:00
c9d18328ef Merge pull request #29932 from miaoyq/container-log-add-archive
add support for compressibility of log file
Upstream-commit: c4e93da8a6fcd206e3fbfb07b821b5743f90f437
Component: engine
2018-03-19 15:20:17 -04:00
559e0317a4 Merge pull request #36507 from dnephin/use-gty-assert
Replace testify/assert with gotestyourself/assert
Upstream-commit: 7bda5be956eb1123d597b3c497d938c4d6df3327
Component: engine
2018-03-19 09:08:06 +01:00
5c48fab022 Merge pull request #36609 from thaJeztah/fix-stats-loop
Fix stats collector spinning CPU if no stats are collected
Upstream-commit: 72ba7f593fa4dbb628cf5ee83cd7daf955934cf5
Component: engine
2018-03-16 10:35:10 -07:00
0331f04e35 Post migration assertion fixes
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: c9e52bd0da0461e605a3678b85702f83081504a7
Component: engine
2018-03-16 11:03:46 -04:00