Commit Graph

2653 Commits

Author SHA1 Message Date
dc6063672c Close readclosers returned by DecompressStream
Signed-off-by: Joe Ferguson <joe@infosiftr.com>
Upstream-commit: 76e99e1a8b60c831415d2f6a6a7954e16c25620b
Component: engine
2018-06-07 11:16:13 -07:00
0e7017a3ae Merge pull request #36688 from cpuguy83/volumes_service
Extract volume interaction to a volumes service
Upstream-commit: 5037c5a8ce762b46638378b7a7d1081572eadba1
Component: engine
2018-06-05 02:16:20 +02:00
d2c390cca0 Merge pull request #37177 from cpuguy83/minor_plugin_addr_change
Move plugin client to separate interface
Upstream-commit: 71cd53e4a197b303c6ba086bd584ffd67a884281
Component: engine
2018-05-31 08:22:04 -07:00
6aff3ef24e Merge pull request #37174 from thaJeztah/no_additional_magic_matching
Be explicit about github.com prefix being a legacy feature
Upstream-commit: 2ea4ece674e04c04174ee9fff8005cdccdc4fad0
Component: engine
2018-05-31 09:45:59 +02:00
285424548c Move plugin client to separate interface
This makes it a bit simpler to remove this interface for v2 plugins
and not break external projects (libnetwork and swarmkit).

Note that before we remove the `Client()` interface from `CompatPlugin`
libnetwork and swarmkit must be updated to explicitly check for the v1
client interface as is done int his PR.

This is just a minor tweak that I realized is needed after trying to
implement the needed changes on libnetwork.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 7c77df8acc597cd4f540d873de5fe53a3d414ba9
Component: engine
2018-05-30 15:22:10 -04:00
90dfa0b4ee Be explicit about github.com prefix being a legacy feature
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: babb0c14fd1281b309de46c3481a5bea88b8031d
Component: engine
2018-05-30 11:09:14 +02:00
4c7a840b18 Move plugin client creation to the extension point
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: f51a96c0165fdcbbe11f62b66b582b7e202f211b
Component: engine
2018-05-25 15:18:53 -04:00
feb51dbad1 Extract volume interaction to a volumes service
This cleans up some of the package API's used for interacting with
volumes, and simplifies management.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: e4b6adc88e967971de634596654d9bc33e7bd7e0
Component: engine
2018-05-25 14:21:07 -04:00
0f96e98e12 Various code-cleanup
remove unnescessary import aliases, brackets, and so on.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f23c00d8701e4bd0f2372a586dacbf66a26f9a51
Component: engine
2018-05-23 17:50:54 +02:00
4056a6dfa4 pkg/mount/TestMount: fix wrt selinux
Sometimes docker-master CI fails on rhel4+selinux configuration,
like this:

--- FAIL: TestMount (0.12s)
    --- FAIL: TestMount/none-remount,size=128k (0.01s)
    	mounter_linux_test.go:209: unexpected mount option "seclabel" expected "rw,size=128k"
    --- FAIL: TestMount/none-remount,ro,size=128k (0.01s)
    	mounter_linux_test.go:209: unexpected mount option "seclabel" expected "ro,size=128k"

Earlier, commit 8bebd42df2 (PR #34965) fixed this failure,
but not entirely (i.e. the test is now flaky). It looks like
either selinux detection code is not always working (it won't
work in d-in-d), or the kernel might or might not add 'seclabel'
option).

As the subject of this test case is definitely not selinux,
it can just ignore the option added by it.

While at it, fix error messages:
 - add missing commas;
 - fix a typo;
 - allow for clear distinction between mount
   and vfs (per-superblock) options.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: d78e885326213e8ef89919c3cc6d16e712e852a8
Component: engine
2018-05-22 23:30:47 -07:00
3b0918309f Merge pull request #37088 from ohbarye/fix-typos-duplicated-the
Fix typos: remove duplicated "the"
Upstream-commit: 3646562a5cfbf93f987d0ccffe7ca1c3487036e2
Component: engine
2018-05-19 20:57:47 +02:00
7f78f7fe15 Fix some linting issues
These showed locally when running `make validate`. CI doesn't seem to have the
same (possibly it's disabled in the configuration)

    builder/fscache/fscache.go:618::error: github.com/docker/docker/vendor/github.com/tonistiigi/fsutil.StatInfo composite literal uses unkeyed fields (vet)
    client/swarm_unlock_test.go:44::error: github.com/docker/docker/api/types/swarm.UnlockRequest composite literal uses unkeyed fields (vet)
    client/swarm_unlock_test.go:20::error: github.com/docker/docker/api/types/swarm.UnlockRequest composite literal uses unkeyed fields (vet)
    cmd/dockerd/daemon_unix.go:113::error: github.com/docker/docker/cmd/dockerd/hack.MalformedHostHeaderOverride composite literal uses unkeyed fields (vet)
    cmd/dockerd/daemon_unix.go:110::error: github.com/docker/docker/cmd/dockerd/hack.MalformedHostHeaderOverride composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay/overlay.go:171::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay/overlay.go:413::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay2/overlay.go:203::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay2/overlay.go:584::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/zfs/zfs.go:109::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/zfs/zfs.go:388::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/volumes_windows.go:27::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    integration/service/network_test.go:31::error: github.com/docker/docker/api/types/network.NetworkingConfig composite literal uses unkeyed fields (vet)
    api/server/server.go:129:10⚠️ should not use basic type string as key in context.WithValue (golint)
    integration/service/network_test.go:54::error: github.com/docker/docker/api/types/network.NetworkingConfig composite literal uses unkeyed fields (vet)
    libcontainerd/client_daemon_linux.go:61::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    libcontainerd/client_daemon_linux.go:74::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    pkg/archive/archive_windows.go:76::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    plugin/manager_linux.go:56::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 4f8c870d623c63b98f8ef8002448f07f4dda4aa9
Component: engine
2018-05-17 19:28:27 +02:00
16c7354fa4 Merge pull request #37074 from skyc024/fix-typos
Fix typos
Upstream-commit: 6a9bcc51df541ea0dedf038e8e88fe556239ef70
Component: engine
2018-05-17 16:00:26 +02:00
111037b920 Fix typos: remove duplicated "the"
Signed-off-by: Masato Ohba <over.rye@gmail.com>
Upstream-commit: 0f95b23d98384a3ae3769b75292cd5b14ba38437
Component: engine
2018-05-17 21:49:51 +09:00
7d44629ea2 Fix typos
Signed-off-by: chenyuzhu <chenyuzhi@oschina.cn>
Upstream-commit: 5ac0d2f0b83e65f49b490e9d20c7a5b0e65f8e55
Component: engine
2018-05-16 15:31:12 +08:00
3a0e583232 jsonmessage: pass message to aux callback
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: f784907eae4f2e8bd4d9dea3d45c790c92689f3d
Component: engine
2018-05-14 15:48:25 -07:00
a0405be5c0 Merge pull request #36918 from Sh4d1/master
add verification to check if escapeKeys have elements (pkg/term/proxy.go)
Upstream-commit: d4e48af48d1a2d7427ccd9f950cf6bc6ff5682b6
Component: engine
2018-04-30 11:57:21 +02:00
48f4272c53 Add verification and test to check if escapeKeys is not empty
In pkg/term/proxy.go and pkg/term/proxy_test.go, check if escapeKeys is empty and if it is, return the one key read

Signed-off-by: Patrik Cyvoct <patrik@ptrk.io>
Upstream-commit: d339130f307634a0edd6aef29b6934c3934e9ba2
Component: engine
2018-04-30 00:32:43 +02:00
e3af8e0774 Switch from x/net/context -> context
Since Go 1.7, context is a standard package. Since Go 1.9, everything
that is provided by "x/net/context" is a couple of type aliases to
types in "context".

Many vendored packages still use x/net/context, so vendor entry remains
for now.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 7d62e40f7e4f3c17d229a7687d6fcca5448de813
Component: engine
2018-04-23 13:52:44 -07:00
5a3297f67b Merge pull request #36913 from vdemeester/test-skip-non-root
Skip some tests requires root uid when run as user…
Upstream-commit: 5c233cf4315747785b27d4b0f0631d220556205f
Component: engine
2018-04-23 11:42:42 -07:00
4521f88522 Merge pull request #36920 from kolyshkin/cancel-func
context.WithTimeout: do call the cancel func
Upstream-commit: 20b524bf2e91466e00cd5fadd87e9d49f8cb8cb3
Component: engine
2018-04-23 20:14:25 +02:00
7f23cb8c8b Skip some tests requires root uid when run as user
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: a7999aaa536818085ec8a23901ef64637587b7e4
Component: engine
2018-04-23 10:14:39 +02:00
4a3d7068f3 Fix typo in idtools tests
It should check `os.Geteuid` with `uid` instead of `os.Getegid`.
On the container (where the tests run), the uid and gid seems to be
the same, thus this doesn't fail.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 5d8b88b114da7c40f04a95909a35287ae1dac37f
Component: engine
2018-04-23 10:14:07 +02:00
554b5427fa Merge pull request #36091 from kolyshkin/mount
pkg/mount improvements
Upstream-commit: 53982e3fc1642f2605189cea48bc6de7b9318611
Component: engine
2018-04-21 11:03:54 +02:00
d80d707e96 context.WithTimeout: do call the cancel func
govet complains (when using standard "context" package):

> the cancel function returned by context.WithTimeout should be called,
> not discarded, to avoid a context leak (vet)

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 05e2f7e2fafd0fbc818c9f4cda7ac513c785d49c
Component: engine
2018-04-20 12:27:13 -07:00
f00e9a9036 mount.Unmount(): don't look into /proc/self/mountinfo
Now, every Unmount() call takes a burden to parse the whole nine yards
of /proc/self/mountinfo to figure out whether the given mount point is
mounted or not (and returns an error in case parsing fails somehow).

Instead, let's just call umount() and ignore EINVAL, which results
in the same behavior, but much better performance.

Note that EINVAL is returned from umount(2) not only in the case when
`target` is not mounted, but also for invalid flags. As the flags are
hardcoded here, it can't be the case.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: a1d095199ddb9b4811e1417b6adcdfadad7d73f4
Component: engine
2018-04-19 14:49:50 -07:00
96e23c4152 pkg/mount/mountinfo_linux: parser speed up
The mountinfo parser implemented via `fmt.Sscanf()` is slower than the one
using `strings.Split()` and `strconv.Atoi()`. This rewrite helps to speed it
up to a factor of 8x, here is a result from go bench:

> BenchmarkParsingScanf-4      	     300	  22294112 ns/op
> BenchmarkParsingSplit-4      	    3000	   2780703 ns/op

I tried other approaches, such as using `fmt.Sscanf()` for the first
three (integer) fields and `strings.Split()` for the rest, but it slows
things down considerably:

> BenchmarkParsingMixed-4      	    1000	   8827058 ns/op

Note the old code uses `fmt.Sscanf`, when a linear search for '-' field,
when a split for the last 3 fields. The new code relies on a single
split.

I have also added more comments to aid in future development.

Finally, the test data is fixed to now have white space before the first field.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: c611f18a7f16d8aa878a5a5c7537d23a0937c40a
Component: engine
2018-04-19 14:49:42 -07:00
cd146e5f0d getSourceMount(): simplify
The flow of getSourceMount was:
 1 get all entries from /proc/self/mountinfo
 2 do a linear search for the `source` directory
 3 if found, return its data
 4 get the parent directory of `source`, goto 2

The repeated linear search through the whole mountinfo (which can have
thousands of records) is inefficient. Instead, let's just

 1 collect all the relevant records (only those mount points
   that can be a parent of `source`)
 2 find the record with the longest mountpath, return its data

This was tested manually with something like

```go
func TestGetSourceMount(t *testing.T) {
	mnt, flags, err := getSourceMount("/sys/devices/msr/")
	assert.NoError(t, err)
	t.Logf("mnt: %v, flags: %v", mnt, flags)
}
```

...but it relies on having a specific mount points on the system
being used for testing.

[v2: add unit tests for ParentsFilter]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 871c957242df9f8c74faf751a2f14eb5178d4140
Component: engine
2018-04-19 14:49:17 -07:00
a97e7a41c8 pkg/mount: implement/use filter for mountinfo parsing
Functions `GetMounts()` and `parseMountTable()` return all the entries
as read and parsed from /proc/self/mountinfo. In many cases the caller
is only interested only one or a few entries, not all of them.

One good example is `Mounted()` function, which looks for a specific
entry only. Another example is `RecursiveUnmount()` which is only
interested in mount under a specific path.

This commit adds `filter` argument to `GetMounts()` to implement
two things:
 1. filter out entries a caller is not interested in
 2. stop processing if a caller is found what it wanted

`nil` can be passed to get a backward-compatible behavior, i.e. return
all the entries.

A few filters are implemented:
 - `PrefixFilter`: filters out all entries not under `prefix`
 - `SingleEntryFilter`: looks for a specific entry

Finally, `Mounted()` is modified to use `SingleEntryFilter()`, and
`RecursiveUnmount()` is using `PrefixFilter()`.

Unit tests are added to check filters are working.

[v2: ditch NoFilter, use nil]
[v3: ditch GetMountsFiltered()]
[v4: add unit test for filters]
[v5: switch to gotestyourself]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: bb934c6aca3e77541dd4fd51b9ab2706294dadda
Component: engine
2018-04-19 14:48:09 -07:00
f6b5cf688c pkg/term add build tag for netbsd
Signed-off-by: Alexandre Jomin <alexandrejomin@gmail.com>
Upstream-commit: 0d82bc142c2448c09b567773867344aa8fc172fb
Component: engine
2018-04-18 15:16:06 +02:00
566b2857cf Merge pull request #35739 from thaJeztah/bump-go-1.10
Bump Golang to 1.10.1
Upstream-commit: 61138fb5fc0f5fc1799cabfa86d28fb88a0f472c
Component: engine
2018-04-13 13:56:38 -10:00
21cd6af613 Merge pull request #36778 from odg0318/master
Using authorization plugin, I changed Content-Type check routine.
Upstream-commit: ff6a103067dec1eec6de023ec813355c419246f6
Component: engine
2018-04-12 17:39:38 -07:00
869a49d9be Set format in archiver
Prevent changing the tar output by setting the format to
PAX and keeping the times truncated.
Without this change the archiver will produce different tar
archives with different hashes with go 1.10.
The addition of the access and changetime timestamps would
also cause diff comparisons to fail.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: fb170206ba12752214630b269a40ac7be6115ed4
Component: engine
2018-04-12 13:34:33 -07:00
49ee24ad6b Fix tarsum for go 1.10
Remove invalid flush commands, flush should only occur when file
has been completely written. This is already handle, remove these calls.
Ensure data gets written after EOF in correct order and before close.
Remove gname and uname from sum for hash compatibility.
Update tarsum tests for gname/uname removal.
Return valid length after eof.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a422774e593b33bd287d9890544ad9e09b380d8c
Component: engine
2018-04-12 13:34:14 -07:00
ad813c4199 Limit authz response buffer
When the authz response buffer limit is hit, perform a flush.
This prevents excessive buffer sizes, especially on large responses
(e.g. `/containers/<id>/archive` or `/containers/<id>/export`).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 74f8e47352e71aad4015d8d9dea8f16e7a055863
Component: engine
2018-04-11 15:36:36 -04:00
dc8e81cea5 Merge pull request #35518 from cyphar/libdm-dlsym-deferred_remove
pkg: devmapper: dynamically load dm_task_deferred_remove
Upstream-commit: 5219725890816a62185b6c7c0f022351e7aaee84
Component: engine
2018-04-11 14:11:16 +02:00
12207142e4 If Content-Type is application/json;charset=UTF-8, RequestBody is empty.
Signed-off-by: odg0318 <odg0318@gmail.com>
Upstream-commit: 6ac73d34e4d8f2b6303f0ed49b58d0a92c19c6f1
Component: engine
2018-04-05 04:03:45 -04:00
7b60a70f28 Fix tests for pkg/archive
Signed-off-by: Manuel Rüger <manuel@rueg.eu>
Upstream-commit: 108bbd96cb6f9aaa2a761e1a33250f36d58882f9
Component: engine
2018-04-03 01:17:52 +02:00
9b41292c7f Support cancellation in directory.Size()
Makes sure that if the user cancels a request that the daemon stops
trying to traverse a directory.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 9d46c4c138d7b3f7778c13fe84857712bd6c97a9
Component: engine
2018-03-29 15:49:15 -04:00
27bd98bb9a Merge pull request #36617 from Microsoft/jjh/kernel-registry
Windows: Move kernel_windows to use golang registry functions
Upstream-commit: 7270b01e5dd1fca3fd7fed0d6fb9d2769ec5f4ba
Component: engine
2018-03-19 11:47:17 +01:00
ad67257ea2 Windows: Move kernel_windows to use golang registry functions
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: 29f93c4bc7bb2fbbaaa8c3f809e858ce1f12a0f2
Component: engine
2018-03-16 09:47:45 -07:00
0fdeea8152 Cleanup pkg/jsonmessage progress tests
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 7d8815ea705e85a73248b5d9e468f9dc65277bb8
Component: engine
2018-03-16 11:03:47 -04:00
0331f04e35 Post migration assertion fixes
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: c9e52bd0da0461e605a3678b85702f83081504a7
Component: engine
2018-03-16 11:03:46 -04:00
60daf5fa97 Automated migration using
gty-migrate-from-testify --ignore-build-tags

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 6be0f709830113966f295401327b027ec2f0bbca
Component: engine
2018-03-16 11:03:43 -04:00
1c9c9f9e72 Cleanup some assertions
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: ef01dea8935932486f03a37069720987e805dce6
Component: engine
2018-03-16 10:54:54 -04:00
e31b1923c7 Merge pull request #36506 from kolyshkin/pkg-mount-slice
pkg/mount: use sort.Slice
Upstream-commit: a21d5bf669571ffc7ad31b8df3c0a632dc207b78
Component: engine
2018-03-09 09:46:53 +01:00
7856345e7b Merge pull request #36505 from kolyshkin/pkg-mount-tests
pkg/mount unit tests: skip some test under non-root
Upstream-commit: 4a1d35c546c4a5df3353f5984d86671f702a315a
Component: engine
2018-03-08 17:27:31 +09:00
2c2dcf7989 Merge pull request #36451 from Microsoft/jjh/ubr
Windows: Report Version and UBR
Upstream-commit: 8cf8fe9cf80a07d4e68829f619ea2c71fbb5af0c
Component: engine
2018-03-07 06:27:01 -08:00
c617524866 pkg/mount: use sort.Slice
Sorting by mount point length can be implemented in a more
straightforward fashion since Go 1.8 introduced sort.Slice()
with an ability to provide a less() function in place.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: a00310b54c0cdcafb402aeea92feca865da9fdf3
Component: engine
2018-03-06 12:46:58 -08:00
db90a864e8 pkg/mount unit tests: skip some test under non-root
This makes `go test .` to pass if run as non-root user, skipping
those tests that require superuser privileges (for `mount`).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 4aae77602a7540b4f977572f3fbdc0891ac57cab
Component: engine
2018-03-06 12:37:27 -08:00