Commit Graph

739 Commits

Author SHA1 Message Date
3a50fe644a Merge pull request #34861 from tklauser/fix-cstring-leaks
Fix CString memory leaks
Upstream-commit: 777d4a1bf45c85db6931205d4adbe38a17c583d7
Component: engine
2017-09-21 09:14:07 -07:00
0dfdf37ab2 Merge pull request #34914 from euank/000003-percent
overlay2: fix faulty errcheck
Upstream-commit: 48cce229330561c104f777dac4f3f39ef2e2de6e
Component: engine
2017-09-20 19:52:10 -07:00
ccdce91e65 overlay2: fix faulty errcheck
The change in 7a7357dae1bcccb17e9b2d4c7c8f5c025fce56ca inadvertently
changed the `defer` error code into a no-op. This restores its behavior
prior to that code change, and also introduces a little more error
logging.

Signed-off-by: Euan Kemp <euan.kemp@coreos.com>
Upstream-commit: 639ab92f011245e17e9a293455a8dae1eb034022
Component: engine
2017-09-20 15:25:57 -07:00
7e0e805bca Merge pull request #34863 from keloyang/close-pipe
Close pipe in overlay2 graphdriver
Upstream-commit: e40d5e665c6de904ea1e22bb3aa615638ec05e2a
Component: engine
2017-09-20 09:37:15 -07:00
da00b40eb4 Merge pull request #34721 from kinvolk/iaguis/add-missing-ecryptfs-string
Add missing eCryptfs translation to FsNames
Upstream-commit: 3fa72d38ec61377b506d31711f5cf1b855219926
Component: engine
2017-09-19 05:45:24 -07:00
07893693ca Merge pull request #34891 from Microsoft/jjh/fixcomment
LCOW: Fix comment in graphdriver code
Upstream-commit: 13e8a7a0067ce5d8d458b3a5985691bed0b0f30e
Component: engine
2017-09-19 14:43:35 +02:00
f8cc2fb021 LCOW: Fix comment in graphdriver code
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: f9fc269c204b96b34a479f3e55e41886ed2626b0
Component: engine
2017-09-18 19:52:55 -07:00
b07dcb72fb Close pipe if mountFrom failed.
Signed-off-by: Shukui Yang <yangshukui@huawei.com>
Upstream-commit: 9f38923901352459bb621d0b3587a6517e67eeb3
Component: engine
2017-09-19 01:25:39 +00:00
3303de2b82 Merge pull request #34759 from kolyshkin/gometalinter
Gometalinter fixups for non-x86
Upstream-commit: 65e88d996ad13ad5ca9c2d2c625a87f9271d1e82
Component: engine
2017-09-18 13:44:15 -07:00
5421741502 Add missing eCryptfs translation to FsNames
It was causing the error message to be

    'overlay' is not supported over <unknown>

instead of

    'overlay' is not supported over ecryptfs

Signed-off-by: Iago López Galeiras <iago@kinvolk.io>
Upstream-commit: ddb31b4fdf7311ad6c7e62aa15b5f6da16518e77
Component: engine
2017-09-18 19:06:13 +02:00
3a4f4fd3c4 overlay gd: fix build for 32-bit ARM
This commit reverts a hunk of commit 2f5f0af3f ("Add unconvert linter")
and adds a hint for unconvert linter to ignore excessive conversion as
it is required on 32-bit platforms (e.g. armhf).

The exact error on armhf is this:

	19:06:45 ---> Making bundle: dynbinary (in bundles/17.06.0-dev/dynbinary)
	19:06:48 Building: bundles/17.06.0-dev/dynbinary-daemon/dockerd-17.06.0-dev
	19:10:58 # github.com/docker/docker/daemon/graphdriver/overlay
	19:10:58 daemon/graphdriver/overlay/copy.go:161: cannot use stat.Atim.Sec (type int32) as type int64 in argument to time.Unix
	19:10:58 daemon/graphdriver/overlay/copy.go:161: cannot use stat.Atim.Nsec (type int32) as type int64 in argument to time.Unix
	19:10:58 daemon/graphdriver/overlay/copy.go:162: cannot use stat.Mtim.Sec (type int32) as type int64 in argument to time.Unix
	19:10:58 daemon/graphdriver/overlay/copy.go:162: cannot use stat.Mtim.Nsec (type int32) as type int64 in argument to time.Unix

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: b569f57890d4cad132be437e5dac55130b6c76e9
Component: engine
2017-09-17 22:04:31 -07:00
a14ef1d5fc devmapper: tell why xfs is not supported
Instead of providing a generic message listing all possible reasons
why xfs is not available on the system, let's be specific.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: c21245c9200ab39a9219b28f8185573b78a55074
Component: engine
2017-09-17 22:04:31 -07:00
e3dc487328 devmapper: show dmesg if mount fails
If mount fails, the reason might be right there in the kernel log ring buffer.
Let's include it in the error message, it might be of great help.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 46833ee1c353c247e3ef817a08d5a35a2a43bdf3
Component: engine
2017-09-17 22:04:31 -07:00
a4c2f1d0c4 devmapper: don't create too new xfs
Since the update to Debian Stretch, devmapper unit test fails. One
reason is, the combination of somewhat old (less than 3.16) kernel and
relatively new xfsprogs leads to creating a filesystem which is not supported
by the kernel:

> [12206.467518] XFS (dm-1): Superblock has unknown read-only compatible features (0x1) enabled.
> [12206.472046] XFS (dm-1): Attempted to mount read-only compatible filesystem read-write.
> Filesystem can only be safely mounted read only.
> [12206.472079] XFS (dm-1): SB validate failed with error 22.

Ideally, that would be automatically and implicitly handled by xfsprogs.
In real life, we have to take care about it here. Sigh.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 6b01bc5adb1255cb48e22e755ab86fd2c3305211
Component: engine
2017-09-17 22:04:31 -07:00
322ee8f812 devmapper gd: disable for static build
Static build with devmapper is impossible now since libudev is required
and no static version of libudev is available (as static libraries are
not supported by systemd which udev is part of).

This should not hurt anyone as "[t]he primary user of static builds
is the Editions, and docker in docker via the containers, and none
of those use device mapper".

Also, since the need for static libdevmapper is gone, there is no need
to self-compile libdevmapper -- let's use the one from Debian Stretch.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 84f1c054e66d3999aaf2751062cda8a77925b7ae
Component: engine
2017-09-17 22:04:31 -07:00
5086fdcfde Fix CString memory leaks
Make sure to call C.free on C string allocated using C.CString in every
exit path.

C.CString allocates memory in the C heap using malloc. It is the callers
responsibility to free them. See
https://golang.org/cmd/cgo/#hdr-Go_references_to_C for details.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
Upstream-commit: 593dbfd1448e8dac08488786fde6fe7fb057bdac
Component: engine
2017-09-15 09:57:26 +02:00
cddfe04f6a LCOW: Implemented support for docker cp + build
This enables docker cp and ADD/COPY docker build support for LCOW.
Originally, the graphdriver.Get() interface returned a local path
to the container root filesystem. This does not work for LCOW, so
the Get() method now returns an interface that LCOW implements to
support copying to and from the container.

Signed-off-by: Akash Gupta <akagup@microsoft.com>
Upstream-commit: 7a7357dae1bcccb17e9b2d4c7c8f5c025fce56ca
Component: engine
2017-09-14 12:07:52 -07:00
4219a83be8 overlay gd: fix build for 32-bit ARM
This commit reverts a hunk of commit 2f5f0af3f ("Add unconvert linter")
and adds a hint for unconvert linter to ignore excessive conversion as
it is required on 32-bit platforms (e.g. armhf).

The exact error on armhf is this:

	19:06:45 ---> Making bundle: dynbinary (in bundles/17.06.0-dev/dynbinary)
	19:06:48 Building: bundles/17.06.0-dev/dynbinary-daemon/dockerd-17.06.0-dev
	19:10:58 # github.com/docker/docker/daemon/graphdriver/overlay
	19:10:58 daemon/graphdriver/overlay/copy.go:161: cannot use stat.Atim.Sec (type int32) as type int64 in argument to time.Unix
	19:10:58 daemon/graphdriver/overlay/copy.go:161: cannot use stat.Atim.Nsec (type int32) as type int64 in argument to time.Unix
	19:10:58 daemon/graphdriver/overlay/copy.go:162: cannot use stat.Mtim.Sec (type int32) as type int64 in argument to time.Unix
	19:10:58 daemon/graphdriver/overlay/copy.go:162: cannot use stat.Mtim.Nsec (type int32) as type int64 in argument to time.Unix

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 21b2c278cc86f0fc411018becbcbf2a7e44b6057
Component: engine
2017-09-12 18:44:55 -07:00
18c29c5c5f Add gosimple linter
Update gometalinter

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: f7f101d57ef8cbf2d8723a18b7d723c5c5dd04b6
Component: engine
2017-09-12 12:09:59 -04:00
eec1d4ef54 Optimize some wrong usage and spelling
Signed-off-by: wgliang <liangcszzu@163.com>
Upstream-commit: 94cefa21459a0c620e5a9c2da04df6d3a43dae17
Component: engine
2017-09-07 09:44:08 +08:00
01392057b0 Add unconvert linter
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 2f5f0af3fdb7e9ee607a0e178dbe2af6e10cccf4
Component: engine
2017-08-24 15:08:31 -04:00
80226e2163 Add more detailed logging to aufs init
Addresses some comments on 276b44608b04f08bdf46ce7c816b1f744bf24b7d

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 0a98025d4b2910c2089325b87d28c32d05803e13
Component: engine
2017-08-22 14:19:03 -04:00
e12074fbb5 Merge pull request #34587 from cpuguy83/aufs_rm_errors
Fix error removing diff path
Upstream-commit: bbcdc7df34eeb01c2f5741235f26f90749e67a05
Component: engine
2017-08-22 11:02:30 -07:00
7b15e791ed Fix error removing diff path
In d42dbdd3d48d0134f8bba7ead92a7067791dffab the code was re-arranged to
better report errors, and ignore non-errors.
In doing so we removed a deferred remove of the AUFS diff path, but did
not replace it with a non-deferred one.

This fixes the issue and makes the code a bit more readable.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 276b44608b04f08bdf46ce7c816b1f744bf24b7d
Component: engine
2017-08-22 12:51:58 -04:00
94c685a721 Add deadcode linter
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 62c1f0ef41e6cd88a8846da1c11976a320ca8b41
Component: engine
2017-08-21 18:18:50 -04:00
6b3b192846 Add goimports to linters.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 372670b5074b077927314cdf46af30f8752e7db0
Component: engine
2017-08-21 18:15:08 -04:00
de626e10fb Fix golint errors.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 9b47b7b1519c5f2138e2933fb1fc459eb00895c0
Component: engine
2017-08-18 14:23:44 -04:00
ded67f686e devmapper autosetup: add check for thin_check
I was able to successfully use device mapper autoconfig feature
(commit 5ef07d79c) but it stopped working after a reboot.

Investigation shown that the dm device was not activated because of
a missing binary, that is not used during initial setup, but every
following time. Here's an error shown when trying to manually activate
the device:

> kir@kd:~/go/src/github.com/docker/docker$ sudo lvchange -a y /dev/docker/thinpool
> /usr/sbin/thin_check: execvp failed: No such file or directory
> Check of pool docker/thinpool failed (status:2). Manual repair required!

Surely, there is no solution to this other than to have a package that
provides the thin_check binary installed beforehand. Due to the fact
the issue revealed itself way later than DM setup was performed, it was
somewhat harder to investigate.

With this in mind, let's check for binary presense before setting up DM,
refusing to proceed if the binary is not there, saving a user from later
frustration.

While at it, eliminate repeated binary checking code. The downside is
that the binary lookup is happening more than once now -- I think the
clarity of code overweights this minor de-optimization.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 58a453f3f06c1daf34544da8aa16bb95e8e18010
Component: engine
2017-08-14 13:25:28 +03:00
94157b8e90 devmapper: refer to dockerd man page
...not the docker one.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 8b7bd58869725dce2f0fcfd582d23dc5e0cfcf8e
Component: engine
2017-08-14 10:09:58 +03:00
d92c40217f devmapper autoconfig: add mkdir
I tried using dm.directlvm_device but it ended up with the following
error:

> Error starting daemon: error initializing graphdriver: error
> writing docker thinp autoextend profile: open
> /etc/lvm/profile/docker-thinpool.profile: no such file or directory

The reason is /etc/lvm/profile directory does not exist. I think it is
better to try creating it beforehand.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 6ca20ec771ab7c0ebf64c20021ca795746cf3ccb
Component: engine
2017-08-14 10:09:34 +03:00
479e9126c5 Merge pull request #34430 from AkihiroSuda/promote-overlay2
graphdriver: promote overlay2 over aufs
Upstream-commit: 2e38c07814fbb99364838df2cd1092744f6bd1e1
Component: engine
2017-08-09 21:41:36 +02:00
b9168879f3 jhoward/opengcs --> Microsoft/opengcs
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: ee0587a45d94443271c695a0d407514fd36f7fa4
Component: engine
2017-08-08 14:43:43 -07:00
308b435aad graphdriver: promote overlay2 over aufs
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Upstream-commit: fd6ffc2337aedf1019582851fe0cf4f6f33f113a
Component: engine
2017-08-08 10:17:37 +00:00
18cbafdfc4 Merge pull request #34396 from Microsoft/jjh/movescratch
LCOW: Move toolsScratchPath to /tmp
Upstream-commit: d4f6db83c21cfc6af54fffb1f13e8acb7199f96a
Component: engine
2017-08-05 11:37:15 +01:00
3b5cce5785 LCOW: Move toolsScratchPath to /tmp
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: 993f4072874ee5cdce93ec9b6525e1fa3ebda4c8
Component: engine
2017-08-03 15:48:53 -07:00
1b5797d860 LCOW: Graphdriver dynamic sandbox management
Signed-off-by: John Howard <jhoward@microsoft.com>

This changes the graphdriver to perform dynamic sandbox management.
Previously, as a temporary 'hack', the service VM had a prebuilt
sandbox in it. With this change, management is under the control
of the client (docker) and executes a mkfs.ext4 on it. This enables
sandboxes of non-default sizes too (a TODO previously in the code).

It also addresses https://github.com/moby/moby/pull/33969#discussion_r127287887

Requires:
- go-winio: v0.4.3
- opengcs:  v0.0.12
- hcsshim:  v0.6.x
Upstream-commit: 8c279ef3ad8cd1f019789b8378d0394c80a1807f
Component: engine
2017-08-03 09:06:45 -07:00
296e1bef94 Revendor microsoft/go-winio @ v0.4.4
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: 45e2dd123bd9ace93d3fa85b12fcea3a8ac642ce
Component: engine
2017-08-03 09:06:26 -07:00
9762198ec1 Merge pull request #34272 from dmcgowan/update-logrus
Update logrus to v1.0.1 (Sirupsen -> sirupsen)
Upstream-commit: 8af4db6f002ac907b6ef8610b237879dfcaa5b7a
Component: engine
2017-08-01 18:53:33 -07:00
970fea45e3 graphdriver/devmapper: clarify a message
Make sure user understands this is about the in-kernel driver
(not the dockerd driver or smth).

While at it, amend the comment as well.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: aab2450e25b397d38cdcb5e173ef1121283196c2
Component: engine
2017-07-31 20:01:02 -07:00
d659edcaf5 Update logrus to v1.0.1
Fixes case sensitivity issue

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: 1009e6a40b295187e038b67e184e9c0384d95538
Component: engine
2017-07-31 13:16:46 -07:00
5c5712c42e Switch Stat syscalls to x/sys/unix
Switch some more usage of the Stat function and the Stat_t type from the
syscall package to golang.org/x/sys. Those were missing in PR #33399.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
Upstream-commit: 01f70b028e9597ef207509e8124e120688dae185
Component: engine
2017-07-27 10:09:02 +02:00
5eb81e4819 Merge pull request #33960 from cpuguy83/ignore_not_exist_err
Fix error handling with not-exist errors on remove
Upstream-commit: 67eeb0490d9beacbd6e00eb4ea20cfe692ebf333
Component: engine
2017-07-21 13:52:27 +02:00
ca6a533d23 Fix error handling with not-exist errors on remove
Specifically, none of the graphdrivers are supposed to return a
not-exist type of error on remove (or at least that's how they are
currently handled).

Found that AUFS still had one case where a not-exist error could escape,
when checking if the directory is mounted we call a `Statfs` on the
path.

This fixes AUFS to not return an error in this case, but also
double-checks at the daemon level on layer remove that the error is not
a `not-exist` type of error.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: d42dbdd3d48d0134f8bba7ead92a7067791dffab
Component: engine
2017-07-20 12:26:27 -04:00
b9deee44fd Merge pull request #33969 from Microsoft/jjh/lifetime
LCOW: Service VM lifetime changes
Upstream-commit: f22cecf9297a467f20bf49bc1c7aef5d2ee3a59c
Component: engine
2017-07-13 15:51:21 -07:00
9ed5ded98d LCOW: Service VM lifetime changes
Signed-off-by: John Howard <jhoward@microsoft.com>

This changes the LCOW driver to support both global SVM lifetime and
per-instance lifetime. It also corrects the scratch implementation.
Upstream-commit: 76b0d4819a50c50edd79890db0b92eb96c8d46e6
Component: engine
2017-07-13 11:09:43 -07:00
a6bf694e20 Merge pull request #33845 from cyphar/devicemapper-show-me-your-logs
devicemapper: rework logging and add --storage-opt dm.libdm_log_level
Upstream-commit: 00b218216d8a2075bf7200df74aa07b749f72f34
Component: engine
2017-07-12 17:46:14 -07:00
8e0ce0ecdb Merge pull request #33877 from rhvgoyal/sync-removal
devicemapper: Wait for device removal if deferredRemoval=true and deferredDeletion=…
Upstream-commit: e04dbe5ac287c2a856b7c96972d931ee5f0e288f
Component: engine
2017-07-12 17:35:45 -07:00
e95f4619cd [project] change syscall to /x/sys/unix|windows
Changes most references of syscall to golang.org/x/sys/
Ones aren't changes include, Errno, Signal and SysProcAttr
as they haven't been implemented in /x/sys/.

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>

[s390x] switch utsname from unsigned to signed

per 33267e036f
char in s390x in the /x/sys/unix package is now signed, so
change the buildtags

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
Upstream-commit: 069fdc8a083cb1663e4f86fe3fd9b9a1aebc3e54
Component: engine
2017-07-11 08:00:32 -04:00
643654c2f0 Spelling fixes
* additional
* ambiguous
* anonymous
* anything
* application
* because
* before
* building
* capabilities
* circumstances
* commit
* committer
* compresses
* concatenated
* config
* container
* container's
* current
* definition
* delimiter
* disassociates
* discovery
* distributed
* doesnotexist
* downloads
* duplicates
* either
* enhancing
* enumerate
* escapable
* exactly
* expect
* expectations
* expected
* explicitly
* false
* filesystem
* following
* forbidden
* git with
* healthcheck
* ignore
* independent
* inheritance
* investigating
* irrelevant
* it
* logging
* looking
* membership
* mimic
* minimum
* modify
* mountpoint
* multiline
* notifier
* outputting
* outside
* overridden
* override
* parsable
* plugins
* precedence
* propagation
* provided
* provides
* registries
* repositories
* returning
* settings
* should
* signals
* someone
* something
* specifically
* successfully
* synchronize
* they've
* thinking
* uninitialized
* unintentionally
* unmarshaling
* unnamed
* unreferenced
* verify

Signed-off-by: Josh Soref <jsoref@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 39bcaee47b8a284a46b761afe218ba7deda0d482
Component: engine
2017-07-03 13:13:09 -07:00
666b2cc734 devicemapper: add --storage-opt dm.libdm_log_level=X option
Because we use our own logging callbacks in order to use libdm
effectively, it is quite difficult to debug complicated devicemapper
issues (because any warnings or notices from libdm are muted by our own
callback function). e07d3cd9a ("devmapper: Fix libdm logging") further
reduced the ability of this debugging by only allowing _LOG_FATAL errors
to be passed to the output.

Unfortunately libdm is very chatty, so in order to avoid making the logs
even more crowded, add a dm.libdm_log_level storage option that allows
people who are debugging the lovely world of libdm to be able to dive in
without recompiling binaries.

The valid values of dm.libdm_log_level map directly to the libdm logging
levels, and are in the range [2,7] as of the time of writing with 7
being _LOG_DEBUG and 2 being _LOG_FATAL. The default is _LOG_FATAL.

Signed-off-by: Aleksa Sarai <asarai@suse.de>
Upstream-commit: 198f83bba120c6c9bda679d33a55acab6cc9f33d
Component: engine
2017-07-04 02:04:26 +10:00