Instead of implementing refcounts at each graphdriver, implement this in
the layer package which is what the engine actually interacts with now.
This means interacting directly with the graphdriver is no longer
explicitly safe with regard to Get/Put calls being refcounted.
In addition, with the containerd, layers may still be mounted after
a daemon restart since we will no longer explicitly kill containers when
we shutdown or startup engine.
Because of this ref counts would need to be repopulated.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 65d79e3e5e537039b244afd7eda29e721a93d84f
Component: engine
Instead of implementing refcounts at each graphdriver, implement this in
the layer package which is what the engine actually interacts with now.
This means interacting directly with the graphdriver is no longer
explicitly safe with regard to Get/Put calls being refcounted.
In addition, with the containerd, layers may still be mounted after
a daemon restart since we will no longer explicitly kill containers when
we shutdown or startup engine.
Because of this ref counts would need to be repopulated.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 563d0711f83952e561a0d7d5c48fef9810b4f010
Component: engine
Closes#16667
Uses the prefix "devmapper:" for all the fmt and logrus error, debug, and info messages.
Signed-off-by: Chris Dituri <csdituri@gmail.com>
Upstream-commit: 0aa6ace6e6e819551d21ebdcaa0c5802f76d7603
Component: engine
This change will allow us to run SELinux in a container with
BTRFS back end. We continue to work on fixing the kernel/BTRFS
but this change will allow SELinux Security separation on BTRFS.
It basically relabels the content on container creation.
Just relabling -init directory in BTRFS use case. Everything looks like it
works. I don't believe tar/achive stores the SELinux labels, so we are good
as far as docker commit.
Tested Speed on startup with BTRFS on top of loopback directory. BTRFS
not on loopback should get even better perfomance on startup time. The
more inodes inside of the container image will increase the relabel time.
This patch will give people who care more about security the option of
runnin BTRFS with SELinux. Those who don't want to take the slow down
can disable SELinux either in individual containers or for all containers
by continuing to disable SELinux in the daemon.
Without relabel:
> time docker run --security-opt label:disable fedora echo test
test
real 0m0.918s
user 0m0.009s
sys 0m0.026s
With Relabel
test
real 0m1.942s
user 0m0.007s
sys 0m0.030s
Signed-off-by: Dan Walsh <dwalsh@redhat.com>
Signed-off-by: Dan Walsh <dwalsh@redhat.com>
Upstream-commit: 1716d497a420f0cd4e53a99535704c6d215e38c7
Component: engine
Add docker info show base filesystem size of container/image when use devicemapper
Upstream-commit: 5ecbc9747fab243136402ea78d65e928e6d4678b
Component: engine
Adds support for the daemon to handle user namespace maps as a
per-daemon setting.
Support for handling uid/gid mapping is added to the builder,
archive/unarchive packages and functions, all graphdrivers (except
Windows), and the test suite is updated to handle user namespace daemon
rootgraph changes.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
Upstream-commit: 442b45628ee12ebd8e8bd08497896d5fa8eec4bd
Component: engine
Keep track of number of deleted devices and export this information through
"docker info".
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Upstream-commit: d295dc66521e2734390473ec1f1da8a73ad3288a
Component: engine
Finally here is the patch to implement deferred deletion functionality.
Deferred deleted devices are marked as "Deleted" in device meta file.
First we try to delete the device and only if deletion fails and user has
enabled deferred deletion, device is marked for deferred deletion.
When docker starts up again, we go through list of deleted devices and
try to delete these again.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Upstream-commit: d929589c1fc4538dcd1b2a7a3dc7d4afbdfa72fd
Component: engine
Provide a command line option dm.use_deferred_deletion to enable deferred
device deletion feature. By default feature will be turned off.
Not sure if there is much value in deferred deletion being turned on
without deferred removal being turned on. So for now, this feature can
be enabled only if deferred removal is on.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Upstream-commit: 51e059e7e90f37848596a0b6ec83f7835e6e4131
Component: engine
Exists() and HasDevice() just check if device file exists or not. It does
not say anything about if device is mounted or not. Fix comments.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Upstream-commit: f5c0eb9ffe9a9d30ac6ff81fa1a4c216908189a6
Component: engine
TL;DR: check for IsExist(err) after a failed MkdirAll() is both
redundant and wrong -- so two reasons to remove it.
Quoting MkdirAll documentation:
> MkdirAll creates a directory named path, along with any necessary
> parents, and returns nil, or else returns an error. If path
> is already a directory, MkdirAll does nothing and returns nil.
This means two things:
1. If a directory to be created already exists, no error is returned.
2. If the error returned is IsExist (EEXIST), it means there exists
a non-directory with the same name as MkdirAll need to use for
directory. Example: we want to MkdirAll("a/b"), but file "a"
(or "a/b") already exists, so MkdirAll fails.
The above is a theory, based on quoted documentation and my UNIX
knowledge.
3. In practice, though, current MkdirAll implementation [1] returns
ENOTDIR in most of cases described in #2, with the exception when
there is a race between MkdirAll and someone else creating the
last component of MkdirAll argument as a file. In this very case
MkdirAll() will indeed return EEXIST.
Because of #1, IsExist check after MkdirAll is not needed.
Because of #2 and #3, ignoring IsExist error is just plain wrong,
as directory we require is not created. It's cleaner to report
the error now.
Note this error is all over the tree, I guess due to copy-paste,
or trying to follow the same usage pattern as for Mkdir(),
or some not quite correct examples on the Internet.
[v2: a separate aufs commit is merged into this one]
[1] https://github.com/golang/go/blob/f9ed2f75/src/os/path.go
Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Upstream-commit: a83a76934787a20e96389d33bd56a09369f9b808
Component: engine
Export image/container metadata stored in graph driver. Right now 3 fields
DeviceId, DeviceSize and DeviceName are being exported from devicemapper.
Other graph drivers can export fields as they see fit.
This data can be used to mount the thin device outside of docker and tools
can look into image/container and do some kind of inspection.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Upstream-commit: 407a626be62996cd6385ea4d80e669ab83f5f04d
Component: engine
This will help with debugging as one could just do "docker info" and figure
out of deferred removal is enabled or not.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Upstream-commit: 66a53819aea2ab1ab0d50be1f8d32fcb2427cd78
Component: engine
Fixes#9960
This adds the output of a "Backing Filesystem:" entry to `docker info`
to overlay, aufs, and devicemapper graphdrivers. The default list
includes a fairly complete list of common filesystem names from
linux/include/uapi/linux/magic.h, but if the backing filesystem is not
recognized, the code will simply show "<unknown>"
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
Upstream-commit: 48b1dd0084904678728817d728bb9ab1c0183aad
Component: engine
There are a couple of drivers that swallow errors that may occur in
their Put() implementation.
This changes the signature of (*Driver).Put for all the drivers implemented.
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
Upstream-commit: 00fd63e55807c36fedf0878645dfec995fba381d
Component: engine
Presenly the "Data file:" shows either the loopback _file_ or the block device.
With this, the "Data file:" will always show the device, and if it is a
loopback, then there will additionally be a "Data loop file:".
(Same for "Metadata file:")
Signed-off-by: Vincent Batts <vbatts@redhat.com>
Upstream-commit: 09c033ff872334cdcc45172ac57dbf21573481ef
Component: engine
This is a first pass at splitting out devicemapper into separate, usable
bindings.
Signed-off-by: Vincent Batts <vbatts@redhat.com>
Upstream-commit: e2f8fbfbcc450432536e387777b1ff080c94a948
Component: engine
The logic is unrelated to graphdriver.
Signed-off-by: Vincent Batts <vbatts@redhat.com>
Upstream-commit: 930a756ad55ad5f4e5e6391b41673743d7254c2b
Component: engine
Some graphdrivers are Differs and type assertions are made
in various places throughout the project. Differ offers some
convenience in generating/applying diffs of filesystem layers
but for most graphdrivers another code path is taken.
This patch brings all of the logic related to filesystem
diffs in one place, and simplifies the implementation of some
common types like Image, Daemon, and Container.
Signed-off-by: Josh Hawn <josh.hawn@docker.com>
Upstream-commit: dee6b481fe0da1d845261ffff2e610fb05898d3c
Component: engine
If this is at the root directory for the daemon you could unmount
somones filesystem when you stop docker and this is actually only needed
for the palces that the graph drivers mount the container's root
filesystems.
Docker-DCO-1.1-Signed-off-by: Michael Crosby <michael@crosbymichael.com> (github: crosbymichael)
Upstream-commit: 3609b051b88565c0fe0615fd47ddb48eed549d27
Component: engine
This allows setting these settings to be passed:
dm.basesize
dm.loopdatasize
dm.loopmetadatasize
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
Upstream-commit: 7f5ba068f438ee159bfca7396de4987bcae45809
Component: engine
This lets you add storage specific options for the daemon.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
Upstream-commit: 822ea97ffcf10645720bb93108a60f8b9ce9931d
Component: engine
This is no longer needed, as the tests are gone.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
Upstream-commit: 39d244a593aad63be58d8b2e452715e25135f255
Component: engine
