Commit Graph

3467 Commits

Author SHA1 Message Date
ff5de9f609 RootIsShared() - Fix array out of bounds error
This happened for me on the last (empty) line, but better safe than sorry
so we make the check general.
Upstream-commit: 7d566b4f761e8942cf9679e96774b320b8496b2f
Component: engine
2013-09-30 17:35:01 -06:00
74b00e51f4 devmapper: Move init layer to top rather than bottom
The init layer needs to be topmost to make sure certain files
are always there (for instance, the ubuntu:12.10 image wrongly
has /dev/shm being a symlink to /run/shm, and we need to override
that). However, previously the devmapper code implemented the
init layer by putting it in the base devmapper device, which meant
layers above it could override these files (so that ubuntu:12.10
broke).

So, instead we put the base layer in *each* images devmapper device.
This is "safe" because we still have the pristine layer data
in the layer directory. Also, it means we diff the container
against the image with the init layer applied, so it won't show
up in diffs/commits.
Upstream-commit: fdbc2695fe00d522c5c1a962f9be2f802bf53943
Component: engine
2013-09-30 17:35:01 -06:00
1d1c891003 lxc: Work around lxc-start need for private mounts
lxc-start requires / to be mounted private, otherwise the changes
it does inside the container (both mounts and unmounts) will propagate
out to the host.

We work around this by starting up lxc-start in its own namespace where
we set / to rprivate.

Unfortunately go can't really execute any code between clone and exec,
so we can't do this in a nice way. Instead we have a horrible hack that
use the unshare command, the shell and the mount command...
Upstream-commit: 429587779a95a4d38ec9cd66202de9729c320ef8
Component: engine
2013-09-30 17:35:01 -06:00
95fd1df21e Utils: Add ShellQuoteArguments
Upstream-commit: 145024c6ccc7bff1eda632823702f91899667ed3
Component: engine
2013-09-30 17:35:01 -06:00
f3c281436d Container: Inject into the mount, not the rwPath
For device-mapper setups we can't just push the file into the rwPath.
Upstream-commit: c1388010731cac1c8ff5159bd72b926545a64e58
Component: engine
2013-09-30 17:35:01 -06:00
4cb2cd0d23 api_test: Fix PostContainersCreate
We can't look for the created file in the rwpath, because that
doesn't exist in the device-mapper world, instead look in the
RootfsPath.
Upstream-commit: 07227866006ea75a0c492814b808fdbb672431ee
Component: engine
2013-09-30 17:35:01 -06:00
0453049ffb Runtime: Only remove device on destroy if it exists
Upstream-commit: 5f8e24f8428bc5fb7076ec757ded44f965c48888
Component: engine
2013-09-30 17:35:01 -06:00
eb4c81b5cf graph test: Unmount image via image.Unmount()
This helps us track the unmount
Upstream-commit: 0d7ab8db03814bac19fe076a0d53b6d423616bc7
Component: engine
2013-09-30 17:35:01 -06:00
89fbc4edd2 deviceset: Cleanup device sets on test end
We unmount all mounts and deactivate all device mapper devices to
make sure we're left with no leftovers after the test.
Upstream-commit: ed741f7b27b1b1cf5b6f8917551ce86bc39e9c78
Component: engine
2013-09-30 17:35:01 -06:00
12fbeaaae6 DeviceSet: Add UnmountDevice()
Right now this does nothing but add a new layer, but it means
that all DeviceMounts are paired with DeviceUnmounts so that we
can track (and cleanup) active mounts.
Upstream-commit: 9e64ebb29549db19a84b8cb514bea60c26184779
Component: engine
2013-09-30 17:35:01 -06:00
16f68cf6d5 Always start tests from a clean set of loopback images
This way we don't get any issues with leftovers
Upstream-commit: d951911b23b26cda087e823f203414a0ad184276
Component: engine
2013-09-30 17:35:01 -06:00
7bed559f55 tests: Store the loopback images for test outside unit-tests
This directory is copied to each test prefix which is really
slow with the large loopback mounts.
Upstream-commit: 7fb60caa5d34f8fc0abeae2ce7841000665414b0
Component: engine
2013-09-30 17:35:01 -06:00
70d3d13622 Allow specifying the docker client path in _DOCKER_INIT_PATH
I currently need this to get the tests running, otherwise it will
mount the docker.test binary inside the containers, which doesn't
work due to the libdevmapper.so dependency.
Upstream-commit: 76a2ab6e34e6cdd66899815aeeaac2048e5bafce
Component: engine
2013-09-30 17:35:00 -06:00
7d0232a89a Limit the amount of prints during normal runs
This removes some Debugf() calls and chages some direct prints to
Debugf(). This means we don't get a bunch of spew when running the
tests.
Upstream-commit: 6094257b28f2e4b5e1a6616c77961b5cec0c9195
Component: engine
2013-09-30 17:35:00 -06:00
816bf23aaf Reuse a single DeviceSetDM for all the tests
We wrap the "real" DeviceSet for each test so that we get only
a single device-mapper pool and loopback mounts, but still
separate out the IDs in the tests. This makes the test run
much faster.
Upstream-commit: 52294192b2a008b624862701cbf8491ad19b0798
Component: engine
2013-09-30 17:35:00 -06:00
0a6a934279 Add DeviceSetWrapper
This wraps an existing DeviceSet and just adds a prefix to all ids in
it. This will be useful for reusing a single DeviceSet for all the tests
(but with separate ids)
Upstream-commit: 381ce94ef4fe2590b0afa16c5f35a508de12e7a3
Component: engine
2013-09-30 17:35:00 -06:00
348d75b3b5 Delete corresponding Devices when deleting Images
If an image is deleted and there is a corresponding device
for that image we also delete the image.
Upstream-commit: 99393cf3cfd08de769f0c37f06b912fb3771a080
Component: engine
2013-09-30 17:35:00 -06:00
fef827ba7b Runtime: Delete corresponding devices when deleting container
Upstream-commit: 30890c7763e775d38c82d3eac5c1650e149137ae
Component: engine
2013-09-30 17:35:00 -06:00
79ab6381a5 Implement container.ExportRW() on device-mapper
Upstream-commit: b0626f403b168b9020a802ec3bc4ad8c9bbc2486
Component: engine
2013-09-30 17:34:59 -06:00
06dfc21edb Make TarFilter more useful
There are a few changes:
* Callers can specify if they want recursive behaviour or not
* All file listings to tar are sent on stdin, to handle long lists better
* We can pass in a list of filenames which will be created as empty
  files in the tarball

This is exactly what we want for the creation of layer tarballs given
a container fs, a set of files to add and a set of whiteout files to create.
Upstream-commit: fda6ff9c2707efbd1c9d1f2bf151b9d1d082d0c6
Component: engine
2013-09-30 17:34:59 -06:00
53e75b5d60 Archive: Fix up tar commandline arguments in TarFilter()
There is no need to duplicate the compression flags for
every element in the filter.
Upstream-commit: b86f67126c86a07aac155f38aefd6c1ef538e24d
Component: engine
2013-09-30 17:34:59 -06:00
534a030a97 Implement docker diff for device-mapper
To do diffing we just compare file metadata, so this relies
on things like size and mtime/ctime to catch any changes.
Its *possible* to trick this by updating a file without
changing the size and setting back the mtime/ctime, but
that seems pretty unlikely to happen in reality, and lets
us avoid comparing the actual file data.
Upstream-commit: 1c5dc26a7c0a0abb7bc59174768ec309f6c5fd4f
Component: engine
2013-09-30 17:34:59 -06:00
8fb407ad51 devmapper: Base the device-mapper names on the root dir name
This means the default is "docker-*", but for tests we get separate
prefixes for each test.
Upstream-commit: 8e7cbbff504b1d5b0680d2a14821d1d7b0757ab0
Component: engine
2013-09-30 17:34:59 -06:00
11ba6503a7 Image: Always create a .docker-id file in the devices we create
Without this there is really no way to map back from the device-mapper
devices to the actual docker image/container ids in case the json file
somehow got lost
Upstream-commit: 074f38d49377411cf0b805095c0d9909d4859f3c
Component: engine
2013-09-30 17:34:59 -06:00
0134fab2c6 Image: Deactivate image device when unmounting container
There is no need to keep all the device-mapper devices active, we
can just activate them on demand if needed.
Upstream-commit: a9ec1dbc9bec91e1c0f1b751a06680570a04e915
Component: engine
2013-09-30 17:34:59 -06:00
f4fd5894a2 Image: Initial support for device-mapper mounts
This supports creating images from layers and mounting them
for running a container.

Not supported yet are:
* Creating diffs between images/containers
* Creating layers for new images from a device-mapper container
Upstream-commit: d2ba3e200576c2bcceb81d8d3d65b86fbf49313b
Component: engine
2013-09-30 17:34:59 -06:00
03b4e52ca7 Runtime: Add MountMethod to allow AUFS and device-mapper to coexist
Upstream-commit: 8f7361279c13660a617886a523c50ecbc6849129
Component: engine
2013-09-30 17:34:59 -06:00
4f27979b41 Runtime: Add DeviceSet singleton
This adds a DeviceSet singleton to the Runtime object which will be used for
any DeviceMapper dependent code.
Upstream-commit: ca2f7f955e697091f2b7bee9a33c6c4e106cecd0
Component: engine
2013-09-30 17:34:59 -06:00
6c1aee42e6 Server: Pass in device-mapper DeviceSet to server
This makes docker (but not docker-init) link to libdevmapper and will
allow it to use the DeviceSet
Upstream-commit: 1d36b8c7b7b0c943ccb7d69b7181b3e33566d77c
Component: engine
2013-09-30 17:34:59 -06:00
acf340a68b Add DeviceSet interface
This interface matches the device-mapper implementation (DeviceSetDM)
but is free from any dependencies. This allows core docker code
to refer to a DeviceSet without having an explicit dependency on
the devmapper package.

This is important, because the devmapper package has external
dependencies which are not wanted in the docker client app, as it
needs to run with minimal dependencies in the docker image.
Upstream-commit: e6216793d91a9b003814b535b0373902326753dd
Component: engine
2013-09-30 17:34:58 -06:00
3b80b99ef6 Image: Add runtime and container id args to Mount()
We will later need the runtime to get access to the VolumeSet
singleton, and the container id to have a name for the volume
for the container
Upstream-commit: e368c8bb01b3c52c8e4c334c3a7f32556af9d632
Component: engine
2013-09-30 17:34:58 -06:00
b25a04bd5f Runtime: Automatically use docker-init if it exists
In some builds the main docker binary is not statically linked,
and as such not usable in as the .dockerinit binary, for those
cases we look for a separately shipped docker-init binary and
use that instead.
Upstream-commit: 167601e85850aa58df96f9d0796f9c26ed2d6fa4
Component: engine
2013-09-30 17:34:58 -06:00
3681b7af0d Add a separate docker-init binary
This may be used for the .dockerinit case if the main binary is not
statically linked.
Upstream-commit: b8dc7b5f1a8111275e2b869dfb67a8689cecf9b9
Component: engine
2013-09-30 17:34:58 -06:00
0553a0af33 devmapper: Add simple tool to test the DeviceSet commands
Upstream-commit: 7fb3bfed03d4d2a88f0e76fd6b8425cc753f4547
Component: engine
2013-09-30 17:34:58 -06:00
bf4813911d devmapper: Add DeviceSet device-mapper helper
This is a module that uses the device-mapper create CoW snapshots
You instantiate a DeviceSetDM object on a specified root (/var/lib/docker),
and it will create a subdirectory there called "loopback". It will
contain two sparse files which are loopback mounted into
a thin-pool device-mapper device called "docker-pool".

We then create a base snapshot in the pool with an empty filesystem
which can be used as a base for docker snapshots. It also keeps track
of the mapping between docker image ids and the snapshots in the pool.

Typical use of is something like (without error checking):

devices = NewDeviceSetDM("/var/lib/docker")
devices.AddDevice(imageId, "") // "" is the base image id
devices.MountDevice(imageId, "/mnt/image")
 ... extract base image to /mnt/image
devices.AddDevice(containerId, imageId)
devices.MountDevice(containerId, "/mnt/container")
... start container at /mnt/container
Upstream-commit: 374a5e9913112c5bde590e532bc0ba5e4afeda49
Component: engine
2013-09-30 17:34:58 -06:00
cdec248ebe Add libdevmapper wrapper
Upstream-commit: 459bac712709db4d188022539f4d59524b4b3670
Component: engine
2013-09-30 17:34:58 -06:00
98a0159ef9 Merge pull request #1994 from metalivedev/1993-legalcontext
Fix #1993: add explanation for export restrictions
Upstream-commit: 23015fa14dd3f58e582d7ef3039262025b06f21c
Component: engine
2013-09-30 16:33:46 -07:00
61073fc069 Merge pull request #2051 from tianon/hack-dockerfile-refactor
Add cleanup/refactor portion of #2010 for hack and Dockerfile updates
Upstream-commit: 85b776995cfcd02a8b0f6f07f6b5f57238363e83
Component: engine
2013-09-30 16:29:32 -07:00
c186883e60 Merge pull request #2035 from dhrp/style-revamp
Changes to a new style for the docs. Includes version switcher.
Upstream-commit: fa44555fb44f370ac01a1c26da3d5c1927833335
Component: engine
2013-09-30 15:51:43 -07:00
bd97f4052d Merge pull request #2033 from metalivedev/1969-cherrypick-fixcommitformat
Fix #1969 formatting, add information about multiline json
Upstream-commit: 3e7c50e8a17c66cbb45895235c7d9d38beafc200
Component: engine
2013-09-30 14:55:11 -07:00
33aa0c2b85 Add cleanup/refactor portion of #2010 for hack and Dockerfile updates
Upstream-commit: ccefe47897bdcef8b56bf54d4bd324989855f3b5
Component: engine
2013-09-30 13:57:30 -06:00
cf42215112 Merge pull request #2039 from tianon/mkimage-centos
Add contrib/mkimage-centos.sh back (from #1621), and associated documentation link
Upstream-commit: ff850319803a4c5910ec0a5a34134bf4af99d58e
Component: engine
2013-09-30 10:51:06 -07:00
3d2cd4cd2d Merge pull request #2046 from dotcloud/2030-dockercfg_panic_err-fix
- Runtime: fix panic with wrong dockercfg file
Upstream-commit: 017ecefd66cbf538e0b17a3f74ece05f92473c0f
Component: engine
2013-09-30 10:02:12 -07:00
bc13d7f295 fix panic with wrong dockercfg file
Upstream-commit: 649605915428e0ee81cf49d15e949d48da20110c
Component: engine
2013-09-30 11:07:32 +00:00
4664dde409 Merge pull request #1995 from dotcloud/fix_attach_2
Fix the attach behavior with -i
Upstream-commit: bcce3cbdd1cb4d79272c0cbfa43ef0847be1b9d9
Component: engine
2013-09-30 03:29:19 -07:00
cc91061a09 Merge pull request #2025 from dotcloud/2020_add_dockercfg_doc
add .dockercfg doc
Upstream-commit: bbf644ed62cf815cf40ef3de3345fac7ed42588a
Component: engine
2013-09-30 03:21:46 -07:00
bda4d2512b Add contrib/mkimage-centos.sh back (from #1621), and associated documentation link
Upstream-commit: d89ce09a76e7934338d4966d05f6b67618c8bfb0
Component: engine
2013-09-28 07:54:34 -06:00
17c973c7fe Changes to a new style for the docs. Includes version switcher.
* added link to edit on GitHub
* Changed image source on homepage
* Made some changes to the structure, added the ability to have l3 navigation. Added warning, note and other styles.
* Fixed an image link, removed the .. :content: links because they were quicky and didn't look good, added pagelinks to current page of other versions.
* Moved the remote client api's to their own doc
Upstream-commit: e3e041b6bf0066fa5e65625694517ff387fee3b1
Component: engine
2013-09-27 19:54:52 -07:00
ac095d59b4 Added more context.
Upstream-commit: 0a35b1fb3681561a1f3f139d989c0bd133122123
Component: engine
2013-09-27 19:07:12 -07:00
024f3a4096 Fix #1969 formatting, add information about multiline json
Upstream-commit: 74d54b9b2eec8f92d688b7d53e813ebe931e4114
Component: engine
2013-09-27 18:16:00 -07:00