Commit Graph

3315 Commits

Author SHA1 Message Date
a29076830e hack: Don't just run the "TestRunHostname" test
Upstream-commit: 9b65c7cf49feacad61cdcd11d2b002319a6b6bae
Component: engine
2013-10-04 15:38:47 +02:00
850aea64f1 Remove overly spewy Debugf
Upstream-commit: f7e374fb3a51bd25e61ae6c4343838a0ca756fba
Component: engine
2013-10-04 15:36:30 +02:00
78b0694228 Make sure we mark the libdevmapper /dev/mapper/control fd CLOEXEC
We do a hack to mark it such, because otherwise lxc-start will not
work.
Upstream-commit: 1a1be5a87c2169def7b0b0c29fc5cbac850a00a5
Component: engine
2013-10-03 21:00:16 +02:00
c413093856 Be better at cleaning up leftover from earlier test runs
When running the test inside a docker container we sometimes are left with
leftover device nodes for device mapper devices that no longer exist.
We were panic:ing in this case, but with this change we just remove such
nodes.
Upstream-commit: 7b58e15b08349dba35bf3813f77a0b900c2e4df5
Component: engine
2013-10-03 19:54:14 +02:00
7dc11f73be Random improvments
Upstream-commit: 8b2f4aab232880cb0d7faa18ae12adb33f483b2c
Component: engine
2013-10-02 20:18:15 -07:00
788a1fc558 Update Dockerfile and hack to support compiling device-mapper code statically (using go1.2rc1)
Upstream-commit: 06d0843a61843e6b16a56d518e21032a5652098b
Component: engine
2013-10-03 10:32:47 -06:00
e87cd90363 rebase
Upstream-commit: deb05a36e8cb0361c7900b128e594cd6db06ba73
Component: engine
2013-10-03 15:49:28 +00:00
705988989e disable: don't create device nodes manually if udev is not availabile as we don't have it in dind
Upstream-commit: 55189307d0ec6992f76572fae675bd371cc36cff
Component: engine
2013-10-03 15:47:38 +00:00
615792fa5b Image: Fix time setting for old kernels
This is a better fix for futimes() on kernels not supporting O_PATH.
The previous fix broke when copying a device, as it tried to open it
and got and error.
Upstream-commit: 55e1782d6623b59af3f1aea1eb9646a36bbb5579
Component: engine
2013-09-30 17:35:02 -06:00
6420d1eea7 go fmt and aufs support removed
Upstream-commit: 152302e379eaa5b2dfc27d901934da0a18b46beb
Component: engine
2013-09-30 17:35:02 -06:00
bc0e976e11 Revert "add a -mount-method flag"
This reverts commit e52d756f40c9ccf8b37ca496cb72be057c909ed7.
Upstream-commit: 72a08a5458d5d8248dec431c82241a7b18b2657b
Component: engine
2013-09-30 17:35:02 -06:00
0728d1e89a add a -mount-method flag
Upstream-commit: aeb89ffbbac8c5625ad4dbba66938e2ef3f5c638
Component: engine
2013-09-30 17:35:02 -06:00
8ed20571fc RootIsShared: Fix root detection
Column 4 is the mount position, column 3 will not always be
"/" for the root. On one of my system its "/root".
Upstream-commit: 0484b2c3254238a6c534f7d417c12c10b694f0d0
Component: engine
2013-09-30 17:35:02 -06:00
1d7e574dd4 Tests: Clean up any old devmapper leftovers before starting tests
Upstream-commit: be6fef02544d9bc31321b2a21d039e261dc02bd8
Component: engine
2013-09-30 17:35:02 -06:00
1461717d2e Image.Changes: Deactivate image device after unmounting it
There is no need to keep the image device around if we were the
onces creating the device.
Upstream-commit: 071cc18b58408eaa71575e0233a056475196b199
Component: engine
2013-09-30 17:35:02 -06:00
64c3059ad8 Add DeviceSet.HasActivatedDevice()
This lets you see if a device has been activated
Upstream-commit: 6ec5585501c6029ef10a474597503511d5074e10
Component: engine
2013-09-30 17:35:02 -06:00
890bafe58d image: Unmount before removing device in error paths
The device remove fails unless we unmount first
Upstream-commit: 6f57e8025a0a03da8052edc30157f7677f65f208
Component: engine
2013-09-30 17:35:02 -06:00
5fc36fa5bf runtime test: Ensure all containers are unmounted at nuke()
Otherwise we may leave around e.g. devmapper mounts
Upstream-commit: b0a9147fd5f0197931b73e18163a12b04ad8e826
Component: engine
2013-09-30 17:35:02 -06:00
c5a71d7b11 ShellQuoteArguments: Fix quoting
This accidentally used two quotes to start/end each quoted string.
Upstream-commit: a7e876e3571965de64615d2012e63fae9c3ebbf8
Component: engine
2013-09-30 17:35:01 -06:00
17aec56f2a Image: unmount device before removing it on failures
If we don't do this the remove will fail due to EBUSY
Upstream-commit: ecdbdfdaea5497c34a0299e7efd97b8a44bd7314
Component: engine
2013-09-30 17:35:01 -06:00
6101a0a475 image: Handle systems that don't support O_PATH when updating timestamp
Older kernel can't handle O_PATH in open() so this will
fail on dirs and symlinks. For dirs wa can fallback to
the normal Utimes, but for symlinks there is not much to do
but ignore their timestamps.
Upstream-commit: 2c71710b74829787a0c78f7e02f45d31935a996f
Component: engine
2013-09-30 17:35:01 -06:00
2af8abef3d devmapper: Fix loopback mount code
Typo in the loop-control code made it always fall back to the
old method of opening loopback devices.
Upstream-commit: bbc72c85f7733fefda499cb0e96f1dda2ce90625
Component: engine
2013-09-30 17:35:01 -06:00
91d411052a applyLayer() use btrfs reflinks if availible
We use the new file copy helper which uses btrfs reflinks if availible.
Upstream-commit: 1a082ed245aaf55cc92e884c988865b195ea0cb7
Component: engine
2013-09-30 17:35:01 -06:00
08576d6eaf Add CopyFile that can use btrfs reflinks if availible
Upstream-commit: 86421e8b5e84452d66e6b16b5cb1a5a438f8fa1f
Component: engine
2013-09-30 17:35:01 -06:00
19c4006853 Remove accidental commit that enabled MountMethodFilesystem
Upstream-commit: 91c69fd3532eaa92cfc21c6331699a995fdbe2a6
Component: engine
2013-09-30 17:35:01 -06:00
9f8a6e47d8 Add trivial copy-based CoW backend
This creates a container by copying the corresponding files
from the layers into the containers. This is not gonna be very useful
on a developer setup, as there is no copy-on-write or general diskspace
sharing. It also makes container instantiation slower.

However, it may be useful in deployment where we don't always have a lot
of containers running (long-running daemons) and where we don't
do a lot of docker commits.
Upstream-commit: 43a7d3d0e9e2fafcdc90cb84855e1bb3869d2729
Component: engine
2013-09-30 17:35:01 -06:00
6dae654f9c Add Changes.ChangesLayers()
This calculates the difference between a set of layers and a
directory tree.
Upstream-commit: 60f552cac3d165dbe4c8d65c1ab82d31700dcc5d
Component: engine
2013-09-30 17:35:01 -06:00
3953036d64 Changes: Better metadata comparison
Change the comparison to better handle files that are copied during
container creation but not actually changed:

* Inode - this will change during a copy
* ctime - this will change during a copy (as we can't set it back)
* blocksize - this will change for sparse files during copy
* size for directories - this can change anytime but doesn't
  necessarily reflect an actual contents change
* Compare mtimes at microsecond precision (as this is what utimes has)
Upstream-commit: ad402763e160ded1924c9b154983d81614e90deb
Component: engine
2013-09-30 17:35:01 -06:00
84b4aeca21 Image.applyLayer: Be better at creating identical files
There are some changes here that make the file metadata better match
the layer files:

* Set the mode of the file after the chown, as otherwise the per-group/uid
  specific flags and e.g. sticky bit is lost
* Use lchown instead of chown
* Delay mtime updates to after all other changes so that later file
  creation doesn't change the mtime for the parent directory
* Use Futimes in combination with O_PATH|O_NOFOLLOW to set mtime on symlinks
Upstream-commit: 5d2ace3424516bd7cc8d4a57fcaddd00fa1c4b5d
Component: engine
2013-09-30 17:35:01 -06:00
ce9d889436 Change how ChangesDirs() works
Rather than scan the files in the old directory twice to detect the
deletions we now scan both directories twice and then do all the
diffing on the in-memory structure.

This is more efficient, but it also lets us diff more complex things
later that are not exact on-disk trees.
Upstream-commit: 727e7fcccadf1d3e286f5a3c8d1aa388f6b4dab8
Component: engine
2013-09-30 17:35:01 -06:00
ff5de9f609 RootIsShared() - Fix array out of bounds error
This happened for me on the last (empty) line, but better safe than sorry
so we make the check general.
Upstream-commit: 7d566b4f761e8942cf9679e96774b320b8496b2f
Component: engine
2013-09-30 17:35:01 -06:00
74b00e51f4 devmapper: Move init layer to top rather than bottom
The init layer needs to be topmost to make sure certain files
are always there (for instance, the ubuntu:12.10 image wrongly
has /dev/shm being a symlink to /run/shm, and we need to override
that). However, previously the devmapper code implemented the
init layer by putting it in the base devmapper device, which meant
layers above it could override these files (so that ubuntu:12.10
broke).

So, instead we put the base layer in *each* images devmapper device.
This is "safe" because we still have the pristine layer data
in the layer directory. Also, it means we diff the container
against the image with the init layer applied, so it won't show
up in diffs/commits.
Upstream-commit: fdbc2695fe00d522c5c1a962f9be2f802bf53943
Component: engine
2013-09-30 17:35:01 -06:00
1d1c891003 lxc: Work around lxc-start need for private mounts
lxc-start requires / to be mounted private, otherwise the changes
it does inside the container (both mounts and unmounts) will propagate
out to the host.

We work around this by starting up lxc-start in its own namespace where
we set / to rprivate.

Unfortunately go can't really execute any code between clone and exec,
so we can't do this in a nice way. Instead we have a horrible hack that
use the unshare command, the shell and the mount command...
Upstream-commit: 429587779a95a4d38ec9cd66202de9729c320ef8
Component: engine
2013-09-30 17:35:01 -06:00
95fd1df21e Utils: Add ShellQuoteArguments
Upstream-commit: 145024c6ccc7bff1eda632823702f91899667ed3
Component: engine
2013-09-30 17:35:01 -06:00
f3c281436d Container: Inject into the mount, not the rwPath
For device-mapper setups we can't just push the file into the rwPath.
Upstream-commit: c1388010731cac1c8ff5159bd72b926545a64e58
Component: engine
2013-09-30 17:35:01 -06:00
4cb2cd0d23 api_test: Fix PostContainersCreate
We can't look for the created file in the rwpath, because that
doesn't exist in the device-mapper world, instead look in the
RootfsPath.
Upstream-commit: 07227866006ea75a0c492814b808fdbb672431ee
Component: engine
2013-09-30 17:35:01 -06:00
0453049ffb Runtime: Only remove device on destroy if it exists
Upstream-commit: 5f8e24f8428bc5fb7076ec757ded44f965c48888
Component: engine
2013-09-30 17:35:01 -06:00
eb4c81b5cf graph test: Unmount image via image.Unmount()
This helps us track the unmount
Upstream-commit: 0d7ab8db03814bac19fe076a0d53b6d423616bc7
Component: engine
2013-09-30 17:35:01 -06:00
89fbc4edd2 deviceset: Cleanup device sets on test end
We unmount all mounts and deactivate all device mapper devices to
make sure we're left with no leftovers after the test.
Upstream-commit: ed741f7b27b1b1cf5b6f8917551ce86bc39e9c78
Component: engine
2013-09-30 17:35:01 -06:00
12fbeaaae6 DeviceSet: Add UnmountDevice()
Right now this does nothing but add a new layer, but it means
that all DeviceMounts are paired with DeviceUnmounts so that we
can track (and cleanup) active mounts.
Upstream-commit: 9e64ebb29549db19a84b8cb514bea60c26184779
Component: engine
2013-09-30 17:35:01 -06:00
16f68cf6d5 Always start tests from a clean set of loopback images
This way we don't get any issues with leftovers
Upstream-commit: d951911b23b26cda087e823f203414a0ad184276
Component: engine
2013-09-30 17:35:01 -06:00
7bed559f55 tests: Store the loopback images for test outside unit-tests
This directory is copied to each test prefix which is really
slow with the large loopback mounts.
Upstream-commit: 7fb60caa5d34f8fc0abeae2ce7841000665414b0
Component: engine
2013-09-30 17:35:01 -06:00
70d3d13622 Allow specifying the docker client path in _DOCKER_INIT_PATH
I currently need this to get the tests running, otherwise it will
mount the docker.test binary inside the containers, which doesn't
work due to the libdevmapper.so dependency.
Upstream-commit: 76a2ab6e34e6cdd66899815aeeaac2048e5bafce
Component: engine
2013-09-30 17:35:00 -06:00
7d0232a89a Limit the amount of prints during normal runs
This removes some Debugf() calls and chages some direct prints to
Debugf(). This means we don't get a bunch of spew when running the
tests.
Upstream-commit: 6094257b28f2e4b5e1a6616c77961b5cec0c9195
Component: engine
2013-09-30 17:35:00 -06:00
816bf23aaf Reuse a single DeviceSetDM for all the tests
We wrap the "real" DeviceSet for each test so that we get only
a single device-mapper pool and loopback mounts, but still
separate out the IDs in the tests. This makes the test run
much faster.
Upstream-commit: 52294192b2a008b624862701cbf8491ad19b0798
Component: engine
2013-09-30 17:35:00 -06:00
0a6a934279 Add DeviceSetWrapper
This wraps an existing DeviceSet and just adds a prefix to all ids in
it. This will be useful for reusing a single DeviceSet for all the tests
(but with separate ids)
Upstream-commit: 381ce94ef4fe2590b0afa16c5f35a508de12e7a3
Component: engine
2013-09-30 17:35:00 -06:00
348d75b3b5 Delete corresponding Devices when deleting Images
If an image is deleted and there is a corresponding device
for that image we also delete the image.
Upstream-commit: 99393cf3cfd08de769f0c37f06b912fb3771a080
Component: engine
2013-09-30 17:35:00 -06:00
fef827ba7b Runtime: Delete corresponding devices when deleting container
Upstream-commit: 30890c7763e775d38c82d3eac5c1650e149137ae
Component: engine
2013-09-30 17:35:00 -06:00
79ab6381a5 Implement container.ExportRW() on device-mapper
Upstream-commit: b0626f403b168b9020a802ec3bc4ad8c9bbc2486
Component: engine
2013-09-30 17:34:59 -06:00
06dfc21edb Make TarFilter more useful
There are a few changes:
* Callers can specify if they want recursive behaviour or not
* All file listings to tar are sent on stdin, to handle long lists better
* We can pass in a list of filenames which will be created as empty
  files in the tarball

This is exactly what we want for the creation of layer tarballs given
a container fs, a set of files to add and a set of whiteout files to create.
Upstream-commit: fda6ff9c2707efbd1c9d1f2bf151b9d1d082d0c6
Component: engine
2013-09-30 17:34:59 -06:00