Commit Graph

34 Commits

Author SHA1 Message Date
044c307631 small batch of edits/corrections to comments
Upstream-commit: 0fccf0f68655fd01973e4c00bbb063fad1bce56c
Component: engine
2013-12-24 16:40:14 -08:00
e69eef70c1 Add -S option to tar for efficient sparse file handling
Fixes issue #3282
Upstream-commit: 733bf5d3ddbfb6dba7c2c0996c4af47a765e4593
Component: engine
2013-12-19 21:41:22 -08:00
3222062547 archive: Re-add XZ compression support
This shells out to the xz binary to support .tar.xz layers, as
there is no compression/xz support in go.
Upstream-commit: b8a4f570fb31091f43caeba5b824ae38a5bc69e8
Component: engine
2013-12-18 10:50:22 +01:00
c725c9d310 Handle compressed tars in ApplyLayer
When pulling from a registry we get a compressed tar archive, so
we need to wrap the stream in the right kind of compress reader.

Unfortunately go doesn't have an Xz decompression method, but I
don't think any docker layers use that atm anyway.
Upstream-commit: a96a26c62f6333e61134759256db43a7f5946631
Component: engine
2013-12-17 14:19:48 +01:00
692e7bb3bd ApplyLayer: Fix TestLookupImage
The TestLookupImage test seems to use a layer that contains
/etc/postgres/postgres.conf, but not e.g. /etc/postgres.

To handle this we ensure that the parent directory always
exists, and if not we create it.
Upstream-commit: 78c22c24b353d77fdab3e1616d9986a8ae95a7c2
Component: engine
2013-12-16 14:35:43 +01:00
7bf68dd3de Re-enable TestApplyLayer
With the previous two changes we now pass this test.
Upstream-commit: a8af12f80a4a1678988b4667e5211d4e576ce903
Component: engine
2013-12-13 15:50:25 +01:00
51b58640b0 Fix change detection when applying tar layers
The default gnu tar format has no sub-second precision mtime support,
and the golang tar writer currently doesn't support that either.
This means if we export the changes from a container we will not
get zeron in the sub-second precision field when the change is applied.

This means we can't compare that to the original without getting a
spurious change. So, we detect this case by treating a case where the
seconds match and either of the two nanoseconds are zero as equal.
Upstream-commit: 10cd902f900392a2f10a6f8763bba70607ea0d41
Component: engine
2013-12-13 15:46:41 +01:00
e5d96fb523 archive: Implement ApplyLayer directly
Rather than calling out to tar we use the golang tar parser
to directly extract the tar files. This has two major advantages:

1) We're able to replace an existing directory with a file in the
   new layer. This currently breaks with the external tar, since
   it refuses to recursively remove the destination directory in
   this case, and there are no options to make it do that.

2) We avoid extracting the whiteout files just to later remove them.
Upstream-commit: 818c249bae8d29842834bf765299c86c09e6913e
Component: engine
2013-12-13 15:43:50 +01:00
26d7cf1b0c fixed some go vet issues.
Upstream-commit: a6c9a332d040a98c5cf4325491fc8d9aa7e5475b
Component: engine
2013-11-29 22:53:20 -02:00
3f2a763338 Move syscall.Stats logic to os specific file.
related to #2909.
Upstream-commit: 8b2a7e35c35f894dca0795a4fde9ec0cfe04ce43
Component: engine
2013-11-28 00:22:47 -02:00
116f6db1b3 Set consistent time for sample dir in archive test
Upstream-commit: 0d1b5d767648ee6200a816786e51d23917bf0218
Component: engine
2013-11-25 19:58:14 -08:00
cf0c94cbee Merge pull request #48 from crosbymichael/reset-mtime
Ensure same atime, mtime after applying whiteouts
Upstream-commit: 91330243b53c5340c80d132379d0e97f2fb0c771
Component: engine
2013-11-19 15:40:12 -08:00
b6950e3878 Merge branch 'master' into creack-merge-master
Conflicts:
	api_params.go
	graph.go
	graph_test.go
	image.go
	integration/api_test.go
	integration/container_test.go
	integration/runtime_test.go
	runtime.go
Upstream-commit: 28d4cbbc59e419f00c1ad75991b106241933ca5d
Component: engine
2013-11-19 14:58:12 -08:00
99bee9426b Implement fallback for getting the size of a container
This moves Driver.Size() to Differ.DiffSize(), removing the empty
implementations in devmapper and dummy, and renaming the one in aufs.

Then we fall back to a container.Changes() implementation in the non-aufs
case.
Upstream-commit: 5d76681c3d0cbb744205a397420603ce029ce754
Component: engine
2013-11-19 10:36:54 +01:00
04ef2191af Ensure same atime, mtime after applying whiteouts
Upstream-commit: 3edb4af663079ad79672bc98fcd20790f8fb2589
Component: engine
2013-11-19 00:35:03 -08:00
6e0c243c2e Lintify code with confidence=1
Upstream-commit: 5e941f1ca035bb1ec014c18d277aecaa41deba85
Component: engine
2013-11-18 16:24:11 -08:00
63db5cd7f5 Mark archive.TestApplyLayer as skipped
Instead of just returning we call Skip to log that the test
was skipped.
Upstream-commit: 8ed4307f50b93d7c49eccf30357a4b9d3fbb1376
Component: engine
2013-11-18 14:31:34 +01:00
d9b9f3ddbd Add tests for the changes detection code
This adds 3 tests:

Verify that ChangesDirs() returns nothing when run on a copy of the
same directory.

Verify that various mutations of a directory get reported with the
right changes.

Verify that ExportChanges() + ApplyLayer() of the above mutation gets
an identical directory.

Unfortunately the last test is disabled because it fails in multiple
ways atm. But I want to get it in so that we can fix it.
Upstream-commit: f7238f94e8b5b31e78360c342676ca84b5984806
Component: engine
2013-11-15 09:49:46 +01:00
b091059403 Use ExportChanges() in runtime.Diff()
This code was duplicated in two places, one which was unused. This
syncs the code and removes the unused version.
Upstream-commit: d69a6a20f0b6657821638ee591920d071a784b0e
Component: engine
2013-11-15 09:49:42 +01:00
39e4a0bc9e ApplyLayer() use RemoveAll to handle removing directories
rmTargetPath can be a directory, so we have to use RemoveAll() or we
will fail to whiteout non-empty directories.
Upstream-commit: 006e2a600ce689770ba2c49805bc4f634976f365
Component: engine
2013-11-14 22:56:18 +01:00
5f7a312280 Merge branch 'dm-plugin-new-ChangesDirs' of https://github.com/alexlarsson/docker into alexlarsson-dm-plugin-new-ChangesDirs
Conflicts:
	archive/changes.go
Upstream-commit: eace2dbe1d49ad6a990f5b42f092d3f4d77bf225
Component: engine
2013-11-14 12:23:01 -08:00
88636cc70a Merge pull request #17 from alexlarsson/dm-plugin-whiteout-directory
archive.ApplyLayer() - handle directory whiteouts
Upstream-commit: 50d80a89381366001d2e888b822a139cd8eb2025
Component: engine
2013-11-12 12:16:30 -08:00
82914826a6 archive.ApplyLayer() - handle directory whiteouts
When directories are white-outed we can get called with the previously
removed directories. Handle this with os.IsNotExist(error).
Upstream-commit: 6f3c32eb18f0a604ba2638b6a9f19f8824412fd1
Component: engine
2013-11-12 21:14:54 +01:00
084cb15f1a Allow drivers to implement ApplyDiff in Differ interface
Upstream-commit: 5d9723002bc764e2c768e5184994d7949f55fc49
Component: engine
2013-11-11 19:09:57 -08:00
b93991acb1 Add ability to exclude files from tar
Upstream-commit: 2c7f50a77dc281289387008b4a08656e7e5328be
Component: engine
2013-11-11 14:30:38 -08:00
d30197ef54 archive.ChagesDir() - faster and better implementation
This replaces the current version with the latest version from the
dm branch. Advantages in this version:

We only scan each directory tree once, which means less i/o and less chance
of container changes inbetween the two scans causing inconsistencies.

We avoid comparing some fields for change detection:

* Inode
* size-in-blocks
  These can change during a copy operation (e.g. in the dummy backend)
  without needing to actually reflect a change in content or metadata.

* Ctime
  Any copy operation will create a new Ctime value, and there is no
  API to change it to the "source" value.

* size for directories
  The size of a directory doesn't have to be the same just because you
  recreated the same content as another director. Internal details in
  the filesystem may make these different with no "real" change.
Upstream-commit: b6ef4bc9521346dc5066d71821c6cadfbeced9d3
Component: engine
2013-11-11 14:51:19 +01:00
22263cd931 Merge pull request #6 from alexlarsson/fix-ApplyLayer-remove
archive.ApplyLayer() remove files recursively
Upstream-commit: 78ef0bd9985d899dda6fdfc8c35d9ab47f7f07d7
Component: engine
2013-11-08 11:46:55 -08:00
07ffba21a6 Don't rely on drivers for diff and export
Upstream-commit: 99210c9c6efbe10df922faa66fa3232ab898ef7b
Component: engine
2013-11-08 02:48:52 +00:00
914824bce7 Move Change to the archive package, and fix a leftover merge in
Container.Inject()
Upstream-commit: 9ae4bcaaf8aa01da340341e47f99f83a59ce91dc
Component: engine
2013-11-08 00:45:07 +00:00
783f801512 gofmt
Upstream-commit: f1127b9308fbb4d4e21c3c5567ace9fdda82559d
Component: engine
2013-11-07 23:58:03 +00:00
65b7bbeca7 archive.ApplyLayer() remove files recursively
Some aufs metadata are directories, and whiteouts can be for
directories, so use os.RemoveAll() for these.
Upstream-commit: 484804abffda2692492e295d4691f90564d05eb2
Component: engine
2013-11-07 22:46:44 +01:00
6fb0ff4f9a archive.ApplyLayer(): parse a tar archive as a standard aufs-compatible diff, and apply it on a directory
Upstream-commit: 318dd33fb778282c3012ee06d892d6375e98cd6b
Component: engine
2013-11-01 05:06:35 +00:00
c2b31012f8 Partial merge of devmapper/ in order to integrate it as a backend
plugin.

The merge is inert, in other words the devmapper code is not called
and the primary aufs backend is untouched.
Upstream-commit: 36c7a7ae943220ffb1bea9cd47d2f586db00eab2
Component: engine
2013-11-01 04:24:17 +00:00
fb9771da9e Move archive.go to sub package
Upstream-commit: 96d1e9bb5a269936afeb9748a894f6995b17e0a4
Component: engine
2013-10-31 16:57:45 -07:00