The "TestChangesWithChanges" case randomlly fails on my development
VM with the following errors:
```
--- FAIL: TestChangesWithChanges (0.00s)
changes_test.go:201: no change for expected change C /dir1/subfolder != A /dir1/subfolder/newFile
```
If I apply the following patch to changes_test.go, the test passes.
```diff
diff --git a/pkg/archive/changes_test.go b/pkg/archive/changes_test.go
index 290b2dd..ba1aca0 100644
--- a/pkg/archive/changes_test.go
+++ b/pkg/archive/changes_test.go
@@ -156,6 +156,7 @@ func TestChangesWithChanges(t *testing.T) {
}
defer os.RemoveAll(layer)
createSampleDir(t, layer)
+ time.Sleep(5 * time.Millisecond)
os.MkdirAll(path.Join(layer, "dir1/subfolder"), 0740)
// Let's modify modtime for dir1 to be sure it's the same for the two layer (to not having false positive)
```
It seems that if a file is created immediately after the directory is created,
the `archive.Changes` function could't recognize that the parent directory of
the new file is modified.
Perhaps the problem may reproduce on machines with low time precision?
I had successfully reproduced the failure on my development VM as well as
a VM on DigitalOcean.
Signed-off-by: Shijiang Wei <mountkin@gmail.com>
Upstream-commit: e2c6a8be7c03d3c8405421c07ec6fb1c5c3a6200
Component: engine
This makes the "Buffering to disk" part of `docker push` 70% faster in
my use-case (having already applied #12833).
fsync'ing here serves no valuable purpose: if the drive's operation is
interrupted, so it the program's, and this archive has no value other
than the immediate and transient one.
Signed-off-by: Burke Libbey <burke.libbey@shopify.com>
Upstream-commit: 236dbc2e59f5b665f9fa30f3f7ba1fe6c8483b24
Component: engine
This case is triggered frequently on ZFS.
Signed-off-by: Burke Libbey <burke.libbey@shopify.com>
Upstream-commit: 03956610e588057e903288b8fb50df46c8d8f5d9
Component: engine
If we tear through a few layers of abstraction, we can get at the inodes
contained in a directory without having to stat all the files. This
allows us to eliminate identical files much earlier in the changelist
generation process.
Signed-off-by: Burke Libbey <burke@libbey.me>
Upstream-commit: 45c45a2c9a71528489a58fe633849f16e245631c
Component: engine
Add tests on:
- changes.go
- archive.go
- wrap.go
Should fix#11603 as the coverage is now 81.2% on the ``pkg/archive``
package. There is still room for improvement though :).
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: c21d408ad24cf8e2b5bd761d562fae7e3ae1bc54
Component: engine
- Trying to add or complete unit test to each ``func``
- Removing dead code (``escapeName``)
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: c4fe5dad1deb45ecde460d7627523dbf032dc205
Component: engine
on overlay fs, the mtime of directories changes in a container where new
files are added in an upper layer (e.g. '/etc'). This flags the
directory as a change where there was none.
Closes#9874
Signed-off-by: Vincent Batts <vbatts@redhat.com>
Upstream-commit: 2ce37f6616762900aa941c0644dece9cdbf90124
Component: engine
cc: @jhowardmsft
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Upstream-commit: 8ba64b383dee8827c70f9c0a0d71aa61d0f0c595
Component: engine
This change modifies the chmod bits of build context archives built on
windows to preserve the execute bit and remove the r/w bits from
grp/others.
Also adjusted integ-cli tests to verify permissions based on the platform
the tests are running.
Fixes#11047.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Upstream-commit: 1a22418f9f1573ca8521831d52c8e0562cb3ef8f
Component: engine
Currently TestBuildRenamedDockerfile fails since passing
custom dockerfile paths like:
docker build -f dir/file .
fails on windows because those are unix paths. Instead, on
windows accept windows style paths like:
docker build -f dir\file .
and convert them to unix style paths using the helper we
have in `pkg/archive` so that daemon can correctly locate
the path in the context.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Upstream-commit: 5ecab9e831c6d39e649430013087a3e2a85fb949
Component: engine
Currently pkg/archive stores nested windows files with
backslashes (e.g. `dir\`, `dir\file.txt`) and this causes
tar not being correctly extracted on Linux daemon.
This change assures we canonicalize all paths to unix
paths and add them to tar with that name independent of platform.
Fixes the following test cases for Windows CI:
- TestBuildAddFileWithWhitespace
- TestBuildCopyFileWithWhitespace
- TestBuildAddDirContentToRoot
- TestBuildAddDirContentToExistingDir
- TestBuildCopyDirContentToRoot
- TestBuildCopyDirContentToExistDir
- TestBuildDockerignore
- TestBuildEnvUsage
- TestBuildEnvUsage2
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Upstream-commit: 28842d3f093e69cf62a090a532aacba1e6ff6d1c
Component: engine
This test was written against
master(abdfb21e3a761efdd70614de42905ff7911c5372) as a failing test, but
works with this patch set.
Signed-off-by: Vincent Batts <vbatts@redhat.com>
Upstream-commit: 899a2dda097a57a12f8794ded4a706781173aa5d
Component: engine
sort changes found and exported.
Sorting the files before appending them to the tar archive
would mean a dependable ordering for types like hardlinks.
Also, combine sort logic used
Signed-off-by: Vincent Batts <vbatts@redhat.com>
Upstream-commit: 32d6d49539f5876e6ef75f00db917d8a3aaa0569
Component: engine
If .dockerignore mentions either then the client will send them to the
daemon but the daemon will erase them after the Dockerfile has been parsed
to simulate them never being sent in the first place.
an events test kept failing for me so I tried to fix that too
Closes#8330
Signed-off-by: Doug Davis <dug@us.ibm.com>
Upstream-commit: 6d801a3caa54ad7ef574bc426aa1ffc412c5af82
Component: engine
To avoid an expensive call to archive.ChangesDirs() which walks two directory
trees and compares every entry, archive.ApplyLayer() has been extended to
also return the size of the layer changes.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Upstream-commit: 35a22c9e12c05e2a0a205964702ced78ea39d7a1
Component: engine
With 32ba6ab from #9261, TempArchive now closes the underlying file and
cleans it up as soon as the file's contents have been read. When pushing
an image, PushImageLayerRegistry attempts to call Close() on the layer,
which is a TempArchive that has already been closed. In this situation,
Close() returns an "invalid argument" error.
Add a Close method to TempArchive that does a no-op if the underlying
file has already been closed.
Signed-off-by: Andy Goldstein <agoldste@redhat.com>
Upstream-commit: 48ec176cd51da20e23564941da2d9906a7779d28
Component: engine
This fixes the removal of TempArchives which can read with only one
read. Such archives weren't getting removed because EOF wasn't being
triggered.
Docker-DCO-1.1-Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com> (github: unclejack)
Upstream-commit: 32ba6ab83c7e47d627a2b971e7f6ca9b56e1be85
Component: engine
pkg/archive contains code both invoked from cli (cross platform) and
daemon (linux only) and Unix-specific dependencies break compilation on
Windows. We extracted those stat-related funcs into platform specific
implementations at pkg/system and added unit tests.
Signed-off-by: Ahmet Alp Balkan <ahmetb@microsoft.com>
Upstream-commit: 2180aa4f6f2ad4d8f284d63ee29e93547263976e
Component: engine
Some parts of pkg/archive is called on both client/daemon code. To get
it compiling on Windows, these funcs are extracted into files with
build tags.
Signed-off-by: Ahmet Alp Balkan <ahmetb@microsoft.com>
Upstream-commit: 3d2fae353f6ddc819d3a3c4db80887a40ac6f5f0
Component: engine
