Build cache uses pgk/tarsum to get a digest of content which is
ADD'd or COPY'd during a build. The builder has always used v0 of
the tarsum algorithm which includes mtimes however since the whole
file is hashed anyway, the mtime doesn't really provide any extra
information about whether the file has changed and many version
control tools like Git strip mtime from files when they are cloned.
This patch updates the build subsystem to use v1 of Tarsum which
explicitly ignores mtime when calculating a digest. Now ADD and
COPY will result in a cache hit if only the mtime and not the file
contents have changed.
NOTE: Tarsum is NOT a meant to be a cryptographically secure hash
function. It is a best-effort approach to determining if two sets of
filesystem content are different.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Upstream-commit: 0e10507a1c5985e6fda0ff48e9313ba7a4de761b
Component: engine
Change CLI error msg because it was too specific and didn't make sense
when there were errors not related to inaccessible files.
Removed some log.Error() calls since they're not really errors we should
log. Returning the error will be enough.
Closes: #13417
Signed-off-by: Doug Davis <dug@us.ibm.com>
Upstream-commit: 82ea6ed2bc33ac1ec2ad2bd8d4a098031dd77095
Component: engine
Previous fix used %q which incorrectly go-escaped things. For example:
```
RUN echoo A \& B C
```
would result in the user seeing:
```
INFO[0000] The command '/bin/sh -c echoo A \\& B\tC' returned a non-zero code: 127
```
Note the double-\ and the \t instead of a tab character
The testcase had to double escape things due to logrus getting in the way
but I'm going to fix that in another PR because its a change to the UX.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Upstream-commit: 006c066b6ce0b8b16f7d68b0ef997717bfb35662
Component: engine
When RUN returns with a non-zero return code it prints the command
that was executed as a Go []string:
```
INFO[0000] The command &{[/bin/sh -c noop a1 a2]} returned a non-zero code: 127
```
instead it should look like this:
```
INFO[0000] The command "/bin/sh -c noop a1 a2" returned a non-zero code: 127
```
Signed-off-by: Doug Davis <dug@us.ibm.com>
Upstream-commit: 54662eae10923a0aca03ffddc1a30b7d25431c79
Component: engine
This fixes an issue where the build output for the "Steps" would look like:
```
Step 1: RUN echo hi echo hi
```
instead of
```
Step 1: RUN echo hi
```
Also, I noticed that there were no checks to make sure invalid Dockerfile
cmd flags were caught on cmds that didn't use cmd flags at all. They would
have been caught on the cmds that had flags, but cmds that didn't bother
to add a new code for flags would have just ignored them. So, I added
checks to each cmd to flag it.
Added testcases for issues.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Upstream-commit: 08b7f30fcd050244026098673b19700485308b5a
Component: engine
in the filename and the command itself
Closes#12267
Signed-off-by: Doug Davis <dug@us.ibm.com>
Upstream-commit: 82daa43844556953101b201bc5983aed4fbe6233
Component: engine
Currently `docker inspect -f` use json.Unmarshal() unmarshal
to interface, it will store all JSON numbers in float64, so
we use `docker inspect 4f0d73b75a0d | grep Memory` and
`docker inspect -f {{.HostConfig.Memory}} 4f0d73b75a0d` will
get different values.
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
Upstream-commit: b0ef3194aaa8c22b674ed5301c59e8e557a7e85e
Component: engine
- Mount struct now called volumeMount
- Merged volume creation for each volume type (volumes-from, binds, normal
volumes) so this only happens in once place
- Simplified container copy of volumes (for when `docker cp` is a
volume)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: f3680e74946d3a773edb118ea3f508b8237e44a8
Component: engine
It prints test name and duration for each test.
Also performs deleteAllContainers after each test.
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
Upstream-commit: dc944ea7e48d11a2906e751d3e61daf08faee054
Component: engine
Changed method declaration. Fixed all calls to dockerCmd
method to reflect the change.
resolves#12355
Signed-off-by: bobby abbott <ttobbaybbob@gmail.com>
Upstream-commit: 621b601b3c602aab5ef0f07903fdf413881bb261
Component: engine
Testcase TestBuildResourceConstraintsAreUsed run build without
--no-cache, so if you run this test twice, it will fail the
second time.
TESTFLAGS='-v -run ^TestBuildResourceConstraintsAreUsed$' ./hack/make.sh binary test-integration-cli
[PASSED]
TESTFLAGS='-v -run ^TestBuildResourceConstraintsAreUsed$' ./hack/make.sh binary test-integration-cli
[FAIL]
Because we'll use cID to inspect field and will get empty cID
if we have cache.
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
Upstream-commit: c49cc1f2fbd3d0256455750c885eadcaa3d17937
Component: engine
This patch include the following fixs:
- fix image name error when docker ps
- fix docker events test failure: use the exact image name for filter
- fix docker build CI test failure due to "docker events" change
Because of change of daemon log behavior. Now we record
the exact Image name as you typed. So docker run -d busybux sh
and docker run -d busybox:latest are not the same in the log.
So it will affect the docker events. So change the related CI
Signed-off-by: Liu Hua <sdu.liu@huawei.com>
Upstream-commit: 645c020f5ab7119cd06d7c6a790e9d99fe1cd309
Component: engine
This was just an alias to `strings.TrimSpace`
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 475c65319b4663d630711519e18d0b134c42c7f1
Component: engine
Just ${xxx:+...} and ${xxx:-...} for now
Signed-off-by: Doug Davis <dug@us.ibm.com>
Upstream-commit: 39908fc6d9a13f0d44196475ba938b6cc352fdbe
Component: engine
My AR couldn't take it any more:
- one logDone per test
- PASSED lines don't wrap
Signed-off-by: Doug Davis <dug@us.ibm.com>
Upstream-commit: 9a9d23dbc40c9b650bd1f9f98a421b7ff2312f36
Component: engine
Set cpuset to "0" so that it works on single core machines.
W/o this (and set to "1") we'll see something like this error
when running:
System error: write /cgroup/cpuset/docker/66689499bbd08cd8dccc9b7bfd1d6b34e85d73ce8c84d3c69b5e91944322da60/docker/79d7c548b58c85c4cfad6cd01eb7c3b30db254d1014c496137edd93ddc528a6f/cpuset.cpus: invalid argument"
Signed-off-by: Doug Davis <dug@us.ibm.com>
Upstream-commit: 389d0ae45327926a9d20b2a3e6c15499f3c9c570
Component: engine
The validation script from #10681 is too pedantic, and does not handle
well situations like:
```
cat <<EOF # or <<-EOF
Whether the leading whitespace is stripped out or not by bash
it should still be considered as valid.
EOF
```
This reverts commit 4e65c1c319afffc325853b88c9aef0c42ec83482.
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 0252ad0adc37a34b88fa908ae74a13b940febdcb
Component: engine
and makes the testing output ugly.
This hides the output since it not used.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Upstream-commit: fbd47969a86793c4e87e2da1abea1f806ba0a526
Component: engine
