Commit Graph

574 Commits

Author SHA1 Message Date
b660bf165d Unify the frozen images to the multi-arch version
Update and unify the `busybox` images on all arches to the `glibc` multi-arch
version and remove the temp workaround on amd64 which uses the old version
busybox (v1.26) before this PR to bypass the failure of those network related
test cases. Also, this PR will fix all the network related issues with `glibc`
version `busybox` image.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 3a971009763387856bb7f162accdf6714100e39b
Component: engine
2018-02-14 03:59:04 +00:00
2802ca3c0e Merge pull request #36018 from thaJeztah/remove-deprecated-testenv-functions
Remove deprecated integration-cli/environment functions
Upstream-commit: 8c31aacebff68fe73f4203f86c5681a0fce648f1
Component: engine
2018-01-15 13:24:52 -08:00
d1c8c580ed Remove deprecated environment.DaemonPlatform()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 18a771a761654e241ae8d1e85aa0c0a6164c5d27
Component: engine
2018-01-15 15:32:06 +01:00
28a714b165 Remove deprecated environment.MinimalBaseImage()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: dfedc9ef6214500526c29fbf9a46aca9cecbeb57
Component: engine
2018-01-15 15:31:02 +01:00
de027de052 Remove deprecated environment.DockerBasePath()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 142b1f8bfb3a1459660a5877b48f2bd7d17ba5d6
Component: engine
2018-01-15 15:29:26 +01:00
9251d7a17a busybox: Fix ls compatible issue
BusyBox v1.26.2 (2017-03-09 00:04:38 UTC) supports `-le` option
to get the full date and time information, while BusyBox v1.27.2
(2017-11-01 23:22:25 UTC, which is used by the official multi-arch
image, uses `--full-time` instead of `-e` to get the same data. As
a result, we will get below error for the `DockerSuite.TestBuildLastModified`
test case in case of multi-arch image used:

> docker_cli_build_test.go:446:
>     out2 = cli.DockerCmd(c, "run", name, "ls", "-le", "/file").Combined()
>   o/src/github.com/docker/docker/vendor/github.com/gotestyourself/gotestyourself/icmd/command.go:61:
>     t.Fatalf("at %s:%d - %s\n", filepath.Base(file), line, err.Error())
> ... Error: at cli.go:33 -
> Command:  /usr/local/bin/docker run testbuildlastmodified ls -le /file
> ExitCode: 1
> Error:    exit status 1
> Stdout:
> Stderr:   ls: invalid option -- e
> BusyBox v1.27.2 (2017-11-01 23:22:25 UTC) multi-call binary.

This PR tries to fix the above compatible issue for busybox image.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: f88c2c04ef4dbae20eaddeaf69e605878f498041
Component: engine
2018-01-11 06:34:54 +00:00
2a34102708 TestBuildNotVerboseFailureRemote: fix false positive
I got the following test failure on power:

10:00:56
10:00:56
----------------------------------------------------------------------
10:00:56 FAIL: docker_cli_build_test.go:3521:
DockerSuite.TestBuildNotVerboseFailureRemote
10:00:56
10:00:56 docker_cli_build_test.go:3536:
10:00:56     c.Fatal(fmt.Errorf("Test[%s] expected that quiet stderr and
verbose stdout are equal; quiet [%v], verbose [%v]", name,
quietResult.Stderr(), result.Combined()))
10:00:56 ... Error: Test[quiet_build_wrong_remote] expected that quiet
stderr and verbose stdout are equal; quiet [
10:00:56 unable to prepare context: unable to download remote context
http://something.invalid: Get http://something.invalid: dial tcp: lookup
something.invalid on 172.29.128.11:53: no such host
10:00:56 ], verbose [unable to prepare context: unable to download
remote context http://something.invalid: Get http://something.invalid:
dial tcp: lookup something.invalid on 8.8.8.8:53: no such host
10:00:56 ]
10:00:56
10:00:56
10:00:56
----------------------------------------------------------------------

The reason is, either more than one name server is configured, or
nameserver was reconfigured in the middle of the test run. In any case,
different nameserver IP in an error messages should not be treated
as a failure, so let's strip those out.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 3676bd8569f4df28a4f850cd4814e3558d8c03f6
Component: engine
2017-12-28 13:08:41 -08:00
d109b596b8 integration-cli: clarify multi-stage tests names
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: e90b7c06b4b08ca2935359667cfac4f05e33819b
Component: engine
2017-12-20 11:55:27 -08:00
be3cbac37e Separate the GenerateRandomAlphaOnlyString function from stringutils
Signed-off-by: chaowang <chaowang@localhost.localdomain>
Upstream-commit: 7c35a2418265336a572976e2ced378ef4b6f1666
Component: engine
2017-10-28 09:03:02 +08:00
e01e198f54 Introduce a typed command system and 2 phase parse/dispatch build
This is a work base to introduce more features like build time
dockerfile optimisations, dependency analysis and parallel build, as
well as a first step to go from a dispatch-inline process to a
frontend+backend process.

Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
Upstream-commit: 669c0677980b04bcbf871bb7c2d9f07caccfd42b
Component: engine
2017-09-18 09:48:56 +02:00
e8bff97a66 Update tests to use icmd
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 92427b3a8146e048c4b85e5ece1530da97d8472d
Component: engine
2017-08-25 12:07:30 -04:00
e13afe9c59 Remove RandomTmpDirPath
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 9db68f4dea6379b7e85ee5c6f2be080a6fcbe4ca
Component: engine
2017-08-22 17:15:26 -04:00
3177188cce Remove testutil.ParseCgroupPaths
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: e8bd67181533222c192cca19a4279c85db48875e
Component: engine
2017-08-22 17:15:26 -04:00
524d0fa645 Remove testutil.CompareDirectoryEntries and IsKilled
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 6151d55ee7301d2b3800d701a9ce4969848e4d6b
Component: engine
2017-08-22 17:15:26 -04:00
643654c2f0 Spelling fixes
* additional
* ambiguous
* anonymous
* anything
* application
* because
* before
* building
* capabilities
* circumstances
* commit
* committer
* compresses
* concatenated
* config
* container
* container's
* current
* definition
* delimiter
* disassociates
* discovery
* distributed
* doesnotexist
* downloads
* duplicates
* either
* enhancing
* enumerate
* escapable
* exactly
* expect
* expectations
* expected
* explicitly
* false
* filesystem
* following
* forbidden
* git with
* healthcheck
* ignore
* independent
* inheritance
* investigating
* irrelevant
* it
* logging
* looking
* membership
* mimic
* minimum
* modify
* mountpoint
* multiline
* notifier
* outputting
* outside
* overridden
* override
* parsable
* plugins
* precedence
* propagation
* provided
* provides
* registries
* repositories
* returning
* settings
* should
* signals
* someone
* something
* specifically
* successfully
* synchronize
* they've
* thinking
* uninitialized
* unintentionally
* unmarshaling
* unnamed
* unreferenced
* verify

Signed-off-by: Josh Soref <jsoref@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 39bcaee47b8a284a46b761afe218ba7deda0d482
Component: engine
2017-07-03 13:13:09 -07:00
1c2b78478c builder: Emit a BuildResult after squashing.
This ensures that the final BuildResult is the actual tagged image.

Fixes #33822.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Upstream-commit: 9777ec3be0e3056d0bedcf36052704f832e45759
Component: engine
2017-06-26 13:19:25 +01:00
13d2adae3f Fix copy when used with scratch and images with empty RootFS
Commit the rwLayer to get the correct DiffID
Refacator copy in thebuilder
move more code into exportImage
cleanup some windows tests
Release the newly commited layer.
Set the imageID on the buildStage after exporting a new image.
Move archiver to BuildManager.
Have ReleaseableLayer.Commit return a layer
and store the Image from exportImage in the local imageSources cache
Remove NewChild from image interface.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 51360965206b0db49cc0365dabb590063a17a9df
Component: engine
2017-06-08 15:07:16 -04:00
033ef466ec Fix resetting image metadata between stages for scratch case
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: b192218503427a49b6de0a093e38e2697a22427a
Component: engine
2017-05-12 12:00:55 -07:00
01b4a22798 Refactor interaction between dispatcher.from and dispatchState
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: ab3a037a5b77220d0524ce2b17105e1daae39425
Component: engine
2017-05-10 17:58:57 -04:00
d0052ddc0d fix expected output in tests
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: f47a61c42957c0a98944a0723b3807aa4d07ed8e
Component: engine
2017-05-05 12:14:30 -07:00
bd17f42bef Add docker build --iidfile=FILE
This is synonymous with `docker run --cidfile=FILE` and writes the digest of
the newly built image to the named file. This is intended to be used by build
systems which want to avoid tagging (perhaps because they are in CI or
otherwise want to avoid fixed names which can clash) by enabling e.g. Makefile
constructs like:

    image.id: Dockerfile
    	docker build --iidfile=image.id .

    do-some-more-stuff: image.id
    	do-stuff-with <image.id

Currently the only way to achieve this is to use `docker build -q` and capture
the stdout, but at the expense of losing the build output.

In non-silent mode (without `-q`) with API >= v1.29 the caller will now see a
`JSONMessage` with the `Aux` field containing a `types.BuildResult` in the
output stream for each image/layer produced during the build, with the final
one being the end product.  Having all of the intermediate images might be
interesting in some cases.

In silent mode (with `-q`) there is no change, on success the only output will
be the resulting image digest as it was previosuly.

There was no wrapper to just output an Aux section without enclosing it in a
Progress, so add one here.

Added some tests to integration cli tests.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Upstream-commit: 5894bc1abf8186802d360d20739b57bfffed51df
Component: engine
2017-05-05 16:35:54 +01:00
d3da06632a Fix run with entrypoint in base image
Update a test to use a base image with entrypoint to that the linux build
has at least one test that behaves like all the windows tests.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: d9371ee80764d0eecf7b8a562121f0a6234167a3
Component: engine
2017-05-03 10:59:38 -04:00
8e846811d4 Remove ContainerUpdateCmdOnBuild, it does nothing.
Set a blank entrypoint to preserve the old behaviour.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 97f860716497f6f75236b72e4af90e01afad832c
Component: engine
2017-05-03 10:59:38 -04:00
dbcc3f98b6 Cleanup all the mutate + defer revert of b.runConfig in the builder
Instead of mutating and reverting, just create a copy and pass the copy
around.

Add a unit test for builder dispatcher.run

Fix two test failures

Fix image history by adding a CreatedBy to commit options. Previously the
createdBy field was being created by modifying a reference to the runConfig that
was held from when the container was created.

Fix a test that expected a trailing slash. Previously the runConfig was being
modified by container create. Now that we're creating a copy of runConfig
instead of sharing a reference the runConfig retains the trailing slash.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 9f738cc574d50d0a2accdf6f6deb30405c24a80c
Component: engine
2017-05-01 18:36:53 -04:00
5d06c101d8 Fix cache miss when builtin build args are used.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 721e1736ae31914902d2c2196aed5f92e1a0982d
Component: engine
2017-04-28 12:49:50 -04:00
619e402989 builder: Make builtin arg pruning work with > 1 arg
The previous implementation would error out with "Unexpected EOF" which
was caused by an underlying "array index out-of-bounds" error.
The root cause was deleting items from the same array that was being
iterated over. The iteration was unaware that the array size had
changed, resulting in an error.

The new implementation builds a new array instead of mutating a copy of
the old one.

Fixes: #32744

Signed-off-by: Dave Tucker <dt@docker.com>
Upstream-commit: 1076ab58ec92637a1a157d8dfbb7625d7bba6c40
Component: engine
2017-04-26 10:24:58 +01:00
7e9b6e3ca4 Change builder.dispatchers to use a dispatchRequest object
This change starts the process of splitting up the Builder into logical
components. Remove builder.flags and move it to the new dispatchRequest
object.

Use runConfig from dispatchRequest instead of from the builder.

More progress removing things from the Builder struct.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: e59327aaac338bd350746e0e6479489f21a0a1fa
Component: engine
2017-04-24 14:21:19 -04:00
87a3307749 Merge pull request #32740 from vdemeester/integration-trusted-cmd-cli
Use `cli` for trusted related command
Upstream-commit: 83ee902ecc3790c33c1e2d87334074436056bb49
Component: engine
2017-04-22 14:51:12 -07:00
f5ac07a8d7 [integration] Move fakegit to its own package in cli
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: a582d9dc42abd96bd88197939e082a1ea3c938a7
Component: engine
2017-04-19 15:52:03 +02:00
987b06e286 Use cli for trusted relate command
This also removed some skipped test (that are skipped for a long while).

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: b0ba39d4312ff20b11fc777092cc0c3e5ddeb19f
Component: engine
2017-04-19 14:04:39 +02:00
0b7785856b Merge pull request #32328 from duglin/fixEnvVars
Minor tweaks to quotes in env vars
Upstream-commit: df3c4254072d110ecfcefeda94279c25e6a9f334
Component: engine
2017-04-13 10:41:28 -04:00
e1a50e229d Minor tweaks to quotes in env vars
Addresses part of #32140, in particular:
- this will make it so that double backslashes in double-quoted
strings will result in a single backslash. While in single quotes it remains
a double backslash.
- missing closing " and ' will now generate an error

Signed-off-by: Doug Davis <dug@us.ibm.com>
Upstream-commit: 2fb7c3c4f0b2fe15e279729a87c465df4fa70879
Component: engine
2017-04-12 12:20:14 -07:00
203ab34184 Merge pull request #32532 from dnephin/refactor-some-builder-parts
[builder] cleanup env dispatcher and Builder.build()
Upstream-commit: d2ab40e5b99bc2ab0abfea146544a20ce6081942
Component: engine
2017-04-12 11:11:06 -07:00
4de7d79620 Merge pull request #29807 from thaJeztah/update-frozen-images
Update frozen images in Dockerfiles
Upstream-commit: 330ddf8eb8f8a551677bdfc757fcefdb0cdaec3f
Component: engine
2017-04-12 15:37:40 +02:00
6fb208126e Cleanup in dispatcher.env
Remove commented code blocks
Remove some duplication in comparing and restructuring env

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: c7fad9b750f8f143a22cc5a85a1dc26573025414
Component: engine
2017-04-11 14:44:28 -04:00
5f5b71f0ea Move FakeContext to integration-cli/cli/build/context package…
… and continue emptying `docker_utils_test.go` from build related function.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 56fb4653e8b867802d2a33121f933d164a4e6325
Component: engine
2017-04-11 18:25:49 +02:00
a89a2c21c5 builder: add an option for specifying build target
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: 33e07f41adc0ec96edad5f22bc2cfaec0d26ea58
Component: engine
2017-04-10 16:21:43 -07:00
2160e9bb7d Update frozen images in Dockerfiles
This updates the versions of the frozen images used to their current
version. The original reason for updating these images was to make sure
they are not affected by [CVE-2016-1252 / DSA-3733-1](https://lwn.net/Articles/709119/),
which is fixed in apt 1.0.9.8.4 and up.

Note that `CVE-2016-1252` won't affect our test-suite, because no packages
are installed during out tests. It is just "good practice" to keep these
images up to date.

The `debian:jessie`, and `buildpack-deps:jessie` in  `Dockerfile.s390x`,
and `Dockerfile.armhf` have not been updated in this patch, because
those images have not yet been updated to contain apt 1.0.9.8.4.

While working on this, the `busybox` and `hello-world` were also updated
to their latest version.

Also removes a reference to `hack/make/.ensure-frozen-images`, which
was removed in ff91276d1f5beab5582d9ca582ee01af13198333.

The new busybox image has one layer less than the original,
so updated `TestBuildSquashParent` to take that into account.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1ecd8ed5186a161c1c43ae7c99b1c81b97188619
Component: engine
2017-04-10 22:05:39 +02:00
15fc803536 Merge pull request #31236 from tonistiigi/docker-stdin
build: accept -f - to read Dockerfile from stdin
Upstream-commit: 778e32a2fadcf1ccb1819ad6534af5c477585300
Component: engine
2017-04-10 20:14:54 +02:00
43f24a2541 Merge pull request #31352 from dnephin/allow-arg-in-fromt
[dockerfile] Allow ARG in FROM
Upstream-commit: 4d9e32a08e8e2613601cbc5d36bd8c7ecf32a772
Component: engine
2017-04-10 13:23:40 -04:00
8c6400b4e4 Refactor BuildArgs
Add MetaArgs for ARG that occur before the first FROM
Integration test for these cases.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 239c53bf836174108dbae445a394a290f5fe2898
Component: engine
2017-04-07 17:57:47 -04:00
4c321e8e20 Test and fix forbidden path for COPY --from on Windows
Paths resolving to c:\ or c:\windows are forbidden

Replaced the obscure (and non-working) regex with a simple case
insensitive comparison to the black listed paths (we should forbid c:\,
c:\windows but not d:\)

Also, add a test ensuring paths are case insensitive on windows

Also, made sure existing multi-staged build tests pass on windows

Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
Upstream-commit: b0e75888738f6e427b545922a195837c00158749
Component: engine
2017-04-07 18:00:36 +02:00
4adfcf1302 Upadte archive.ReplaceFileTarWrapper() to not expect a sorted archive
Improve test coverage of ReplaceFileTarWrapper()

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 8cd6c30a489f7a0210526b0f94469c525ba8e0ee
Component: engine
2017-04-06 13:36:34 -04:00
06ea8bf159 build: accept -f - to read Dockerfile from stdin
Heavily based on implementation by David Sheets

Signed-off-by: David Sheets <sheets@alum.mit.edu>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: 3f6dc81e10b8b813fffaa9b4167a60c5a507fa38
Component: engine
2017-04-05 19:42:31 -04:00
077947c0d9 Add a multi stage trusted build testcase
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: 79ccb3478f2d1c541d8689a8b5c60ab51d95ca79
Component: engine
2017-04-04 09:56:00 -07:00
71418f1299 Add support for FROM using named block
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: 672ea30a94933eec228d71217b033da5d6572b8d
Component: engine
2017-04-04 09:34:28 -07:00
da4a3b24f4 Implicit copy-from with reference
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: 73b4b8ed7f86b8df70d7028c5ac7c2ba61ee0a8c
Component: engine
2017-04-04 09:34:28 -07:00
d51100a9ef Add named context support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: 87512bbc8490aca261133a7f3a3ea6518d703c34
Component: engine
2017-04-04 09:34:28 -07:00
b4a1c08afc builder: Fix setting command with custom shell
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: a1fa59ea123555a910c1de1f7b88b75000f2bec4
Component: engine
2017-03-30 13:29:21 -07:00
de10a198b8 [test-integration] Migrate some more tests to cli package
Add some required command operators to the `cli` package, and update
some tests to use this package, in order to remove a few functions
from `docker_utils_test.go`

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: eeaa6c96d83575da765dfc626c2c73d8a29dda32
Component: engine
2017-03-28 15:17:31 +02:00