Both names have no real sense, but one allows to make sure these packages
aren't used outside of `integration`.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: af306d149e76b100e08972cda364647bd7bcfe1e
Component: engine
We don't need the test image namespace anymore since we've already
upgrade those images to the latest multi-arch ones.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 662bdb4a5638e56d78e66bb2cb5d7a4a751135c9
Component: engine
The `.integration-test-helpers` script was sourced by `/etc/bash/bash.rc`.
However, the `$SCRIPTDIR` environment variable is set through `hack/make.sh`,
so will not be set when calling the `.integration-test-helpers` script directly.
Before this patch;
make BIND_DIR=. shell
...
bash: /make/.go-autogen: No such file or directory
After this patch, the warning is no longer printed
Also removed sourcing `.go-autogen` from test-integration and build-integration-test-binary,
as they already sourced `.integration-test-helpers` (which sources
`.go-autogen`).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 6d5e468db1f07c12140af0cf36ba0d1a09bb4ca9
Component: engine
Also skip.IfCondition directly from the test, so that the skip message is correct
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 104c1c68438c5c59ab0e7a5fb2da6bae4ce6d080
Component: engine
I strongly disagree with the design of this pull request.
Signed-off-by: David Sheets <dsheets@docker.com>
Upstream-commit: 1574d91463efd4741c2f6620718abdb624967584
Component: engine
Failures from the integration suite were not propagating to the outter shell
for some reason. Handle the failure with an if exit 1.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 96707bc600747257e82917ca079fa5006d636b2c
Component: engine
This adds a new package `integration` where `engine` integration tests
should live. Those integration tests should not depends on any `cli`
components (except from the `dockerd` daemon for now — to actually
start a daemon).
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 6b025a8b665f7b976bb2560488ba07f55dbf0fb6
Component: engine
Starting with this commit, integration tests should no longer rely on
the docker cli, they should be API tests instead. For the existing tests
the scripts will use a frozen version of the docker cli with a
DOCKER_API_VERSION frozen to 1.30, which should ensure that the CI remains
green at all times.
To help contributors develop and test manually with a modified docker
cli, this commit also adds a DOCKER_CLI_PATH environment variable to the
Makefile. This allows to set the path of a custom cli that will be
available inside the development container and used to run the
integration tests.
Signed-off-by: Arnaud Porterie (icecrime) <arnaud.porterie@docker.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 32915b1d0a315598edb737785d0357b5a1b8aa11
Component: engine
This commit adds contrib/integration-cli-on-swarm/integration-cli-on-swarm.sh,
which enables IT to be running in parallel, using Swarm-mode and Funker.
Please refer to contrib/integration-cli-on-swarm/README.md
The test takes almost 5 to 6 minutes, with 10 n1-standard-4 GCE instances.
$ ./contrib/integration-cli-on-swarm/integration-cli-on-swarm.sh --push-worker-image example.gcr.io/foo/bar --replicas 30 --shuffle
2016/12/29 08:32:15 Loaded 1618 tests (30 chunks)
2016/12/29 08:32:15 Executing 30 chunks in parallel, against "integration-cli-worker"
2016/12/29 08:32:15 Executing chunk 0 (contains 54 test filters)
..
2016/12/29 08:34:34 Finished chunk 28 [1/30] with 54 test filters in 2m19.098068003s, code=0.
2016/12/29 08:34:38 Finished chunk 12 [2/30] with 54 test filters in 2m23.088569511s, code=0.
2016/12/29 08:34:48 Finished chunk 10 [3/30] with 54 test filters in 2m33.880679079s, code=0.
2016/12/29 08:34:54 Finished chunk 20 [4/30] with 54 test filters in 2m39.973747028s, code=0.
2016/12/29 08:35:11 Finished chunk 18 [5/30] with 54 test filters in 2m56.28384361s, code=0.
2016/12/29 08:35:11 Finished chunk 29 [6/30] with 52 test filters in 2m56.54047088s, code=0.
2016/12/29 08:35:15 Finished chunk 1 [7/30] with 54 test filters in 3m0.285044426s, code=0.
2016/12/29 08:35:22 Finished chunk 6 [8/30] with 54 test filters in 3m7.211775338s, code=0.
2016/12/29 08:35:24 Finished chunk 25 [9/30] with 54 test filters in 3m9.938413009s, code=0.
2016/12/29 08:35:30 Finished chunk 27 [10/30] with 54 test filters in 3m15.219834368s, code=0.
2016/12/29 08:35:36 Finished chunk 9 [11/30] with 54 test filters in 3m21.615434162s, code=0.
2016/12/29 08:35:41 Finished chunk 13 [12/30] with 54 test filters in 3m26.576907401s, code=0.
2016/12/29 08:35:45 Finished chunk 17 [13/30] with 54 test filters in 3m30.290752537s, code=0.
2016/12/29 08:35:53 Finished chunk 2 [14/30] with 54 test filters in 3m38.148423321s, code=0.
2016/12/29 08:35:55 Finished chunk 24 [15/30] with 54 test filters in 3m40.09669137s, code=0.
2016/12/29 08:35:57 Finished chunk 8 [16/30] with 54 test filters in 3m42.299945108s, code=0.
2016/12/29 08:35:57 Finished chunk 22 [17/30] with 54 test filters in 3m42.946558809s, code=0.
2016/12/29 08:35:59 Finished chunk 23 [18/30] with 54 test filters in 3m44.232557165s, code=0.
2016/12/29 08:36:02 Finished chunk 3 [19/30] with 54 test filters in 3m47.112051358s, code=0.
2016/12/29 08:36:11 Finished chunk 15 [20/30] with 54 test filters in 3m56.340656645s, code=0.
2016/12/29 08:36:11 Finished chunk 11 [21/30] with 54 test filters in 3m56.882401231s, code=0.
2016/12/29 08:36:22 Finished chunk 19 [22/30] with 54 test filters in 4m7.551093516s, code=0.
2016/12/29 08:36:23 Finished chunk 21 [23/30] with 54 test filters in 4m8.221093446s, code=0.
2016/12/29 08:36:25 Finished chunk 16 [24/30] with 54 test filters in 4m10.450451705s, code=0.
2016/12/29 08:36:27 Finished chunk 5 [25/30] with 54 test filters in 4m12.162272692s, code=0.
2016/12/29 08:36:28 Finished chunk 14 [26/30] with 54 test filters in 4m13.977801031s, code=0.
2016/12/29 08:36:29 Finished chunk 0 [27/30] with 54 test filters in 4m14.34086812s, code=0.
2016/12/29 08:36:49 Finished chunk 26 [28/30] with 54 test filters in 4m34.437085539s, code=0.
2016/12/29 08:37:14 Finished chunk 7 [29/30] with 54 test filters in 4m59.22902721s, code=0.
2016/12/29 08:37:20 Finished chunk 4 [30/30] with 54 test filters in 5m5.103469214s, code=0.
2016/12/29 08:37:20 Executed 30 chunks in 5m5.104379119s. PASS: 30, FAIL: 0.
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Upstream-commit: 28797019b3f0eb288c4b201237cb992ec148dd4c
Component: engine
This is especially important for distributions like NixOS where `/bin/bash` doesn't exist, or for MacOS users who've installed a newer version of Bash than the one that comes with their OS.
Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
Upstream-commit: 52379fa76dee07ca038624d639d9e14f4fb719ff
Component: engine
Otherwise, while using test-integration-shell, it runs the tests using
the previously compiled test binary.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: aefbc034445df17c98f1247b100e2b6a2ed22e84
Component: engine
test.main was unexpectedly created under docker/integration-cli/bundles/VERSION/test-integration-cli directory.
This commit moves test.main to docker/bundles/VERSION/test-integration-cli.
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Upstream-commit: 4522f14f8cd0d51ced18c24e0d7929409b82a212
Component: engine
moves ensure-frozen-images to go
moves ensure-syscall-test to go
moves ensure-nnp-test to go
moves ensure-httpserver to go
Also makes some of the fixtures load only for the required tests.
This makes sure that fixtures that won't be needed for a test run such as
`make TESTFLAGS='-check.f Swarm' test-integration-cli` (for example)
aren't loaded... like the syscall tests.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: ff91276d1f5beab5582d9ca582ee01af13198333
Component: engine
Adds a new bundle `verify-integration-tests` which pre-compiles a test
binary for the integration tests.
This makes sure that the integration tests will actually compile before
doing other tasks which take much longer, such as building dockerd and
loading test fixtures.
When it comes time to actually run the tests, the pre-compiled binary
will be used so it doesn't have to compile the tests a 2nd time.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 4d0275c8192ca39d88f7dbe1e59ec03b544b6696
Component: engine
Interactive integration testing is useful when you're developing new tests, or
making changes to cli code.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 1d945a9743748373cd5c5bfface3209d241007d7
Component: engine
Vincent Demeester
Dennis Chen
Sebastiaan van Stijn
Daniel Nephin
David Sheets
Akihiro Suda
Kenfe-Mickael Laventure
Arnaud Porterie (icecrime)
Tianon Gravi
YuPengZTE
Brian Goff