Commit Graph

643 Commits

Author SHA1 Message Date
6044c2ae89 Merge pull request #32792 from rogaha/extra_prometheus_metrics
Add extra prometheus metrics
Upstream-commit: 1a6f8a92b2c5912058828d560f52281ba0424472
Component: engine
2017-05-09 21:48:34 +02:00
a7451e12cd Add extra prometheus metrics
- buildsTriggered
 - buildsFailed
    - valid options:
        metricsDockerfileSyntaxError,
        metricsDockerfileEmptyError,
        metricsCommandNotSupportedError,
        metricsErrorProcessingCommandsError,
        metricsBuildTargetNotReachableError,
        metricsMissingOnbuildArgumentsError,
        metricsUnknownInstructionError,
        metricsBuildCanceled,
- engineInfo

Signed-off-by: Roberto Gandolfo Hashioka <roberto_hashioka@hotmail.com>
Upstream-commit: a28b173a780cd06db6d93197c54b00a7d616b3dc
Component: engine
2017-05-09 01:04:40 -07:00
bd17f42bef Add docker build --iidfile=FILE
This is synonymous with `docker run --cidfile=FILE` and writes the digest of
the newly built image to the named file. This is intended to be used by build
systems which want to avoid tagging (perhaps because they are in CI or
otherwise want to avoid fixed names which can clash) by enabling e.g. Makefile
constructs like:

    image.id: Dockerfile
    	docker build --iidfile=image.id .

    do-some-more-stuff: image.id
    	do-stuff-with <image.id

Currently the only way to achieve this is to use `docker build -q` and capture
the stdout, but at the expense of losing the build output.

In non-silent mode (without `-q`) with API >= v1.29 the caller will now see a
`JSONMessage` with the `Aux` field containing a `types.BuildResult` in the
output stream for each image/layer produced during the build, with the final
one being the end product.  Having all of the intermediate images might be
interesting in some cases.

In silent mode (with `-q`) there is no change, on success the only output will
be the resulting image digest as it was previosuly.

There was no wrapper to just output an Aux section without enclosing it in a
Progress, so add one here.

Added some tests to integration cli tests.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Upstream-commit: 5894bc1abf8186802d360d20739b57bfffed51df
Component: engine
2017-05-05 16:35:54 +01:00
5ce7dab3ff Merge pull request #33033 from YuPengZTE/devReadAll
check err
Upstream-commit: b61ffbfb527cf9d9998b8d2c514749a51e7640ae
Component: engine
2017-05-05 22:33:23 +09:00
a7c561e6df check err
Signed-off-by: yupengzte <yu.peng36@zte.com.cn>
Upstream-commit: 0d99cb862997d9421d5c5c4dc96983067ddc4d84
Component: engine
2017-05-05 13:33:40 +08:00
965d054208 Some refactoring of dispatch()
Remove runConfig from Builder and dispatchRequest. It is not only on
dispatchState.

Move dispatch state fields from Builder to dispatchState

Move stageName tracking to dispatchRequest.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 2f0ebba0e7f43908a41d5ff9bbb1b9b78743a6ab
Component: engine
2017-05-04 17:11:08 -04:00
9494cbba92 Merge pull request #32749 from dnephin/rm-ContainerUpdateCmdOnBuild
[Builder] Remove ContainerUpdateCmdOnBuild
Upstream-commit: 37be26382614d833e4855b4fc1eb89a6817d8426
Component: engine
2017-05-03 14:02:08 -04:00
0046c26e1e Merge pull request #32949 from dnephin/refactor-streamformatter-and-progress
Refacator pkg/streamformatter
Upstream-commit: 08d7fad45dce832dc03934e9f4fc904899220c96
Component: engine
2017-05-03 09:05:26 -07:00
d3da06632a Fix run with entrypoint in base image
Update a test to use a base image with entrypoint to that the linux build
has at least one test that behaves like all the windows tests.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: d9371ee80764d0eecf7b8a562121f0a6234167a3
Component: engine
2017-05-03 10:59:38 -04:00
8e846811d4 Remove ContainerUpdateCmdOnBuild, it does nothing.
Set a blank entrypoint to preserve the old behaviour.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 97f860716497f6f75236b72e4af90e01afad832c
Component: engine
2017-05-03 10:59:38 -04:00
4886c78246 Fix setting b.runConfig.Image at arbitrary places.
Previously this value was set at some point attrbitrarily between when it was updated and when it was going to be used next.

Instead always set it as the last step of dispatch.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 3dcab289821ddd4575b7e48d463ba8ef2af492ea
Component: engine
2017-05-03 10:59:37 -04:00
755f9321e1 Refacator pkg/streamformatter
StreamFormatter suffered was two distinct structs mixed into a single struct
without any overlap.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: c87d67b0ad788a6a80d1af89488e2d1f22726c34
Component: engine
2017-05-02 17:38:12 -04:00
dbcc3f98b6 Cleanup all the mutate + defer revert of b.runConfig in the builder
Instead of mutating and reverting, just create a copy and pass the copy
around.

Add a unit test for builder dispatcher.run

Fix two test failures

Fix image history by adding a CreatedBy to commit options. Previously the
createdBy field was being created by modifying a reference to the runConfig that
was held from when the container was created.

Fix a test that expected a trailing slash. Previously the runConfig was being
modified by container create. Now that we're creating a copy of runConfig
instead of sharing a reference the runConfig retains the trailing slash.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 9f738cc574d50d0a2accdf6f6deb30405c24a80c
Component: engine
2017-05-01 18:36:53 -04:00
3d193f829d Extract squash and tagging from the Dockerfile builder.
Remove pathCache and replace it with syncmap
Cleanup NewBuilder
Create an api/server/backend/build
Extract BuildTagger

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 0296797f0f39477d675128c93c1646b3186937ee
Component: engine
2017-05-01 12:07:32 -04:00
5d06c101d8 Fix cache miss when builtin build args are used.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 721e1736ae31914902d2c2196aed5f92e1a0982d
Component: engine
2017-04-28 12:49:50 -04:00
ad91de1337 Merge pull request #32772 from dnephin/refactor-builder-split-commit
[Builder] Refactor `Builder.commit()`
Upstream-commit: ec67cea96e0fd90a7a266e1adac9fe1867c8ae13
Component: engine
2017-04-28 11:18:55 -04:00
9f6af10310 Merge pull request #32763 from dave-tucker/fix-32744
builder: Make builtin arg pruning work with > 1 arg
Upstream-commit: 9f0ebea33579b1c6cec8ae445ac3b21655aa3bcd
Component: engine
2017-04-28 10:24:13 -04:00
b5c88cd04e Merge pull request #32504 from dongluochen/healthcheck_duration
do not allow duration less than 1 ms in healthcheck parameters
Upstream-commit: a7519152d9be7a0dd6941d529ea4b83cf4b7f1d4
Component: engine
2017-04-27 23:54:00 -04:00
f6bbbde171 Extract commitContainer from commit() to separate the distinct branches
Remove unused arguments to commit.
This will allow us to remove all the runConfig mutate+revert code that
is scattered around builder dispatchers/internals

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 0d9e66b98ac9237b561d9beee5ecfe6a133ffc8f
Component: engine
2017-04-27 20:49:38 -04:00
ce59d98dee Merge pull request #32858 from dnephin/builder-shell-words-interface
[Builder] Remove b.escapeToken, create ShellLex
Upstream-commit: aee2da3bdf6665939722dacf3b0c149499c35d90
Component: engine
2017-04-27 13:16:40 -04:00
3b753edcdd Merge pull request #31984 from tonistiigi/remote-context
builder: Refactor remote context
Upstream-commit: ae0f8c7ba16362e869894e0121cf3052069d7fe4
Component: engine
2017-04-27 11:50:18 -04:00
e25a81277d Create a new ShellLex struct which provides the expected interface to Builder.
Remove b.escapeToken from builder

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 2414166e1e38a4de8aa4af40b90c7e1bc90102b0
Component: engine
2017-04-26 19:39:16 -04:00
0411944839 Merge pull request #31856 from jim-minter/more_races
Resolve a set of race conditions in logging and attaching code
Upstream-commit: 5eca7f7c5d7678f6548cd7d90199d67ce98d4c98
Component: engine
2017-04-26 16:04:02 -07:00
619e402989 builder: Make builtin arg pruning work with > 1 arg
The previous implementation would error out with "Unexpected EOF" which
was caused by an underlying "array index out-of-bounds" error.
The root cause was deleting items from the same array that was being
iterated over. The iteration was unaware that the array size had
changed, resulting in an error.

The new implementation builds a new array instead of mutating a copy of
the old one.

Fixes: #32744

Signed-off-by: Dave Tucker <dt@docker.com>
Upstream-commit: 1076ab58ec92637a1a157d8dfbb7625d7bba6c40
Component: engine
2017-04-26 10:24:58 +01:00
aadacb3d07 Refactor remote context parsing
Redefine a better interface for remote context dependency.

Separate Dockerfile build instruction from remote context.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: d1faf3df27207af111daf4bd0191478c374dbc55
Component: engine
2017-04-25 14:35:31 -07:00
6c0f9bd02c Reduce the number of calls to b.image
In preparation for removing it from builder.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 52626bb9199ca3562fa70f14d8fc7b8c12c8b0f5
Component: engine
2017-04-24 14:21:19 -04:00
7e9b6e3ca4 Change builder.dispatchers to use a dispatchRequest object
This change starts the process of splitting up the Builder into logical
components. Remove builder.flags and move it to the new dispatchRequest
object.

Use runConfig from dispatchRequest instead of from the builder.

More progress removing things from the Builder struct.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: e59327aaac338bd350746e0e6479489f21a0a1fa
Component: engine
2017-04-24 14:21:19 -04:00
57d7fd25ff Merge pull request #32500 from dnephin/refactor-from-dispatcher
Refactor from dispatcher
Upstream-commit: 3beb8d9c8bc77762abceea52a742c1221636ea44
Component: engine
2017-04-24 10:10:42 -07:00
3df8030542 Fix race on ContainerAttachRaw
Signed-off-by: Jim Minter <jminter@redhat.com>
Upstream-commit: 32ca1214fa55f55c54a54061ecf752b75f2c72c3
Component: engine
2017-04-20 12:27:50 +01:00
142883d5b3 Remove pkg/testutil/assert in favor of testify
I noticed that we're using a homegrown package for assertions. The
functions are extremely similar to testify, but with enough slight
differences to be confusing (for example, Equal takes its arguments in a
different order). We already vendor testify, and it's used in a few
places by tests.

I also found some problems with pkg/testutil/assert. For example, the
NotNil function seems to be broken. It checks the argument against
"nil", which only works for an interface. If you pass in a nil map or
slice, the equality check will fail.

In the interest of avoiding NIH, I'm proposing replacing
pkg/testutil/assert with testify. The test code looks almost the same,
but we avoid the confusion of having two similar but slightly different
assertion packages, and having to maintain our own package instead of
using a commonly-used one.

In the process, I found a few places where the tests should halt if an
assertion fails, so I've made those cases (that I noticed) use "require"
instead of "assert", and I've vendored the "require" package from
testify alongside the already-present "assert" package.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 6052f2b3969feadb01662d8e2f30337d9c7f61af
Component: engine
2017-04-14 12:03:21 -07:00
834babfd94 Merge pull request #32601 from dnephin/builder-shell-words-buffer
Use a bytes.Buffer for shell_words string concat
Upstream-commit: 41f4c3cf7e270086bb2f6a6bd4622d9aeb8f741f
Component: engine
2017-04-13 23:22:18 +02:00
aa3095ef47 remove increment flag on imageContexts.new()
Rename the methods to match my understanding of the behaviour.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: aafd7fa96909f48e43fd1b785c3c3a1f8b656ede
Component: engine
2017-04-13 14:56:57 -04:00
b84c939d61 Extract two functions from the FROM dispatcher
This helps clarify the difference cases for parsing ctxName, and for
getting an image.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 4d62f67117ca515671fcc882807bfc4862067ea6
Component: engine
2017-04-13 14:56:57 -04:00
80987246bf Merge pull request #32580 from dnephin/refactor-builder-parser-directive
Refactor Dockerfile.parser and directive
Upstream-commit: 700b4807c3a120181973c675a8f69192d961e13f
Component: engine
2017-04-13 20:47:42 +02:00
964c3672d4 Use a bytes.Buffer for shell_words string concat
It's much faster

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 0055a48277b4dcec644ddfbf94d8b96c58d5e03e
Component: engine
2017-04-13 14:39:34 -04:00
0b7785856b Merge pull request #32328 from duglin/fixEnvVars
Minor tweaks to quotes in env vars
Upstream-commit: df3c4254072d110ecfcefeda94279c25e6a9f334
Component: engine
2017-04-13 10:41:28 -04:00
2f0c087a4c set 1ms as container duration minimum value
Signed-off-by: Dong Chen <dongluo.chen@docker.com>
Upstream-commit: d8b6a35d0272d9bb121dda7d0bc53d0e53d8bd22
Component: engine
2017-04-12 15:45:29 -07:00
78d1a8154c Refactor dockerfile parser
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 4cbc953a5d80c850df7107b28e743e933bbeb1d3
Component: engine
2017-04-12 18:31:52 -04:00
96891f1f0d Cleanup processing of directives.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 9c53fa2d0c1796d32aaba2193e7d802cd9b66763
Component: engine
2017-04-12 15:42:35 -04:00
e1a50e229d Minor tweaks to quotes in env vars
Addresses part of #32140, in particular:
- this will make it so that double backslashes in double-quoted
strings will result in a single backslash. While in single quotes it remains
a double backslash.
- missing closing " and ' will now generate an error

Signed-off-by: Doug Davis <dug@us.ibm.com>
Upstream-commit: 2fb7c3c4f0b2fe15e279729a87c465df4fa70879
Component: engine
2017-04-12 12:20:14 -07:00
7e99490a6a Keep parser.Directive internal to parser
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 64c4c1c3d5e0fe364d83db5a8dc99a24fd121754
Component: engine
2017-04-12 14:48:09 -04:00
bbeb6127dc Hide builder.parser.Directive internals
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: bb429da9a977a9d89121052fe1143528294b8a0c
Component: engine
2017-04-12 14:48:09 -04:00
203ab34184 Merge pull request #32532 from dnephin/refactor-some-builder-parts
[builder] cleanup env dispatcher and Builder.build()
Upstream-commit: d2ab40e5b99bc2ab0abfea146544a20ce6081942
Component: engine
2017-04-12 11:11:06 -07:00
cdb97bb321 Merge pull request #32370 from AkihiroSuda/improve-build-error
builder/dockerfile: improve error message about build stage name
Upstream-commit: 3c072598820e8b435151289ee0d8b74cb7e047ba
Component: engine
2017-04-12 16:33:23 +02:00
5aac0fb7d6 builder/dockerfile: improve error message about build stage name
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Upstream-commit: 50a9be4bf5b8a57acc455eaf7c3842b2df046f71
Component: engine
2017-04-12 05:27:39 +00:00
862dffcf24 Remove unused Builder.Cancel()
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 068f344e032ad4489a88665adec683e06ad6f3c7
Component: engine
2017-04-11 20:33:13 -04:00
a00fca972d Remove unused id field from Builder.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: a6abd57b83dc0aaf0cedeeb488c8a41262e46b7d
Component: engine
2017-04-11 14:44:32 -04:00
a5b8a0845e Factor out functions from builder/dockerfile/builder.go:Builder.build()
Remove the block comment which is stale, and redundant now that the
function is just as readable as the comment.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: bfcd95817afaedb078022fc2f335ead64afee55c
Component: engine
2017-04-11 14:44:32 -04:00
6fb208126e Cleanup in dispatcher.env
Remove commented code blocks
Remove some duplication in comparing and restructuring env

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: c7fad9b750f8f143a22cc5a85a1dc26573025414
Component: engine
2017-04-11 14:44:28 -04:00
443e2161ff Merge pull request #32495 from dnephin/cleanup-builder-dispatcher-env
Remove Builder.dockerfile field
Upstream-commit: b764c471bf1a648342a4b4f58de1e9768b49b4ed
Component: engine
2017-04-11 14:50:55 +09:00