Commit Graph

122 Commits

Author SHA1 Message Date
2f23577c7a vendor: use dockerfile parser from buildkit
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: c9ebd2f13b3aab49c2d90cc38dcf2cd0d059697f
Component: engine
2018-06-02 11:10:34 -07:00
0f96e98e12 Various code-cleanup
remove unnescessary import aliases, brackets, and so on.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f23c00d8701e4bd0f2372a586dacbf66a26f9a51
Component: engine
2018-05-23 17:50:54 +02:00
7273d67395 Remove unused 'label' related functions
Since we use `NewLabelCommand()` instead of `addNodesForLabelOption()`
to create the 'LABEL' commands from '--label' options, so all the related
functions should be removed.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: c7b543164daed58fbea36471592438b4e53ab748
Component: engine
2018-05-17 09:41:06 +08:00
0b32e0ce4f Add 'LABEL' command from '--label' to the last stage
This PR is tring to fix issue #36996.

Currently for multi-stage build, if `--target` specified, the `--label` option
will be ignored. The root cause is the last stage build will remove the `LABEL`
command(s) node created from the `--label` option. In order to address this issue,
we can create `LABEL` command(s) and add it/tem to the last stage.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 9c238ebd55e4105ad7f7edc04231ea61bb278ae8
Component: engine
2018-05-17 09:40:19 +08:00
d39da73d16 Export BuildArgs
Signed-off-by: Priya Wadhwa <priyawadhwa@google.com>
Upstream-commit: 593255ffb07989e416743d4235fef65e75414e6b
Component: engine
2018-05-08 14:57:26 -07:00
4c5e76b26e Fix the target name issue for multi-stage build
This PR is trying to fix issue #36956.

The stage name is case-insensitive by design, so we should use
`strings.EqualFold()` as the comparison method to eliminate the
case sensitive noise.

Also we need to return a pre-defined error code order to avoid below
message like:
"FIXME: Got an API for which error does not match any expected type!!!:
failed to reach build target dev in Dockerfile"

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 7c0570473cfa181aeb3278072cc9af4f9298cb98
Component: engine
2018-05-01 10:28:40 +08:00
e3af8e0774 Switch from x/net/context -> context
Since Go 1.7, context is a standard package. Since Go 1.9, everything
that is provided by "x/net/context" is a couple of type aliases to
types in "context".

Many vendored packages still use x/net/context, so vendor entry remains
for now.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 7d62e40f7e4f3c17d229a7687d6fcca5448de813
Component: engine
2018-04-23 13:52:44 -07:00
d7c0222450 Builder: Plumbing through platform in FROM statement
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: 69fa84bc3d57dafd19800642c5ba196bc6d45f90
Component: engine
2018-03-19 14:29:36 -07:00
bf670c6435 Builder - dockerfile - just use API for now, and unit test fix
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: 735e5d22b7ca208acc9ad7373bb8f93167ee3f85
Component: engine
2018-03-19 14:29:35 -07:00
be83c11fb0 Add canonical import comment
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 4f0d95fa6ee7f865597c03b9e63702cdcb0f7067
Component: engine
2018-02-05 16:51:57 -05:00
3ea1f0df2c Move builder shell parser into subpackage
Moves builder/shell_parser and into its own subpackage at builder/shell since it
has no dependencies other than the standard library. This will make it
much easier to vendor for downstream libraries, without pulling all the
dependencies of builder/.

Fixes #36154

Signed-off-by: Matt Rickard <mrick@google.com>
Upstream-commit: a634526d14639c6b98c509a069ef29e2b69c0ef0
Component: engine
2018-01-30 17:54:39 -08:00
40b95b8e94 Address feedback from Tonis
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: 0cba7740d41369eee33b671f26276325580bc07b
Component: engine
2018-01-18 12:30:39 -08:00
852153685d LCOW: Refactor to multiple layer-stores based on feedback
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: afd305c4b5682fbc297e1685e2b7a49628b7c7f0
Component: engine
2018-01-18 08:31:05 -08:00
33860da10b LCOW: Re-coalesce stores
Signed-off-by: John Howard <jhoward@microsoft.com>

The re-coalesces the daemon stores which were split as part of the
original LCOW implementation.

This is part of the work discussed in https://github.com/moby/moby/issues/34617,
in particular see the document linked to in that issue.
Upstream-commit: ce8e529e182bde057cdfafded62c210b7293b8ba
Component: engine
2018-01-18 08:29:19 -08:00
d4d0b5c268 Move api/errdefs to errdefs
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: d453fe35b9b8b52d0677fe0c3cc8373f2f5d30d0
Component: engine
2018-01-11 21:21:43 -05:00
952c29f8da Add helpers to create errdef errors
Instead of having to create a bunch of custom error types that are doing
nothing but wrapping another error in sub-packages, use a common helper
to create errors of the requested type.

e.g. instead of re-implementing this over and over:

```go
type notFoundError struct {
  cause error
}

func(e notFoundError) Error() string {
  return e.cause.Error()
}

func(e notFoundError) NotFound() {}

func(e notFoundError) Cause() error {
  return e.cause
}
```

Packages can instead just do:

```
  errdefs.NotFound(err)
```

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 87a12421a94faac294079bebc97c8abb4180dde5
Component: engine
2018-01-11 21:21:43 -05:00
2af5d43021 Fix environ substitutions in docker commit --change ...
The building machinery was being handed an uninitialized container
Config.  This changes it to use the target container's Config.

Resolves #30538

Signed-off-by: Anthony Sottile <asottile@umich.edu>
Upstream-commit: 0785836c4b440a8d4a5dfdb8df82e50f9f4d23a1
Component: engine
2017-12-20 11:03:38 -08:00
b9b2e53e40 builder: fix long stream sync
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: c6703b722e1c0914342d61ca6af77aea93150873
Component: engine
2017-11-03 12:53:17 -07:00
817827c1bc Test & Fix build with rm/force-rm matrix
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
Upstream-commit: 172e73a1dfb894c54fdc93fa6ef338eb1374d06c
Component: engine
2017-10-12 10:22:40 +02:00
3d973055bf LCOW: API change JSON header to string POST parameter
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: d98ecf2d6cdad7dae65868398440cfdc855e5263
Component: engine
2017-10-06 15:26:48 -07:00
35db73fa01 LCOW: API: Add platform to /images/create and /build
Signed-off-by: John Howard <jhoward@microsoft.com>

This PR has the API changes described in https://github.com/moby/moby/issues/34617.
Specifically, it adds an HTTP header "X-Requested-Platform" which is a JSON-encoded
OCI Image-spec `Platform` structure.

In addition, it renames (almost all) uses of a string variable platform (and associated)
methods/functions to os. This makes it much clearer to disambiguate with the swarm
"platform" which is really os/arch. This is a stepping stone to getting the daemon towards
fully multi-platform/arch-aware, and makes it clear when "operating system" is being
referred to rather than "platform" which is misleadingly used - sometimes in the swarm
meaning, but more often as just the operating system.
Upstream-commit: 0380fbff37922cadf294851b1546f4c212c7f364
Component: engine
2017-10-06 11:44:18 -07:00
e01e198f54 Introduce a typed command system and 2 phase parse/dispatch build
This is a work base to introduce more features like build time
dockerfile optimisations, dependency analysis and parallel build, as
well as a first step to go from a dispatch-inline process to a
frontend+backend process.

Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
Upstream-commit: 669c0677980b04bcbf871bb7c2d9f07caccfd42b
Component: engine
2017-09-18 09:48:56 +02:00
cddfe04f6a LCOW: Implemented support for docker cp + build
This enables docker cp and ADD/COPY docker build support for LCOW.
Originally, the graphdriver.Get() interface returned a local path
to the container root filesystem. This does not work for LCOW, so
the Get() method now returns an interface that LCOW implements to
support copying to and from the container.

Signed-off-by: Akash Gupta <akagup@microsoft.com>
Upstream-commit: 7a7357dae1bcccb17e9b2d4c7c8f5c025fce56ca
Component: engine
2017-09-14 12:07:52 -07:00
30f1b651e2 Remove string checking in API error handling
Use strongly typed errors to set HTTP status codes.
Error interfaces are defined in the api/errors package and errors
returned from controllers are checked against these interfaces.

Errors can be wraeped in a pkg/errors.Causer, as long as somewhere in the
line of causes one of the interfaces is implemented. The special error
interfaces take precedence over Causer, meaning if both Causer and one
of the new error interfaces are implemented, the Causer is not
traversed.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: ebcb7d6b406fe50ea9a237c73004d75884184c33
Component: engine
2017-08-15 16:01:11 -04:00
d659edcaf5 Update logrus to v1.0.1
Fixes case sensitivity issue

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: 1009e6a40b295187e038b67e184e9c0384d95538
Component: engine
2017-07-31 13:16:46 -07:00
7deeee36b6 Remove client/session package, depend on buildkit's session package
gofmt -w -r '"github.com/docker/docker/client/session" -> "github.com/moby/buildkit/session"'
gofmt -w -r '"github.com/docker/docker/client/session/filesync" -> "github.com/moby/buildkit/session/filesync"'

Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 41445a4745451b99d1c361c6c87447da790cb01b
Component: engine
2017-07-28 16:34:23 -07:00
1c25f0d80a builder: enable path filtering for filesync session
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: ad46348d7c9f8a84d61e1ba5f08c8fd0727e4ebc
Component: engine
2017-06-28 09:12:30 -07:00
0a2d82735e Add a test for warning on empty continuation lines.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: b47b375cb8bb0dca7ee0ebfa093bfc163ad867fb
Component: engine
2017-06-23 13:15:04 -04:00
e529bcd027 Implement incremental file sync using client session
Also exposes shared cache and garbage collection/prune
for the source data.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: 5c3d2d552b0430672d5f481ab2d37036f6e92166
Component: engine
2017-06-22 11:52:35 -07:00
b1e4d7d884 Add long-running client session endpoint
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: ec7b6238c31e26702ecf366731532e189b5da142
Component: engine
2017-06-22 11:22:41 -07:00
ebc74326cd LCOW: Fix ImageCache to address right store
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: ba40132366e196c740cdf2d2f1cbee7ead955d2f
Component: engine
2017-06-20 19:49:53 -07:00
2665467b29 LCOW: Builder plumbing
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: 08252bc963a20b2fb3052fc8db7143771c8c3248
Component: engine
2017-06-20 19:49:53 -07:00
13d2adae3f Fix copy when used with scratch and images with empty RootFS
Commit the rwLayer to get the correct DiffID
Refacator copy in thebuilder
move more code into exportImage
cleanup some windows tests
Release the newly commited layer.
Set the imageID on the buildStage after exporting a new image.
Move archiver to BuildManager.
Have ReleaseableLayer.Commit return a layer
and store the Image from exportImage in the local imageSources cache
Remove NewChild from image interface.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 51360965206b0db49cc0365dabb590063a17a9df
Component: engine
2017-06-08 15:07:16 -04:00
932c771fa1 Remove CopyOnBuild from the daemon.
Add CreateImage() to the daemon
Refactor daemon.Comit() and expose a Image.NewChild()
Update copy to use IDMappings.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: bd5f92d2631df7c932b93e72e45b39cba19f2f3b
Component: engine
2017-06-08 15:06:54 -04:00
4358607de5 Fix ONBUILD COPY
the source was missing from the second dispatch

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 3f2604157790408acf5ad05c74cebe105f2b6979
Component: engine
2017-06-02 17:47:03 -04:00
850d525b5f Extract imageProber and ContainerBackend from Builder
Extract a common function for builder.createContainer
Extract imageCache for doing cache probes
Removes the cacheBuested field from Builder
Create a new containerManager class which reduces the interface between the
builder and managing containers to 3 functions (from 6)

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 19f3b0715cf89f336efa286ea1d3f8b57d8c4973
Component: engine
2017-05-23 15:12:58 -04:00
1f72326d67 Refactor builder.calcCopyInfo and dependencies
Remove pathCache from imageContexts
Extract validateCopySourcePath
Extract copyWithWildcards
Extract copyInfoForFile and walkSource from calcCopyInfo
Move copy internals to copy.go
remove source from Builder

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 213ed02e255d881826aab002ad1d5df67de56b20
Component: engine
2017-05-17 09:54:39 -04:00
df464dbe2f Refactor imageContexts into two different structs.
buildStages now tracks the imageID and runConfig for a build stage

imageMounter tracks image mounts so they can released when the build ends.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 6c28e8edd5f047d5b1438f773d49882f28d7a006
Component: engine
2017-05-10 17:59:30 -04:00
e526b1e1d8 Expose a smaller interface for the Builder retrieving images from daemon
Removes 3 methods from the builder.Backend interface
Remove the coupling between imageContexts, imageMounts and the builder.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: b3bc7b28d09138a37ab5476eb46dfe74f8984f18
Component: engine
2017-05-10 17:58:57 -04:00
f04e158c49 Merge pull request #33114 from dnephin/fix-builder-warn-on-unused-builtint
Fix warning for unused build args
Upstream-commit: 22a03192fbf10f583003135421fcbc72c38f1ee4
Component: engine
2017-05-10 09:38:07 +02:00
6044c2ae89 Merge pull request #32792 from rogaha/extra_prometheus_metrics
Add extra prometheus metrics
Upstream-commit: 1a6f8a92b2c5912058828d560f52281ba0424472
Component: engine
2017-05-09 21:48:34 +02:00
3a59c54458 Fix warning for unused build args.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 72cc81ee8dcfca5503f3a02289f8f3d4a08582ad
Component: engine
2017-05-09 11:25:33 -04:00
a7451e12cd Add extra prometheus metrics
- buildsTriggered
 - buildsFailed
    - valid options:
        metricsDockerfileSyntaxError,
        metricsDockerfileEmptyError,
        metricsCommandNotSupportedError,
        metricsErrorProcessingCommandsError,
        metricsBuildTargetNotReachableError,
        metricsMissingOnbuildArgumentsError,
        metricsUnknownInstructionError,
        metricsBuildCanceled,
- engineInfo

Signed-off-by: Roberto Gandolfo Hashioka <roberto_hashioka@hotmail.com>
Upstream-commit: a28b173a780cd06db6d93197c54b00a7d616b3dc
Component: engine
2017-05-09 01:04:40 -07:00
bd17f42bef Add docker build --iidfile=FILE
This is synonymous with `docker run --cidfile=FILE` and writes the digest of
the newly built image to the named file. This is intended to be used by build
systems which want to avoid tagging (perhaps because they are in CI or
otherwise want to avoid fixed names which can clash) by enabling e.g. Makefile
constructs like:

    image.id: Dockerfile
    	docker build --iidfile=image.id .

    do-some-more-stuff: image.id
    	do-stuff-with <image.id

Currently the only way to achieve this is to use `docker build -q` and capture
the stdout, but at the expense of losing the build output.

In non-silent mode (without `-q`) with API >= v1.29 the caller will now see a
`JSONMessage` with the `Aux` field containing a `types.BuildResult` in the
output stream for each image/layer produced during the build, with the final
one being the end product.  Having all of the intermediate images might be
interesting in some cases.

In silent mode (with `-q`) there is no change, on success the only output will
be the resulting image digest as it was previosuly.

There was no wrapper to just output an Aux section without enclosing it in a
Progress, so add one here.

Added some tests to integration cli tests.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Upstream-commit: 5894bc1abf8186802d360d20739b57bfffed51df
Component: engine
2017-05-05 16:35:54 +01:00
965d054208 Some refactoring of dispatch()
Remove runConfig from Builder and dispatchRequest. It is not only on
dispatchState.

Move dispatch state fields from Builder to dispatchState

Move stageName tracking to dispatchRequest.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 2f0ebba0e7f43908a41d5ff9bbb1b9b78743a6ab
Component: engine
2017-05-04 17:11:08 -04:00
dbcc3f98b6 Cleanup all the mutate + defer revert of b.runConfig in the builder
Instead of mutating and reverting, just create a copy and pass the copy
around.

Add a unit test for builder dispatcher.run

Fix two test failures

Fix image history by adding a CreatedBy to commit options. Previously the
createdBy field was being created by modifying a reference to the runConfig that
was held from when the container was created.

Fix a test that expected a trailing slash. Previously the runConfig was being
modified by container create. Now that we're creating a copy of runConfig
instead of sharing a reference the runConfig retains the trailing slash.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 9f738cc574d50d0a2accdf6f6deb30405c24a80c
Component: engine
2017-05-01 18:36:53 -04:00
3d193f829d Extract squash and tagging from the Dockerfile builder.
Remove pathCache and replace it with syncmap
Cleanup NewBuilder
Create an api/server/backend/build
Extract BuildTagger

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 0296797f0f39477d675128c93c1646b3186937ee
Component: engine
2017-05-01 12:07:32 -04:00
ce59d98dee Merge pull request #32858 from dnephin/builder-shell-words-interface
[Builder] Remove b.escapeToken, create ShellLex
Upstream-commit: aee2da3bdf6665939722dacf3b0c149499c35d90
Component: engine
2017-04-27 13:16:40 -04:00
e25a81277d Create a new ShellLex struct which provides the expected interface to Builder.
Remove b.escapeToken from builder

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 2414166e1e38a4de8aa4af40b90c7e1bc90102b0
Component: engine
2017-04-26 19:39:16 -04:00
aadacb3d07 Refactor remote context parsing
Redefine a better interface for remote context dependency.

Separate Dockerfile build instruction from remote context.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: d1faf3df27207af111daf4bd0191478c374dbc55
Component: engine
2017-04-25 14:35:31 -07:00