From 65b28186fc4172670b3279e882429496dcbe7ce6 Mon Sep 17 00:00:00 2001 From: Sebastiaan van Stijn Date: Sat, 20 Apr 2019 13:19:07 +0200 Subject: [PATCH] bump buildkit 05766c5c21a1e528eeb1c3522b2f05493fe9ac47 (docker-18.09 branch) full diff: https://github.com/moby/buildkit/compare/520201006c9dc676da9cf9655337ac711f7f127d..05766c5c21a1e528eeb1c3522b2f05493fe9ac47 - moby/buildkit#952 [18.09 backport] Have parser error on dockerfiles without instructions - backport of moby/buildkit#771 Have parser error on dockerfiles without instructions Signed-off-by: Sebastiaan van Stijn --- vendor.conf | 2 +- vendor/github.com/moby/buildkit/README.md | 36 ++++++++++++++----- .../moby/buildkit/client/llb/source.go | 25 ++----------- .../moby/buildkit/client/llb/state.go | 27 ++++++++++++++ .../frontend/gateway/grpcclient/client.go | 6 ++++ .../moby/buildkit/session/session.go | 4 +-- vendor/github.com/moby/buildkit/vendor.conf | 16 ++++----- 7 files changed, 74 insertions(+), 42 deletions(-) diff --git a/vendor.conf b/vendor.conf index 293bc6539f..85365724c5 100755 --- a/vendor.conf +++ b/vendor.conf @@ -51,7 +51,7 @@ github.com/Microsoft/hcsshim 44c060121b68e8bdc40b411beba551f3b4ee9e55 github.com/Microsoft/go-winio v0.4.10 github.com/miekg/pkcs11 6120d95c0e9576ccf4a78ba40855809dca31a9ed github.com/mitchellh/mapstructure f15292f7a699fcc1a38a80977f80a046874ba8ac -github.com/moby/buildkit 520201006c9dc676da9cf9655337ac711f7f127d +github.com/moby/buildkit 05766c5c21a1e528eeb1c3522b2f05493fe9ac47 github.com/modern-go/concurrent bacd9c7ef1dd9b15be4a9909b8ac7a4e313eec94 # 1.0.3 github.com/modern-go/reflect2 4b7aa43c6742a2c18fdef89dd197aaae7dac7ccd # 1.0.1 github.com/morikuni/aec 39771216ff4c63d11f5e604076f9c45e8be1067b diff --git a/vendor/github.com/moby/buildkit/README.md b/vendor/github.com/moby/buildkit/README.md index ae8c566f21..67aa5ade42 100644 --- a/vendor/github.com/moby/buildkit/README.md +++ b/vendor/github.com/moby/buildkit/README.md @@ -27,9 +27,11 @@ Read the proposal from https://github.com/moby/moby/issues/32925 Introductory blog post https://blog.mobyproject.org/introducing-buildkit-17e056cc5317 +:information_source: If you are visiting this repo for the usage of experimental Dockerfile features like `RUN --mount=type=(bind|cache|tmpfs|secret|ssh)`, please refer to [`frontend/dockerfile/docs/experimental.md`](frontend/dockerfile/docs/experimental.md). + ### Used by -[Moby](https://github.com/moby/moby/pull/37151) +[Moby & Docker](https://github.com/moby/moby/pull/37151) [img](https://github.com/genuinetools/img) @@ -37,6 +39,12 @@ Introductory blog post https://blog.mobyproject.org/introducing-buildkit-17e056c [container build interface](https://github.com/containerbuilding/cbi) +[Knative Build Templates](https://github.com/knative/build-templates) + +[boss](https://github.com/crosbymichael/boss) + +[Rio](https://github.com/rancher/rio) (on roadmap) + ### Quick start Dependencies: @@ -79,6 +87,7 @@ See [`solver/pb/ops.proto`](./solver/pb/ops.proto) for the format definition. Currently, following high-level languages has been implemented for LLB: - Dockerfile (See [Exploring Dockerfiles](#exploring-dockerfiles)) +- [Buildpacks](https://github.com/tonistiigi/buildkit-pack) - (open a PR to add your own language) For understanding the basics of LLB, `examples/buildkit*` directory contains scripts that define how to build different configurations of BuildKit itself and its dependencies using the `client` package. Running one of these scripts generates a protobuf definition of a build graph. Note that the script itself does not execute any steps of the build. @@ -136,15 +145,19 @@ build-using-dockerfile -t mybuildkit -f ./hack/dockerfiles/test.Dockerfile . docker inspect myimage ``` -##### Building a Dockerfile using [external frontend](https://hub.docker.com/r/tonistiigi/dockerfile/tags/): +##### Building a Dockerfile using [external frontend](https://hub.docker.com/r/docker/dockerfile/tags/): -During development, an external version of the Dockerfile frontend is pushed to https://hub.docker.com/r/tonistiigi/dockerfile that can be used with the gateway frontend. The source for the external frontend is currently located in `./frontend/dockerfile/cmd/dockerfile-frontend` but will move out of this repository in the future ([#163](https://github.com/moby/buildkit/issues/163)). For automatic build from master branch of this repository `tonistiigi/dockerfile:master` image can be used. +External versions of the Dockerfile frontend are pushed to https://hub.docker.com/r/docker/dockerfile-upstream and https://hub.docker.com/r/docker/dockerfile and can be used with the gateway frontend. The source for the external frontend is currently located in `./frontend/dockerfile/cmd/dockerfile-frontend` but will move out of this repository in the future ([#163](https://github.com/moby/buildkit/issues/163)). For automatic build from master branch of this repository `docker/dockerfile-upsteam:master` or `docker/dockerfile-upstream:master-experimental` image can be used. ``` -buildctl build --frontend=gateway.v0 --frontend-opt=source=tonistiigi/dockerfile --local context=. --local dockerfile=. -buildctl build --frontend gateway.v0 --frontend-opt=source=tonistiigi/dockerfile --frontend-opt=context=git://github.com/moby/moby --frontend-opt build-arg:APT_MIRROR=cdn-fastly.deb.debian.org +buildctl build --frontend=gateway.v0 --frontend-opt=source=docker/dockerfile --local context=. --local dockerfile=. +buildctl build --frontend gateway.v0 --frontend-opt=source=docker/dockerfile --frontend-opt=context=git://github.com/moby/moby --frontend-opt build-arg:APT_MIRROR=cdn-fastly.deb.debian.org ```` +##### Building a Dockerfile with experimental features like `RUN --mount=type=(bind|cache|tmpfs|secret|ssh)` + +See [`frontend/dockerfile/docs/experimental.md`](frontend/dockerfile/docs/experimental.md). + ### Exporters By default, the build result and intermediate cache will only remain internally in BuildKit. Exporter needs to be specified to retrieve the result. @@ -207,15 +220,22 @@ buildctl debug workers -v BuildKit can also be used by running the `buildkitd` daemon inside a Docker container and accessing it remotely. The client tool `buildctl` is also available for Mac and Windows. +We provide `buildkitd` container images as [`moby/buildkit`](https://hub.docker.com/r/moby/buildkit/tags/): + +* `moby/buildkit:latest`: built from the latest regular [release](https://github.com/moby/buildkit/releases) +* `moby/buildkit:rootless`: same as `latest` but runs as an unprivileged user, see [`docs/rootless.md`](docs/rootless.md) +* `moby/buildkit:master`: built from the master branch +* `moby/buildkit:master-rootless`: same as master but runs as an unprivileged user, see [`docs/rootless.md`](docs/rootless.md) + To run daemon in a container: ``` -docker run -d --privileged -p 1234:1234 tonistiigi/buildkit --addr tcp://0.0.0.0:1234 +docker run -d --privileged -p 1234:1234 moby/buildkit:latest --addr tcp://0.0.0.0:1234 export BUILDKIT_HOST=tcp://0.0.0.0:1234 buildctl build --help ``` -The `tonistiigi/buildkit` image can be built locally using the Dockerfile in `./hack/dockerfiles/test.Dockerfile`. +The images can be also built locally using `./hack/dockerfiles/test.Dockerfile` (or `./hack/dockerfiles/test.buildkit.Dockerfile` if you already have BuildKit). ### Opentracing support @@ -232,7 +252,7 @@ export JAEGER_TRACE=0.0.0.0:6831 ### Supported runc version -During development, BuildKit is tested with the version of runc that is being used by the containerd repository. Please refer to [runc.md](https://github.com/containerd/containerd/blob/v1.1.3/RUNC.md) for more information. +During development, BuildKit is tested with the version of runc that is being used by the containerd repository. Please refer to [runc.md](https://github.com/containerd/containerd/blob/v1.2.0-rc.1/RUNC.md) for more information. ### Running BuildKit without root privileges diff --git a/vendor/github.com/moby/buildkit/client/llb/source.go b/vendor/github.com/moby/buildkit/client/llb/source.go index ef03b19f1b..52d40be8b7 100644 --- a/vendor/github.com/moby/buildkit/client/llb/source.go +++ b/vendor/github.com/moby/buildkit/client/llb/source.go @@ -126,30 +126,11 @@ func Image(ref string, opts ...ImageOption) State { if err != nil { src.err = err } else { - var img struct { - Config struct { - Env []string `json:"Env,omitempty"` - WorkingDir string `json:"WorkingDir,omitempty"` - User string `json:"User,omitempty"` - } `json:"config,omitempty"` - } - if err := json.Unmarshal(dt, &img); err != nil { - src.err = err - } else { - st := NewState(src.Output()) - for _, env := range img.Config.Env { - parts := strings.SplitN(env, "=", 2) - if len(parts[0]) > 0 { - var v string - if len(parts) > 1 { - v = parts[1] - } - st = st.AddEnv(parts[0], v) - } - } - st = st.Dir(img.Config.WorkingDir) + st, err := NewState(src.Output()).WithImageConfig(dt) + if err == nil { return st } + src.err = err } } return NewState(src.Output()) diff --git a/vendor/github.com/moby/buildkit/client/llb/state.go b/vendor/github.com/moby/buildkit/client/llb/state.go index 15639fd3ee..24e3949bf8 100644 --- a/vendor/github.com/moby/buildkit/client/llb/state.go +++ b/vendor/github.com/moby/buildkit/client/llb/state.go @@ -2,8 +2,10 @@ package llb import ( "context" + "encoding/json" "fmt" "net" + "strings" "github.com/containerd/containerd/platforms" "github.com/moby/buildkit/identity" @@ -171,6 +173,31 @@ func (s State) WithOutput(o Output) State { return s } +func (s State) WithImageConfig(c []byte) (State, error) { + var img struct { + Config struct { + Env []string `json:"Env,omitempty"` + WorkingDir string `json:"WorkingDir,omitempty"` + User string `json:"User,omitempty"` + } `json:"config,omitempty"` + } + if err := json.Unmarshal(c, &img); err != nil { + return State{}, err + } + for _, env := range img.Config.Env { + parts := strings.SplitN(env, "=", 2) + if len(parts[0]) > 0 { + var v string + if len(parts) > 1 { + v = parts[1] + } + s = s.AddEnv(parts[0], v) + } + } + s = s.Dir(img.Config.WorkingDir) + return s, nil +} + func (s State) Run(ro ...RunOption) ExecState { ei := &ExecInfo{State: s} if p := s.GetPlatform(); p != nil { diff --git a/vendor/github.com/moby/buildkit/frontend/gateway/grpcclient/client.go b/vendor/github.com/moby/buildkit/frontend/gateway/grpcclient/client.go index 5e87fa57e5..d4e0525d89 100644 --- a/vendor/github.com/moby/buildkit/frontend/gateway/grpcclient/client.go +++ b/vendor/github.com/moby/buildkit/frontend/gateway/grpcclient/client.go @@ -356,6 +356,9 @@ func (r *reference) ReadFile(ctx context.Context, req client.ReadRequest) ([]byt } func (r *reference) ReadDir(ctx context.Context, req client.ReadDirRequest) ([]*fstypes.Stat, error) { + if err := r.c.caps.Supports(pb.CapReadDir); err != nil { + return nil, err + } rdr := &pb.ReadDirRequest{ DirPath: req.Path, IncludePattern: req.IncludePattern, @@ -369,6 +372,9 @@ func (r *reference) ReadDir(ctx context.Context, req client.ReadDirRequest) ([]* } func (r *reference) StatFile(ctx context.Context, req client.StatRequest) (*fstypes.Stat, error) { + if err := r.c.caps.Supports(pb.CapStatFile); err != nil { + return nil, err + } rdr := &pb.StatFileRequest{ Path: req.Path, Ref: r.id, diff --git a/vendor/github.com/moby/buildkit/session/session.go b/vendor/github.com/moby/buildkit/session/session.go index 47c9579633..5d04738ef7 100644 --- a/vendor/github.com/moby/buildkit/session/session.go +++ b/vendor/github.com/moby/buildkit/session/session.go @@ -24,7 +24,7 @@ const ( // Dialer returns a connection that can be used by the session type Dialer func(ctx context.Context, proto string, meta map[string][]string) (net.Conn, error) -// Attachable defines a feature that can be expsed on a session +// Attachable defines a feature that can be exposed on a session type Attachable interface { Register(*grpc.Server) } @@ -66,7 +66,7 @@ func NewSession(ctx context.Context, name, sharedKey string) (*Session, error) { return s, nil } -// Allow enable a given service to be reachable through the grpc session +// Allow enables a given service to be reachable through the grpc session func (s *Session) Allow(a Attachable) { a.Register(s.grpcServer) } diff --git a/vendor/github.com/moby/buildkit/vendor.conf b/vendor/github.com/moby/buildkit/vendor.conf index ba9a01ed3c..33f167b7a0 100644 --- a/vendor/github.com/moby/buildkit/vendor.conf +++ b/vendor/github.com/moby/buildkit/vendor.conf @@ -6,7 +6,7 @@ github.com/davecgh/go-spew v1.1.0 github.com/pmezard/go-difflib v1.0.0 golang.org/x/sys 1b2967e3c290b7c545b3db0deeda16e9be4f98a2 -github.com/containerd/containerd d97a907f7f781c0ab8340877d8e6b53cc7f1c2f6 +github.com/containerd/containerd 1a5f9a3434ac53c0e9d27093ecc588e0c281c333 github.com/containerd/typeurl a93fcdb778cd272c6e9b3028b2f42d813e785d40 golang.org/x/sync 450f422ab23cf9881c94e2db30cac0eb1b7cf80c github.com/sirupsen/logrus v1.0.0 @@ -16,9 +16,9 @@ golang.org/x/net 0ed95abb35c445290478a5348a7b38bb154135fd github.com/gogo/protobuf v1.0.0 github.com/gogo/googleapis b23578765ee54ff6bceff57f397d833bf4ca6869 github.com/golang/protobuf v1.1.0 -github.com/containerd/continuity f44b615e492bdfb371aae2f76ec694d9da1db537 +github.com/containerd/continuity bd77b46c8352f74eb12c85bdc01f4b90f69d66b4 github.com/opencontainers/image-spec v1.0.1 -github.com/opencontainers/runc 20aff4f0488c6d4b8df4d85b4f63f1f704c11abd +github.com/opencontainers/runc a00bf0190895aa465a5fbed0268888e2c8ddfe85 github.com/Microsoft/go-winio v0.4.11 github.com/containerd/fifo 3d5202aec260678c48179c56f40e6f38a095738c github.com/opencontainers/runtime-spec eba862dc2470385a233c7507392675cbeadf7353 # v1.0.1-45-geba862d @@ -28,8 +28,9 @@ google.golang.org/genproto d80a6e20e776b0b17a324d0ba1ab50a39c8e8944 golang.org/x/text 19e51611da83d6be54ddafce4a4af510cb3e9ea4 github.com/docker/go-events 9461782956ad83b30282bf90e31fa6a70c255ba9 github.com/syndtr/gocapability db04d3cc01c8b54962a58ec7e491717d06cfcc16 -github.com/Microsoft/hcsshim v0.7.3 +github.com/Microsoft/hcsshim v0.7.9 golang.org/x/crypto 0709b304e793a5edb4a2c0145f281ecdc20838a4 +github.com/containerd/cri 8506fe836677cc3bb23a16b68145128243d843b5 # release/1.2 branch github.com/urfave/cli 7bc6a0acffa589f415f88aca16cc1de5ffd66f9c github.com/morikuni/aec 39771216ff4c63d11f5e604076f9c45e8be1067b @@ -40,8 +41,8 @@ golang.org/x/time f51c12702a4d776e4c1fa9b0fabab841babae631 github.com/docker/docker 71cd53e4a197b303c6ba086bd584ffd67a884281 github.com/pkg/profile 5b67d428864e92711fcbd2f8629456121a56d91f -github.com/tonistiigi/fsutil f567071bed2416e4d87d260d3162722651182317 -github.com/hashicorp/go-immutable-radix 826af9ccf0feeee615d546d69b11f8e98da8c8f1 git://github.com/tonistiigi/go-immutable-radix.git +github.com/tonistiigi/fsutil 2862f6bc5ac9b97124e552a5c108230b38a1b0ca +github.com/hashicorp/go-immutable-radix 826af9ccf0feeee615d546d69b11f8e98da8c8f1 https://github.com/tonistiigi/go-immutable-radix github.com/hashicorp/golang-lru a0d98a5f288019575c6d1f4bb1573fef2d1fcdc4 github.com/mitchellh/hashstructure 2bca23e0e452137f789efbc8610126fd8b94f73b github.com/docker/go-connections 3ede32e2033de7505e6500d6c868c2b9ed9f169d @@ -66,6 +67,3 @@ github.com/opentracing-contrib/go-stdlib b1a47cfbdd7543e70e9ef3e73d0802ad306cc1c # used by dockerfile tests gotest.tools v2.1.0 github.com/google/go-cmp v0.2.0 - -# used by rootless spec conv test -github.com/seccomp/libseccomp-golang 32f571b70023028bd57d9288c20efbcb237f3ce0