Compare commits

...

35 Commits

Author SHA1 Message Date
ad6c667034 Merge pull request #2178 from thaJeztah/19.03_backport_docs_updates
[19.03 backport] docs updates
2019-10-29 16:36:30 -07:00
5cef8fc8e5 Merge pull request #2176 from thaJeztah/19.03_backport_completion
[19.03 backport] completion script updates and fixes
2019-10-29 16:36:23 -07:00
9597e681cb update Jenkins Build Status urls for README
Signed-off-by: danielhelfand <helfand.4@gmail.com>
(cherry picked from commit 46f3f9ac04)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-29 15:27:54 +01:00
c050ae7fb1 Adjusting glossary reference and clarrifying the start of a Dockerfile
Signed-off-by: Brandon Mitchell <git@bmitch.net>
(cherry picked from commit 937ce69b42)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-29 15:27:42 +01:00
aa45ea366c Minor formatting changes and a typo fix.
Signed-off-by: Adrian Plata <adrian.plata@docker.com>
(cherry picked from commit ba7ec36de2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-29 15:27:28 +01:00
df5c7130df Updating deprecated command example in search.md
--stars example is deprecated. Changing to --filter=stars=5

Signed-off-by: danielartine <daniel.artine@ufrj.br>
(cherry picked from commit bba0a4d5ac)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-29 15:26:32 +01:00
a90e5598a4 Fix phrasing when referring to the freezer cgroup
Signed-off-by: Odin Ugedal <odin@ugedal.com>
(cherry picked from commit a0dd6e025d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-29 15:26:20 +01:00
66f8d279c1 builder.md: Correct copy/paste error
Signed-off-by: Daniel Cassidy <mail@danielcassidy.me.uk>
(cherry picked from commit e9ada56a88)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-29 15:26:09 +01:00
c4e5fd7e53 Explain -s flag on docker ps
https://github.com/docker/docker.github.io/issues/1520#issuecomment-467971886

Signed-off-by: Forest Johnson <fjohnson@peoplenetonline.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f7ab2987c3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-29 15:25:56 +01:00
57332bcea3 Add bash completion for builder build options
Signed-off-by: Harald Albers <github@albersweb.de>
(cherry picked from commit cd9b39fc52)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-29 15:11:42 +01:00
2fe281d170 Add bash completion for builder build
Signed-off-by: Trapier Marshall <trapier.marshall@docker.com>
(cherry picked from commit 86ae871e0a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-29 15:11:28 +01:00
69f0132397 Fix erroneous space indentation of shell script
Signed-off-by: Morten Hekkvang <morten.hekkvang@sbab.se>
(cherry picked from commit fc19fb6d95)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-29 15:11:15 +01:00
9447d7431b Move filtering to __docker_complete_services, as suggested by @albers
Signed-off-by: Morten Hekkvang <morten.hekkvang@sbab.se>
(cherry picked from commit 6ebcc211a7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-29 15:11:13 +01:00
c70ce79adc Fix substitution of comma to space character, suggested by @albers
Signed-off-by: Morten Hekkvang <morten.hekkvang@sbab.se>
(cherry picked from commit b85e1e2efe)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-29 15:11:10 +01:00
667bd9a1b8 Make service completion faster
Signed-off-by: Morten Hekkvang <morten.hekkvang@sbab.se>
(cherry picked from commit f55c5b6566)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-29 15:11:08 +01:00
6c66c799c7 add rudimentary fish completions for docker network
Signed-off-by: Tim Sampson <tim@sampson.fi>
(cherry picked from commit 2f7b364cb3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-29 15:10:42 +01:00
8a9e86c728 Merge pull request #2124 from chris-crone/backport-19.03-env-merge
[19.03 backport] Add option to remove `env_file` entry once it's merged in the `environment` section
2019-10-22 14:52:48 -07:00
e176053a3f Merge pull request #2150 from thaJeztah/19.03_backport_bump_golang_1.12.12
[19.03 backport] Update Golang 1.12.12 (CVE-2019-17596)
2019-10-22 14:50:04 -07:00
f38bea4ac3 Update Golang 1.12.12 (CVE-2019-17596)
Golang 1.12.12
-------------------------------

full diff: https://github.com/golang/go/compare/go1.12.11...go1.12.12

go1.12.12 (released 2019/10/17) includes fixes to the go command, runtime,
syscall and net packages. See the Go 1.12.12 milestone on our issue tracker for
details.

https://github.com/golang/go/issues?q=milestone%3AGo1.12.12

Golang 1.12.11 (CVE-2019-17596)
-------------------------------

full diff: https://github.com/golang/go/compare/go1.12.10...go1.12.11

go1.12.11 (released 2019/10/17) includes security fixes to the crypto/dsa
package. See the Go 1.12.11 milestone on our issue tracker for details.

https://github.com/golang/go/issues?q=milestone%3AGo1.12.11

    [security] Go 1.13.2 and Go 1.12.11 are released

    Hi gophers,

    We have just released Go 1.13.2 and Go 1.12.11 to address a recently reported
    security issue. We recommend that all affected users update to one of these
    releases (if you're not sure which, choose Go 1.13.2).

    Invalid DSA public keys can cause a panic in dsa.Verify. In particular, using
    crypto/x509.Verify on a crafted X.509 certificate chain can lead to a panic,
    even if the certificates don't chain to a trusted root. The chain can be
    delivered via a crypto/tls connection to a client, or to a server that accepts
    and verifies client certificates. net/http clients can be made to crash by an
    HTTPS server, while net/http servers that accept client certificates will
    recover the panic and are unaffected.

    Moreover, an application might crash invoking
    crypto/x509.(*CertificateRequest).CheckSignature on an X.509 certificate
    request, parsing a golang.org/x/crypto/openpgp Entity, or during a
    golang.org/x/crypto/otr conversation. Finally, a golang.org/x/crypto/ssh client
    can panic due to a malformed host key, while a server could panic if either
    PublicKeyCallback accepts a malformed public key, or if IsUserAuthority accepts
    a certificate with a malformed public key.

    The issue is CVE-2019-17596 and Go issue golang.org/issue/34960.

    Thanks to Daniel Mandragona for discovering and reporting this issue. We'd also
    like to thank regilero for a previous disclosure of CVE-2019-16276.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 474d522ee2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-22 03:48:54 +02:00
379470969b Add option remove env_file entry after merged
This avoids having a redundant `env_file` entry
output when rendering the compose file

Signed-off-by: Ulysses Souza <ulysses.souza@docker.com>
(cherry picked from commit 821f5ecf53)
Signed-off-by: Christopher Crone <christopher.crone@docker.com>
2019-10-04 17:32:48 +02:00
2355349d8b Merge pull request #2119 from thaJeztah/19.03_backport_bump_yaml.v2_2.2.3
[19.03 backport] vendor: Bump gopkg.in/yaml.v2
2019-10-03 10:52:44 -07:00
73d513e56a Merge pull request #2120 from thaJeztah/19.03_backport_experimental_note
[19.03 backport] backport docs changes
2019-10-01 14:27:56 -07:00
9fd5604d80 Adding experimental feature info
Signed-off-by: Adrian Plata <adrian.plata@docker.com>
(cherry picked from commit 031bd2d530)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-01 19:11:06 +02:00
82ac89901f Clarifying docker config rules
Signed-off-by: Adrian Plata <adrian.plata@docker.com>
(cherry picked from commit 131eefc54f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-01 19:11:01 +02:00
29e3a70b04 vendor: Bump gopkg.in/yaml.v2
Signed-off-by: Christopher Crone <christopher.crone@docker.com>
(cherry picked from commit 91cf8b04c9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-01 14:54:49 +02:00
3fb239756d Merge pull request #2113 from thaJeztah/19.03_backport_bump_golang_1.12.10
[19.03 backport] bump golang 1.12.10 (CVE-2019-16276)
2019-09-28 01:53:31 +02:00
ff72c27a7a bump golang 1.12.10 (CVE-2019-16276)
full diff: https://github.com/golang/go/compare/go1.12.9...go1.12.10

```
Hi gophers,

We have just released Go 1.13.1 and Go 1.12.10 to address a recently reported security issue. We recommend that all affected users update to one of these releases (if you're not sure which, choose Go 1.13.1).

net/http (through net/textproto) used to accept and normalize invalid HTTP/1.1 headers with a space before the colon, in violation of RFC 7230. If a Go server is used behind an uncommon reverse proxy that accepts and forwards but doesn't normalize such invalid headers, the reverse proxy and the server can interpret the headers differently. This can lead to filter bypasses or request smuggling, the latter if requests from separate clients are multiplexed onto the same upstream connection by the proxy. Such invalid headers are now rejected by Go servers, and passed without normalization to Go client applications.

The issue is CVE-2019-16276 and Go issue golang.org/issue/34540.

Thanks to Andrew Stucki, Adam Scarr (99designs.com), and Jan Masarik (masarik.sh) for discovering and reporting this issue.

Downloads are available at https://golang.org/dl for all supported platforms.

Alla prossima,
Filippo on behalf of the Go team
```

From the patch: 6e6f4aaf70

```
net/textproto: don't normalize headers with spaces before the colon

RFC 7230 is clear about headers with a space before the colon, like

X-Answer : 42

being invalid, but we've been accepting and normalizing them for compatibility
purposes since CL 5690059 in 2012.

On the client side, this is harmless and indeed most browsers behave the same
to this day. On the server side, this becomes a security issue when the
behavior doesn't match that of a reverse proxy sitting in front of the server.

For example, if a WAF accepts them without normalizing them, it might be
possible to bypass its filters, because the Go server would interpret the
header differently. Worse, if the reverse proxy coalesces requests onto a
single HTTP/1.1 connection to a Go server, the understanding of the request
boundaries can get out of sync between them, allowing an attacker to tack an
arbitrary method and path onto a request by other clients, including
authentication headers unknown to the attacker.

This was recently presented at multiple security conferences:
https://portswigger.net/blog/http-desync-attacks-request-smuggling-reborn

net/http servers already reject header keys with invalid characters.
Simply stop normalizing extra spaces in net/textproto, let it return them
unchanged like it does for other invalid headers, and let net/http enforce
RFC 7230, which is HTTP specific. This loses us normalization on the client
side, but there's no right answer on the client side anyway, and hiding the
issue sounds worse than letting the application decide.
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8743e36a45)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-27 23:33:06 +02:00
ed54e85144 Merge pull request #2079 from thaJeztah/19.03_backport_bump_pkcs11
[19.03 backport] bump miekg/pkcs11 v1.0.2
2019-09-27 13:41:10 +02:00
0bc3d1fd2d bump miekg/pkcs11 v1.0.2
full diff: 6120d95c0e...v1.0.2

relevant changes:

- miekg/pkcs11#110 Fix issue freeing memory on GetOperationState when NOT CK_OK
- miekg/pkcs11#106 Move to go modules
- miekg/pkcs11#104 Expose login API for vendor specific login types

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 54428b1f37)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-27 11:51:03 +02:00
50bb8c70f3 Merge pull request #2102 from tiborvass/19.03-buildkit-vendor
[19.03] vendor: update buildkit to docker-19.03
2019-09-23 18:18:48 -07:00
25168137bd vendor: update buildkit to docker-19.03
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-09-24 00:36:30 +00:00
3af5c3f8c6 Merge pull request #2099 from thaJeztah/19.03_backport_bump_vndr
[19.03 backport] bump LK4D4/vndr v0.0.3 and revendor
2019-09-23 16:59:17 -07:00
3143ba5700 bump LK4D4/vndr v0.0.3 and revendor
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 909b85460c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-23 19:30:55 +02:00
a89484e338 Merge pull request #2090 from thaJeztah/19.03_backport_bump_go_1.12.9
[19.03 backport] Bump golang 1.12.9
2019-09-23 10:21:02 -07:00
2ac9213069 Bump golang 1.12.9
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
(cherry picked from commit d9856049e5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-16 16:35:02 +02:00
61 changed files with 743 additions and 243 deletions

View File

@ -1,4 +1,5 @@
[![build status](https://circleci.com/gh/docker/cli.svg?style=shield)](https://circleci.com/gh/docker/cli/tree/master) [![Build Status](https://jenkins.dockerproject.org/job/docker/job/cli/job/master/badge/icon)](https://jenkins.dockerproject.org/job/docker/job/cli/job/master/)
[![build status](https://circleci.com/gh/docker/cli.svg?style=shield)](https://circleci.com/gh/docker/cli/tree/master)
[![Build Status](https://ci.docker.com/public/job/cli/job/master/badge/icon)](https://ci.docker.com/public/job/cli/job/master)
docker/cli
==========

View File

@ -4,7 +4,7 @@ clone_folder: c:\gopath\src\github.com\docker\cli
environment:
GOPATH: c:\gopath
GOVERSION: 1.12.8
GOVERSION: 1.12.12
DEPVERSION: v0.4.1
install:

View File

@ -156,7 +156,8 @@ func runBuildBuildKit(dockerCli command.Cli, options buildOptions) error {
}
w = f
}
s.Allow(filesync.NewFSSyncTarget(w))
output := func(map[string]string) (io.WriteCloser, error) { return w, nil }
s.Allow(filesync.NewFSSyncTarget(output))
}
}

View File

@ -32,6 +32,14 @@ type Options struct {
SkipInterpolation bool
// Interpolation options
Interpolate *interp.Options
// Discard 'env_file' entries after resolving to 'environment' section
discardEnvFiles bool
}
// WithDiscardEnvFiles sets the Options to discard the `env_file` section after resolving to
// the `environment` section
func WithDiscardEnvFiles(opts *Options) {
opts.discardEnvFiles = true
}
// ParseYAML reads the bytes from a file, parses the bytes into a mapping
@ -105,6 +113,11 @@ func Load(configDetails types.ConfigDetails, options ...func(*Options)) (*types.
return nil, err
}
cfg.Filename = file.Filename
if opts.discardEnvFiles {
for i := range cfg.Services {
cfg.Services[i].EnvFile = nil
}
}
configs = append(configs, cfg)
}

View File

@ -759,6 +759,38 @@ services:
assert.Check(t, is.DeepEqual([]string{"build", "links", "pid"}, unsupported))
}
func TestDiscardEnvFileOption(t *testing.T) {
dict, err := ParseYAML([]byte(`version: "3"
services:
web:
image: nginx
env_file:
- example1.env
- example2.env
`))
expectedEnvironmentMap := types.MappingWithEquals{
"FOO": strPtr("foo_from_env_file"),
"BAZ": strPtr("baz_from_env_file"),
"BAR": strPtr("bar_from_env_file_2"), // Original value is overwritten by example2.env
"QUX": strPtr("quz_from_env_file_2"),
}
assert.NilError(t, err)
configDetails := buildConfigDetails(dict, nil)
// Default behavior keeps the `env_file` entries
configWithEnvFiles, err := Load(configDetails)
assert.NilError(t, err)
assert.DeepEqual(t, configWithEnvFiles.Services[0].EnvFile, types.StringList{"example1.env",
"example2.env"})
assert.DeepEqual(t, configWithEnvFiles.Services[0].Environment, expectedEnvironmentMap)
// Custom behavior removes the `env_file` entries
configWithoutEnvFiles, err := Load(configDetails, WithDiscardEnvFiles)
assert.NilError(t, err)
assert.DeepEqual(t, configWithoutEnvFiles.Services[0].EnvFile, types.StringList(nil))
assert.DeepEqual(t, configWithoutEnvFiles.Services[0].Environment, expectedEnvironmentMap)
}
func TestBuildProperties(t *testing.T) {
dict, err := ParseYAML([]byte(`
version: "3"

View File

@ -550,17 +550,18 @@ __docker_complete_nodes() {
# output to the IDs or names of matching items. This setting takes
# precedence over the environment setting.
__docker_services() {
local fields='$2' # default: service name only
[ "${DOCKER_COMPLETION_SHOW_SERVICE_IDS}" = yes ] && fields='$1,$2' # ID & name
local format='{{.Name}}' # default: service name only
[ "${DOCKER_COMPLETION_SHOW_SERVICE_IDS}" = yes ] && format='{{.ID}} {{.Name}}' # ID & name
if [ "$1" = "--id" ] ; then
fields='$1' # IDs only
format='{{.ID}}' # IDs only
shift
elif [ "$1" = "--name" ] ; then
fields='$2' # names only
format='{{.Name}}' # names only
shift
fi
__docker_q service ls "$@" | awk "NR>1 {print $fields}"
__docker_q service ls --quiet --format "$format" "$@"
}
# __docker_complete_services applies completion of services based on the current
@ -572,7 +573,7 @@ __docker_complete_services() {
current="$2"
shift 2
fi
COMPREPLY=( $(compgen -W "$(__docker_services "$@")" -- "$current") )
COMPREPLY=( $(__docker_services "$@" --filter "name=$current") )
}
# __docker_tasks returns a list of all task IDs.
@ -1204,6 +1205,7 @@ _docker_build() {
_docker_builder() {
local subcommands="
build
prune
"
__docker_subcommands "$subcommands" && return
@ -1218,6 +1220,10 @@ _docker_builder() {
esac
}
_docker_builder_build() {
_docker_image_build
}
_docker_builder_prune() {
case "$prev" in
--filter)

View File

@ -16,7 +16,7 @@
function __fish_docker_no_subcommand --description 'Test if docker has yet to be given the subcommand'
for i in (commandline -opc)
if contains -- $i attach build commit cp create diff events exec export history images import info inspect kill load login logout logs pause port ps pull push rename restart rm rmi run save search start stop tag top trust unpause version wait stats
if contains -- $i attach build commit cp create diff events exec export history images import info inspect kill load login logout logs network pause port ps pull push rename restart rm rmi run save search start stop tag top trust unpause version wait stats
return 1
end
end
@ -34,6 +34,11 @@ function __fish_print_docker_containers --description 'Print a list of docker co
end
end
function __fish_print_docker_networks --description 'Print a list of docker networks'
docker network ls --format "{{.ID}}\n{{.Name}}" | tr ',' '\n'
end
function __fish_docker_no_subcommand_trust --description 'Test if docker has yet to be given the trust subcommand'
if __fish_seen_subcommand_from trust
for i in (commandline -opc)
@ -370,6 +375,21 @@ complete -c docker -A -f -n '__fish_seen_subcommand_from logs' -l since -d 'Show
complete -c docker -A -f -n '__fish_seen_subcommand_from logs' -l tail -d 'Output the specified number of lines at the end of logs (defaults to all logs)'
complete -c docker -A -f -n '__fish_seen_subcommand_from logs' -a '(__fish_print_docker_containers running)' -d "Container"
# network
complete -c docker -f -n '__fish_docker_no_subcommand' -a network -d 'Manage networks'
complete -c docker -A -f -n '__fish_seen_subcommand_from network' -a connect -d 'Connect a container to a network'
complete -c docker -A -f -n '__fish_seen_subcommand_from network' -a create -d 'Create a network'
complete -c docker -A -f -n '__fish_seen_subcommand_from network' -a disconnect -d 'Disconnect a container from a network'
complete -c docker -A -f -n '__fish_seen_subcommand_from network' -a inspect -d 'Display detailed information on one or more networks'
complete -c docker -A -f -n '__fish_seen_subcommand_from network' -a ls -d 'List networks'
complete -c docker -A -f -n '__fish_seen_subcommand_from network' -a prune -d 'Remove all unused networks'
complete -c docker -A -f -n '__fish_seen_subcommand_from network' -a rm -d 'Remove one or more networks'
complete -c docker -A -f -n '__fish_seen_subcommand_from network' -l help -d 'Print usage'
complete -c docker -A -f -n '__fish_seen_subcommand_from network rm' -a '(__fish_print_docker_networks)' -d "Network"
complete -c docker -A -f -n '__fish_seen_subcommand_from network connect' -a '(__fish_print_docker_networks)' -d "Network"
complete -c docker -A -f -n '__fish_seen_subcommand_from network disconnect' -a '(__fish_print_docker_networks)' -d "Network"
complete -c docker -A -f -n '__fish_seen_subcommand_from network inspect' -a '(__fish_print_docker_networks)' -d "Network"
# port
complete -c docker -f -n '__fish_docker_no_subcommand' -a port -d 'Lookup the public-facing port that is NAT-ed to PRIVATE_PORT'
complete -c docker -A -f -n '__fish_seen_subcommand_from port' -l help -d 'Print usage'

View File

@ -1,4 +1,4 @@
ARG GO_VERSION=1.12.8
ARG GO_VERSION=1.12.12
FROM golang:${GO_VERSION}-alpine

View File

@ -1,4 +1,4 @@
ARG GO_VERSION=1.12.8
ARG GO_VERSION=1.12.12
FROM dockercore/golang-cross:${GO_VERSION}
ENV DISABLE_WARN_OUTSIDE_CONTAINER=1

View File

@ -1,10 +1,10 @@
ARG GO_VERSION=1.12.8
ARG GO_VERSION=1.12.12
FROM golang:${GO_VERSION}-alpine
RUN apk add -U git make bash coreutils ca-certificates curl
ARG VNDR_SHA=1fc68ee0c852556a9ed53cbde16247033f104111
ARG VNDR_SHA=b177b583eb9d44bd5abfca3083a4aeb971b75861
RUN go get -d github.com/LK4D4/vndr && \
cd /go/src/github.com/LK4D4/vndr && \
git checkout -q "$VNDR_SHA" && \

View File

@ -1,4 +1,4 @@
ARG GO_VERSION=1.12.8
ARG GO_VERSION=1.12.12
# Use Debian based image as docker-compose requires glibc.
FROM golang:${GO_VERSION}

View File

@ -1,4 +1,4 @@
ARG GO_VERSION=1.12.8
ARG GO_VERSION=1.12.12
FROM golang:${GO_VERSION}-alpine

View File

@ -157,10 +157,12 @@ be UPPERCASE to distinguish them from arguments more easily.
Docker runs instructions in a `Dockerfile` in order. A `Dockerfile` **must
start with a \`FROM\` instruction**. The `FROM` instruction specifies the [*Base
Image*](glossary.md#base-image) from which you are building. `FROM` may only be
preceded by one or more `ARG` instructions, which declare arguments that are used
in `FROM` lines in the `Dockerfile`.
begin with a \`FROM\` instruction**. This may be after [parser
directives](#parser-directives), [comments](#format), and globally scoped
[ARGs](#arg). The `FROM` instruction specifies the [*Parent
Image*](glossary.md#parent-image) from which you are building. `FROM`
may only be preceded by one or more `ARG` instructions, which declare arguments
that are used in `FROM` lines in the `Dockerfile`.
Docker treats lines that *begin* with `#` as a comment, unless the line is
a valid [parser directive](#parser-directives). A `#` marker anywhere
@ -1720,8 +1722,8 @@ The following `ARG` variables are set automatically:
* `TARGETVARIANT` - variant component of TARGETPLATFORM
* `BUILDPLATFORM` - platform of the node performing the build.
* `BUILDOS` - OS component of BUILDPLATFORM
* `BUILDARCH` - OS component of BUILDPLATFORM
* `BUILDVARIANT` - OS component of BUILDPLATFORM
* `BUILDARCH` - architecture component of BUILDPLATFORM
* `BUILDVARIANT` - variant component of BUILDPLATFORM
These arguments are defined in the global scope so are not automatically
available inside build stages or for your `RUN` commands. To expose one of

View File

@ -54,7 +54,7 @@ each `docker` command with `sudo`. To avoid having to use `sudo` with the
For more information about installing Docker or `sudo` configuration, refer to
the [installation](https://docs.docker.com/install/) instructions for your operating system.
### Environment variables
## Environment variables
For easy reference, the following list of environment variables are supported
by the `docker` command line:
@ -96,28 +96,43 @@ variables.
### Configuration files
By default, the Docker command line stores its configuration files in a
directory called `.docker` within your `$HOME` directory. However, you can
specify a different location via the `DOCKER_CONFIG` environment variable
or the `--config` command line option. If both are specified, then the
`--config` option overrides the `DOCKER_CONFIG` environment variable.
For example:
docker --config ~/testconfigs/ ps
Instructs Docker to use the configuration files in your `~/testconfigs/`
directory when running the `ps` command.
directory called `.docker` within your `$HOME` directory.
Docker manages most of the files in the configuration directory
and you should not modify them. However, you *can modify* the
and you should not modify them. However, you *can* modify the
`config.json` file to control certain aspects of how the `docker`
command behaves.
Currently, you can modify the `docker` command behavior using environment
You can modify the `docker` command behavior using environment
variables or command-line options. You can also use options within
`config.json` to modify some of the same behavior. When using these
mechanisms, you must keep in mind the order of precedence among them. Command
line options override environment variables and environment variables override
properties you specify in a `config.json` file.
`config.json` to modify some of the same behavior. If an environment variable
and the `--config` flag are set, the flag takes precedent over the environment
variable. Command line options override environment variables and environment
variables override properties you specify in a `config.json` file.
### Change the `.docker` directory
To specify a different directory, use the `DOCKER_CONFIG`
environment variable or the `--config` command line option. If both are
specified, then the `--config` option overrides the `DOCKER_CONFIG` environment
variable. The example below overrides the `docker ps` command using a
`config.json` file located in the `~/testconfigs/` directory.
```bash
$ docker --config ~/testconfigs/ ps
```
This flag only applies to whatever command is being ran. For persistent
configuration, you can set the `DOCKER_CONFIG` environment variable in your
shell (e.g. `~/.profile` or `~/.bashrc`). The example below sets the new
directory to be `HOME/newdir/.docker`.
```bash
echo export DOCKER_CONFIG=$HOME/newdir/.docker > ~/.profile
```
### `config.json` properties
The `config.json` file stores a JSON encoding of several properties:
@ -265,6 +280,31 @@ Following is a sample `config.json` file:
{% endraw %}
```
### Experimental features
Experimental features provide early access to future product functionality.
These features are intended only for testing and feedback as they may change
between releases without warning or can be removed entirely from a future
release.
> Experimental features must not be used in production environments.
{: .warning }
To enable experimental features, edit the `config.json` file and set
`experimental` to `enabled`. The example below enables experimental features
in a `config.json` file that already enables a debug feature.
```json
{
"experimental": "enabled",
"debug": true
}
```
You can also enable experimental features from the Docker Desktop menu. See the
[Docker Desktop Getting Started page](https://docs.docker.com/docker-for-mac#experimental-features)
for more information.
### Notary
If using your own notary server and a self-signed certificate or an internal

View File

@ -27,14 +27,14 @@ Options:
## Description
The `docker pause` command suspends all processes in the specified containers.
On Linux, this uses the cgroups freezer. Traditionally, when suspending a process
On Linux, this uses the freezer cgroup. Traditionally, when suspending a process
the `SIGSTOP` signal is used, which is observable by the process being suspended.
With the cgroups freezer the process is unaware, and unable to capture,
With the freezer cgroup the process is unaware, and unable to capture,
that it is being suspended, and subsequently resumed. On Windows, only Hyper-V
containers can be paused.
See the
[cgroups freezer documentation](https://www.kernel.org/doc/Documentation/cgroup-v1/freezer-subsystem.txt)
[freezer cgroup documentation](https://www.kernel.org/doc/Documentation/cgroup-v1/freezer-subsystem.txt)
for further details.
## Examples

View File

@ -75,6 +75,22 @@ $ docker ps -a
container that exposes TCP ports `100, 101, 102` displays `100-102/tcp` in
the `PORTS` column.
### Show disk usage by container
The `docker ps -s` command displays two different on-disk-sizes for each container:
```bash
$ docker ps -s
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES SIZE SIZE
e90b8831a4b8 nginx "/bin/bash -c 'mkdir " 11 weeks ago Up 4 hours my_nginx 35.58 kB (virtual 109.2 MB)
00c6131c5e30 telegraf:1.5 "/entrypoint.sh" 11 weeks ago Up 11 weeks my_telegraf 0 B (virtual 209.5 MB)
```
* The "size" information shows the amount of data (on disk) that is used for the _writable_ layer of each container
* The "virtual size" is the total amount of disk-space used for the read-only _image_ data used by the container and the writable layer.
For more information, refer to the [container size on disk](https://docs.docker.com/storage/storagedriver/#container-size-on-disk) section.
### Filtering
The filtering flag (`-f` or `--filter`) format is a `key=value` pair. If there is more
@ -431,4 +447,4 @@ a87ecb4f327c com.docker.swarm.node=ubuntu,com.docker.swarm.storage=ssd
01946d9d34d8
c1d3b0166030 com.docker.swarm.node=debian,com.docker.swarm.cpu=6
41d50ecd2f57 com.docker.swarm.node=fedora,com.docker.swarm.cpu=3,com.docker.swarm.storage=ssd
```
```

View File

@ -83,7 +83,7 @@ This example displays images with a name containing 'busybox',
at least 3 stars and the description isn't truncated in the output:
```bash
$ docker search --stars=3 --no-trunc busybox
$ docker search --filter=stars=3 --no-trunc busybox
NAME DESCRIPTION STARS OFFICIAL AUTOMATED
busybox Busybox base image. 325 [OK]
progrium/busybox 50 [OK]

View File

@ -27,10 +27,10 @@ Options:
## Description
The `docker unpause` command un-suspends all processes in the specified containers.
On Linux, it does this using the cgroups freezer.
On Linux, it does this using the freezer cgroup.
See the
[cgroups freezer documentation](https://www.kernel.org/doc/Documentation/cgroup-v1/freezer-subsystem.txt)
[freezer cgroup documentation](https://www.kernel.org/doc/Documentation/cgroup-v1/freezer-subsystem.txt)
for further details.
## Examples

View File

@ -1,11 +1,11 @@
The `docker container pause` command suspends all processes in the specified containers.
On Linux, this uses the cgroups freezer. Traditionally, when suspending a process
On Linux, this uses the freezer cgroup. Traditionally, when suspending a process
the `SIGSTOP` signal is used, which is observable by the process being suspended.
With the cgroups freezer the process is unaware, and unable to capture,
With the freezer cgroup the process is unaware, and unable to capture,
that it is being suspended, and subsequently resumed. On Windows, only Hyper-V
containers can be paused.
See the [cgroups freezer documentation]
See the [freezer cgroup documentation]
(https://www.kernel.org/doc/Documentation/cgroup-v1/freezer-subsystem.txt) for
further details.

View File

@ -1,6 +1,6 @@
The `docker container unpause` command un-suspends all processes in a container.
On Linux, it does this using the cgroups freezer.
On Linux, it does this using the freezer cgroup.
See the [cgroups freezer documentation]
See the [freezer cgroup documentation]
(https://www.kernel.org/doc/Documentation/cgroup-v1/freezer-subsystem.txt) for
further details.

View File

@ -50,9 +50,9 @@ github.com/mattn/go-shellwords a72fbe27a1b0ed0df2f027549450
github.com/matttproud/golang_protobuf_extensions c12348ce28de40eed0136aa2b644d0ee0650e56c # v1.0.1
github.com/Microsoft/go-winio 84b4ab48a50763fe7b3abcef38e5205c12027fac
github.com/Microsoft/hcsshim 672e52e9209d1e53718c1b6a7d68cc9272654ab5
github.com/miekg/pkcs11 6120d95c0e9576ccf4a78ba40855809dca31a9ed
github.com/miekg/pkcs11 cb39313ec884f2cd77f4762875fe96aecf68f8e3 # v1.0.2
github.com/mitchellh/mapstructure f15292f7a699fcc1a38a80977f80a046874ba8ac
github.com/moby/buildkit f238f1efb04f00bf0cc147141fda9ddb55c8bc49
github.com/moby/buildkit ae10b292fefb00e0fbf9fecd1419c5f252e58895
github.com/modern-go/concurrent bacd9c7ef1dd9b15be4a9909b8ac7a4e313eec94 # 1.0.3
github.com/modern-go/reflect2 4b7aa43c6742a2c18fdef89dd197aaae7dac7ccd # 1.0.1
github.com/morikuni/aec 39771216ff4c63d11f5e604076f9c45e8be1067b
@ -73,7 +73,7 @@ github.com/spf13/cobra ef82de70bb3f60c65fb8eebacbb2
github.com/spf13/pflag 4cb166e4f25ac4e8016a3595bbf7ea2e9aa85a2c https://github.com/thaJeztah/pflag.git # temporary fork with https://github.com/spf13/pflag/pull/170 applied, which isn't merged yet upstream
github.com/syndtr/gocapability d98352740cb2c55f81556b63d4a1ec64c5a319c2
github.com/theupdateframework/notary d6e1431feb32348e0650bf7551ac5cffd01d857b # v0.6.1
github.com/tonistiigi/fsutil 3bbb99cdbd76619ab717299830c60f6f2a533a6b
github.com/tonistiigi/fsutil 3d2716dd0a4d06ff854241c7e8b6f3f904e1719f
github.com/tonistiigi/units 6950e57a87eaf136bbe44ef2ec8e75b9e3569de2
github.com/xeipuuv/gojsonpointer 4e3ac2762d5f479393488629ee9370b50873b3a6
github.com/xeipuuv/gojsonreference bd5ef7bd5415a7ac448318e64f11a24cd21e594b
@ -88,7 +88,7 @@ golang.org/x/time fbb02b2291d28baffd63558aa44b
google.golang.org/genproto 02b4e95473316948020af0b7a4f0f22c73929b0e
google.golang.org/grpc 25c4f928eaa6d96443009bd842389fb4fa48664e # v1.20.1
gopkg.in/inf.v0 d2d2541c53f18d2a059457998ce2876cc8e67cbf # v0.9.1
gopkg.in/yaml.v2 5420a8b6744d3b0345ab293f6fcba19c978f1183 # v2.2.1
gopkg.in/yaml.v2 bb4e33bf68bf89cad44d386192cbed201f35b241 # v2.2.3
gotest.tools 1083505acf35a0bd8a696b26837e1fb3187a7a83 # v2.3.0
k8s.io/api 40a48860b5abbba9aa891b02b32da429b08d96a0 # kubernetes-1.14.0
k8s.io/apimachinery d7deff9243b165ee192f5551710ea4285dcfd615 # kubernetes-1.14.0

5
vendor/github.com/gogo/googleapis/go.mod generated vendored Normal file
View File

@ -0,0 +1,5 @@
module github.com/gogo/googleapis
go 1.12
require github.com/gogo/protobuf v1.2.1

3
vendor/github.com/gogo/protobuf/go.mod generated vendored Normal file
View File

@ -0,0 +1,3 @@
module github.com/gogo/protobuf
require github.com/kisielk/errcheck v1.1.0 // indirect

1
vendor/github.com/google/uuid/go.mod generated vendored Normal file
View File

@ -0,0 +1 @@
module github.com/google/uuid

1
vendor/github.com/gorilla/mux/go.mod generated vendored Normal file
View File

@ -0,0 +1 @@
module github.com/gorilla/mux

5
vendor/github.com/jaguilar/vt100/go.mod generated vendored Normal file
View File

@ -0,0 +1,5 @@
module github.com/jaguilar/vt100
go 1.12
require github.com/stretchr/testify v1.3.0

View File

@ -0,0 +1 @@
module github.com/konsorten/go-windows-terminal-sequences

1
vendor/github.com/mattn/go-shellwords/go.mod generated vendored Normal file
View File

@ -0,0 +1 @@
module github.com/mattn/go-shellwords

3
vendor/github.com/miekg/pkcs11/go.mod generated vendored Normal file
View File

@ -0,0 +1,3 @@
module github.com/miekg/pkcs11
go 1.12

View File

@ -42,19 +42,21 @@ type GCMParams struct {
// NewGCMParams returns a pointer to AES-GCM parameters that can be used with the CKM_AES_GCM mechanism.
// The Free() method must be called after the operation is complete.
//
// *NOTE*
// Some HSMs, like CloudHSM, will ignore the IV you pass in and write their
// Note that some HSMs, like CloudHSM, will ignore the IV you pass in and write their
// own. As a result, to support all libraries, memory is not freed
// automatically, so that after the EncryptInit/Encrypt operation the HSM's IV
// can be read back out. It is up to the caller to ensure that Free() is called
// on the GCMParams object at an appropriate time, which is after
//
// Encrypt/Decrypt. As an example:
//
// gcmParams := pkcs11.NewGCMParams(make([]byte, 12), nil, 128)
// p.ctx.EncryptInit(session, []*pkcs11.Mechanism{pkcs11.NewMechanism(pkcs11.CKM_AES_GCM, gcmParams)}, aesObjHandle)
// ct, _ := p.ctx.Encrypt(session, pt)
// iv := gcmParams.IV()
// gcmParams.Free()
// gcmParams := pkcs11.NewGCMParams(make([]byte, 12), nil, 128)
// p.ctx.EncryptInit(session, []*pkcs11.Mechanism{pkcs11.NewMechanism(pkcs11.CKM_AES_GCM, gcmParams)},
// aesObjHandle)
// ct, _ := p.ctx.Encrypt(session, pt)
// iv := gcmParams.IV()
// gcmParams.Free()
//
func NewGCMParams(iv, aad []byte, tagSize int) *GCMParams {
return &GCMParams{
iv: iv,
@ -112,7 +114,7 @@ func (p *GCMParams) Free() {
p.arena = nil
}
// NewPSSParams creates a CK_RSA_PKCS_PSS_PARAMS structure and returns it as a byte array for use with the CKM_RSA_PKCS_PSS mechanism
// NewPSSParams creates a CK_RSA_PKCS_PSS_PARAMS structure and returns it as a byte array for use with the CKM_RSA_PKCS_PSS mechanism.
func NewPSSParams(hashAlg, mgf, saltLength uint) []byte {
p := C.CK_RSA_PKCS_PSS_PARAMS{
hashAlg: C.CK_MECHANISM_TYPE(hashAlg),
@ -122,7 +124,7 @@ func NewPSSParams(hashAlg, mgf, saltLength uint) []byte {
return C.GoBytes(unsafe.Pointer(&p), C.int(unsafe.Sizeof(p)))
}
// OAEPParams can be passed to NewMechanism to implement CKM_RSA_PKCS_OAEP
// OAEPParams can be passed to NewMechanism to implement CKM_RSA_PKCS_OAEP.
type OAEPParams struct {
HashAlg uint
MGF uint
@ -130,7 +132,7 @@ type OAEPParams struct {
SourceData []byte
}
// NewOAEPParams creates a CK_RSA_PKCS_OAEP_PARAMS structure suitable for use with the CKM_RSA_PKCS_OAEP mechanism
// NewOAEPParams creates a CK_RSA_PKCS_OAEP_PARAMS structure suitable for use with the CKM_RSA_PKCS_OAEP mechanism.
func NewOAEPParams(hashAlg, mgf, sourceType uint, sourceData []byte) *OAEPParams {
return &OAEPParams{
HashAlg: hashAlg,
@ -154,14 +156,14 @@ func cOAEPParams(p *OAEPParams, arena arena) ([]byte, arena) {
return C.GoBytes(unsafe.Pointer(&params), C.int(unsafe.Sizeof(params))), arena
}
// ECDH1DeriveParams can be passed to NewMechanism to implement CK_ECDH1_DERIVE_PARAMS
// ECDH1DeriveParams can be passed to NewMechanism to implement CK_ECDH1_DERIVE_PARAMS.
type ECDH1DeriveParams struct {
KDF uint
SharedData []byte
PublicKeyData []byte
}
// NewECDH1DeriveParams creates a CK_ECDH1_DERIVE_PARAMS structure suitable for use with the CKM_ECDH1_DERIVE mechanism
// NewECDH1DeriveParams creates a CK_ECDH1_DERIVE_PARAMS structure suitable for use with the CKM_ECDH1_DERIVE mechanism.
func NewECDH1DeriveParams(kdf uint, sharedData []byte, publicKeyData []byte) *ECDH1DeriveParams {
return &ECDH1DeriveParams{
KDF: kdf,

View File

@ -800,13 +800,13 @@ func (c *Ctx) Destroy() {
c.ctx = nil
}
// Initialize initializes the Cryptoki library. */
// Initialize initializes the Cryptoki library.
func (c *Ctx) Initialize() error {
e := C.Initialize(c.ctx)
return toError(e)
}
// Finalize indicates that an application is done with the Cryptoki library. */
// Finalize indicates that an application is done with the Cryptoki library.
func (c *Ctx) Finalize() error {
if c.ctx == nil {
return toError(CKR_CRYPTOKI_NOT_INITIALIZED)
@ -815,7 +815,7 @@ func (c *Ctx) Finalize() error {
return toError(e)
}
// GetInfo returns general information about Cryptoki. */
// GetInfo returns general information about Cryptoki.
func (c *Ctx) GetInfo() (Info, error) {
var p C.ckInfo
e := C.GetInfo(c.ctx, &p)
@ -829,7 +829,7 @@ func (c *Ctx) GetInfo() (Info, error) {
return i, toError(e)
}
// GetSlotList obtains a list of slots in the system. */
// GetSlotList obtains a list of slots in the system.
func (c *Ctx) GetSlotList(tokenPresent bool) ([]uint, error) {
var (
slotList C.CK_ULONG_PTR
@ -843,7 +843,7 @@ func (c *Ctx) GetSlotList(tokenPresent bool) ([]uint, error) {
return l, nil
}
// GetSlotInfo obtains information about a particular slot in the system. */
// GetSlotInfo obtains information about a particular slot in the system.
func (c *Ctx) GetSlotInfo(slotID uint) (SlotInfo, error) {
var csi C.CK_SLOT_INFO
e := C.GetSlotInfo(c.ctx, C.CK_ULONG(slotID), &csi)
@ -885,7 +885,7 @@ func (c *Ctx) GetTokenInfo(slotID uint) (TokenInfo, error) {
return s, toError(e)
}
// GetMechanismList obtains a list of mechanism types supported by a token. */
// GetMechanismList obtains a list of mechanism types supported by a token.
func (c *Ctx) GetMechanismList(slotID uint) ([]*Mechanism, error) {
var (
mech C.CK_ULONG_PTR // in pkcs#11 we're all CK_ULONGs \o/
@ -997,11 +997,11 @@ func (c *Ctx) GetOperationState(sh SessionHandle) ([]byte, error) {
statelen C.CK_ULONG
)
e := C.GetOperationState(c.ctx, C.CK_SESSION_HANDLE(sh), &state, &statelen)
defer C.free(unsafe.Pointer(state))
if toError(e) != nil {
return nil, toError(e)
}
b := C.GoBytes(unsafe.Pointer(state), C.int(statelen))
C.free(unsafe.Pointer(state))
return b, nil
}

17
vendor/github.com/miekg/pkcs11/release.go generated vendored Normal file
View File

@ -0,0 +1,17 @@
// +build release
package pkcs11
import "fmt"
// Release is current version of the pkcs11 library.
var Release = R{1, 0, 2}
// R holds the version of this library.
type R struct {
Major, Minor, Patch int
}
func (r R) String() string {
return fmt.Sprintf("%d.%d.%d", r.Major, r.Minor, r.Patch)
}

View File

@ -1,27 +1,25 @@
[![asciicinema example](https://asciinema.org/a/gPEIEo1NzmDTUu2bEPsUboqmU.png)](https://asciinema.org/a/gPEIEo1NzmDTUu2bEPsUboqmU)
## BuildKit
[![GoDoc](https://godoc.org/github.com/moby/buildkit?status.svg)](https://godoc.org/github.com/moby/buildkit/client/llb)
[![Build Status](https://travis-ci.org/moby/buildkit.svg?branch=master)](https://travis-ci.org/moby/buildkit)
[![Go Report Card](https://goreportcard.com/badge/github.com/moby/buildkit)](https://goreportcard.com/report/github.com/moby/buildkit)
BuildKit is a toolkit for converting source code to build artifacts in an efficient, expressive and repeatable manner.
Key features:
- Automatic garbage collection
- Extendable frontend formats
- Concurrent dependency resolution
- Efficient instruction caching
- Build cache import/export
- Nested build job invocations
- Distributable workers
- Multiple output formats
- Pluggable architecture
- Execution without root privileges
- Automatic garbage collection
- Extendable frontend formats
- Concurrent dependency resolution
- Efficient instruction caching
- Build cache import/export
- Nested build job invocations
- Distributable workers
- Multiple output formats
- Pluggable architecture
- Execution without root privileges
Read the proposal from https://github.com/moby/moby/issues/32925
@ -33,20 +31,21 @@ Introductory blog post https://blog.mobyproject.org/introducing-buildkit-17e056c
BuildKit is used by the following projects:
- [Moby & Docker](https://github.com/moby/moby/pull/37151)
- [img](https://github.com/genuinetools/img)
- [OpenFaaS Cloud](https://github.com/openfaas/openfaas-cloud)
- [container build interface](https://github.com/containerbuilding/cbi)
- [Knative Build Templates](https://github.com/knative/build-templates)
- [vab](https://github.com/stellarproject/vab)
- [Rio](https://github.com/rancher/rio) (on roadmap)
- [Moby & Docker](https://github.com/moby/moby/pull/37151)
- [img](https://github.com/genuinetools/img)
- [OpenFaaS Cloud](https://github.com/openfaas/openfaas-cloud)
- [container build interface](https://github.com/containerbuilding/cbi)
- [Knative Build Templates](https://github.com/knative/build-templates)
- [the Sanic build tool](https://github.com/distributed-containers-inc/sanic)
- [vab](https://github.com/stellarproject/vab)
- [Rio](https://github.com/rancher/rio) (on roadmap)
### Quick start
Dependencies:
- [runc](https://github.com/opencontainers/runc)
- [containerd](https://github.com/containerd/containerd) (if you want to use containerd worker)
- [runc](https://github.com/opencontainers/runc)
- [containerd](https://github.com/containerd/containerd) (if you want to use containerd worker)
The following command installs `buildkitd` and `buildctl` to `/usr/local/bin`:
@ -58,14 +57,13 @@ You can also use `make binaries-all` to prepare `buildkitd.containerd_only` and
#### Starting the buildkitd daemon:
```
```bash
buildkitd --debug --root /var/lib/buildkit
```
The buildkitd daemon supports two worker backends: OCI (runc) and containerd.
By default, the OCI (runc) worker is used.
You can set `--oci-worker=false --containerd-worker=true` to use the containerd worker.
By default, the OCI (runc) worker is used. You can set `--oci-worker=false --containerd-worker=true` to use the containerd worker.
We are open to adding more backends.
@ -73,44 +71,46 @@ We are open to adding more backends.
BuildKit builds are based on a binary intermediate format called LLB that is used for defining the dependency graph for processes running part of your build. tl;dr: LLB is to Dockerfile what LLVM IR is to C.
- Marshaled as Protobuf messages
- Concurrently executable
- Efficiently cacheable
- Vendor-neutral (i.e. non-Dockerfile languages can be easily implemented)
- Marshaled as Protobuf messages
- Concurrently executable
- Efficiently cacheable
- Vendor-neutral (i.e. non-Dockerfile languages can be easily implemented)
See [`solver/pb/ops.proto`](./solver/pb/ops.proto) for the format definition.
Currently, following high-level languages has been implemented for LLB:
- Dockerfile (See [Exploring Dockerfiles](#exploring-dockerfiles))
- [Buildpacks](https://github.com/tonistiigi/buildkit-pack)
- (open a PR to add your own language)
- Dockerfile (See [Exploring Dockerfiles](#exploring-dockerfiles))
- [Buildpacks](https://github.com/tonistiigi/buildkit-pack)
- (open a PR to add your own language)
For understanding the basics of LLB, `examples/buildkit*` directory contains scripts that define how to build different configurations of BuildKit itself and its dependencies using the `client` package. Running one of these scripts generates a protobuf definition of a build graph. Note that the script itself does not execute any steps of the build.
You can use `buildctl debug dump-llb` to see what data is in this definition. Add `--dot` to generate dot layout.
```bash
go run examples/buildkit0/buildkit.go | buildctl debug dump-llb | jq .
go run examples/buildkit0/buildkit.go \
| buildctl debug dump-llb \
| jq .
```
To start building use `buildctl build` command. The example script accepts `--with-containerd` flag to choose if containerd binaries and support should be included in the end result as well.
To start building use `buildctl build` command. The example script accepts `--with-containerd` flag to choose if containerd binaries and support should be included in the end result as well.
```bash
go run examples/buildkit0/buildkit.go | buildctl build
go run examples/buildkit0/buildkit.go \
| buildctl build
```
`buildctl build` will show interactive progress bar by default while the build job is running. If the path to the trace file is specified, the trace file generated will contain all information about the timing of the individual steps and logs.
Different versions of the example scripts show different ways of describing the build definition for this project to show the capabilities of the library. New versions have been added when new features have become available.
- `./examples/buildkit0` - uses only exec operations, defines a full stage per component.
- `./examples/buildkit1` - cloning git repositories has been separated for extra concurrency.
- `./examples/buildkit2` - uses git sources directly instead of running `git clone`, allowing better performance and much safer caching.
- `./examples/buildkit3` - allows using local source files for separate components eg. `./buildkit3 --runc=local | buildctl build --local runc-src=some/local/path`
- `./examples/dockerfile2llb` - can be used to convert a Dockerfile to LLB for debugging purposes
- `./examples/gobuild` - shows how to use nested invocation to generate LLB for Go package internal dependencies
- `./examples/buildkit0` - uses only exec operations, defines a full stage per component.
- `./examples/buildkit1` - cloning git repositories has been separated for extra concurrency.
- `./examples/buildkit2` - uses git sources directly instead of running `git clone`, allowing better performance and much safer caching.
- `./examples/buildkit3` - allows using local source files for separate components eg. `./buildkit3 --runc=local | buildctl build --local runc-src=some/local/path`
- `./examples/dockerfile2llb` - can be used to convert a Dockerfile to LLB for debugging purposes
- `./examples/gobuild` - shows how to use nested invocation to generate LLB for Go package internal dependencies
#### Exploring Dockerfiles
@ -120,9 +120,18 @@ During development, Dockerfile frontend (dockerfile.v0) is also part of the Buil
##### Building a Dockerfile with `buildctl`
```
buildctl build --frontend=dockerfile.v0 --local context=. --local dockerfile=.
buildctl build --frontend=dockerfile.v0 --local context=. --local dockerfile=. --opt target=foo --opt build-arg:foo=bar
```bash
buildctl build \
--frontend=dockerfile.v0 \
--local context=. \
--local dockerfile=.
# or
buildctl build \
--frontend=dockerfile.v0 \
--local context=. \
--local dockerfile=. \
--opt target=foo \
--opt build-arg:foo=bar
```
`--local` exposes local source files from client to the builder. `context` and `dockerfile` are the names Dockerfile frontend looks for build context and Dockerfile location.
@ -131,8 +140,9 @@ buildctl build --frontend=dockerfile.v0 --local context=. --local dockerfile=. -
For people familiar with `docker build` command, there is an example wrapper utility in `./examples/build-using-dockerfile` that allows building Dockerfiles with BuildKit using a syntax similar to `docker build`.
```
go build ./examples/build-using-dockerfile && sudo install build-using-dockerfile /usr/local/bin
```bash
go build ./examples/build-using-dockerfile \
&& sudo install build-using-dockerfile /usr/local/bin
build-using-dockerfile -t myimage .
build-using-dockerfile -t mybuildkit -f ./hack/dockerfiles/test.Dockerfile .
@ -145,10 +155,18 @@ docker inspect myimage
External versions of the Dockerfile frontend are pushed to https://hub.docker.com/r/docker/dockerfile-upstream and https://hub.docker.com/r/docker/dockerfile and can be used with the gateway frontend. The source for the external frontend is currently located in `./frontend/dockerfile/cmd/dockerfile-frontend` but will move out of this repository in the future ([#163](https://github.com/moby/buildkit/issues/163)). For automatic build from master branch of this repository `docker/dockerfile-upsteam:master` or `docker/dockerfile-upstream:master-experimental` image can be used.
```bash
buildctl build \
--frontend gateway.v0 \
--opt source=docker/dockerfile \
--local context=. \
--local dockerfile=.
buildctl build \
--frontend gateway.v0 \
--opt source=docker/dockerfile \
--opt context=git://github.com/moby/moby \
--opt build-arg:APT_MIRROR=cdn-fastly.deb.debian.org
```
buildctl build --frontend gateway.v0 --opt source=docker/dockerfile --local context=. --local dockerfile=.
buildctl build --frontend gateway.v0 --opt source=docker/dockerfile --opt context=git://github.com/moby/moby --opt build-arg:APT_MIRROR=cdn-fastly.deb.debian.org
````
##### Building a Dockerfile with experimental features like `RUN --mount=type=(bind|cache|tmpfs|secret|ssh)`
@ -162,46 +180,58 @@ By default, the build result and intermediate cache will only remain internally
The containerd worker needs to be used
```
```bash
buildctl build ... --output type=image,name=docker.io/username/image
ctr --namespace=buildkit images ls
```
##### Push resulting image to registry
```
```bash
buildctl build ... --output type=image,name=docker.io/username/image,push=true
```
If credentials are required, `buildctl` will attempt to read Docker configuration file.
##### Exporting build result back to client
The local client will copy the files directly to the client. This is useful if BuildKit is being used for building something else than container images.
```
```bash
buildctl build ... --output type=local,dest=path/to/output-dir
```
To export specific files use multi-stage builds with a scratch stage and copy the needed files into that stage with `COPY --from`.
```dockerfile
...
FROM scratch as testresult
COPY --from=builder /usr/src/app/testresult.xml .
...
```
```bash
buildctl build ... --opt target=testresult --output type=local,dest=path/to/output-dir
```
Tar exporter is similar to local exporter but transfers the files through a tarball.
```
```bash
buildctl build ... --output type=tar,dest=out.tar
buildctl build ... --output type=tar > out.tar
```
##### Exporting built image to Docker
```
```bash
# exported tarball is also compatible with OCI spec
buildctl build ... --output type=docker,name=myimage | docker load
```
##### Exporting [OCI Image Format](https://github.com/opencontainers/image-spec) tarball to client
```
```bash
buildctl build ... --output type=oci,dest=path/to/output.tar
buildctl build ... --output type=oci > output.tar
```
@ -210,14 +240,14 @@ buildctl build ... --output type=oci > output.tar
#### To/From registry
```
```bash
buildctl build ... --export-cache type=registry,ref=localhost:5000/myrepo:buildcache
buildctl build ... --import-cache type=registry,ref=localhost:5000/myrepo:buildcache
```
#### To/From local filesystem
```
```bash
buildctl build ... --export-cache type=local,dest=path/to/output-dir
buildctl build ... --import-cache type=local,src=path/to/input-dir
```
@ -225,27 +255,29 @@ buildctl build ... --import-cache type=local,src=path/to/input-dir
The directory layout conforms to OCI Image Spec v1.0.
#### `--export-cache` options
* `mode=min` (default): only export layers for the resulting image
* `mode=max`: export all the layers of all intermediate steps
* `ref=docker.io/user/image:tag`: reference for `registry` cache exporter
* `dest=path/to/output-dir`: directory for `local` cache exporter
- `mode=min` (default): only export layers for the resulting image
- `mode=max`: export all the layers of all intermediate steps
- `ref=docker.io/user/image:tag`: reference for `registry` cache exporter
- `dest=path/to/output-dir`: directory for `local` cache exporter
#### `--import-cache` options
* `ref=docker.io/user/image:tag`: reference for `registry` cache importer
* `src=path/to/input-dir`: directory for `local` cache importer
* `digest=sha256:deadbeef`: digest of the manifest list to import for `local` cache importer. Defaults to the digest of "latest" tag in `index.json`
- `ref=docker.io/user/image:tag`: reference for `registry` cache importer
- `src=path/to/input-dir`: directory for `local` cache importer
- `digest=sha256:deadbeef`: digest of the manifest list to import for `local` cache importer. Defaults to the digest of "latest" tag in `index.json`
### Other
#### View build cache
```
```bash
buildctl du -v
```
#### Show enabled workers
```
```bash
buildctl debug workers -v
```
@ -255,31 +287,65 @@ BuildKit can also be used by running the `buildkitd` daemon inside a Docker cont
We provide `buildkitd` container images as [`moby/buildkit`](https://hub.docker.com/r/moby/buildkit/tags/):
* `moby/buildkit:latest`: built from the latest regular [release](https://github.com/moby/buildkit/releases)
* `moby/buildkit:rootless`: same as `latest` but runs as an unprivileged user, see [`docs/rootless.md`](docs/rootless.md)
* `moby/buildkit:master`: built from the master branch
* `moby/buildkit:master-rootless`: same as master but runs as an unprivileged user, see [`docs/rootless.md`](docs/rootless.md)
- `moby/buildkit:latest`: built from the latest regular [release](https://github.com/moby/buildkit/releases)
- `moby/buildkit:rootless`: same as `latest` but runs as an unprivileged user, see [`docs/rootless.md`](docs/rootless.md)
- `moby/buildkit:master`: built from the master branch
- `moby/buildkit:master-rootless`: same as master but runs as an unprivileged user, see [`docs/rootless.md`](docs/rootless.md)
To run daemon in a container:
```
```bash
docker run -d --privileged -p 1234:1234 moby/buildkit:latest --addr tcp://0.0.0.0:1234
export BUILDKIT_HOST=tcp://0.0.0.0:1234
buildctl build --help
```
The images can be also built locally using `./hack/dockerfiles/test.Dockerfile` (or `./hack/dockerfiles/test.buildkit.Dockerfile` if you already have BuildKit).
Run `make images` to build the images as `moby/buildkit:local` and `moby/buildkit:local-rootless`.
To run client and an ephemeral daemon in a single container ("daemonless mode"):
```bash
docker run \
-it \
--rm \
--privileged \
-v /path/to/dir:/tmp/work \
--entrypoint buildctl-daemonless.sh \
moby/buildkit:master \
build \
--frontend dockerfile.v0 \
--local context=/tmp/work \
--local dockerfile=/tmp/work
```
or
```bash
docker run \
-it \
--rm \
--security-opt seccomp=unconfined \
--security-opt apparmor=unconfined \
-e BUILDKITD_FLAGS=--oci-worker-no-process-sandbox \
-v /path/to/dir:/tmp/work \
--entrypoint buildctl-daemonless.sh \
moby/buildkit:master-rootless \
build \
--frontend \
dockerfile.v0 \
--local context=/tmp/work \
--local dockerfile=/tmp/work
```
The images can be also built locally using `./hack/dockerfiles/test.Dockerfile` (or `./hack/dockerfiles/test.buildkit.Dockerfile` if you already have BuildKit). Run `make images` to build the images as `moby/buildkit:local` and `moby/buildkit:local-rootless`.
#### Connection helpers
If you are running `moby/buildkit:master` or `moby/buildkit:master-rootless` as a Docker/Kubernetes container, you can use special `BUILDKIT_HOST` URL for connecting to the BuildKit daemon in the container:
```
export BUILDKIT_HOST=docker://<container>
```bash
export BUILDKIT_HOST=docker-container://<container>
```
```
```bash
export BUILDKIT_HOST=kube-pod://<pod>
```
@ -287,15 +353,13 @@ export BUILDKIT_HOST=kube-pod://<pod>
BuildKit supports opentracing for buildkitd gRPC API and buildctl commands. To capture the trace to [Jaeger](https://github.com/jaegertracing/jaeger), set `JAEGER_TRACE` environment variable to the collection address.
```
```bash
docker run -d -p6831:6831/udp -p16686:16686 jaegertracing/all-in-one:latest
export JAEGER_TRACE=0.0.0.0:6831
# restart buildkitd and buildctl so they know JAEGER_TRACE
# any buildctl command should be traced to http://127.0.0.1:16686/
```
### Supported runc version
During development, BuildKit is tested with the version of runc that is being used by the containerd repository. Please refer to [runc.md](https://github.com/containerd/containerd/blob/v1.2.1/RUNC.md) for more information.
@ -306,5 +370,4 @@ Please refer to [`docs/rootless.md`](docs/rootless.md).
### Contributing
Want to contribute to BuildKit? Awesome! You can find information about
contributing to this project in the [CONTRIBUTING.md](/.github/CONTRIBUTING.md)
Want to contribute to BuildKit? Awesome! You can find information about contributing to this project in the [CONTRIBUTING.md](/.github/CONTRIBUTING.md)

View File

@ -427,11 +427,13 @@ func Security(s pb.SecurityMode) RunOption {
}
func Shlex(str string) RunOption {
return Shlexf(str)
return runOptionFunc(func(ei *ExecInfo) {
ei.State = shlexf(str, false)(ei.State)
})
}
func Shlexf(str string, v ...interface{}) RunOption {
return runOptionFunc(func(ei *ExecInfo) {
ei.State = shlexf(str, v...)(ei.State)
ei.State = shlexf(str, true, v...)(ei.State)
})
}
@ -442,7 +444,9 @@ func Args(a []string) RunOption {
}
func AddEnv(key, value string) RunOption {
return AddEnvf(key, value)
return runOptionFunc(func(ei *ExecInfo) {
ei.State = ei.State.AddEnv(key, value)
})
}
func AddEnvf(key, value string, v ...interface{}) RunOption {
@ -458,7 +462,9 @@ func User(str string) RunOption {
}
func Dir(str string) RunOption {
return Dirf(str)
return runOptionFunc(func(ei *ExecInfo) {
ei.State = ei.State.Dir(str)
})
}
func Dirf(str string, v ...interface{}) RunOption {
return runOptionFunc(func(ei *ExecInfo) {

View File

@ -24,19 +24,24 @@ var (
keySecurity = contextKeyT("llb.security")
)
func addEnvf(key, value string, v ...interface{}) StateOption {
func addEnvf(key, value string, replace bool, v ...interface{}) StateOption {
if replace {
value = fmt.Sprintf(value, v...)
}
return func(s State) State {
return s.WithValue(keyEnv, getEnv(s).AddOrReplace(key, fmt.Sprintf(value, v...)))
return s.WithValue(keyEnv, getEnv(s).AddOrReplace(key, value))
}
}
func dir(str string) StateOption {
return dirf(str)
return dirf(str, false)
}
func dirf(str string, v ...interface{}) StateOption {
func dirf(value string, replace bool, v ...interface{}) StateOption {
if replace {
value = fmt.Sprintf(value, v...)
}
return func(s State) State {
value := fmt.Sprintf(str, v...)
if !path.IsAbs(value) {
prev := getDir(s)
if prev == "" {
@ -100,9 +105,12 @@ func args(args ...string) StateOption {
}
}
func shlexf(str string, v ...interface{}) StateOption {
func shlexf(str string, replace bool, v ...interface{}) StateOption {
if replace {
str = fmt.Sprintf(str, v...)
}
return func(s State) State {
arg, err := shlex.Split(fmt.Sprintf(str, v...))
arg, err := shlex.Split(str)
if err != nil {
// TODO: handle error
}

View File

@ -240,18 +240,18 @@ func (s State) File(a *FileAction, opts ...ConstraintsOpt) State {
}
func (s State) AddEnv(key, value string) State {
return s.AddEnvf(key, value)
return addEnvf(key, value, false)(s)
}
func (s State) AddEnvf(key, value string, v ...interface{}) State {
return addEnvf(key, value, v...)(s)
return addEnvf(key, value, true, v...)(s)
}
func (s State) Dir(str string) State {
return s.Dirf(str)
return dirf(str, false)(s)
}
func (s State) Dirf(str string, v ...interface{}) State {
return dirf(str, v...)(s)
return dirf(str, true, v...)(s)
}
func (s State) GetEnv(key string) (string, bool) {

View File

@ -46,8 +46,8 @@ type SolveOpt struct {
type ExportEntry struct {
Type string
Attrs map[string]string
Output io.WriteCloser // for ExporterOCI and ExporterDocker
OutputDir string // for ExporterLocal
Output func(map[string]string) (io.WriteCloser, error) // for ExporterOCI and ExporterDocker
OutputDir string // for ExporterLocal
}
type CacheOptionsEntry struct {

View File

@ -128,7 +128,7 @@ func (c *grpcClient) Run(ctx context.Context, f client.BuildFunc) (retError erro
}
}
if retError != nil {
st, _ := status.FromError(retError)
st, _ := status.FromError(errors.Cause(retError))
stp := st.Proto()
req.Error = &rpc.Status{
Code: stp.Code,

77
vendor/github.com/moby/buildkit/go.mod generated vendored Normal file
View File

@ -0,0 +1,77 @@
module github.com/moby/buildkit
go 1.11
require (
github.com/BurntSushi/toml v0.3.1
github.com/Microsoft/go-winio v0.4.13-0.20190408173621-84b4ab48a507
github.com/apache/thrift v0.0.0-20161221203622-b2a4d4ae21c7 // indirect
github.com/codahale/hdrhistogram v0.0.0-20160425231609-f8ad88b59a58 // indirect
github.com/containerd/cgroups v0.0.0-20190226200435-dbea6f2bd416 // indirect
github.com/containerd/console v0.0.0-20181022165439-0650fd9eeb50
github.com/containerd/containerd v1.3.0-0.20190507210959-7c1e88399ec0
github.com/containerd/continuity v0.0.0-20190827140505-75bee3e2ccb6
github.com/containerd/fifo v0.0.0-20180307165137-3d5202aec260 // indirect
github.com/containerd/go-cni v0.0.0-20190610170741-5a4663dad645
github.com/containerd/go-runc v0.0.0-20190911050354-e029b79d8cda
github.com/containerd/ttrpc v0.0.0-20190411181408-699c4e40d1e7 // indirect
github.com/containerd/typeurl v0.0.0-20180627222232-a93fcdb778cd // indirect
github.com/containernetworking/cni v0.6.1-0.20180218032124-142cde0c766c // indirect
github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e
github.com/docker/cli v0.0.0-20190321234815-f40f9c240ab0
github.com/docker/distribution v2.7.1-0.20190205005809-0d3efadf0154+incompatible
github.com/docker/docker v1.14.0-0.20190319215453-e7b5f7dbe98c
github.com/docker/docker-credential-helpers v0.6.0 // indirect
github.com/docker/go-connections v0.3.0
github.com/docker/go-events v0.0.0-20170721190031-9461782956ad // indirect
github.com/docker/libnetwork v0.8.0-dev.2.0.20190604151032-3c26b4e7495e
github.com/godbus/dbus v4.1.0+incompatible // indirect
github.com/gofrs/flock v0.7.0
github.com/gogo/googleapis v1.1.0
github.com/gogo/protobuf v1.2.0
github.com/golang/protobuf v1.2.0
github.com/google/go-cmp v0.2.0
github.com/google/shlex v0.0.0-20150127133951-6f45313302b9
github.com/grpc-ecosystem/grpc-opentracing v0.0.0-20180507213350-8e809c8a8645
github.com/hashicorp/go-immutable-radix v1.0.0
github.com/hashicorp/golang-lru v0.0.0-20160207214719-a0d98a5f2880
github.com/hashicorp/uuid v0.0.0-20160311170451-ebb0a03e909c // indirect
github.com/ishidawataru/sctp v0.0.0-20180213033435-07191f837fed // indirect
github.com/jaguilar/vt100 v0.0.0-20150826170717-2703a27b14ea
github.com/kr/pretty v0.1.0 // indirect
github.com/mitchellh/hashstructure v0.0.0-20170609045927-2bca23e0e452
github.com/morikuni/aec v0.0.0-20170113033406-39771216ff4c
github.com/opencontainers/go-digest v1.0.0-rc1
github.com/opencontainers/image-spec v1.0.1
github.com/opencontainers/runc v1.0.0-rc8
github.com/opencontainers/runtime-spec v0.0.0-20180909173843-eba862dc2470
github.com/opentracing-contrib/go-stdlib v0.0.0-20171029140428-b1a47cfbdd75
github.com/opentracing/opentracing-go v0.0.0-20171003133519-1361b9cd60be
github.com/pkg/errors v0.8.1
github.com/pkg/profile v1.2.1
github.com/serialx/hashring v0.0.0-20190422032157-8b2912629002
github.com/sirupsen/logrus v1.3.0
github.com/stretchr/testify v1.3.0
github.com/syndtr/gocapability v0.0.0-20180916011248-d98352740cb2 // indirect
github.com/tonistiigi/fsutil v0.0.0-20190819224149-3d2716dd0a4d
github.com/tonistiigi/units v0.0.0-20180711220420-6950e57a87ea
github.com/uber/jaeger-client-go v0.0.0-20180103221425-e02c85f9069e
github.com/uber/jaeger-lib v1.2.1 // indirect
github.com/urfave/cli v0.0.0-20171014202726-7bc6a0acffa5
github.com/vishvananda/netlink v1.0.0 // indirect
github.com/vishvananda/netns v0.0.0-20180720170159-13995c7128cc // indirect
go.etcd.io/bbolt v1.3.2
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2
golang.org/x/net v0.0.0-20190311183353-d8887717615a
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
golang.org/x/sys v0.0.0-20190303122642-d455e41777fc
golang.org/x/time v0.0.0-20161028155119-f51c12702a4d
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8
google.golang.org/grpc v1.20.1
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 // indirect
gotest.tools v2.2.0+incompatible
)
replace github.com/hashicorp/go-immutable-radix => github.com/tonistiigi/go-immutable-radix v0.0.0-20170803185627-826af9ccf0fe
replace github.com/jaguilar/vt100 => github.com/tonistiigi/vt100 v0.0.0-20190402012908-ad4c4a574305

View File

@ -4,6 +4,7 @@ import (
"context"
"github.com/moby/buildkit/session"
"github.com/pkg/errors"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)
@ -16,10 +17,10 @@ func CredentialsFunc(ctx context.Context, c session.Caller) func(string) (string
Host: host,
})
if err != nil {
if st, ok := status.FromError(err); ok && st.Code() == codes.Unimplemented {
if st, ok := status.FromError(errors.Cause(err)); ok && st.Code() == codes.Unimplemented {
return "", "", nil
}
return "", "", err
return "", "", errors.WithStack(err)
}
return resp.Username, resp.Secret, nil
}

View File

@ -9,6 +9,7 @@ import (
"github.com/moby/buildkit/session"
digest "github.com/opencontainers/go-digest"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
"google.golang.org/grpc/metadata"
)
@ -31,47 +32,53 @@ func (cs *callerContentStore) choose(ctx context.Context) context.Context {
func (cs *callerContentStore) Info(ctx context.Context, dgst digest.Digest) (content.Info, error) {
ctx = cs.choose(ctx)
return cs.store.Info(ctx, dgst)
info, err := cs.store.Info(ctx, dgst)
return info, errors.WithStack(err)
}
func (cs *callerContentStore) Update(ctx context.Context, info content.Info, fieldpaths ...string) (content.Info, error) {
ctx = cs.choose(ctx)
return cs.store.Update(ctx, info, fieldpaths...)
info, err := cs.store.Update(ctx, info, fieldpaths...)
return info, errors.WithStack(err)
}
func (cs *callerContentStore) Walk(ctx context.Context, fn content.WalkFunc, fs ...string) error {
ctx = cs.choose(ctx)
return cs.store.Walk(ctx, fn, fs...)
return errors.WithStack(cs.store.Walk(ctx, fn, fs...))
}
func (cs *callerContentStore) Delete(ctx context.Context, dgst digest.Digest) error {
ctx = cs.choose(ctx)
return cs.store.Delete(ctx, dgst)
return errors.WithStack(cs.store.Delete(ctx, dgst))
}
func (cs *callerContentStore) ListStatuses(ctx context.Context, fs ...string) ([]content.Status, error) {
ctx = cs.choose(ctx)
return cs.store.ListStatuses(ctx, fs...)
resp, err := cs.store.ListStatuses(ctx, fs...)
return resp, errors.WithStack(err)
}
func (cs *callerContentStore) Status(ctx context.Context, ref string) (content.Status, error) {
ctx = cs.choose(ctx)
return cs.store.Status(ctx, ref)
st, err := cs.store.Status(ctx, ref)
return st, errors.WithStack(err)
}
func (cs *callerContentStore) Abort(ctx context.Context, ref string) error {
ctx = cs.choose(ctx)
return cs.store.Abort(ctx, ref)
return errors.WithStack(cs.store.Abort(ctx, ref))
}
func (cs *callerContentStore) Writer(ctx context.Context, opts ...content.WriterOpt) (content.Writer, error) {
ctx = cs.choose(ctx)
return cs.store.Writer(ctx, opts...)
w, err := cs.store.Writer(ctx, opts...)
return w, errors.WithStack(err)
}
func (cs *callerContentStore) ReaderAt(ctx context.Context, desc ocispec.Descriptor) (content.ReaderAt, error) {
ctx = cs.choose(ctx)
return cs.store.ReaderAt(ctx, desc)
ra, err := cs.store.ReaderAt(ctx, desc)
return ra, errors.WithStack(err)
}
// NewCallerStore creates content.Store from session.Caller with specified storeID

View File

@ -14,7 +14,7 @@ import (
)
func sendDiffCopy(stream grpc.Stream, fs fsutil.FS, progress progressCb) error {
return fsutil.Send(stream.Context(), stream, fs, progress)
return errors.WithStack(fsutil.Send(stream.Context(), stream, fs, progress))
}
func newStreamWriter(stream grpc.ClientStream) io.WriteCloser {
@ -29,7 +29,7 @@ type bufferedWriteCloser struct {
func (bwc *bufferedWriteCloser) Close() error {
if err := bwc.Writer.Flush(); err != nil {
return err
return errors.WithStack(err)
}
return bwc.Closer.Close()
}
@ -40,19 +40,25 @@ type streamWriterCloser struct {
func (wc *streamWriterCloser) Write(dt []byte) (int, error) {
if err := wc.ClientStream.SendMsg(&BytesMessage{Data: dt}); err != nil {
return 0, err
// SendMsg return EOF on remote errors
if errors.Cause(err) == io.EOF {
if err := errors.WithStack(wc.ClientStream.RecvMsg(struct{}{})); err != nil {
return 0, err
}
}
return 0, errors.WithStack(err)
}
return len(dt), nil
}
func (wc *streamWriterCloser) Close() error {
if err := wc.ClientStream.CloseSend(); err != nil {
return err
return errors.WithStack(err)
}
// block until receiver is done
var bm BytesMessage
if err := wc.ClientStream.RecvMsg(&bm); err != io.EOF {
return err
return errors.WithStack(err)
}
return nil
}
@ -69,19 +75,19 @@ func recvDiffCopy(ds grpc.Stream, dest string, cu CacheUpdater, progress progres
cf = cu.HandleChange
ch = cu.ContentHasher()
}
return fsutil.Receive(ds.Context(), ds, dest, fsutil.ReceiveOpt{
return errors.WithStack(fsutil.Receive(ds.Context(), ds, dest, fsutil.ReceiveOpt{
NotifyHashed: cf,
ContentHasher: ch,
ProgressCb: progress,
Filter: fsutil.FilterFunc(filter),
})
}))
}
func syncTargetDiffCopy(ds grpc.Stream, dest string) error {
if err := os.MkdirAll(dest, 0700); err != nil {
return err
return errors.Wrapf(err, "failed to create synctarget dest dir %s", dest)
}
return fsutil.Receive(ds.Context(), ds, dest, fsutil.ReceiveOpt{
return errors.WithStack(fsutil.Receive(ds.Context(), ds, dest, fsutil.ReceiveOpt{
Merge: true,
Filter: func() func(string, *fstypes.Stat) bool {
uid := os.Getuid()
@ -92,7 +98,7 @@ func syncTargetDiffCopy(ds grpc.Stream, dest string) error {
return true
}
}(),
})
}))
}
func writeTargetFile(ds grpc.Stream, wc io.WriteCloser) error {
@ -102,10 +108,10 @@ func writeTargetFile(ds grpc.Stream, wc io.WriteCloser) error {
if errors.Cause(err) == io.EOF {
return nil
}
return err
return errors.WithStack(err)
}
if _, err := wc.Write(bm.Data); err != nil {
return err
return errors.WithStack(err)
}
}
}

View File

@ -18,11 +18,12 @@ import (
)
const (
keyOverrideExcludes = "override-excludes"
keyIncludePatterns = "include-patterns"
keyExcludePatterns = "exclude-patterns"
keyFollowPaths = "followpaths"
keyDirName = "dir-name"
keyOverrideExcludes = "override-excludes"
keyIncludePatterns = "include-patterns"
keyExcludePatterns = "exclude-patterns"
keyFollowPaths = "followpaths"
keyDirName = "dir-name"
keyExporterMetaPrefix = "exporter-md-"
)
type fsSyncProvider struct {
@ -238,16 +239,16 @@ func NewFSSyncTargetDir(outdir string) session.Attachable {
}
// NewFSSyncTarget allows writing into an io.WriteCloser
func NewFSSyncTarget(w io.WriteCloser) session.Attachable {
func NewFSSyncTarget(f func(map[string]string) (io.WriteCloser, error)) session.Attachable {
p := &fsSyncTarget{
outfile: w,
f: f,
}
return p
}
type fsSyncTarget struct {
outdir string
outfile io.WriteCloser
outdir string
f func(map[string]string) (io.WriteCloser, error)
}
func (sp *fsSyncTarget) Register(server *grpc.Server) {
@ -258,11 +259,26 @@ func (sp *fsSyncTarget) DiffCopy(stream FileSend_DiffCopyServer) error {
if sp.outdir != "" {
return syncTargetDiffCopy(stream, sp.outdir)
}
if sp.outfile == nil {
if sp.f == nil {
return errors.New("empty outfile and outdir")
}
defer sp.outfile.Close()
return writeTargetFile(stream, sp.outfile)
opts, _ := metadata.FromIncomingContext(stream.Context()) // if no metadata continue with empty object
md := map[string]string{}
for k, v := range opts {
if strings.HasPrefix(k, keyExporterMetaPrefix) {
md[strings.TrimPrefix(k, keyExporterMetaPrefix)] = strings.Join(v, ",")
}
}
wc, err := sp.f(md)
if err != nil {
return err
}
if wc == nil {
return status.Errorf(codes.AlreadyExists, "target already exists")
}
defer wc.Close()
return writeTargetFile(stream, wc)
}
func CopyToCaller(ctx context.Context, fs fsutil.FS, c session.Caller, progress func(int, bool)) error {
@ -275,13 +291,13 @@ func CopyToCaller(ctx context.Context, fs fsutil.FS, c session.Caller, progress
cc, err := client.DiffCopy(ctx)
if err != nil {
return err
return errors.WithStack(err)
}
return sendDiffCopy(cc, fs, progress)
}
func CopyFileWriter(ctx context.Context, c session.Caller) (io.WriteCloser, error) {
func CopyFileWriter(ctx context.Context, md map[string]string, c session.Caller) (io.WriteCloser, error) {
method := session.MethodURL(_FileSend_serviceDesc.ServiceName, "diffcopy")
if !c.Supports(method) {
return nil, errors.Errorf("method %s not supported by the client", method)
@ -289,9 +305,16 @@ func CopyFileWriter(ctx context.Context, c session.Caller) (io.WriteCloser, erro
client := NewFileSendClient(c.Conn())
opts := make(map[string][]string, len(md))
for k, v := range md {
opts[keyExporterMetaPrefix+k] = []string{v}
}
ctx = metadata.NewOutgoingContext(ctx, opts)
cc, err := client.DiffCopy(ctx)
if err != nil {
return nil, err
return nil, errors.WithStack(err)
}
return newStreamWriter(cc), nil

View File

@ -21,10 +21,10 @@ func GetSecret(ctx context.Context, c session.Caller, id string) ([]byte, error)
ID: id,
})
if err != nil {
if st, ok := status.FromError(err); ok && (st.Code() == codes.Unimplemented || st.Code() == codes.NotFound) {
if st, ok := status.FromError(errors.Cause(err)); ok && (st.Code() == codes.Unimplemented || st.Code() == codes.NotFound) {
return nil, errors.Wrapf(ErrNotFound, "secret %s not found", id)
}
return nil, err
return nil, errors.WithStack(err)
}
return resp.Data, nil
}

View File

@ -3,23 +3,24 @@ package sshforward
import (
io "io"
"github.com/pkg/errors"
context "golang.org/x/net/context"
"golang.org/x/sync/errgroup"
"google.golang.org/grpc"
)
func Copy(ctx context.Context, conn io.ReadWriteCloser, stream grpc.Stream) error {
func Copy(ctx context.Context, conn io.ReadWriteCloser, stream grpc.Stream, closeStream func() error) error {
g, ctx := errgroup.WithContext(ctx)
g.Go(func() (retErr error) {
p := &BytesMessage{}
for {
if err := stream.RecvMsg(p); err != nil {
conn.Close()
if err == io.EOF {
return nil
}
conn.Close()
return err
return errors.WithStack(err)
}
select {
case <-ctx.Done():
@ -29,7 +30,7 @@ func Copy(ctx context.Context, conn io.ReadWriteCloser, stream grpc.Stream) erro
}
if _, err := conn.Write(p.Data); err != nil {
conn.Close()
return err
return errors.WithStack(err)
}
p.Data = p.Data[:0]
}
@ -41,9 +42,12 @@ func Copy(ctx context.Context, conn io.ReadWriteCloser, stream grpc.Stream) erro
n, err := conn.Read(buf)
switch {
case err == io.EOF:
if closeStream != nil {
closeStream()
}
return nil
case err != nil:
return err
return errors.WithStack(err)
}
select {
case <-ctx.Done():
@ -52,7 +56,7 @@ func Copy(ctx context.Context, conn io.ReadWriteCloser, stream grpc.Stream) erro
}
p := &BytesMessage{Data: buf[:n]}
if err := stream.SendMsg(p); err != nil {
return err
return errors.WithStack(err)
}
}
})

View File

@ -7,6 +7,7 @@ import (
"path/filepath"
"github.com/moby/buildkit/session"
"github.com/pkg/errors"
context "golang.org/x/net/context"
"golang.org/x/sync/errgroup"
"google.golang.org/grpc/metadata"
@ -48,7 +49,7 @@ func (s *server) run(ctx context.Context, l net.Listener, id string) error {
return err
}
go Copy(ctx, conn, stream)
go Copy(ctx, conn, stream, stream.CloseSend)
}
})
@ -65,7 +66,7 @@ type SocketOpt struct {
func MountSSHSocket(ctx context.Context, c session.Caller, opt SocketOpt) (sockPath string, closer func() error, err error) {
dir, err := ioutil.TempDir("", ".buildkit-ssh-sock")
if err != nil {
return "", nil, err
return "", nil, errors.WithStack(err)
}
defer func() {
@ -78,16 +79,16 @@ func MountSSHSocket(ctx context.Context, c session.Caller, opt SocketOpt) (sockP
l, err := net.Listen("unix", sockPath)
if err != nil {
return "", nil, err
return "", nil, errors.WithStack(err)
}
if err := os.Chown(sockPath, opt.UID, opt.GID); err != nil {
l.Close()
return "", nil, err
return "", nil, errors.WithStack(err)
}
if err := os.Chmod(sockPath, os.FileMode(opt.Mode)); err != nil {
l.Close()
return "", nil, err
return "", nil, errors.WithStack(err)
}
s := &server{caller: c}
@ -102,12 +103,12 @@ func MountSSHSocket(ctx context.Context, c session.Caller, opt SocketOpt) (sockP
return sockPath, func() error {
err := l.Close()
os.RemoveAll(sockPath)
return err
return errors.WithStack(err)
}, nil
}
func CheckSSHID(ctx context.Context, c session.Caller, id string) error {
client := NewSSHClient(c.Conn())
_, err := client.CheckAgent(ctx, &CheckAgentRequest{ID: id})
return err
return errors.WithStack(err)
}

View File

@ -114,7 +114,7 @@ func (sp *socketProvider) ForwardAgent(stream sshforward.SSH_ForwardAgentServer)
eg.Go(func() error {
defer s1.Close()
return sshforward.Copy(ctx, s2, stream)
return sshforward.Copy(ctx, s2, stream, nil)
})
return eg.Wait()

View File

@ -30,19 +30,20 @@ const (
CapBuildOpLLBFileName apicaps.CapID = "source.buildop.llbfilename"
CapExecMetaBase apicaps.CapID = "exec.meta.base"
CapExecMetaProxy apicaps.CapID = "exec.meta.proxyenv"
CapExecMetaNetwork apicaps.CapID = "exec.meta.network"
CapExecMetaSecurity apicaps.CapID = "exec.meta.security"
CapExecMetaSetsDefaultPath apicaps.CapID = "exec.meta.setsdefaultpath"
CapExecMountBind apicaps.CapID = "exec.mount.bind"
CapExecMountCache apicaps.CapID = "exec.mount.cache"
CapExecMountCacheSharing apicaps.CapID = "exec.mount.cache.sharing"
CapExecMountSelector apicaps.CapID = "exec.mount.selector"
CapExecMountTmpfs apicaps.CapID = "exec.mount.tmpfs"
CapExecMountSecret apicaps.CapID = "exec.mount.secret"
CapExecMountSSH apicaps.CapID = "exec.mount.ssh"
CapExecCgroupsMounted apicaps.CapID = "exec.cgroup"
CapExecMetaBase apicaps.CapID = "exec.meta.base"
CapExecMetaProxy apicaps.CapID = "exec.meta.proxyenv"
CapExecMetaNetwork apicaps.CapID = "exec.meta.network"
CapExecMetaSecurity apicaps.CapID = "exec.meta.security"
CapExecMetaSetsDefaultPath apicaps.CapID = "exec.meta.setsdefaultpath"
CapExecMountBind apicaps.CapID = "exec.mount.bind"
CapExecMountBindReadWriteNoOuput apicaps.CapID = "exec.mount.bind.readwrite-nooutput"
CapExecMountCache apicaps.CapID = "exec.mount.cache"
CapExecMountCacheSharing apicaps.CapID = "exec.mount.cache.sharing"
CapExecMountSelector apicaps.CapID = "exec.mount.selector"
CapExecMountTmpfs apicaps.CapID = "exec.mount.tmpfs"
CapExecMountSecret apicaps.CapID = "exec.mount.secret"
CapExecMountSSH apicaps.CapID = "exec.mount.ssh"
CapExecCgroupsMounted apicaps.CapID = "exec.cgroup"
CapFileBase apicaps.CapID = "file.base"
@ -193,6 +194,12 @@ func init() {
Status: apicaps.CapStatusExperimental,
})
Caps.Init(apicaps.Cap{
ID: CapExecMountBindReadWriteNoOuput,
Enabled: true,
Status: apicaps.CapStatusExperimental,
})
Caps.Init(apicaps.Cap{
ID: CapExecMountCache,
Enabled: true,

10
vendor/github.com/sirupsen/logrus/go.mod generated vendored Normal file
View File

@ -0,0 +1,10 @@
module github.com/sirupsen/logrus
require (
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/konsorten/go-windows-terminal-sequences v1.0.1
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/stretchr/objx v0.1.1 // indirect
github.com/stretchr/testify v1.2.2
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33
)

28
vendor/github.com/tonistiigi/fsutil/go.mod generated vendored Normal file
View File

@ -0,0 +1,28 @@
module github.com/tonistiigi/fsutil
require (
github.com/Microsoft/go-winio v0.4.11 // indirect
github.com/Microsoft/hcsshim v0.8.5 // indirect
github.com/containerd/containerd v1.2.4
github.com/containerd/continuity v0.0.0-20181001140422-bd77b46c8352
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/docker/docker v0.0.0-20180531152204-71cd53e4a197
github.com/docker/go-units v0.3.1 // indirect
github.com/gogo/protobuf v1.0.0
github.com/google/go-cmp v0.2.0 // indirect
github.com/gotestyourself/gotestyourself v2.2.0+incompatible // indirect
github.com/onsi/ginkgo v1.7.0 // indirect
github.com/onsi/gomega v1.4.3 // indirect
github.com/opencontainers/go-digest v1.0.0-rc1
github.com/opencontainers/image-spec v1.0.1 // indirect
github.com/opencontainers/runc v1.0.0-rc6 // indirect
github.com/pkg/errors v0.8.1
github.com/sirupsen/logrus v1.0.3 // indirect
github.com/stretchr/testify v1.3.0
golang.org/x/crypto v0.0.0-20190129210102-0709b304e793 // indirect
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e
gopkg.in/airbrake/gobrake.v2 v2.0.9 // indirect
gopkg.in/gemnasium/logrus-airbrake-hook.v2 v2.1.2 // indirect
gotest.tools v2.1.0+incompatible // indirect
)

View File

@ -49,6 +49,9 @@ func mkstat(path, relpath string, fi os.FileInfo, inodemap map[uint64]string) (*
stat.Mode = noPermPart | permPart
}
// Clear the socket bit since archive/tar.FileInfoHeader does not handle it
stat.Mode &^= uint32(os.ModeSocket)
return stat, nil
}

3
vendor/golang.org/x/crypto/go.mod generated vendored Normal file
View File

@ -0,0 +1,3 @@
module golang.org/x/crypto
require golang.org/x/sys v0.0.0-20190403152447-81d4e9dc473e

6
vendor/golang.org/x/net/go.mod generated vendored Normal file
View File

@ -0,0 +1,6 @@
module golang.org/x/net
require (
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2
golang.org/x/text v0.3.0
)

1
vendor/golang.org/x/sync/go.mod generated vendored Normal file
View File

@ -0,0 +1 @@
module golang.org/x/sync

3
vendor/golang.org/x/sys/go.mod generated vendored Normal file
View File

@ -0,0 +1,3 @@
module golang.org/x/sys
go 1.12

19
vendor/google.golang.org/grpc/go.mod generated vendored Normal file
View File

@ -0,0 +1,19 @@
module google.golang.org/grpc
require (
cloud.google.com/go v0.26.0 // indirect
github.com/BurntSushi/toml v0.3.1 // indirect
github.com/client9/misspell v0.3.4
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
github.com/golang/mock v1.1.1
github.com/golang/protobuf v1.2.0
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3
golang.org/x/net v0.0.0-20190311183353-d8887717615a
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f // indirect
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a
golang.org/x/tools v0.0.0-20190311212946-11955173bddd
google.golang.org/appengine v1.1.0 // indirect
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099
)

13
vendor/gopkg.in/yaml.v2/decode.go generated vendored
View File

@ -229,6 +229,10 @@ type decoder struct {
mapType reflect.Type
terrors []string
strict bool
decodeCount int
aliasCount int
aliasDepth int
}
var (
@ -315,6 +319,13 @@ func (d *decoder) prepare(n *node, out reflect.Value) (newout reflect.Value, unm
}
func (d *decoder) unmarshal(n *node, out reflect.Value) (good bool) {
d.decodeCount++
if d.aliasDepth > 0 {
d.aliasCount++
}
if d.aliasCount > 100 && d.decodeCount > 1000 && float64(d.aliasCount)/float64(d.decodeCount) > 0.99 {
failf("document contains excessive aliasing")
}
switch n.kind {
case documentNode:
return d.document(n, out)
@ -353,7 +364,9 @@ func (d *decoder) alias(n *node, out reflect.Value) (good bool) {
failf("anchor '%s' value contains itself", n.value)
}
d.aliases[n] = true
d.aliasDepth++
good = d.unmarshal(n.alias, out)
d.aliasDepth--
delete(d.aliases, n)
return good
}

28
vendor/gopkg.in/yaml.v2/encode.go generated vendored
View File

@ -13,6 +13,19 @@ import (
"unicode/utf8"
)
// jsonNumber is the interface of the encoding/json.Number datatype.
// Repeating the interface here avoids a dependency on encoding/json, and also
// supports other libraries like jsoniter, which use a similar datatype with
// the same interface. Detecting this interface is useful when dealing with
// structures containing json.Number, which is a string under the hood. The
// encoder should prefer the use of Int64(), Float64() and string(), in that
// order, when encoding this type.
type jsonNumber interface {
Float64() (float64, error)
Int64() (int64, error)
String() string
}
type encoder struct {
emitter yaml_emitter_t
event yaml_event_t
@ -89,6 +102,21 @@ func (e *encoder) marshal(tag string, in reflect.Value) {
}
iface := in.Interface()
switch m := iface.(type) {
case jsonNumber:
integer, err := m.Int64()
if err == nil {
// In this case the json.Number is a valid int64
in = reflect.ValueOf(integer)
break
}
float, err := m.Float64()
if err == nil {
// In this case the json.Number is a valid float64
in = reflect.ValueOf(float)
break
}
// fallback case - no number could be obtained
in = reflect.ValueOf(m.String())
case time.Time, *time.Time:
// Although time.Time implements TextMarshaler,
// we don't want to treat it as a string for YAML

5
vendor/gopkg.in/yaml.v2/go.mod generated vendored Normal file
View File

@ -0,0 +1,5 @@
module "gopkg.in/yaml.v2"
require (
"gopkg.in/check.v1" v0.0.0-20161208181325-20d25e280405
)

2
vendor/gopkg.in/yaml.v2/resolve.go generated vendored
View File

@ -81,7 +81,7 @@ func resolvableTag(tag string) bool {
return false
}
var yamlStyleFloat = regexp.MustCompile(`^[-+]?[0-9]*\.?[0-9]+([eE][-+][0-9]+)?$`)
var yamlStyleFloat = regexp.MustCompile(`^[-+]?(\.[0-9]+|[0-9]+(\.[0-9]*)?)([eE][-+]?[0-9]+)?$`)
func resolve(tag string, in string) (rtag string, out interface{}) {
if !resolvableTag(tag) {

8
vendor/gotest.tools/go.mod vendored Normal file
View File

@ -0,0 +1,8 @@
module gotest.tools
require (
github.com/google/go-cmp v0.2.0
github.com/pkg/errors v0.8.0
github.com/spf13/pflag v1.0.3
golang.org/x/tools v0.0.0-20180810170437-e96c4e24768d
)