Adds a local fork of this package for use in the classic builder.
Code was taken at commit [d33d46d01656e1d9ee26743f0c0d7779f685dd4e][1].
Migration was done using the following steps:
# install filter-repo (https://github.com/newren/git-filter-repo/blob/main/INSTALL.md)
brew install git-filter-repo
# create a temporary clone of docker
cd ~/Projects
git clone https://github.com/docker/docker.git build_context_temp
cd build_context_temp
# commit taken from
git rev-parse --verify HEAD
d33d46d01656e1d9ee26743f0c0d7779f685dd4e
git filter-repo --analyze
# remove all code, except for the remotecontext packages, and move to build/internal docs and previous locations of it
git filter-repo \
--path 'builder/remotecontext/git' \
--path 'builder/remotecontext/urlutil' \
--path-rename builder/remotecontext:cli/command/image/build/internal
# go to the target repository
cd ~/go/src/github.com/docker/cli
# create a branch to work with
git checkout -b fork_remotecontext
# add the temporary repository as an upstream and make sure it's up-to-date
git remote add build_context_temp ~/Projects/build_context_temp
git fetch build_context_temp
# merge the upstream code
git merge --allow-unrelated-histories --signoff -S build_context_temp/master
[1]: https://github.com/docker/docker/d33d46d01656e1d9ee26743f0c0d7779f685dd4e
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was the only utility we consumed from the package, and it's trivial
to implement, so let's create local copies of it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This utility was only used for testing, and to generate a random
suffix for Dockerfiles. As we don't need the same contract as
pkg/stringid.GenerateRandomID() (not allow all-numeric IDs as they
would not be usable for hostnames), we can use a local test-utility,
and local implementation for the random suffix instead.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The use of pools here is unnecessary and creates a code dependency on
moby library which will move to internal.
The pool does not provide any value here since it is only done for a
single operation and unlikely performed in alongside other operations
which are re-using the buffers many times between garbage collections.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Looks like it's broken, so use a blanket "nolint:gosec" instead;
cli/command/image/build/context.go:238:17: G107: Potential HTTP request made with variable url (gosec)
if resp, err = http.Get(url); err != nil {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
cli/registry/client/endpoint.go:128:34: fmt.Sprintf can be replaced with string concatenation (perfsprint)
req.Header.Set("Authorization", fmt.Sprintf("Bearer %s", th.token))
^
cli/command/telemetry_docker.go:88:14: fmt.Sprintf can be replaced with string concatenation (perfsprint)
endpoint = fmt.Sprintf("unix://%s", path.Join(u.Host, u.Path))
^
cli/command/cli_test.go:195:47: fmt.Sprintf can be replaced with string concatenation (perfsprint)
opts := &flags.ClientOptions{Hosts: []string{fmt.Sprintf("unix://%s", socket)}}
^
cli/command/registry_test.go:59:24: fmt.Sprintf can be replaced with string concatenation (perfsprint)
inputServerAddress: fmt.Sprintf("https://%s", testAuthConfigs[1].ServerAddress),
^
cli/command/container/opts_test.go:338:35: fmt.Sprintf can be replaced with string concatenation (perfsprint)
if config, _, _ := mustParse(t, fmt.Sprintf("--hostname=%s", hostname)); config.Hostname != expectedHostname {
^
cli/command/context/options.go:79:24: fmt.Sprintf can be replaced with string concatenation (perfsprint)
errs = append(errs, fmt.Sprintf("%s: unrecognized config key", k))
^
cli/command/image/build.go:461:68: fmt.Sprintf can be replaced with string concatenation (perfsprint)
line = dockerfileFromLinePattern.ReplaceAllLiteralString(line, fmt.Sprintf("FROM %s", reference.FamiliarString(trustedRef)))
^
cli/command/image/remove_test.go:21:9: fmt.Sprintf can be replaced with string concatenation (perfsprint)
return fmt.Sprintf("Error: No such image: %s", n.imageID)
^
cli/command/image/build/context.go:229:102: fmt.Sprintf can be replaced with string concatenation (perfsprint)
progReader := progress.NewProgressReader(response.Body, progressOutput, response.ContentLength, "", fmt.Sprintf("Downloading build context from remote url: %s", remoteURL))
^
cli/command/service/logs.go:215:16: fmt.Sprintf can be replaced with string concatenation (perfsprint)
taskName += fmt.Sprintf(".%s", task.ID)
^
cli/command/service/logs.go:217:16: fmt.Sprintf can be replaced with string concatenation (perfsprint)
taskName += fmt.Sprintf(".%s", stringid.TruncateID(task.ID))
^
cli/command/service/progress/progress_test.go:877:18: fmt.Sprintf can be replaced with string concatenation (perfsprint)
ID: fmt.Sprintf("task%s", nodeID),
^
cli/command/stack/swarm/remove.go:61:24: fmt.Sprintf can be replaced with string concatenation (perfsprint)
errs = append(errs, fmt.Sprintf("Failed to remove some resources from stack: %s", namespace))
^
cli/command/swarm/ipnet_slice_test.go:32:9: fmt.Sprintf can be replaced with string concatenation (perfsprint)
arg := fmt.Sprintf("--cidrs=%s", strings.Join(vals, ","))
^
cli/command/swarm/ipnet_slice_test.go:137:30: fmt.Sprintf can be replaced with string concatenation (perfsprint)
if err := f.Parse([]string{fmt.Sprintf("--cidrs=%s", strings.Join(test.FlagArg, ","))}); err != nil {
^
cli/compose/schema/schema.go:105:11: fmt.Sprintf can be replaced with string concatenation (perfsprint)
return fmt.Sprintf("must be a %s", humanReadableType(expectedType))
^
cli/manifest/store/store.go:165:9: fmt.Sprintf can be replaced with string concatenation (perfsprint)
return fmt.Sprintf("No such manifest: %s", n.object)
^
e2e/image/push_test.go:340:4: fmt.Sprintf can be replaced with string concatenation (perfsprint)
fmt.Sprintf("NOTARY_ROOT_PASSPHRASE=%s", pwd),
^
e2e/image/push_test.go:341:4: fmt.Sprintf can be replaced with string concatenation (perfsprint)
fmt.Sprintf("NOTARY_TARGETS_PASSPHRASE=%s", pwd),
^
e2e/image/push_test.go:342:4: fmt.Sprintf can be replaced with string concatenation (perfsprint)
fmt.Sprintf("NOTARY_SNAPSHOT_PASSPHRASE=%s", pwd),
^
e2e/image/push_test.go:343:4: fmt.Sprintf can be replaced with string concatenation (perfsprint)
fmt.Sprintf("NOTARY_DELEGATION_PASSPHRASE=%s", pwd),
^
e2e/plugin/trust_test.go:23:16: fmt.Sprintf can be replaced with string concatenation (perfsprint)
pluginName := fmt.Sprintf("%s/plugin-content-trust", registryPrefix)
^
e2e/plugin/trust_test.go:53:8: fmt.Sprintf can be replaced with string concatenation (perfsprint)
Out: fmt.Sprintf("Installed plugin %s", pluginName),
^
e2e/trust/revoke_test.go:62:57: fmt.Sprintf can be replaced with string concatenation (perfsprint)
icmd.RunCommand("docker", "tag", fixtures.AlpineImage, fmt.Sprintf("%s:v1", revokeRepo)).Assert(t, icmd.Success)
^
e2e/trust/revoke_test.go:64:49: fmt.Sprintf can be replaced with string concatenation (perfsprint)
icmd.Command("docker", "-D", "trust", "sign", fmt.Sprintf("%s:v1", revokeRepo)),
^
e2e/trust/revoke_test.go:68:58: fmt.Sprintf can be replaced with string concatenation (perfsprint)
icmd.RunCommand("docker", "tag", fixtures.BusyboxImage, fmt.Sprintf("%s:v2", revokeRepo)).Assert(t, icmd.Success)
^
e2e/trust/revoke_test.go:70:49: fmt.Sprintf can be replaced with string concatenation (perfsprint)
icmd.Command("docker", "-D", "trust", "sign", fmt.Sprintf("%s:v2", revokeRepo)),
^
e2e/trust/sign_test.go:36:47: fmt.Sprintf can be replaced with string concatenation (perfsprint)
assert.Check(t, is.Contains(result.Stdout(), fmt.Sprintf("v1: digest: sha256:%s", fixtures.AlpineSha)))
^
e2e/trust/sign_test.go:53:47: fmt.Sprintf can be replaced with string concatenation (perfsprint)
assert.Check(t, is.Contains(result.Stdout(), fmt.Sprintf("v1: digest: sha256:%s", fixtures.BusyboxSha)))
^
e2e/trust/sign_test.go:65:47: fmt.Sprintf can be replaced with string concatenation (perfsprint)
assert.Check(t, is.Contains(result.Stdout(), fmt.Sprintf("v1: digest: sha256:%s", fixtures.AlpineSha)))
^
opts/file.go:21:9: fmt.Sprintf can be replaced with string concatenation (perfsprint)
return fmt.Sprintf("poorly formatted environment: %s", e.msg)
^
opts/hosts_test.go:26:31: fmt.Sprintf can be replaced with string concatenation (perfsprint)
"tcp://host:": fmt.Sprintf("tcp://host:%s", defaultHTTPPort),
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit 9e1f8d646e changed this to a
"nolint" comment due to a regression in GoSec. That regression was
fixed, so we can go back to use the more fine-grained "nosec" comment.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When passing a Dockerfile through stdin, it's not possible to specify the
name of the Dockerfile (using the `-f` option). When building with BuildKit
enabled, an error is already produced for this case, but the classic builder
silently ignored it.
This patch adds an error for this situation:
echo -e 'FROM busybox' | DOCKER_BUILDKIT=0 docker build -f some.Dockerfile -
DEPRECATED: The legacy builder is deprecated and will be removed in a future release.
BuildKit is currently disabled; enable it by removing the DOCKER_BUILDKIT=0
environment-variable.
unable to prepare context: ambiguous Dockerfile source: both stdin and flag correspond to Dockerfiles
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
the "golang.org/x/sys/execabs" package was introduced to address a security
issue on Windows, and changing the default behavior of os/exec was considered
a breaking change. go1.19 applied the behavior that was previously implemented
in the execabs package;
from the release notes: https://go.dev/doc/go1.19#os-exec-path
> Command and LookPath no longer allow results from a PATH search to be found
> relative to the current directory. This removes a common source of security
> problems but may also break existing programs that depend on using, say,
> exec.Command("prog") to run a binary named prog (or, on Windows, prog.exe)
> in the current directory. See the os/exec package documentation for information
> about how best to update such programs.
>
> On Windows, Command and LookPath now respect the NoDefaultCurrentDirectoryInExePath
> environment variable, making it possible to disable the default implicit search
> of “.” in PATH lookups on Windows systems.
With those changes, we no longer need to use the execabs package, and we can
switch back to os/exec.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Looks like the linter uses an explicit -lang, which (for go1.19)
results in some additional formatting for octal values.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
cli/command/image/build/context.go:238:23: "400" can be replaced by http.StatusBadRequest (usestdlibvars)
if resp.StatusCode < 400 {
^
cli/trust/trust.go:139:30: "GET" can be replaced by http.MethodGet (usestdlibvars)
req, err := http.NewRequest("GET", endpointStr, nil)
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Changed `matcher.Matches(file)` to `matcher.MatchesOrParentMatches(file)`:
cli/command/image/build/context.go:95:9: SA1019: matcher.Matches is deprecated: This implementation is buggy (it only checks a single parent dir against the pattern) and will be removed soon. Use either MatchesOrParentMatches or MatchesUsingParentResults instead. (staticcheck)
return matcher.Matches(file)
^
And updated a test to match the JSON omitting empty RootFS.Type fields (in
practice, this field should never be empty in real situations, and always
be "layer"). Changed the test to use subtests to easier find which case
is failing.
full diff: 343665850e...83b51522df
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 73aef6edfe
modified archive.ReplaceFileTarWrapper to set the Name field in the tar header,
if the field was not set.
That change exposed an issue in how a Dockerfile from stdin was sent to the daemon.
When attempting to build using a build-context, and a Dockerfile from stdin, the
following happened:
```bash
mkdir build-stdin && cd build-stdin && echo hello > hello.txt
DOCKER_BUILDKIT=0 docker build --no-cache -t foo -f- . <<'EOF'
FROM alpine
COPY . .
EOF
Sending build context to Docker daemon 2.607kB
Error response from daemon: dockerfile parse error line 1: unknown instruction: .DOCKERIGNORE
```
Removing the `-t foo`, oddly lead to a different failure:
```bash
DOCKER_BUILDKIT=0 docker build --no-cache -f- . <<'EOF'
FROM alpine
COPY . .
EOF
Sending build context to Docker daemon 2.581kB
Error response from daemon: Cannot locate specified Dockerfile: .dockerfile.701d0d71fb1497d6a7ce
```
From the above, it looks like the tar headers got mangled, causing (in the first
case) the daemon to use the build-context tar as a plain-text file, and therefore
parsing it as Dockerfile, and in the second case, causing it to not being able to
find the Dockerfile in the context.
I noticed that both TarModifierFuncs were using the same `hdrTmpl` struct, which
looks to caused them to step on each other's toes. Changing them to each initialize
their own struct made the issue go away.
After this change:
```bash
DOCKER_BUILDKIT=0 docker build --no-cache -t foo -f- . <<'EOF'
FROM alpine
COPY . .
EOF
Sending build context to Docker daemon 2.607kB
Step 1/2 : FROM alpine
---> d4ff818577bc
Step 2/2 : COPY . .
---> 556f745e6938
Successfully built 556f745e6938
Successfully tagged foo:latest
DOCKER_BUILDKIT=0 docker build --no-cache -f- . <<'EOF'
FROM alpine
COPY . .
EOF
Sending build context to Docker daemon 2.607kB
Step 1/2 : FROM alpine
---> d4ff818577bc
Step 2/2 : COPY . .
---> aaaee43bec5e
Successfully built aaaee43bec5e
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
On Windows, the os/exec.{Command,CommandContext,LookPath} functions
resolve command names that have neither path separators nor file extension
(e.g., "git") by first looking in the current working directory before
looking in the PATH environment variable.
Go maintainers intended to match cmd.exe's historical behavior.
However, this is pretty much never the intended behavior and as an abundance of precaution
this patch prevents that when executing commands.
Example of commands that docker.exe may execute: `git`, `docker-buildx` (or other cli plugin), `docker-credential-wincred`, `docker`.
Note that this was prompted by the [Go 1.15.7 security fixes](https://blog.golang.org/path-security), but unlike in `go.exe`,
the windows path lookups in docker are not in a code path allowing remote code execution, thus there is no security impact on docker.
Signed-off-by: Tibor Vass <tibor@docker.com>
As pointed out in #1459, docker cli fails to detect that the input is a tarball,
in case it is generated by `git archive --format=tgz`.
This happens because `git archive` adds some metadata to the initial tar header,
and so it is more than 1 block (of 512 bytes) long, while we only provide 1 block
to archive/tar.Next() and it fails.
To fix, give it 2 blocks :)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Historically, the Dockerfile had to be insde the build-context, because it was
sent as part of the build-context.
3f6dc81e10
added support for passing the Dockerfile through stdin, in which case the
contents of the Dockerfile is injected into the build-context.
This patch uses the same mechanism for situations where the location of the
Dockerfile is passed, and its path is outside of the build-context.
Before this change:
$ mkdir -p myproject/context myproject/dockerfiles && cd myproject
$ echo "hello" > context/hello
$ echo -e "FROM busybox\nCOPY /hello /\nRUN cat /hello" > dockerfiles/Dockerfile
$ docker build --no-cache -f $PWD/dockerfiles/Dockerfile $PWD/context
unable to prepare context: the Dockerfile (/Users/sebastiaan/projects/test/dockerfile-outside/myproject/dockerfiles/Dockerfile) must be within the build context
After this change:
$ mkdir -p myproject/context myproject/dockerfiles && cd myproject
$ echo "hello" > context/hello
$ echo -e "FROM busybox\nCOPY /hello /\nRUN cat /hello" > dockerfiles/Dockerfile
$ docker build --no-cache -f $PWD/dockerfiles/Dockerfile $PWD/context
Sending build context to Docker daemon 2.607kB
Step 1/3 : FROM busybox
---> 6ad733544a63
Step 2/3 : COPY /hello /
---> 9a5ae1c7be9e
Step 3/3 : RUN cat /hello
---> Running in 20dfef2d180f
hello
Removing intermediate container 20dfef2d180f
---> ce1748f91bb2
Successfully built ce1748f91bb2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
