Adds a flag to the create and run command, `--use-api-socket`, that can
be used to start a container with the correctly configured parameters to
ensure that accessing the docker socket will work with out managing bind
mounts and authentication injection.
The implementation in this PR resolves the tokens for the current
credential set in the client and then copies it into a container at the
well know location of /run/secrets/docker/config.json, setting
DOCKER_CONFIG to ensure it is resolved by existing tooling. We use a
compose-compatible secret location with the hope that the CLI and
compose can work together seamlessly.
The bind mount for the socket is resolved from the current context,
erroring out if the flag is set and the provided socket is not a unix
socket.
There are a few drawbacks to this approach but it resolves a long
standing pain point. We'll continue to develop this as we understand
more use cases but it is marked as experimental for now.
Signed-off-by: Stephen Day <stephen.day@docker.com>
The [`docker buildx bake`][1] command has reached GA; this patch adds
a top-level `docker bake` command as alias for `docker buildx bake` to
improve discoverability and make it more convenient to use.
With this patch:
docker --help
Usage: docker [OPTIONS] COMMAND
A self-sufficient runtime for containers
Common Commands:
run Create and run a new container from an image
exec Execute a command in a running container
ps List containers
build Build an image from a Dockerfile
bake Build from a file
pull Download an image from a registry
push Upload an image to a registry
images List images
...
The command is hidden if buildx is not installed;
docker --help
Usage: docker [OPTIONS] COMMAND
A self-sufficient runtime for containers
Common Commands:
run Create and run a new container from an image
exec Execute a command in a running container
ps List containers
build Build an image from a Dockerfile
pull Download an image from a registry
push Upload an image to a registry
images List images
...
We can do some tweaking after this; currently it show an error
in situations where buildx is missing. We don't account for
"DOCKER_BUILDKIT=0", because this is a new feature that requires
buildx, and cannot be "disabled";
buildx missing;
docker bake
ERROR: bake requires the buildx component but it is missing or broken.
Install the buildx component to use bake:
https://docs.docker.com/go/buildx/
BuildKit disabled:
DOCKER_BUILDKIT=0 docker bake
ERROR: bake requires the buildx component but it is missing or broken.
Install the buildx component to use bake:
https://docs.docker.com/go/buildx/
[1]: https://www.docker.com/blog/ga-launch-docker-bake/
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before this patch, `docker image ls` / `docker image ls` would always
show untagged images, but hide "dangling" images (which effectively
only were produced by the legacy builder) unless `-a` / `--all` was
used. This often resulted in many `<none>:<none>` or `<untagged>` images
to be shown, which had little value to interact with, other than to
garbage collect (`docker system prune`).
In future, we want to take more advantage of containerd's garbage-collecting
features (removing unused images automatically), and this UX change is
a stepping stone toward that.
For now, this patch only changes the behavior for `docker image ls --tree`,
but we should make this the same for "non" --tree as well.
This patch:
- changes `docker image ls` to hide both "untagged" and "dangling" images
by default.
- changes the behavior of `--all` on the client side to make them visible
The API response remains the same for now, but this is something we can
consider changing in future (possibly more granular than a single boolean).
Before this patch;
docker image ls --tree
i Info → U In Use
IMAGE ID DISK USAGE CONTENT SIZE EXTRA
docker:cli 28fb556c1ea1 276MB 69.8MB
├─ linux/amd64 828f4f57525d 0B 0B
├─ linux/arm/v6 563c0b58e54b 0B 0B
├─ linux/arm/v7 6045d4846c59 0B 0B
└─ linux/arm64/v8 11e8dfd68841 276MB 69.8MB
alpine:latest a8560b36e8b8 12.8MB 3.99MB U
├─ linux/amd64 1c4eef651f65 0B 0B
├─ linux/arm/v6 903bfe2ae994 0B 0B
├─ linux/arm/v7 9c2d245b3c01 0B 0B
├─ linux/arm64/v8 757d680068d7 12.8MB 3.99MB U
├─ linux/386 2436f2b3b7d2 0B 0B
├─ linux/ppc64le 9ed53fd3b831 0B 0B
├─ linux/riscv64 1de5eb4a9a67 0B 0B
└─ linux/s390x fe0dcdd1f783 0B 0B
<untagged> c6c1bcb0fd8d 12.8MB 3.99MB
└─ linux/arm64 cb171c618ae8 12.8MB 3.99MB
<untagged> 7361ef970703 12.8MB 3.99MB
└─ linux/arm64 07033f43e44a 12.8MB 3.99MB
<untagged> 0c62c63b81ec 12.8MB 3.99MB
└─ linux/arm64 94742272117f 12.8MB 3.99MB
<untagged> 91dd947eebd0 12.8MB 3.99MB
└─ linux/arm64 ee55d203e26f 12.8MB 3.99MB
<untagged> 382d9f57e8d8 12.8MB 3.99MB
└─ linux/arm64 5256d47804e3 12.8MB 3.99MB
<untagged> 56fa17d2a7e7 12.8MB 3.99MB
├─ linux/amd64 483f502c0e6a 0B 0B
├─ linux/arm/v6 c79529000bdf 0B 0B
├─ linux/arm/v7 cc455d4b2c47 0B 0B
├─ linux/arm64/v8 508c1b94e1d2 12.8MB 3.99MB
├─ linux/386 f32403957113 0B 0B
├─ linux/ppc64le 23dbce23b88f 0B 0B
├─ linux/riscv64 f9d2da150cee 0B 0B
└─ linux/s390x 6bb03952a007 0B 0B
After this patch
docker image ls --tree
i Info → U In Use
IMAGE ID DISK USAGE CONTENT SIZE EXTRA
docker:cli 28fb556c1ea1 276MB 69.8MB
├─ linux/amd64 828f4f57525d 0B 0B
├─ linux/arm/v6 563c0b58e54b 0B 0B
├─ linux/arm/v7 6045d4846c59 0B 0B
└─ linux/arm64/v8 11e8dfd68841 276MB 69.8MB
alpine:latest a8560b36e8b8 12.8MB 3.99MB U
├─ linux/amd64 1c4eef651f65 0B 0B
├─ linux/arm/v6 903bfe2ae994 0B 0B
├─ linux/arm/v7 9c2d245b3c01 0B 0B
├─ linux/arm64/v8 757d680068d7 12.8MB 3.99MB U
├─ linux/386 2436f2b3b7d2 0B 0B
├─ linux/ppc64le 9ed53fd3b831 0B 0B
├─ linux/riscv64 1de5eb4a9a67 0B 0B
└─ linux/s390x fe0dcdd1f783 0B 0B
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before this patch, image total content size would only include
container images content size.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
We can return early without executing the regular expression or evaluating
the template for `--format=json` or `--format='{{json .}}'`.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Same functionality, but implemented with atomicwriter. There's a slight
difference in error-messages produced (but can be adjusted if we want).
Before:
docker container export -o ./no/such/foo mycontainer
failed to export container: invalid output path: directory "no/such" does not exist
docker container export -o /no/permissions mycontainer
failed to export container: stat /no/permissions: permission denied
After:
docker container export -o ./no/such/foo mycontainer
failed to export container: invalid file path: stat no/such: no such file or directory
docker container export -o /no/permissions mycontainer
failed to export container: failed to stat output path: lstat /no/permissions: permission denied
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Same functionality, but implemented with atomicwriter. There's a slight
difference in error-messages produced (but can be adjusted if we want).
Before:
docker image save -o ./no/such/foo busybox:latest
failed to save image: invalid output path: directory "no/such" does not exist
docker image save -o /no/permissions busybox:latest
failed to save image: stat /no/permissions: permission denied
After:
docker image save -o ./no/such/foo busybox:latest
failed to save image: invalid file path: stat no/such: no such file or directory
docker image save -o /no/permissions busybox:latest
failed to save image: failed to stat output path: lstat /no/permissions: permission denied
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
gotest.tools' fs package only provides very minimal benefits here;
use stdlib functions to make things slightly more transparent.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Assert that the write succeeded; also changing `Fprintf` to `Fprint`,
because we were not using templating (we should check why no linter
complained about this).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test verifies the default behavior, but when running the test
in an environment that already has a ~/.docker/config.json present,
it may fail.
This patch updates the test to configure the config-directory to
point to an empty directory, making sure it's not affected by
state.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
In most situations, the CLI is created through the `NewDockerCli` constructor,
however, it's possible to construct a CLI manually (`&DockerCli{}`). We
should probably prevent this (and un-export the `DockerCli` implementation),
but currently have some code-paths that depend on the type being exported.
When constructing the CLI with this approach, the CLI would not be fully
initialized and not have the context-store configuration set up.
Using the default context store without a config set will result in Endpoints
from contexts not being type-mapped correctly, and used as a generic
`map[string]any`, instead of a [docker.EndpointMeta].
When looking up the API endpoint (using [EndpointFromContext]), no endpoint
will be found, and a default, empty endpoint will be used instead which in
its turn, causes [newAPIClientFromEndpoint] to be initialized with the default
config instead of settings for the current context (which may mean; connecting
with the wrong endpoint and/or TLS Config to be missing).
I'm not sure if this situation could happen in practice, but it caused some
of our unit-tests ([TestInitializeFromClient] among others) to fail when
running outside of the dev-container on a host that used Docker Desktop's
"desktop-linux" context. In that situation, the test would produce the wrong
"Ping" results (using defaults, instead of the results produced in the test).
This patch:
- updates the contextStoreConfig field to be a pointer, so that we are
able to detect if a config was already set.
- updates the `Initialize` function to set the default context-store config
if no config was found (technically the field is mostly immutable, and
can only set through `WithDefaultContextStoreConfig`, so this may be
slightly redundant).
We should update this code to be less error-prone to use; the combination
of an exported type (`DockerCli`), a constructor `NewDockerCli` and a
`Initialize` function (as well as some internal contructors to allow
lazy initialization) make constructing the "CLI" hard to use, and there's
various codepaths where it can be in a partially initialized state. The
same applies to the default context store, which also requires too much
"domain" knowledge to use properly.
I'm leaving improvements around that for a follow-up.
[EndpointFromContext]: 33494921b8/cli/context/docker/load.go (L139-L149)
[docker.EndpointMeta]: 33494921b8/cli/context/docker/load.go (L19-L21)
[newAPIClientFromEndpoint]: 33494921b8/cli/command/cli.go (L295-L305)
[TestInitializeFromClient]: 33494921b8/cli/command/cli_test.go (L157-L205)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This reverts commit fed9fa0f72.
This removes the ContextType field, which was temporarily added to provide
compatibility with the "compose-cli" wrapper that shipped with Docker Desktop.
The compose-cli wrapper extended the context struct with an additional field
that was not part of the CLI itself, but was used by Visual Studio to detect
the type of context.
This temporary field shipped as part of Docker 27.0 June 2024), which should
be enough time for Visual Studio to have adjusted their integration.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test was only testing whether we could load a legacy config-file that
contained the "experimental" (experimental CLI) option. Experimental cli
options are disabled since 977d3ae046 (20.10),
and now enabled by default, but we should not fail to start the cli if the
config-file contains the option.
Move the test to the config package, as it doesn't need the cli for this.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This looks like a copy/paste from other tests, because this test
does not test anything related to docker content trust.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function was exported in 812f113685
for use in other parts of the CLI, but it's now only used locally.
Make it internal again, as it was never designed to be exported. There
are no known external consumers of this function, but deprecating it
first, in case there are.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function was exported in e43c7920ea
for use of "docker app", which is now deprecated. The signature of this
function also depended on a non-exported type, so it could not be used
externally.
Make it internal again, as it was never designed to be exported. There
are no known external consumers of this function.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function was exported in e43c7920ea
for use of "docker app", which is now deprecated. The signature of this
function also depended on a non-exported type so it could not be used
externally.
Make it internal again, as it was never designed to be exported. There
are no known external consumers of this function.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It's only used internally, and has no external consumers. Un-export
it, rename it to something more descriptive, and move it to a separate
file to align with other packages.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was introduced in 79141ce5eb, which
was reverted in f596202125, and re-applied
in the previous commit.
Before this patch, saving credentials worked correctly;
docker login -u thajeztah
Password:
Login Succeeded
cat ~/.docker/config.json
{
"auths": {
"https://index.docker.io/v1/": {
"auth": "REDACTED"
}
}
}
But when resolving the credentials, the credentials stored would not be found;
docker pull -q thajeztah/private-test-image
Error response from daemon: pull access denied for thajeztah/private-test-image, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
With this patch applied:
docker pull -q thajeztah/private-test-image
docker.io/thajeztah/private-test-image:latest
Thanks to mtrmac (Miloslav Trmač) for spotting this mistake!
Suggested-by: Miloslav Trmač <mitr@redhat.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This reverts commit f596202125, and reapplies
79141ce5eb.
> cli/command: remove uses of GetAuthConfigKey, ParseRepositoryInfo
>
> Re-implement locally, based on the code in github.com/docker/docker/registry,
> but leaving out bits that are not used on the client-side, such as
> configuration of Mirrors, and configurable insecure-registry, which
> are not used on the client side.
This commit contains a regression due to a typo in `authConfigKey`;
const authConfigKey = "https:/index.docker.io/v1/"
Which is missing a `/` after the scheme.
Which currently fails the TestRetrieveAuthTokenFromImage test.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It's currently slower because it calls registry.ParseRepositoryInfo,
which does a DNS lookup for hostnames to determine if they're a loopback
address (and marked "insecure");
go test -v -run TestRetrieveAuthTokenFromImage
=== RUN TestRetrieveAuthTokenFromImage
=== RUN TestRetrieveAuthTokenFromImage/no-prefix
=== RUN TestRetrieveAuthTokenFromImage/docker.io
=== RUN TestRetrieveAuthTokenFromImage/index.docker.io
=== RUN TestRetrieveAuthTokenFromImage/registry-1.docker.io
=== RUN TestRetrieveAuthTokenFromImage/registry.hub.docker.com
=== RUN TestRetrieveAuthTokenFromImage/[::1]
=== RUN TestRetrieveAuthTokenFromImage/[::1]:5000
=== RUN TestRetrieveAuthTokenFromImage/127.0.0.1
=== RUN TestRetrieveAuthTokenFromImage/localhost
=== RUN TestRetrieveAuthTokenFromImage/localhost:5000
=== RUN TestRetrieveAuthTokenFromImage/no-auth.example.com
--- PASS: TestRetrieveAuthTokenFromImage (0.35s)
--- PASS: TestRetrieveAuthTokenFromImage/no-prefix (0.00s)
--- PASS: TestRetrieveAuthTokenFromImage/docker.io (0.00s)
--- PASS: TestRetrieveAuthTokenFromImage/index.docker.io (0.00s)
--- PASS: TestRetrieveAuthTokenFromImage/registry-1.docker.io (0.08s)
--- PASS: TestRetrieveAuthTokenFromImage/registry.hub.docker.com (0.12s)
--- PASS: TestRetrieveAuthTokenFromImage/[::1] (0.13s)
--- PASS: TestRetrieveAuthTokenFromImage/[::1]:5000 (0.00s)
--- PASS: TestRetrieveAuthTokenFromImage/127.0.0.1 (0.00s)
--- PASS: TestRetrieveAuthTokenFromImage/localhost (0.00s)
--- PASS: TestRetrieveAuthTokenFromImage/localhost:5000 (0.00s)
--- PASS: TestRetrieveAuthTokenFromImage/no-auth.example.com (0.01s)
PASS
ok github.com/docker/cli/cli/command 1.367s
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
