Compare commits

...

19 Commits

Author SHA1 Message Date
b8857225a0 Merge pull request #6013 from thaJeztah/bump_engine
Some checks failed
build / bin-image (push) Has been cancelled
build / prepare-plugins (push) Has been cancelled
build / plugins (push) Has been cancelled
codeql / codeql (push) Has been cancelled
e2e / tests (alpine, 23, connhelper-ssh) (push) Has been cancelled
e2e / tests (alpine, 23, local) (push) Has been cancelled
e2e / tests (alpine, 26, connhelper-ssh) (push) Has been cancelled
e2e / tests (alpine, 26, local) (push) Has been cancelled
e2e / tests (alpine, 27, connhelper-ssh) (push) Has been cancelled
e2e / tests (alpine, 27, local) (push) Has been cancelled
e2e / tests (alpine, 28, connhelper-ssh) (push) Has been cancelled
e2e / tests (alpine, 28, local) (push) Has been cancelled
e2e / tests (debian, 23, connhelper-ssh) (push) Has been cancelled
e2e / tests (debian, 23, local) (push) Has been cancelled
e2e / tests (debian, 26, connhelper-ssh) (push) Has been cancelled
e2e / tests (debian, 26, local) (push) Has been cancelled
e2e / tests (debian, 27, connhelper-ssh) (push) Has been cancelled
e2e / tests (debian, 27, local) (push) Has been cancelled
e2e / tests (debian, 28, connhelper-ssh) (push) Has been cancelled
e2e / tests (debian, 28, local) (push) Has been cancelled
test / ctn (push) Has been cancelled
test / host (macos-13) (push) Has been cancelled
test / host (macos-14) (push) Has been cancelled
validate / validate (lint) (push) Has been cancelled
validate / validate (shellcheck) (push) Has been cancelled
validate / validate (update-authors) (push) Has been cancelled
validate / validate (validate-vendor) (push) Has been cancelled
validate / validate-md (push) Has been cancelled
validate / validate-make (manpages) (push) Has been cancelled
validate / validate-make (yamldocs) (push) Has been cancelled
vendor: github.com/docker/docker 3f46cadf398a (master, v28.0.0-rc.2)
2025-04-16 12:28:40 +00:00
fc04a49c35 vendor: github.com/docker/docker 3f46cadf398a (master, v28.0.0-rc.2)
full diff: https://github.com/docker/docker/compare/v28.1.0-rc.1...3f46cadf398abdf3196230fea41dac96b5d4016e

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-04-16 14:22:08 +02:00
129ab99109 Merge pull request #6011 from thaJeztah/bump_archive
vendor: github.com/moby/go-archive v0.1.0
2025-04-16 13:39:56 +02:00
59a723bda6 Merge pull request #6012 from thaJeztah/bump_dev_tools
Dockerfile: update buildx to v0.23.0, compose v2.33.1
2025-04-16 13:39:33 +02:00
6ca77b6529 Merge pull request #6009 from zhangwenlong8911/master
set CGO_ENABLED=1 on loong64
2025-04-16 13:28:57 +02:00
50900c0da7 Dockerfile: update compose to v2.33.1
Looks like later versions are currently missing on Docker Hub

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-04-16 13:24:28 +02:00
2dcc881d4d Dockerfile: update buildx to v0.23.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-04-16 13:19:14 +02:00
e7a091eceb vendor: github.com/moby/go-archive v0.1.0
full diff: https://github.com/moby/go-archive/compare/21f3f3385ab7...v0.1.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-04-16 13:08:08 +02:00
e04b67f51d Merge pull request #6010 from thaJeztah/tweak_platform_completion
completion: remove generic "os-only" for platforms
2025-04-16 09:44:31 +00:00
557d721299 completion: remove generic "os-only" for platforms
Using `--platform=linux` or `--platform=windows` is not commonly
used (or recommended). Let's remove these from the list of suggested
platforms.

We should tweak this completion further, and sort the list based
on the daemon's platform (putting linux first for a Linux daemon,
and windows first on a Windows daemon), possibly with the correct
architecture (and os-version) included, but we don't yet provide
that information in `/_ping`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-04-16 11:30:26 +02:00
219d3fe25a Merge pull request #5858 from stevvooe/sjd/include-docker-socket
run: flag to include the docker socket
2025-04-16 10:49:27 +02:00
2b28bb649b set CGO_ENABLED=1 on loong64
Signed-off-by: Wenlong Zhang <zhangwenlong@loongson.cn>
2025-04-16 14:52:19 +08:00
1a502e91c9 run: flag to include the Docker API socket
Adds a flag to the create and run command, `--use-api-socket`, that can
be used to start a container with the correctly configured parameters to
ensure that accessing the docker socket will work with out managing bind
mounts and authentication injection.

The implementation in this PR resolves the tokens for the current
credential set in the client and then copies it into a container at the
well know location of /run/secrets/docker/config.json, setting
DOCKER_CONFIG to ensure it is resolved by existing tooling. We use a
compose-compatible secret location with the hope that the CLI and
compose can work together seamlessly.

The bind mount for the socket is resolved from the current context,
erroring out if the flag is set and the provided socket is not a unix
socket.

There are a few drawbacks to this approach but it resolves a long
standing pain point. We'll continue to develop this as we understand
more use cases but it is marked as experimental for now.

Signed-off-by: Stephen Day <stephen.day@docker.com>
2025-04-15 10:57:44 -07:00
1adc1583a7 Merge pull request #6006 from thaJeztah/bump_engine_28.1
vendor: github.com/docker/docker v28.1.0-rc.1
2025-04-15 17:01:40 +02:00
785a12eeef vendor: github.com/docker/docker v28.1.0-rc.1
no diff; same commit, but tagged;
https://github.com/docker/docker/compare/250792c1a540...v28.1.0-rc.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-04-12 08:15:26 +02:00
fc99fe2d08 Merge pull request #5994 from aevesdocker/oss-5
docs: replace sshfs with rclone
2025-04-11 17:00:42 +02:00
b501283743 docs: replace sshfs with rclone
Signed-off-by: aevesdocker <allie.sadler@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-04-11 16:54:46 +02:00
3372bcf821 Merge pull request #6001 from thaJeztah/tweak_prompt
Dockerfile: fix and clean up shell prompt
2025-04-11 16:53:28 +02:00
1f9a55de6a Dockerfile: fix and clean up shell prompt
The existing approach had some issues with how the control-chars
were escaped; also switching to use Dockerfile here-doc to make
it a bit more readable, and add some comments to the `.bashrc`.

Also make sure the MOTD isn't printed multiple times, and only
for interactive shells, and slightly tweak it with some colors.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2025-04-11 13:48:30 +02:00
36 changed files with 787 additions and 530 deletions

View File

@ -12,8 +12,8 @@ ARG GOTESTSUM_VERSION=v1.12.0
# BUILDX_VERSION sets the version of buildx to use for the e2e tests.
# It must be a tag in the docker.io/docker/buildx-bin image repository
# on Docker Hub.
ARG BUILDX_VERSION=0.20.1
ARG COMPOSE_VERSION=v2.32.4
ARG BUILDX_VERSION=0.23.0
ARG COMPOSE_VERSION=v2.33.1
FROM --platform=$BUILDPLATFORM tonistiigi/xx:${XX_VERSION} AS xx

View File

@ -152,7 +152,6 @@ func NoComplete(_ *cobra.Command, _ []string, _ string) ([]string, cobra.ShellCo
}
var commonPlatforms = []string{
"linux",
"linux/386",
"linux/amd64",
"linux/arm",
@ -169,10 +168,8 @@ var commonPlatforms = []string{
// Not yet supported
"linux/riscv64",
"windows",
"windows/amd64",
"wasip1",
"wasip1/wasm",
}

View File

@ -11,6 +11,7 @@ import (
"github.com/docker/cli/internal/test"
"github.com/docker/docker/api/types/container"
"github.com/moby/go-archive"
"github.com/moby/go-archive/compression"
"gotest.tools/v3/assert"
is "gotest.tools/v3/assert/cmp"
"gotest.tools/v3/fs"
@ -74,7 +75,7 @@ func TestRunCopyFromContainerToFilesystem(t *testing.T) {
cli := test.NewFakeCli(&fakeClient{
containerCopyFromFunc: func(ctr, srcPath string) (io.ReadCloser, container.PathStat, error) {
assert.Check(t, is.Equal("container", ctr))
readCloser, err := archive.Tar(srcDir.Path(), archive.Uncompressed)
readCloser, err := archive.Tar(srcDir.Path(), compression.None)
return readCloser, container.PathStat{}, err
},
})

View File

@ -1,11 +1,15 @@
package container
import (
"archive/tar"
"bytes"
"context"
"fmt"
"io"
"net/netip"
"os"
"path"
"strings"
"github.com/containerd/platforms"
"github.com/distribution/reference"
@ -13,13 +17,17 @@ import (
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/command/completion"
"github.com/docker/cli/cli/command/image"
"github.com/docker/cli/cli/config/configfile"
"github.com/docker/cli/cli/config/types"
"github.com/docker/cli/cli/internal/jsonstream"
"github.com/docker/cli/cli/streams"
"github.com/docker/cli/cli/trust"
"github.com/docker/cli/opts"
"github.com/docker/docker/api/types/container"
imagetypes "github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/mount"
"github.com/docker/docker/api/types/versions"
"github.com/docker/docker/client"
"github.com/docker/docker/errdefs"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
@ -35,11 +43,12 @@ const (
)
type createOptions struct {
name string
platform string
untrusted bool
pull string // always, missing, never
quiet bool
name string
platform string
untrusted bool
pull string // always, missing, never
quiet bool
useAPISocket bool
}
// NewCreateCommand creates a new cobra.Command for `docker create`
@ -70,6 +79,8 @@ func NewCreateCommand(dockerCli command.Cli) *cobra.Command {
flags.StringVar(&options.name, "name", "", "Assign a name to the container")
flags.StringVar(&options.pull, "pull", PullImageMissing, `Pull image before creating ("`+PullImageAlways+`", "|`+PullImageMissing+`", "`+PullImageNever+`")`)
flags.BoolVarP(&options.quiet, "quiet", "q", false, "Suppress the pull output")
flags.BoolVarP(&options.useAPISocket, "use-api-socket", "", false, "Bind mount Docker API socket and required auth")
flags.SetAnnotation("use-api-socket", "experimentalCLI", nil) // Marks flag as experimental for now.
// Add an explicit help that doesn't have a `-h` to prevent the conflict
// with hostname
@ -179,20 +190,20 @@ func (cid *cidFile) Write(id string) error {
return nil
}
func newCIDFile(path string) (*cidFile, error) {
if path == "" {
func newCIDFile(cidPath string) (*cidFile, error) {
if cidPath == "" {
return &cidFile{}, nil
}
if _, err := os.Stat(path); err == nil {
return nil, errors.Errorf("container ID file found, make sure the other container isn't running or delete %s", path)
if _, err := os.Stat(cidPath); err == nil {
return nil, errors.Errorf("container ID file found, make sure the other container isn't running or delete %s", cidPath)
}
f, err := os.Create(path)
f, err := os.Create(cidPath)
if err != nil {
return nil, errors.Wrap(err, "failed to create the container ID file")
}
return &cidFile{path: path, file: f}, nil
return &cidFile{path: cidPath, file: f}, nil
}
//nolint:gocyclo
@ -239,6 +250,73 @@ func createContainer(ctx context.Context, dockerCli command.Cli, containerCfg *c
return nil
}
const dockerConfigPathInContainer = "/run/secrets/docker/config.json"
var apiSocketCreds map[string]types.AuthConfig
if options.useAPISocket {
// We'll create two new mounts to handle this flag:
// 1. Mount the actual docker socket.
// 2. A synthezised ~/.docker/config.json with resolved tokens.
socket := dockerCli.DockerEndpoint().Host
if !strings.HasPrefix(socket, "unix://") {
return "", fmt.Errorf("flag --use-api-socket can only be used with unix sockets: docker endpoint %s incompatible", socket)
}
socket = strings.TrimPrefix(socket, "unix://") // should we confirm absolute path?
containerCfg.HostConfig.Mounts = append(containerCfg.HostConfig.Mounts, mount.Mount{
Type: mount.TypeBind,
Source: socket,
Target: "/var/run/docker.sock",
BindOptions: &mount.BindOptions{},
})
/*
Ideally, we'd like to copy the config into a tmpfs but unfortunately,
the mounts won't be in place until we start the container. This can
leave around the config if the container doesn't get deleted.
We are using the most compose-secret-compatible approach,
which is implemented at
https://github.com/docker/compose/blob/main/pkg/compose/convergence.go#L737
// Prepare a tmpfs mount for our credentials so they go away after the
// container exits. We'll copy into this mount after the container is
// created.
containerCfg.HostConfig.Mounts = append(containerCfg.HostConfig.Mounts, mount.Mount{
Type: mount.TypeTmpfs,
Target: "/docker/",
TmpfsOptions: &mount.TmpfsOptions{
SizeBytes: 1 << 20, // only need a small partition
Mode: 0o600,
},
})
*/
var envvarPresent bool
for _, envvar := range containerCfg.Config.Env {
if strings.HasPrefix(envvar, "DOCKER_CONFIG=") {
envvarPresent = true
}
}
// If the DOCKER_CONFIG env var is already present, we assume the client knows
// what they're doing and don't inject the creds.
if !envvarPresent {
// Set our special little location for the config file.
containerCfg.Config.Env = append(containerCfg.Config.Env,
"DOCKER_CONFIG="+path.Dir(dockerConfigPathInContainer))
// Resolve this here for later, ensuring we error our before we create the container.
creds, err := dockerCli.ConfigFile().GetAllCredentials()
if err != nil {
return "", fmt.Errorf("resolving credentials failed: %w", err)
}
apiSocketCreds = creds // inject these after container creation.
}
}
var platform *specs.Platform
// Engine API version 1.41 first introduced the option to specify platform on
// create. It will produce an error if you try to set a platform on older API
@ -286,11 +364,25 @@ func createContainer(ctx context.Context, dockerCli command.Cli, containerCfg *c
if warn := localhostDNSWarning(*hostConfig); warn != "" {
response.Warnings = append(response.Warnings, warn)
}
containerID = response.ID
for _, w := range response.Warnings {
_, _ = fmt.Fprintln(dockerCli.Err(), "WARNING:", w)
}
err = containerIDFile.Write(response.ID)
return response.ID, err
err = containerIDFile.Write(containerID)
if options.useAPISocket && apiSocketCreds != nil {
// Create a new config file with just the auth.
newConfig := &configfile.ConfigFile{
AuthConfigs: apiSocketCreds,
}
if err := copyDockerConfigIntoContainer(ctx, dockerCli.Client(), containerID, dockerConfigPathInContainer, newConfig); err != nil {
return "", fmt.Errorf("injecting docker config.json into container failed: %w", err)
}
}
return containerID, err
}
// check the DNS settings passed via --dns against localhost regexp to warn if
@ -321,3 +413,39 @@ func validatePullOpt(val string) error {
)
}
}
// copyDockerConfigIntoContainer takes the client configuration and copies it
// into the container.
//
// The path should be an absolute path in the container, commonly
// /root/.docker/config.json.
func copyDockerConfigIntoContainer(ctx context.Context, dockerAPI client.APIClient, containerID string, configPath string, config *configfile.ConfigFile) error {
var configBuf bytes.Buffer
if err := config.SaveToWriter(&configBuf); err != nil {
return fmt.Errorf("saving creds: %w", err)
}
// We don't need to get super fancy with the tar creation.
var tarBuf bytes.Buffer
tarWriter := tar.NewWriter(&tarBuf)
tarWriter.WriteHeader(&tar.Header{
Name: configPath,
Size: int64(configBuf.Len()),
Mode: 0o600,
})
if _, err := io.Copy(tarWriter, &configBuf); err != nil {
return fmt.Errorf("writing config to tar file for config copy: %w", err)
}
if err := tarWriter.Close(); err != nil {
return fmt.Errorf("closing tar for config copy failed: %w", err)
}
if err := dockerAPI.CopyToContainer(ctx, containerID, "/",
&tarBuf, container.CopyToContainerOptions{}); err != nil {
return fmt.Errorf("copying config.json into container failed: %w", err)
}
return nil
}

View File

@ -60,6 +60,7 @@ func NewRunCommand(dockerCli command.Cli) *cobra.Command {
flags.StringVar(&options.detachKeys, "detach-keys", "", "Override the key sequence for detaching a container")
flags.StringVar(&options.pull, "pull", PullImageMissing, `Pull image before running ("`+PullImageAlways+`", "`+PullImageMissing+`", "`+PullImageNever+`")`)
flags.BoolVarP(&options.quiet, "quiet", "q", false, "Suppress the pull output")
flags.BoolVarP(&options.createOptions.useAPISocket, "use-api-socket", "", false, "Bind mount Docker API socket and required auth")
// Add an explicit help that doesn't have a `-h` to prevent the conflict
// with hostname

View File

@ -20,6 +20,7 @@ import (
"github.com/docker/docker/pkg/streamformatter"
"github.com/docker/docker/pkg/stringid"
"github.com/moby/go-archive"
"github.com/moby/go-archive/compression"
"github.com/moby/patternmatcher"
"github.com/pkg/errors"
)
@ -163,7 +164,7 @@ func GetContextFromReader(rc io.ReadCloser, dockerfileName string) (out io.ReadC
return nil, "", err
}
tarArchive, err := archive.Tar(dockerfileDir, archive.Uncompressed)
tarArchive, err := archive.Tar(dockerfileDir, compression.None)
if err != nil {
return nil, "", err
}
@ -178,8 +179,7 @@ func GetContextFromReader(rc io.ReadCloser, dockerfileName string) (out io.ReadC
// IsArchive checks for the magic bytes of a tar or any supported compression
// algorithm.
func IsArchive(header []byte) bool {
compression := archive.DetectCompression(header)
if compression != archive.Uncompressed {
if compression.Detect(header) != compression.None {
return true
}
r := tar.NewReader(bytes.NewBuffer(header))
@ -427,7 +427,7 @@ func Compress(buildCtx io.ReadCloser) (io.ReadCloser, error) {
pipeReader, pipeWriter := io.Pipe()
go func() {
compressWriter, err := archive.CompressStream(pipeWriter, archive.Gzip)
compressWriter, err := compression.CompressStream(pipeWriter, archive.Gzip)
if err != nil {
pipeWriter.CloseWithError(err)
}

View File

@ -11,6 +11,7 @@ import (
"testing"
"github.com/moby/go-archive"
"github.com/moby/go-archive/compression"
"github.com/moby/patternmatcher"
"gotest.tools/v3/assert"
is "gotest.tools/v3/assert/cmp"
@ -165,7 +166,7 @@ func TestGetContextFromReaderTar(t *testing.T) {
contextDir := createTestTempDir(t)
createTestTempFile(t, contextDir, DefaultDockerfileName, dockerfileContents)
tarStream, err := archive.Tar(contextDir, archive.Uncompressed)
tarStream, err := archive.Tar(contextDir, compression.None)
assert.NilError(t, err)
tarArchive, relDockerfile, err := GetContextFromReader(tarStream, DefaultDockerfileName)

View File

@ -15,7 +15,7 @@ import (
"github.com/docker/cli/internal/test"
"github.com/docker/docker/api/types"
"github.com/google/go-cmp/cmp"
"github.com/moby/go-archive"
"github.com/moby/go-archive/compression"
"gotest.tools/v3/assert"
"gotest.tools/v3/fs"
"gotest.tools/v3/skip"
@ -54,7 +54,7 @@ func TestRunBuildDockerfileFromStdinWithCompress(t *testing.T) {
assert.DeepEqual(t, expected, fakeBuild.filenames(t))
header := buffer.Bytes()[:10]
assert.Equal(t, archive.Gzip, archive.DetectCompression(header))
assert.Equal(t, compression.Gzip, compression.Detect(header))
}
func TestRunBuildResetsUidAndGidInContext(t *testing.T) {

View File

@ -13,6 +13,7 @@ import (
"github.com/docker/cli/cli/command/completion"
"github.com/docker/docker/api/types"
"github.com/moby/go-archive"
"github.com/moby/go-archive/compression"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
@ -99,14 +100,14 @@ func runCreate(ctx context.Context, dockerCli command.Cli, options pluginCreateO
return err
}
compression := archive.Uncompressed
comp := compression.None
if options.compress {
logrus.Debugf("compression enabled")
compression = archive.Gzip
comp = compression.Gzip
}
createCtx, err := archive.TarWithOptions(absContextDir, &archive.TarOptions{
Compression: compression,
Compression: comp,
})
if err != nil {
return err

View File

@ -6,7 +6,7 @@ ARG ALPINE_VERSION=3.21
# BUILDX_VERSION sets the version of buildx to install in the dev container.
# It must be a valid tag in the docker.io/docker/buildx-bin image repository
# on Docker Hub.
ARG BUILDX_VERSION=0.20.1
ARG BUILDX_VERSION=0.23.0
FROM docker/buildx-bin:${BUILDX_VERSION} AS buildx
FROM golang:${GO_VERSION}-alpine${ALPINE_VERSION} AS golang
@ -47,9 +47,28 @@ RUN apk add --no-cache \
jq \
nano
RUN echo -e "\nYou are now in a development container. Run '\e\033[1mmake help\e\033[0m' to learn about\navailable make targets.\n" > /etc/motd \
&& echo -e "cat /etc/motd\nPS1=\"\e[0;32m\u@docker-cli-dev\\$ \e[0m\"" >> /root/.bashrc \
&& echo -e "source /etc/bash/bash_completion.sh" >> /root/.bashrc
RUN <<-'EOF'
cat > /etc/motd <<-'EOM'
\e[1;32mYou are now in a development container.\e[0m
Run \e[1;36mmake help\e[0m to see available targets.
EOM
cat >> /root/.bashrc <<-'EOB'
# print the MOTD when opening the dev-container (interactive shell only).
if [[ $- == *i* ]] && [[ -z "$MOTD_SHOWN" ]]; then
printf "%b\n" "$(cat /etc/motd)"
export MOTD_SHOWN=1
fi
# set a custom prompt to make it more visible when inside the dev-container.
PS1='\[\e[0;32m\]\u@docker-cli-dev\$ \[\e[0m\]'
# set-up bash completion for testing.
source /etc/bash/bash_completion.sh
EOB
EOF
CMD ["/bin/bash"]
ENV DISABLE_WARN_OUTSIDE_CONTAINER=1
ENV PATH=$PATH:/go/src/github.com/docker/cli/build

View File

@ -35,31 +35,38 @@ Plugins that start successfully are listed as enabled in the output.
After a plugin is installed, you can use it as an option for another Docker
operation, such as creating a volume.
In the following example, you install the `sshfs` plugin, verify that it is
In the following example, you install the [`rclone` plugin](https://rclone.org/docker/), verify that it is
enabled, and use it to create a volume.
> [!NOTE]
> This example is intended for instructional purposes only. Once the volume is
> created, your SSH password to the remote host is exposed as plaintext when
> inspecting the volume. Delete the volume as soon as you are done with the
> example.
> This example is intended for instructional purposes only.
1. Install the `sshfs` plugin.
1. Set up the pre-requisite directories. By default they must exist on the host at the following locations:
- `/var/lib/docker-plugins/rclone/config`. Reserved for the `rclone.conf` config file and must exist even if it's empty and the config file is not present.
- `/var/lib/docker-plugins/rclone/cache`. Holds the plugin state file as well as optional VFS caches.
2. Install the `rclone` plugin.
```console
$ docker plugin install vieux/sshfs
$ docker plugin install rclone/docker-volume-rclone --alias rclone
Plugin "vieux/sshfs" is requesting the following privileges:
- network: [host]
- capabilities: [CAP_SYS_ADMIN]
Do you grant the above permissions? [y/N] y
vieux/sshfs
Plugin "rclone/docker-volume-rclone" is requesting the following privileges:
- network: [host]
- mount: [/var/lib/docker-plugins/rclone/config]
- mount: [/var/lib/docker-plugins/rclone/cache]
- device: [/dev/fuse]
- capabilities: [CAP_SYS_ADMIN]
Do you grant the above permissions? [y/N]
```
The plugin requests 2 privileges:
The plugin requests 5 privileges:
- It needs access to the `host` network.
- Access to pre-requisite directories to mount to store:
- Your Rclone config files
- Temporary cache data
- Gives access to the FUSE (Filesystem in Userspace) device. This is required because Rclone uses FUSE to mount remote storage as if it were a local filesystem.
- It needs the `CAP_SYS_ADMIN` capability, which allows the plugin to run
the `mount` command.
@ -68,24 +75,25 @@ enabled, and use it to create a volume.
```console
$ docker plugin ls
ID NAME TAG DESCRIPTION ENABLED
69553ca1d789 vieux/sshfs latest the `sshfs` plugin true
ID NAME DESCRIPTION ENABLED
aede66158353 rclone:latest Rclone volume plugin for Docker true
```
3. Create a volume using the plugin.
This example mounts the `/remote` directory on host `1.2.3.4` into a
volume named `sshvolume`.
volume named `rclonevolume`.
This volume can now be mounted into containers.
```console
$ docker volume create \
-d vieux/sshfs \
--name sshvolume \
-o sshcmd=user@1.2.3.4:/remote \
-o password=$(cat file_containing_password_for_remote_host)
sshvolume
-d rclone \
--name rclonevolume \
-o type=sftp \
-o path=remote \
-o sftp-host=1.2.3.4 \
-o sftp-user=user \
-o "sftp-password=$(cat file_containing_password_for_remote_host)"
```
4. Verify that the volume was created successfully.
@ -94,21 +102,21 @@ enabled, and use it to create a volume.
$ docker volume ls
DRIVER NAME
vieux/sshfs sshvolume
rclone rclonevolume
```
5. Start a container that uses the volume `sshvolume`.
5. Start a container that uses the volume `rclonevolume`.
```console
$ docker run --rm -v sshvolume:/data busybox ls /data
$ docker run --rm -v rclonevolume:/data busybox ls /data
<content of /remote on machine 1.2.3.4>
```
6. Remove the volume `sshvolume`
6. Remove the volume `rclonevolume`
```console
$ docker volume rm sshvolume
$ docker volume rm rclonevolume
sshvolume
```

View File

@ -104,6 +104,7 @@ Create a new container
| `--tmpfs` | `list` | | Mount a tmpfs directory |
| `-t`, `--tty` | `bool` | | Allocate a pseudo-TTY |
| `--ulimit` | `ulimit` | | Ulimit options |
| `--use-api-socket` | `bool` | | Bind mount Docker API socket and required auth |
| `-u`, `--user` | `string` | | Username or UID (format: <name\|uid>[:<group\|gid>]) |
| `--userns` | `string` | | User namespace to use |
| `--uts` | `string` | | UTS namespace to use |

View File

@ -107,6 +107,7 @@ Create and run a new container from an image
| [`--tmpfs`](#tmpfs) | `list` | | Mount a tmpfs directory |
| [`-t`](#tty), [`--tty`](#tty) | `bool` | | Allocate a pseudo-TTY |
| [`--ulimit`](#ulimit) | `ulimit` | | Ulimit options |
| `--use-api-socket` | `bool` | | Bind mount Docker API socket and required auth |
| `-u`, `--user` | `string` | | Username or UID (format: <name\|uid>[:<group\|gid>]) |
| [`--userns`](#userns) | `string` | | User namespace to use |
| [`--uts`](#uts) | `string` | | UTS namespace to use |

View File

@ -104,6 +104,7 @@ Create a new container
| `--tmpfs` | `list` | | Mount a tmpfs directory |
| `-t`, `--tty` | `bool` | | Allocate a pseudo-TTY |
| `--ulimit` | `ulimit` | | Ulimit options |
| `--use-api-socket` | `bool` | | Bind mount Docker API socket and required auth |
| `-u`, `--user` | `string` | | Username or UID (format: <name\|uid>[:<group\|gid>]) |
| `--userns` | `string` | | User namespace to use |
| `--uts` | `string` | | UTS namespace to use |

View File

@ -107,6 +107,7 @@ Create and run a new container from an image
| `--tmpfs` | `list` | | Mount a tmpfs directory |
| `-t`, `--tty` | `bool` | | Allocate a pseudo-TTY |
| `--ulimit` | `ulimit` | | Ulimit options |
| `--use-api-socket` | `bool` | | Bind mount Docker API socket and required auth |
| `-u`, `--user` | `string` | | Username or UID (format: <name\|uid>[:<group\|gid>]) |
| `--userns` | `string` | | User namespace to use |
| `--uts` | `string` | | UTS namespace to use |

View File

@ -58,7 +58,7 @@ if [ -z "$CGO_ENABLED" ]; then
case "$(go env GOOS)" in
linux)
case "$(go env GOARCH)" in
amd64|arm64|arm|s390x|riscv64)
amd64|arm64|arm|s390x|riscv64|loong64)
CGO_ENABLED=1
;;
*)

View File

@ -14,7 +14,7 @@ require (
github.com/distribution/reference v0.6.0
github.com/docker/cli-docs-tool v0.9.0
github.com/docker/distribution v2.8.3+incompatible
github.com/docker/docker v28.0.5-0.20250411131724-250792c1a540+incompatible // v28.1.0-dev
github.com/docker/docker v28.1.0-rc.1.0.20250416120748-3f46cadf398a+incompatible // master (v28.0.0-rc.2)
github.com/docker/docker-credential-helpers v0.9.2
github.com/docker/go-connections v0.5.0
github.com/docker/go-units v0.5.0
@ -26,7 +26,7 @@ require (
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510
github.com/google/uuid v1.6.0
github.com/mattn/go-runewidth v0.0.16
github.com/moby/go-archive v0.0.0-20250404171912-21f3f3385ab7
github.com/moby/go-archive v0.1.0
github.com/moby/patternmatcher v0.6.0
github.com/moby/swarmkit/v2 v2.0.0-20250103191802-8c1959736554
github.com/moby/sys/atomicwriter v0.1.0

View File

@ -52,8 +52,8 @@ github.com/docker/cli-docs-tool v0.9.0/go.mod h1:ClrwlNW+UioiRyH9GiAOe1o3J/TsY3T
github.com/docker/distribution v2.7.1+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/distribution v2.8.3+incompatible h1:AtKxIZ36LoNK51+Z6RpzLpddBirtxJnzDrHLEKxTAYk=
github.com/docker/distribution v2.8.3+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/docker v28.0.5-0.20250411131724-250792c1a540+incompatible h1:kI3XNcOjX01jlSUpE47xI/u72GnbrNhaeULmdW76IOc=
github.com/docker/docker v28.0.5-0.20250411131724-250792c1a540+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker v28.1.0-rc.1.0.20250416120748-3f46cadf398a+incompatible h1:6crmQOQPsBh2CAs7EUV/ME+H5a1m6cYI0neATzoViJU=
github.com/docker/docker v28.1.0-rc.1.0.20250416120748-3f46cadf398a+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker-credential-helpers v0.9.2 h1:50JF7ADQiHdAVBRtg/vy883Y4U5+5GmPOBNtUU+X+6A=
github.com/docker/docker-credential-helpers v0.9.2/go.mod h1:x+4Gbw9aGmChi3qTLZj8Dfn0TD20M/fuWy0E5+WDeCo=
github.com/docker/go v1.5.1-1.0.20160303222718-d30aec9fd63c h1:lzqkGL9b3znc+ZUgi7FlLnqjQhcXxkNM/quxIjBVMD0=
@ -165,8 +165,8 @@ github.com/mitchellh/mapstructure v1.0.0 h1:vVpGvMXJPqSDh2VYHF7gsfQj8Ncx+Xw5Y1KH
github.com/mitchellh/mapstructure v1.0.0/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0=
github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo=
github.com/moby/go-archive v0.0.0-20250404171912-21f3f3385ab7 h1:CWAY9uG9JhmLmnM7T64+bV0C9IraDrvxEwXq1HJ7hhk=
github.com/moby/go-archive v0.0.0-20250404171912-21f3f3385ab7/go.mod h1:G9B+YoujNohJmrIYFBpSd54GTUB4lt9S+xVQvsJyFuo=
github.com/moby/go-archive v0.1.0 h1:Kk/5rdW/g+H8NHdJW2gsXyZ7UnzvJNOy6VKJqueWdcQ=
github.com/moby/go-archive v0.1.0/go.mod h1:G9B+YoujNohJmrIYFBpSd54GTUB4lt9S+xVQvsJyFuo=
github.com/moby/patternmatcher v0.6.0 h1:GmP9lR19aU5GqSSFko+5pRqHi+Ohk1O69aFiKkVGiPk=
github.com/moby/patternmatcher v0.6.0/go.mod h1:hDPoyOpDY7OrrMDLaYoY3hf52gNCR/YOUYxkhApJIxc=
github.com/moby/swarmkit/v2 v2.0.0-20250103191802-8c1959736554 h1:DMHJbgyNZWyrPKYjCYt2IxEO7KA0eSd4fo6KQsv2W84=

View File

@ -10491,13 +10491,9 @@ paths:
### Image tarball format
An image tarball contains one directory per image layer (named using its long ID), each containing these files:
An image tarball contains [Content as defined in the OCI Image Layout Specification](https://github.com/opencontainers/image-spec/blob/v1.1.1/image-layout.md#content).
- `VERSION`: currently `1.0` - the file format version
- `json`: detailed layer information, similar to `docker inspect layer_id`
- `layer.tar`: A tarfile containing the filesystem changes in this layer
The `layer.tar` file contains `aufs` style `.wh..wh.aufs` files and directories for storing attribute changes and deletions.
Additionally, includes the manifest.json file associated with a backwards compatible docker save format.
If the tarball defines a repository, the tarball should also include a `repositories` file at the root that contains a list of repository and tag names mapped to layer IDs.

2
vendor/github.com/moby/go-archive/.gitattributes generated vendored Normal file
View File

@ -0,0 +1,2 @@
*.go -text diff=golang
*.go text eol=lf

1
vendor/github.com/moby/go-archive/.gitignore generated vendored Normal file
View File

@ -0,0 +1 @@
/coverage.txt

33
vendor/github.com/moby/go-archive/.golangci.yml generated vendored Normal file
View File

@ -0,0 +1,33 @@
version: "2"
issues:
# Disable maximum issues count per one linter.
max-issues-per-linter: 0
# Disable maximum count of issues with the same text.
max-same-issues: 0
linters:
enable:
- errorlint
- unconvert
- unparam
exclusions:
generated: disable
presets:
- comments
- std-error-handling
settings:
staticcheck:
# Enable all options, with some exceptions.
# For defaults, see https://golangci-lint.run/usage/linters/#staticcheck
checks:
- all
- -QF1008 # Omit embedded fields from selector expression; https://staticcheck.dev/docs/checks/#QF1008
- -ST1003 # Poorly chosen identifier; https://staticcheck.dev/docs/checks/#ST1003
formatters:
enable:
- gofumpt
- goimports
exclusions:
generated: disable

View File

@ -3,32 +3,24 @@ package archive
import (
"archive/tar"
"bufio"
"bytes"
"compress/bzip2"
"compress/gzip"
"context"
"encoding/binary"
"errors"
"fmt"
"io"
"os"
"os/exec"
"path/filepath"
"runtime"
"runtime/debug"
"strconv"
"strings"
"sync"
"sync/atomic"
"syscall"
"time"
"github.com/containerd/log"
"github.com/klauspost/compress/zstd"
"github.com/moby/patternmatcher"
"github.com/moby/sys/sequential"
"github.com/moby/sys/user"
"github.com/moby/go-archive/compression"
"github.com/moby/go-archive/tarheader"
)
// ImpliedDirectoryMode represents the mode (Unix permissions) applied to directories that are implied by files in a
@ -45,7 +37,9 @@ const ImpliedDirectoryMode = 0o755
type (
// Compression is the state represents if compressed or not.
Compression int
//
// Deprecated: use [compression.Compression].
Compression = compression.Compression
// WhiteoutFormat is the format of whiteouts unpacked
WhiteoutFormat int
@ -58,7 +52,7 @@ type (
TarOptions struct {
IncludeFiles []string
ExcludePatterns []string
Compression Compression
Compression compression.Compression
NoLchown bool
IDMap user.IdentityMapping
ChownOpts *ChownOpts
@ -102,11 +96,11 @@ func NewDefaultArchiver() *Archiver {
type breakoutError error
const (
Uncompressed Compression = 0 // Uncompressed represents the uncompressed.
Bzip2 Compression = 1 // Bzip2 is bzip2 compression algorithm.
Gzip Compression = 2 // Gzip is gzip compression algorithm.
Xz Compression = 3 // Xz is xz compression algorithm.
Zstd Compression = 4 // Zstd is zstd compression algorithm.
Uncompressed = compression.None // Deprecated: use [compression.None].
Bzip2 = compression.Bzip2 // Deprecated: use [compression.Bzip2].
Gzip = compression.Gzip // Deprecated: use [compression.Gzip].
Xz = compression.Xz // Deprecated: use [compression.Xz].
Zstd = compression.Zstd // Deprecated: use [compression.Zstd].
)
const (
@ -122,7 +116,7 @@ func IsArchivePath(path string) bool {
return false
}
defer file.Close()
rdr, err := DecompressStream(file)
rdr, err := compression.DecompressStream(file)
if err != nil {
return false
}
@ -132,242 +126,25 @@ func IsArchivePath(path string) bool {
return err == nil
}
const (
zstdMagicSkippableStart = 0x184D2A50
zstdMagicSkippableMask = 0xFFFFFFF0
)
var (
bzip2Magic = []byte{0x42, 0x5A, 0x68}
gzipMagic = []byte{0x1F, 0x8B, 0x08}
xzMagic = []byte{0xFD, 0x37, 0x7A, 0x58, 0x5A, 0x00}
zstdMagic = []byte{0x28, 0xb5, 0x2f, 0xfd}
)
type matcher = func([]byte) bool
func magicNumberMatcher(m []byte) matcher {
return func(source []byte) bool {
return bytes.HasPrefix(source, m)
}
}
// zstdMatcher detects zstd compression algorithm.
// Zstandard compressed data is made of one or more frames.
// There are two frame formats defined by Zstandard: Zstandard frames and Skippable frames.
// See https://datatracker.ietf.org/doc/html/rfc8878#section-3 for more details.
func zstdMatcher() matcher {
return func(source []byte) bool {
if bytes.HasPrefix(source, zstdMagic) {
// Zstandard frame
return true
}
// skippable frame
if len(source) < 8 {
return false
}
// magic number from 0x184D2A50 to 0x184D2A5F.
if binary.LittleEndian.Uint32(source[:4])&zstdMagicSkippableMask == zstdMagicSkippableStart {
return true
}
return false
}
}
// DetectCompression detects the compression algorithm of the source.
func DetectCompression(source []byte) Compression {
compressionMap := map[Compression]matcher{
Bzip2: magicNumberMatcher(bzip2Magic),
Gzip: magicNumberMatcher(gzipMagic),
Xz: magicNumberMatcher(xzMagic),
Zstd: zstdMatcher(),
}
for _, compression := range []Compression{Bzip2, Gzip, Xz, Zstd} {
fn := compressionMap[compression]
if fn(source) {
return compression
}
}
return Uncompressed
}
func xzDecompress(ctx context.Context, archive io.Reader) (io.ReadCloser, error) {
args := []string{"xz", "-d", "-c", "-q"}
return cmdStream(exec.CommandContext(ctx, args[0], args[1:]...), archive)
}
func gzDecompress(ctx context.Context, buf io.Reader) (io.ReadCloser, error) {
if noPigzEnv := os.Getenv("MOBY_DISABLE_PIGZ"); noPigzEnv != "" {
noPigz, err := strconv.ParseBool(noPigzEnv)
if err != nil {
log.G(ctx).WithError(err).Warn("invalid value in MOBY_DISABLE_PIGZ env var")
}
if noPigz {
log.G(ctx).Debugf("Use of pigz is disabled due to MOBY_DISABLE_PIGZ=%s", noPigzEnv)
return gzip.NewReader(buf)
}
}
unpigzPath, err := exec.LookPath("unpigz")
if err != nil {
log.G(ctx).Debugf("unpigz binary not found, falling back to go gzip library")
return gzip.NewReader(buf)
}
log.G(ctx).Debugf("Using %s to decompress", unpigzPath)
return cmdStream(exec.CommandContext(ctx, unpigzPath, "-d", "-c"), buf)
}
type readCloserWrapper struct {
io.Reader
closer func() error
closed atomic.Bool
}
func (r *readCloserWrapper) Close() error {
if !r.closed.CompareAndSwap(false, true) {
log.G(context.TODO()).Error("subsequent attempt to close readCloserWrapper")
if log.GetLevel() >= log.DebugLevel {
log.G(context.TODO()).Errorf("stack trace: %s", string(debug.Stack()))
}
return nil
}
if r.closer != nil {
return r.closer()
}
return nil
}
var bufioReader32KPool = &sync.Pool{
New: func() interface{} { return bufio.NewReaderSize(nil, 32*1024) },
}
type bufferedReader struct {
buf *bufio.Reader
}
func newBufferedReader(r io.Reader) *bufferedReader {
buf := bufioReader32KPool.Get().(*bufio.Reader)
buf.Reset(r)
return &bufferedReader{buf}
}
func (r *bufferedReader) Read(p []byte) (int, error) {
if r.buf == nil {
return 0, io.EOF
}
n, err := r.buf.Read(p)
if err == io.EOF {
r.buf.Reset(nil)
bufioReader32KPool.Put(r.buf)
r.buf = nil
}
return n, err
}
func (r *bufferedReader) Peek(n int) ([]byte, error) {
if r.buf == nil {
return nil, io.EOF
}
return r.buf.Peek(n)
//
// Deprecated: use [compression.Detect].
func DetectCompression(source []byte) compression.Compression {
return compression.Detect(source)
}
// DecompressStream decompresses the archive and returns a ReaderCloser with the decompressed archive.
//
// Deprecated: use [compression.DecompressStream].
func DecompressStream(archive io.Reader) (io.ReadCloser, error) {
buf := newBufferedReader(archive)
bs, err := buf.Peek(10)
if err != nil && err != io.EOF {
// Note: we'll ignore any io.EOF error because there are some odd
// cases where the layer.tar file will be empty (zero bytes) and
// that results in an io.EOF from the Peek() call. So, in those
// cases we'll just treat it as a non-compressed stream and
// that means just create an empty layer.
// See Issue 18170
return nil, err
}
compression := DetectCompression(bs)
switch compression {
case Uncompressed:
return &readCloserWrapper{
Reader: buf,
}, nil
case Gzip:
ctx, cancel := context.WithCancel(context.Background())
gzReader, err := gzDecompress(ctx, buf)
if err != nil {
cancel()
return nil, err
}
return &readCloserWrapper{
Reader: gzReader,
closer: func() error {
cancel()
return gzReader.Close()
},
}, nil
case Bzip2:
bz2Reader := bzip2.NewReader(buf)
return &readCloserWrapper{
Reader: bz2Reader,
}, nil
case Xz:
ctx, cancel := context.WithCancel(context.Background())
xzReader, err := xzDecompress(ctx, buf)
if err != nil {
cancel()
return nil, err
}
return &readCloserWrapper{
Reader: xzReader,
closer: func() error {
cancel()
return xzReader.Close()
},
}, nil
case Zstd:
zstdReader, err := zstd.NewReader(buf)
if err != nil {
return nil, err
}
return &readCloserWrapper{
Reader: zstdReader,
closer: func() error {
zstdReader.Close()
return nil
},
}, nil
default:
return nil, fmt.Errorf("Unsupported compression format %s", (&compression).Extension())
}
return compression.DecompressStream(archive)
}
type nopWriteCloser struct {
io.Writer
}
func (nopWriteCloser) Close() error { return nil }
// CompressStream compresses the dest with specified compression algorithm.
func CompressStream(dest io.Writer, compression Compression) (io.WriteCloser, error) {
switch compression {
case Uncompressed:
return nopWriteCloser{dest}, nil
case Gzip:
return gzip.NewWriter(dest), nil
case Bzip2, Xz:
// archive/bzip2 does not support writing, and there is no xz support at all
// However, this is not a problem as docker only currently generates gzipped tars
return nil, fmt.Errorf("Unsupported compression format %s", (&compression).Extension())
default:
return nil, fmt.Errorf("Unsupported compression format %s", (&compression).Extension())
}
//
// Deprecated: use [compression.CompressStream].
func CompressStream(dest io.Writer, comp compression.Compression) (io.WriteCloser, error) {
return compression.CompressStream(dest, comp)
}
// TarModifierFunc is a function that can be passed to ReplaceFileTarWrapper to
@ -416,7 +193,7 @@ func ReplaceFileTarWrapper(inputTarStream io.ReadCloser, mods map[string]TarModi
var originalHeader *tar.Header
for {
originalHeader, err = tarReader.Next()
if err == io.EOF {
if errors.Is(err, io.EOF) {
break
}
if err != nil {
@ -458,90 +235,11 @@ func ReplaceFileTarWrapper(inputTarStream io.ReadCloser, mods map[string]TarModi
return pipeReader
}
// Extension returns the extension of a file that uses the specified compression algorithm.
func (compression *Compression) Extension() string {
switch *compression {
case Uncompressed:
return "tar"
case Bzip2:
return "tar.bz2"
case Gzip:
return "tar.gz"
case Xz:
return "tar.xz"
case Zstd:
return "tar.zst"
}
return ""
}
// assert that we implement [tar.FileInfoNames].
//
// TODO(thaJeztah): disabled to allow compiling on < go1.23. un-comment once we drop support for older versions of go.
// var _ tar.FileInfoNames = (*nosysFileInfo)(nil)
// nosysFileInfo hides the system-dependent info of the wrapped FileInfo to
// prevent tar.FileInfoHeader from introspecting it and potentially calling into
// glibc.
//
// It implements [tar.FileInfoNames] to further prevent [tar.FileInfoHeader]
// from performing any lookups on go1.23 and up. see https://go.dev/issue/50102
type nosysFileInfo struct {
os.FileInfo
}
// Uname stubs out looking up username. It implements [tar.FileInfoNames]
// to prevent [tar.FileInfoHeader] from loading libraries to perform
// username lookups.
func (fi nosysFileInfo) Uname() (string, error) {
return "", nil
}
// Gname stubs out looking up group-name. It implements [tar.FileInfoNames]
// to prevent [tar.FileInfoHeader] from loading libraries to perform
// username lookups.
func (fi nosysFileInfo) Gname() (string, error) {
return "", nil
}
func (fi nosysFileInfo) Sys() interface{} {
// A Sys value of type *tar.Header is safe as it is system-independent.
// The tar.FileInfoHeader function copies the fields into the returned
// header without performing any OS lookups.
if sys, ok := fi.FileInfo.Sys().(*tar.Header); ok {
return sys
}
return nil
}
// sysStat, if non-nil, populates hdr from system-dependent fields of fi.
var sysStat func(fi os.FileInfo, hdr *tar.Header) error
// FileInfoHeaderNoLookups creates a partially-populated tar.Header from fi.
//
// Compared to the archive/tar.FileInfoHeader function, this function is safe to
// call from a chrooted process as it does not populate fields which would
// require operating system lookups. It behaves identically to
// tar.FileInfoHeader when fi is a FileInfo value returned from
// tar.Header.FileInfo().
//
// When fi is a FileInfo for a native file, such as returned from os.Stat() and
// os.Lstat(), the returned Header value differs from one returned from
// tar.FileInfoHeader in the following ways. The Uname and Gname fields are not
// set as OS lookups would be required to populate them. The AccessTime and
// ChangeTime fields are not currently set (not yet implemented) although that
// is subject to change. Callers which require the AccessTime or ChangeTime
// fields to be zeroed should explicitly zero them out in the returned Header
// value to avoid any compatibility issues in the future.
// Deprecated: use [tarheader.FileInfoHeaderNoLookups].
func FileInfoHeaderNoLookups(fi os.FileInfo, link string) (*tar.Header, error) {
hdr, err := tar.FileInfoHeader(nosysFileInfo{fi}, link)
if err != nil {
return nil, err
}
if sysStat != nil {
return hdr, sysStat(fi, hdr)
}
return hdr, nil
return tarheader.FileInfoHeaderNoLookups(fi, link)
}
// FileInfoHeader creates a populated Header from fi.
@ -552,7 +250,7 @@ func FileInfoHeaderNoLookups(fi os.FileInfo, link string) (*tar.Header, error) {
// precision, and the Uname and Gname fields are only set when fi is a FileInfo
// value returned from tar.Header.FileInfo().
func FileInfoHeader(name string, fi os.FileInfo, link string) (*tar.Header, error) {
hdr, err := FileInfoHeaderNoLookups(fi, link)
hdr, err := tarheader.FileInfoHeaderNoLookups(fi, link)
if err != nil {
return nil, err
}
@ -768,7 +466,7 @@ func createTarFile(path, extractDir string, hdr *tar.Header, reader io.Reader, o
case tar.TypeDir:
// Create directory unless it exists as a directory already.
// In that case we just want to merge the two
if fi, err := os.Lstat(path); !(err == nil && fi.IsDir()) {
if fi, err := os.Lstat(path); err != nil || !fi.IsDir() {
if err := os.Mkdir(path, hdrInfo.Mode()); err != nil {
return err
}
@ -908,8 +606,8 @@ func createTarFile(path, extractDir string, hdr *tar.Header, reader io.Reader, o
// Tar creates an archive from the directory at `path`, and returns it as a
// stream of bytes.
func Tar(path string, compression Compression) (io.ReadCloser, error) {
return TarWithOptions(path, &TarOptions{Compression: compression})
func Tar(path string, comp compression.Compression) (io.ReadCloser, error) {
return TarWithOptions(path, &TarOptions{Compression: comp})
}
// TarWithOptions creates an archive from the directory at `path`, only including files whose relative
@ -945,7 +643,7 @@ func NewTarballer(srcPath string, options *TarOptions) (*Tarballer, error) {
pipeReader, pipeWriter := io.Pipe()
compressWriter, err := CompressStream(pipeWriter, options.Compression)
compressWriter, err := compression.CompressStream(pipeWriter, options.Compression)
if err != nil {
return nil, err
}
@ -1031,7 +729,8 @@ func (t *Tarballer) Do() {
)
walkRoot := getWalkRoot(t.srcPath, include)
filepath.WalkDir(walkRoot, func(filePath string, f os.DirEntry, err error) error {
// TODO(thaJeztah): should this error be handled?
_ = filepath.WalkDir(walkRoot, func(filePath string, f os.DirEntry, err error) error {
if err != nil {
log.G(context.TODO()).Errorf("Tar: Can't stat file %s to tar: %s", t.srcPath, err)
return nil
@ -1135,7 +834,7 @@ func (t *Tarballer) Do() {
if err := ta.addTarFile(filePath, relFilePath); err != nil {
log.G(context.TODO()).Errorf("Can't add file %s to tar: %s", filePath, err)
// if pipe is broken, stop writing tar stream to it
if err == io.ErrClosedPipe {
if errors.Is(err, io.ErrClosedPipe) {
return err
}
}
@ -1155,7 +854,7 @@ func Unpack(decompressedArchive io.Reader, dest string, options *TarOptions) err
loop:
for {
hdr, err := tr.Next()
if err == io.EOF {
if errors.Is(err, io.EOF) {
// end of tar archive
break
}
@ -1217,7 +916,7 @@ loop:
continue
}
if !(fi.IsDir() && hdr.Typeflag == tar.TypeDir) {
if !fi.IsDir() || hdr.Typeflag != tar.TypeDir {
if err := os.RemoveAll(path); err != nil {
return err
}
@ -1309,7 +1008,7 @@ func UntarUncompressed(tarArchive io.Reader, dest string, options *TarOptions) e
// Handler for teasing out the automatic decompression
func untarHandler(tarArchive io.Reader, dest string, options *TarOptions, decompress bool) error {
if tarArchive == nil {
return fmt.Errorf("Empty archive")
return errors.New("empty archive")
}
dest = filepath.Clean(dest)
if options == nil {
@ -1321,7 +1020,7 @@ func untarHandler(tarArchive io.Reader, dest string, options *TarOptions, decomp
r := tarArchive
if decompress {
decompressedArchive, err := DecompressStream(tarArchive)
decompressedArchive, err := compression.DecompressStream(tarArchive)
if err != nil {
return err
}
@ -1335,15 +1034,14 @@ func untarHandler(tarArchive io.Reader, dest string, options *TarOptions, decomp
// TarUntar is a convenience function which calls Tar and Untar, with the output of one piped into the other.
// If either Tar or Untar fails, TarUntar aborts and returns the error.
func (archiver *Archiver) TarUntar(src, dst string) error {
archive, err := TarWithOptions(src, &TarOptions{Compression: Uncompressed})
archive, err := Tar(src, compression.None)
if err != nil {
return err
}
defer archive.Close()
options := &TarOptions{
return archiver.Untar(archive, dst, &TarOptions{
IDMap: archiver.IDMapping,
}
return archiver.Untar(archive, dst, options)
})
}
// UntarPath untar a file from path to a destination, src is the source tar file path.
@ -1353,10 +1051,9 @@ func (archiver *Archiver) UntarPath(src, dst string) error {
return err
}
defer archive.Close()
options := &TarOptions{
return archiver.Untar(archive, dst, &TarOptions{
IDMap: archiver.IDMapping,
}
return archiver.Untar(archive, dst, options)
})
}
// CopyWithTar creates a tar archive of filesystem path `src`, and
@ -1393,7 +1090,7 @@ func (archiver *Archiver) CopyFileWithTar(src, dst string) (err error) {
}
if srcSt.IsDir() {
return fmt.Errorf("Can't copy a directory")
return errors.New("can't copy a directory")
}
// Clean up the trailing slash. This must be done in an operating
@ -1421,7 +1118,7 @@ func (archiver *Archiver) CopyFileWithTar(src, dst string) (err error) {
}
defer srcF.Close()
hdr, err := FileInfoHeaderNoLookups(srcSt, "")
hdr, err := tarheader.FileInfoHeaderNoLookups(srcSt, "")
if err != nil {
return err
}
@ -1470,43 +1167,3 @@ func remapIDs(idMapping user.IdentityMapping, hdr *tar.Header) error {
hdr.Uid, hdr.Gid = uid, gid
return err
}
// cmdStream executes a command, and returns its stdout as a stream.
// If the command fails to run or doesn't complete successfully, an error
// will be returned, including anything written on stderr.
func cmdStream(cmd *exec.Cmd, input io.Reader) (io.ReadCloser, error) {
cmd.Stdin = input
pipeR, pipeW := io.Pipe()
cmd.Stdout = pipeW
var errBuf bytes.Buffer
cmd.Stderr = &errBuf
// Run the command and return the pipe
if err := cmd.Start(); err != nil {
return nil, err
}
// Ensure the command has exited before we clean anything up
done := make(chan struct{})
// Copy stdout to the returned pipe
go func() {
if err := cmd.Wait(); err != nil {
pipeW.CloseWithError(fmt.Errorf("%s: %s", err, errBuf.String()))
} else {
pipeW.Close()
}
close(done)
}()
return &readCloserWrapper{
Reader: pipeR,
closer: func() error {
// Close pipeR, and then wait for the command to complete before returning. We have to close pipeR first, as
// cmd.Wait waits for any non-file stdout/stderr/stdin to close.
err := pipeR.Close()
<-done
return err
},
}, nil
}

View File

@ -7,17 +7,12 @@ import (
"errors"
"os"
"path/filepath"
"runtime"
"strings"
"syscall"
"golang.org/x/sys/unix"
)
func init() {
sysStat = statUnix
}
// addLongPathPrefix adds the Windows long path prefix to the path provided if
// it does not already have it. It is a no-op on platforms other than Windows.
func addLongPathPrefix(srcPath string) string {
@ -38,40 +33,6 @@ func chmodTarEntry(perm os.FileMode) os.FileMode {
return perm // noop for unix as golang APIs provide perm bits correctly
}
// statUnix populates hdr from system-dependent fields of fi without performing
// any OS lookups.
func statUnix(fi os.FileInfo, hdr *tar.Header) error {
// Devmajor and Devminor are only needed for special devices.
// In FreeBSD, RDev for regular files is -1 (unless overridden by FS):
// https://cgit.freebsd.org/src/tree/sys/kern/vfs_default.c?h=stable/13#n1531
// (NODEV is -1: https://cgit.freebsd.org/src/tree/sys/sys/param.h?h=stable/13#n241).
// ZFS in particular does not override the default:
// https://cgit.freebsd.org/src/tree/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c?h=stable/13#n2027
// Since `Stat_t.Rdev` is uint64, the cast turns -1 into (2^64 - 1).
// Such large values cannot be encoded in a tar header.
if runtime.GOOS == "freebsd" && hdr.Typeflag != tar.TypeBlock && hdr.Typeflag != tar.TypeChar {
return nil
}
s, ok := fi.Sys().(*syscall.Stat_t)
if !ok {
return nil
}
hdr.Uid = int(s.Uid)
hdr.Gid = int(s.Gid)
if s.Mode&unix.S_IFBLK != 0 ||
s.Mode&unix.S_IFCHR != 0 {
hdr.Devmajor = int64(unix.Major(uint64(s.Rdev))) //nolint: unconvert
hdr.Devminor = int64(unix.Minor(uint64(s.Rdev))) //nolint: unconvert
}
return nil
}
func getInodeFromStat(stat interface{}) (uint64, error) {
s, ok := stat.(*syscall.Stat_t)
if !ok {

View File

@ -41,11 +41,6 @@ func chmodTarEntry(perm os.FileMode) os.FileMode {
return perm | 0o111
}
func setHeaderForSpecialDevice(hdr *tar.Header, name string, stat interface{}) (err error) {
// do nothing. no notion of Rdev, Nlink in stat on Windows
return
}
func getInodeFromStat(stat interface{}) (uint64, error) {
// do nothing. no notion of Inode in stat on Windows
return 0, nil

View File

@ -75,7 +75,7 @@ func sameFsTime(a, b time.Time) bool {
// Changes walks the path rw and determines changes for the files in the path,
// with respect to the parent layers
func Changes(layers []string, rw string) ([]Change, error) {
return changes(layers, rw, aufsDeletedFile, aufsMetadataSkip)
return collectChanges(layers, rw, aufsDeletedFile, aufsMetadataSkip)
}
func aufsMetadataSkip(path string) (skip bool, err error) {
@ -103,7 +103,7 @@ type (
deleteChange func(string, string, os.FileInfo) (string, error)
)
func changes(layers []string, rw string, dc deleteChange, sc skipChange) ([]Change, error) {
func collectChanges(layers []string, rw string, dc deleteChange, sc skipChange) ([]Change, error) {
var (
changes []Change
changedDirs = make(map[string]struct{})

View File

@ -132,14 +132,7 @@ func (w *walker) walk(path string, i1, i2 os.FileInfo) (err error) {
ix1 := 0
ix2 := 0
for {
if ix1 >= len(names1) {
break
}
if ix2 >= len(names2) {
break
}
for ix1 < len(names1) && ix2 < len(names2) {
ni1 := names1[ix1]
ni2 := names2[ix2]

View File

@ -0,0 +1,263 @@
package compression
import (
"bufio"
"bytes"
"compress/bzip2"
"compress/gzip"
"context"
"errors"
"fmt"
"io"
"os"
"os/exec"
"strconv"
"sync"
"github.com/containerd/log"
"github.com/klauspost/compress/zstd"
)
// Compression is the state represents if compressed or not.
type Compression int
const (
None Compression = 0 // None represents the uncompressed.
Bzip2 Compression = 1 // Bzip2 is bzip2 compression algorithm.
Gzip Compression = 2 // Gzip is gzip compression algorithm.
Xz Compression = 3 // Xz is xz compression algorithm.
Zstd Compression = 4 // Zstd is zstd compression algorithm.
)
// Extension returns the extension of a file that uses the specified compression algorithm.
func (c *Compression) Extension() string {
switch *c {
case None:
return "tar"
case Bzip2:
return "tar.bz2"
case Gzip:
return "tar.gz"
case Xz:
return "tar.xz"
case Zstd:
return "tar.zst"
default:
return ""
}
}
type readCloserWrapper struct {
io.Reader
closer func() error
}
func (r *readCloserWrapper) Close() error {
if r.closer != nil {
return r.closer()
}
return nil
}
type nopWriteCloser struct {
io.Writer
}
func (nopWriteCloser) Close() error { return nil }
var bufioReader32KPool = &sync.Pool{
New: func() interface{} { return bufio.NewReaderSize(nil, 32*1024) },
}
type bufferedReader struct {
buf *bufio.Reader
}
func newBufferedReader(r io.Reader) *bufferedReader {
buf := bufioReader32KPool.Get().(*bufio.Reader)
buf.Reset(r)
return &bufferedReader{buf}
}
func (r *bufferedReader) Read(p []byte) (int, error) {
if r.buf == nil {
return 0, io.EOF
}
n, err := r.buf.Read(p)
if errors.Is(err, io.EOF) {
r.buf.Reset(nil)
bufioReader32KPool.Put(r.buf)
r.buf = nil
}
return n, err
}
func (r *bufferedReader) Peek(n int) ([]byte, error) {
if r.buf == nil {
return nil, io.EOF
}
return r.buf.Peek(n)
}
// DecompressStream decompresses the archive and returns a ReaderCloser with the decompressed archive.
func DecompressStream(archive io.Reader) (io.ReadCloser, error) {
buf := newBufferedReader(archive)
bs, err := buf.Peek(10)
if err != nil && !errors.Is(err, io.EOF) {
// Note: we'll ignore any io.EOF error because there are some odd
// cases where the layer.tar file will be empty (zero bytes) and
// that results in an io.EOF from the Peek() call. So, in those
// cases we'll just treat it as a non-compressed stream and
// that means just create an empty layer.
// See Issue 18170
return nil, err
}
switch compression := Detect(bs); compression {
case None:
return &readCloserWrapper{
Reader: buf,
}, nil
case Gzip:
ctx, cancel := context.WithCancel(context.Background())
gzReader, err := gzipDecompress(ctx, buf)
if err != nil {
cancel()
return nil, err
}
return &readCloserWrapper{
Reader: gzReader,
closer: func() error {
cancel()
return gzReader.Close()
},
}, nil
case Bzip2:
bz2Reader := bzip2.NewReader(buf)
return &readCloserWrapper{
Reader: bz2Reader,
}, nil
case Xz:
ctx, cancel := context.WithCancel(context.Background())
xzReader, err := xzDecompress(ctx, buf)
if err != nil {
cancel()
return nil, err
}
return &readCloserWrapper{
Reader: xzReader,
closer: func() error {
cancel()
return xzReader.Close()
},
}, nil
case Zstd:
zstdReader, err := zstd.NewReader(buf)
if err != nil {
return nil, err
}
return &readCloserWrapper{
Reader: zstdReader,
closer: func() error {
zstdReader.Close()
return nil
},
}, nil
default:
return nil, fmt.Errorf("unsupported compression format (%d)", compression)
}
}
// CompressStream compresses the dest with specified compression algorithm.
func CompressStream(dest io.Writer, compression Compression) (io.WriteCloser, error) {
switch compression {
case None:
return nopWriteCloser{dest}, nil
case Gzip:
return gzip.NewWriter(dest), nil
case Bzip2:
// archive/bzip2 does not support writing.
return nil, errors.New("unsupported compression format: tar.bz2")
case Xz:
// there is no xz support at all
// However, this is not a problem as docker only currently generates gzipped tars
return nil, errors.New("unsupported compression format: tar.xz")
default:
return nil, fmt.Errorf("unsupported compression format (%d)", compression)
}
}
func xzDecompress(ctx context.Context, archive io.Reader) (io.ReadCloser, error) {
args := []string{"xz", "-d", "-c", "-q"}
return cmdStream(exec.CommandContext(ctx, args[0], args[1:]...), archive)
}
func gzipDecompress(ctx context.Context, buf io.Reader) (io.ReadCloser, error) {
if noPigzEnv := os.Getenv("MOBY_DISABLE_PIGZ"); noPigzEnv != "" {
noPigz, err := strconv.ParseBool(noPigzEnv)
if err != nil {
log.G(ctx).WithError(err).Warn("invalid value in MOBY_DISABLE_PIGZ env var")
}
if noPigz {
log.G(ctx).Debugf("Use of pigz is disabled due to MOBY_DISABLE_PIGZ=%s", noPigzEnv)
return gzip.NewReader(buf)
}
}
unpigzPath, err := exec.LookPath("unpigz")
if err != nil {
log.G(ctx).Debugf("unpigz binary not found, falling back to go gzip library")
return gzip.NewReader(buf)
}
log.G(ctx).Debugf("Using %s to decompress", unpigzPath)
return cmdStream(exec.CommandContext(ctx, unpigzPath, "-d", "-c"), buf)
}
// cmdStream executes a command, and returns its stdout as a stream.
// If the command fails to run or doesn't complete successfully, an error
// will be returned, including anything written on stderr.
func cmdStream(cmd *exec.Cmd, in io.Reader) (io.ReadCloser, error) {
reader, writer := io.Pipe()
cmd.Stdin = in
cmd.Stdout = writer
var errBuf bytes.Buffer
cmd.Stderr = &errBuf
// Run the command and return the pipe
if err := cmd.Start(); err != nil {
return nil, err
}
// Ensure the command has exited before we clean anything up
done := make(chan struct{})
// Copy stdout to the returned pipe
go func() {
if err := cmd.Wait(); err != nil {
_ = writer.CloseWithError(fmt.Errorf("%w: %s", err, errBuf.String()))
} else {
_ = writer.Close()
}
close(done)
}()
return &readCloserWrapper{
Reader: reader,
closer: func() error {
// Close pipeR, and then wait for the command to complete before returning. We have to close pipeR first, as
// cmd.Wait waits for any non-file stdout/stderr/stdin to close.
err := reader.Close()
<-done
return err
},
}, nil
}

View File

@ -0,0 +1,65 @@
package compression
import (
"bytes"
"encoding/binary"
)
const (
zstdMagicSkippableStart = 0x184D2A50
zstdMagicSkippableMask = 0xFFFFFFF0
)
var (
bzip2Magic = []byte{0x42, 0x5A, 0x68}
gzipMagic = []byte{0x1F, 0x8B, 0x08}
xzMagic = []byte{0xFD, 0x37, 0x7A, 0x58, 0x5A, 0x00}
zstdMagic = []byte{0x28, 0xb5, 0x2f, 0xfd}
)
type matcher = func([]byte) bool
// Detect detects the compression algorithm of the source.
func Detect(source []byte) Compression {
compressionMap := map[Compression]matcher{
Bzip2: magicNumberMatcher(bzip2Magic),
Gzip: magicNumberMatcher(gzipMagic),
Xz: magicNumberMatcher(xzMagic),
Zstd: zstdMatcher(),
}
for _, compression := range []Compression{Bzip2, Gzip, Xz, Zstd} {
fn := compressionMap[compression]
if fn(source) {
return compression
}
}
return None
}
func magicNumberMatcher(m []byte) matcher {
return func(source []byte) bool {
return bytes.HasPrefix(source, m)
}
}
// zstdMatcher detects zstd compression algorithm.
// Zstandard compressed data is made of one or more frames.
// There are two frame formats defined by Zstandard: Zstandard frames and Skippable frames.
// See https://datatracker.ietf.org/doc/html/rfc8878#section-3 for more details.
func zstdMatcher() matcher {
return func(source []byte) bool {
if bytes.HasPrefix(source, zstdMagic) {
// Zstandard frame
return true
}
// skippable frame
if len(source) < 8 {
return false
}
// magic number from 0x184D2A50 to 0x184D2A5F.
if binary.LittleEndian.Uint32(source[:4])&zstdMagicSkippableMask == zstdMagicSkippableStart {
return true
}
return false
}
}

View File

@ -128,7 +128,6 @@ func TarResourceRebase(sourcePath, rebaseName string) (content io.ReadCloser, _
func TarResourceRebaseOpts(sourceBase string, rebaseName string) *TarOptions {
filter := []string{sourceBase}
return &TarOptions{
Compression: Uncompressed,
IncludeFiles: filter,
IncludeSourceDir: true,
RebaseNames: map[string]string{
@ -335,7 +334,7 @@ func RebaseArchiveEntries(srcContent io.Reader, oldBase, newBase string) io.Read
for {
hdr, err := srcTar.Next()
if err == io.EOF {
if errors.Is(err, io.EOF) {
// Signals end of archive.
rebasedTar.Close()
w.Close()

View File

@ -4,4 +4,6 @@ package archive
import "golang.org/x/sys/unix"
var mknod = unix.Mknod
func mknod(path string, mode uint32, dev uint64) error {
return unix.Mknod(path, mode, dev)
}

View File

@ -3,6 +3,7 @@ package archive
import (
"archive/tar"
"context"
"errors"
"fmt"
"io"
"os"
@ -11,6 +12,8 @@ import (
"strings"
"github.com/containerd/log"
"github.com/moby/go-archive/compression"
)
// UnpackLayer unpack `layer` to a `dest`. The stream `layer` can be
@ -35,7 +38,7 @@ func UnpackLayer(dest string, layer io.Reader, options *TarOptions) (size int64,
// Iterate through the files in the archive.
for {
hdr, err := tr.Next()
if err == io.EOF {
if errors.Is(err, io.EOF) {
// end of tar archive
break
}
@ -149,7 +152,7 @@ func UnpackLayer(dest string, layer io.Reader, options *TarOptions) (size int64,
// the layer is also a directory. Then we want to merge them (i.e.
// just apply the metadata from the layer).
if fi, err := os.Lstat(path); err == nil {
if !(fi.IsDir() && hdr.Typeflag == tar.TypeDir) {
if !fi.IsDir() || hdr.Typeflag != tar.TypeDir {
if err := os.RemoveAll(path); err != nil {
return 0, err
}
@ -165,7 +168,7 @@ func UnpackLayer(dest string, layer io.Reader, options *TarOptions) (size int64,
linkBasename := filepath.Base(hdr.Linkname)
srcHdr = aufsHardlinks[linkBasename]
if srcHdr == nil {
return 0, fmt.Errorf("Invalid aufs hardlink")
return 0, errors.New("invalid aufs hardlink")
}
tmpFile, err := os.Open(filepath.Join(aufsTempdir, linkBasename))
if err != nil {
@ -221,18 +224,18 @@ func ApplyUncompressedLayer(dest string, layer io.Reader, options *TarOptions) (
// IsEmpty checks if the tar archive is empty (doesn't contain any entries).
func IsEmpty(rd io.Reader) (bool, error) {
decompRd, err := DecompressStream(rd)
decompRd, err := compression.DecompressStream(rd)
if err != nil {
return true, fmt.Errorf("failed to decompress archive: %v", err)
return true, fmt.Errorf("failed to decompress archive: %w", err)
}
defer decompRd.Close()
tarReader := tar.NewReader(decompRd)
if _, err := tarReader.Next(); err != nil {
if err == io.EOF {
if errors.Is(err, io.EOF) {
return true, nil
}
return false, fmt.Errorf("failed to read next archive header: %v", err)
return false, fmt.Errorf("failed to read next archive header: %w", err)
}
return false, nil
@ -247,7 +250,7 @@ func applyLayerHandler(dest string, layer io.Reader, options *TarOptions, decomp
defer restore()
if decompress {
decompLayer, err := DecompressStream(layer)
decompLayer, err := compression.DecompressStream(layer)
if err != nil {
return 0, err
}

View File

@ -0,0 +1,67 @@
package tarheader
import (
"archive/tar"
"os"
)
// assert that we implement [tar.FileInfoNames].
var _ tar.FileInfoNames = (*nosysFileInfo)(nil)
// nosysFileInfo hides the system-dependent info of the wrapped FileInfo to
// prevent tar.FileInfoHeader from introspecting it and potentially calling into
// glibc.
//
// It implements [tar.FileInfoNames] to further prevent [tar.FileInfoHeader]
// from performing any lookups on go1.23 and up. see https://go.dev/issue/50102
type nosysFileInfo struct {
os.FileInfo
}
// Uname stubs out looking up username. It implements [tar.FileInfoNames]
// to prevent [tar.FileInfoHeader] from loading libraries to perform
// username lookups.
func (fi nosysFileInfo) Uname() (string, error) {
return "", nil
}
// Gname stubs out looking up group-name. It implements [tar.FileInfoNames]
// to prevent [tar.FileInfoHeader] from loading libraries to perform
// username lookups.
func (fi nosysFileInfo) Gname() (string, error) {
return "", nil
}
func (fi nosysFileInfo) Sys() interface{} {
// A Sys value of type *tar.Header is safe as it is system-independent.
// The tar.FileInfoHeader function copies the fields into the returned
// header without performing any OS lookups.
if sys, ok := fi.FileInfo.Sys().(*tar.Header); ok {
return sys
}
return nil
}
// FileInfoHeaderNoLookups creates a partially-populated tar.Header from fi.
//
// Compared to the archive/tar.FileInfoHeader function, this function is safe to
// call from a chrooted process as it does not populate fields which would
// require operating system lookups. It behaves identically to
// tar.FileInfoHeader when fi is a FileInfo value returned from
// tar.Header.FileInfo().
//
// When fi is a FileInfo for a native file, such as returned from os.Stat() and
// os.Lstat(), the returned Header value differs from one returned from
// tar.FileInfoHeader in the following ways. The Uname and Gname fields are not
// set as OS lookups would be required to populate them. The AccessTime and
// ChangeTime fields are not currently set (not yet implemented) although that
// is subject to change. Callers which require the AccessTime or ChangeTime
// fields to be zeroed should explicitly zero them out in the returned Header
// value to avoid any compatibility issues in the future.
func FileInfoHeaderNoLookups(fi os.FileInfo, link string) (*tar.Header, error) {
hdr, err := tar.FileInfoHeader(nosysFileInfo{fi}, link)
if err != nil {
return nil, err
}
return hdr, sysStat(fi, hdr)
}

View File

@ -0,0 +1,46 @@
//go:build !windows
package tarheader
import (
"archive/tar"
"os"
"runtime"
"syscall"
"golang.org/x/sys/unix"
)
// sysStat populates hdr from system-dependent fields of fi without performing
// any OS lookups.
func sysStat(fi os.FileInfo, hdr *tar.Header) error {
// Devmajor and Devminor are only needed for special devices.
// In FreeBSD, RDev for regular files is -1 (unless overridden by FS):
// https://cgit.freebsd.org/src/tree/sys/kern/vfs_default.c?h=stable/13#n1531
// (NODEV is -1: https://cgit.freebsd.org/src/tree/sys/sys/param.h?h=stable/13#n241).
// ZFS in particular does not override the default:
// https://cgit.freebsd.org/src/tree/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c?h=stable/13#n2027
// Since `Stat_t.Rdev` is uint64, the cast turns -1 into (2^64 - 1).
// Such large values cannot be encoded in a tar header.
if runtime.GOOS == "freebsd" && hdr.Typeflag != tar.TypeBlock && hdr.Typeflag != tar.TypeChar {
return nil
}
s, ok := fi.Sys().(*syscall.Stat_t)
if !ok {
return nil
}
hdr.Uid = int(s.Uid)
hdr.Gid = int(s.Gid)
if s.Mode&unix.S_IFBLK != 0 ||
s.Mode&unix.S_IFCHR != 0 {
hdr.Devmajor = int64(unix.Major(uint64(s.Rdev))) //nolint: unconvert
hdr.Devminor = int64(unix.Minor(uint64(s.Rdev))) //nolint: unconvert
}
return nil
}

View File

@ -0,0 +1,12 @@
package tarheader
import (
"archive/tar"
"os"
)
// sysStat populates hdr from system-dependent fields of fi without performing
// any OS lookups. It is a no-op on Windows.
func sysStat(os.FileInfo, *tar.Header) error {
return nil
}

6
vendor/modules.txt vendored
View File

@ -58,7 +58,7 @@ github.com/docker/distribution/registry/client/transport
github.com/docker/distribution/registry/storage/cache
github.com/docker/distribution/registry/storage/cache/memory
github.com/docker/distribution/uuid
# github.com/docker/docker v28.0.5-0.20250411131724-250792c1a540+incompatible
# github.com/docker/docker v28.1.0-rc.1.0.20250416120748-3f46cadf398a+incompatible
## explicit
github.com/docker/docker/api
github.com/docker/docker/api/types
@ -195,9 +195,11 @@ github.com/miekg/pkcs11
# github.com/moby/docker-image-spec v1.3.1
## explicit; go 1.18
github.com/moby/docker-image-spec/specs-go/v1
# github.com/moby/go-archive v0.0.0-20250404171912-21f3f3385ab7
# github.com/moby/go-archive v0.1.0
## explicit; go 1.23.0
github.com/moby/go-archive
github.com/moby/go-archive/compression
github.com/moby/go-archive/tarheader
# github.com/moby/patternmatcher v0.6.0
## explicit; go 1.19
github.com/moby/patternmatcher