d6d8ca6ebe
Swarm has size constraints on the size of configs, but the client-side would
read content into memory, regardless its size. This could lead to either the
client reading too much into memory, or it sending data that's larger than
the size limit of gRPC, which resulted in the error not being handled by
SwarmKit and a generic gRPC error returned.
Reading a config from a file used a system.OpenSequential for reading
([FILE_FLAG_SEQUENTIAL_SCAN]). While there could be a very marginal benefit
to prevent polluting the system's cache (Windows won’t aggressively keep it
in the cache, freeing up system memory for other tasks). These details were
not documented in code, and possibly may be too marginal, but adding a comment
to outline won't hurt so this patch also adds a comment.
This patch:
- Factors out the reading code to a readConfigData, analogous to the
equivalent in secret create.
- Implements reading the data with a limit-reader to prevent reading
large files into memory.
- The limit is based on SwarmKits limits ([MaxConfigSize]), but made
twice that size, just in case larger sizes are supported in future;
the main goal is to have some constraints, and to prevent hitting
the gRPC limit.
- Updates some error messages to include STDIN (when used), or the
filename (when used).
Before this patch:
ls -lh largefile
-rw------- 1 thajeztah staff 8.1M Mar 9 00:19 largefile
docker config create nosuchfile ./nosuchfile
Error reading content from "./nosuchfile": open ./nosuchfile: no such file or directory
docker config create toolarge ./largefile
Error response from daemon: rpc error: code = ResourceExhausted desc = grpc: received message larger than max (8462870 vs. 4194304)
docker config create empty ./emptyfile
Error response from daemon: rpc error: code = InvalidArgument desc = config data must be larger than 0 and less than 1024000 bytes
cat ./largefile | docker config create toolarge -
Error response from daemon: rpc error: code = ResourceExhausted desc = grpc: received message larger than max (8462870 vs. 4194304)
cat ./emptyfile | docker config create empty -
Error response from daemon: rpc error: code = InvalidArgument desc = config data must be larger than 0 and less than 1024000 bytes
With this patch:
docker config create nosuchfile ./nosuchfile
error reading from ./nosuchfile: open ./nosuchfile: no such file or directory
docker config create empty ./emptyfile
error reading from ./emptyfile: data is empty
docker config create toolarge ./largefile
Error response from daemon: rpc error: code = InvalidArgument desc = config data must be larger than 0 and less than 1024000 bytes
cat ./largefile | docker config create toolarge -
Error response from daemon: rpc error: code = InvalidArgument desc = secret data must be larger than 0 and less than 1024000 bytes
cat ./emptyfile | docker config create empty -
error reading from STDIN: data is empty
[FILE_FLAG_SEQUENTIAL_SCAN]: https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilea#FILE_FLAG_SEQUENTIAL_SCAN
[MaxConfigSize]: https://pkg.go.dev/github.com/moby/swarmkit/v2@v2.0.0-20250103191802-8c1959736554/manager/controlapi#MaxConfigSize
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Docker CLI
About
This repository is the home of the Docker CLI.
Development
docker/cli is developed using Docker.
Build CLI from source:
docker buildx bake
Build binaries for all supported platforms:
docker buildx bake cross
Build for a specific platform:
docker buildx bake --set binary.platform=linux/arm64
Build dynamic binary for glibc or musl:
USE_GLIBC=1 docker buildx bake dynbinary
Run all linting:
docker buildx bake lint shellcheck
Run test:
docker buildx bake test
List all the available targets:
make help
In-container development environment
Start an interactive development environment:
make -f docker.Makefile shell
Legal
Brought to you courtesy of our legal counsel. For more context, see the NOTICE document in this repo.
Use and transfer of Docker may be subject to certain restrictions by the United States and other governments.
It is your responsibility to ensure that your use and/or transfer does not violate applicable laws.
For more information, see https://www.bis.doc.gov
Licensing
docker/cli is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.