This commit adds a transfer manager which deduplicates and schedules
transfers, and also an upload manager and download manager that build on
top of the transfer manager to provide high-level interfaces for uploads
and downloads. The push and pull code is modified to use these building
blocks.
Some benefits of the changes:
- Simplification of push/pull code
- Pushes can upload layers concurrently
- Failed downloads and uploads are retried after backoff delays
- Cancellation is supported, but individual transfers will only be
cancelled if all pushes or pulls using them are cancelled.
- The distribution code is decoupled from Docker Engine packages and API
conventions (i.e. streamformatter), which will make it easier to split
out.
This commit also includes unit tests for the new distribution/xfer
package. The tests cover 87.8% of the statements in the package.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 572ce802306a4e919802e5b77cbeca94acda7c0a
Component: engine
Closes#9798
@maintainers please note that this is a change to the UX. We no longer
require the -f flag on `docker tag` to move a tag from an existing image.
However, this does make us more consistent across our commands,
see https://github.com/docker/docker/issues/9798 for the history.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Upstream-commit: 8d4fe141c4c3f456df9c2be75ffe5071d1665717
Component: engine
Add distribution package for managing pulls and pushes. This is based on
the old code in the graph package, with major changes to work with the
new image/layer model.
Add v1 migration code.
Update registry, api/*, and daemon packages to use the reference
package's types where applicable.
Update daemon package to use image/layer/tag stores instead of the graph
package
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: 4352da7803d182a6013a5238ce20a7c749db979a
Component: engine
- Optional "--shm-size=" was added to the sub-command(run, create,and build).
- The size of /dev/shm in the container can be changed
when container is made.
- Being able to specify is a numerical value that applies number,
b, k, m, and g.
- The default value is 64MB, when this option is not set.
- It deals with both native and lxc drivers.
Signed-off-by: NIWA Hideyuki <niwa.hiedyuki@jp.fujitsu.com>
Upstream-commit: 5aeaf2a0c4236711e0981515d8627b30e22a1637
Component: engine
By adding a (*WriteFlusher).Close, we limit the Write calls to possibly
deallocated http response buffers to the lifetime of an http request.
Typically, this is seen as a very confusing panic, the cause is usually a
situation where an http.ResponseWriter is held after request completion. We
avoid the panic by disallowing further writes to the response writer after the
request is completed.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: ec2289b2d9ac79fd5e0f69f56f023dfe8ee78bf8
Component: engine
This is useful for cluster systems such as swarm to sync the image
state when new images are successfully built.
Signed-off-by: Shijiang Wei <mountkin@gmail.com>
Upstream-commit: 2968fa44eb2a53c121ad2b27c519ae47ca7653c3
Component: engine
We should not check if the mux framework internals work as expected in every handler.
The missing parameter error doesn't make sense from the user point of view.
This change initializes a proper vars context if the mux fails to do so and delegates
specific parameter error checks to the handlers.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Upstream-commit: 389ce0aae6a303660e591ef80272322ac82854e2
Component: engine
- use Exists instead of Get
- push Get inside of daemonbuilder
Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
Upstream-commit: 844fb2961976d1bca52cc8560abf6b1ac2ae754a
Component: engine
Adds support for the daemon to handle user namespace maps as a
per-daemon setting.
Support for handling uid/gid mapping is added to the builder,
archive/unarchive packages and functions, all graphdrivers (except
Windows), and the test suite is updated to handle user namespace daemon
rootgraph changes.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
Upstream-commit: 442b45628ee12ebd8e8bd08497896d5fa8eec4bd
Component: engine
- use daemon member directly rather than through a function call
- create GetImage method for use external to deamon
Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
Upstream-commit: ee36b57c8f3c33f1c50ecb4305bd62a962876fe6
Component: engine
- create pass through in daemon for access to functions on daemon
member
- import image
push image
export image and corrections
lookup image & comments
load image
list images
image history & comments
Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
Upstream-commit: 77c592d26a83aa5a56726d6e1741f18f044e64ab
Component: engine
This patch creates interfaces in builder/ for building Docker images.
It is a first step in a series of patches to remove the daemon
dependency on builder and later allow a client-side Dockerfile builder
as well as potential builder plugins.
It is needed because we cannot remove the /build API endpoint, so we
need to keep the server-side Dockerfile builder, but we also want to
reuse the same Dockerfile parser and evaluator for both server-side and
client-side.
builder/dockerfile/ and api/server/builder.go contain implementations
of those interfaces as a refactoring of the current code.
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: e0ef11a4c2c6789e08b61df082b5b9aa70898e7a
Component: engine
Implement basic interfaces to write custom routers that can be plugged
to the server. Remove server coupling with the daemon.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Upstream-commit: da982cf5511814b6897244ecaa9c016f8800340a
Component: engine
Tibor Vass
Daniel Nephin
Justas Brazauskas
Aaron Lehmann
Jess Frazelle
Antonio Murdaca
Doug Davis
Tonis Tiigi
NIWA Hideyuki
Alexander Morozov
Stephen J Day
John Howard
Shijiang Wei
Doug Davis
David Calavera
Morgan Bauer
Vincent Demeester
Phil Estes