Remove forked reference package. Use normalized named values
everywhere and familiar functions to convert back to familiar
strings for UX and storage compatibility.
Enforce that the source repository in the distribution metadata
is always a normalized string, ignore invalid values which are not.
Update distribution tests to use normalized values.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: 3a1279393faf78632bf169619d407e584da84b66
Component: engine
Use resolving to repo info as the split point between the
legitimate reference package and forked reference package.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: 0421f5173dbdcb4e4eade5267f274302bb6ab97c
Component: engine
The `digest` data type, used throughout docker for image verification
and identity, has been broken out into `opencontainers/go-digest`. This
PR updates the dependencies and moves uses over to the new type.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: 7a855799175b6b984886ef1cfa337d6df1d4c668
Component: engine
Move configurations into a single file.
Abstract download manager in pull config.
Add supports for schema2 only and schema2 type checking.
Add interface for providing push layers.
Abstract image store to generically handle configurations.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: 3c7676a057a4c0103895f793e407dc6736df139a
Component: engine
When building a manifest during a push operation, all layers must have
an associated descriptor. If a layer is missing a descriptor, that leads
to a panic.
A break inside a switch in layerAlreadyExists meant to break from the
loop surrounding the switch, but instead breaks from the switch. This
causes the loop to continue, and can overwrite the descriptor with an
empty one, leading to the panic.
Also, fix layerAlreadyExists not to abort the push when a speculative
stat on a candidate layer digest fails with an error. This could happen
in situations like a potential cross-repository mount where the user
does not have permission to access the source repository.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: e0702e9f373f5086f53de2485262dde9fc1e7ac2
Component: engine
Classify blobs into three categories based on size.
Use a very limited number of mount attempts and no existence check for
small blobs. Use more attempts for bigger blobs.
Also remember blob associations during layer existence check.
Blob digests are now checked in the target repository from newest to
latest. If the blob exists and the metadata entry does not, it will be
created. If the blob is not found, the metadata entry will be removed.
Signed-off-by: Michal Minář <miminar@redhat.com>
Upstream-commit: 81f7b1f1e50252e9fdd52111f0899d59f19ecb87
Component: engine
to avoid unnecessary blob re-uploads.
Cross repository mount from particular repo will most probably fail if
the user pushing to the registry is not the same as the one who pulled
or pushed to the source repo.
This PR attempts first to cross-repo mount from the source repositories
associated with the pusher's auth config. Then it falls back to other
repositories sorted from the most similar to the target repo to the
least.
It also prevents metadata deletion in cases where cross-repo mount fails
and the auth config hashes differ.
Signed-off-by: Michal Minář <miminar@redhat.com>
Upstream-commit: 0928f3f2e3eda75a295b651d27f9dd992fd951a4
Component: engine
Currently this always uses the schema1 manifest builder. Later, it will
be changed to attempt schema2 first, and fall back when necessary.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: f33fa1b8d3befec31fdf1952ea1190012c812dc7
Component: engine
This commit adds a transfer manager which deduplicates and schedules
transfers, and also an upload manager and download manager that build on
top of the transfer manager to provide high-level interfaces for uploads
and downloads. The push and pull code is modified to use these building
blocks.
Some benefits of the changes:
- Simplification of push/pull code
- Pushes can upload layers concurrently
- Failed downloads and uploads are retried after backoff delays
- Cancellation is supported, but individual transfers will only be
cancelled if all pushes or pulls using them are cancelled.
- The distribution code is decoupled from Docker Engine packages and API
conventions (i.e. streamformatter), which will make it easier to split
out.
This commit also includes unit tests for the new distribution/xfer
package. The tests cover 87.8% of the statements in the package.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 572ce802306a4e919802e5b77cbeca94acda7c0a
Component: engine
