With digests being added by default, all images have multiple references.
The check for whether force is required to remove the reference should use the new check for single reference which accounts for digest references.
This change restores pre-1.12 behavior and ensures images are not accidentally left dangling while a container is running.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: 1f7a9b1ab3d261de5be7d490e7e4f978f317242f
Component: engine
The error message;
unable to delete .... (must be forced) - image is referenced in one or more repositories
Looks to be incorrect
Given the following images:
docker images | grep 2d36b1c0ea40
registry 2 2d36b1c0ea40 7 weeks ago 171.2 MB
registry 2.4.1 2d36b1c0ea40 7 weeks ago 171.2 MB
Removing the image by *id* fails:
docker rmi 2d36b1c0ea40
Error response from daemon: conflict: unable to delete 2d36b1c0ea40 (must be forced) - image is referenced in one or more repositories
However, after untagging one image:
docker rmi registry:2
Untagged: registry:2
Removing the image works:
docker rmi 2d36b1c0ea40
Untagged: registry:2.4.1
Deleted: sha256:2d36b1c0ea40159adc8b36f7563f1d7a6d443384fe2611e8b393c1cb3ae2e6ad
Deleted: sha256:7abfddbf4e61927307b6646010845eeb7513ecc6541f33ea6103b2493e36aa4e
Deleted: sha256:f512d7699dbb2994fe15d30ee1d404e57b58c3c310617b4471db649680b4cfa0
Deleted: sha256:7291e34714908270aeda93f1dc681485f9734d41314e2fdc6c11f32ffa782a21
Deleted: sha256:548f6562929484f3d78267e4b1e31dcfeb9f303059668888b4423ce5501c7fbc
Deleted: sha256:afe56b46629e6d8e0bfc36fed13395a7cfa2bd83b58bb489976ef13553eff20b
Deleted: sha256:cf0c3bd5d6a0a14ebf96cc7d3df79a37774a70f3086273e09da71a86fe74fec2
Deleted: sha256:c5fdaf8b055f544d0211043e687905315c3a0b71c1c08df07e473dd0a30e43c8
docker images | grep 2d36b1c0ea40
<empty>
This changes the error message to reflect that behavior
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f0988dd39598a46d911abc54c19fc3377698d630
Component: engine
Updates the rmi code to treat canonical references as related to tagged references from the same repository during deletion.
Canonical references with a different repository name will be treated as separate references.
Updates the remove by ID logic to still remove an image if there is a single tag reference and only canonical references to the same repository remaining.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: a281be1c11d61deed7cb2853125e6d489f16aa2e
Component: engine
Moving all strings to the errors package wasn't a good idea after all.
Our custom implementation of Go errors predates everything that's nice
and good about working with errors in Go. Take as an example what we
have to do to get an error message:
```go
func GetErrorMessage(err error) string {
switch err.(type) {
case errcode.Error:
e, _ := err.(errcode.Error)
return e.Message
case errcode.ErrorCode:
ec, _ := err.(errcode.ErrorCode)
return ec.Message()
default:
return err.Error()
}
}
```
This goes against every good practice for Go development. The language already provides a simple, intuitive and standard way to get error messages, that is calling the `Error()` method from an error. Reinventing the error interface is a mistake.
Our custom implementation also makes very hard to reason about errors, another nice thing about Go. I found several (>10) error declarations that we don't use anywhere. This is a clear sign about how little we know about the errors we return. I also found several error usages where the number of arguments was different than the parameters declared in the error, another clear example of how difficult is to reason about errors.
Moreover, our custom implementation didn't really make easier for people to return custom HTTP status code depending on the errors. Again, it's hard to reason about when to set custom codes and how. Take an example what we have to do to extract the message and status code from an error before returning a response from the API:
```go
switch err.(type) {
case errcode.ErrorCode:
daError, _ := err.(errcode.ErrorCode)
statusCode = daError.Descriptor().HTTPStatusCode
errMsg = daError.Message()
case errcode.Error:
// For reference, if you're looking for a particular error
// then you can do something like :
// import ( derr "github.com/docker/docker/errors" )
// if daError.ErrorCode() == derr.ErrorCodeNoSuchContainer { ... }
daError, _ := err.(errcode.Error)
statusCode = daError.ErrorCode().Descriptor().HTTPStatusCode
errMsg = daError.Message
default:
// This part of will be removed once we've
// converted everything over to use the errcode package
// FIXME: this is brittle and should not be necessary.
// If we need to differentiate between different possible error types,
// we should create appropriate error types with clearly defined meaning
errStr := strings.ToLower(err.Error())
for keyword, status := range map[string]int{
"not found": http.StatusNotFound,
"no such": http.StatusNotFound,
"bad parameter": http.StatusBadRequest,
"conflict": http.StatusConflict,
"impossible": http.StatusNotAcceptable,
"wrong login/password": http.StatusUnauthorized,
"hasn't been activated": http.StatusForbidden,
} {
if strings.Contains(errStr, keyword) {
statusCode = status
break
}
}
}
```
You can notice two things in that code:
1. We have to explain how errors work, because our implementation goes against how easy to use Go errors are.
2. At no moment we arrived to remove that `switch` statement that was the original reason to use our custom implementation.
This change removes all our status errors from the errors package and puts them back in their specific contexts.
IT puts the messages back with their contexts. That way, we know right away when errors used and how to generate their messages.
It uses custom interfaces to reason about errors. Errors that need to response with a custom status code MUST implementent this simple interface:
```go
type errorWithStatus interface {
HTTPErrorStatusCode() int
}
```
This interface is very straightforward to implement. It also preserves Go errors real behavior, getting the message is as simple as using the `Error()` method.
I included helper functions to generate errors that use custom status code in `errors/errors.go`.
By doing this, we remove the hard dependency we have eeverywhere to our custom errors package. Yes, you can use it as a helper to generate error, but it's still very easy to generate errors without it.
Please, read this fantastic blog post about errors in Go: http://dave.cheney.net/2014/12/24/inspecting-errors
Signed-off-by: David Calavera <david.calavera@gmail.com>
Upstream-commit: a793564b2591035aec5412fbcbcccf220c773a4c
Component: engine
- Generalize in an interface.
- Stop abusing of List for everything.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Upstream-commit: 3c82fad44112dc73861f325bbecd68b9922b0ad3
Component: engine
Rather than using 2 different functions for different
types of conflicts use a bitmask to specify what
conflicts need to be checked. This allows a better way
to make exceptions.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: 883be4893049aa97cb874d239cf6b06325fc74f2
Component: engine
Do not untag image if it would later get a hard conflict because of running containers.
Fixes#18873
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: 38a45eed8850a15d2f737ce7455f29c5ae53ab49
Component: engine
When pulling an image with content trust enabled, two references are
created: a digest reference and a tag reference. Deleting by tag
wouldn't actually remove the image, because the digest reference keeps
it alive.
This change modifies the rmi logic so that digest references don't keep
an image alive. If the last tag referencing a given image is deleted,
any digest references to it will be removed as well, so the image can
actually get deleted. This fixes the usability problem with deletions
when content trust is in use, so something like "docker pull busybox;
docker rmi busybox" will work as expected.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 2f048f73e122ab90b8f35a088b4be52bd255caad
Component: engine
- Stop serializing JSONMessage in favor of events.Message.
- Keep backwards compatibility with JSONMessage for container events.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Upstream-commit: 72f1881df102fce9ad31e98045b91c204dd44513
Component: engine
So other packages don't need to import the daemon package when they
want to use this struct.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 6bb0d1816acd8d4f7a542a6aac047da2b874f476
Component: engine
Add distribution package for managing pulls and pushes. This is based on
the old code in the graph package, with major changes to work with the
new image/layer model.
Add v1 migration code.
Update registry, api/*, and daemon packages to use the reference
package's types where applicable.
Update daemon package to use image/layer/tag stores instead of the graph
package
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: 4352da7803d182a6013a5238ce20a7c749db979a
Component: engine
When an image has multiple tags and rmi is called with force on a tag, only the single tag should be removed.
The current behavior is broken and removes all tags and the image.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: 48e7f7963e142a0a45b583c5e65015d896c59433
Component: engine
Lookup the graph parent reference to detect a builder cache miss before
looping the whole graph image index to build a parent-children tree.
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
Upstream-commit: f9e81b40f4065e3d0851172759ef58fda6572cce
Component: engine
also fix a typo in pkg/truncindex package comment
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
Upstream-commit: 56f5e3459f8d7477d2aa60dee02bc7cd8a8731ad
Component: engine
- use daemon member directly rather than through a function call
- create GetImage method for use external to deamon
Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
Upstream-commit: ee36b57c8f3c33f1c50ecb4305bd62a962876fe6
Component: engine
Although having a request ID available throughout the codebase is very
valuable, the impact of requiring a Context as an argument to every
function in the codepath of an API request, is too significant and was
not properly understood at the time of the review.
Furthermore, mixing API-layer code with non-API-layer code makes the
latter usable only by API-layer code (one that has a notion of Context).
This reverts commit de4164043546d2b9ee3bf323dbc41f4979c84480, reversing
changes made to 7daeecd42d7bb112bfe01532c8c9a962bb0c7967.
Signed-off-by: Tibor Vass <tibor@docker.com>
Conflicts:
api/server/container.go
builder/internals.go
daemon/container_unix.go
daemon/create.go
Upstream-commit: b08f071e18043abe8ce15f56826d38dd26bedb78
Component: engine
This reverts commit ff92f45be49146cd7ac7716c36d89de989cb262e, reversing
changes made to 80e31df3b6fdf6c1fbd6a5d0aceb0a148066508c.
Reverting to make the next revert easier.
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: 79c31f4b13d331d4011b2975a96dcdeab2036865
Component: engine
Avoid creating a global context object that will be used while the daemon is running.
Not only this object won't ever be garbage collected, but it won't ever be used for anything else than creating other contexts in each request. I think it's a bad practive to have something like this sprawling aroud the code.
This change removes that global object and initializes a context in the cases we don't have already one, like shutting down the server.
This also removes a bunch of context arguments from functions that did nothing with it.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Upstream-commit: 27c76522dea91ec585f0b5f0ae1fec8c255b7b22
Component: engine
This PR adds a "request ID" to each event generated, the 'docker events'
stream now looks like this:
```
2015-09-10T15:02:50.000000000-07:00 [reqid: c01e3534ddca] de7c5d4ca927253cf4e978ee9c4545161e406e9b5a14617efb52c658b249174a: (from ubuntu) create
```
Note the `[reqID: c01e3534ddca]` part, that's new.
Each HTTP request will generate its own unique ID. So, if you do a
`docker build` you'll see a series of events all with the same reqID.
This allow for log processing tools to determine which events are all related
to the same http request.
I didn't propigate the context to all possible funcs in the daemon,
I decided to just do the ones that needed it in order to get the reqID
into the events. I'd like to have people review this direction first, and
if we're ok with it then I'll make sure we're consistent about when
we pass around the context - IOW, make sure that all funcs at the same level
have a context passed in even if they don't call the log funcs - this will
ensure we're consistent w/o passing it around for all calls unnecessarily.
ping @icecrime @calavera @crosbymichael
Signed-off-by: Doug Davis <dug@us.ibm.com>
Upstream-commit: 26b1064967d9fcefd4c35f60e96bf6d7c9a3b5f8
Component: engine
This file was not well documented and had very high cyclomatic complexity.
This patch completely rearranges this file and the ImageDelete method to
be easier to follow and more maintainable in the future.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Upstream-commit: 111d2f348767692a34366ff5cd01f607cf54ad57
Component: engine
- some method names were changed to have a 'Locking' suffix, as the
downcased versions already existed, and the existing functions simply
had locks around the already downcased version.
- deleting unused functions
- package comment
- magic numbers replaced by golang constants
- comments all over
Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
Upstream-commit: abd72d4008dde7ee8249170d49eb4bc963c51e24
Component: engine
Expand the godoc documentation for the graph package.
Centralize DefaultTag in the graphs/tag package instead of defining it
twice.
Remove some unnecessary "config" structs that are only used to pass
a few parameters to a function.
Simplify the GetParentsSize function - there's no reason for it to take
an accumulator argument.
Unexport some functions that aren't needed outside the package.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: d4836cd7ec1c085c5a5caa7eb7f5eda4ace10eb6
Component: engine
Move graph related functions in image to graph package.
Consolidating graph functionality is the first step in refactoring graph into an image store model.
Subsequent refactors will involve breaking up graph into multiple types with a strongly defined interface.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Upstream-commit: 2b58b677a54950bc78c13760ef79fe6284154847
Component: engine
I ran into a situation where I was trying:
`docker rmi busybox`
and it kept failing saying:
`could not find image: Prefix can't be empty`
While I have no idea how I got into this situation, it turns out this is
error message is from `daemon.canDeleteImage()`. In that func we loop over
all containers checking to see if they're using the image we're trying to
delete. In my case though, I had a container with no ImageID. So the code
would die tryig to find that image (hence the "Prefix can't be empty" err).
This would stop all processing despite the fact that the container we're
checking had nothing to do with 'busybox'.
My change logs the bad situation in the logs and then skips that container.
There's no reason to fail all `docker rmi ...` calls just because of one
bad container.
Will continue to try to figure out how I got a container w/o an ImageID
but as of now I have no idea, I didn't do anything but normal docker cli
commands.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Upstream-commit: 71a499022904e1da074ded3e1ed874c1b75ccf23
Component: engine
If an image has been tagged to multiple repos and tags, 'docker
rmi -f IMAGE_ID' will just untag one random repo instead of
untagging all and deleting the image. This patch implement
this. This commit is composed of:
*untag all names and delete the image
*add test to this feature
*modify commandline/cli.md to explain this
Signed-off-by: Deng Guangxing <dengguangxing@huawei.com>
Upstream-commit: 795a58fb44a2bd18ec37d78c82d75c025f786c50
Component: engine
