remove unnescessary import aliases, brackets, and so on.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f23c00d8701e4bd0f2372a586dacbf66a26f9a51
Component: engine
This is synonymous with `docker run --cidfile=FILE` and writes the digest of
the newly built image to the named file. This is intended to be used by build
systems which want to avoid tagging (perhaps because they are in CI or
otherwise want to avoid fixed names which can clash) by enabling e.g. Makefile
constructs like:
image.id: Dockerfile
docker build --iidfile=image.id .
do-some-more-stuff: image.id
do-stuff-with <image.id
Currently the only way to achieve this is to use `docker build -q` and capture
the stdout, but at the expense of losing the build output.
In non-silent mode (without `-q`) with API >= v1.29 the caller will now see a
`JSONMessage` with the `Aux` field containing a `types.BuildResult` in the
output stream for each image/layer produced during the build, with the final
one being the end product. Having all of the intermediate images might be
interesting in some cases.
In silent mode (with `-q`) there is no change, on success the only output will
be the resulting image digest as it was previosuly.
There was no wrapper to just output an Aux section without enclosing it in a
Progress, so add one here.
Added some tests to integration cli tests.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Upstream-commit: 5894bc1abf8186802d360d20739b57bfffed51df
Component: engine
Change "service create" and "service update" to wait until the creation
or update finishes, when --detach=false is specified. Show progress bars
for the overall operation and for each individual task (when there are a
small enough number of tasks), unless "-q" / "--quiet" is specified.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 330a0035334871d92207b583c1c36d52a244753f
Component: engine
This seems clearer to me and avoids a miniscule amount of string formatting.
I have deliberately not changed the tests here to avoid the possibility of
changing both wrongly.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Upstream-commit: 3c233b13a01b9f44e31a158768a06b97a775f095
Component: engine
Although our use of ANSI codes here is rather simple it is generally good
practice to use terminfo in order to be portable to different terminal
emulators.
Vendor github.com/Nvveen/Gotty (actually my fork with a fix, see
https://github.com/Nvveen/Gotty/pull/1) and use that to parse the terminfo
files.
Note that "\e]2K" (clear entire line) is not covered by terminfo. We can
achieve the same end by first clearing from begining of line to cursor
(el1="\e]1K") and then clearing from cursor to end of line (el="\e]k").
Test suite has been updated and forces (either directly or by setting $TERM to
something highly unlikely to exist) the use of the non-terminfo fallbacks which
retains the same output behaviour as previously. This is preferable even to
relying on a well-known and relatively static terminfo (like vt102) since even
that in principal might have different terminfo encodings.
In case terminfo is not available at all for $TERM or doesn't expose the
specific capabilities which we use then fall back to the previous manual
escapes, with the exception that we avoid "\e]2K" as discussed above.
Tested with a manual docker pull with rxvt-unicode ($TERM=rxvt-unicode), xterm
($TERM=xterm), mlterm ($TERM=mlterm) and aterm ($TERM=kterm).
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Upstream-commit: f02221a7941948017df68db8fd9a5de7f19453bf
Component: engine
When logging the failure of a case which involves control characters add line
feeds and spacing such that the expected and actual strings are aligned in the
same columns making comparisons easier.
Turns:
jsonmessage_test.go:156: Expected "\x1b[2K\rstatus 1 B\r", got "\x1b[1K\x1b[K\rstatus 1 B\r"
Into:
jsonmessage_test.go:156:
Expected "\x1b[2K\rstatus 1 B\r"
got "\x1b[1K\x1b[K\rstatus 1 B\r"
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Upstream-commit: 2e3baa98eb0f40898005db1b40303848bf5c7de2
Component: engine
The majority already did so, but a few used %v.
Compared with %v using %q escapes any control characters ensuring they are
visible in the logs (otherwise they can cause scrolling and overwriting etc).
The %q formatter also includes the surrounding "s making the manual use of []
to wrap/delimit the %v output unnecessary.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Upstream-commit: 6582cfb1d4fec92b17e10291d30c2e6312ceb5f5
Component: engine
The ANSI escape codes \e[0A (cursor up 0 lines) and \e[0B (cursor down 0 lines)
are not well defined and are treated differently by different terminals. In
particular xterm treats 0 as a missing parameter and therefore defaults to 1,
whereas rxvt-unicode treats these escapes as a request to move 0 lines.
However the use of these codes is unnecessary and were really just hiding the
fact that we were not correctly computing diff when adding a new line. Having
added the new line to the ids map and output the corresponding \n we need to
then calculate a correct diff of 1 rather than leaving it as the default 0
(which xterm then interprets as 1). The fix is to pull the diff calculation out
of the else case and to always do it.
With this in place we can then avoid outputting escapes for moving 0 lines.
Actually diff should never be 0 to start with any more, but check to be safe.
This fixes corruption of `docker pull` seen with rxvt-unicode (and likely other
terminals in that family) seen in #28111. Tested with rxvt-unicode
($TERM=rxvt-unicode), xterm ($TERM=xterm), mlterm ($TERM=mlterm) and aterm
($TERM=kterm).
The test cases have been updated to match the new behaviour.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Upstream-commit: b08b437acc3bf52bd2c3435e632ed09f3312e489
Component: engine
The trust code used to parse the console output of `docker push` to
extract the digest, tag, and size information and determine what to
sign. This is fragile and might give an attacker control over what gets
signed if the attacker can find a way to influence what gets printed as
part of the push output.
This commit sends the push metadata out-of-band. It introduces an `Aux`
field in JSONMessage that can carry application-specific data alongside
progress updates. Instead of parsing formatted output, the client looks
in this field to get the digest, size, and tag from the push.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 65370be888d940899593a001024f53d6b83b4bb0
Component: engine
- Move time json marshaling to the jsonlog package: this is a docker
internal hack that we should not promote as a library.
- Move Timestamp encoding/decoding functions to the API types: This is
only used there. It could be a standalone library but I don't this
it's worth having a separated repo for this. It could introduce more
complexity than it solves.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Upstream-commit: 27220ecc6b1eedf650ca9cf94965cb0dc2054efd
Component: engine
jm.ID is already checked in the outer "if",
so theres no reason to check it again here.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 7a6f5d9b3186de2116686d5e2c40eff673dae6ec
Component: engine
In the existing code, "diff" has function scope and the value from the
previous iteration may be used if it is not reset. This appears to be an
oversight. This commit changes its scope to the for loop body.
One confusing point is that the cursor movement escape sequences appear
to be necessary even if the requested movement is 0. I haven't been able
to figure out why this makes a difference.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 59df2adc071e0186ccd0ba7ea9387142e168d735
Component: engine
When we handle a message that isn't tracked in the "line" map (for
example, one with no ID), clear the line map. This means we won't update
lines that were part of a previous, completed set of operations when
doing something like pull -a. It also has the beneficial side effect
of avoiding terminal glitching in these types of situations, since
messages that don't get tracked in the "line" map cause the count of the
number of lines to get out of sync.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: fc41d393946e6da59792da9ed6e5ab8ed6f58814
Component: engine
Although having a request ID available throughout the codebase is very
valuable, the impact of requiring a Context as an argument to every
function in the codepath of an API request, is too significant and was
not properly understood at the time of the review.
Furthermore, mixing API-layer code with non-API-layer code makes the
latter usable only by API-layer code (one that has a notion of Context).
This reverts commit de4164043546d2b9ee3bf323dbc41f4979c84480, reversing
changes made to 7daeecd42d7bb112bfe01532c8c9a962bb0c7967.
Signed-off-by: Tibor Vass <tibor@docker.com>
Conflicts:
api/server/container.go
builder/internals.go
daemon/container_unix.go
daemon/create.go
Upstream-commit: b08f071e18043abe8ce15f56826d38dd26bedb78
Component: engine
This PR adds a "request ID" to each event generated, the 'docker events'
stream now looks like this:
```
2015-09-10T15:02:50.000000000-07:00 [reqid: c01e3534ddca] de7c5d4ca927253cf4e978ee9c4545161e406e9b5a14617efb52c658b249174a: (from ubuntu) create
```
Note the `[reqID: c01e3534ddca]` part, that's new.
Each HTTP request will generate its own unique ID. So, if you do a
`docker build` you'll see a series of events all with the same reqID.
This allow for log processing tools to determine which events are all related
to the same http request.
I didn't propigate the context to all possible funcs in the daemon,
I decided to just do the ones that needed it in order to get the reqID
into the events. I'd like to have people review this direction first, and
if we're ok with it then I'll make sure we're consistent about when
we pass around the context - IOW, make sure that all funcs at the same level
have a context passed in even if they don't call the log funcs - this will
ensure we're consistent w/o passing it around for all calls unnecessarily.
ping @icecrime @calavera @crosbymichael
Signed-off-by: Doug Davis <dug@us.ibm.com>
Upstream-commit: 26b1064967d9fcefd4c35f60e96bf6d7c9a3b5f8
Component: engine
This way provide both Time and TimeNano in the event. For the display of
the JSONMessage, use either, but prefer TimeNano Proving only TimeNano
would break Subscribers that are using the `Time` field, so both are set
for backwards compatibility.
The events logging uses nano formatting, but only provides a Unix()
time, therefor ordering may get lost in the output. Example:
```
2015-09-15T14:18:51.000000000-04:00 ee46febd64ac629f7de9cd8bf58582e6f263d97ff46896adc5b508db804682da: (from busybox) resize
2015-09-15T14:18:51.000000000-04:00 a78c9149b1c0474502a117efaa814541926c2ae6ec3c76607e1c931b84c3a44b: (from busybox) resize
```
By having a field just for Nano time, when set, the marshalling back to
`time.Unix(sec int64, nsec int64)` has zeros exactly where it needs to.
This does not break any existing use of jsonmessage.JSONMessage, but now
allows for use of `UnixNano()` and get event formatting that has
distinguishable order. Example:
```
2015-09-15T15:37:23.810295632-04:00 6adcf8ed9f5f5ec059a915466cd1cde86a18b4a085fc3af405e9cc9fecbbbbaf: (from busybox) resize
2015-09-15T15:37:23.810412202-04:00 6b7c5bfdc3f902096f5a91e628f21bd4b56e32590c5b4b97044aafc005ddcb0d: (from busybox) resize
```
Including tests for TimeNano and updated event API reference doc.
Signed-off-by: Vincent Batts <vbatts@redhat.com>
Upstream-commit: 4026512a2c535fe377a4cf834b7a78fccfd82dab
Component: engine
The practice of buffering to a tempfile during a pushing contributes massively
to slow V2 push performance perception. The protocol was actually designed to
avoid precalculation, supporting cut-through data push. This means we can
assemble the layer, calculate its digest and push to the remote endpoint, all
at the same time.
This should increase performance massively on systems with slow disks or IO
bottlenecks.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Upstream-commit: 349aef06d51a9425f70a877585fe45283366c2c4
Component: engine
Some structures use int for sizes and UNIX timestamps. On some
platforms, int is 32 bits, so this can lead to the year 2038 issues and
overflows when dealing with large containers or layers.
Consistently use int64 to store sizes and UNIX timestamps in
api/types/types.go. Update related to code accordingly (i.e.
strconv.FormatInt instead of strconv.Itoa).
Use int64 in progressreader package to avoid integer overflow when
dealing with large quantities. Update related code accordingly.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 1f61084d83aea37b212468aaa975020094b7f7c9
Component: engine
