… so moving it in test files for now.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
(cherry picked from commit 37ca5d6813)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 62aed95bc1
Component: cli
osusergo build tag is needed for better chances to have
a proper static binary when Go >= 1.11 is used, and is
harmless otherwise.
seccomp build tag is needed so the resulting dockerd binary
has seccomp support.
[v2: add libseccomp-dev installation]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 97596405f4300a7ae6eb7664f6c58785955dd8d3
Component: packaging
Should generate correct bits whether or not the suffix is present.
Working example:
❯ ./gen-deb-ver . 18.09.0-beta1
18.09.0~1.1.beta1 18.09.0-beta1
❯ ./gen-deb-ver . 18.09.0-ce-beta1
18.09.0~ce~1.1.beta1 18.09.0-ce-beta1
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 9de3f6fb573806abc2e546447c41552ff90731a4)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: cc336d0804076b31d6d059b164e9b4e3b3cec886
Component: packaging
Includes fixes related to upgrade cycles
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 83a20d53f14adcd2bcc9e932435aa66b6e056afa)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: befa3c61954d12f2413df56a384989d7ae2cb9b3
Component: packaging
A bit overdue
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 11643a30e0bdde4016afe42ab5f1f85e0fc83ecf)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: f38206fa477c59ec39b06b8ddde40be1f65c7cf0
Component: packaging
docker load doesn't support OCI based tar files so we have to make 2
different ones.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit 696cdc009ca73b070d8877ee5cdd13c4b0e4a39b)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: ea811c67ee1e558ecec1b394fc591458b57f7ed1
Component: packaging
The official access point for the Q3 engine images will
be prefixed by store.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 3597d75281
Component: cli
This is set in a top-level makefile, but apparently
isn't getting passed all the way through as expected.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: a5ad5471419cb2a0935148f891c5ada8b13b4779
Component: packaging
Also cleans up the target in rpm and deb Makfiles because
the tar target already output a tar.
Signed-off-by: Jose Bigio <jose.bigio@docker.com>
(cherry picked from commit 8a4744f48aa13bea0b4e9174165be4add0a5a42f)
Signed-off-by: Jose Bigio <jose.bigio@docker.com>
Upstream-commit: 8e4a92dc667e2143b6d5707e19c5564b7da46ba1
Component: packaging
… this removes a whole lot of dependencies from people depending on docker/cli…
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
(cherry picked from commit 2d344b2f61)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: fc3dc8f058
Component: cli
… this is, for now, the only platform that is supported
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
(cherry picked from commit a3a955f204)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2a46a3d46c
Component: cli
This will make it easier for anybody to build with a specific version of a distro they want to build on.
They could lock into a specific sha of a docker image if they wanted, as an example.
Signed-off-by: corbin-coleman <corbin.coleman@docker.com>
Upstream-commit: 7afc35b95ac46bfef383788341ddb0b157af14eb
Component: packaging
This should accomodate the new versioning fix
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: be00095e1d0d5674665a63b6e5932178632feb0e
Component: packaging
Removes the need for the offline installer to install the shim process
and instead installs the shim process as part of the packaging.
May be easier in the future to just package the shim process on it's own
but that'll come after this 18.09 release
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit f8bd366d58f8bdf8a82b9a033353ca5bf4eda948)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Upstream-commit: 31d0cb047b98ab43f661bd026bdd63deef62543d
Component: packaging
This should test that
- all the messages produced are delivered (i.e. not lost)
- followLogs() exits
Loosely based on the test having the same name by Brian Goff, see
https://gist.github.com/cpuguy83/e538793de18c762608358ee0eaddc197
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit f845d76d047760c91dc0c7076aea840291fbdbc5)
Upstream-commit: 2a82480df9ad91593d59be4b5283917dbea2da39
Component: engine
When daemon.ContainerLogs() is called with options.follow=true
(as in "docker logs --follow"), the "loggerutils.followLogs()"
function never returns (even then the logs consumer is gone).
As a result, all the resources associated with it (including
an opened file descriptor for the log file being read, two FDs
for a pipe, and two FDs for inotify watch) are never released.
If this is repeated (such as by running "docker logs --follow"
and pressing Ctrl-C a few times), this results in DoS caused by
either hitting the limit of inotify watches, or the limit of
opened files. The only cure is daemon restart.
Apparently, what happens is:
1. logs producer (a container) is gone, calling (*LogWatcher).Close()
for all its readers (daemon/logger/jsonfilelog/jsonfilelog.go:175).
2. WatchClose() is properly handled by a dedicated goroutine in
followLogs(), cancelling the context.
3. Upon receiving the ctx.Done(), the code in followLogs()
(daemon/logger/loggerutils/logfile.go#L626-L638) keeps to
send messages _synchronously_ (which is OK for now).
4. Logs consumer is gone (Ctrl-C is pressed on a terminal running
"docker logs --follow"). Method (*LogWatcher).Close() is properly
called (see daemon/logs.go:114). Since it was called before and
due to to once.Do(), nothing happens (which is kinda good, as
otherwise it will panic on closing a closed channel).
5. A goroutine (see item 3 above) keeps sending log messages
synchronously to the logWatcher.Msg channel. Since the
channel reader is gone, the channel send operation blocks forever,
and resource cleanup set up in defer statements at the beginning
of followLogs() never happens.
Alas, the fix is somewhat complicated:
1. Distinguish between close from logs producer and logs consumer.
To that effect,
- yet another channel is added to LogWatcher();
- {Watch,}Close() are renamed to {Watch,}ProducerGone();
- {Watch,}ConsumerGone() are added;
*NOTE* that ProducerGone()/WatchProducerGone() pair is ONLY needed
in order to stop ConsumerLogs(follow=true) when a container is stopped;
otherwise we're not interested in it. In other words, we're only
using it in followLogs().
2. Code that was doing (logWatcher*).Close() is modified to either call
ProducerGone() or ConsumerGone(), depending on the context.
3. Code that was waiting for WatchClose() is modified to wait for
either ConsumerGone() or ProducerGone(), or both, depending on the
context.
4. followLogs() are modified accordingly:
- context cancellation is happening on WatchProducerGone(),
and once it's received the FileWatcher is closed and waitRead()
returns errDone on EOF (i.e. log rotation handling logic is disabled);
- due to this, code that was writing synchronously to logWatcher.Msg
can be and is removed as the code above it handles this case;
- function returns once ConsumerGone is received, freeing all the
resources -- this is the bugfix itself.
While at it,
1. Let's also remove the ctx usage to simplify the code a bit.
It was introduced by commit a69a59ffc7e3d ("Decouple removing the
fileWatcher from reading") in order to fix a bug. The bug was actually
a deadlock in fsnotify, and the fix was just a workaround. Since then
the fsnofify bug has been fixed, and a new fsnotify was vendored in.
For more details, please see
https://github.com/moby/moby/pull/27782#issuecomment-416794490
2. Since `(*filePoller).Close()` is fixed to remove all the files
being watched, there is no need to explicitly call
fileWatcher.Remove(name) anymore, so get rid of the extra code.
Should fix https://github.com/moby/moby/issues/37391
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 916eabd459fe707b5c4a86377d12e2ad1871b353)
Upstream-commit: 84a5b528aede5579861201e869870d10fc98c07c
Component: engine
This test case checks that followLogs() exits once the reader is gone.
Currently it does not (i.e. this test is supposed to fail) due to #37391.
[kolyshkin@: test case Brian Goff, changelog and all bugs are by me]
Source: https://gist.github.com/cpuguy83/e538793de18c762608358ee0eaddc197
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit d37a11bfbab83ab42b1160f116e863daac046192)
Upstream-commit: 511741735e0aa2fe68a66d99384c00d187d1a157
Component: engine
This code has many return statements, for some of them the
"end logs" or "end stream" message was not printed, giving
the impression that this "for" loop never ended.
Make sure that "begin logs" is to be followed by "end logs".
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 2e4c2a6bf9cb47fd07e42f9c043024ed3dbcd04d)
Upstream-commit: 2b8bc86679b7153bb4ace063a858637df0f16a2e
Component: engine
The code in Close() that removes the watches was not working,
because it first sets `w.closed = true` and then calls w.close(),
which starts with
```
if w.closed {
return errPollerClosed
}
```
Fix by setting w.closed only after calling w.remove() for all the
files being watched.
While at it, remove the duplicated `delete(w.watches, name)` code.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit fffa8958d00860b4e3563327a2cc6836a12d4ba9)
Upstream-commit: 4e2dbfa1af48191126b0910b9463bf94d8371886
Component: engine
There is no need to wait for up to 200ms in order to close
the file descriptor once the chClose is received.
This commit might reduce the chances for occasional "The process
cannot access the file because it is being used by another process"
error on Windows, where an opened file can't be removed.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit dfbb64ea7d042d5b2bb0c1c2b88e3682b7069b10)
Upstream-commit: 3a3bfcbf47e98212abfc9cfed860d9e99fc41cdc
Component: engine
In case of errors, the file descriptor is never closed. Fix it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 88bcf1573ca2eaffc15da346a1651a3749567554)
Upstream-commit: 7be43586af6824c1e55cb502d9d2bab45c9b4505
Component: engine
To include https://github.com/vbatts/tar-split/pull/48 which
fixes the issue of creating an image with >8GB file in it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 92e75439037d205c218208945c70cfb3633e87aa)
Upstream-commit: d7085abec2e445630bedd3e79782c5ec33f62682
Component: engine
