Commit Graph

5123 Commits

Author SHA1 Message Date
b538302e2c Merge pull request #90 from andrewhsu/fix-leak
[17.06] Prevent a goroutine leak when healthcheck gets stopped
2017-07-13 13:58:07 +02:00
136b90fbd0 Merge pull request #102 from tiborvass/runtime-spec-fix
[17.06][engine][vendor] forks of containerd + runc + runtime-spec
2017-07-12 15:29:33 -07:00
2d8beb7256 Merge pull request #95 from andrewhsu/fix-logging
[17.06] Fix stderr logging for journald and syslog
2017-07-12 14:40:48 -07:00
2206628703 [engine][vendor] forks of containerd + runc + runtime-spec
This vendors from the docker org:
- containerd to 6e23458c129b551d5c9871e5174f6b1b7f6d1170
- runc to 810190ceaa507aa2727d7ae6f4790c76ec150bd2
- runtime-spec to a45ba0989fc26c695fe166a49c45bb8b7618ab36

This fixes two issues:
- if the container is paused, it now responds properly to SIGKILL
- on buggy kernels such as RHEL7.2, a int64->uint64 conversion bug
  prevented containers to start when memory cgroup was specified.

Signed-off-by: Tibor Vass <tibor@docker.com>
2017-07-12 03:13:56 +00:00
bac91bd031 Fix stderr logging for journald and syslog
logger.PutMessage, added in #28762 (v17.04.0-ce), clears msg.Source. So journald
and syslog were treating stderr messages as if they were stdout.

Signed-off-by: David Glasser <glasser@davidglasser.net>
(cherry picked from commit 917050c5728f2fb9958ccb3ab66a23766f741adc)
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2017-07-11 22:53:07 +00:00
4295f109dc Prevent a goroutine leak when healthcheck gets stopped
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
(cherry picked from commit 67297ba0051d39be544009ba76abea14bc0be8a4)
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2017-07-11 22:49:56 +00:00
728c4a5bfe use latestChunk instead of latestFile to get cherry-pick commit 7bd7bde working
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2017-07-07 18:31:52 +00:00
7bd7bdeada Fix log readers can block writes indefinitely
Before this patch, a log reader is able to block all log writes
indefinitely (and other operations) by simply opening the log stream and
not consuming all the messages.

The reason for this is we protect the read stream from corruption by
ensuring there are no new writes while the log stream is consumed (and
caught up with the live entries).

We can get around this issue because log files are append only, so we
can limit reads to only the section of the file that was written to when
the log stream was first requested.

Now logs are only blocked until all files are opened, rather than
streamed to the client.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit e2209185ed1c959131d4068ec7fc93e194dc0802)

Conflicts:
components/engine/daemon/logger/jsonfilelog/read.go
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2017-07-06 01:29:57 +00:00
bb23c93d28 Redact the swarm's spec's signing CA cert when getting swarm info, because
otherwise if the user gets the info from the API, makes a non-CA related change,
then updates, swarm will interpret this as the user trying to remove the signing
key from the swarm.  We are redacting due to usability reasons, not because
the signing cert is secret.  The signing KEY is secret, hence it's redacted.

Signed-off-by: Ying Li <ying.li@docker.com>
(cherry picked from commit bdfbd22afbbf16a07f0316656c6c17453df3e0f7)
Signed-off-by: Tibor Vass <tibor@docker.com>
2017-06-20 01:52:18 +00:00
97a704b2a7 Merge pull request #52 from seemethere/cherry_pick_33335
Check signal is unset before using user stopsignal
2017-06-12 19:24:57 -07:00
ca7df6fe42 Merge pull request #61 from seemethere/cherry_pick_33583
Logging driver should receive same file in start/stop request
2017-06-10 22:09:47 -07:00
ab5798a6f7 daemon: Remove daemon datastructure dump functionality
When sending SIGUSR1 to the daemon, it can crash because of a concurrent
map access panic, showing a stack trace involving dumpDaemon. It appears
it's not possible to recover from a concurrent map access panic. Since
it's important that SIGUSR1 not be a destructive operation, sadly the
best course of action I can think of is to remove this functionality.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
(cherry picked from commit a4c68ee8574c9b8a3309ebebee0d90108042ba61)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
2017-06-09 17:34:45 -07:00
899ed1b641 Logging driver should receive same file in start/stop request
Signed-off-by: Peter Bücker <peter.buecker@gmail.com>
(cherry picked from commit e908e1a357b435d7fab497d51cdd3e58458a0590)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
2017-06-09 17:33:46 -07:00
0cbae9b2ee Merge pull request #48 from seemethere/cherry_pick_33330
Don't create source directory while the daemon is being shutdown, fix…
2017-06-09 15:33:19 -07:00
dcd1f685c8 Merge pull request #47 from seemethere/cherry_pick_33249
Add container environment variables correctly to the health check
2017-06-09 15:32:27 -07:00
3910d5b571 Check signal is unset before using user stopsignal
This fixes an issue where if a stop signal is set, and a user sends
SIGKILL, `container.ExitOnNext()` is not set, thus causing the container
to restart.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 114652ab86609e5c0cbfad84f642942b466a0596)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
2017-06-09 13:21:29 -07:00
d09575fb8f Don't create source directory while the daemon is being shutdown, fix #30348
If a container mount the socket the daemon is listening on into
container while the daemon is being shutdown, the socket will
not exist on the host, then daemon will assume it's a directory
and create it on the host, this will cause the daemon can't start
next time.

fix issue https://github.com/moby/moby/issues/30348

To reproduce this issue, you can add following code

```
--- a/daemon/oci_linux.go
+++ b/daemon/oci_linux.go
@@ -8,6 +8,7 @@ import (
        "sort"
        "strconv"
        "strings"
+       "time"

        "github.com/Sirupsen/logrus"
        "github.com/docker/docker/container"
@@ -666,7 +667,8 @@ func (daemon *Daemon) createSpec(c *container.Container) (*libcontainerd.Spec, e
        if err := daemon.setupIpcDirs(c); err != nil {
                return nil, err
        }
-
+       fmt.Printf("===please stop the daemon===\n")
+       time.Sleep(time.Second * 2)
        ms, err := daemon.setupMounts(c)
        if err != nil {
                return nil, err

```

step1 run a container which has `--restart always` and `-v /var/run/docker.sock:/sock`
```
$ docker run -ti --restart always -v /var/run/docker.sock:/sock busybox
/ #

```
step2 exit the the container
```
/ # exit
```
and kill the daemon when you see
```
===please stop the daemon===
```
in the daemon log

The daemon can't restart again and fail with `can't create unix socket /var/run/docker.sock: is a directory`.

Signed-off-by: Lei Jitang <leijitang@huawei.com>

(cherry picked from commit 7318eba5b2f8bb4b867ca943c3229260ca98a3bc)

Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>

Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
2017-06-09 13:13:40 -07:00
13934b618c Add container environment variables correctly to the health check
The health check process doesn't have all the environment
varialbes in the container or has them set incorrectly.

This patch should fix that problem.

Signed-off-by: Boaz Shuster <ripcurld.github@gmail.com>
(cherry picked from commit 5836d86ac4d617e837d94010aa60384648ab59ea)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
2017-06-09 13:09:40 -07:00
94f4d72c55 Service alias should not be copied to task alias
If a service alias is copied to task, then the DNS resolution on the
service name will resolve to service VIP and all of Task-IPs and that
will break the concept of vip based load-balancing resulting in all the
dns-rr caching issues.

This is a regression introduced in #33130

Signed-off-by: Madhu Venugopal <madhu@docker.com>
(cherry picked from commit 38c15531501578b96d34be5ce7f33a0be6be078f)
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
2017-06-09 12:55:46 -07:00
b2dacf38db Merge pull request #8 from mlaventure/chp-moby-33341
Do not clear swarm directory at the begining of swarm init and swarm …
2017-06-05 09:57:27 -07:00
2bdf7010a1 Merge pull request #7 from mlaventure/chp-moby-33308
remove RuntimeData from cluster and types
2017-06-05 06:10:46 -07:00
46f679cc1d Merge pull request #5 from mlaventure/chp-moby-33311
Adding support for DNS search on RS1
2017-06-05 06:09:52 -07:00
30e966bc97 Do not clear swarm directory at the begining of swarm init and swarm join now.
However, do clear the directory if init or join fails, because we don't
want to leave it in a half-finished state.

Signed-off-by: Ying Li <ying.li@docker.com>
(cherry picked from commit bf3e9293a66c77a2fddf4e691222898846b4af9f)
2017-05-31 11:25:56 -07:00
bb1fe809d2 remove RuntimeData from cluster and types
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
(cherry picked from commit 8eeba751983d6499a07fcd21c26f381eb1d17d19)
2017-05-31 11:08:24 -07:00
e724dcd16b Adding support for DNS search on RS1
Signed-off-by: Sandeep Bansal <sabansal@microsoft.com>
(cherry picked from commit b8e8dcd6e031d005a2d46d95f84da35b436cd870)
2017-05-31 10:55:38 -07:00
43484d9284 prevent image prune panic
Signed-off-by: Alfred Landrum <alfred.landrum@docker.com>
(cherry picked from commit 32da2a4234c5c68ff466dc1afc91ba98dbbe199a)
2017-05-31 09:58:11 -07:00
b0a3ababea Merge pull request #33286 from aboch/ingr
Properly identify ingress network created with older swarm
Upstream-commit: 39a604c0d419424bab0314e7070c49bcacca3bda
Component: engine
2017-05-19 10:04:29 -04:00
1e0f9dbab8 arg filt
Signed-off-by: yupengzte <yu.peng36@zte.com.cn>
Upstream-commit: 49bf533fe72fd2390da89d133fbe984d0284ebe2
Component: engine
2017-05-19 13:47:32 +08:00
ba221c0328 Properly identify ingress network created with older swarm
- otherwise docker network prune will remove it

Signed-off-by: Alessandro Boch <aboch@docker.com>
Upstream-commit: 93763f11eeec5e9b1d0308a3ad85bbf069f9107f
Component: engine
2017-05-18 19:47:45 -07:00
01eae47819 Merge pull request #33281 from dmcgowan/overlay2-on-btrfs
Support overlay2 on btrfs
Upstream-commit: 01af41ed99f96693bd04bec1cf0f7a6b97bc2408
Component: engine
2017-05-19 03:09:39 +02:00
10c5371927 Support overlay2 on btrfs
OverlayFS is supported on top of btrfs as of Linux Kernel 4.7.
Skip the hard enforcement when on kernel 4.7 or newer and
respect the kernel check override flag on older kernels.
https://btrfs.wiki.kernel.org/index.php/Changelog#By_feature

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: f64a4ad008e68996afcec3ab34a869887716f944
Component: engine
2017-05-18 15:26:03 -07:00
6b7f9e397d Merge pull request #33250 from cyphar/apparmor-readonly-root-fix
apparmor: make pkg/aaparser work on read-only root
Upstream-commit: 6b262572028cc249b7da8cc55459197395fdb33e
Component: engine
2017-05-18 23:17:31 +02:00
b7e3ae2808 Merge pull request #33264 from wenjianhn/chkp-ls
Fix checkpoint ls
Upstream-commit: cc24049f77eb94ad486bd22c2470f0d13d6158a1
Component: engine
2017-05-18 13:37:42 -07:00
17712f857a Improved poor memory efficiency of awslogs
Signed-off-by: YAMASAKI Masahide <masahide.y@gmail.com>
Upstream-commit: 524f30634018ce619da61aa0a13dad245e098226
Component: engine
2017-05-18 23:30:00 +09:00
a65ff11d52 Fix checkpoint ls
Use create=false when calling getCheckpointDir
Fix #33263

Signed-off-by: Jacob Wen <jian.w.wen@oracle.com>
Upstream-commit: c0d2ec1eb585a31e3d7734aef7acb753644c49c0
Component: engine
2017-05-18 10:57:10 +08:00
cee45f7710 Adding network specific options to service create/update
The commit adds capability to accept csv parameters
for network option in service create/update commands.The change
includes name,alias driver options specific to the network.
With this the following will be supported

docker service create --name web --network name=docknet,alias=web1,driver-opt=field1=value1 nginx
docker service create --name web --network docknet nginx
docker service update web --network-add name=docknet,alias=web1,driver-opt=field1=value1
docker service update web --network-rm docknet

Signed-off-by: Abhinandan Prativadi <abhi@docker.com>
Upstream-commit: fe11de01772ec7c7501d3b679ce94d41bcc14060
Component: engine
2017-05-17 18:46:34 -07:00
4f94cc50a4 Integrate local datascope network with swarm
Signed-off-by: Alessandro Boch <aboch@docker.com>
Upstream-commit: b34d3e730fe3eee4b058e4dc165a7d4964e00a2a
Component: engine
2017-05-17 15:50:28 -07:00
eefcf59562 Allow to control network scope
- User can now promote a network's scope to swarm,
  so that the network is visible to swarm

Signed-off-by: Alessandro Boch <aboch@docker.com>
Upstream-commit: fcafc7108b7b7f2ca9c2982b29b47de424f2da52
Component: engine
2017-05-17 15:50:28 -07:00
9f01da606a Support configuration networks
- They are configuration-only networks which
  can be used to supply the configuration
  when creating regular networks.
- They do not get allocated and do net get plumbed.
  Drivers do not get to know about them.
- They can be removed, once no other network is
  using them.
- When user creates a network specifying a
  configuration network for the config, no
  other network specific configuration field
  is are accepted. User can only specify
  network operator fields (attachable, internal,...)

Signed-off-by: Alessandro Boch <aboch@docker.com>
Upstream-commit: 9ee7b4dda926a1444dc0ea50c4ca6d90c8684060
Component: engine
2017-05-17 15:50:28 -07:00
5f22605796 support cluster events
Signed-off-by: Dong Chen <dongluo.chen@docker.com>
Upstream-commit: 59d45c384a2de7bca73296ce1471646db14cb0c8
Component: engine
2017-05-17 11:46:30 -07:00
e7e4e93352 apparmor: make pkg/aaparser work on read-only root
This is necessary because normally `apparmor_parser -r` will try to
create a temporary directory on the host (which is not allowed if the
host has a rootfs). However, the -K option bypasses saving things to the
cache (which avoids this issue).

  % apparmor_parser -r /tmp/docker-profile
  mkstemp: Read-only file system
  % apparmor_parser -Kr /tmp/docker-profile
  %

In addition, add extra information to the ensureDefaultAppArmorProfile
errors so that problems like this are easier to debug.

Fixes: 2f7596aaef3a ("apparmor: do not save profile to /etc/apparmor.d")
Signed-off-by: Aleksa Sarai <asarai@suse.de>
Upstream-commit: dd340c52cb37d29d8bae687fdc60e039341b4295
Component: engine
2017-05-18 00:05:13 +10:00
f81cca9824 Merge pull request #33075 from x1022as/stop
fix inconsistent state string with containerd
Upstream-commit: 4dd3e5b77c2f5ab8203cb8fc239e8441a504d799
Component: engine
2017-05-17 09:27:47 -04:00
ad53a9e7a1 Merge pull request #30891 from mixja/awslogs-multiline-support
Add awslogs multiline support
Upstream-commit: 50342883819194f6d86594d076efc948f92c757d
Component: engine
2017-05-17 09:27:21 -04:00
6f26254653 Merge pull request #32237 from jlhawn/update_container_wait
Update Container Wait
Upstream-commit: c053a2069e1c8e99488517071540fb3e526233f3
Component: engine
2017-05-17 02:39:52 +02:00
0e311d1c4e Merge pull request #33151 from nwt/push-foreign-layers
Add daemon option to push foreign layers
Upstream-commit: a30ef99e8dd2c3e7a54b6410a5709f61db59c07f
Component: engine
2017-05-17 02:04:31 +02:00
4fcc705035 Merge pull request #33169 from johnstep/windows-configs
Add Windows configs support
Upstream-commit: 7658851e74b63fa7a1c3dfa5bf950712cfe72d4e
Component: engine
2017-05-16 16:46:34 -07:00
caabf0e2ac Merge pull request #32388 from nishanttotla/pin-by-digest-on-client-alternative
Moving docker service digest pinning to client side
Upstream-commit: d6f4fe9e38b60f63e429fff7ffced9c26cbf8236
Component: engine
2017-05-16 15:24:07 -07:00
fa44d02011 Update ContainerWait API
This patch adds the untilRemoved option to the ContainerWait API which
allows the client to wait until the container is not only exited but
also removed.

This patch also adds some more CLI integration tests for waiting for a
created container and waiting with the new --until-removed flag.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

Handle detach sequence in CLI

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

Update Container Wait Conditions

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

Apply container wait changes to API 1.30

The set of changes to the containerWait API missed the cut for the
Docker 17.05 release (API version 1.29). This patch bumps the version
checks to use 1.30 instead.

This patch also makes a minor update to a testfile which was added to
the builder/dockerfile package.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

Remove wait changes from CLI

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

Address minor nits on wait changes

- Changed the name of the tty Proxy wrapper to `escapeProxy`
- Removed the unnecessary Error() method on container.State
- Fixes a typo in comment (repeated word)

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

Use router.WithCancel in the containerWait handler

This handler previously added this functionality manually but now uses
the existing wrapper which does it for us.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

Add WaitCondition constants to api/types/container

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

Address more ContainerWait review comments

- Update ContainerWait backend interface to not return pointer values
  for container.StateStatus type.
- Updated container state's Wait() method comments to clarify that a
  context MUST be used for cancelling the request, setting timeouts,
  and to avoid goroutine leaks.
- Removed unnecessary buffering when making channels in the client's
  ContainerWait methods.
- Renamed result and error channels in client's ContainerWait methods
  to clarify that only a single result or error value would be sent
  on the channel.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

Move container.WaitCondition type to separate file

... to avoid conflict with swagger-generated code for API response

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)

Address more ContainerWait review comments

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Upstream-commit: 4921171587c09d0fcd8086a62a25813332f44112
Component: engine
2017-05-16 15:11:39 -07:00
3f22cbeb2a Update Container Wait Backend
This patch consolidates the two WaitStop and WaitWithContext methods
on the container.State type. Now there is a single method, Wait, which
takes a context and a bool specifying whether to wait for not just a
container exit but also removal.

The behavior has been changed slightly so that a wait call during a
Created state will not return immediately but instead wait for the
container to be started and then exited.

The interface has been changed to no longer block, but instead returns
a channel on which the caller can receive a *StateStatus value which
indicates the ExitCode or an error if there was one (like a context
timeout or state transition error).

These changes have been propagated through the rest of the deamon to
preserve all other existing behavior.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Upstream-commit: cfdf84d5d04c8ee656e5c4ad3db993c258e52674
Component: engine
2017-05-16 15:09:14 -07:00
67b57727bd Add daemon option to push foreign layers
The --allow-nondistributable-artifacts daemon option specifies
registries to which foreign layers should be pushed.  (By default,
foreign layers are not pushed to registries.)

Additionally, to make this option effective, foreign layers are now
pulled from the registry if possible, falling back to the URLs in the
image manifest otherwise.

This option is useful when pushing images containing foreign layers to a
registry on an air-gapped network so hosts on that network can pull the
images without connecting to another server.

Signed-off-by: Noah Treuhaft <noah.treuhaft@docker.com>
Upstream-commit: 67fdf574d5acd6ddccb6ece0ffe0ace1c1608712
Component: engine
2017-05-16 14:36:36 -07:00