Commit Graph

5965 Commits

Author SHA1 Message Date
68b3c3dd7c Fix regression when filtering container names using a leading slash
Commit 5c8da2e96738addd8a175e5b9a23b8c37d4a4dfe updated the filtering behavior
to match container-names without having to specify the leading slash.

This change caused a regression in situations where a regex was provided as
filter, using an explicit leading slash (`--filter name=^/mycontainername`).

This fix changes the filters to match containers both with, and without the
leading slash, effectively making the leading slash optional when filtering.

With this fix, filters with and without a leading slash produce the same result:

    $ docker ps --filter name=^a
    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
    21afd6362b0c        busybox             "sh"                2 minutes ago       Up 2 minutes                            a2
    56e53770e316        busybox             "sh"                2 minutes ago       Up 2 minutes                            a1

    $ docker ps --filter name=^/a
    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
    21afd6362b0c        busybox             "sh"                2 minutes ago       Up 2 minutes                            a2
    56e53770e316        busybox             "sh"                3 minutes ago       Up 3 minutes                            a1

    $ docker ps --filter name=^b
    CONTAINER ID        IMAGE               COMMAND             CREATED              STATUS              PORTS               NAMES
    b69003b6a6fe        busybox             "sh"                About a minute ago   Up About a minute                       b1

    $ docker ps --filter name=^/b
    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
    b69003b6a6fe        busybox             "sh"                56 seconds ago      Up 54 seconds                           b1

    $ docker ps --filter name=/a
    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
    21afd6362b0c        busybox             "sh"                3 minutes ago       Up 3 minutes                            a2
    56e53770e316        busybox             "sh"                4 minutes ago       Up 4 minutes                            a1

    $ docker ps --filter name=a
    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
    21afd6362b0c        busybox             "sh"                3 minutes ago       Up 3 minutes                            a2
    56e53770e316        busybox             "sh"                4 minutes ago       Up 4 minutes                            a1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6f9b5ba810e9314ab9335490cd41316940a32b5e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5fa80da2d385520f2c3c91e8ce23d181e4f93097
Component: engine
2018-08-29 12:54:06 +02:00
b9ccf0e1c6 Merge pull request #37688 from tiborvass/features-not-flat
Fix logic when enabling buildkit
Upstream-commit: ef50da44b30dd041896cfc6be45b777e10cc1d75
Component: engine
2018-08-21 18:56:41 -07:00
65b88273a4 Merge pull request #37684 from thaJeztah/add_remote_api_warning
Add warning if REST API is accessible through an insecure connection
Upstream-commit: 1cc3deebc40733fb6dcd98fbb5412399dc764876
Component: engine
2018-08-21 16:52:37 -07:00
ed97b30d09 Fix logic when enabling buildkit
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: c973cde7606dc7a2557094fc90d8e6bb595fa354
Component: engine
2018-08-21 23:49:08 +00:00
c0b7cd4ffb Merge pull request #37636 from thaJeztah/add_swarm_label_tests
Add unit test for swarm labels on containers
Upstream-commit: 3bd1e7b59b0bf4ee4e581e1d27c49aee7987f1f1
Component: engine
2018-08-21 23:02:46 +01:00
7469deec1a Add warning if REST API is accessible through an insecure connection
The remote API allows full privilege escalation and is equivalent to
having root access on the host. Because of this, the API should never
be accessible through an insecure connection (TCP without TLS, or TCP
without TLS  verification).

Although a warning is already logged on startup if the daemon uses an
insecure configuration, this warning is not very visible (unless someone
decides to read the logs).

This patch attempts to make insecure configuration more visible by sending
back warnings through the API (which will be printed when using `docker info`).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 547b993e07330f3e74cba935975fce05e8661381
Component: engine
2018-08-21 22:03:24 +02:00
07c22f44e1 Add "Warnings" to /info endpoint, and move detection to the daemon
When requesting information about the daemon's configuration through the `/info`
endpoint, missing features (or non-recommended settings) may have to be presented
to the user.

Detecting these situations, and printing warnings currently is handled by the
cli, which results in some complications:

- duplicated effort: each client has to re-implement detection and warnings.
- it's not possible to generate warnings for reasons outside of the information
  returned in the `/info` response.
- cli-side detection has to be updated for new conditions. This means that an
  older cli connecting to a new daemon may not print all warnings (due to
  it not detecting the new conditions)
- some warnings (in particular, warnings about storage-drivers) depend on
  driver-status (`DriverStatus`) information. The format of the information
  returned in this field is not part of the API specification and can change
  over time, resulting in cli-side detection no longer being functional.

This patch adds a new `Warnings` field to the `/info` response. This field is
to return warnings to be presented by the user.

Existing warnings that are currently handled by the CLI are copied to the daemon
as part of this patch; This change is backward-compatible with existing
clients; old client can continue to use the client-side warnings, whereas new
clients can skip client-side detection, and print warnings that are returned by
the daemon.

Example response with this patch applied;

```bash
curl --unix-socket /var/run/docker.sock http://localhost/info | jq .Warnings
```

```json
[
  "WARNING: bridge-nf-call-iptables is disabled",
  "WARNING: bridge-nf-call-ip6tables is disabled"
]
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a3d4238b9ce653d2863fbc93057ed4162a83221e
Component: engine
2018-08-21 11:36:15 +02:00
e4dc477de6 Merge pull request #37558 from selansen/master
Global Default Address Pool feature support
Upstream-commit: 1800883bd16664846db1572b8c8fbe8c85892cee
Component: engine
2018-08-20 18:15:44 -07:00
438b026631 Merge pull request #37604 from dperny/task-wait-for-attachments
Block task starting until node attachments are ready
Upstream-commit: 9d71a574fe0197434b543d77332597c722129c1b
Component: engine
2018-08-20 18:14:44 -07:00
3569cbf496 Merge pull request #37620 from tonistiigi/buildkit-net-modes
buildkit: enable net modes and bridge
Upstream-commit: cf72051c3746fa60d7e7995c0342eaca91f07bfd
Component: engine
2018-08-20 14:56:24 -07:00
524fe098d4 Merge pull request #37675 from cpuguy83/unused_logging_code
Remove now unused multireader.
Upstream-commit: 715aa064ade3576b778f0f71ba43829ba625bf69
Component: engine
2018-08-20 22:12:38 +01:00
e44dbe9123 Block task starting until node attachments are ready
Blocks the execution of tasks during the Prepare phase until there
exists an IP address for every overlay network in use by the task. This
prevents a task from starting before the NetworkAttachment containing
the IP address has been sent down to the node.

Includes a basic test for the correct use case.

Signed-off-by: Drew Erny <drew.erny@docker.com>
Upstream-commit: 3c81dc3103d9c88cb333c80e0810f80ab80c374e
Component: engine
2018-08-20 15:28:15 -05:00
c9c36cf7eb Global Default Address Pool feature support
This feature allows user to specify list of subnets for global
default address pool. User can configure subnet list using
'swarm init' command. Daemon passes the information to swarmkit.
We validate the information in swarmkit, then store it in cluster
object. when IPAM init is called, we pass subnet list to IPAM driver.

Signed-off-by: selansen <elango.siva@docker.com>
Upstream-commit: f7ad95cab9cc7ba8925673a933028d53284c13f5
Component: engine
2018-08-20 15:07:08 -04:00
3214172ab1 builder: setup code for a bridge networking
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: d6424a088dacc902b674f402e6a62aa8f5f4a803
Component: engine
2018-08-20 18:55:01 +00:00
ebd369251b Merge pull request #37593 from AntaresS/add-enable-buildkit
[enhancement] add optional fields in daemon.json to enable buildkit
Upstream-commit: 991682749612d6613d5f49035f62e2a479c0dc59
Component: engine
2018-08-20 19:41:56 +01:00
f36591e04d Remove now unused multireader.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 5da8bc2e5b4f19d2201f8642cbb3663a7cc70c8a
Component: engine
2018-08-20 09:42:19 -07:00
d2993cf016 Merge pull request #37092 from cpuguy83/local_logger
Add "local" log driver
Upstream-commit: e0ad6d045c752e0523d8591b235ec2db32bc71fc
Component: engine
2018-08-20 07:01:41 +01:00
02ca9d77c5 add optional fields in daemon.json to enable buildkit
Signed-off-by: Anda Xu <anda.xu@docker.com>
Upstream-commit: 2be17666b4cf21995da3e4ddf4ed615e6e6ca63a
Component: engine
2018-08-19 14:58:23 -07:00
8701fc500f Expose license status in Info (#37612)
* Expose license status in Info

This wires up a new field in the Info payload that exposes the license.
For moby this is hardcoded to always report a community edition.
Downstream enterprise dockerd will have additional licensing logic wired
into this function to report details about the current license status.

Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>

* Code review comments

Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>

* Add windows autogen support

Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
Upstream-commit: 896d1b1c61a48e2df1a7b4644ddde6ee97db6111
Component: engine
2018-08-17 17:05:21 -07:00
dc90119f4b Merge pull request #37659 from Microsoft/jjh/37356
LCOW: Mount to short container paths to avoid command-line length limit
Upstream-commit: 8613b34a7e363948555eb986fd6078d9f9c808d7
Component: engine
2018-08-17 15:48:35 -07:00
98e7273292 Merge pull request #35521 from salah-khan/35507
Add --chown flag support for ADD/COPY commands for Windows
Upstream-commit: b3e9f7b13b0f0c414fa6253e1f17a86b2cff68b5
Component: engine
2018-08-17 11:31:16 -07:00
e5b37c1dc8 Add new local log driver
This driver uses protobuf to store log messages and has better defaults
for log file handling (e.g. compression and file rotation enabled by
default).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: a351b38e7217af059eb2f8fc3dba14dc03836a45
Component: engine
2018-08-17 09:36:56 -07:00
2db873cf38 LCOW: Capture stderr on external process. Log actual error rather than throwaway
Signed-off-by: John Howard <jhoward@microsoft.com>
Upstream-commit: dffc96616503936fc282c0e3a3eea8695ee11b67
Component: engine
2018-08-16 19:20:14 -07:00
cef8826091 LCOW: Mount to short container paths to avoid command-line length limit
Signed-off-by: John Howard <jhoward@microsoft.com>

Fixes #36764

@johnstep PTAL. @jterry75 FYI.

There are two commits in this PR. The first ensure that errors are actually returned to the caller - it was being thrown away.

The second commit changes the LCOW driver to map, on a per service VM basis, "long" container paths such as `/tmp/c8fa0ae1b348f505df2707060f6a49e63280d71b83b7936935c827e2e9bde16d` to much shorter paths, based on a per-service VM counter, so something more like /tmp/d3. This means that the root cause of the failure where the mount call to create the overlay was failing due to command line length becomes something much shorter such as below.

`mount -t overlay overlay -olowerdir=/tmp/d3:/tmp/d4:/tmp/d5:/tmp/d6:/tmp/d7:/tmp/d8:/tmp/d9:/tmp/d10:/tmp/d11:/tmp/d12:/tmp/d13:/tmp/d14:/tmp/d15:/tmp/d16:/tmp/d17:/tmp/d18:/tmp/d19:/tmp/d20:/tmp/d21:/tmp/d22:/tmp/d23:/tmp/d24:/tmp/d25:/tmp/d26:/tmp/d27:/tmp/d28:/tmp/d29:/tmp/d30:/tmp/d31:/tmp/d32:/tmp/d33:/tmp/d34:/tmp/d35:/tmp/d36:/tmp/d37:/tmp/d38:/tmp/d39:/tmp/d40:/tmp/d41:/tmp/d42:/tmp/d43:/tmp/d44:/tmp/d45:/tmp/d46:/tmp/d47:/tmp/d48:/tmp/d49:/tmp/d50:/tmp/d51:/tmp/d52:/tmp/d53:/tmp/d54:/tmp/d55:/tmp/d56:/tmp/d57:/tmp/d58:/tmp/d59:/tmp/d60:/tmp/d61:/tmp/d62,upperdir=/tmp/d2/upper,workdir=/tmp/d2/work /tmp/c8fa0ae1b348f505df2707060f6a49e63280d71b83b7936935c827e2e9bde16d-mount`

For those worrying about overflow (which I'm sure @thaJeztah will mention...): It's safe to use a counter here as SVMs are disposable in the default configuration. The exception is when running the daemon in unsafe LCOW "global" mode (ie `--storage-opt lcow.globalmode=1`) where the SVMs aren't disposed of, but a single one is reused. However, to overflow the command line length, it would require several hundred-thousand trillion (conservative, I should sit down and work it out accurately if I get -really- bored) of SCSI hot-add operations, and even to hit that would be hard as just running containers normally uses the VPMEM path for the containers UVM, not to the global SVM on SCSI. It gets incremented by one per build step (commit more accurately) as a general rule. Hence it would be necessary to have to be doing automated builds without restarting the daemon for literally years on end in unsafe mode. 😇

Note that in reality, the previous limit of ~47 layers before hitting the command line length limit is close to what is possible in the platform, at least as of RS5/Windows Server 2019 where, in the HCS v1 schema, a single SCSI controller is used, and that can only support 64 disks per controller per the Hyper-V VDEV. And remember we have one slot taken up for the SVMs scratch, and another for the containers scratch when committing a layer. So the best you can architecturally get on the platform is around the following (it's also different by 1 depending on whether in unsafe or default mode)

```
PS E:\docker\build\36764\short> docker build --no-cache .
Sending build context to Docker daemon  2.048kB
Step 1/4 : FROM alpine as first
 ---> 11cd0b38bc3c
Step 2/4 : RUN echo test > /test
 ---> Running in 8ddfe20e5bfb
Removing intermediate container 8ddfe20e5bfb
 ---> b0103a00b1c9
Step 3/4 : FROM alpine
 ---> 11cd0b38bc3c
Step 4/4 : COPY --from=first /test /test
 ---> 54bfae391eba
Successfully built 54bfae391eba
PS E:\docker\build\36764\short> cd ..
PS E:\docker\build\36764> docker build --no-cache .
Sending build context to Docker daemon  4.689MB
Step 1/61 : FROM alpine as first
 ---> 11cd0b38bc3c
Step 2/61 : RUN echo test > /test
 ---> Running in 02597ff870db
Removing intermediate container 02597ff870db
 ---> 3096de6fc454
Step 3/61 : RUN echo test > /test
 ---> Running in 9a8110f4ff19
Removing intermediate container 9a8110f4ff19
 ---> 7691808cf28e
Step 4/61 : RUN echo test > /test
 ---> Running in 9afb8f51510b
Removing intermediate container 9afb8f51510b
 ---> e42a0df2bb1c
Step 5/61 : RUN echo test > /test
 ---> Running in fe977ed6804e
Removing intermediate container fe977ed6804e
 ---> 55850c9b0479
Step 6/61 : RUN echo test > /test
 ---> Running in be65cbfad172
Removing intermediate container be65cbfad172
 ---> 0cf8acba70f0
Step 7/61 : RUN echo test > /test
 ---> Running in fd5b0907b6a9
Removing intermediate container fd5b0907b6a9
 ---> 257a4493d85d
Step 8/61 : RUN echo test > /test
 ---> Running in f7ca0ffd9076
Removing intermediate container f7ca0ffd9076
 ---> 3baa6f4fa2d5
Step 9/61 : RUN echo test > /test
 ---> Running in 5146814d4727
Removing intermediate container 5146814d4727
 ---> 485b9d5cf228
Step 10/61 : RUN echo test > /test
 ---> Running in a090eec1b743
Removing intermediate container a090eec1b743
 ---> a7eb10155b51
Step 11/61 : RUN echo test > /test
 ---> Running in 942660b288df
Removing intermediate container 942660b288df
 ---> 9d286a1e2133
Step 12/61 : RUN echo test > /test
 ---> Running in c3d369aa91df
Removing intermediate container c3d369aa91df
 ---> f78be4788992
Step 13/61 : RUN echo test > /test
 ---> Running in a03c3ac6888f
Removing intermediate container a03c3ac6888f
 ---> 6504363f61ab
Step 14/61 : RUN echo test > /test
 ---> Running in 0c3c2fca3f90
Removing intermediate container 0c3c2fca3f90
 ---> fe3448b8bb29
Step 15/61 : RUN echo test > /test
 ---> Running in 828d51c76d3b
Removing intermediate container 828d51c76d3b
 ---> 870684e3aea0
Step 16/61 : RUN echo test > /test
 ---> Running in 59a2f7c5f3ad
Removing intermediate container 59a2f7c5f3ad
 ---> cf84556ca5c0
Step 17/61 : RUN echo test > /test
 ---> Running in bfb4e088eeb3
Removing intermediate container bfb4e088eeb3
 ---> 9c8f9f652cef
Step 18/61 : RUN echo test > /test
 ---> Running in f1b88bb5a2d7
Removing intermediate container f1b88bb5a2d7
 ---> a6233ad21648
Step 19/61 : RUN echo test > /test
 ---> Running in 45f70577d709
Removing intermediate container 45f70577d709
 ---> 1b5cc52d370d
Step 20/61 : RUN echo test > /test
 ---> Running in 2ce231d5043d
Removing intermediate container 2ce231d5043d
 ---> 4a0e17cbebaa
Step 21/61 : RUN echo test > /test
 ---> Running in 52e4b0928f1f
Removing intermediate container 52e4b0928f1f
 ---> 99b50e989bcb
Step 22/61 : RUN echo test > /test
 ---> Running in f7ba3da7460d
Removing intermediate container f7ba3da7460d
 ---> bfa3cad88285
Step 23/61 : RUN echo test > /test
 ---> Running in 60180bf60f88
Removing intermediate container 60180bf60f88
 ---> fe7271988bcb
Step 24/61 : RUN echo test > /test
 ---> Running in 20324d396531
Removing intermediate container 20324d396531
 ---> e930bc039128
Step 25/61 : RUN echo test > /test
 ---> Running in b3ac70fd4404
Removing intermediate container b3ac70fd4404
 ---> 39d0a11ea6d8
Step 26/61 : RUN echo test > /test
 ---> Running in 0193267d3787
Removing intermediate container 0193267d3787
 ---> 8062d7aab0a5
Step 27/61 : RUN echo test > /test
 ---> Running in f41f45fb7985
Removing intermediate container f41f45fb7985
 ---> 1f5f18f2315b
Step 28/61 : RUN echo test > /test
 ---> Running in 90dd09c63d6e
Removing intermediate container 90dd09c63d6e
 ---> 02f0a1141f11
Step 29/61 : RUN echo test > /test
 ---> Running in c557e5386e0a
Removing intermediate container c557e5386e0a
 ---> dbcd6fb1f6f4
Step 30/61 : RUN echo test > /test
 ---> Running in 65369385d855
Removing intermediate container 65369385d855
 ---> e6e9058a0650
Step 31/61 : RUN echo test > /test
 ---> Running in d861fcc388fd
Removing intermediate container d861fcc388fd
 ---> 6e4c2c0f741f
Step 32/61 : RUN echo test > /test
 ---> Running in 1483962b7e1c
Removing intermediate container 1483962b7e1c
 ---> cf8f142aa055
Step 33/61 : RUN echo test > /test
 ---> Running in 5868934816c1
Removing intermediate container 5868934816c1
 ---> d5ff87cdc204
Step 34/61 : RUN echo test > /test
 ---> Running in e057f3201f3a
Removing intermediate container e057f3201f3a
 ---> b4031b7ab4ac
Step 35/61 : RUN echo test > /test
 ---> Running in 22b769b9079c
Removing intermediate container 22b769b9079c
 ---> 019d898510b6
Step 36/61 : RUN echo test > /test
 ---> Running in f1d364ef4ff8
Removing intermediate container f1d364ef4ff8
 ---> 9525cafdf04d
Step 37/61 : RUN echo test > /test
 ---> Running in 5bf505b8bdcc
Removing intermediate container 5bf505b8bdcc
 ---> cd5002b33bfd
Step 38/61 : RUN echo test > /test
 ---> Running in be24a921945c
Removing intermediate container be24a921945c
 ---> 8675db44d1b7
Step 39/61 : RUN echo test > /test
 ---> Running in 352dc6beef3d
Removing intermediate container 352dc6beef3d
 ---> 0ab0ece43c71
Step 40/61 : RUN echo test > /test
 ---> Running in eebde33e5d9b
Removing intermediate container eebde33e5d9b
 ---> 46ca4b0dfc03
Step 41/61 : RUN echo test > /test
 ---> Running in f920313a1e85
Removing intermediate container f920313a1e85
 ---> 7f3888414d58
Step 42/61 : RUN echo test > /test
 ---> Running in 10e2f4dc1ac7
Removing intermediate container 10e2f4dc1ac7
 ---> 14db9e15f2dc
Step 43/61 : RUN echo test > /test
 ---> Running in c849d6e89aa5
Removing intermediate container c849d6e89aa5
 ---> fdb770494dd6
Step 44/61 : RUN echo test > /test
 ---> Running in 419d1a8353db
Removing intermediate container 419d1a8353db
 ---> d12e9cf078be
Step 45/61 : RUN echo test > /test
 ---> Running in 0f1805263e4c
Removing intermediate container 0f1805263e4c
 ---> cd005e7b08a4
Step 46/61 : RUN echo test > /test
 ---> Running in 5bde05b46441
Removing intermediate container 5bde05b46441
 ---> 05aa426a3d4a
Step 47/61 : RUN echo test > /test
 ---> Running in 01ebc84bd1bc
Removing intermediate container 01ebc84bd1bc
 ---> 35d371fa4342
Step 48/61 : RUN echo test > /test
 ---> Running in 49f6c2f51dd4
Removing intermediate container 49f6c2f51dd4
 ---> 1090b5dfa130
Step 49/61 : RUN echo test > /test
 ---> Running in f8a9089cd725
Removing intermediate container f8a9089cd725
 ---> b2d0eec0716d
Step 50/61 : RUN echo test > /test
 ---> Running in a1697a0b2db0
Removing intermediate container a1697a0b2db0
 ---> 10d96ac8f497
Step 51/61 : RUN echo test > /test
 ---> Running in 33a2332c06eb
Removing intermediate container 33a2332c06eb
 ---> ba5bf5609c1c
Step 52/61 : RUN echo test > /test
 ---> Running in e8920392be0d
Removing intermediate container e8920392be0d
 ---> 5b3a95685c7e
Step 53/61 : RUN echo test > /test
 ---> Running in 4b9298587c65
Removing intermediate container 4b9298587c65
 ---> d4961a349141
Step 54/61 : RUN echo test > /test
 ---> Running in 8a0c960c2ba1
Removing intermediate container 8a0c960c2ba1
 ---> b413197fcfa2
Step 55/61 : RUN echo test > /test
 ---> Running in 536ee3b9596b
Removing intermediate container 536ee3b9596b
 ---> fc16b69b224a
Step 56/61 : RUN echo test > /test
 ---> Running in 8b817b8d7b59
Removing intermediate container 8b817b8d7b59
 ---> 2f0896400ff9
Step 57/61 : RUN echo test > /test
 ---> Running in ab0ed79ec3d4
Removing intermediate container ab0ed79ec3d4
 ---> b4fb420e736c
Step 58/61 : RUN echo test > /test
 ---> Running in 8548d7eead1f
Removing intermediate container 8548d7eead1f
 ---> 745103fd5a38
Step 59/61 : RUN echo test > /test
 ---> Running in 1980559ad5d6
Removing intermediate container 1980559ad5d6
 ---> 08c1c74a5618
Step 60/61 : FROM alpine
 ---> 11cd0b38bc3c
Step 61/61 : COPY --from=first /test /test
 ---> 67f053c66c27
Successfully built 67f053c66c27
PS E:\docker\build\36764>
```

Note also that subsequent error messages once you go beyond current platform limitations kind of suck (such as insufficient resources with a bunch of spew which is incomprehensible to most) and we could do better to detect this earlier in the daemon. That'll be for a (reasonably low-priority) follow-up though as and when I have time. Theoretically we *may*, if the platform doesn't require additional changes for RS5, be able to have bigger platform limits using the v2 schema with up to 127 VPMem devices, and the possibility to have multiple SCSI controllers per SVM/UVM. However, currently LCOW is using HCS v1 schema calls, and there's no plans to rewrite the graphdriver/libcontainerd components outside of the moving LCOW fully over to the containerd runtime/snapshotter using HCS v2 schema, which is still some time off fruition.

PS OK, while waiting for a full run to complete, I did get bored. Turns out it won't overflow line length as max(uint64) is 18446744073709551616 which would still be short enough at 127 layers, double the current platform limit. And I could always change it to hex or base36 to make it even shorter, or remove the 'd' from /tmp/dN. IOW, pretty sure no-one is going to hit the limit even if we could get the platform to 256 which is the current Hyper-V SCSI limit per VM (4x64), although PMEM at 127 would be the next immediate limit.
Upstream-commit: f586fe5637bbadd919ce4a126c6d95e4f3b1523b
Component: engine
2018-08-16 19:18:55 -07:00
9dae42985b Merge pull request #37149 from dmcgowan/split-libcontainerd
libcontainerd: split client and daemon supervision
Upstream-commit: 4d6219264655bbb4510642e40d7914fd56962596
Component: engine
2018-08-16 19:00:10 +02:00
adf79edc07 Merge pull request #37472 from thaJeztah/no_unknown
Do not return "<unknown>" in /info response
Upstream-commit: 0a4f89566a22a07427d850a8d70f264210062da0
Component: engine
2018-08-16 14:56:31 +02:00
1eafd46aad Merge pull request #37412 from AzureCR/naduggar/logissue
Select polling based watcher for docker log file watcher on Windows
Upstream-commit: 678d4b3a6d4cd7824d26592090b4dec507ce4bc2
Component: engine
2018-08-14 14:40:44 +02:00
94eb6513c1 Add unit test for swarm labels on containers
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 6f8d17dad3cf29c2ff2d019204c68227ebadfcc8
Component: engine
2018-08-14 12:31:53 +02:00
7414934b6c Add ADD/COPY --chown flag support to Windows
This implements chown support on Windows. Built-in accounts as well
as accounts included in the SAM database of the container are supported.

NOTE: IDPair is now named Identity and IDMappings is now named
IdentityMapping.

The following are valid examples:
ADD --chown=Guest . <some directory>
COPY --chown=Administrator . <some directory>
COPY --chown=Guests . <some directory>
COPY --chown=ContainerUser . <some directory>

On Windows an owner is only granted the permission to read the security
descriptor and read/write the discretionary access control list. This
fix also grants read/write and execute permissions to the owner.

Signed-off-by: Salahuddin Khan <salah@docker.com>
Upstream-commit: 763d8392612942ff5c32a35f8bdafd7ae93d3321
Component: engine
2018-08-13 21:59:11 -07:00
feccebcffe Decouple logfile from tailfile.
This makes it so consumers of `LogFile` should pass in how to get an
io.Reader to the requested number of lines to tail.

This is also much more efficient when tailing a large number of lines.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 94a10150f64a24793216f5324a34e62be3f31a2d
Component: engine
2018-08-10 21:02:19 -07:00
71a3e53c36 Fix the several typos detected by github.com/client9/misspell
Signed-off-by: Kazuhiro Sera <seratch@gmail.com>
Upstream-commit: 1e49fdcafc6601cd9dd5c46551ee283baa0272ef
Component: engine
2018-08-09 00:45:00 +09:00
084e7423dc libcontainerd: split client and supervisor
Adds a supervisor package for starting and monitoring containerd.
Separates grpc connection allowing access from daemon.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: dd2e19ebd58cd8896d79b4b8db61144b93717b33
Component: engine
2018-08-06 10:23:04 -07:00
b6bdcc10a5 Merge pull request #37485 from fcrisciani/resolv
Systemd-resolved proper handling
Upstream-commit: e1584514c15f6ad91b9ef5bc530a1eef46f18267
Component: engine
2018-08-01 14:52:33 +02:00
a8fa741f70 loggerutils: fix a typo
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 2c6fbd864ae089e9cb544666949b8b2d897b3b23
Component: engine
2018-07-31 15:44:46 +03:00
3cb14e951e Merge pull request #37462 from cwgem/37453_slash_prefix_removal
Remove slash prefix when matching name filters (Fixes #37453)
Upstream-commit: 7b46fbc17ab991804126588a8dfd09e54ec5902b
Component: engine
2018-07-30 13:17:40 +02:00
1d8c80a782 Fixes for resolv.conf
Handle the case of systemd-resolved, and if in place
use a different resolv.conf source.
Set appropriately the option on libnetwork.
Move unix specific code to container_operation_unix

Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
Upstream-commit: e353e7e3f0ce8eceeff657393cba2876375403fa
Component: engine
2018-07-26 11:17:56 -07:00
b9ebe2f524 Add configuration option for containerd cri
Disable cri plugin by default in containerd and
allows an option to enable the plugin. This only
has an effect on containerd when supervised by
dockerd. When containerd is managed outside of
dockerd, the configuration is not effected.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: 8fb5f4d5c9b4933be31bf5371d65a95edb037261
Component: engine
2018-07-24 11:34:47 -07:00
59e4a5a574 Do not return "<unknown>" in /info response
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e6e8ab50fad5397f7be07fa00ba2bca85860cbe8
Component: engine
2018-07-16 16:09:58 +02:00
e409641b96 Remove slash prefix when matching name filters (Fixes #37453)
* Regex name filters were display undesired behavior due to
  names containing the trailing slash when being compared
* Adjusted filterByNameIDMatches and includeContainerInList to
  strip the slash prefix before doing name comparisons
* Added test case and helper functions for the test to list_test
* Force failed tests during development to ensure there were
  no false positives

Signed-off-by: Chris White <me@cwprogram.com>
Upstream-commit: 5c8da2e96738addd8a175e5b9a23b8c37d4a4dfe
Component: engine
2018-07-15 05:57:29 -05:00
3c9a878585 Merge pull request #37451 from thaJeztah/refactor_system_info
Refactor daemon.info to reduce cyclomatic complexity
Upstream-commit: a2a1d04bcae54129b5c4ddad4e4f68458fb5ef3d
Component: engine
2018-07-13 16:47:28 -07:00
c1dba76a94 Unexport daemon.FillPlatformInfo
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: c03d3a416b24c18dfbb938397052eebd8b67d1a3
Component: engine
2018-07-13 13:16:34 +02:00
91b7b13fc2 Refactor daemon.info to reduce cyclomatic complexity
Before this change;

    gocyclo daemon/info.go
    17 daemon (*Daemon).SystemInfo daemon/info.go:27:1
    2 daemon (*Daemon).SystemVersion daemon/info.go:150:1
    1 daemon (*Daemon).showPluginsInfo daemon/info.go:195:1

After this change;

    gocyclo daemon/info.go
    8 daemon (*Daemon).fillSecurityOptions daemon/info.go:150:1
    5 daemon operatingSystem daemon/info.go:201:1
    3 daemon (*Daemon).fillDriverInfo daemon/info.go:121:1
    2 daemon hostName daemon/info.go:172:1
    2 daemon memInfo daemon/info.go:192:1
    2 daemon kernelVersion daemon/info.go:182:1
    1 daemon (*Daemon).SystemVersion daemon/info.go:81:1
    1 daemon (*Daemon).SystemInfo daemon/info.go:27:1
    1 daemon (*Daemon).fillPluginsInfo daemon/info.go:138:1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 50eaed4d0c1492479ed4d83eb97424d21a3e458d
Component: engine
2018-07-13 13:10:05 +02:00
8a9efc106f lcow: fix debug in startServiceVMIfNotRunning()
When go-1.11beta1 is used for building, the following error is
reported:

> 14:56:20 daemon\graphdriver\lcow\lcow.go:236: Debugf format %s reads
> arg #2, but call has 1 arg

While fixing this, let's also fix a few other things in this
very function (startServiceVMIfNotRunning):

1. Do not use fmt.Printf when not required.
2. Use `title` whenever possible.
3. Don't add `id` to messages as `title` already has it.
4. Remove duplicated colons.
5. Try to unify style of messages.
6. s/startservicevmifnotrunning/startServiceVMIfNotRunning/
...

In general, logging/debugging here is a mess and requires much more
love than I can give it at the moment. Areas for improvement:

1. Add a global var logger = logrus.WithField("storage-driver", "lcow")
and use it everywhere else in the code.
2. Use logger.WithField("id", id) whenever possible (same for "context"
and other similar fields).
3. Revise all the errors returned to be uniform.
4. Make use of errors.Wrap[f] whenever possible.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: b7a95a3ce4c731c0fca204435be758ea89d6050f
Component: engine
2018-07-12 13:42:23 -07:00
99d1e1cf76 loggerutils: build fixes, improve errors
There are two build errors when using go-1.11beta1:

> daemon/logger/loggerutils/logfile.go:367: Warningf format %q arg f.Name is a func value, not called
> daemon/logger/loggerutils/logfile.go:564: Debug call has possible formatting directive %v

In the first place, the file name is actually not required as error
message already includes it.

While at it, fix a couple of other places for more correct messages, and
make sure to not add a file name if an error already has it.

Fixes: f69f09f44c
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 09ad434f10cff48741322854a3003686b28295b5
Component: engine
2018-07-11 15:52:25 +02:00
d01a9704de aufs: fix Wrapf args
Fix the following go-1.11beta1 build error:

> daemon/graphdriver/aufs/aufs.go:376: Wrapf format %s reads arg #1, but call has 0 args

While at it, change '%s' to %q.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2e30e9e6db42043cb2bd67d25a7152488c834f9f
Component: engine
2018-07-11 15:52:22 +02:00
14a71772f5 Compile fix
Go 1.11beta1 (rightfully) complains:

> 15:38:37 daemon/cluster/controllers/plugin/controller.go:183:
> Entry.Debugf format %#T has unrecognized flag #

This debug print was added by commit 72c3bcf2a5.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a9a136572dc2a2c2ec5da320c4d0a32b5a15c550
Component: engine
2018-07-11 15:52:19 +02:00
2a7994f372 daemon/*.go: fix some Wrap[f]/Warn[f] errors
In particular, these two:
> daemon/daemon_unix.go:1129: Wrapf format %v reads arg #1, but call has 0 args
> daemon/kill.go:111: Warn call has possible formatting directive %s

and a few more.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 3737194b9f2875c10f3f2117c1816054ba0ff262
Component: engine
2018-07-11 15:51:51 +02:00
a6e2e0bba5 Select polling based watcher for Windows log watcher
Signed-off-by: Tejaswini Duggaraju <naduggar@microsoft.com>
Upstream-commit: df84cdd091631f32abaedd805791db89ac5c83d4
Component: engine
2018-07-10 10:20:10 -07:00
730b7c1410 Merge pull request #37372 from ctelfer/scalable-lb
Improve scalability of the Linux load balancing
Upstream-commit: 328c089b5bca66594c4937e88cf9879879c4eed7
Component: engine
2018-07-09 20:06:37 +02:00
f1b7597661 Get err type in removeNetworks() w/ errors.Cause()
Commit c0bc14e8 wrapped the return value of nw.Delete() with some extra
information.  However, this breaks the code in
containerAdaptor.removeNetworks() which ignores certain specific
libnetwork error return codes.  Said codes actually don't represent
errors, but just regular conditions to be expected in normal operation.
The removeNetworks() call checked for these errors by type assertions
which the errors.Wrap(err...) breaks.

This has a cascading effect, because controller.Remove() invokes
containerAdaptor.removeNetworks() and if the latter returns an error,
then Remove() fails to remove the container itself.  This is not
necessarily catastrophic since the container reaper apparently will
purge the container later, but it is clearly not the behavior we want.

Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: 6225d1f15c5fd916c3e0ef3afe022f6cc14ac696
Component: engine
2018-07-07 18:18:53 -04:00