Commit Graph

54 Commits

Author SHA1 Message Date
4f76283213 builder: use buildkit's GC for build cache
This allows users to configure the buildkit GC.

The following enables the default GC:
```
{
  "builder": {
    "gc": {
      "enabled": true
    }
  }
}
```

The default GC policy has a simple config:
```
{
  "builder": {
    "gc": {
      "enabled": true,
      "defaultKeepStorage": "30GB"
    }
  }
}
```

A custom GC policy can be used instead by specifying a list of cache prune rules:
```
{
  "builder": {
    "gc": {
      "enabled": true,
      "policy": [
        {"keepStorage": "512MB", "filter": ["unused-for=1400h"]]},
        {"keepStorage": "30GB", "all": true}
      ]
    }
  }
}
```

Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 4a776d0ca76c4bdf4399aef8c102361d6c2819eb)
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Upstream-commit: 73e2f72a7c5bd6d6f8306e0ffe4371e1c3b00a21
Component: engine
2018-09-21 17:06:25 -07:00
812694c0cc fixed the dockerd won't start bug when 'runtimes' field is defined in both daemon config file and cli flags
Signed-off-by: Anda Xu <anda.xu@docker.com>
(cherry picked from commit 8392d0930b511402aa7aa71ccfe6c0d4a8159237)
Upstream-commit: 66ed41aec82dbcdfbc38027e3d800e429af1cd58
Component: engine
2018-09-20 10:54:47 -07:00
ed97b30d09 Fix logic when enabling buildkit
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: c973cde7606dc7a2557094fc90d8e6bb595fa354
Component: engine
2018-08-21 23:49:08 +00:00
02ca9d77c5 add optional fields in daemon.json to enable buildkit
Signed-off-by: Anda Xu <anda.xu@docker.com>
Upstream-commit: 2be17666b4cf21995da3e4ddf4ed615e6e6ca63a
Component: engine
2018-08-19 14:58:23 -07:00
b6bdcc10a5 Merge pull request #37485 from fcrisciani/resolv
Systemd-resolved proper handling
Upstream-commit: e1584514c15f6ad91b9ef5bc530a1eef46f18267
Component: engine
2018-08-01 14:52:33 +02:00
1d8c80a782 Fixes for resolv.conf
Handle the case of systemd-resolved, and if in place
use a different resolv.conf source.
Set appropriately the option on libnetwork.
Move unix specific code to container_operation_unix

Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
Upstream-commit: e353e7e3f0ce8eceeff657393cba2876375403fa
Component: engine
2018-07-26 11:17:56 -07:00
b9ebe2f524 Add configuration option for containerd cri
Disable cri plugin by default in containerd and
allows an option to enable the plugin. This only
has an effect on containerd when supervised by
dockerd. When containerd is managed outside of
dockerd, the configuration is not effected.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: 8fb5f4d5c9b4933be31bf5371d65a95edb037261
Component: engine
2018-07-24 11:34:47 -07:00
4cadaa03f8 Update tests to use gotest.tools 👼
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: 38457285242e57306c5b7ee652c7ccbb9fbd6713
Component: engine
2018-06-13 09:04:30 +02:00
0f96e98e12 Various code-cleanup
remove unnescessary import aliases, brackets, and so on.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f23c00d8701e4bd0f2372a586dacbf66a26f9a51
Component: engine
2018-05-23 17:50:54 +02:00
fac8e1d52b Replace deprecated testutil.ErrorContains()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 55bebbaecf5e40db9d83b28080ce08dc8642f407
Component: engine
2018-05-21 00:13:04 +02:00
194bacfc7b Merge pull request #36921 from cyli/filter-namespaced-labels
Warn when reserved-namespace engine labels are configured
Upstream-commit: 57493cd60628ebf6e8b02725f44e1a4ef152a39e
Component: engine
2018-05-07 15:12:52 +02:00
b9f36ffdf0 Allow user to control the default address pools
- Via daemon flag --default-address-pools base=<CIDR>,size=<int>

Signed-off-by: Elango Siva  <elango@docker.com>
Upstream-commit: 173b3c364ebedd56bd5b6c190a9b41f592ba57d6
Component: engine
2018-04-30 11:14:08 -04:00
d865a73a75 Filter engine labels to exclude com.docker.*, io.docker.*, and org.dockerproject.*
labels as per https://docs.docker.com/config/labels-custom-metadata/.

Signed-off-by: Ying Li <ying.li@docker.com>
Upstream-commit: d1d7bcd6d73516af5ae70c01bf08a0f73ad87743
Component: engine
2018-04-24 11:42:09 -07:00
7f23cb8c8b Skip some tests requires root uid when run as user
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Upstream-commit: a7999aaa536818085ec8a23901ef64637587b7e4
Component: engine
2018-04-23 10:14:39 +02:00
9a7c928884 Expose swarm raft tuning parameters in engine config
Signed-off-by: David Chung <david.chung@docker.com>
Upstream-commit: 275a1ca7c8ec1d556c37bb50ac574beffbb16e5a
Component: engine
2018-03-28 16:54:43 -07:00
60daf5fa97 Automated migration using
gty-migrate-from-testify --ignore-build-tags

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 6be0f709830113966f295401327b027ec2f0bbca
Component: engine
2018-03-16 11:03:43 -04:00
be83c11fb0 Add canonical import comment
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 4f0d95fa6ee7f865597c03b9e63702cdcb0f7067
Component: engine
2018-02-05 16:51:57 -05:00
36e0e57cbe Log active configuration when reloading
When succesfully reloading the daemon configuration, print a message
in the logs with the active configuration:

    INFO[2018-01-15T15:36:20.901688317Z] Got signal to reload configuration, reloading from: /etc/docker/daemon.json
    INFO[2018-01-14T02:23:48.782769942Z] Reloaded configuration: {"mtu":1500,"pidfile":"/var/run/docker.pid","data-root":"/var/lib/docker","exec-root":"/var/run/docker","group":"docker","deprecated-key-path":"/etc/docker/key.json","max-concurrent-downloads":3,"max-concurrent-uploads":5,"shutdown-timeout":15,"debug":true,"hosts":["unix:///var/run/docker.sock"],"log-level":"info","swarm-default-advertise-addr":"","metrics-addr":"","log-driver":"json-file","ip":"0.0.0.0","icc":true,"iptables":true,"ip-forward":true,"ip-masq":true,"userland-proxy":true,"disable-legacy-registry":true,"experimental":false,"network-control-plane-mtu":1500,"runtimes":{"runc":{"path":"docker-runc"}},"default-runtime":"runc","oom-score-adjust":-500,"default-shm-size":67108864,"default-ipc-mode":"shareable"}

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8378dcf46d017c70df97d6f851e0196b113b422e
Component: engine
2018-01-21 00:56:02 +01:00
621388138c Golint: remove redundant ifs
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b4a63139696aea2c73ec361a9af8b36a118f0423
Component: engine
2018-01-15 00:42:25 +01:00
ef4dfd2f67 Remove Solaris files
Solaris is no longer being worked on, so these files
are now just dead code.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1589cc0a85396e2768bfe9e558c7c2100dc3bc87
Component: engine
2017-12-18 17:22:25 +01:00
4d38282082 Add option to enable networkDB debug
Add a new configuration option to allow the enabling
of the networkDB debug. The option is only parsed using the
reload event. This will protect the daemon on start or restart
if the option is left behind in the config file

Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
Upstream-commit: a97e45794ea8318a08daf763a5b63b04184a886b
Component: engine
2017-12-06 13:19:47 -08:00
aa1c9fab4f Remove deprecated support for duplicate label-keys
Support for duplicate labels (but different values) was
deprecated in commit e4c9079d091a2eeac8a74a0356e3f348db873b87
(Docker 1.13), and scheduled for removal in 17.12

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8c6322338c91cdb88b1fef4def393d9a7b670366
Component: engine
2017-11-12 03:11:37 +01:00
aee7705a5f Added NodeGenericResource config tests
Signed-off-by: Renaud Gaubert <renaud.gaubert@gmail.com>
Upstream-commit: 734346a37e0cd5d2576f759d302beed5033ff14e
Component: engine
2017-11-06 21:47:26 +01:00
e5124fd7ef Updated GenericResource CLI
Signed-off-by: Renaud Gaubert <renaud.gaubert@gmail.com>
Upstream-commit: ebe14310b7728cca6d092ff66236577a3f3016d5
Component: engine
2017-11-06 21:47:26 +01:00
eefbd135ae Remove solaris build tag and `contrib/mkimage/solaris
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 4785f1a7ab7ec857dc3ca849ee6ecadf519ef30e
Component: engine
2017-11-02 00:01:46 +00:00
7f1869d232 Merge pull request #35285 from crosbymichael/solaris
Remove solaris files
Upstream-commit: 17bb1d3663f6586e83b453670526e3186bb56dd3
Component: engine
2017-10-25 15:14:04 +02:00
d78181e968 Remove solaris files
For obvious reasons that it is not really supported now.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Upstream-commit: 5a9b5f10cf967f31f0856871ad08f9a0286b4a46
Component: engine
2017-10-24 15:39:34 -04:00
2740139c0c Merge pull request #35125 from ripcurld0/reload_no_config
Reload daemon even if "/etc/docker/daemon.json" does not exist
Upstream-commit: 04e8d7b8bfc531fdfc6fb201fae39820002455fe
Component: engine
2017-10-24 21:23:49 +02:00
b49329396e Reload daemon even if "/etc/docker/daemon.json" does not exist
Before this commit if "--config-file" wasn't set the daemon would use
the default configuration file which is "/etc/docker/daemon.json".

When attempting to reload the daemon if that file didn't exist
and error message would display.

This behaviour is changed in a way that if the default configuration
file does not exist and no other configuration file is set
the daemon uses an empty configuration which later will be updated
and reloaded using the "reload" function given as an argument in Reload.

However, if the "--config-file" is set and the file is removed
or renamed an error message will be displayed and no reload will be done.

Signed-off-by: Boaz Shuster <ripcurld.github@gmail.com>
Upstream-commit: 0f8119a87dc301e9543c45c416d6b4545da1fc46
Component: engine
2017-10-22 15:20:52 +03:00
044d7f995b Update libcontainerd to use containerd 1.0
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
Upstream-commit: ddae20c032058a0fd42c34c2e9750ee8f6296ac8
Component: engine
2017-10-20 07:11:37 -07:00
803b5f0e94 Merge pull request #34821 from thaJeztah/remove-enable-api-cors
Remove deprecated --enable-api-cors flag
Upstream-commit: dc35a8a5d22b6c828cd105be7c62b72919df25cd
Component: engine
2017-09-13 20:10:27 -07:00
18c29c5c5f Add gosimple linter
Update gometalinter

Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: f7f101d57ef8cbf2d8723a18b7d723c5c5dd04b6
Component: engine
2017-09-12 12:09:59 -04:00
3e4a18de4d Remove deprecated --enable-api-cors flag
The `--enable-api-cors` flag was deprecated in f3dd2db4ff7de1399a49af928cd3eae4fccf8764,
and marked for removal in docker 17.09 through 85f92ef3590b386ea17e3948262725a2d3ce4db5.

This patch removes the deprecated flag.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 7d4eab554379524c2421a7ecd3319d1d087d1de2
Component: engine
2017-09-12 12:43:34 +02:00
931cac7408 Fix bad import graph from opts/opts.go
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: b68221c37ee597950364788204546f9c9d0e46a1
Component: engine
2017-08-29 15:32:43 -04:00
9cb0439501 Move ErrorContains to an internal package.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: e62b2d410cda6ac9166ac8d052fd3fb99a9460bf
Component: engine
2017-08-25 12:04:58 -04:00
b03ce80e95 Update tests to use gotestyourself/fs
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 60672382c7eb1f65e063c4bf07b0880559cea91b
Component: engine
2017-08-23 17:25:00 -04:00
6b3b192846 Add goimports to linters.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Upstream-commit: 372670b5074b077927314cdf46af30f8752e7db0
Component: engine
2017-08-21 18:15:08 -04:00
eca24fb7b0 Implement none, private, and shareable ipc modes
Since the commit d88fe447df0e8 ("Add support for sharing /dev/shm/ and
/dev/mqueue between containers") container's /dev/shm is mounted on the
host first, then bind-mounted inside the container. This is done that
way in order to be able to share this container's IPC namespace
(and the /dev/shm mount point) with another container.

Unfortunately, this functionality breaks container checkpoint/restore
(even if IPC is not shared). Since /dev/shm is an external mount, its
contents is not saved by `criu checkpoint`, and so upon restore any
application that tries to access data under /dev/shm is severily
disappointed (which usually results in a fatal crash).

This commit solves the issue by introducing new IPC modes for containers
(in addition to 'host' and 'container:ID'). The new modes are:

 - 'shareable':	enables sharing this container's IPC with others
		(this used to be the implicit default);

 - 'private':	disables sharing this container's IPC.

In 'private' mode, container's /dev/shm is truly mounted inside the
container, without any bind-mounting from the host, which solves the
issue.

While at it, let's also implement 'none' mode. The motivation, as
eloquently put by Justin Cormack, is:

> I wondered a while back about having a none shm mode, as currently it is
> not possible to have a totally unwriteable container as there is always
> a /dev/shm writeable mount. It is a bit of a niche case (and clearly
> should never be allowed to be daemon default) but it would be trivial to
> add now so maybe we should...

...so here's yet yet another mode:

 - 'none':	no /dev/shm mount inside the container (though it still
		has its own private IPC namespace).

Now, to ultimately solve the abovementioned checkpoint/restore issue, we'd
need to make 'private' the default mode, but unfortunately it breaks the
backward compatibility. So, let's make the default container IPC mode
per-daemon configurable (with the built-in default set to 'shareable'
for now). The default can be changed either via a daemon CLI option
(--default-shm-mode) or a daemon.json configuration file parameter
of the same name.

Note one can only set either 'shareable' or 'private' IPC modes as a
daemon default (i.e. in this context 'host', 'container', or 'none'
do not make much sense).

Some other changes this patch introduces are:

1. A mount for /dev/shm is added to default OCI Linux spec.

2. IpcMode.Valid() is simplified to remove duplicated code that parsed
   'container:ID' form. Note the old version used to check that ID does
   not contain a semicolon -- this is no longer the case (tests are
   modified accordingly). The motivation is we should either do a
   proper check for container ID validity, or don't check it at all
   (since it is checked in other places anyway). I chose the latter.

3. IpcMode.Container() is modified to not return container ID if the
   mode value does not start with "container:", unifying the check to
   be the same as in IpcMode.IsContainer().

3. IPC mode unit tests (runconfig/hostconfig_test.go) are modified
   to add checks for newly added values.

[v2: addressed review at https://github.com/moby/moby/pull/34087#pullrequestreview-51345997]
[v3: addressed review at https://github.com/moby/moby/pull/34087#pullrequestreview-53902833]
[v4: addressed the case of upgrading from older daemon, in this case
     container.HostConfig.IpcMode is unset and this is valid]
[v5: document old and new IpcMode values in api/swagger.yaml]
[v6: add the 'none' mode, changelog entry to docs/api/version-history.md]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 7120976d74195a60334c688a061270a4d95f9aeb
Component: engine
2017-08-14 10:50:39 +03:00
d659edcaf5 Update logrus to v1.0.1
Fixes case sensitivity issue

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: 1009e6a40b295187e038b67e184e9c0384d95538
Component: engine
2017-07-31 13:16:46 -07:00
c202b3f97f Allow to set the control plane MTU
Add daemon config to allow the user to specify the MTU of the control plane network.
The first user of this new parameter is actually libnetwork that can seed the
gossip with the proper MTU value allowing to pack multiple messages per UDP packet sent.
If the value is not specified or is lower than 1500 the logic will set it to the default.

Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
Upstream-commit: f9f25ca5e44c89d7c1ebdfa9865076eb2cde9bb2
Component: engine
2017-07-28 13:52:03 -07:00
2029cf1c2d Added support for Generic Resources
Signed-off-by: Renaud Gaubert <rgaubert@nvidia.com>
Upstream-commit: 87e1464c438f4163e2694a0bf29ae5a45903449b
Component: engine
2017-07-24 17:49:56 -07:00
ead60478b8 add NamedUlimitOpt implement NamedOption to fix 32528
Signed-off-by: allencloud <allen.sun@daocloud.io>
Upstream-commit: 148f2711e8e78c9cf43b4c30b03b33b16d9e7098
Component: engine
2017-06-16 10:14:09 +08:00
bd56d0486b daemon: correctly try to retrieve init/runtime versions
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
Upstream-commit: 858b4b44c8172eb2c92767c8f624f4138db5212b
Component: engine
2017-06-06 19:36:24 +02:00
6481987490 Expose trust key path in config
Allows storing key under any directory. In the case where the
"/etc/docker" directory is not preserved, this file can be
specified to a location where it will be preserved to ensure
the ID does not change across restarts.

Note this key is currently only used today to generate the ID
used in Docker info and for manifest schema v1 pushes. The key
signature and finger on these manifests are not checked or
used any longer for security, deprecated by notary.

Removes old key migration from a pre-release of Docker which put
the key under the home directory and was used to preserve ID used
for swarm v1 after the file moved.

closes #32135

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Upstream-commit: e428c824c35e85a02fffee592b79ab7db1a0c4d2
Component: engine
2017-05-10 14:16:27 -07:00
3f0220808f daemon/info: don't sort authorization plugins as order matters
plugins exist in a chain that composes potentially mutating requests and
responses. This simply reverts the sorting of AuthZ plugins so that the
/info API endpoint returns the internal ordering used for AuthZ composition.

Volume driver plugins are not affected because they are just a set.

Signed-off-by: David Sheets <dsheets@docker.com>
Upstream-commit: cfcf2a0cec8d83c1584020ccaab0c94f5bb6d7d4
Component: engine
2017-05-02 09:39:32 +01:00
142883d5b3 Remove pkg/testutil/assert in favor of testify
I noticed that we're using a homegrown package for assertions. The
functions are extremely similar to testify, but with enough slight
differences to be confusing (for example, Equal takes its arguments in a
different order). We already vendor testify, and it's used in a few
places by tests.

I also found some problems with pkg/testutil/assert. For example, the
NotNil function seems to be broken. It checks the argument against
"nil", which only works for an interface. If you pass in a nil map or
slice, the equality check will fail.

In the interest of avoiding NIH, I'm proposing replacing
pkg/testutil/assert with testify. The test code looks almost the same,
but we avoid the confusion of having two similar but slightly different
assertion packages, and having to maintain our own package instead of
using a commonly-used one.

In the process, I found a few places where the tests should halt if an
assertion fails, so I've made those cases (that I noticed) use "require"
instead of "assert", and I've vendored the "require" package from
testify alongside the already-present "assert" package.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 6052f2b3969feadb01662d8e2f30337d9c7f61af
Component: engine
2017-04-14 12:03:21 -07:00
3d6e7463ee Fix missing Init Binary in docker info output
- Moved DefaultInitBinary from daemon/daemon.go to
daemon/config/config.go since it's a daemon config and is referred in
config package files.
- Added condition in GetInitPath to check for any explicitly configured
DefaultInitBinary. If not, the default value of DefaultInitBinary is
returned.
- Changed all references of DefaultInitBinary to refer to the variable
from new location.
- Added TestCommonUnixGetInitPath to test for the various values of
GetInitPath.

Fixes #32314

Signed-off-by: Sunny Gogoi <indiasuny000@gmail.com>
Upstream-commit: 17b128876028022991e2dbcb2cc402cc81b451e5
Component: engine
2017-04-10 16:54:07 +05:30
fd940e1e8e Deprecate --graph flag; Replace with --data-root
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Upstream-commit: 261ef1fa27c4dfefa5f686b0a4ef354a43040e58
Component: engine
2017-03-30 11:24:56 +02:00
2034662b7a When authz plugin is disabled, remove from authz middleware chain.
When the daemon is configured to run with an authorization-plugin and if
the plugin is disabled, the daemon continues to send API requests to the
plugin and expect it to respond. But the plugin has been disabled. As a
result, all API requests are blocked. Fix this behavior by removing the
disabled plugin from the authz middleware chain.

Tested using riyaz/authz-no-volume-plugin and observed that after
disabling the plugin, API request/response is functional.

Fixes #31836

Signed-off-by: Anusha Ragunathan <anusha.ragunathan@docker.com>
Upstream-commit: 38de272bd4dfea945985b7031cd353ac5f6507c5
Component: engine
2017-03-22 12:07:39 -07:00
f3bfd3cf98 Merge pull request #28279 from chchliang/updown
--max-concurrent-downloads,--max-concurrent-uploads must great than or equal to 0 
Upstream-commit: dc1b0e1b2c8483e0f2eaa28316be01aae6d6ae4a
Component: engine
2017-03-03 10:06:59 +01:00