When a plugin has an activation error, it was not being checked in the
`waitActive` loop. This means it will just wait forever for a manifest
to be populated even though it may never come.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: f2d384fca6fa08da13fdc01c7991e8e35b081198
Component: engine
c54b717 caused a regression for pluginv1 on Windows, where extraneous
backslashes were added to BasePath of the plugin. For pluginv1 on windows,
BasePath() should return an empty string, since the plugin is fully aware
of the mount path. Also, unlike Linux where all paths are relative to "/",
Windows paths are dependent on system drives and mapped drives.
Fixes#30148
Signed-off-by: Anusha Ragunathan <anusha.ragunathan@docker.com>
Upstream-commit: 0ef21eb0e30d2ea036730a7c5502f888c6b763d1
Component: engine
`plugins.GetAll()` was not locking the plugin map when checking if a
plugin exists, this can cause a race and potentially a panic if another
goroutine is trying to load a plugin into the map at the same time.
Also fixes a race during activation where a plugin inserts itself into
the plugin map but does not check if something else is already there.
This is already checked before trying to activate the plugin, however
the map lock is not held for this entire period, so other plugins may be
loaded during this time.
To fix, before inserting the plugin into the map, check if one with the
same name already exists and use that instead.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 21fcbb39b73310e69d6403a1cfa8b26799cc1355
Component: engine
When a plugin is activated, and then `plugins.Handle` is called to
register a new handler for a given plugin type, a deadlock occurs when
for anything which calls `waitActive`, including `Get`, and `GetAll`.
This happens because `Handle()` is setting `activated` to `false` to
ensure that plugin handlers are run on next activation.
Maybe these handlers should be called immediately for any plugins which
are already registered... but to preserve the existing behavior while
fixing the deadlock, track if handlers have been run on plugins and
reset when a new handler is registered.
The simplest way to reproduce the deadlock with Docker is to add a `-v
/foo` to the test container created for the external graphdriver tests.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 2938dce794be7559ba73b4e9630015020a7fa937
Component: engine
Signed-off-by: Jie Luo <luo612@zju.edu.cn>
typo
Signed-off-by: Jie Luo <luo612@zju.edu.cn>
fix some typos
Signed-off-by: Jie Luo <luo612@zju.edu.cn>
Upstream-commit: ea2dd4b5d0b41552d047814d9e39ddaa3662ab41
Component: engine
Legacy plugins expect host-relative paths (such as for Volume.Mount).
However, a containerized plugin cannot respond with a host-relative
path. Therefore, this commit modifies new volume plugins' paths in Mount
and List to prepend the container's rootfs path.
This introduces a new PropagatedMount field in the Plugin Config.
When it is set for volume plugins, RootfsPropagation is set to rshared
and the path specified by PropagatedMount is bind-mounted with rshared
prior to launching the container. This is so that the daemon code can
access the paths returned by the plugin from the host mount namespace.
Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: c54b717caf1a55e525ce180bfcb42addd59c6633
Component: engine
Currently the plugins pkg allows a single handler. This assumption
breaks down if there are mutiple listeners to a plugin of a certain
Manifest such as NetworkDriver or IpamDriver when swarm-mode is enabled.
Signed-off-by: Madhu Venugopal <madhu@docker.com>
Upstream-commit: 5e9c78aeaf1d88000921190b88a1d91d6261208c
Component: engine
As part of making graphdrivers support pluginv2, a PluginGetter
interface was necessary for cleaner separation and avoiding import
cycles.
This commit creates a PluginGetter interface and makes pluginStore
implement it. Then the pluginStore object is created in the daemon
(rather than by the plugin manager) and passed to plugin init as
well as to the different subsystems (eg. graphdrivers, volumedrivers).
A side effect of this change was that some code was moved out of
experimental. This is good, since plugin support will be stable soon.
Signed-off-by: Anusha Ragunathan <anusha@docker.com>
Upstream-commit: fefea805e930a67fb6327f8e59415932861358cb
Component: engine
Plugin discovery on Windows is not possible using named pipes. However,
it is possible using spec file (tcp based). This adds Windows specific
paths for discovery.
Fixes#23605
Signed-off-by: Anusha Ragunathan <anusha@docker.com>
Upstream-commit: 36cf93fb0c45ce62b3cb1e82e1ecae1486017c9b
Component: engine
Legacy plugin model maintained a map of plugins. This is
not used by the new model. Using this map in the new model
causes incorrect lookup of plugins. This change uses adds
a plugin to the map only if its legacy.
Signed-off-by: Anusha Ragunathan <anusha@docker.com>
Upstream-commit: 8fd779dc28a11d8727d76e9553379b0c854f7c4c
Component: engine
This patch introduces a new experimental engine-level plugin management
with a new API and command line. Plugins can be distributed via a Docker
registry, and their lifecycle is managed by the engine.
This makes plugins a first-class construct.
For more background, have a look at issue #20363.
Documentation is in a separate commit. If you want to understand how the
new plugin system works, you can start by reading the documentation.
Note: backwards compatibility with existing plugins is maintained,
albeit they won't benefit from the advantages of the new system.
Signed-off-by: Tibor Vass <tibor@docker.com>
Signed-off-by: Anusha Ragunathan <anusha@docker.com>
Upstream-commit: f37117045c5398fd3dca8016ea8ca0cb47e7312b
Component: engine
This is similar to network scopes where a volume can either be `local`
or `global`. A `global` volume is one that exists across the entire
cluster where as a `local` volume exists on a single engine.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 2f40b1b281a3be8f34d82a5170988ee46ea1f442
Component: engine
Now handles `package.Type` and `*package.Type`
Fixes parsing issues with slice and map types.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 79ff6eaf21dfebad0f8131a1ede235249cd6638f
Component: engine
This generates an ID string for calls to Mount/Unmount, allowing drivers
to differentiate between two callers of `Mount` and `Unmount`.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 2b6bc294fc7f9e08a9091833b021b7d2a01ad2a6
Component: engine
When a plugin is first found, it is loaded into the available plugins
even though it's not activated yet.
If activation fails it is taken out of the list.
While it is in the list, other callers may see it and try to check it's
manifest. If it is not fully activated yet, the manifest will be nil and
cause a panic.
This is especially problematic for drivers that are down and have not
been activated yet.
We could just not load the plugin into the available list until it's
fully active, however that will just cause multiple of the same plugin
to attemp to be loaded.
We could check if the manifest is nil and return early (instead of
panicing on a nil manifest), but this will cause a 2nd caller to receive
a response while the first caller is still waiting, which can be
awkward.
This change uses a condition variable to handle activation (instead of
sync.Once). If the plugin is not activated, callers will all wait until
it is activated and receive a broadcast from the condition variable
signaling that it's ok to proceed, in which case we'll check if their
was an error in activation and proceed accordingly.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: baac2f48674d42863f9fdca5ef3056b4c6ed707e
Component: engine
Small refactor to be able to use custom transports
to call remote plugins.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Upstream-commit: 1a630234508bdb12d55425ceebdb0b6523a38578
Component: engine
Most of them were found and fixed by codespell.
Signed-off-by: Stefan Weil <sw@weilnetz.de>
Upstream-commit: 2eee613326fb59fd168849618d14a9054a40f9f5
Component: engine
Use a back-compat struct to handle listing volumes for volumes we know
about (because, presumably, they are being used by a container) for
volume drivers which don't yet support `List`.
Adds a fall-back for the volume driver `Get` call, which will use
`Create` when the driver returns a `404` for `Get`. The old behavior was
to always use `Create` to get a volume reference.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: f6c20d9b22ec9913f67b6c2ebdb5ef07c87b8cd7
Component: engine
We don't want to error out when there is a json unmarshal error since
the `old way` will cause this to error.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 279dd092b6e9d5f6bb405440fa8d407349f5ad84
Component: engine
Makes `docker volume ls` and `docker volume inspect` ask the volume
drivers rather than only using what is cached locally.
Previously in order to use a volume from an external driver, one would
either have to use `docker volume create` or have a container that is
already using that volume for it to be visible to the other volume
API's.
For keeping uniqueness of volume names in the daemon, names are bound to
a driver on a first come first serve basis. If two drivers have a volume
with the same name, the first one is chosen, and a warning is logged
about the second one.
Adds 2 new methods to the plugin API, `List` and `Get`.
If a plugin does not implement these endpoints, a user will not be able
to find the specified volumes as well requests go through the drivers.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: d3eca4451d264aac564594fe46b8c097bd85a5cc
Component: engine
- Use the ones provided by docker/go-connections, they are a drop in replacement.
- Remove pkg/sockets from docker.
- Keep pkg/tlsconfig because libnetwork still needs it and there is a
circular dependency issue.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Upstream-commit: 8e034802b7ad92a29f08785e553415adcd1348a3
Component: engine
This patches avoids registering (and calling) the same plugin more than
once. Using an helper map which indexes by name guarantees this and keeps
the order.
The behavior of overriding the same name in a flag is consistent with,
for instance, the `docker run -v /test -v /test` flag which register
the volume just once.
Adds integration tests.
Without this patch:
```
Dec 20 19:34:52 localhost.localdomain docker[9988]:
time="2015-12-20T19:34:52.080901676+01:00" level=debug msg="Calling
GET
/v1.22/info"
Dec 20 19:34:52 localhost.localdomain docker[9988]:
time="2015-12-20T19:34:52.081213202+01:00" level=debug msg="AuthZ
request using plugin docker-novolume-plugin"
Dec 20 19:34:52 localhost.localdomain docker[9988]:
time="2015-12-20T19:34:52.081268132+01:00" level=debug
msg="docker-novolume-plugin implements: authz"
Dec 20 19:34:52 localhost.localdomain docker[9988]:
time="2015-12-20T19:34:52.081699788+01:00" level=debug msg="AuthZ
request using plugin docker-novolume-plugin"
Dec 20 19:34:52 localhost.localdomain docker[9988]:
time="2015-12-20T19:34:52.081762507+01:00" level=debug
msg="docker-novolume-plugin implements: authz"
Dec 20 19:34:52 localhost.localdomain docker[9988]:
time="2015-12-20T19:34:52.082092480+01:00" level=debug msg="GET
/v1.22/info"
Dec 20 19:34:52 localhost.localdomain docker[9988]:
time="2015-12-20T19:34:52.628691038+01:00" level=debug msg="AuthZ
response using plugin docker-novolume-plugin"
Dec 20 19:34:52 localhost.localdomain docker[9988]:
time="2015-12-20T19:34:52.629880930+01:00" level=debug msg="AuthZ
response using plugin docker-novolume-plugin"
```
With this patch:
```
Dec 20 19:37:32 localhost.localdomain docker[16620]:
time="2015-12-20T19:37:32.376523958+01:00" level=debug msg="Calling
GET
/v1.22/info"
Dec 20 19:37:32 localhost.localdomain docker[16620]:
time="2015-12-20T19:37:32.376715483+01:00" level=debug msg="AuthZ
request using plugin docker-novolume-plugin"
Dec 20 19:37:32 localhost.localdomain docker[16620]:
time="2015-12-20T19:37:32.376771230+01:00" level=debug
msg="docker-novolume-plugin implements: authz"
Dec 20 19:37:32 localhost.localdomain docker[16620]:
time="2015-12-20T19:37:32.377698897+01:00" level=debug msg="GET
/v1.22/info"
Dec 20 19:37:32 localhost.localdomain docker[16620]:
time="2015-12-20T19:37:32.951016441+01:00" level=debug msg="AuthZ
response using plugin docker-novolume-plugin"
```
Also removes a somehow duplicate debug statement (leaving only the
second one as it's a loop of plugin's manifest):
```
Dec 20 19:52:30 localhost.localdomain docker[25767]:
time="2015-12-20T19:52:30.544090518+01:00" level=debug
msg="docker-novolume-plugin's manifest: &{[authz]}"
Dec 20 19:52:30 localhost.localdomain docker[25767]:
time="2015-12-20T19:52:30.544170677+01:00" level=debug
msg="docker-novolume-plugin implements: authz"
```
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
Upstream-commit: c28fc06e002e06deed3437da76bc213b7bd752ba
Component: engine
- fix naming and formatting
- provide more context when erroring auth
- do not capitalize errors
- fix wrong documentation
- remove ugly remoteError{}
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
Upstream-commit: 5a64c8027ecd2bebe773c9abd2e73c2fd7d23de1
Component: engine
When user call the `Call()` method, they don't always want to sent
some args or get the return value, so they use `nil` when call `Call()`
method and this will casue an error. It's better to not trying to
encode or decode if it's nil.
Signed-off-by: Lei Jitang <leijitang@huawei.com>
Upstream-commit: a2d348968fe754df41d0a1b56c7ba29ebbaa6849
Component: engine
Adds support for the daemon to handle user namespace maps as a
per-daemon setting.
Support for handling uid/gid mapping is added to the builder,
archive/unarchive packages and functions, all graphdrivers (except
Windows), and the test suite is updated to handle user namespace daemon
rootgraph changes.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
Upstream-commit: 442b45628ee12ebd8e8bd08497896d5fa8eec4bd
Component: engine
