This patch is required for the updated version of libnetwork and entails
two minor changes.
First, it uses the new libnetwork.NetworkDeleteOptionRemoveLB option to
the network.Delete() method to automatically remove the load balancing
endpoint for ingress networks. This allows removal of the
deleteLoadBalancerSandbox() function whose functionality is now within
libnetwork.
The second change is to allocate a load balancer endpoint IP address for
all overlay networks rather than just "ingress" and windows overlay
networks. Swarmkit is already performing this allocation, but moby was
not making use of these IP addresses for Linux overlay networks (except
ingress). The current version of libnetwork makes use of these IP
addresses by creating a load balancing sandbox and endpoint similar to
ingress's for all overlay network and putting all load balancing state
for a given node in that sandbox only. This reduces the amount of linux
kernel state required per node.
In the prior scheme, libnetwork would program each container's network
namespace with every piece of load balancing state for every other
container that shared *any* network with the first container. This
meant that the amount of kernel state on a given node scaled with the
square of the number of services in the cluster and with the square of
the number of containers per service. With the new scheme, kernel state
at each node scales linearly with the number of services and the number
of containers per service. This also reduces the number of system calls
required to add or remove tasks and containers. Previously the number
of system calls required grew linearly with the number of other
tasks that shared a network with the container. Now the number of
system calls grows linearly only with the number of networks that the
task/container is attached to. This results in a significant
performance improvement when adding and removing services to a cluster
that already heavily loaded.
The primary disadvantage to this scheme is that it requires the
allocation of an additional IP address per node per subnet for every
node in the cluster that has a task on the given subnet. However, as
mentioned, swarmkit is already allocating these IP addresses for every
node and they are going unused. Future swarmkit modifications should be
examined to only allocate said IP addresses when nodes actually require
them.
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: 8e0f6bc90351525f3e52f3bc357378fcccccdd27
Component: engine
This stuff doesn't belong here and is causing imports of libnetwork into
the router, which is not what we want.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: c0bc14e8dd5a31f5edc804e9a1347bb3eb44483e
Component: engine
These network operations really don't have anything to do with the
container but rather are setting up the networking.
Ideally these wouldn't get shoved into the daemon package, but doing
something else (e.g. extract a network service into a new package) but
there's a lot more work to do in that regard.
In reality, this probably simplifies some of that work as it moves all
the network operations to the same place.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: cc8f358c23d307d19b06cd5e7d039d8fad01a947
Component: engine
Since Go 1.7, context is a standard package. Since Go 1.9, everything
that is provided by "x/net/context" is a couple of type aliases to
types in "context".
Many vendored packages still use x/net/context, so vendor entry remains
for now.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 7d62e40f7e4f3c17d229a7687d6fcca5448de813
Component: engine
Ingress networks will no longer automatically remove their
load-balancing endpoint (and sandbox) automatically when the network is
otherwise upopulated. This is to prevent automatic removal of the
ingress networks when all the containers leave them. Therefore
explicit removal of an ingress network also requires explicit removal
of its load-balancing endpoint.
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Upstream-commit: 3da4ebf355d3494d1403b2878a1ae6958b2724e9
Component: engine
With the inclusion of PR 30897, creating service for host network
fails in 18.02. Modified IsPreDefinedNetwork check and return
NetworkNameError instead of errdefs.Forbidden to address this issue
Signed-off-by: selansen <elango.siva@docker.com>
Upstream-commit: 7cf8b20762cc9491f52ff3f3d94c880378183696
Component: engine
This fix is a follow up to 30397, with `FindUniqueNetwork`
changed to `FindNetwork` based on the review feedback.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: ccc2ed01894a1950eaf47db2ad0860ad87cd78d1
Component: engine
Instead of having to create a bunch of custom error types that are doing
nothing but wrapping another error in sub-packages, use a common helper
to create errors of the requested type.
e.g. instead of re-implementing this over and over:
```go
type notFoundError struct {
cause error
}
func(e notFoundError) Error() string {
return e.cause.Error()
}
func(e notFoundError) NotFound() {}
func(e notFoundError) Cause() error {
return e.cause
}
```
Packages can instead just do:
```
errdefs.NotFound(err)
```
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 87a12421a94faac294079bebc97c8abb4180dde5
Component: engine
This fix is part of the effort to address 30242 where
issue arise because of the fact that multiple networks
may share the same name (within or across local/swarm scopes).
The focus of this fix is to allow creation of service
when a network in local scope has the same name as the
service network.
An integration test has been added.
This fix fixes 30242.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: cafed80cd019a8b40025eaa5e5b37459362607fb
Component: engine
The error type libnetwork.ErrNoSuchNetwork is used in the controller
to retry the network creation as a managed network though the manager.
The change of the type was breaking the logic causing the network to
not being created anymore so that no new container on that network
was able to be launched
Added unit test
Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
Upstream-commit: 51cea0a53c2fd36832277402e9faac81bfb4abd4
Component: engine
For obvious reasons that it is not really supported now.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Upstream-commit: 5a9b5f10cf967f31f0856871ad08f9a0286b4a46
Component: engine
If a network is created with a name that matches another
network's ID, the network with that name was masking the
other network's ID.
As a result, it was not possible to remove the network
with a given ID.
This patch changes the order in which networks are
matched to be what we use for other cases;
1. Match on full ID
2. Match on full Name
3. Match on Partial ID
Before this patch:
$ docker network create foo
336717eac9eaa3da6557042a04efc803f7e8862ce6cf96f6b9565265ba5c618b
$ docker network create 336717eac9eaa3da6557042a04efc803f7e8862ce6cf96f6b9565265ba5c618b
4a698333f1197f20224583abce14876d7f25fdfe416a8545927006c315915a2a
$ docker network ls
NETWORK ID NAME DRIVER SCOPE
4a698333f119 336717eac9eaa3da6557042a04efc803f7e8862ce6cf96f6b9565265ba5c618b bridge local
d1e40d43a2c0 bridge bridge local
336717eac9ea foo bridge local
13cf280a1bbf host host local
d9e4c03728a0 none null local
$ docker network rm 336717eac9eaa3da6557042a04efc803f7e8862ce6cf96f6b9565265ba5c618b
4a698333f1197f20224583abce14876d7f25fdfe416a8545927006c315915a2a
$ docker network ls
NETWORK ID NAME DRIVER SCOPE
d1e40d43a2c0 bridge bridge local
336717eac9ea foo bridge local
13cf280a1bbf host host local
d9e4c03728a0 none null local
After this patch:
$ docker network create foo
2d1791a7def4e2a1ef0f6b83c6add333df0bb4ced2f196c584cb64e6bd94b835
$ docker network create 2d1791a7def4e2a1ef0f6b83c6add333df0bb4ced2f196c584cb64e6bd94b835
6cbc749a529cd2d9d3b10566c84e56c4203dd88b67417437b5fc7a6e955dd48f
$ docker network ls
NETWORK ID NAME DRIVER SCOPE
6cbc749a529c 2d1791a7def4e2a1ef0f6b83c6add333df0bb4ced2f196c584cb64e6bd94b835 bridge local
166c943dbeb5 bridge bridge local
2d1791a7def4 foo bridge local
6c45b8aa6d8e host host local
b11c96b51ea7 none null local
$ docker network rm 2d1791a7def4e2a1ef0f6b83c6add333df0bb4ced2f196c584cb64e6bd94b835
2d1791a7def4e2a1ef0f6b83c6add333df0bb4ced2f196c584cb64e6bd94b835
$ docker network ls
NETWORK ID NAME DRIVER SCOPE
6cbc749a529c 2d1791a7def4e2a1ef0f6b83c6add333df0bb4ced2f196c584cb64e6bd94b835 bridge local
166c943dbeb5 bridge bridge local
6c45b8aa6d8e host host local
b11c96b51ea7 none null local
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: e52001c56e12e4fc63fb5d89ef919295d6ddd5d5
Component: engine
Use strongly typed errors to set HTTP status codes.
Error interfaces are defined in the api/errors package and errors
returned from controllers are checked against these interfaces.
Errors can be wraeped in a pkg/errors.Causer, as long as somewhere in the
line of causes one of the interfaces is implemented. The special error
interfaces take precedence over Causer, meaning if both Causer and one
of the new error interfaces are implemented, the Causer is not
traversed.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: ebcb7d6b406fe50ea9a237c73004d75884184c33
Component: engine
- User can now promote a network's scope to swarm,
so that the network is visible to swarm
Signed-off-by: Alessandro Boch <aboch@docker.com>
Upstream-commit: fcafc7108b7b7f2ca9c2982b29b47de424f2da52
Component: engine
- They are configuration-only networks which
can be used to supply the configuration
when creating regular networks.
- They do not get allocated and do net get plumbed.
Drivers do not get to know about them.
- They can be removed, once no other network is
using them.
- When user creates a network specifying a
configuration network for the config, no
other network specific configuration field
is are accepted. User can only specify
network operator fields (attachable, internal,...)
Signed-off-by: Alessandro Boch <aboch@docker.com>
Upstream-commit: 9ee7b4dda926a1444dc0ea50c4ca6d90c8684060
Component: engine
This commit in conjunction with a libnetwork side commit,
cleans up the libnetwork SetClusterProvider logic interaction.
The previous code was inducing libnetwork to spawn several go
routines that were racing between each other during the agent
init and close.
A test got added to verify that back to back swarm init and leave
are properly processed and not raise crashes
Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
Upstream-commit: e2ec006797fa14f59bcf7b9c23505ccdf1d3ded3
Component: engine
before: Error response from daemon: datastore for scope "global" is not initialized
after: Error response from daemon: This node is not a swarm manager. Use "docker swarm init" or "docker swarm join" to connect this node to swarm and try again.
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Upstream-commit: 9c77a4c2973a62beb5e6cc57b1970e97edca97d8
Component: engine
- libnetwork controller Networks() already returns
a copy list. Also Networks() correctly skips any
network which ahs already been marked for deletion
while getNetworks implementation bypass this.
Signed-off-by: Alessandro Boch <aboch@docker.com>
Upstream-commit: 5d71cc01b6bb089a70fa1e855943dab0d88439bb
Component: engine
- When the node leaves the cluster, if any user run
container(s) is connected to the swarm network,
then daemon needs to detach the container(s) and
remove the network.
Signed-off-by: Alessandro Boch <aboch@docker.com>
Upstream-commit: 3cedca5d532958ffc007d9b62cc871d3d113f054
Component: engine
returned in "docker info". Currently info endpoint isnt using the
GetAllByCap, but relies on existing networks to get the plugin names.
This causes a basic issue when it comes to global network plugins which
swarm-mode relies on, wherein swarmkit will not be able to schedule the
network on the worker nodes due to the filtering logic.
In order to break this chicken & egg issue, we must start to use the
GetAllManagedPluginsByCap. We are unable to use GetAllByCap due to
various issues with Plugin-V1's lazy loading approach causing issues
especially during daemon restarts (which uses SystemInfo)
Signed-off-by: Madhu Venugopal <madhu@docker.com>
Upstream-commit: 6aaa85f0b0dd030f68512ca9004823926419f5fd
Component: engine
Go style calls for mixed caps instead of all caps:
https://golang.org/doc/effective_go.html#mixed-caps
Change LOOKUP, ACQUIRE, and RELEASE to Lookup, Acquire, and Release.
This vendors a fork of libnetwork for now, to deal with a cyclic
dependency issue. The change will be upstream to libnetwork once this is
merged.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Upstream-commit: 428600108cce0a11e65ec4ebd9e439e947b55da7
Component: engine
This adds a metrics packages that creates additional metrics. Add the
metrics endpoint to the docker api server under `/metrics`.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Add metrics to daemon package
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
api: use standard way for metrics route
Also add "type" query parameter
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
Convert timers to ms
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Upstream-commit: 3343d234f3b131d4be1d4ca84385e184633a79bd
Component: engine
This fix tries to address the issue raised in 27695 where
duplicate `nat` and `null` has been listed in `docker info`
for Windows.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 3347aba95762955fe17916e8cfc66aabbd6d66d6
Component: engine
Currently the plugins pkg allows a single handler. This assumption
breaks down if there are mutiple listeners to a plugin of a certain
Manifest such as NetworkDriver or IpamDriver when swarm-mode is enabled.
Signed-off-by: Madhu Venugopal <madhu@docker.com>
Upstream-commit: 5e9c78aeaf1d88000921190b88a1d91d6261208c
Component: engine
When calling the /networks/ endpoint with a trailing
slash, the default network was returned.
This changes the endpoint to return the list of networks
instead (same response as `/networks` without trailing
slash).
Also updated the description for GetNetworkByName to
explain that the "default" network is returned if
no name or id is provided.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 6ad4bf0a533f8851b0ddccb56fe3b457819f6146
Component: engine
