diff --git a/components/engine/.github/ISSUE_TEMPLATE.md b/components/engine/.github/ISSUE_TEMPLATE.md
new file mode 100644
index 0000000000..c347950212
--- /dev/null
+++ b/components/engine/.github/ISSUE_TEMPLATE.md
@@ -0,0 +1,51 @@
+
+
+Output of `docker version`:
+
+```
+(paste your output here)
+```
+
+
+Output of `docker info`:
+
+```
+(paste your output here)
+```
+
+Provide additional environment details (AWS, VirtualBox, physical, etc.):
+
+
+
+List the steps to reproduce the issue:
+1.
+2.
+3.
+
+
+Describe the results you received:
+
+
+Describe the results you expected:
+
+
+Provide additional info you think is important:
diff --git a/components/engine/.github/PULL_REQUEST_TEMPLATE.md b/components/engine/.github/PULL_REQUEST_TEMPLATE.md
new file mode 100644
index 0000000000..fee0c7864c
--- /dev/null
+++ b/components/engine/.github/PULL_REQUEST_TEMPLATE.md
@@ -0,0 +1,23 @@
+
+
+Please provide the following information:
+
+- What did you do?
+
+- How did you do it?
+
+- How do I see it or verify it?
+
+- A picture of a cute animal (not mandatory but encouraged)
+
diff --git a/components/engine/CHANGELOG.md b/components/engine/CHANGELOG.md
index 2d19c2094a..c27e96bf36 100644
--- a/components/engine/CHANGELOG.md
+++ b/components/engine/CHANGELOG.md
@@ -5,6 +5,73 @@ information on the list of deprecated flags and APIs please have a look at
https://docs.docker.com/misc/deprecated/ where target removal dates can also
be found.
+## 1.10.2 (2016-02-22)
+
+### Runtime
+
+- Prevent systemd from deleting containers' cgroups when its configuration is reloaded [#20518](https://github.com/docker/docker/pull/20518)
+- Fix SELinux issues by disregarding `--read-only` when mounting `/dev/mqueue` [#20333](https://github.com/docker/docker/pull/20333)
+- Fix chown permissions used during `docker cp` when userns is used [#20446](https://github.com/docker/docker/pull/20446)
+- Fix configuration loading issue with all booleans defaulting to `true` [#20471](https://github.com/docker/docker/pull/20471)
+- Fix occasional panic with `docker logs -f` [#20522](https://github.com/docker/docker/pull/20522)
+
+### Distribution
+
+- Keep layer reference if deletion failed to avoid a badly inconsistent state [#20513](https://github.com/docker/docker/pull/20513)
+- Handle gracefully a corner case when canceling migration [#20372](https://github.com/docker/docker/pull/20372)
+- Fix docker import on compressed data [#20367](https://github.com/docker/docker/pull/20367)
+- Fix tar-split files corruption during migration that later cause docker push and docker save to fail [#20458](https://github.com/docker/docker/pull/20458)
+
+### Networking
+
+- Fix daemon crash if embedded DNS is sent garbage [#20510](https://github.com/docker/docker/pull/20510)
+
+### Volumes
+
+- Fix issue with multiple volume references with same name [#20381](https://github.com/docker/docker/pull/20381)
+
+### Security
+
+- Fix potential cache corruption and delegation conflict issues [#20523](https://github.com/docker/docker/pull/20523)
+
+## 1.10.1 (2016-02-11)
+
+### Runtime
+
+* Do not stop daemon on migration hard failure [#20156](https://github.com/docker/docker/pull/20156)
+- Fix various issues with migration to content-addressable images [#20058](https://github.com/docker/docker/pull/20058)
+- Fix ZFS permission bug with user namespaces [#20045](https://github.com/docker/docker/pull/20045)
+- Do not leak /dev/mqueue from the host to all containers, keep it container-specific [#19876](https://github.com/docker/docker/pull/19876) [#20133](https://github.com/docker/docker/pull/20133)
+- Fix `docker ps --filter before=...` to not show stopped containers without providing `-a` flag [#20135](https://github.com/docker/docker/pull/20135)
+
+### Security
+
+- Fix issue preventing docker events to work properly with authorization plugin [#20002](https://github.com/docker/docker/pull/20002)
+
+### Distribution
+
+* Add additional verifications and prevent from uploading invalid data to registries [#20164](https://github.com/docker/docker/pull/20164)
+- Fix regression preventing uppercase characters in image reference hostname [#20175](https://github.com/docker/docker/pull/20175)
+
+### Networking
+
+- Fix embedded DNS for user-defined networks in the presence of firewalld [#20060](https://github.com/docker/docker/pull/20060)
+- Fix issue where removing a network during shutdown left Docker inoperable [#20181](https://github.com/docker/docker/issues/20181) [#20235](https://github.com/docker/docker/issues/20235)
+- Embedded DNS is now able to return compressed results [#20181](https://github.com/docker/docker/issues/20181)
+- Fix port-mapping issue with `userland-proxy=false` [#20181](https://github.com/docker/docker/issues/20181)
+
+### Logging
+
+- Fix bug where tcp+tls protocol would be rejected [#20109](https://github.com/docker/docker/pull/20109)
+
+### Volumes
+
+- Fix issue whereby older volume drivers would not receive volume options [#19983](https://github.com/docker/docker/pull/19983)
+
+### Misc
+
+- Remove TasksMax from Docker systemd service [#20167](https://github.com/docker/docker/pull/20167)
+
## 1.10.0 (2016-02-04)
**IMPORTANT**: Docker 1.10 uses a new content-addressable storage for images and layers.
@@ -1771,7 +1838,7 @@ With the ongoing changes to the networking and execution subsystems of docker te
+ Containers can expose public UDP ports (eg, '-p 123/udp')
+ Optionally specify an exact public port (eg. '-p 80:4500')
* 'docker login' supports additional options
-- Dont save a container`s hostname when committing an image.
+- Don't save a container`s hostname when committing an image.
#### Registry
diff --git a/components/engine/CONTRIBUTING.md b/components/engine/CONTRIBUTING.md
index e499e1a9d6..6b875d6901 100644
--- a/components/engine/CONTRIBUTING.md
+++ b/components/engine/CONTRIBUTING.md
@@ -154,6 +154,8 @@ However, there might be a way to implement that feature *on top of* Docker.
The docker-dev
group is for contributors and other people contributing to the Docker
project.
+ You can join them without an google account by sending an email to e.g. "docker-user+subscribe@googlegroups.com".
+ After receiving the join-request message, you can simply reply to that to confirm the subscribtion.
diff --git a/components/engine/Dockerfile b/components/engine/Dockerfile
index 45486db4ec..a3430d58bc 100644
--- a/components/engine/Dockerfile
+++ b/components/engine/Dockerfile
@@ -23,14 +23,16 @@
# the case. Therefore, you don't have to disable it anymore.
#
-FROM ubuntu:trusty
+FROM debian:jessie
# add zfs ppa
-RUN apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys E871F18B51E0147C77796AC81196BA81F6B0FC61
+RUN apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys E871F18B51E0147C77796AC81196BA81F6B0FC61 \
+ || apt-key adv --keyserver hkp://pgp.mit.edu:80 --recv-keys E871F18B51E0147C77796AC81196BA81F6B0FC61
RUN echo deb http://ppa.launchpad.net/zfs-native/stable/ubuntu trusty main > /etc/apt/sources.list.d/zfs.list
# add llvm repo
-RUN apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 6084F3CF814B57C1CF12EFD515CF4D18AF4F7421
+RUN apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 6084F3CF814B57C1CF12EFD515CF4D18AF4F7421 \
+ || apt-key adv --keyserver hkp://pgp.mit.edu:80 --recv-keys 6084F3CF814B57C1CF12EFD515CF4D18AF4F7421
RUN echo deb http://llvm.org/apt/trusty/ llvm-toolchain-trusty main > /etc/apt/sources.list.d/llvm.list
# Packaged dependencies
@@ -56,12 +58,13 @@ RUN apt-get update && apt-get install -y \
libsystemd-journal-dev \
libtool \
mercurial \
+ net-tools \
pkg-config \
python-dev \
python-mock \
python-pip \
python-websocket \
- s3cmd=1.1.0* \
+ s3cmd=1.5.0* \
ubuntu-zfs \
xfsprogs \
libzfs-dev \
@@ -88,9 +91,11 @@ RUN cd /usr/local/lvm2 \
# Configure the container for OSX cross compilation
ENV OSX_SDK MacOSX10.11.sdk
+ENV OSX_CROSS_COMMIT 8aa9b71a394905e6c5f4b59e2b97b87a004658a4
RUN set -x \
&& export OSXCROSS_PATH="/osxcross" \
- && git clone --depth 1 https://github.com/tpoechtrager/osxcross.git $OSXCROSS_PATH \
+ && git clone https://github.com/tpoechtrager/osxcross.git $OSXCROSS_PATH \
+ && ( cd $OSXCROSS_PATH && git checkout -q $OSX_CROSS_COMMIT) \
&& curl -sSL https://s3.dockerproject.org/darwin/${OSX_SDK}.tar.xz -o "${OSXCROSS_PATH}/tarballs/${OSX_SDK}.tar.xz" \
&& UNATTENDED=yes OSX_VERSION_MIN=10.6 ${OSXCROSS_PATH}/build.sh
ENV PATH /osxcross/target/bin:$PATH
@@ -114,7 +119,7 @@ RUN set -x \
# IMPORTANT: If the version of Go is updated, the Windows to Linux CI machines
# will need updating, to avoid errors. Ping #docker-maintainers on IRC
# with a heads-up.
-ENV GO_VERSION 1.5.3
+ENV GO_VERSION 1.6
RUN curl -fsSL "https://storage.googleapis.com/golang/go${GO_VERSION}.linux-amd64.tar.gz" \
| tar -xzC /usr/local
ENV PATH /go/bin:/usr/local/go/bin:$PATH
@@ -166,7 +171,7 @@ RUN set -x \
&& rm -rf "$GOPATH"
# Install notary server
-ENV NOTARY_VERSION docker-v1.10-5
+ENV NOTARY_VERSION v0.2.0
RUN set -x \
&& export GOPATH="$(mktemp -d)" \
&& git clone https://github.com/docker/notary.git "$GOPATH/src/github.com/docker/notary" \
diff --git a/components/engine/Dockerfile.aarch64 b/components/engine/Dockerfile.aarch64
index ee8c889e77..fa8fde9ac7 100644
--- a/components/engine/Dockerfile.aarch64
+++ b/components/engine/Dockerfile.aarch64
@@ -11,19 +11,11 @@
# # Run the test suite:
# docker run --privileged docker hack/make.sh test
#
-# # Publish a release:
-# docker run --privileged \
-# -e AWS_S3_BUCKET=baz \
-# -e AWS_ACCESS_KEY=foo \
-# -e AWS_SECRET_KEY=bar \
-# -e GPG_PASSPHRASE=gloubiboulga \
-# docker hack/release.sh
-#
# Note: AppArmor used to mess with privileged mode, but this is no longer
# the case. Therefore, you don't have to disable it anymore.
#
-FROM aarch64/ubuntu:trusty
+FROM aarch64/ubuntu:wily
# Packaged dependencies
RUN apt-get update && apt-get install -y \
@@ -45,15 +37,16 @@ RUN apt-get update && apt-get install -y \
libc6-dev \
libcap-dev \
libsqlite3-dev \
- libsystemd-journal-dev \
+ libsystemd-dev \
mercurial \
+ net-tools \
parallel \
pkg-config \
python-dev \
python-mock \
python-pip \
python-websocket \
- s3cmd=1.1.0* \
+ gccgo \
--no-install-recommends
# Install armhf loader to use armv6 binaries on armv8
@@ -103,14 +96,11 @@ RUN set -x \
# We don't have official binary tarballs for ARM64, eigher for Go or bootstrap,
# so we use the official armv6 released binaries as a GOROOT_BOOTSTRAP, and
# build Go from source code.
-ENV BOOT_STRAP_VERSION 1.6beta1
ENV GO_VERSION 1.5.3
-RUN mkdir -p /usr/src/go-bootstrap \
- && curl -fsSL https://storage.googleapis.com/golang/go${BOOT_STRAP_VERSION}.linux-arm6.tar.gz | tar -v -C /usr/src/go-bootstrap -xz --strip-components=1 \
- && mkdir /usr/src/go \
- && curl -fsSL https://storage.googleapis.com/golang/go${GO_VERSION}.src.tar.gz | tar -v -C /usr/src/go -xz --strip-components=1 \
+RUN mkdir /usr/src/go && curl -fsSL https://storage.googleapis.com/golang/go${GO_VERSION}.src.tar.gz | tar -v -C /usr/src/go -xz --strip-components=1 \
&& cd /usr/src/go/src \
- && GOOS=linux GOARCH=arm64 GOROOT_BOOTSTRAP=/usr/src/go-bootstrap ./make.bash
+ && GOOS=linux GOARCH=arm64 GOROOT_BOOTSTRAP="$(go env GOROOT)" ./make.bash
+
ENV PATH /usr/src/go/bin:$PATH
ENV GOPATH /go:/go/src/github.com/docker/docker/vendor
@@ -127,7 +117,7 @@ RUN set -x \
&& rm -rf "$GOPATH"
# Install notary server
-ENV NOTARY_VERSION docker-v1.10-5
+ENV NOTARY_VERSION v0.2.0
RUN set -x \
&& export GOPATH="$(mktemp -d)" \
&& git clone https://github.com/docker/notary.git "$GOPATH/src/github.com/docker/notary" \
diff --git a/components/engine/Dockerfile.armhf b/components/engine/Dockerfile.armhf
index 8b48cf9261..fd6f8721fa 100644
--- a/components/engine/Dockerfile.armhf
+++ b/components/engine/Dockerfile.armhf
@@ -11,19 +11,11 @@
# # Run the test suite:
# docker run --privileged docker hack/make.sh test
#
-# # Publish a release:
-# docker run --privileged \
-# -e AWS_S3_BUCKET=baz \
-# -e AWS_ACCESS_KEY=foo \
-# -e AWS_SECRET_KEY=bar \
-# -e GPG_PASSPHRASE=gloubiboulga \
-# docker hack/release.sh
-#
# Note: AppArmor used to mess with privileged mode, but this is no longer
# the case. Therefore, you don't have to disable it anymore.
#
-FROM armhf/ubuntu:trusty
+FROM armhf/debian:jessie
# Packaged dependencies
RUN apt-get update && apt-get install -y \
@@ -143,7 +135,7 @@ RUN set -x \
&& rm -rf "$GOPATH"
# Install notary server
-ENV NOTARY_VERSION docker-v1.10-5
+ENV NOTARY_VERSION v0.2.0
RUN set -x \
&& export GOPATH="$(mktemp -d)" \
&& git clone https://github.com/docker/notary.git "$GOPATH/src/github.com/docker/notary" \
diff --git a/components/engine/Dockerfile.gccgo b/components/engine/Dockerfile.gccgo
index 96f92f6145..c01f5dd895 100644
--- a/components/engine/Dockerfile.gccgo
+++ b/components/engine/Dockerfile.gccgo
@@ -23,6 +23,7 @@ RUN apt-get update && apt-get install -y \
libcap-dev \
libsqlite3-dev \
mercurial \
+ net-tools \
parallel \
python-dev \
python-mock \
diff --git a/components/engine/Dockerfile.ppc64le b/components/engine/Dockerfile.ppc64le
index d4a7e6f1a6..fc1d929f48 100644
--- a/components/engine/Dockerfile.ppc64le
+++ b/components/engine/Dockerfile.ppc64le
@@ -11,14 +11,6 @@
# # Run the test suite:
# docker run --privileged docker hack/make.sh test
#
-# # Publish a release:
-# docker run --privileged \
-# -e AWS_S3_BUCKET=baz \
-# -e AWS_ACCESS_KEY=foo \
-# -e AWS_SECRET_KEY=bar \
-# -e GPG_PASSPHRASE=gloubiboulga \
-# docker hack/release.sh
-#
# Note: AppArmor used to mess with privileged mode, but this is no longer
# the case. Therefore, you don't have to disable it anymore.
#
@@ -82,7 +74,21 @@ RUN cd /usr/local/lvm2 \
# TODO install Go, using gccgo as GOROOT_BOOTSTRAP (Go 1.5+ supports ppc64le properly)
# possibly a ppc64le/golang image?
-ENV PATH /go/bin:$PATH
+## BUILD GOLANG 1.6
+ENV GO_VERSION 1.6
+ENV GO_DOWNLOAD_URL https://golang.org/dl/go${GO_VERSION}.src.tar.gz
+ENV GO_DOWNLOAD_SHA256 a96cce8ce43a9bf9b2a4c7d470bc7ee0cb00410da815980681c8353218dcf146
+ENV GOROOT_BOOTSTRAP /usr/local
+
+RUN curl -fsSL "$GO_DOWNLOAD_URL" -o golang.tar.gz \
+ && echo "$GO_DOWNLOAD_SHA256 golang.tar.gz" | sha256sum -c - \
+ && tar -C /usr/src -xzf golang.tar.gz \
+ && rm golang.tar.gz \
+ && cd /usr/src/go/src && ./make.bash 2>&1
+
+ENV GOROOT_BOOTSTRAP /usr/src/
+
+ENV PATH /usr/src/go/bin/:/go/bin:$PATH
ENV GOPATH /go:/go/src/github.com/docker/docker/vendor
# This has been commented out and kept as reference because we don't support compiling with older Go anymore.
@@ -90,7 +96,7 @@ ENV GOPATH /go:/go/src/github.com/docker/docker/vendor
# RUN curl -sSL https://storage.googleapis.com/golang/go${GOFMT_VERSION}.$(go env GOOS)-$(go env GOARCH).tar.gz | tar -C /go/bin -xz --strip-components=2 go/bin/gofmt
# TODO update this sha when we upgrade to Go 1.5+
-ENV GO_TOOLS_COMMIT 069d2f3bcb68257b627205f0486d6cc69a231ff9
+ENV GO_TOOLS_COMMIT d02228d1857b9f49cd0252788516ff5584266eb6
# Grab Go's cover tool for dead-simple code coverage testing
# Grab Go's vet tool for examining go code to find suspicious constructs
# and help prevent errors that the compiler might not catch
@@ -99,7 +105,7 @@ RUN git clone https://github.com/golang/tools.git /go/src/golang.org/x/tools \
&& go install -v golang.org/x/tools/cmd/cover \
&& go install -v golang.org/x/tools/cmd/vet
# Grab Go's lint tool
-ENV GO_LINT_COMMIT f42f5c1c440621302702cb0741e9d2ca547ae80f
+ENV GO_LINT_COMMIT 32a87160691b3c96046c0c678fe57c5bef761456
RUN git clone https://github.com/golang/lint.git /go/src/github.com/golang/lint \
&& (cd /go/src/github.com/golang/lint && git checkout -q $GO_LINT_COMMIT) \
&& go install -v github.com/golang/lint/golint
@@ -121,16 +127,16 @@ RUN set -x \
go build -o /usr/local/bin/registry-v2-schema1 github.com/docker/distribution/cmd/registry \
&& rm -rf "$GOPATH"
-# TODO update this when we upgrade to Go 1.5.1+
+
# Install notary server
-#ENV NOTARY_VERSION docker-v1.10-5
-#RUN set -x \
-# && export GOPATH="$(mktemp -d)" \
-# && git clone https://github.com/docker/notary.git "$GOPATH/src/github.com/docker/notary" \
-# && (cd "$GOPATH/src/github.com/docker/notary" && git checkout -q "$NOTARY_VERSION") \
-# && GOPATH="$GOPATH/src/github.com/docker/notary/Godeps/_workspace:$GOPATH" \
-# go build -o /usr/local/bin/notary-server github.com/docker/notary/cmd/notary-server \
-# && rm -rf "$GOPATH"
+ENV NOTARY_VERSION v0.2.0
+RUN set -x \
+ && export GOPATH="$(mktemp -d)" \
+ && git clone https://github.com/docker/notary.git "$GOPATH/src/github.com/docker/notary" \
+ && (cd "$GOPATH/src/github.com/docker/notary" && git checkout -q "$NOTARY_VERSION") \
+ && GOPATH="$GOPATH/src/github.com/docker/notary/Godeps/_workspace:$GOPATH" \
+ go build -o /usr/local/bin/notary-server github.com/docker/notary/cmd/notary-server \
+ && rm -rf "$GOPATH"
# Get the "docker-py" source so we can run their integration tests
ENV DOCKER_PY_COMMIT e2878cbcc3a7eef99917adc1be252800b0e41ece
diff --git a/components/engine/Dockerfile.s390x b/components/engine/Dockerfile.s390x
index 90b7fb38aa..f46e1e0c9b 100644
--- a/components/engine/Dockerfile.s390x
+++ b/components/engine/Dockerfile.s390x
@@ -11,14 +11,6 @@
# # Run the test suite:
# docker run --privileged docker hack/make.sh test
#
-# # Publish a release:
-# docker run --privileged \
-# -e AWS_S3_BUCKET=baz \
-# -e AWS_ACCESS_KEY=foo \
-# -e AWS_SECRET_KEY=bar \
-# -e GPG_PASSPHRASE=gloubiboulga \
-# docker hack/release.sh
-#
# Note: AppArmor used to mess with privileged mode, but this is no longer
# the case. Therefore, you don't have to disable it anymore.
#
@@ -116,7 +108,7 @@ RUN set -x \
&& rm -rf "$GOPATH"
# Install notary server
-ENV NOTARY_VERSION docker-v1.10-5
+ENV NOTARY_VERSION v0.2.0
RUN set -x \
&& export GOPATH="$(mktemp -d)" \
&& git clone https://github.com/docker/notary.git "$GOPATH/src/github.com/docker/notary" \
diff --git a/components/engine/Dockerfile.windows b/components/engine/Dockerfile.windows
index 691bd6910f..78db5a3b34 100755
--- a/components/engine/Dockerfile.windows
+++ b/components/engine/Dockerfile.windows
@@ -19,14 +19,7 @@
# Important notes:
# ---------------
#
-# Multiple commands in a single powershell RUN command are deliberately not done. This is
-# because PS doesn't have a concept quite like set -e in bash. It would be possible to use
-# try-catch script blocks, but that would make this file unreadable. The problem is that
-# if there are two commands eg "RUN powershell -command fail; succeed", as far as docker
-# would be concerned, the return code from the overall RUN is succeed. This doesn't apply to
-# RUN which uses cmd as the command interpreter such as "RUN fail; succeed".
-#
-# 'sleep 5' is a deliberate workaround for a current problem on containers in Windows
+# 'Start-Sleep' is a deliberate workaround for a current problem on containers in Windows
# Server 2016. It ensures that the network is up and available for when the command is
# network related. This bug is being tracked internally at Microsoft and exists in TP4.
# Generally sleep 1 or 2 is probably enough, but making it 5 to make the build file
@@ -39,55 +32,70 @@
# Don't try to use a volume for passing the source through. The cygwin posix utilities will
# balk at reparse points. Again, see the example at the top of this file on how use a volume
# to get the built binary out of the container.
+#
+# The steps are minimised dramatically to improve performance (TP4 is slow on commit)
FROM windowsservercore
# Environment variable notes:
-# - GOLANG_VERSION should be updated to be consistent with the Linux dockerfile.
+# - GOLANG_VERSION must consistent with 'Dockerfile' used by Linux'.
# - FROM_DOCKERFILE is used for detection of building within a container.
-ENV GOLANG_VERSION=1.5.3 \
- GIT_VERSION=2.7.0 \
+ENV GOLANG_VERSION=1.6 \
+ GIT_LOCATION=https://github.com/git-for-windows/git/releases/download/v2.7.2.windows.1/Git-2.7.2-64-bit.exe \
RSRC_COMMIT=ba14da1f827188454a4591717fff29999010887f \
GOPATH=C:/go;C:/go/src/github.com/docker/docker/vendor \
FROM_DOCKERFILE=1
-# Make sure we're in temp for the downloads
-WORKDIR c:/windows/temp
-
-# Download everything else we need to install
-# We want a 64-bit make.exe, not 16 or 32-bit. This was hard to find, so documenting the links
-# - http://sourceforge.net/p/mingw-w64/wiki2/Make/ -->
-# - http://sourceforge.net/projects/mingw-w64/files/External%20binary%20packages%20%28Win64%20hosted%29/ -->
-# - http://sourceforge.net/projects/mingw-w64/files/External binary packages %28Win64 hosted%29/make/
-RUN powershell -command sleep 5; Invoke-WebRequest -UserAgent 'DockerCI' -outfile make.zip http://downloads.sourceforge.net/project/mingw-w64/External%20binary%20packages%20%28Win64%20hosted%29/make/make-3.82.90-20111115.zip
-RUN powershell -command sleep 5; Invoke-WebRequest -UserAgent 'DockerCI' -outfile gcc.zip http://downloads.sourceforge.net/project/tdm-gcc/TDM-GCC%205%20series/5.1.0-tdm64-1/gcc-5.1.0-tdm64-1-core.zip
-RUN powershell -command sleep 5; Invoke-WebRequest -UserAgent 'DockerCI' -outfile runtime.zip http://downloads.sourceforge.net/project/tdm-gcc/MinGW-w64%20runtime/GCC%205%20series/mingw64runtime-v4-git20150618-gcc5-tdm64-1.zip
-RUN powershell -command sleep 5; Invoke-WebRequest -UserAgent 'DockerCI' -outfile binutils.zip http://downloads.sourceforge.net/project/tdm-gcc/GNU%20binutils/binutils-2.25-tdm64-1.zip
-RUN powershell -command sleep 5; Invoke-WebRequest -UserAgent 'DockerCI' -outfile 7zsetup.exe http://www.7-zip.org/a/7z1514-x64.exe
-RUN powershell -command sleep 5; Invoke-WebRequest -UserAgent 'DockerCI' -outfile lzma.7z http://www.7-zip.org/a/lzma1514.7z
-RUN powershell -command sleep 5; Invoke-WebRequest -UserAgent 'DockerCI' -outfile gitsetup.exe https://github.com/git-for-windows/git/releases/download/v%GIT_VERSION%.windows.1/Git-%GIT_VERSION%-64-bit.exe
-RUN powershell -command sleep 5; Invoke-WebRequest -UserAgent 'DockerCI' -outfile go.msi https://storage.googleapis.com/golang/go%GOLANG_VERSION%.windows-amd64.msi
-
-# Path
-RUN setx /M Path "c:\git\cmd;c:\git\bin;c:\git\usr\bin;%Path%;c:\gcc\bin;c:\7zip"
-
-# Install and expand the bits we downloaded.
-# Note: The git, 7z and go.msi installers execute asynchronously.
-RUN powershell -command start-process .\gitsetup.exe -ArgumentList '/VERYSILENT /SUPPRESSMSGBOXES /CLOSEAPPLICATIONS /DIR=c:\git' -Wait
-RUN powershell -command start-process .\7zsetup -ArgumentList '/S /D=c:/7zip' -Wait
-RUN powershell -command start-process .\go.msi -ArgumentList '/quiet' -Wait
-RUN powershell -command Expand-Archive gcc.zip \gcc -Force
-RUN powershell -command Expand-Archive runtime.zip \gcc -Force
-RUN powershell -command Expand-Archive binutils.zip \gcc -Force
-RUN powershell -command 7z e lzma.7z bin/lzma.exe
-RUN powershell -command 7z x make.zip make-3.82.90-20111115/bin_amd64/make.exe
-RUN powershell -command mv make-3.82.90-20111115/bin_amd64/make.exe /gcc/bin/
-
-# RSRC for manifest and icon
-RUN powershell -command sleep 5 ; git clone https://github.com/akavel/rsrc.git c:\go\src\github.com\akavel\rsrc
-RUN cd c:/go/src/github.com/akavel/rsrc && git checkout -q %RSRC_COMMIT% && go install -v
-
-# Prepare for building
WORKDIR c:/
+
+# Everything downloaded/installed in one go (better performance, esp on TP4)
+RUN \
+ setx /M Path "c:\git\cmd;c:\git\bin;c:\git\usr\bin;%Path%;c:\gcc\bin;c:\go\bin" && \
+ setx GOROOT "c:\go" && \
+ powershell -command \
+ $ErrorActionPreference = 'Stop'; \
+ Start-Sleep -Seconds 5; \
+ Function Download-File([string] $source, [string] $target) { \
+ $wc = New-Object net.webclient; $wc.Downloadfile($source, $target) \
+ } \
+ \
+ Write-Host INFO: Downloading git...; \
+ Download-File %GIT_LOCATION% gitsetup.exe; \
+ \
+ Write-Host INFO: Downloading go...; \
+ Download-File https://storage.googleapis.com/golang/go%GOLANG_VERSION%.windows-amd64.msi go.msi; \
+ \
+ Write-Host INFO: Downloading compiler 1 of 3...; \
+ Download-File https://raw.githubusercontent.com/jhowardmsft/docker-tdmgcc/master/gcc.zip gcc.zip; \
+ \
+ Write-Host INFO: Downloading compiler 2 of 3...; \
+ Download-File https://raw.githubusercontent.com/jhowardmsft/docker-tdmgcc/master/runtime.zip runtime.zip; \
+ \
+ Write-Host INFO: Downloading compiler 3 of 3...; \
+ Download-File https://raw.githubusercontent.com/jhowardmsft/docker-tdmgcc/master/binutils.zip binutils.zip; \
+ \
+ Write-Host INFO: Installing git...; \
+ Start-Process gitsetup.exe -ArgumentList '/VERYSILENT /SUPPRESSMSGBOXES /CLOSEAPPLICATIONS /DIR=c:\git\' -Wait; \
+ \
+ Write-Host INFO: Installing go..."; \
+ Start-Process msiexec -ArgumentList '-i go.msi -quiet' -Wait; \
+ \
+ Write-Host INFO: Unzipping compiler...; \
+ c:\git\usr\bin\unzip.exe -q -o gcc.zip -d /c/gcc; \
+ c:\git\usr\bin\unzip.exe -q -o runtime.zip -d /c/gcc; \
+ c:\git\usr\bin\unzip.exe -q -o binutils.zip -d /c/gcc"; \
+ \
+ Write-Host INFO: Removing interim files; \
+ Remove-Item *.zip; \
+ Remove-Item go.msi; \
+ Remove-Item gitsetup.exe; \
+ \
+ Write-Host INFO: Cloning and installing RSRC; \
+ c:\git\bin\git.exe clone https://github.com/akavel/rsrc.git c:\go\src\github.com\akavel\rsrc; \
+ cd \go\src\github.com\akavel\rsrc; c:\git\bin\git.exe checkout -q %RSRC_COMMIT%; c:\go\bin\go.exe install -v; \
+ \
+ Write-Host INFO: Completed
+
+# Prepare for building
COPY . /go/src/github.com/docker/docker
diff --git a/components/engine/MAINTAINERS b/components/engine/MAINTAINERS
index 43a2f8753d..99bbd1dc85 100644
--- a/components/engine/MAINTAINERS
+++ b/components/engine/MAINTAINERS
@@ -26,6 +26,7 @@
# the release process is clear and up-to-date.
people = [
+ "aaronlehmann",
"calavera",
"coolljt0725",
"cpuguy83",
@@ -111,6 +112,11 @@
# ADD YOURSELF HERE IN ALPHABETICAL ORDER
+ [people.aaronlehmann]
+ Name = "Aaron Lehmann"
+ Email = "aaron.lehmann@docker.com"
+ GitHub = "aaronlehmann"
+
[people.calavera]
Name = "David Calavera"
Email = "david.calavera@gmail.com"
@@ -196,11 +202,6 @@
Email = "github@gone.nl"
GitHub = "thaJeztah"
- [people.theadactyl]
- Name = "Thea Lamkin"
- Email = "thea@docker.com"
- GitHub = "theadactyl"
-
[people.tianon]
Name = "Tianon Gravi"
Email = "admwiggin@gmail.com"
diff --git a/components/engine/Makefile b/components/engine/Makefile
index ba45c4664e..b455c12f43 100644
--- a/components/engine/Makefile
+++ b/components/engine/Makefile
@@ -54,6 +54,10 @@ DOCKER_ENVS := \
BIND_DIR := $(if $(BINDDIR),$(BINDDIR),$(if $(DOCKER_HOST),,bundles))
DOCKER_MOUNT := $(if $(BIND_DIR),-v "$(CURDIR)/$(BIND_DIR):/go/src/github.com/docker/docker/$(BIND_DIR)")
+# This allows the test suite to be able to run without worrying about the underlying fs used by the container running the daemon (e.g. aufs-on-aufs), so long as the host running the container is running a supported fs.
+# The volume will be cleaned up when the container is removed due to `--rm`.
+# Note that `BIND_DIR` will already be set to `bundles` if `DOCKER_HOST` is not set (see above BIND_DIR line), in such case this will do nothing since `DOCKER_MOUNT` will already be set.
+DOCKER_MOUNT := $(if $(DOCKER_MOUNT),$(DOCKER_MOUNT),-v "/go/src/github.com/docker/docker/bundles")
GIT_BRANCH := $(shell git rev-parse --abbrev-ref HEAD 2>/dev/null)
DOCKER_IMAGE := docker-dev$(if $(GIT_BRANCH),:$(GIT_BRANCH))
@@ -80,6 +84,16 @@ binary: build
$(DOCKER_RUN_DOCKER) hack/make.sh binary
build: bundles
+ifeq ($(DOCKER_OSARCH), linux/arm)
+ # A few libnetwork integration tests require that the kernel be
+ # configured with "dummy" network interface and has the module
+ # loaded. However, the dummy module is not available by default
+ # on arm images. This ensures that it's built and loaded.
+ echo "Syncing kernel modules"
+ oc-sync-kernel-modules
+ depmod
+ modprobe dummy
+endif
docker build ${DOCKER_BUILD_ARGS} -t "$(DOCKER_IMAGE)" -f "$(DOCKERFILE)" .
bundles:
diff --git a/components/engine/README.md b/components/engine/README.md
index e10fbce9df..aa9cc0ea81 100644
--- a/components/engine/README.md
+++ b/components/engine/README.md
@@ -216,7 +216,7 @@ We are always open to suggestions on process improvements, and are always lookin
| Internet Relay Chat (IRC) |
- IRC a direct line to our most knowledgeable Docker users; we have
+ IRC is a direct line to our most knowledgeable Docker users; we have
both the #docker and #docker-dev group on
irc.freenode.net.
IRC is a rich chat protocol but it can overwhelm new users. You can search
@@ -234,6 +234,8 @@ We are always open to suggestions on process improvements, and are always lookin
The docker-dev
group is for contributors and other people contributing to the Docker
project.
+ You can join them without an google account by sending an email to e.g. "docker-user+subscribe@googlegroups.com".
+ After receiving the join-request message, you can simply reply to that to confirm the subscribtion.
|
diff --git a/components/engine/ROADMAP.md b/components/engine/ROADMAP.md
index 4ec0bf0a05..514fdb7423 100644
--- a/components/engine/ROADMAP.md
+++ b/components/engine/ROADMAP.md
@@ -33,97 +33,58 @@ won't be accepting pull requests adding or removing items from this file.
# 1. Features and refactoring
-## 1.1 Security
+## 1.1 Runtime improvements
-Security is a top objective for the Docker Engine. The most notable items we intend to provide in
-the near future are:
+We recently introduced [`runC`](https://runc.io) as a standalone low-level tool for container
+execution. The initial goal was to integrate runC as a replacement in the Engine for the traditional
+default libcontainer `execdriver`, but the Engine internals were not ready for this.
-- Trusted distribution of images: the effort is driven by the [distribution](https://github.com/docker/distribution)
-group but will have significant impact on the Engine
-- [User namespaces](https://github.com/docker/docker/pull/12648)
-- [Seccomp support](https://github.com/docker/libcontainer/pull/613)
+As runC continued evolving, and the OCI specification along with it, we created
+[`containerd`](https://containerd.tools/), a daemon to control and monitor multiple `runC`. This is
+the new target for Engine integration, as it can entirely replace the whole `execdriver`
+architecture, and container monitoring along with it.
-## 1.2 Plumbing project
+Docker Engine will rely on a long-running `containerd` companion daemon for all container execution
+related operations. This could open the door in the future for Engine restarts without interrupting
+running containers.
-We define a plumbing tool as a standalone piece of software usable and meaningful on its own. In
-the current state of the Docker Engine, most subsystems provide independent functionalities (such
-the builder, pushing and pulling images, running applications in a containerized environment, etc)
-but all are coupled in a single binary. We want to offer the users to flexibility to use only the
-pieces they need, and we will also gain in maintainability by splitting the project among multiple
-repositories.
+## 1.2 Plugins improvements
-As it currently stands, the rough design outlines is to have:
-- Low level plumbing tools, each dealing with one responsibility (e.g., [runC](https://runc.io))
-- Docker subsystems services, each exposing an elementary concept over an API, and relying on one or
-multiple lower level plumbing tools for their implementation (e.g., network management)
-- Docker Engine to expose higher level actions (e.g., create a container with volume `V` and network
-`N`), while still providing pass-through access to the individual subsystems.
+Docker Engine 1.7.0 introduced plugin support, initially for the use cases of volumes and networks
+extensions. The plugin infrastructure was kept minimal as we were collecting use cases and real
+world feedback before optimizing for any particular workflow.
-The architectural details are still being worked on, but one thing we know for sure is that we need
-to technically decouple the pieces.
+In the future, we'd like plugins to become first class citizens, and encourage an ecosystem of
+plugins. This implies in particular making it trivially easy to distribute plugins as containers
+through any Registry instance, as well as solving the commonly heard pain points of plugins needing
+to be treated as somewhat special (being active at all time, started before any other user
+containers, and not as easily dismissed).
-### 1.2.1 Runtime
+## 1.3 Internal decoupling
-A Runtime tool already exists today in the form of [runC](https://github.com/opencontainers/runc).
-We intend to modify the Engine to directly call out to a binary implementing the Open Containers
-Specification such as runC rather than relying on libcontainer to set the container runtime up.
+A lot of work has been done in trying to decouple the Docker Engine's internals. In particular, the
+API implementation has been refactored and ongoing work is happening to move the code to a separate
+repository ([`docker/engine-api`](https://github.com/docker/engine-api)), and the Builder side of
+the daemon is now [fully independent](https://github.com/docker/docker/tree/master/builder) while
+still residing in the same repository.
-This plan will deprecate the existing [`execdriver`](https://github.com/docker/docker/tree/master/daemon/execdriver)
-as different runtime backends will be implemented as separated binaries instead of being compiled
-into the Engine.
+We are exploring ways to go further with that decoupling, capitalizing on the work introduced by the
+runtime renovation and plugins improvement efforts. Indeed, the combination of `containerd` support
+with the concept of "special" containers opens the door for bootstrapping more Engine internals
+using the same facilities.
-### 1.2.2 Builder
+## 1.4 Cluster capable Engine
-The Builder (i.e., the ability to build an image from a Dockerfile) is already nicely decoupled,
-but would benefit from being entirely separated from the Engine, and rely on the standard Engine
-API for its operations.
+The community has been pushing for a more cluster capable Docker Engine, and a huge effort was spent
+adding features such as multihost networking, and node discovery down at the Engine level. Yet, the
+Engine is currently incapable of taking scheduling decisions alone, and continues relying on Swarm
+for that.
-### 1.2.3 Distribution
-
-Distribution already has a [dedicated repository](https://github.com/docker/distribution) which
-holds the implementation for Registry v2 and client libraries. We could imagine going further by
-having the Engine call out to a binary providing image distribution related functionalities.
-
-There are two short term goals related to image distribution. The first is stabilize and simplify
-the push/pull code. Following that is the conversion to the more secure Registry V2 protocol.
-
-### 1.2.4 Networking
-
-Most of networking related code was already decoupled today in [libnetwork](https://github.com/docker/libnetwork).
-As with other ingredients, we might want to take it a step further and make it a meaningful utility
-that the Engine would call out to instead of a library.
-
-## 1.3 Plugins
-
-An initiative around plugins started with Docker 1.7.0, with the goal of allowing for out of
-process extensibility of some Docker functionalities, starting with volumes and networking. The
-approach is to provide specific extension points rather than generic hooking facilities. We also
-deliberately keep the extensions API the simplest possible, expanding as we discover valid use
-cases that cannot be implemented.
-
-At the time of writing:
-
-- Plugin support is merged as an experimental feature: real world use cases and user feedback will
-help us refine the UX to make the feature more user friendly.
-- There are no immediate plans to expand on the number of pluggable subsystems.
-- Golang 1.5 might add language support for [plugins](https://docs.google.com/document/d/1nr-TQHw_er6GOQRsF6T43GGhFDelrAP0NqSS_00RgZQ)
-which we consider supporting as an alternative to JSON/HTTP.
-
-## 1.4 Volume management
-
-Volumes are not a first class citizen in the Engine today: we would like better volume management,
-similar to the way network are managed in the new [CNM](https://github.com/docker/docker/issues/9983).
-
-## 1.5 Better API implementation
-
-The current Engine API is insufficiently typed, versioned, and ultimately hard to maintain. We
-also suffer from the lack of a common implementation with [Swarm](https://github.com/docker/swarm).
-
-## 1.6 Checkpoint/restore
-
-Support for checkpoint/restore was [merged](https://github.com/docker/libcontainer/pull/479) in
-[libcontainer](https://github.com/docker/libcontainer) and made available through [runC](https://runc.io):
-we intend to take advantage of it in the Engine.
+We plan to complete this effort and make Engine fully cluster capable. Multiple instances of the
+Docker Engine being already capable of discovering each other and establish overlay networking for
+their container to communicate, the next step is for a given Engine to gain ability to dispatch work
+to another node in the cluster. This will be introduced in a backward compatible way, such that a
+`docker run` invocation on a particular node remains fully deterministic.
# 2 Frozen features
@@ -139,45 +100,41 @@ The Dockerfile syntax as we know it is simple, and has proven successful in supp
definitive move, we temporarily won't accept more patches to the Dockerfile syntax for several
reasons:
-- Long term impact of syntax changes is a sensitive matter that require an amount of attention
-the volume of Engine codebase and activity today doesn't allow us to provide.
-- Allowing the Builder to be implemented as a separate utility consuming the Engine's API will
-open the door for many possibilities, such as offering alternate syntaxes or DSL for existing
-languages without cluttering the Engine's codebase.
-- A standalone Builder will also offer the opportunity for a better dedicated group of maintainers
-to own the Dockerfile syntax and decide collectively on the direction to give it.
-- Our experience with official images tend to show that no new instruction or syntax expansion is
-*strictly* necessary for the majority of use cases, and although we are aware many things are still
-lacking for many, we cannot make it a priority yet for the above reasons.
+ - Long term impact of syntax changes is a sensitive matter that require an amount of attention the
+ volume of Engine codebase and activity today doesn't allow us to provide.
+ - Allowing the Builder to be implemented as a separate utility consuming the Engine's API will
+ open the door for many possibilities, such as offering alternate syntaxes or DSL for existing
+ languages without cluttering the Engine's codebase.
+ - A standalone Builder will also offer the opportunity for a better dedicated group of maintainers
+ to own the Dockerfile syntax and decide collectively on the direction to give it.
+ - Our experience with official images tend to show that no new instruction or syntax expansion is
+ *strictly* necessary for the majority of use cases, and although we are aware many things are
+ still lacking for many, we cannot make it a priority yet for the above reasons.
Again, this is not about saying that the Dockerfile syntax is done, it's about making choices about
what we want to do first!
## 2.3 Remote Registry Operations
-A large amount of work is ongoing in the area of image distribution and
-provenance. This includes moving to the V2 Registry API and heavily
-refactoring the code that powers these features. The desired result is more
-secure, reliable and easier to use image distribution.
+A large amount of work is ongoing in the area of image distribution and provenance. This includes
+moving to the V2 Registry API and heavily refactoring the code that powers these features. The
+desired result is more secure, reliable and easier to use image distribution.
-Part of the problem with this part of the code base is the lack of a stable
-and flexible interface. If new features are added that access the registry
-without solidifying these interfaces, achieving feature parity will continue
-to be elusive. While we get a handle on this situation, we are imposing a
-moratorium on new code that accesses the Registry API in commands that don't
-already make remote calls.
+Part of the problem with this part of the code base is the lack of a stable and flexible interface.
+If new features are added that access the registry without solidifying these interfaces, achieving
+feature parity will continue to be elusive. While we get a handle on this situation, we are imposing
+a moratorium on new code that accesses the Registry API in commands that don't already make remote
+calls.
-Currently, only the following commands cause interaction with a remote
-registry:
+Currently, only the following commands cause interaction with a remote registry:
-- push
-- pull
-- run
-- build
-- search
-- login
+ - push
+ - pull
+ - run
+ - build
+ - search
+ - login
-In the interest of stabilizing the registry access model during this ongoing
-work, we are not accepting additions to other commands that will cause remote
-interaction with the Registry API. This moratorium will lift when the goals of
-the distribution project have been met.
+In the interest of stabilizing the registry access model during this ongoing work, we are not
+accepting additions to other commands that will cause remote interaction with the Registry API. This
+moratorium will lift when the goals of the distribution project have been met.
diff --git a/components/engine/api/client/attach.go b/components/engine/api/client/attach.go
index efdad108ce..c7b1eae3f3 100644
--- a/components/engine/api/client/attach.go
+++ b/components/engine/api/client/attach.go
@@ -41,12 +41,6 @@ func (cli *DockerCli) CmdAttach(args ...string) error {
return err
}
- if c.Config.Tty && cli.isTerminalOut {
- if err := cli.monitorTtySize(cmd.Arg(0), false); err != nil {
- logrus.Debugf("Error monitoring TTY size: %s", err)
- }
- }
-
if *detachKeys != "" {
cli.configFile.DetachKeys = *detachKeys
}
@@ -82,6 +76,21 @@ func (cli *DockerCli) CmdAttach(args ...string) error {
defer cli.restoreTerminal(in)
}
+ if c.Config.Tty && cli.isTerminalOut {
+ height, width := cli.getTtySize()
+ // To handle the case where a user repeatedly attaches/detaches without resizing their
+ // terminal, the only way to get the shell prompt to display for attaches 2+ is to artificially
+ // resize it, then go back to normal. Without this, every attach after the first will
+ // require the user to manually resize or hit enter.
+ cli.resizeTtyTo(cmd.Arg(0), height+1, width+1, false)
+
+ // After the above resizing occurs, the call to monitorTtySize below will handle resetting back
+ // to the actual size.
+ if err := cli.monitorTtySize(cmd.Arg(0), false); err != nil {
+ logrus.Debugf("Error monitoring TTY size: %s", err)
+ }
+ }
+
if err := cli.holdHijackedConnection(c.Config.Tty, in, cli.out, cli.err, resp); err != nil {
return err
}
diff --git a/components/engine/api/client/build.go b/components/engine/api/client/build.go
index e6a4749d39..eef716a99e 100644
--- a/components/engine/api/client/build.go
+++ b/components/engine/api/client/build.go
@@ -6,25 +6,20 @@ import (
"bytes"
"fmt"
"io"
- "io/ioutil"
"os"
- "os/exec"
"path/filepath"
"regexp"
"runtime"
- "strings"
"golang.org/x/net/context"
"github.com/docker/docker/api"
+ "github.com/docker/docker/builder"
"github.com/docker/docker/builder/dockerignore"
Cli "github.com/docker/docker/cli"
"github.com/docker/docker/opts"
"github.com/docker/docker/pkg/archive"
"github.com/docker/docker/pkg/fileutils"
- "github.com/docker/docker/pkg/gitutils"
- "github.com/docker/docker/pkg/httputils"
- "github.com/docker/docker/pkg/ioutils"
"github.com/docker/docker/pkg/jsonmessage"
flag "github.com/docker/docker/pkg/mflag"
"github.com/docker/docker/pkg/progress"
@@ -65,7 +60,7 @@ func (cli *DockerCli) CmdBuild(args ...string) error {
flCgroupParent := cmd.String([]string{"-cgroup-parent"}, "", "Optional parent cgroup for the container")
flBuildArg := opts.NewListOpts(runconfigopts.ValidateEnv)
cmd.Var(&flBuildArg, []string{"-build-arg"}, "Set build-time variables")
- isolation := cmd.String([]string{"-isolation"}, "", "Container isolation level")
+ isolation := cmd.String([]string{"-isolation"}, "", "Container isolation technology")
ulimits := make(map[string]*units.Ulimit)
flUlimits := runconfigopts.NewUlimitOpt(&ulimits)
@@ -102,13 +97,13 @@ func (cli *DockerCli) CmdBuild(args ...string) error {
switch {
case specifiedContext == "-":
- ctx, relDockerfile, err = getContextFromReader(cli.in, *dockerfileName)
+ ctx, relDockerfile, err = builder.GetContextFromReader(cli.in, *dockerfileName)
case urlutil.IsGitURL(specifiedContext):
- tempDir, relDockerfile, err = getContextFromGitURL(specifiedContext, *dockerfileName)
+ tempDir, relDockerfile, err = builder.GetContextFromGitURL(specifiedContext, *dockerfileName)
case urlutil.IsURL(specifiedContext):
- ctx, relDockerfile, err = getContextFromURL(progBuff, specifiedContext, *dockerfileName)
+ ctx, relDockerfile, err = builder.GetContextFromURL(progBuff, specifiedContext, *dockerfileName)
default:
- contextDir, relDockerfile, err = getContextFromLocalDir(specifiedContext, *dockerfileName)
+ contextDir, relDockerfile, err = builder.GetContextFromLocalDir(specifiedContext, *dockerfileName)
}
if err != nil {
@@ -143,7 +138,7 @@ func (cli *DockerCli) CmdBuild(args ...string) error {
}
}
- if err := validateContextDirectory(contextDir, excludes); err != nil {
+ if err := builder.ValidateContextDirectory(contextDir, excludes); err != nil {
return fmt.Errorf("Error checking context: '%s'.", err)
}
@@ -223,7 +218,7 @@ func (cli *DockerCli) CmdBuild(args ...string) error {
Remove: *rm,
ForceRemove: *forceRm,
PullParent: *pull,
- IsolationLevel: container.IsolationLevel(*isolation),
+ Isolation: container.Isolation(*isolation),
CPUSetCPUs: *flCPUSetCpus,
CPUSetMems: *flCPUSetMems,
CPUShares: *flCPUShares,
@@ -234,7 +229,7 @@ func (cli *DockerCli) CmdBuild(args ...string) error {
ShmSize: shmSize,
Ulimits: flUlimits.GetList(),
BuildArgs: runconfigopts.ConvertKVStringsToMap(flBuildArg.GetAll()),
- AuthConfigs: cli.configFile.AuthConfigs,
+ AuthConfigs: cli.retrieveAuthConfigs(),
}
response, err := cli.client.ImageBuild(context.Background(), options)
@@ -281,54 +276,6 @@ func (cli *DockerCli) CmdBuild(args ...string) error {
return nil
}
-// validateContextDirectory checks if all the contents of the directory
-// can be read and returns an error if some files can't be read
-// symlinks which point to non-existing files don't trigger an error
-func validateContextDirectory(srcPath string, excludes []string) error {
- contextRoot, err := getContextRoot(srcPath)
- if err != nil {
- return err
- }
- return filepath.Walk(contextRoot, func(filePath string, f os.FileInfo, err error) error {
- // skip this directory/file if it's not in the path, it won't get added to the context
- if relFilePath, err := filepath.Rel(contextRoot, filePath); err != nil {
- return err
- } else if skip, err := fileutils.Matches(relFilePath, excludes); err != nil {
- return err
- } else if skip {
- if f.IsDir() {
- return filepath.SkipDir
- }
- return nil
- }
-
- if err != nil {
- if os.IsPermission(err) {
- return fmt.Errorf("can't stat '%s'", filePath)
- }
- if os.IsNotExist(err) {
- return nil
- }
- return err
- }
-
- // skip checking if symlinks point to non-existing files, such symlinks can be useful
- // also skip named pipes, because they hanging on open
- if f.Mode()&(os.ModeSymlink|os.ModeNamedPipe) != 0 {
- return nil
- }
-
- if !f.IsDir() {
- currentFile, err := os.Open(filePath)
- if err != nil && os.IsPermission(err) {
- return fmt.Errorf("no permission to read from '%s'", filePath)
- }
- currentFile.Close()
- }
- return nil
- })
-}
-
// validateTag checks if the given image name can be resolved.
func validateTag(rawRepo string) (string, error) {
_, err := reference.ParseNamed(rawRepo)
@@ -339,96 +286,6 @@ func validateTag(rawRepo string) (string, error) {
return rawRepo, nil
}
-// isUNC returns true if the path is UNC (one starting \\). It always returns
-// false on Linux.
-func isUNC(path string) bool {
- return runtime.GOOS == "windows" && strings.HasPrefix(path, `\\`)
-}
-
-// getDockerfileRelPath uses the given context directory for a `docker build`
-// and returns the absolute path to the context directory, the relative path of
-// the dockerfile in that context directory, and a non-nil error on success.
-func getDockerfileRelPath(givenContextDir, givenDockerfile string) (absContextDir, relDockerfile string, err error) {
- if absContextDir, err = filepath.Abs(givenContextDir); err != nil {
- return "", "", fmt.Errorf("unable to get absolute context directory: %v", err)
- }
-
- // The context dir might be a symbolic link, so follow it to the actual
- // target directory.
- //
- // FIXME. We use isUNC (always false on non-Windows platforms) to workaround
- // an issue in golang. On Windows, EvalSymLinks does not work on UNC file
- // paths (those starting with \\). This hack means that when using links
- // on UNC paths, they will not be followed.
- if !isUNC(absContextDir) {
- absContextDir, err = filepath.EvalSymlinks(absContextDir)
- if err != nil {
- return "", "", fmt.Errorf("unable to evaluate symlinks in context path: %v", err)
- }
- }
-
- stat, err := os.Lstat(absContextDir)
- if err != nil {
- return "", "", fmt.Errorf("unable to stat context directory %q: %v", absContextDir, err)
- }
-
- if !stat.IsDir() {
- return "", "", fmt.Errorf("context must be a directory: %s", absContextDir)
- }
-
- absDockerfile := givenDockerfile
- if absDockerfile == "" {
- // No -f/--file was specified so use the default relative to the
- // context directory.
- absDockerfile = filepath.Join(absContextDir, api.DefaultDockerfileName)
-
- // Just to be nice ;-) look for 'dockerfile' too but only
- // use it if we found it, otherwise ignore this check
- if _, err = os.Lstat(absDockerfile); os.IsNotExist(err) {
- altPath := filepath.Join(absContextDir, strings.ToLower(api.DefaultDockerfileName))
- if _, err = os.Lstat(altPath); err == nil {
- absDockerfile = altPath
- }
- }
- }
-
- // If not already an absolute path, the Dockerfile path should be joined to
- // the base directory.
- if !filepath.IsAbs(absDockerfile) {
- absDockerfile = filepath.Join(absContextDir, absDockerfile)
- }
-
- // Evaluate symlinks in the path to the Dockerfile too.
- //
- // FIXME. We use isUNC (always false on non-Windows platforms) to workaround
- // an issue in golang. On Windows, EvalSymLinks does not work on UNC file
- // paths (those starting with \\). This hack means that when using links
- // on UNC paths, they will not be followed.
- if !isUNC(absDockerfile) {
- absDockerfile, err = filepath.EvalSymlinks(absDockerfile)
- if err != nil {
- return "", "", fmt.Errorf("unable to evaluate symlinks in Dockerfile path: %v", err)
- }
- }
-
- if _, err := os.Lstat(absDockerfile); err != nil {
- if os.IsNotExist(err) {
- return "", "", fmt.Errorf("Cannot locate Dockerfile: %q", absDockerfile)
- }
- return "", "", fmt.Errorf("unable to stat Dockerfile: %v", err)
- }
-
- if relDockerfile, err = filepath.Rel(absContextDir, absDockerfile); err != nil {
- return "", "", fmt.Errorf("unable to get relative Dockerfile path: %v", err)
- }
-
- if strings.HasPrefix(relDockerfile, ".."+string(filepath.Separator)) {
- return "", "", fmt.Errorf("The Dockerfile (%s) must be within the build context (%s)", givenDockerfile, givenContextDir)
- }
-
- return absContextDir, relDockerfile, nil
-}
-
// writeToFile copies from the given reader and writes it to a file with the
// given filename.
func writeToFile(r io.Reader, filename string) error {
@@ -445,107 +302,6 @@ func writeToFile(r io.Reader, filename string) error {
return nil
}
-// getContextFromReader will read the contents of the given reader as either a
-// Dockerfile or tar archive. Returns a tar archive used as a context and a
-// path to the Dockerfile inside the tar.
-func getContextFromReader(r io.ReadCloser, dockerfileName string) (out io.ReadCloser, relDockerfile string, err error) {
- buf := bufio.NewReader(r)
-
- magic, err := buf.Peek(archive.HeaderSize)
- if err != nil && err != io.EOF {
- return nil, "", fmt.Errorf("failed to peek context header from STDIN: %v", err)
- }
-
- if archive.IsArchive(magic) {
- return ioutils.NewReadCloserWrapper(buf, func() error { return r.Close() }), dockerfileName, nil
- }
-
- // Input should be read as a Dockerfile.
- tmpDir, err := ioutil.TempDir("", "docker-build-context-")
- if err != nil {
- return nil, "", fmt.Errorf("unbale to create temporary context directory: %v", err)
- }
-
- f, err := os.Create(filepath.Join(tmpDir, api.DefaultDockerfileName))
- if err != nil {
- return nil, "", err
- }
- _, err = io.Copy(f, buf)
- if err != nil {
- f.Close()
- return nil, "", err
- }
-
- if err := f.Close(); err != nil {
- return nil, "", err
- }
- if err := r.Close(); err != nil {
- return nil, "", err
- }
-
- tar, err := archive.Tar(tmpDir, archive.Uncompressed)
- if err != nil {
- return nil, "", err
- }
-
- return ioutils.NewReadCloserWrapper(tar, func() error {
- err := tar.Close()
- os.RemoveAll(tmpDir)
- return err
- }), api.DefaultDockerfileName, nil
-
-}
-
-// getContextFromGitURL uses a Git URL as context for a `docker build`. The
-// git repo is cloned into a temporary directory used as the context directory.
-// Returns the absolute path to the temporary context directory, the relative
-// path of the dockerfile in that context directory, and a non-nil error on
-// success.
-func getContextFromGitURL(gitURL, dockerfileName string) (absContextDir, relDockerfile string, err error) {
- if _, err := exec.LookPath("git"); err != nil {
- return "", "", fmt.Errorf("unable to find 'git': %v", err)
- }
- if absContextDir, err = gitutils.Clone(gitURL); err != nil {
- return "", "", fmt.Errorf("unable to 'git clone' to temporary context directory: %v", err)
- }
-
- return getDockerfileRelPath(absContextDir, dockerfileName)
-}
-
-// getContextFromURL uses a remote URL as context for a `docker build`. The
-// remote resource is downloaded as either a Dockerfile or a tar archive.
-// Returns the tar archive used for the context and a path of the
-// dockerfile inside the tar.
-func getContextFromURL(out io.Writer, remoteURL, dockerfileName string) (io.ReadCloser, string, error) {
- response, err := httputils.Download(remoteURL)
- if err != nil {
- return nil, "", fmt.Errorf("unable to download remote context %s: %v", remoteURL, err)
- }
- progressOutput := streamformatter.NewStreamFormatter().NewProgressOutput(out, true)
-
- // Pass the response body through a progress reader.
- progReader := progress.NewProgressReader(response.Body, progressOutput, response.ContentLength, "", fmt.Sprintf("Downloading build context from remote url: %s", remoteURL))
-
- return getContextFromReader(ioutils.NewReadCloserWrapper(progReader, func() error { return response.Body.Close() }), dockerfileName)
-}
-
-// getContextFromLocalDir uses the given local directory as context for a
-// `docker build`. Returns the absolute path to the local context directory,
-// the relative path of the dockerfile in that context directory, and a non-nil
-// error on success.
-func getContextFromLocalDir(localDir, dockerfileName string) (absContextDir, relDockerfile string, err error) {
- // When using a local context directory, when the Dockerfile is specified
- // with the `-f/--file` option then it is considered relative to the
- // current directory and not the context directory.
- if dockerfileName != "" {
- if dockerfileName, err = filepath.Abs(dockerfileName); err != nil {
- return "", "", fmt.Errorf("unable to get absolute path to Dockerfile: %v", err)
- }
- }
-
- return getDockerfileRelPath(localDir, dockerfileName)
-}
-
var dockerfileFromLinePattern = regexp.MustCompile(`(?i)^[\s]*FROM[ \f\r\t\v]+(?P[^ \f\r\t\v\n#]+)`)
// resolvedTag records the repository, tag, and resolved digest reference
diff --git a/components/engine/api/client/cli.go b/components/engine/api/client/cli.go
index fd76fc9dbb..e49c5351d5 100644
--- a/components/engine/api/client/cli.go
+++ b/components/engine/api/client/cli.go
@@ -11,6 +11,7 @@ import (
"github.com/docker/docker/api"
"github.com/docker/docker/cli"
"github.com/docker/docker/cliconfig"
+ "github.com/docker/docker/cliconfig/credentials"
"github.com/docker/docker/dockerversion"
"github.com/docker/docker/opts"
"github.com/docker/docker/pkg/term"
@@ -125,6 +126,9 @@ func NewDockerCli(in io.ReadCloser, out, err io.Writer, clientFlags *cli.ClientF
if e != nil {
fmt.Fprintf(cli.err, "WARNING: Error loading config file:%v\n", e)
}
+ if !configFile.ContainsAuth() {
+ credentials.DetectDefaultStore(configFile)
+ }
cli.configFile = configFile
host, err := getServerHost(clientFlags.Common.Hosts, clientFlags.Common.TLSOptions)
diff --git a/components/engine/api/client/create.go b/components/engine/api/client/create.go
index d0417322e9..0c19145463 100644
--- a/components/engine/api/client/create.go
+++ b/components/engine/api/client/create.go
@@ -42,7 +42,7 @@ func (cli *DockerCli) pullImageCustomOut(image string, out io.Writer) error {
return err
}
- authConfig := cli.resolveAuthConfig(cli.configFile.AuthConfigs, repoInfo.Index)
+ authConfig := cli.resolveAuthConfig(repoInfo.Index)
encodedAuth, err := encodeAuthToBase64(authConfig)
if err != nil {
return err
diff --git a/components/engine/api/client/events.go b/components/engine/api/client/events.go
index 54b88f054b..d2408c192e 100644
--- a/components/engine/api/client/events.go
+++ b/components/engine/api/client/events.go
@@ -6,10 +6,12 @@ import (
"io"
"sort"
"strings"
+ "sync"
"time"
"golang.org/x/net/context"
+ "github.com/Sirupsen/logrus"
Cli "github.com/docker/docker/cli"
"github.com/docker/docker/opts"
"github.com/docker/docker/pkg/jsonlog"
@@ -115,3 +117,30 @@ func printOutput(event eventtypes.Message, output io.Writer) {
}
fmt.Fprint(output, "\n")
}
+
+type eventHandler struct {
+ handlers map[string]func(eventtypes.Message)
+ mu sync.Mutex
+}
+
+func (w *eventHandler) Handle(action string, h func(eventtypes.Message)) {
+ w.mu.Lock()
+ w.handlers[action] = h
+ w.mu.Unlock()
+}
+
+// Watch ranges over the passed in event chan and processes the events based on the
+// handlers created for a given action.
+// To stop watching, close the event chan.
+func (w *eventHandler) Watch(c <-chan eventtypes.Message) {
+ for e := range c {
+ w.mu.Lock()
+ h, exists := w.handlers[e.Action]
+ w.mu.Unlock()
+ if !exists {
+ continue
+ }
+ logrus.Debugf("event handler: received event: %v", e)
+ go h(e)
+ }
+}
diff --git a/components/engine/api/client/formatter/custom.go b/components/engine/api/client/formatter/custom.go
index 8a680705ca..9ac457a414 100644
--- a/components/engine/api/client/formatter/custom.go
+++ b/components/engine/api/client/formatter/custom.go
@@ -31,6 +31,7 @@ const (
repositoryHeader = "REPOSITORY"
tagHeader = "TAG"
digestHeader = "DIGEST"
+ mountsHeader = "MOUNTS"
)
type containerContext struct {
@@ -142,6 +143,20 @@ func (c *containerContext) Label(name string) string {
return c.c.Labels[name]
}
+func (c *containerContext) Mounts() string {
+ c.addHeader(mountsHeader)
+
+ var mounts []string
+ for _, m := range c.c.Mounts {
+ name := m.Name
+ if c.trunc {
+ name = stringutils.Truncate(name, 15)
+ }
+ mounts = append(mounts, name)
+ }
+ return strings.Join(mounts, ",")
+}
+
type imageContext struct {
baseSubContext
trunc bool
diff --git a/components/engine/api/client/formatter/custom_test.go b/components/engine/api/client/formatter/custom_test.go
index 608622564a..6a21f2bcd4 100644
--- a/components/engine/api/client/formatter/custom_test.go
+++ b/components/engine/api/client/formatter/custom_test.go
@@ -12,7 +12,7 @@ import (
func TestContainerPsContext(t *testing.T) {
containerID := stringid.GenerateRandomID()
- unix := time.Now().Unix()
+ unix := time.Now().Add(-65 * time.Second).Unix()
var ctx containerContext
cases := []struct {
@@ -55,7 +55,7 @@ func TestContainerPsContext(t *testing.T) {
{types.Container{SizeRw: 10, SizeRootFs: 20}, true, "10 B (virtual 20 B)", sizeHeader, ctx.Size},
{types.Container{}, true, "", labelsHeader, ctx.Labels},
{types.Container{Labels: map[string]string{"cpu": "6", "storage": "ssd"}}, true, "cpu=6,storage=ssd", labelsHeader, ctx.Labels},
- {types.Container{Created: unix}, true, "Less than a second", runningForHeader, ctx.RunningFor},
+ {types.Container{Created: unix}, true, "About a minute", runningForHeader, ctx.RunningFor},
}
for _, c := range cases {
diff --git a/components/engine/api/client/info.go b/components/engine/api/client/info.go
index 42820cd6d3..2d02af3a58 100644
--- a/components/engine/api/client/info.go
+++ b/components/engine/api/client/info.go
@@ -50,6 +50,7 @@ func (cli *DockerCli) CmdInfo(args ...string) error {
}
ioutils.FprintfIfNotEmpty(cli.out, "Execution Driver: %s\n", info.ExecutionDriver)
ioutils.FprintfIfNotEmpty(cli.out, "Logging Driver: %s\n", info.LoggingDriver)
+ ioutils.FprintfIfNotEmpty(cli.out, "Cgroup Driver: %s\n", info.CgroupDriver)
fmt.Fprintf(cli.out, "Plugins: \n")
fmt.Fprintf(cli.out, " Volume:")
@@ -73,7 +74,7 @@ func (cli *DockerCli) CmdInfo(args ...string) error {
fmt.Fprintf(cli.out, "Total Memory: %s\n", units.BytesSize(float64(info.MemTotal)))
ioutils.FprintfIfNotEmpty(cli.out, "Name: %s\n", info.Name)
ioutils.FprintfIfNotEmpty(cli.out, "ID: %s\n", info.ID)
-
+ fmt.Fprintf(cli.out, "Docker Root Dir: %s\n", info.DockerRootDir)
fmt.Fprintf(cli.out, "Debug mode (client): %v\n", utils.IsDebugEnabled())
fmt.Fprintf(cli.out, "Debug mode (server): %v\n", info.Debug)
@@ -82,7 +83,6 @@ func (cli *DockerCli) CmdInfo(args ...string) error {
fmt.Fprintf(cli.out, " Goroutines: %d\n", info.NGoroutines)
fmt.Fprintf(cli.out, " System Time: %s\n", info.SystemTime)
fmt.Fprintf(cli.out, " EventsListeners: %d\n", info.NEventsListener)
- fmt.Fprintf(cli.out, " Docker Root Dir: %s\n", info.DockerRootDir)
}
ioutils.FprintfIfNotEmpty(cli.out, "Http Proxy: %s\n", info.HTTPProxy)
@@ -105,6 +105,9 @@ func (cli *DockerCli) CmdInfo(args ...string) error {
if !info.SwapLimit {
fmt.Fprintln(cli.err, "WARNING: No swap limit support")
}
+ if !info.KernelMemory {
+ fmt.Fprintln(cli.err, "WARNING: No kernel memory limit support")
+ }
if !info.OomKillDisable {
fmt.Fprintln(cli.err, "WARNING: No oom kill disable support")
}
diff --git a/components/engine/api/client/login.go b/components/engine/api/client/login.go
index 18ce831911..6696feb8cf 100644
--- a/components/engine/api/client/login.go
+++ b/components/engine/api/client/login.go
@@ -9,13 +9,14 @@ import (
"strings"
Cli "github.com/docker/docker/cli"
+ "github.com/docker/docker/cliconfig"
+ "github.com/docker/docker/cliconfig/credentials"
flag "github.com/docker/docker/pkg/mflag"
"github.com/docker/docker/pkg/term"
- "github.com/docker/engine-api/client"
"github.com/docker/engine-api/types"
)
-// CmdLogin logs in or registers a user to a Docker registry service.
+// CmdLogin logs in a user to a Docker registry service.
//
// If no server is specified, the user will be logged into or registered to the registry's index server.
//
@@ -26,7 +27,9 @@ func (cli *DockerCli) CmdLogin(args ...string) error {
flUser := cmd.String([]string{"u", "-username"}, "", "Username")
flPassword := cmd.String([]string{"p", "-password"}, "", "Password")
- flEmail := cmd.String([]string{"e", "-email"}, "", "Email")
+
+ // Deprecated in 1.11: Should be removed in docker 1.13
+ cmd.String([]string{"#e", "#-email"}, "", "Email")
cmd.ParseFlags(args, true)
@@ -36,32 +39,27 @@ func (cli *DockerCli) CmdLogin(args ...string) error {
}
var serverAddress string
+ var isDefaultRegistry bool
if len(cmd.Args()) > 0 {
serverAddress = cmd.Arg(0)
} else {
serverAddress = cli.electAuthServer()
+ isDefaultRegistry = true
}
- authConfig, err := cli.configureAuth(*flUser, *flPassword, *flEmail, serverAddress)
+ authConfig, err := cli.configureAuth(*flUser, *flPassword, serverAddress, isDefaultRegistry)
if err != nil {
return err
}
response, err := cli.client.RegistryLogin(authConfig)
if err != nil {
- if client.IsErrUnauthorized(err) {
- delete(cli.configFile.AuthConfigs, serverAddress)
- if err2 := cli.configFile.Save(); err2 != nil {
- fmt.Fprintf(cli.out, "WARNING: could not save config file: %v\n", err2)
- }
- }
return err
}
- if err := cli.configFile.Save(); err != nil {
- return fmt.Errorf("Error saving config file: %v", err)
+ if err := storeCredentials(cli.configFile, authConfig); err != nil {
+ return fmt.Errorf("Error saving credentials: %v", err)
}
- fmt.Fprintf(cli.out, "WARNING: login credentials saved in %s\n", cli.configFile.Filename())
if response.Status != "" {
fmt.Fprintf(cli.out, "%s\n", response.Status)
@@ -77,14 +75,19 @@ func (cli *DockerCli) promptWithDefault(prompt string, configDefault string) {
}
}
-func (cli *DockerCli) configureAuth(flUser, flPassword, flEmail, serverAddress string) (types.AuthConfig, error) {
- authconfig, ok := cli.configFile.AuthConfigs[serverAddress]
- if !ok {
- authconfig = types.AuthConfig{}
+func (cli *DockerCli) configureAuth(flUser, flPassword, serverAddress string, isDefaultRegistry bool) (types.AuthConfig, error) {
+ authconfig, err := getCredentials(cli.configFile, serverAddress)
+ if err != nil {
+ return authconfig, err
}
+
authconfig.Username = strings.TrimSpace(authconfig.Username)
if flUser = strings.TrimSpace(flUser); flUser == "" {
+ if isDefaultRegistry {
+ // if this is a defauly registry (docker hub), then display the following message.
+ fmt.Fprintln(cli.out, "Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.")
+ }
cli.promptWithDefault("Username", authconfig.Username)
flUser = readInput(cli.in, cli.out)
flUser = strings.TrimSpace(flUser)
@@ -114,30 +117,10 @@ func (cli *DockerCli) configureAuth(flUser, flPassword, flEmail, serverAddress s
}
}
- // Assume that a different username means they may not want to use
- // the email from the config file, so prompt it
- if flUser != authconfig.Username {
- if flEmail == "" {
- cli.promptWithDefault("Email", authconfig.Email)
- flEmail = readInput(cli.in, cli.out)
- if flEmail == "" {
- flEmail = authconfig.Email
- }
- }
- } else {
- // However, if they don't override the username use the
- // email from the cmd line if specified. IOW, allow
- // then to change/override them. And if not specified, just
- // use what's in the config file
- if flEmail == "" {
- flEmail = authconfig.Email
- }
- }
authconfig.Username = flUser
authconfig.Password = flPassword
- authconfig.Email = flEmail
authconfig.ServerAddress = serverAddress
- cli.configFile.AuthConfigs[serverAddress] = authconfig
+
return authconfig, nil
}
@@ -150,3 +133,38 @@ func readInput(in io.Reader, out io.Writer) string {
}
return string(line)
}
+
+// getCredentials loads the user credentials from a credentials store.
+// The store is determined by the config file settings.
+func getCredentials(c *cliconfig.ConfigFile, serverAddress string) (types.AuthConfig, error) {
+ s := loadCredentialsStore(c)
+ return s.Get(serverAddress)
+}
+
+func getAllCredentials(c *cliconfig.ConfigFile) (map[string]types.AuthConfig, error) {
+ s := loadCredentialsStore(c)
+ return s.GetAll()
+}
+
+// storeCredentials saves the user credentials in a credentials store.
+// The store is determined by the config file settings.
+func storeCredentials(c *cliconfig.ConfigFile, auth types.AuthConfig) error {
+ s := loadCredentialsStore(c)
+ return s.Store(auth)
+}
+
+// eraseCredentials removes the user credentials from a credentials store.
+// The store is determined by the config file settings.
+func eraseCredentials(c *cliconfig.ConfigFile, serverAddress string) error {
+ s := loadCredentialsStore(c)
+ return s.Erase(serverAddress)
+}
+
+// loadCredentialsStore initializes a new credentials store based
+// in the settings provided in the configuration file.
+func loadCredentialsStore(c *cliconfig.ConfigFile) credentials.Store {
+ if c.CredentialsStore != "" {
+ return credentials.NewNativeStore(c)
+ }
+ return credentials.NewFileStore(c)
+}
diff --git a/components/engine/api/client/logout.go b/components/engine/api/client/logout.go
index f81eb8dd12..b5ff59ddd2 100644
--- a/components/engine/api/client/logout.go
+++ b/components/engine/api/client/logout.go
@@ -25,15 +25,16 @@ func (cli *DockerCli) CmdLogout(args ...string) error {
serverAddress = cli.electAuthServer()
}
+ // check if we're logged in based on the records in the config file
+ // which means it couldn't have user/pass cause they may be in the creds store
if _, ok := cli.configFile.AuthConfigs[serverAddress]; !ok {
fmt.Fprintf(cli.out, "Not logged in to %s\n", serverAddress)
return nil
}
fmt.Fprintf(cli.out, "Remove login credentials for %s\n", serverAddress)
- delete(cli.configFile.AuthConfigs, serverAddress)
- if err := cli.configFile.Save(); err != nil {
- return fmt.Errorf("Failed to save docker config: %v", err)
+ if err := eraseCredentials(cli.configFile, serverAddress); err != nil {
+ fmt.Fprintf(cli.out, "WARNING: could not erase credentials: %v\n", err)
}
return nil
diff --git a/components/engine/api/client/network.go b/components/engine/api/client/network.go
index 56adabc00b..f992af3da4 100644
--- a/components/engine/api/client/network.go
+++ b/components/engine/api/client/network.go
@@ -3,6 +3,7 @@ package client
import (
"fmt"
"net"
+ "sort"
"strings"
"text/tabwriter"
@@ -50,6 +51,7 @@ func (cli *DockerCli) CmdNetworkCreate(args ...string) error {
cmd.Var(flIpamOpt, []string{"-ipam-opt"}, "set IPAM driver specific options")
flInternal := cmd.Bool([]string{"-internal"}, false, "restricts external access to the network")
+ flIPv6 := cmd.Bool([]string{"-ipv6"}, false, "enable IPv6 networking")
cmd.Require(flag.Exact, 1)
err := cmd.ParseFlags(args, true)
@@ -77,6 +79,7 @@ func (cli *DockerCli) CmdNetworkCreate(args ...string) error {
Options: flOpts.GetAll(),
CheckDuplicate: true,
Internal: *flInternal,
+ EnableIPv6: *flIPv6,
}
resp, err := cli.client.NetworkCreate(nc)
@@ -192,7 +195,7 @@ func (cli *DockerCli) CmdNetworkLs(args ...string) error {
if !*quiet {
fmt.Fprintln(wr, "NETWORK ID\tNAME\tDRIVER")
}
-
+ sort.Sort(byNetworkName(networkResources))
for _, networkResource := range networkResources {
ID := networkResource.ID
netName := networkResource.Name
@@ -214,6 +217,12 @@ func (cli *DockerCli) CmdNetworkLs(args ...string) error {
return nil
}
+type byNetworkName []types.NetworkResource
+
+func (r byNetworkName) Len() int { return len(r) }
+func (r byNetworkName) Swap(i, j int) { r[i], r[j] = r[j], r[i] }
+func (r byNetworkName) Less(i, j int) bool { return r[i].Name < r[j].Name }
+
// CmdNetworkInspect inspects the network object for more details
//
// Usage: docker network inspect [OPTIONS] [NETWORK...]
diff --git a/components/engine/api/client/pull.go b/components/engine/api/client/pull.go
index cd15caaa4f..29d9677e95 100644
--- a/components/engine/api/client/pull.go
+++ b/components/engine/api/client/pull.go
@@ -56,7 +56,7 @@ func (cli *DockerCli) CmdPull(args ...string) error {
return err
}
- authConfig := cli.resolveAuthConfig(cli.configFile.AuthConfigs, repoInfo.Index)
+ authConfig := cli.resolveAuthConfig(repoInfo.Index)
requestPrivilege := cli.registryAuthenticationPrivilegedFunc(repoInfo.Index, "pull")
if isTrusted() && !ref.HasDigest() {
diff --git a/components/engine/api/client/push.go b/components/engine/api/client/push.go
index f06a9892ef..29f26c4673 100644
--- a/components/engine/api/client/push.go
+++ b/components/engine/api/client/push.go
@@ -44,7 +44,7 @@ func (cli *DockerCli) CmdPush(args ...string) error {
return err
}
// Resolve the Auth config relevant for this server
- authConfig := cli.resolveAuthConfig(cli.configFile.AuthConfigs, repoInfo.Index)
+ authConfig := cli.resolveAuthConfig(repoInfo.Index)
requestPrivilege := cli.registryAuthenticationPrivilegedFunc(repoInfo.Index, "push")
if isTrusted() {
diff --git a/components/engine/api/client/run.go b/components/engine/api/client/run.go
index 223964d595..b65a0a1bbe 100644
--- a/components/engine/api/client/run.go
+++ b/components/engine/api/client/run.go
@@ -11,7 +11,6 @@ import (
"github.com/Sirupsen/logrus"
Cli "github.com/docker/docker/cli"
- derr "github.com/docker/docker/errors"
"github.com/docker/docker/opts"
"github.com/docker/docker/pkg/promise"
"github.com/docker/docker/pkg/signal"
@@ -21,6 +20,11 @@ import (
"github.com/docker/libnetwork/resolvconf/dns"
)
+const (
+ errCmdNotFound = "Container command not found or does not exist."
+ errCmdCouldNotBeInvoked = "Container command could not be invoked."
+)
+
func (cid *cidFile) Close() error {
cid.file.Close()
@@ -46,20 +50,13 @@ func (cid *cidFile) Write(id string) error {
// return 125 for generic docker daemon failures
func runStartContainerErr(err error) error {
trimmedErr := strings.Trim(err.Error(), "Error response from daemon: ")
- statusError := Cli.StatusError{}
- derrCmdNotFound := derr.ErrorCodeCmdNotFound.Message()
- derrCouldNotInvoke := derr.ErrorCodeCmdCouldNotBeInvoked.Message()
- derrNoSuchImage := derr.ErrorCodeNoSuchImageHash.Message()
- derrNoSuchImageTag := derr.ErrorCodeNoSuchImageTag.Message()
+ statusError := Cli.StatusError{StatusCode: 125}
+
switch trimmedErr {
- case derrCmdNotFound:
+ case errCmdNotFound:
statusError = Cli.StatusError{StatusCode: 127}
- case derrCouldNotInvoke:
+ case errCmdCouldNotBeInvoked:
statusError = Cli.StatusError{StatusCode: 126}
- case derrNoSuchImage, derrNoSuchImageTag:
- statusError = Cli.StatusError{StatusCode: 125}
- default:
- statusError = Cli.StatusError{StatusCode: 125}
}
return statusError
}
diff --git a/components/engine/api/client/search.go b/components/engine/api/client/search.go
index 2e1fcdafd3..c7196a74cf 100644
--- a/components/engine/api/client/search.go
+++ b/components/engine/api/client/search.go
@@ -36,7 +36,7 @@ func (cli *DockerCli) CmdSearch(args ...string) error {
return err
}
- authConfig := cli.resolveAuthConfig(cli.configFile.AuthConfigs, indexInfo)
+ authConfig := cli.resolveAuthConfig(indexInfo)
requestPrivilege := cli.registryAuthenticationPrivilegedFunc(indexInfo, "search")
encodedAuth, err := encodeAuthToBase64(authConfig)
diff --git a/components/engine/api/client/stats.go b/components/engine/api/client/stats.go
index 0bfbe43f9d..3b9b8d8d48 100644
--- a/components/engine/api/client/stats.go
+++ b/components/engine/api/client/stats.go
@@ -1,10 +1,8 @@
package client
import (
- "encoding/json"
"fmt"
"io"
- "sort"
"strings"
"sync"
"text/tabwriter"
@@ -16,125 +14,8 @@ import (
"github.com/docker/engine-api/types"
"github.com/docker/engine-api/types/events"
"github.com/docker/engine-api/types/filters"
- "github.com/docker/go-units"
)
-type containerStats struct {
- Name string
- CPUPercentage float64
- Memory float64
- MemoryLimit float64
- MemoryPercentage float64
- NetworkRx float64
- NetworkTx float64
- BlockRead float64
- BlockWrite float64
- mu sync.RWMutex
- err error
-}
-
-type stats struct {
- mu sync.Mutex
- cs []*containerStats
-}
-
-func (s *containerStats) Collect(cli *DockerCli, streamStats bool) {
- responseBody, err := cli.client.ContainerStats(context.Background(), s.Name, streamStats)
- if err != nil {
- s.mu.Lock()
- s.err = err
- s.mu.Unlock()
- return
- }
- defer responseBody.Close()
-
- var (
- previousCPU uint64
- previousSystem uint64
- dec = json.NewDecoder(responseBody)
- u = make(chan error, 1)
- )
- go func() {
- for {
- var v *types.StatsJSON
- if err := dec.Decode(&v); err != nil {
- u <- err
- return
- }
-
- var memPercent = 0.0
- var cpuPercent = 0.0
-
- // MemoryStats.Limit will never be 0 unless the container is not running and we haven't
- // got any data from cgroup
- if v.MemoryStats.Limit != 0 {
- memPercent = float64(v.MemoryStats.Usage) / float64(v.MemoryStats.Limit) * 100.0
- }
-
- previousCPU = v.PreCPUStats.CPUUsage.TotalUsage
- previousSystem = v.PreCPUStats.SystemUsage
- cpuPercent = calculateCPUPercent(previousCPU, previousSystem, v)
- blkRead, blkWrite := calculateBlockIO(v.BlkioStats)
- s.mu.Lock()
- s.CPUPercentage = cpuPercent
- s.Memory = float64(v.MemoryStats.Usage)
- s.MemoryLimit = float64(v.MemoryStats.Limit)
- s.MemoryPercentage = memPercent
- s.NetworkRx, s.NetworkTx = calculateNetwork(v.Networks)
- s.BlockRead = float64(blkRead)
- s.BlockWrite = float64(blkWrite)
- s.mu.Unlock()
- u <- nil
- if !streamStats {
- return
- }
- }
- }()
- for {
- select {
- case <-time.After(2 * time.Second):
- // zero out the values if we have not received an update within
- // the specified duration.
- s.mu.Lock()
- s.CPUPercentage = 0
- s.Memory = 0
- s.MemoryPercentage = 0
- s.MemoryLimit = 0
- s.NetworkRx = 0
- s.NetworkTx = 0
- s.BlockRead = 0
- s.BlockWrite = 0
- s.mu.Unlock()
- case err := <-u:
- if err != nil {
- s.mu.Lock()
- s.err = err
- s.mu.Unlock()
- return
- }
- }
- if !streamStats {
- return
- }
- }
-}
-
-func (s *containerStats) Display(w io.Writer) error {
- s.mu.RLock()
- defer s.mu.RUnlock()
- if s.err != nil {
- return s.err
- }
- fmt.Fprintf(w, "%s\t%.2f%%\t%s / %s\t%.2f%%\t%s / %s\t%s / %s\n",
- s.Name,
- s.CPUPercentage,
- units.HumanSize(s.Memory), units.HumanSize(s.MemoryLimit),
- s.MemoryPercentage,
- units.HumanSize(s.NetworkRx), units.HumanSize(s.NetworkTx),
- units.HumanSize(s.BlockRead), units.HumanSize(s.BlockWrite))
- return nil
-}
-
// CmdStats displays a live stream of resource usage statistics for one or more containers.
//
// This shows real-time information on CPU usage, memory usage, and network I/O.
@@ -149,125 +30,143 @@ func (cli *DockerCli) CmdStats(args ...string) error {
names := cmd.Args()
showAll := len(names) == 0
+ closeChan := make(chan error)
- if showAll {
+ // monitorContainerEvents watches for container creation and removal (only
+ // used when calling `docker stats` without arguments).
+ monitorContainerEvents := func(started chan<- struct{}, c chan events.Message) {
+ f := filters.NewArgs()
+ f.Add("type", "container")
+ options := types.EventsOptions{
+ Filters: f,
+ }
+ resBody, err := cli.client.Events(context.Background(), options)
+ // Whether we successfully subscribed to events or not, we can now
+ // unblock the main goroutine.
+ close(started)
+ if err != nil {
+ closeChan <- err
+ return
+ }
+ defer resBody.Close()
+
+ decodeEvents(resBody, func(event events.Message, err error) error {
+ if err != nil {
+ closeChan <- err
+ return nil
+ }
+ c <- event
+ return nil
+ })
+ }
+
+ // waitFirst is a WaitGroup to wait first stat data's reach for each container
+ waitFirst := &sync.WaitGroup{}
+
+ cStats := stats{}
+ // getContainerList simulates creation event for all previously existing
+ // containers (only used when calling `docker stats` without arguments).
+ getContainerList := func() {
options := types.ContainerListOptions{
All: *all,
}
cs, err := cli.client.ContainerList(options)
if err != nil {
- return err
+ closeChan <- err
}
- for _, c := range cs {
- names = append(names, c.ID[:12])
+ for _, container := range cs {
+ s := &containerStats{Name: container.ID[:12]}
+ if cStats.add(s) {
+ waitFirst.Add(1)
+ go s.Collect(cli.client, !*noStream, waitFirst)
+ }
}
}
- if len(names) == 0 && !showAll {
- return fmt.Errorf("No containers found")
- }
- sort.Strings(names)
- var (
- cStats = stats{}
- w = tabwriter.NewWriter(cli.out, 20, 1, 3, ' ', 0)
- )
+ if showAll {
+ // If no names were specified, start a long running goroutine which
+ // monitors container events. We make sure we're subscribed before
+ // retrieving the list of running containers to avoid a race where we
+ // would "miss" a creation.
+ started := make(chan struct{})
+ eh := eventHandler{handlers: make(map[string]func(events.Message))}
+ eh.Handle("create", func(e events.Message) {
+ if *all {
+ s := &containerStats{Name: e.ID[:12]}
+ if cStats.add(s) {
+ waitFirst.Add(1)
+ go s.Collect(cli.client, !*noStream, waitFirst)
+ }
+ }
+ })
+
+ eh.Handle("start", func(e events.Message) {
+ s := &containerStats{Name: e.ID[:12]}
+ if cStats.add(s) {
+ waitFirst.Add(1)
+ go s.Collect(cli.client, !*noStream, waitFirst)
+ }
+ })
+
+ eh.Handle("die", func(e events.Message) {
+ if !*all {
+ cStats.remove(e.ID[:12])
+ }
+ })
+
+ eventChan := make(chan events.Message)
+ go eh.Watch(eventChan)
+ go monitorContainerEvents(started, eventChan)
+ defer close(eventChan)
+ <-started
+
+ // Start a short-lived goroutine to retrieve the initial list of
+ // containers.
+ getContainerList()
+ } else {
+ // Artificially send creation events for the containers we were asked to
+ // monitor (same code path than we use when monitoring all containers).
+ for _, name := range names {
+ s := &containerStats{Name: name}
+ if cStats.add(s) {
+ waitFirst.Add(1)
+ go s.Collect(cli.client, !*noStream, waitFirst)
+ }
+ }
+
+ // We don't expect any asynchronous errors: closeChan can be closed.
+ close(closeChan)
+
+ // Do a quick pause to detect any error with the provided list of
+ // container names.
+ time.Sleep(1500 * time.Millisecond)
+ var errs []string
+ cStats.mu.Lock()
+ for _, c := range cStats.cs {
+ c.mu.Lock()
+ if c.err != nil {
+ errs = append(errs, fmt.Sprintf("%s: %v", c.Name, c.err))
+ }
+ c.mu.Unlock()
+ }
+ cStats.mu.Unlock()
+ if len(errs) > 0 {
+ return fmt.Errorf("%s", strings.Join(errs, ", "))
+ }
+ }
+
+ // before print to screen, make sure each container get at least one valid stat data
+ waitFirst.Wait()
+
+ w := tabwriter.NewWriter(cli.out, 20, 1, 3, ' ', 0)
printHeader := func() {
if !*noStream {
fmt.Fprint(cli.out, "\033[2J")
fmt.Fprint(cli.out, "\033[H")
}
- io.WriteString(w, "CONTAINER\tCPU %\tMEM USAGE / LIMIT\tMEM %\tNET I/O\tBLOCK I/O\n")
+ io.WriteString(w, "CONTAINER\tCPU %\tMEM USAGE / LIMIT\tMEM %\tNET I/O\tBLOCK I/O\tPIDS\n")
}
- for _, n := range names {
- s := &containerStats{Name: n}
- // no need to lock here since only the main goroutine is running here
- cStats.cs = append(cStats.cs, s)
- go s.Collect(cli, !*noStream)
- }
- closeChan := make(chan error)
- if showAll {
- type watch struct {
- cid string
- event string
- err error
- }
- getNewContainers := func(c chan<- watch) {
- f := filters.NewArgs()
- f.Add("type", "container")
- options := types.EventsOptions{
- Filters: f,
- }
- resBody, err := cli.client.Events(context.Background(), options)
- if err != nil {
- c <- watch{err: err}
- return
- }
- defer resBody.Close()
- decodeEvents(resBody, func(event events.Message, err error) error {
- if err != nil {
- c <- watch{err: err}
- return nil
- }
-
- c <- watch{event.ID[:12], event.Action, nil}
- return nil
- })
- }
- go func(stopChan chan<- error) {
- cChan := make(chan watch)
- go getNewContainers(cChan)
- for {
- c := <-cChan
- if c.err != nil {
- stopChan <- c.err
- return
- }
- switch c.event {
- case "create":
- s := &containerStats{Name: c.cid}
- cStats.mu.Lock()
- cStats.cs = append(cStats.cs, s)
- cStats.mu.Unlock()
- go s.Collect(cli, !*noStream)
- case "stop":
- case "die":
- if !*all {
- var remove int
- // cStats cannot be O(1) with a map cause ranging over it would cause
- // containers in stats to move up and down in the list...:(
- cStats.mu.Lock()
- for i, s := range cStats.cs {
- if s.Name == c.cid {
- remove = i
- break
- }
- }
- cStats.cs = append(cStats.cs[:remove], cStats.cs[remove+1:]...)
- cStats.mu.Unlock()
- }
- }
- }
- }(closeChan)
- } else {
- close(closeChan)
- }
- // do a quick pause so that any failed connections for containers that do not exist are able to be
- // evicted before we display the initial or default values.
- time.Sleep(1500 * time.Millisecond)
- var errs []string
- cStats.mu.Lock()
- for _, c := range cStats.cs {
- c.mu.Lock()
- if c.err != nil {
- errs = append(errs, fmt.Sprintf("%s: %v", c.Name, c.err))
- }
- c.mu.Unlock()
- }
- cStats.mu.Unlock()
- if len(errs) > 0 {
- return fmt.Errorf("%s", strings.Join(errs, ", "))
- }
for range time.Tick(500 * time.Millisecond) {
printHeader()
toRemove := []int{}
@@ -307,40 +206,3 @@ func (cli *DockerCli) CmdStats(args ...string) error {
}
return nil
}
-
-func calculateCPUPercent(previousCPU, previousSystem uint64, v *types.StatsJSON) float64 {
- var (
- cpuPercent = 0.0
- // calculate the change for the cpu usage of the container in between readings
- cpuDelta = float64(v.CPUStats.CPUUsage.TotalUsage) - float64(previousCPU)
- // calculate the change for the entire system between readings
- systemDelta = float64(v.CPUStats.SystemUsage) - float64(previousSystem)
- )
-
- if systemDelta > 0.0 && cpuDelta > 0.0 {
- cpuPercent = (cpuDelta / systemDelta) * float64(len(v.CPUStats.CPUUsage.PercpuUsage)) * 100.0
- }
- return cpuPercent
-}
-
-func calculateBlockIO(blkio types.BlkioStats) (blkRead uint64, blkWrite uint64) {
- for _, bioEntry := range blkio.IoServiceBytesRecursive {
- switch strings.ToLower(bioEntry.Op) {
- case "read":
- blkRead = blkRead + bioEntry.Value
- case "write":
- blkWrite = blkWrite + bioEntry.Value
- }
- }
- return
-}
-
-func calculateNetwork(network map[string]types.NetworkStats) (float64, float64) {
- var rx, tx float64
-
- for _, v := range network {
- rx += float64(v.RxBytes)
- tx += float64(v.TxBytes)
- }
- return rx, tx
-}
diff --git a/components/engine/api/client/stats_helpers.go b/components/engine/api/client/stats_helpers.go
new file mode 100644
index 0000000000..5c88b7424d
--- /dev/null
+++ b/components/engine/api/client/stats_helpers.go
@@ -0,0 +1,217 @@
+package client
+
+import (
+ "encoding/json"
+ "fmt"
+ "io"
+ "strings"
+ "sync"
+ "time"
+
+ "github.com/docker/engine-api/client"
+ "github.com/docker/engine-api/types"
+ "github.com/docker/go-units"
+ "golang.org/x/net/context"
+)
+
+type containerStats struct {
+ Name string
+ CPUPercentage float64
+ Memory float64
+ MemoryLimit float64
+ MemoryPercentage float64
+ NetworkRx float64
+ NetworkTx float64
+ BlockRead float64
+ BlockWrite float64
+ PidsCurrent uint64
+ mu sync.RWMutex
+ err error
+}
+
+type stats struct {
+ mu sync.Mutex
+ cs []*containerStats
+}
+
+func (s *stats) add(cs *containerStats) bool {
+ s.mu.Lock()
+ defer s.mu.Unlock()
+ if _, exists := s.isKnownContainer(cs.Name); !exists {
+ s.cs = append(s.cs, cs)
+ return true
+ }
+ return false
+}
+
+func (s *stats) remove(id string) {
+ s.mu.Lock()
+ if i, exists := s.isKnownContainer(id); exists {
+ s.cs = append(s.cs[:i], s.cs[i+1:]...)
+ }
+ s.mu.Unlock()
+}
+
+func (s *stats) isKnownContainer(cid string) (int, bool) {
+ for i, c := range s.cs {
+ if c.Name == cid {
+ return i, true
+ }
+ }
+ return -1, false
+}
+
+func (s *containerStats) Collect(cli client.APIClient, streamStats bool, waitFirst *sync.WaitGroup) {
+ var (
+ getFirst bool
+ previousCPU uint64
+ previousSystem uint64
+ u = make(chan error, 1)
+ )
+
+ defer func() {
+ // if error happens and we get nothing of stats, release wait group whatever
+ if !getFirst {
+ getFirst = true
+ waitFirst.Done()
+ }
+ }()
+
+ responseBody, err := cli.ContainerStats(context.Background(), s.Name, streamStats)
+ if err != nil {
+ s.mu.Lock()
+ s.err = err
+ s.mu.Unlock()
+ return
+ }
+ defer responseBody.Close()
+
+ dec := json.NewDecoder(responseBody)
+ go func() {
+ for {
+ var v *types.StatsJSON
+ if err := dec.Decode(&v); err != nil {
+ u <- err
+ return
+ }
+
+ var memPercent = 0.0
+ var cpuPercent = 0.0
+
+ // MemoryStats.Limit will never be 0 unless the container is not running and we haven't
+ // got any data from cgroup
+ if v.MemoryStats.Limit != 0 {
+ memPercent = float64(v.MemoryStats.Usage) / float64(v.MemoryStats.Limit) * 100.0
+ }
+
+ previousCPU = v.PreCPUStats.CPUUsage.TotalUsage
+ previousSystem = v.PreCPUStats.SystemUsage
+ cpuPercent = calculateCPUPercent(previousCPU, previousSystem, v)
+ blkRead, blkWrite := calculateBlockIO(v.BlkioStats)
+ s.mu.Lock()
+ s.CPUPercentage = cpuPercent
+ s.Memory = float64(v.MemoryStats.Usage)
+ s.MemoryLimit = float64(v.MemoryStats.Limit)
+ s.MemoryPercentage = memPercent
+ s.NetworkRx, s.NetworkTx = calculateNetwork(v.Networks)
+ s.BlockRead = float64(blkRead)
+ s.BlockWrite = float64(blkWrite)
+ s.mu.Unlock()
+ u <- nil
+ if !streamStats {
+ return
+ }
+ }
+ }()
+ for {
+ select {
+ case <-time.After(2 * time.Second):
+ // zero out the values if we have not received an update within
+ // the specified duration.
+ s.mu.Lock()
+ s.CPUPercentage = 0
+ s.Memory = 0
+ s.MemoryPercentage = 0
+ s.MemoryLimit = 0
+ s.NetworkRx = 0
+ s.NetworkTx = 0
+ s.BlockRead = 0
+ s.BlockWrite = 0
+ s.mu.Unlock()
+ // if this is the first stat you get, release WaitGroup
+ if !getFirst {
+ getFirst = true
+ waitFirst.Done()
+ }
+ case err := <-u:
+ if err != nil {
+ s.mu.Lock()
+ s.err = err
+ s.mu.Unlock()
+ return
+ }
+ // if this is the first stat you get, release WaitGroup
+ if !getFirst {
+ getFirst = true
+ waitFirst.Done()
+ }
+ }
+ if !streamStats {
+ return
+ }
+ }
+}
+
+func (s *containerStats) Display(w io.Writer) error {
+ s.mu.RLock()
+ defer s.mu.RUnlock()
+ if s.err != nil {
+ return s.err
+ }
+ fmt.Fprintf(w, "%s\t%.2f%%\t%s / %s\t%.2f%%\t%s / %s\t%s / %s\t%d\n",
+ s.Name,
+ s.CPUPercentage,
+ units.HumanSize(s.Memory), units.HumanSize(s.MemoryLimit),
+ s.MemoryPercentage,
+ units.HumanSize(s.NetworkRx), units.HumanSize(s.NetworkTx),
+ units.HumanSize(s.BlockRead), units.HumanSize(s.BlockWrite),
+ s.PidsCurrent)
+ return nil
+}
+
+func calculateCPUPercent(previousCPU, previousSystem uint64, v *types.StatsJSON) float64 {
+ var (
+ cpuPercent = 0.0
+ // calculate the change for the cpu usage of the container in between readings
+ cpuDelta = float64(v.CPUStats.CPUUsage.TotalUsage) - float64(previousCPU)
+ // calculate the change for the entire system between readings
+ systemDelta = float64(v.CPUStats.SystemUsage) - float64(previousSystem)
+ )
+
+ if systemDelta > 0.0 && cpuDelta > 0.0 {
+ cpuPercent = (cpuDelta / systemDelta) * float64(len(v.CPUStats.CPUUsage.PercpuUsage)) * 100.0
+ }
+ return cpuPercent
+}
+
+func calculateBlockIO(blkio types.BlkioStats) (blkRead uint64, blkWrite uint64) {
+ for _, bioEntry := range blkio.IoServiceBytesRecursive {
+ switch strings.ToLower(bioEntry.Op) {
+ case "read":
+ blkRead = blkRead + bioEntry.Value
+ case "write":
+ blkWrite = blkWrite + bioEntry.Value
+ }
+ }
+ return
+}
+
+func calculateNetwork(network map[string]types.NetworkStats) (float64, float64) {
+ var rx, tx float64
+
+ for _, v := range network {
+ rx += float64(v.RxBytes)
+ tx += float64(v.TxBytes)
+ }
+ return rx, tx
+}
diff --git a/components/engine/api/client/stats_unit_test.go b/components/engine/api/client/stats_unit_test.go
index ce1c3a1741..36081c5772 100644
--- a/components/engine/api/client/stats_unit_test.go
+++ b/components/engine/api/client/stats_unit_test.go
@@ -19,6 +19,7 @@ func TestDisplay(t *testing.T) {
NetworkTx: 800 * 1024 * 1024,
BlockRead: 100 * 1024 * 1024,
BlockWrite: 800 * 1024 * 1024,
+ PidsCurrent: 1,
mu: sync.RWMutex{},
}
var b bytes.Buffer
@@ -26,7 +27,7 @@ func TestDisplay(t *testing.T) {
t.Fatalf("c.Display() gave error: %s", err)
}
got := b.String()
- want := "app\t30.00%\t104.9 MB / 2.147 GB\t4.88%\t104.9 MB / 838.9 MB\t104.9 MB / 838.9 MB\n"
+ want := "app\t30.00%\t104.9 MB / 2.147 GB\t4.88%\t104.9 MB / 838.9 MB\t104.9 MB / 838.9 MB\t1\n"
if got != want {
t.Fatalf("c.Display() = %q, want %q", got, want)
}
diff --git a/components/engine/api/client/trust.go b/components/engine/api/client/trust.go
index 753bcd6fc7..88cfb3244b 100644
--- a/components/engine/api/client/trust.go
+++ b/components/engine/api/client/trust.go
@@ -107,7 +107,10 @@ func (scs simpleCredentialStore) Basic(u *url.URL) (string, string) {
return scs.auth.Username, scs.auth.Password
}
-func (cli *DockerCli) getNotaryRepository(repoInfo *registry.RepositoryInfo, authConfig types.AuthConfig) (*client.NotaryRepository, error) {
+// getNotaryRepository returns a NotaryRepository which stores all the
+// information needed to operate on a notary repository.
+// It creates a HTTP transport providing authentication support.
+func (cli *DockerCli) getNotaryRepository(repoInfo *registry.RepositoryInfo, authConfig types.AuthConfig, actions ...string) (*client.NotaryRepository, error) {
server, err := trustServer(repoInfo.Index)
if err != nil {
return nil, err
@@ -169,7 +172,7 @@ func (cli *DockerCli) getNotaryRepository(repoInfo *registry.RepositoryInfo, aut
}
creds := simpleCredentialStore{auth: authConfig}
- tokenHandler := auth.NewTokenHandler(authTransport, creds, repoInfo.FullName(), "push", "pull")
+ tokenHandler := auth.NewTokenHandler(authTransport, creds, repoInfo.FullName(), actions...)
basicHandler := auth.NewBasicHandler(creds)
modifiers = append(modifiers, transport.RequestModifier(auth.NewAuthorizer(challengeManager, tokenHandler, basicHandler)))
tr := transport.NewTransport(base, modifiers...)
@@ -235,7 +238,7 @@ func (cli *DockerCli) trustedReference(ref reference.NamedTagged) (reference.Can
}
// Resolve the Auth config relevant for this server
- authConfig := cli.resolveAuthConfig(cli.configFile.AuthConfigs, repoInfo.Index)
+ authConfig := cli.resolveAuthConfig(repoInfo.Index)
notaryRepo, err := cli.getNotaryRepository(repoInfo, authConfig)
if err != nil {
@@ -302,7 +305,7 @@ func notaryError(repoName string, err error) error {
func (cli *DockerCli) trustedPull(repoInfo *registry.RepositoryInfo, ref registry.Reference, authConfig types.AuthConfig, requestPrivilege apiclient.RequestPrivilegeFunc) error {
var refs []target
- notaryRepo, err := cli.getNotaryRepository(repoInfo, authConfig)
+ notaryRepo, err := cli.getNotaryRepository(repoInfo, authConfig, "pull")
if err != nil {
fmt.Fprintf(cli.out, "Error establishing connection to trust repository: %s\n", err)
return err
@@ -372,60 +375,74 @@ func (cli *DockerCli) trustedPush(repoInfo *registry.RepositoryInfo, tag string,
defer responseBody.Close()
- targets := []target{}
+ // If it is a trusted push we would like to find the target entry which match the
+ // tag provided in the function and then do an AddTarget later.
+ target := &client.Target{}
+ // Count the times of calling for handleTarget,
+ // if it is called more that once, that should be considered an error in a trusted push.
+ cnt := 0
handleTarget := func(aux *json.RawMessage) {
+ cnt++
+ if cnt > 1 {
+ // handleTarget should only be called one. This will be treated as an error.
+ return
+ }
+
var pushResult distribution.PushResult
err := json.Unmarshal(*aux, &pushResult)
if err == nil && pushResult.Tag != "" && pushResult.Digest.Validate() == nil {
- targets = append(targets, target{
- reference: registry.ParseReference(pushResult.Tag),
- digest: pushResult.Digest,
- size: int64(pushResult.Size),
- })
+ h, err := hex.DecodeString(pushResult.Digest.Hex())
+ if err != nil {
+ target = nil
+ return
+ }
+ target.Name = registry.ParseReference(pushResult.Tag).String()
+ target.Hashes = data.Hashes{string(pushResult.Digest.Algorithm()): h}
+ target.Length = int64(pushResult.Size)
}
}
- err = jsonmessage.DisplayJSONMessagesStream(responseBody, cli.out, cli.outFd, cli.isTerminalOut, handleTarget)
- if err != nil {
+ // We want trust signatures to always take an explicit tag,
+ // otherwise it will act as an untrusted push.
+ if tag == "" {
+ if err = jsonmessage.DisplayJSONMessagesStream(responseBody, cli.out, cli.outFd, cli.isTerminalOut, nil); err != nil {
+ return err
+ }
+ fmt.Fprintln(cli.out, "No tag specified, skipping trust metadata push")
+ return nil
+ }
+
+ if err = jsonmessage.DisplayJSONMessagesStream(responseBody, cli.out, cli.outFd, cli.isTerminalOut, handleTarget); err != nil {
return err
}
- if tag == "" {
- fmt.Fprintf(cli.out, "No tag specified, skipping trust metadata push\n")
- return nil
+ if cnt > 1 {
+ return fmt.Errorf("internal error: only one call to handleTarget expected")
}
- if len(targets) == 0 {
- fmt.Fprintf(cli.out, "No targets found, skipping trust metadata push\n")
+
+ if target == nil {
+ fmt.Fprintln(cli.out, "No targets found, please provide a specific tag in order to sign it")
return nil
}
- fmt.Fprintf(cli.out, "Signing and pushing trust metadata\n")
+ fmt.Fprintln(cli.out, "Signing and pushing trust metadata")
- repo, err := cli.getNotaryRepository(repoInfo, authConfig)
+ repo, err := cli.getNotaryRepository(repoInfo, authConfig, "push", "pull")
if err != nil {
fmt.Fprintf(cli.out, "Error establishing connection to notary repository: %s\n", err)
return err
}
- for _, target := range targets {
- h, err := hex.DecodeString(target.digest.Hex())
- if err != nil {
- return err
- }
- t := &client.Target{
- Name: target.reference.String(),
- Hashes: data.Hashes{
- string(target.digest.Algorithm()): h,
- },
- Length: int64(target.size),
- }
- if err := repo.AddTarget(t, releasesRole); err != nil {
- return err
- }
+ if err := repo.AddTarget(target, releasesRole); err != nil {
+ return err
}
err = repo.Publish()
- if _, ok := err.(client.ErrRepoNotInitialized); !ok {
+ if err == nil {
+ fmt.Fprintf(cli.out, "Successfully signed %q:%s\n", repoInfo.FullName(), tag)
+ return nil
+ } else if _, ok := err.(client.ErrRepoNotInitialized); !ok {
+ fmt.Fprintf(cli.out, "Failed to sign %q:%s - %s\n", repoInfo.FullName(), tag, err.Error())
return notaryError(repoInfo.FullName(), err)
}
@@ -444,7 +461,8 @@ func (cli *DockerCli) trustedPush(repoInfo *registry.RepositoryInfo, tag string,
rootKeyID = rootPublicKey.ID()
}
- if err := repo.Initialize(rootKeyID); err != nil {
+ // Initialize the notary repository with a remotely managed snapshot key
+ if err := repo.Initialize(rootKeyID, data.CanonicalSnapshotRole); err != nil {
return notaryError(repoInfo.FullName(), err)
}
fmt.Fprintf(cli.out, "Finished initializing %q\n", repoInfo.FullName())
diff --git a/components/engine/api/client/update.go b/components/engine/api/client/update.go
index b33ed476a6..0a576e7c73 100644
--- a/components/engine/api/client/update.go
+++ b/components/engine/api/client/update.go
@@ -6,6 +6,7 @@ import (
Cli "github.com/docker/docker/cli"
flag "github.com/docker/docker/pkg/mflag"
+ "github.com/docker/docker/runconfig/opts"
"github.com/docker/engine-api/types/container"
"github.com/docker/go-units"
)
@@ -25,6 +26,7 @@ func (cli *DockerCli) CmdUpdate(args ...string) error {
flMemoryReservation := cmd.String([]string{"-memory-reservation"}, "", "Memory soft limit")
flMemorySwap := cmd.String([]string{"-memory-swap"}, "", "Swap limit equal to memory plus swap: '-1' to enable unlimited swap")
flKernelMemory := cmd.String([]string{"-kernel-memory"}, "", "Kernel memory limit")
+ flRestartPolicy := cmd.String([]string{"-restart"}, "", "Restart policy to apply when a container exits")
cmd.Require(flag.Min, 1)
cmd.ParseFlags(args, true)
@@ -69,6 +71,14 @@ func (cli *DockerCli) CmdUpdate(args ...string) error {
}
}
+ var restartPolicy container.RestartPolicy
+ if *flRestartPolicy != "" {
+ restartPolicy, err = opts.ParseRestartPolicy(*flRestartPolicy)
+ if err != nil {
+ return err
+ }
+ }
+
resources := container.Resources{
BlkioWeight: *flBlkioWeight,
CpusetCpus: *flCpusetCpus,
@@ -83,7 +93,8 @@ func (cli *DockerCli) CmdUpdate(args ...string) error {
}
updateConfig := container.UpdateConfig{
- Resources: resources,
+ Resources: resources,
+ RestartPolicy: restartPolicy,
}
names := cmd.Args()
diff --git a/components/engine/api/client/utils.go b/components/engine/api/client/utils.go
index e87e593fe2..0ea3cb060c 100644
--- a/components/engine/api/client/utils.go
+++ b/components/engine/api/client/utils.go
@@ -10,7 +10,6 @@ import (
gosignal "os/signal"
"path/filepath"
"runtime"
- "strings"
"time"
"github.com/Sirupsen/logrus"
@@ -49,7 +48,7 @@ func (cli *DockerCli) registryAuthenticationPrivilegedFunc(index *registrytypes.
return func() (string, error) {
fmt.Fprintf(cli.out, "\nPlease login prior to %s:\n", cmdName)
indexServer := registry.GetAuthConfigKey(index)
- authConfig, err := cli.configureAuth("", "", "", indexServer)
+ authConfig, err := cli.configureAuth("", "", indexServer, false)
if err != nil {
return "", err
}
@@ -59,6 +58,10 @@ func (cli *DockerCli) registryAuthenticationPrivilegedFunc(index *registrytypes.
func (cli *DockerCli) resizeTty(id string, isExec bool) {
height, width := cli.getTtySize()
+ cli.resizeTtyTo(id, height, width, isExec)
+}
+
+func (cli *DockerCli) resizeTtyTo(id string, height, width int, isExec bool) {
if height == 0 && width == 0 {
return
}
@@ -181,38 +184,17 @@ func copyToFile(outfile string, r io.Reader) error {
// resolveAuthConfig is like registry.ResolveAuthConfig, but if using the
// default index, it uses the default index name for the daemon's platform,
// not the client's platform.
-func (cli *DockerCli) resolveAuthConfig(authConfigs map[string]types.AuthConfig, index *registrytypes.IndexInfo) types.AuthConfig {
+func (cli *DockerCli) resolveAuthConfig(index *registrytypes.IndexInfo) types.AuthConfig {
configKey := index.Name
if index.Official {
configKey = cli.electAuthServer()
}
- // First try the happy case
- if c, found := authConfigs[configKey]; found || index.Official {
- return c
- }
+ a, _ := getCredentials(cli.configFile, configKey)
+ return a
+}
- convertToHostname := func(url string) string {
- stripped := url
- if strings.HasPrefix(url, "http://") {
- stripped = strings.Replace(url, "http://", "", 1)
- } else if strings.HasPrefix(url, "https://") {
- stripped = strings.Replace(url, "https://", "", 1)
- }
-
- nameParts := strings.SplitN(stripped, "/", 2)
-
- return nameParts[0]
- }
-
- // Maybe they have a legacy config file, we will iterate the keys converting
- // them to the new format and testing
- for registry, ac := range authConfigs {
- if configKey == convertToHostname(registry) {
- return ac
- }
- }
-
- // When all else fails, return an empty auth config
- return types.AuthConfig{}
+func (cli *DockerCli) retrieveAuthConfigs() map[string]types.AuthConfig {
+ acs, _ := getAllCredentials(cli.configFile)
+ return acs
}
diff --git a/components/engine/api/client/volume.go b/components/engine/api/client/volume.go
index 284e30c3cb..8bb37951ee 100644
--- a/components/engine/api/client/volume.go
+++ b/components/engine/api/client/volume.go
@@ -2,6 +2,7 @@ package client
import (
"fmt"
+ "sort"
"text/tabwriter"
Cli "github.com/docker/docker/cli"
@@ -72,6 +73,7 @@ func (cli *DockerCli) CmdVolumeLs(args ...string) error {
fmt.Fprintf(w, "\n")
}
+ sort.Sort(byVolumeName(volumes.Volumes))
for _, vol := range volumes.Volumes {
if *quiet {
fmt.Fprintln(w, vol.Name)
@@ -83,6 +85,14 @@ func (cli *DockerCli) CmdVolumeLs(args ...string) error {
return nil
}
+type byVolumeName []*types.Volume
+
+func (r byVolumeName) Len() int { return len(r) }
+func (r byVolumeName) Swap(i, j int) { r[i], r[j] = r[j], r[i] }
+func (r byVolumeName) Less(i, j int) bool {
+ return r[i].Name < r[j].Name
+}
+
// CmdVolumeInspect displays low-level information on one or more volumes.
//
// Usage: docker volume inspect [OPTIONS] VOLUME [VOLUME...]
diff --git a/components/engine/api/common.go b/components/engine/api/common.go
index 51be1e27ed..63560c6dea 100644
--- a/components/engine/api/common.go
+++ b/components/engine/api/common.go
@@ -23,9 +23,6 @@ const (
// MinVersion represents Minimum REST API version supported
MinVersion version.Version = "1.12"
- // DefaultDockerfileName is the Default filename with Docker commands, read by docker build
- DefaultDockerfileName string = "Dockerfile"
-
// NoBaseImageSpecifier is the symbol used by the FROM
// command to specify that no base image is to be used.
NoBaseImageSpecifier string = "scratch"
diff --git a/components/engine/api/common_test.go b/components/engine/api/common_test.go
index 4f36b45471..c214660cc4 100644
--- a/components/engine/api/common_test.go
+++ b/components/engine/api/common_test.go
@@ -310,7 +310,7 @@ func TestLoadOrCreateTrustKeyCreateKey(t *testing.T) {
}
// With the need to create the folder hierarchy as tmpKeyFie is in a path
- // where some folder do not exists.
+ // where some folders do not exist.
tmpKeyFile = filepath.Join(tmpKeyFolderPath, "folder/hierarchy/keyfile")
if key, err := LoadOrCreateTrustKey(tmpKeyFile); err != nil || key == nil {
diff --git a/components/engine/api/server/httputils/errors.go b/components/engine/api/server/httputils/errors.go
new file mode 100644
index 0000000000..c6b0a6b2d8
--- /dev/null
+++ b/components/engine/api/server/httputils/errors.go
@@ -0,0 +1,69 @@
+package httputils
+
+import (
+ "net/http"
+ "strings"
+
+ "github.com/Sirupsen/logrus"
+)
+
+// httpStatusError is an interface
+// that errors with custom status codes
+// implement to tell the api layer
+// which response status to set.
+type httpStatusError interface {
+ HTTPErrorStatusCode() int
+}
+
+// inputValidationError is an interface
+// that errors generated by invalid
+// inputs can implement to tell the
+// api layer to set a 400 status code
+// in the response.
+type inputValidationError interface {
+ IsValidationError() bool
+}
+
+// WriteError decodes a specific docker error and sends it in the response.
+func WriteError(w http.ResponseWriter, err error) {
+ if err == nil || w == nil {
+ logrus.WithFields(logrus.Fields{"error": err, "writer": w}).Error("unexpected HTTP error handling")
+ return
+ }
+
+ var statusCode int
+ errMsg := err.Error()
+
+ switch e := err.(type) {
+ case httpStatusError:
+ statusCode = e.HTTPErrorStatusCode()
+ case inputValidationError:
+ statusCode = http.StatusBadRequest
+ default:
+ // FIXME: this is brittle and should not be necessary, but we still need to identify if
+ // there are errors falling back into this logic.
+ // If we need to differentiate between different possible error types,
+ // we should create appropriate error types that implement the httpStatusError interface.
+ errStr := strings.ToLower(errMsg)
+ for keyword, status := range map[string]int{
+ "not found": http.StatusNotFound,
+ "no such": http.StatusNotFound,
+ "bad parameter": http.StatusBadRequest,
+ "conflict": http.StatusConflict,
+ "impossible": http.StatusNotAcceptable,
+ "wrong login/password": http.StatusUnauthorized,
+ "hasn't been activated": http.StatusForbidden,
+ } {
+ if strings.Contains(errStr, keyword) {
+ statusCode = status
+ break
+ }
+ }
+ }
+
+ if statusCode == 0 {
+ statusCode = http.StatusInternalServerError
+ }
+
+ http.Error(w, errMsg, statusCode)
+}
diff --git a/components/engine/api/server/httputils/httputils.go b/components/engine/api/server/httputils/httputils.go
index ecf26e2a14..c81256b145 100644
--- a/components/engine/api/server/httputils/httputils.go
+++ b/components/engine/api/server/httputils/httputils.go
@@ -9,8 +9,6 @@ import (
"golang.org/x/net/context"
- "github.com/Sirupsen/logrus"
- "github.com/docker/distribution/registry/api/errcode"
"github.com/docker/docker/api"
"github.com/docker/docker/pkg/version"
)
@@ -19,7 +17,7 @@ import (
const APIVersionKey = "api-version"
// APIFunc is an adapter to allow the use of ordinary functions as Docker API endpoints.
-// Any function that has the appropriate signature can be register as a API endpoint (e.g. getVersion).
+// Any function that has the appropriate signature can be registered as a API endpoint (e.g. getVersion).
type APIFunc func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error
// HijackConnection interrupts the http response writer to get the
@@ -77,7 +75,7 @@ func ParseForm(r *http.Request) error {
return nil
}
-// ParseMultipartForm ensure the request form is parsed, even with invalid content types.
+// ParseMultipartForm ensures the request form is parsed, even with invalid content types.
func ParseMultipartForm(r *http.Request) error {
if err := r.ParseMultipartForm(4096); err != nil && !strings.HasPrefix(err.Error(), "mime:") {
return err
@@ -85,78 +83,6 @@ func ParseMultipartForm(r *http.Request) error {
return nil
}
-// WriteError decodes a specific docker error and sends it in the response.
-func WriteError(w http.ResponseWriter, err error) {
- if err == nil || w == nil {
- logrus.WithFields(logrus.Fields{"error": err, "writer": w}).Error("unexpected HTTP error handling")
- return
- }
-
- statusCode := http.StatusInternalServerError
- errMsg := err.Error()
-
- // Based on the type of error we get we need to process things
- // slightly differently to extract the error message.
- // In the 'errcode.*' cases there are two different type of
- // error that could be returned. errocode.ErrorCode is the base
- // type of error object - it is just an 'int' that can then be
- // used as the look-up key to find the message. errorcode.Error
- // extends errorcode.Error by adding error-instance specific
- // data, like 'details' or variable strings to be inserted into
- // the message.
- //
- // Ideally, we should just be able to call err.Error() for all
- // cases but the errcode package doesn't support that yet.
- //
- // Additionally, in both errcode cases, there might be an http
- // status code associated with it, and if so use it.
- switch err.(type) {
- case errcode.ErrorCode:
- daError, _ := err.(errcode.ErrorCode)
- statusCode = daError.Descriptor().HTTPStatusCode
- errMsg = daError.Message()
-
- case errcode.Error:
- // For reference, if you're looking for a particular error
- // then you can do something like :
- // import ( derr "github.com/docker/docker/errors" )
- // if daError.ErrorCode() == derr.ErrorCodeNoSuchContainer { ... }
-
- daError, _ := err.(errcode.Error)
- statusCode = daError.ErrorCode().Descriptor().HTTPStatusCode
- errMsg = daError.Message
-
- default:
- // This part of will be removed once we've
- // converted everything over to use the errcode package
-
- // FIXME: this is brittle and should not be necessary.
- // If we need to differentiate between different possible error types,
- // we should create appropriate error types with clearly defined meaning
- errStr := strings.ToLower(err.Error())
- for keyword, status := range map[string]int{
- "not found": http.StatusNotFound,
- "no such": http.StatusNotFound,
- "bad parameter": http.StatusBadRequest,
- "conflict": http.StatusConflict,
- "impossible": http.StatusNotAcceptable,
- "wrong login/password": http.StatusUnauthorized,
- "hasn't been activated": http.StatusForbidden,
- } {
- if strings.Contains(errStr, keyword) {
- statusCode = status
- break
- }
- }
- }
-
- if statusCode == 0 {
- statusCode = http.StatusInternalServerError
- }
-
- http.Error(w, errMsg, statusCode)
-}
-
// WriteJSON writes the value v to the http response stream as json with standard json encoding.
func WriteJSON(w http.ResponseWriter, code int, v interface{}) error {
w.Header().Set("Content-Type", "application/json")
diff --git a/components/engine/api/server/middleware.go b/components/engine/api/server/middleware.go
index 5b277bc1f3..2622bf1bbe 100644
--- a/components/engine/api/server/middleware.go
+++ b/components/engine/api/server/middleware.go
@@ -1,195 +1,41 @@
package server
import (
- "bufio"
- "encoding/json"
- "io"
- "net/http"
- "runtime"
- "strings"
-
"github.com/Sirupsen/logrus"
"github.com/docker/docker/api"
"github.com/docker/docker/api/server/httputils"
+ "github.com/docker/docker/api/server/middleware"
"github.com/docker/docker/dockerversion"
- "github.com/docker/docker/errors"
"github.com/docker/docker/pkg/authorization"
- "github.com/docker/docker/pkg/ioutils"
- "github.com/docker/docker/pkg/version"
- "golang.org/x/net/context"
)
-// middleware is an adapter to allow the use of ordinary functions as Docker API filters.
-// Any function that has the appropriate signature can be register as a middleware.
-type middleware func(handler httputils.APIFunc) httputils.APIFunc
-
-// debugRequestMiddleware dumps the request to logger
-func debugRequestMiddleware(handler httputils.APIFunc) httputils.APIFunc {
- return func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
- logrus.Debugf("%s %s", r.Method, r.RequestURI)
-
- if r.Method != "POST" {
- return handler(ctx, w, r, vars)
- }
- if err := httputils.CheckForJSON(r); err != nil {
- return handler(ctx, w, r, vars)
- }
- maxBodySize := 4096 // 4KB
- if r.ContentLength > int64(maxBodySize) {
- return handler(ctx, w, r, vars)
- }
-
- body := r.Body
- bufReader := bufio.NewReaderSize(body, maxBodySize)
- r.Body = ioutils.NewReadCloserWrapper(bufReader, func() error { return body.Close() })
-
- b, err := bufReader.Peek(maxBodySize)
- if err != io.EOF {
- // either there was an error reading, or the buffer is full (in which case the request is too large)
- return handler(ctx, w, r, vars)
- }
-
- var postForm map[string]interface{}
- if err := json.Unmarshal(b, &postForm); err == nil {
- if _, exists := postForm["password"]; exists {
- postForm["password"] = "*****"
- }
- formStr, errMarshal := json.Marshal(postForm)
- if errMarshal == nil {
- logrus.Debugf("form data: %s", string(formStr))
- } else {
- logrus.Debugf("form data: %q", postForm)
- }
- }
-
- return handler(ctx, w, r, vars)
- }
-}
-
-// authorizationMiddleware perform authorization on the request.
-func (s *Server) authorizationMiddleware(handler httputils.APIFunc) httputils.APIFunc {
- return func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
- // FIXME: fill when authN gets in
- // User and UserAuthNMethod are taken from AuthN plugins
- // Currently tracked in https://github.com/docker/docker/pull/13994
- user := ""
- userAuthNMethod := ""
- authCtx := authorization.NewCtx(s.authZPlugins, user, userAuthNMethod, r.Method, r.RequestURI)
-
- if err := authCtx.AuthZRequest(w, r); err != nil {
- logrus.Errorf("AuthZRequest for %s %s returned error: %s", r.Method, r.RequestURI, err)
- return err
- }
-
- rw := authorization.NewResponseModifier(w)
-
- if err := handler(ctx, rw, r, vars); err != nil {
- logrus.Errorf("Handler for %s %s returned error: %s", r.Method, r.RequestURI, err)
- return err
- }
-
- if err := authCtx.AuthZResponse(rw, r); err != nil {
- logrus.Errorf("AuthZResponse for %s %s returned error: %s", r.Method, r.RequestURI, err)
- return err
- }
- return nil
- }
-}
-
-// userAgentMiddleware checks the User-Agent header looking for a valid docker client spec.
-func (s *Server) userAgentMiddleware(handler httputils.APIFunc) httputils.APIFunc {
- return func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
- if strings.Contains(r.Header.Get("User-Agent"), "Docker-Client/") {
- dockerVersion := version.Version(s.cfg.Version)
-
- userAgent := strings.Split(r.Header.Get("User-Agent"), "/")
-
- // v1.20 onwards includes the GOOS of the client after the version
- // such as Docker/1.7.0 (linux)
- if len(userAgent) == 2 && strings.Contains(userAgent[1], " ") {
- userAgent[1] = strings.Split(userAgent[1], " ")[0]
- }
-
- if len(userAgent) == 2 && !dockerVersion.Equal(version.Version(userAgent[1])) {
- logrus.Warnf("Warning: client and server don't have the same version (client: %s, server: %s)", userAgent[1], dockerVersion)
- }
- }
- return handler(ctx, w, r, vars)
- }
-}
-
-// corsMiddleware sets the CORS header expectations in the server.
-func (s *Server) corsMiddleware(handler httputils.APIFunc) httputils.APIFunc {
- return func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
- // If "api-cors-header" is not given, but "api-enable-cors" is true, we set cors to "*"
- // otherwise, all head values will be passed to HTTP handler
- corsHeaders := s.cfg.CorsHeaders
- if corsHeaders == "" && s.cfg.EnableCors {
- corsHeaders = "*"
- }
-
- if corsHeaders != "" {
- writeCorsHeaders(w, r, corsHeaders)
- }
- return handler(ctx, w, r, vars)
- }
-}
-
-// versionMiddleware checks the api version requirements before passing the request to the server handler.
-func versionMiddleware(handler httputils.APIFunc) httputils.APIFunc {
- return func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
- apiVersion := version.Version(vars["version"])
- if apiVersion == "" {
- apiVersion = api.DefaultVersion
- }
-
- if apiVersion.GreaterThan(api.DefaultVersion) {
- return errors.ErrorCodeNewerClientVersion.WithArgs(apiVersion, api.DefaultVersion)
- }
- if apiVersion.LessThan(api.MinVersion) {
- return errors.ErrorCodeOldClientVersion.WithArgs(apiVersion, api.MinVersion)
- }
-
- w.Header().Set("Server", "Docker/"+dockerversion.Version+" ("+runtime.GOOS+")")
- ctx = context.WithValue(ctx, httputils.APIVersionKey, apiVersion)
- return handler(ctx, w, r, vars)
- }
-}
-
// handleWithGlobalMiddlwares wraps the handler function for a request with
// the server's global middlewares. The order of the middlewares is backwards,
// meaning that the first in the list will be evaluated last.
-//
-// Example: handleWithGlobalMiddlewares(s.getContainersName)
-//
-// s.loggingMiddleware(
-// s.userAgentMiddleware(
-// s.corsMiddleware(
-// versionMiddleware(s.getContainersName)
-// )
-// )
-// )
-// )
func (s *Server) handleWithGlobalMiddlewares(handler httputils.APIFunc) httputils.APIFunc {
- middlewares := []middleware{
- versionMiddleware,
- s.corsMiddleware,
- s.userAgentMiddleware,
+ next := handler
+
+ handleVersion := middleware.NewVersionMiddleware(dockerversion.Version, api.DefaultVersion, api.MinVersion)
+ next = handleVersion(next)
+
+ if s.cfg.EnableCors {
+ handleCORS := middleware.NewCORSMiddleware(s.cfg.CorsHeaders)
+ next = handleCORS(next)
}
+ handleUserAgent := middleware.NewUserAgentMiddleware(s.cfg.Version)
+ next = handleUserAgent(next)
+
// Only want this on debug level
if s.cfg.Logging && logrus.GetLevel() == logrus.DebugLevel {
- middlewares = append(middlewares, debugRequestMiddleware)
+ next = middleware.DebugRequestMiddleware(next)
}
if len(s.cfg.AuthorizationPluginNames) > 0 {
s.authZPlugins = authorization.NewPlugins(s.cfg.AuthorizationPluginNames)
- middlewares = append(middlewares, s.authorizationMiddleware)
+ handleAuthorization := middleware.NewAuthorizationMiddleware(s.authZPlugins)
+ next = handleAuthorization(next)
}
- h := handler
- for _, m := range middlewares {
- h = m(h)
- }
- return h
+ return next
}
diff --git a/components/engine/api/server/middleware/authorization.go b/components/engine/api/server/middleware/authorization.go
new file mode 100644
index 0000000000..cbfa99e7b3
--- /dev/null
+++ b/components/engine/api/server/middleware/authorization.go
@@ -0,0 +1,42 @@
+package middleware
+
+import (
+ "net/http"
+
+ "github.com/Sirupsen/logrus"
+ "github.com/docker/docker/api/server/httputils"
+ "github.com/docker/docker/pkg/authorization"
+ "golang.org/x/net/context"
+)
+
+// NewAuthorizationMiddleware creates a new Authorization middleware.
+func NewAuthorizationMiddleware(plugins []authorization.Plugin) Middleware {
+ return func(handler httputils.APIFunc) httputils.APIFunc {
+ return func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
+ // FIXME: fill when authN gets in
+ // User and UserAuthNMethod are taken from AuthN plugins
+ // Currently tracked in https://github.com/docker/docker/pull/13994
+ user := ""
+ userAuthNMethod := ""
+ authCtx := authorization.NewCtx(plugins, user, userAuthNMethod, r.Method, r.RequestURI)
+
+ if err := authCtx.AuthZRequest(w, r); err != nil {
+ logrus.Errorf("AuthZRequest for %s %s returned error: %s", r.Method, r.RequestURI, err)
+ return err
+ }
+
+ rw := authorization.NewResponseModifier(w)
+
+ if err := handler(ctx, rw, r, vars); err != nil {
+ logrus.Errorf("Handler for %s %s returned error: %s", r.Method, r.RequestURI, err)
+ return err
+ }
+
+ if err := authCtx.AuthZResponse(rw, r); err != nil {
+ logrus.Errorf("AuthZResponse for %s %s returned error: %s", r.Method, r.RequestURI, err)
+ return err
+ }
+ return nil
+ }
+ }
+}
diff --git a/components/engine/api/server/middleware/cors.go b/components/engine/api/server/middleware/cors.go
new file mode 100644
index 0000000000..de21897d2c
--- /dev/null
+++ b/components/engine/api/server/middleware/cors.go
@@ -0,0 +1,33 @@
+package middleware
+
+import (
+ "net/http"
+
+ "github.com/Sirupsen/logrus"
+ "github.com/docker/docker/api/server/httputils"
+ "golang.org/x/net/context"
+)
+
+// NewCORSMiddleware creates a new CORS middleware.
+func NewCORSMiddleware(defaultHeaders string) Middleware {
+ return func(handler httputils.APIFunc) httputils.APIFunc {
+ return func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
+ // If "api-cors-header" is not given, but "api-enable-cors" is true, we set cors to "*"
+ // otherwise, all head values will be passed to HTTP handler
+ corsHeaders := defaultHeaders
+ if corsHeaders == "" {
+ corsHeaders = "*"
+ }
+
+ writeCorsHeaders(w, r, corsHeaders)
+ return handler(ctx, w, r, vars)
+ }
+ }
+}
+
+func writeCorsHeaders(w http.ResponseWriter, r *http.Request, corsHeaders string) {
+ logrus.Debugf("CORS header is enabled and set to: %s", corsHeaders)
+ w.Header().Add("Access-Control-Allow-Origin", corsHeaders)
+ w.Header().Add("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept, X-Registry-Auth")
+ w.Header().Add("Access-Control-Allow-Methods", "HEAD, GET, POST, DELETE, PUT, OPTIONS")
+}
diff --git a/components/engine/api/server/middleware/debug.go b/components/engine/api/server/middleware/debug.go
new file mode 100644
index 0000000000..be7056f6c6
--- /dev/null
+++ b/components/engine/api/server/middleware/debug.go
@@ -0,0 +1,56 @@
+package middleware
+
+import (
+ "bufio"
+ "encoding/json"
+ "io"
+ "net/http"
+
+ "github.com/Sirupsen/logrus"
+ "github.com/docker/docker/api/server/httputils"
+ "github.com/docker/docker/pkg/ioutils"
+ "golang.org/x/net/context"
+)
+
+// DebugRequestMiddleware dumps the request to logger
+func DebugRequestMiddleware(handler httputils.APIFunc) httputils.APIFunc {
+ return func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
+ logrus.Debugf("Calling %s %s", r.Method, r.RequestURI)
+
+ if r.Method != "POST" {
+ return handler(ctx, w, r, vars)
+ }
+ if err := httputils.CheckForJSON(r); err != nil {
+ return handler(ctx, w, r, vars)
+ }
+ maxBodySize := 4096 // 4KB
+ if r.ContentLength > int64(maxBodySize) {
+ return handler(ctx, w, r, vars)
+ }
+
+ body := r.Body
+ bufReader := bufio.NewReaderSize(body, maxBodySize)
+ r.Body = ioutils.NewReadCloserWrapper(bufReader, func() error { return body.Close() })
+
+ b, err := bufReader.Peek(maxBodySize)
+ if err != io.EOF {
+ // either there was an error reading, or the buffer is full (in which case the request is too large)
+ return handler(ctx, w, r, vars)
+ }
+
+ var postForm map[string]interface{}
+ if err := json.Unmarshal(b, &postForm); err == nil {
+ if _, exists := postForm["password"]; exists {
+ postForm["password"] = "*****"
+ }
+ formStr, errMarshal := json.Marshal(postForm)
+ if errMarshal == nil {
+ logrus.Debugf("form data: %s", string(formStr))
+ } else {
+ logrus.Debugf("form data: %q", postForm)
+ }
+ }
+
+ return handler(ctx, w, r, vars)
+ }
+}
diff --git a/components/engine/api/server/middleware/middleware.go b/components/engine/api/server/middleware/middleware.go
new file mode 100644
index 0000000000..588331ae7e
--- /dev/null
+++ b/components/engine/api/server/middleware/middleware.go
@@ -0,0 +1,7 @@
+package middleware
+
+import "github.com/docker/docker/api/server/httputils"
+
+// Middleware is an adapter to allow the use of ordinary functions as Docker API filters.
+// Any function that has the appropriate signature can be registered as a middleware.
+type Middleware func(handler httputils.APIFunc) httputils.APIFunc
diff --git a/components/engine/api/server/middleware/user_agent.go b/components/engine/api/server/middleware/user_agent.go
new file mode 100644
index 0000000000..be4e171cd2
--- /dev/null
+++ b/components/engine/api/server/middleware/user_agent.go
@@ -0,0 +1,35 @@
+package middleware
+
+import (
+ "net/http"
+ "strings"
+
+ "github.com/Sirupsen/logrus"
+ "github.com/docker/docker/api/server/httputils"
+ "github.com/docker/docker/pkg/version"
+ "golang.org/x/net/context"
+)
+
+// NewUserAgentMiddleware creates a new UserAgent middleware.
+func NewUserAgentMiddleware(versionCheck string) Middleware {
+ serverVersion := version.Version(versionCheck)
+
+ return func(handler httputils.APIFunc) httputils.APIFunc {
+ return func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
+ if strings.Contains(r.Header.Get("User-Agent"), "Docker-Client/") {
+ userAgent := strings.Split(r.Header.Get("User-Agent"), "/")
+
+ // v1.20 onwards includes the GOOS of the client after the version
+ // such as Docker/1.7.0 (linux)
+ if len(userAgent) == 2 && strings.Contains(userAgent[1], " ") {
+ userAgent[1] = strings.Split(userAgent[1], " ")[0]
+ }
+
+ if len(userAgent) == 2 && !serverVersion.Equal(version.Version(userAgent[1])) {
+ logrus.Debugf("Client and server don't have the same version (client: %s, server: %s)", userAgent[1], serverVersion)
+ }
+ }
+ return handler(ctx, w, r, vars)
+ }
+ }
+}
diff --git a/components/engine/api/server/middleware/version.go b/components/engine/api/server/middleware/version.go
new file mode 100644
index 0000000000..41d518bcbc
--- /dev/null
+++ b/components/engine/api/server/middleware/version.go
@@ -0,0 +1,45 @@
+package middleware
+
+import (
+ "fmt"
+ "net/http"
+ "runtime"
+
+ "github.com/docker/docker/api/server/httputils"
+ "github.com/docker/docker/pkg/version"
+ "golang.org/x/net/context"
+)
+
+type badRequestError struct {
+ error
+}
+
+func (badRequestError) HTTPErrorStatusCode() int {
+ return http.StatusBadRequest
+}
+
+// NewVersionMiddleware creates a new Version middleware.
+func NewVersionMiddleware(versionCheck string, defaultVersion, minVersion version.Version) Middleware {
+ serverVersion := version.Version(versionCheck)
+
+ return func(handler httputils.APIFunc) httputils.APIFunc {
+ return func(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
+ apiVersion := version.Version(vars["version"])
+ if apiVersion == "" {
+ apiVersion = defaultVersion
+ }
+
+ if apiVersion.GreaterThan(defaultVersion) {
+ return badRequestError{fmt.Errorf("client is newer than server (client API version: %s, server API version: %s)", apiVersion, defaultVersion)}
+ }
+ if apiVersion.LessThan(minVersion) {
+ return badRequestError{fmt.Errorf("client version %s is too old. Minimum supported API version is %s, please upgrade your client to a newer version", apiVersion, minVersion)}
+ }
+
+ header := fmt.Sprintf("Docker/%s (%s)", serverVersion, runtime.GOOS)
+ w.Header().Set("Server", header)
+ ctx = context.WithValue(ctx, httputils.APIVersionKey, apiVersion)
+ return handler(ctx, w, r, vars)
+ }
+ }
+}
diff --git a/components/engine/api/server/middleware_test.go b/components/engine/api/server/middleware/version_test.go
similarity index 61%
rename from components/engine/api/server/middleware_test.go
rename to components/engine/api/server/middleware/version_test.go
index 4f48b20990..f60a98e518 100644
--- a/components/engine/api/server/middleware_test.go
+++ b/components/engine/api/server/middleware/version_test.go
@@ -1,13 +1,13 @@
-package server
+package middleware
import (
"net/http"
"net/http/httptest"
+ "strings"
"testing"
- "github.com/docker/distribution/registry/api/errcode"
"github.com/docker/docker/api/server/httputils"
- "github.com/docker/docker/errors"
+ "github.com/docker/docker/pkg/version"
"golang.org/x/net/context"
)
@@ -19,7 +19,10 @@ func TestVersionMiddleware(t *testing.T) {
return nil
}
- h := versionMiddleware(handler)
+ defaultVersion := version.Version("1.10.0")
+ minVersion := version.Version("1.2.0")
+ m := NewVersionMiddleware(defaultVersion.String(), defaultVersion, minVersion)
+ h := m(handler)
req, _ := http.NewRequest("GET", "/containers/json", nil)
resp := httptest.NewRecorder()
@@ -37,7 +40,10 @@ func TestVersionMiddlewareWithErrors(t *testing.T) {
return nil
}
- h := versionMiddleware(handler)
+ defaultVersion := version.Version("1.10.0")
+ minVersion := version.Version("1.2.0")
+ m := NewVersionMiddleware(defaultVersion.String(), defaultVersion, minVersion)
+ h := m(handler)
req, _ := http.NewRequest("GET", "/containers/json", nil)
resp := httptest.NewRecorder()
@@ -45,13 +51,14 @@ func TestVersionMiddlewareWithErrors(t *testing.T) {
vars := map[string]string{"version": "0.1"}
err := h(ctx, resp, req, vars)
- if derr, ok := err.(errcode.Error); !ok || derr.ErrorCode() != errors.ErrorCodeOldClientVersion {
- t.Fatalf("Expected ErrorCodeOldClientVersion, got %v", err)
+
+ if !strings.Contains(err.Error(), "client version 0.1 is too old. Minimum supported API version is 1.2.0") {
+ t.Fatalf("Expected too old client error, got %v", err)
}
vars["version"] = "100000"
err = h(ctx, resp, req, vars)
- if derr, ok := err.(errcode.Error); !ok || derr.ErrorCode() != errors.ErrorCodeNewerClientVersion {
- t.Fatalf("Expected ErrorCodeNewerClientVersion, got %v", err)
+ if !strings.Contains(err.Error(), "client is newer than server") {
+ t.Fatalf("Expected client newer than server error, got %v", err)
}
}
diff --git a/components/engine/api/server/profiler.go b/components/engine/api/server/profiler.go
index 766462bd6a..3c0dfd08f2 100644
--- a/components/engine/api/server/profiler.go
+++ b/components/engine/api/server/profiler.go
@@ -9,8 +9,10 @@ import (
"github.com/gorilla/mux"
)
-func profilerSetup(mainRouter *mux.Router, path string) {
- var r = mainRouter.PathPrefix(path).Subrouter()
+const debugPathPrefix = "/debug/"
+
+func profilerSetup(mainRouter *mux.Router) {
+ var r = mainRouter.PathPrefix(debugPathPrefix).Subrouter()
r.HandleFunc("/vars", expVars)
r.HandleFunc("/pprof/", pprof.Index)
r.HandleFunc("/pprof/cmdline", pprof.Cmdline)
diff --git a/components/engine/api/server/router/build/build_routes.go b/components/engine/api/server/router/build/build_routes.go
index 152dfff328..0025c85a93 100644
--- a/components/engine/api/server/router/build/build_routes.go
+++ b/components/engine/api/server/router/build/build_routes.go
@@ -4,7 +4,6 @@ import (
"bytes"
"encoding/base64"
"encoding/json"
- "errors"
"fmt"
"io"
"net/http"
@@ -17,7 +16,6 @@ import (
"github.com/docker/docker/pkg/ioutils"
"github.com/docker/docker/pkg/progress"
"github.com/docker/docker/pkg/streamformatter"
- "github.com/docker/docker/utils"
"github.com/docker/engine-api/types"
"github.com/docker/engine-api/types/container"
"github.com/docker/go-units"
@@ -60,11 +58,11 @@ func newImageBuildOptions(ctx context.Context, r *http.Request) (*types.ImageBui
options.ShmSize = shmSize
}
- if i := container.IsolationLevel(r.FormValue("isolation")); i != "" {
- if !container.IsolationLevel.IsValid(i) {
+ if i := container.Isolation(r.FormValue("isolation")); i != "" {
+ if !container.Isolation.IsValid(i) {
return nil, fmt.Errorf("Unsupported isolation: %q", i)
}
- options.IsolationLevel = i
+ options.Isolation = i
}
var buildUlimits = []*units.Ulimit{}
@@ -117,7 +115,7 @@ func (br *buildRouter) postBuild(ctx context.Context, w http.ResponseWriter, r *
if !output.Flushed() {
return err
}
- _, err = w.Write(sf.FormatError(errors.New(utils.GetErrorMessage(err))))
+ _, err = w.Write(sf.FormatError(err))
if err != nil {
logrus.Warnf("could not write error response: %v", err)
}
@@ -159,6 +157,8 @@ func (br *buildRouter) postBuild(ctx context.Context, w http.ResponseWriter, r *
buildOptions.Dockerfile = dockerfileName
}
+ buildOptions.AuthConfigs = authConfigs
+
out = output
if buildOptions.SuppressOutput {
out = notVerboseBuffer
diff --git a/components/engine/api/server/router/container/backend.go b/components/engine/api/server/router/container/backend.go
index cc25f10919..bd8919759c 100644
--- a/components/engine/api/server/router/container/backend.go
+++ b/components/engine/api/server/router/container/backend.go
@@ -5,7 +5,6 @@ import (
"time"
"github.com/docker/docker/api/types/backend"
- "github.com/docker/docker/daemon/exec"
"github.com/docker/docker/pkg/archive"
"github.com/docker/docker/pkg/version"
"github.com/docker/engine-api/types"
@@ -15,7 +14,7 @@ import (
// execBackend includes functions to implement to provide exec functionality.
type execBackend interface {
ContainerExecCreate(config *types.ExecConfig) (string, error)
- ContainerExecInspect(id string) (*exec.Config, error)
+ ContainerExecInspect(id string) (*backend.ExecInspect, error)
ContainerExecResize(name string, height, width int) error
ContainerExecStart(name string, stdin io.ReadCloser, stdout io.Writer, stderr io.Writer) error
ExecExists(name string) (bool, error)
diff --git a/components/engine/api/server/router/container/container_routes.go b/components/engine/api/server/router/container/container_routes.go
index 07d40168be..016e00f05b 100644
--- a/components/engine/api/server/router/container/container_routes.go
+++ b/components/engine/api/server/router/container/container_routes.go
@@ -11,15 +11,12 @@ import (
"time"
"github.com/Sirupsen/logrus"
- "github.com/docker/distribution/registry/api/errcode"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/types/backend"
- derr "github.com/docker/docker/errors"
"github.com/docker/docker/pkg/ioutils"
"github.com/docker/docker/pkg/signal"
"github.com/docker/docker/pkg/term"
"github.com/docker/docker/runconfig"
- "github.com/docker/docker/utils"
"github.com/docker/engine-api/types"
"github.com/docker/engine-api/types/container"
"github.com/docker/engine-api/types/filters"
@@ -126,7 +123,7 @@ func (s *containerRouter) getContainersLogs(ctx context.Context, w http.Response
// The client may be expecting all of the data we're sending to
// be multiplexed, so send it through OutStream, which will
// have been set up to handle that if needed.
- fmt.Fprintf(logsConfig.OutStream, "Error running logs job: %s\n", utils.GetErrorMessage(err))
+ fmt.Fprintf(logsConfig.OutStream, "Error running logs job: %v\n", err)
default:
return err
}
@@ -182,6 +179,10 @@ func (s *containerRouter) postContainersStop(ctx context.Context, w http.Respons
return nil
}
+type errContainerIsRunning interface {
+ ContainerIsRunning() bool
+}
+
func (s *containerRouter) postContainersKill(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
if err := httputils.ParseForm(r); err != nil {
return err
@@ -199,15 +200,17 @@ func (s *containerRouter) postContainersKill(ctx context.Context, w http.Respons
}
if err := s.backend.ContainerKill(name, uint64(sig)); err != nil {
- theErr, isDerr := err.(errcode.ErrorCoder)
- isStopped := isDerr && theErr.ErrorCode() == derr.ErrorCodeNotRunning
+ var isStopped bool
+ if e, ok := err.(errContainerIsRunning); ok {
+ isStopped = !e.ContainerIsRunning()
+ }
// Return error that's not caused because the container is stopped.
// Return error if the container is not running and the api is >= 1.20
// to keep backwards compatibility.
version := httputils.VersionFromContext(ctx)
if version.GreaterThanOrEqualTo("1.20") || !isStopped {
- return fmt.Errorf("Cannot kill container %s: %v", name, utils.GetErrorMessage(err))
+ return fmt.Errorf("Cannot kill container %s: %v", name, err)
}
}
@@ -322,7 +325,8 @@ func (s *containerRouter) postContainerUpdate(ctx context.Context, w http.Respon
}
hostConfig := &container.HostConfig{
- Resources: updateConfig.Resources,
+ Resources: updateConfig.Resources,
+ RestartPolicy: updateConfig.RestartPolicy,
}
name := vars["name"]
@@ -429,7 +433,7 @@ func (s *containerRouter) postContainersAttach(ctx context.Context, w http.Respo
hijacker, ok := w.(http.Hijacker)
if !ok {
- return derr.ErrorCodeNoHijackConnection.WithArgs(containerName)
+ return fmt.Errorf("error attaching to container %s, hijack connection missing", containerName)
}
setupStreams := func() (io.ReadCloser, io.Writer, io.Writer, error) {
diff --git a/components/engine/api/server/router/container/exec.go b/components/engine/api/server/router/container/exec.go
index caa5da061d..bc336f6039 100644
--- a/components/engine/api/server/router/container/exec.go
+++ b/components/engine/api/server/router/container/exec.go
@@ -10,7 +10,6 @@ import (
"github.com/Sirupsen/logrus"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/pkg/stdcopy"
- "github.com/docker/docker/utils"
"github.com/docker/engine-api/types"
"golang.org/x/net/context"
)
@@ -46,7 +45,7 @@ func (s *containerRouter) postContainerExecCreate(ctx context.Context, w http.Re
// Register an instance of Exec in container.
id, err := s.backend.ContainerExecCreate(execConfig)
if err != nil {
- logrus.Errorf("Error setting up exec command in container %s: %s", name, utils.GetErrorMessage(err))
+ logrus.Errorf("Error setting up exec command in container %s: %v", name, err)
return err
}
@@ -113,7 +112,7 @@ func (s *containerRouter) postContainerExecStart(ctx context.Context, w http.Res
if execStartCheck.Detach {
return err
}
- logrus.Errorf("Error running exec in container: %v\n", utils.GetErrorMessage(err))
+ logrus.Errorf("Error running exec in container: %v\n", err)
}
return nil
}
diff --git a/components/engine/api/server/router/image/backend.go b/components/engine/api/server/router/image/backend.go
index 73e8216025..8c76ef9260 100644
--- a/components/engine/api/server/router/image/backend.go
+++ b/components/engine/api/server/router/image/backend.go
@@ -20,7 +20,6 @@ type Backend interface {
type containerBackend interface {
Commit(name string, config *types.ContainerCommitConfig) (imageID string, err error)
- Exists(containerName string) bool
}
type imageBackend interface {
diff --git a/components/engine/api/server/router/image/image.go b/components/engine/api/server/router/image/image.go
index e1b6bed9d7..d6a1297a97 100644
--- a/components/engine/api/server/router/image/image.go
+++ b/components/engine/api/server/router/image/image.go
@@ -4,14 +4,14 @@ import "github.com/docker/docker/api/server/router"
// imageRouter is a router to talk with the image controller
type imageRouter struct {
- daemon Backend
- routes []router.Route
+ backend Backend
+ routes []router.Route
}
// NewRouter initializes a new image router
-func NewRouter(daemon Backend) router.Router {
+func NewRouter(backend Backend) router.Router {
r := &imageRouter{
- daemon: daemon,
+ backend: backend,
}
r.initRoutes()
return r
diff --git a/components/engine/api/server/router/image/image_routes.go b/components/engine/api/server/router/image/image_routes.go
index c84317aec8..dade346925 100644
--- a/components/engine/api/server/router/image/image_routes.go
+++ b/components/engine/api/server/router/image/image_routes.go
@@ -14,7 +14,6 @@ import (
"github.com/docker/distribution/registry/api/errcode"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/builder/dockerfile"
- derr "github.com/docker/docker/errors"
"github.com/docker/docker/pkg/ioutils"
"github.com/docker/docker/pkg/streamformatter"
"github.com/docker/docker/reference"
@@ -49,10 +48,6 @@ func (s *imageRouter) postCommit(ctx context.Context, w http.ResponseWriter, r *
c = &container.Config{}
}
- if !s.daemon.Exists(cname) {
- return derr.ErrorCodeNoSuchContainer.WithArgs(cname)
- }
-
newConfig, err := dockerfile.BuildFromConfig(c, r.Form["changes"])
if err != nil {
return err
@@ -68,7 +63,7 @@ func (s *imageRouter) postCommit(ctx context.Context, w http.ResponseWriter, r *
MergeConfigs: true,
}
- imgID, err := s.daemon.Commit(cname, commitCfg)
+ imgID, err := s.backend.Commit(cname, commitCfg)
if err != nil {
return err
}
@@ -134,7 +129,7 @@ func (s *imageRouter) postImagesCreate(ctx context.Context, w http.ResponseWrite
}
}
- err = s.daemon.PullImage(ref, metaHeaders, authConfig, output)
+ err = s.backend.PullImage(ref, metaHeaders, authConfig, output)
}
}
// Check the error from pulling an image to make sure the request
@@ -175,7 +170,7 @@ func (s *imageRouter) postImagesCreate(ctx context.Context, w http.ResponseWrite
return err
}
- err = s.daemon.ImportImage(src, newRef, message, r.Body, output, newConfig)
+ err = s.backend.ImportImage(src, newRef, message, r.Body, output, newConfig)
}
if err != nil {
if !output.Flushed() {
@@ -233,7 +228,7 @@ func (s *imageRouter) postImagesPush(ctx context.Context, w http.ResponseWriter,
w.Header().Set("Content-Type", "application/json")
- if err := s.daemon.PushImage(ref, metaHeaders, authConfig, output); err != nil {
+ if err := s.backend.PushImage(ref, metaHeaders, authConfig, output); err != nil {
if !output.Flushed() {
return err
}
@@ -259,7 +254,7 @@ func (s *imageRouter) getImagesGet(ctx context.Context, w http.ResponseWriter, r
names = r.Form["names"]
}
- if err := s.daemon.ExportImage(names, output); err != nil {
+ if err := s.backend.ExportImage(names, output); err != nil {
if !output.Flushed() {
return err
}
@@ -275,7 +270,7 @@ func (s *imageRouter) postImagesLoad(ctx context.Context, w http.ResponseWriter,
}
quiet := httputils.BoolValueOrDefault(r, "quiet", true)
w.Header().Set("Content-Type", "application/json")
- return s.daemon.LoadImage(r.Body, w, quiet)
+ return s.backend.LoadImage(r.Body, w, quiet)
}
func (s *imageRouter) deleteImages(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
@@ -292,7 +287,7 @@ func (s *imageRouter) deleteImages(ctx context.Context, w http.ResponseWriter, r
force := httputils.BoolValue(r, "force")
prune := !httputils.BoolValue(r, "noprune")
- list, err := s.daemon.ImageDelete(name, force, prune)
+ list, err := s.backend.ImageDelete(name, force, prune)
if err != nil {
return err
}
@@ -301,7 +296,7 @@ func (s *imageRouter) deleteImages(ctx context.Context, w http.ResponseWriter, r
}
func (s *imageRouter) getImagesByName(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
- imageInspect, err := s.daemon.LookupImage(vars["name"])
+ imageInspect, err := s.backend.LookupImage(vars["name"])
if err != nil {
return err
}
@@ -315,7 +310,7 @@ func (s *imageRouter) getImagesJSON(ctx context.Context, w http.ResponseWriter,
}
// FIXME: The filter parameter could just be a match filter
- images, err := s.daemon.Images(r.Form.Get("filters"), r.Form.Get("filter"), httputils.BoolValue(r, "all"))
+ images, err := s.backend.Images(r.Form.Get("filters"), r.Form.Get("filter"), httputils.BoolValue(r, "all"))
if err != nil {
return err
}
@@ -325,7 +320,7 @@ func (s *imageRouter) getImagesJSON(ctx context.Context, w http.ResponseWriter,
func (s *imageRouter) getImagesHistory(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
name := vars["name"]
- history, err := s.daemon.ImageHistory(name)
+ history, err := s.backend.ImageHistory(name)
if err != nil {
return err
}
@@ -348,7 +343,7 @@ func (s *imageRouter) postImagesTag(ctx context.Context, w http.ResponseWriter,
return err
}
}
- if err := s.daemon.TagImage(newTag, vars["name"]); err != nil {
+ if err := s.backend.TagImage(newTag, vars["name"]); err != nil {
return err
}
w.WriteHeader(http.StatusCreated)
@@ -378,7 +373,7 @@ func (s *imageRouter) getImagesSearch(ctx context.Context, w http.ResponseWriter
headers[k] = v
}
}
- query, err := s.daemon.SearchRegistryForImages(r.Form.Get("term"), config, headers)
+ query, err := s.backend.SearchRegistryForImages(r.Form.Get("term"), config, headers)
if err != nil {
return err
}
diff --git a/components/engine/api/server/router/network/backend.go b/components/engine/api/server/router/network/backend.go
index 113c497ce1..731f92e7de 100644
--- a/components/engine/api/server/router/network/backend.go
+++ b/components/engine/api/server/router/network/backend.go
@@ -8,13 +8,11 @@ import (
// Backend is all the methods that need to be implemented
// to provide network specific functionality.
type Backend interface {
- NetworkControllerEnabled() bool
-
FindNetwork(idName string) (libnetwork.Network, error)
GetNetworkByName(idName string) (libnetwork.Network, error)
GetNetworksByID(partialID string) []libnetwork.Network
GetAllNetworks() []libnetwork.Network
- CreateNetwork(name, driver string, ipam network.IPAM, options map[string]string, internal bool) (libnetwork.Network, error)
+ CreateNetwork(name, driver string, ipam network.IPAM, options map[string]string, internal bool, enableIPv6 bool) (libnetwork.Network, error)
ConnectContainerToNetwork(containerName, networkName string, endpointConfig *network.EndpointSettings) error
DisconnectContainerFromNetwork(containerName string, network libnetwork.Network, force bool) error
DeleteNetwork(name string) error
diff --git a/components/engine/api/server/router/network/filter.go b/components/engine/api/server/router/network/filter.go
index 31d8d0c521..f1648cc2ae 100644
--- a/components/engine/api/server/router/network/filter.go
+++ b/components/engine/api/server/router/network/filter.go
@@ -84,8 +84,8 @@ func filterNetworkByID(nws []libnetwork.Network, id string) (retNws []libnetwork
return retNws, nil
}
-// filterAllNetworks filter network list according to user specified filter
-// and return user chosen networks
+// filterAllNetworks filters network list according to user specified filter
+// and returns user chosen networks
func filterNetworks(nws []libnetwork.Network, filter filters.Args) ([]libnetwork.Network, error) {
// if filter is empty, return original network list
if filter.Len() == 0 {
diff --git a/components/engine/api/server/router/network/network.go b/components/engine/api/server/router/network/network.go
index 59641bc03b..7c88089623 100644
--- a/components/engine/api/server/router/network/network.go
+++ b/components/engine/api/server/router/network/network.go
@@ -1,13 +1,6 @@
package network
-import (
- "net/http"
-
- "github.com/docker/docker/api/server/httputils"
- "github.com/docker/docker/api/server/router"
- "github.com/docker/docker/errors"
- "golang.org/x/net/context"
-)
+import "github.com/docker/docker/api/server/router"
// networkRouter is a router to talk with the network controller
type networkRouter struct {
@@ -32,24 +25,13 @@ func (r *networkRouter) Routes() []router.Route {
func (r *networkRouter) initRoutes() {
r.routes = []router.Route{
// GET
- router.NewGetRoute("/networks", r.controllerEnabledMiddleware(r.getNetworksList)),
- router.NewGetRoute("/networks/{id:.*}", r.controllerEnabledMiddleware(r.getNetwork)),
+ router.NewGetRoute("/networks", r.getNetworksList),
+ router.NewGetRoute("/networks/{id:.*}", r.getNetwork),
// POST
- router.NewPostRoute("/networks/create", r.controllerEnabledMiddleware(r.postNetworkCreate)),
- router.NewPostRoute("/networks/{id:.*}/connect", r.controllerEnabledMiddleware(r.postNetworkConnect)),
- router.NewPostRoute("/networks/{id:.*}/disconnect", r.controllerEnabledMiddleware(r.postNetworkDisconnect)),
+ router.NewPostRoute("/networks/create", r.postNetworkCreate),
+ router.NewPostRoute("/networks/{id:.*}/connect", r.postNetworkConnect),
+ router.NewPostRoute("/networks/{id:.*}/disconnect", r.postNetworkDisconnect),
// DELETE
- router.NewDeleteRoute("/networks/{id:.*}", r.controllerEnabledMiddleware(r.deleteNetwork)),
+ router.NewDeleteRoute("/networks/{id:.*}", r.deleteNetwork),
}
}
-
-func (r *networkRouter) controllerEnabledMiddleware(handler httputils.APIFunc) httputils.APIFunc {
- if r.backend.NetworkControllerEnabled() {
- return handler
- }
- return networkControllerDisabled
-}
-
-func networkControllerDisabled(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
- return errors.ErrorNetworkControllerNotEnabled.WithArgs()
-}
diff --git a/components/engine/api/server/router/network/network_routes.go b/components/engine/api/server/router/network/network_routes.go
index 25f0e83ca6..851a10cb89 100644
--- a/components/engine/api/server/router/network/network_routes.go
+++ b/components/engine/api/server/router/network/network_routes.go
@@ -91,7 +91,7 @@ func (n *networkRouter) postNetworkCreate(ctx context.Context, w http.ResponseWr
warning = fmt.Sprintf("Network with name %s (id : %s) already exists", nw.Name(), nw.ID())
}
- nw, err = n.backend.CreateNetwork(create.Name, create.Driver, create.IPAM, create.Options, create.Internal)
+ nw, err = n.backend.CreateNetwork(create.Name, create.Driver, create.IPAM, create.Options, create.Internal, create.EnableIPv6)
if err != nil {
return err
}
@@ -160,6 +160,8 @@ func buildNetworkResource(nw libnetwork.Network) *types.NetworkResource {
r.ID = nw.ID()
r.Scope = nw.Info().Scope()
r.Driver = nw.Type()
+ r.EnableIPv6 = nw.Info().IPv6Enabled()
+ r.Internal = nw.Info().Internal()
r.Options = nw.Info().DriverOptions()
r.Containers = make(map[string]types.EndpointResource)
buildIpamResources(r, nw)
diff --git a/components/engine/api/server/router_swapper.go b/components/engine/api/server/router_swapper.go
index b5f1d06d8d..1ecc7a7f39 100644
--- a/components/engine/api/server/router_swapper.go
+++ b/components/engine/api/server/router_swapper.go
@@ -7,7 +7,7 @@ import (
"github.com/gorilla/mux"
)
-// routerSwapper is an http.Handler that allow you to swap
+// routerSwapper is an http.Handler that allows you to swap
// mux routers.
type routerSwapper struct {
mu sync.Mutex
diff --git a/components/engine/api/server/server.go b/components/engine/api/server/server.go
index 5e8337a7c2..1379b7372e 100644
--- a/components/engine/api/server/server.go
+++ b/components/engine/api/server/server.go
@@ -9,17 +9,7 @@ import (
"github.com/Sirupsen/logrus"
"github.com/docker/docker/api/server/httputils"
"github.com/docker/docker/api/server/router"
- "github.com/docker/docker/api/server/router/build"
- "github.com/docker/docker/api/server/router/container"
- "github.com/docker/docker/api/server/router/image"
- "github.com/docker/docker/api/server/router/network"
- "github.com/docker/docker/api/server/router/system"
- "github.com/docker/docker/api/server/router/volume"
- "github.com/docker/docker/builder/dockerfile"
- "github.com/docker/docker/daemon"
"github.com/docker/docker/pkg/authorization"
- "github.com/docker/docker/utils"
- "github.com/docker/go-connections/sockets"
"github.com/gorilla/mux"
"golang.org/x/net/context"
)
@@ -37,7 +27,6 @@ type Config struct {
Version string
SocketGroup string
TLSConfig *tls.Config
- Addrs []Addr
}
// Server contains instance details for the server
@@ -49,27 +38,25 @@ type Server struct {
routerSwapper *routerSwapper
}
-// Addr contains string representation of address and its protocol (tcp, unix...).
-type Addr struct {
- Proto string
- Addr string
-}
-
// New returns a new instance of the server based on the specified configuration.
// It allocates resources which will be needed for ServeAPI(ports, unix-sockets).
-func New(cfg *Config) (*Server, error) {
- s := &Server{
+func New(cfg *Config) *Server {
+ return &Server{
cfg: cfg,
}
- for _, addr := range cfg.Addrs {
- srv, err := s.newServer(addr.Proto, addr.Addr)
- if err != nil {
- return nil, err
+}
+
+// Accept sets a listener the server accepts connections into.
+func (s *Server) Accept(addr string, listeners ...net.Listener) {
+ for _, listener := range listeners {
+ httpServer := &HTTPServer{
+ srv: &http.Server{
+ Addr: addr,
+ },
+ l: listener,
}
- logrus.Debugf("Server created for HTTP on %s (%s)", addr.Proto, addr.Addr)
- s.servers = append(s.servers, srv...)
+ s.servers = append(s.servers, httpServer)
}
- return s, nil
}
// Close closes servers and thus stop receiving requests
@@ -84,8 +71,6 @@ func (s *Server) Close() {
// serveAPI loops through all initialized servers and spawns goroutine
// with Server method for each. It sets createMux() as Handler also.
func (s *Server) serveAPI() error {
- s.initRouterSwapper()
-
var chErrors = make(chan error, len(s.servers))
for _, srv := range s.servers {
srv.srv.Handler = s.routerSwapper
@@ -127,31 +112,8 @@ func (s *HTTPServer) Close() error {
return s.l.Close()
}
-func writeCorsHeaders(w http.ResponseWriter, r *http.Request, corsHeaders string) {
- logrus.Debugf("CORS header is enabled and set to: %s", corsHeaders)
- w.Header().Add("Access-Control-Allow-Origin", corsHeaders)
- w.Header().Add("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept, X-Registry-Auth")
- w.Header().Add("Access-Control-Allow-Methods", "HEAD, GET, POST, DELETE, PUT, OPTIONS")
-}
-
-func (s *Server) initTCPSocket(addr string) (l net.Listener, err error) {
- if s.cfg.TLSConfig == nil || s.cfg.TLSConfig.ClientAuth != tls.RequireAndVerifyClientCert {
- logrus.Warn("/!\\ DON'T BIND ON ANY IP ADDRESS WITHOUT setting -tlsverify IF YOU DON'T KNOW WHAT YOU'RE DOING /!\\")
- }
- if l, err = sockets.NewTCPSocket(addr, s.cfg.TLSConfig); err != nil {
- return nil, err
- }
- if err := allocateDaemonPort(addr); err != nil {
- return nil, err
- }
- return
-}
-
func (s *Server) makeHTTPHandler(handler httputils.APIFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
- // log the handler call
- logrus.Debugf("Calling %s %s", r.Method, r.URL.Path)
-
// Define the context that we'll pass around to share info
// like the docker-request-id.
//
@@ -168,33 +130,31 @@ func (s *Server) makeHTTPHandler(handler httputils.APIFunc) http.HandlerFunc {
}
if err := handlerFunc(ctx, w, r, vars); err != nil {
- logrus.Errorf("Handler for %s %s returned error: %s", r.Method, r.URL.Path, utils.GetErrorMessage(err))
+ logrus.Errorf("Handler for %s %s returned error: %v", r.Method, r.URL.Path, err)
httputils.WriteError(w, err)
}
}
}
-// InitRouters initializes a list of routers for the server.
-func (s *Server) InitRouters(d *daemon.Daemon) {
- s.addRouter(container.NewRouter(d))
- s.addRouter(image.NewRouter(d))
- s.addRouter(network.NewRouter(d))
- s.addRouter(system.NewRouter(d))
- s.addRouter(volume.NewRouter(d))
- s.addRouter(build.NewRouter(dockerfile.NewBuildManager(d)))
-}
+// InitRouter initializes the list of routers for the server.
+// This method also enables the Go profiler if enableProfiler is true.
+func (s *Server) InitRouter(enableProfiler bool, routers ...router.Router) {
+ for _, r := range routers {
+ s.routers = append(s.routers, r)
+ }
-// addRouter adds a new router to the server.
-func (s *Server) addRouter(r router.Router) {
- s.routers = append(s.routers, r)
+ m := s.createMux()
+ if enableProfiler {
+ profilerSetup(m)
+ }
+ s.routerSwapper = &routerSwapper{
+ router: m,
+ }
}
// createMux initializes the main router the server uses.
func (s *Server) createMux() *mux.Router {
m := mux.NewRouter()
- if utils.IsDebugEnabled() {
- profilerSetup(m, "/debug/")
- }
logrus.Debugf("Registering routers")
for _, apiRouter := range s.routers {
@@ -222,23 +182,14 @@ func (s *Server) Wait(waitChan chan error) {
waitChan <- nil
}
-func (s *Server) initRouterSwapper() {
- s.routerSwapper = &routerSwapper{
- router: s.createMux(),
- }
+// DisableProfiler reloads the server mux without adding the profiler routes.
+func (s *Server) DisableProfiler() {
+ s.routerSwapper.Swap(s.createMux())
}
-// Reload reads configuration changes and modifies the
-// server according to those changes.
-// Currently, only the --debug configuration is taken into account.
-func (s *Server) Reload(config *daemon.Config) {
- debugEnabled := utils.IsDebugEnabled()
- switch {
- case debugEnabled && !config.Debug: // disable debug
- utils.DisableDebug()
- s.routerSwapper.Swap(s.createMux())
- case config.Debug && !debugEnabled: // enable debug
- utils.EnableDebug()
- s.routerSwapper.Swap(s.createMux())
- }
+// EnableProfiler reloads the server mux adding the profiler routes.
+func (s *Server) EnableProfiler() {
+ m := s.createMux()
+ profilerSetup(m)
+ s.routerSwapper.Swap(m)
}
diff --git a/components/engine/api/types/backend/backend.go b/components/engine/api/types/backend/backend.go
index c871a148c4..ffe9b709e1 100644
--- a/components/engine/api/types/backend/backend.go
+++ b/components/engine/api/types/backend/backend.go
@@ -42,3 +42,28 @@ type ContainerStatsConfig struct {
Stop <-chan bool
Version string
}
+
+// ExecInspect holds information about a running process started
+// with docker exec.
+type ExecInspect struct {
+ ID string
+ Running bool
+ ExitCode *int
+ ProcessConfig *ExecProcessConfig
+ OpenStdin bool
+ OpenStderr bool
+ OpenStdout bool
+ CanRemove bool
+ ContainerID string
+ DetachKeys []byte
+}
+
+// ExecProcessConfig holds information about the exec process
+// running on the host.
+type ExecProcessConfig struct {
+ Tty bool `json:"tty"`
+ Entrypoint string `json:"entrypoint"`
+ Arguments []string `json:"arguments"`
+ Privileged *bool `json:"privileged,omitempty"`
+ User string `json:"user,omitempty"`
+}
diff --git a/components/engine/builder/builder.go b/components/engine/builder/builder.go
index e20893f18e..43586a1e4f 100644
--- a/components/engine/builder/builder.go
+++ b/components/engine/builder/builder.go
@@ -14,6 +14,11 @@ import (
"github.com/docker/engine-api/types/container"
)
+const (
+ // DefaultDockerfileName is the Default filename with Docker commands, read by docker build
+ DefaultDockerfileName string = "Dockerfile"
+)
+
// Context represents a file system tree.
type Context interface {
// Close allows to signal that the filesystem tree won't be used anymore.
@@ -141,7 +146,7 @@ type Image interface {
// ImageCache abstracts an image cache store.
// (parent image, child runconfig) -> child image
type ImageCache interface {
- // GetCachedImage returns a reference to a cached image whose parent equals `parent`
+ // GetCachedImageOnBuild returns a reference to a cached image whose parent equals `parent`
// and runconfig equals `cfg`. A cache miss is expected to return an empty ID and a nil error.
GetCachedImageOnBuild(parentID string, cfg *container.Config) (imageID string, err error)
}
diff --git a/components/engine/builder/context.go b/components/engine/builder/context.go
new file mode 100644
index 0000000000..53a90f1be2
--- /dev/null
+++ b/components/engine/builder/context.go
@@ -0,0 +1,260 @@
+package builder
+
+import (
+ "bufio"
+ "fmt"
+ "io"
+ "io/ioutil"
+ "os"
+ "os/exec"
+ "path/filepath"
+ "runtime"
+ "strings"
+
+ "github.com/docker/docker/pkg/archive"
+ "github.com/docker/docker/pkg/fileutils"
+ "github.com/docker/docker/pkg/gitutils"
+ "github.com/docker/docker/pkg/httputils"
+ "github.com/docker/docker/pkg/ioutils"
+ "github.com/docker/docker/pkg/progress"
+ "github.com/docker/docker/pkg/streamformatter"
+)
+
+// ValidateContextDirectory checks if all the contents of the directory
+// can be read and returns an error if some files can't be read
+// symlinks which point to non-existing files don't trigger an error
+func ValidateContextDirectory(srcPath string, excludes []string) error {
+ contextRoot, err := getContextRoot(srcPath)
+ if err != nil {
+ return err
+ }
+ return filepath.Walk(contextRoot, func(filePath string, f os.FileInfo, err error) error {
+ // skip this directory/file if it's not in the path, it won't get added to the context
+ if relFilePath, err := filepath.Rel(contextRoot, filePath); err != nil {
+ return err
+ } else if skip, err := fileutils.Matches(relFilePath, excludes); err != nil {
+ return err
+ } else if skip {
+ if f.IsDir() {
+ return filepath.SkipDir
+ }
+ return nil
+ }
+
+ if err != nil {
+ if os.IsPermission(err) {
+ return fmt.Errorf("can't stat '%s'", filePath)
+ }
+ if os.IsNotExist(err) {
+ return nil
+ }
+ return err
+ }
+
+ // skip checking if symlinks point to non-existing files, such symlinks can be useful
+ // also skip named pipes, because they hanging on open
+ if f.Mode()&(os.ModeSymlink|os.ModeNamedPipe) != 0 {
+ return nil
+ }
+
+ if !f.IsDir() {
+ currentFile, err := os.Open(filePath)
+ if err != nil && os.IsPermission(err) {
+ return fmt.Errorf("no permission to read from '%s'", filePath)
+ }
+ currentFile.Close()
+ }
+ return nil
+ })
+}
+
+// GetContextFromReader will read the contents of the given reader as either a
+// Dockerfile or tar archive. Returns a tar archive used as a context and a
+// path to the Dockerfile inside the tar.
+func GetContextFromReader(r io.ReadCloser, dockerfileName string) (out io.ReadCloser, relDockerfile string, err error) {
+ buf := bufio.NewReader(r)
+
+ magic, err := buf.Peek(archive.HeaderSize)
+ if err != nil && err != io.EOF {
+ return nil, "", fmt.Errorf("failed to peek context header from STDIN: %v", err)
+ }
+
+ if archive.IsArchive(magic) {
+ return ioutils.NewReadCloserWrapper(buf, func() error { return r.Close() }), dockerfileName, nil
+ }
+
+ // Input should be read as a Dockerfile.
+ tmpDir, err := ioutil.TempDir("", "docker-build-context-")
+ if err != nil {
+ return nil, "", fmt.Errorf("unbale to create temporary context directory: %v", err)
+ }
+
+ f, err := os.Create(filepath.Join(tmpDir, DefaultDockerfileName))
+ if err != nil {
+ return nil, "", err
+ }
+ _, err = io.Copy(f, buf)
+ if err != nil {
+ f.Close()
+ return nil, "", err
+ }
+
+ if err := f.Close(); err != nil {
+ return nil, "", err
+ }
+ if err := r.Close(); err != nil {
+ return nil, "", err
+ }
+
+ tar, err := archive.Tar(tmpDir, archive.Uncompressed)
+ if err != nil {
+ return nil, "", err
+ }
+
+ return ioutils.NewReadCloserWrapper(tar, func() error {
+ err := tar.Close()
+ os.RemoveAll(tmpDir)
+ return err
+ }), DefaultDockerfileName, nil
+
+}
+
+// GetContextFromGitURL uses a Git URL as context for a `docker build`. The
+// git repo is cloned into a temporary directory used as the context directory.
+// Returns the absolute path to the temporary context directory, the relative
+// path of the dockerfile in that context directory, and a non-nil error on
+// success.
+func GetContextFromGitURL(gitURL, dockerfileName string) (absContextDir, relDockerfile string, err error) {
+ if _, err := exec.LookPath("git"); err != nil {
+ return "", "", fmt.Errorf("unable to find 'git': %v", err)
+ }
+ if absContextDir, err = gitutils.Clone(gitURL); err != nil {
+ return "", "", fmt.Errorf("unable to 'git clone' to temporary context directory: %v", err)
+ }
+
+ return getDockerfileRelPath(absContextDir, dockerfileName)
+}
+
+// GetContextFromURL uses a remote URL as context for a `docker build`. The
+// remote resource is downloaded as either a Dockerfile or a tar archive.
+// Returns the tar archive used for the context and a path of the
+// dockerfile inside the tar.
+func GetContextFromURL(out io.Writer, remoteURL, dockerfileName string) (io.ReadCloser, string, error) {
+ response, err := httputils.Download(remoteURL)
+ if err != nil {
+ return nil, "", fmt.Errorf("unable to download remote context %s: %v", remoteURL, err)
+ }
+ progressOutput := streamformatter.NewStreamFormatter().NewProgressOutput(out, true)
+
+ // Pass the response body through a progress reader.
+ progReader := progress.NewProgressReader(response.Body, progressOutput, response.ContentLength, "", fmt.Sprintf("Downloading build context from remote url: %s", remoteURL))
+
+ return GetContextFromReader(ioutils.NewReadCloserWrapper(progReader, func() error { return response.Body.Close() }), dockerfileName)
+}
+
+// GetContextFromLocalDir uses the given local directory as context for a
+// `docker build`. Returns the absolute path to the local context directory,
+// the relative path of the dockerfile in that context directory, and a non-nil
+// error on success.
+func GetContextFromLocalDir(localDir, dockerfileName string) (absContextDir, relDockerfile string, err error) {
+ // When using a local context directory, when the Dockerfile is specified
+ // with the `-f/--file` option then it is considered relative to the
+ // current directory and not the context directory.
+ if dockerfileName != "" {
+ if dockerfileName, err = filepath.Abs(dockerfileName); err != nil {
+ return "", "", fmt.Errorf("unable to get absolute path to Dockerfile: %v", err)
+ }
+ }
+
+ return getDockerfileRelPath(localDir, dockerfileName)
+}
+
+// getDockerfileRelPath uses the given context directory for a `docker build`
+// and returns the absolute path to the context directory, the relative path of
+// the dockerfile in that context directory, and a non-nil error on success.
+func getDockerfileRelPath(givenContextDir, givenDockerfile string) (absContextDir, relDockerfile string, err error) {
+ if absContextDir, err = filepath.Abs(givenContextDir); err != nil {
+ return "", "", fmt.Errorf("unable to get absolute context directory: %v", err)
+ }
+
+ // The context dir might be a symbolic link, so follow it to the actual
+ // target directory.
+ //
+ // FIXME. We use isUNC (always false on non-Windows platforms) to workaround
+ // an issue in golang. On Windows, EvalSymLinks does not work on UNC file
+ // paths (those starting with \\). This hack means that when using links
+ // on UNC paths, they will not be followed.
+ if !isUNC(absContextDir) {
+ absContextDir, err = filepath.EvalSymlinks(absContextDir)
+ if err != nil {
+ return "", "", fmt.Errorf("unable to evaluate symlinks in context path: %v", err)
+ }
+ }
+
+ stat, err := os.Lstat(absContextDir)
+ if err != nil {
+ return "", "", fmt.Errorf("unable to stat context directory %q: %v", absContextDir, err)
+ }
+
+ if !stat.IsDir() {
+ return "", "", fmt.Errorf("context must be a directory: %s", absContextDir)
+ }
+
+ absDockerfile := givenDockerfile
+ if absDockerfile == "" {
+ // No -f/--file was specified so use the default relative to the
+ // context directory.
+ absDockerfile = filepath.Join(absContextDir, DefaultDockerfileName)
+
+ // Just to be nice ;-) look for 'dockerfile' too but only
+ // use it if we found it, otherwise ignore this check
+ if _, err = os.Lstat(absDockerfile); os.IsNotExist(err) {
+ altPath := filepath.Join(absContextDir, strings.ToLower(DefaultDockerfileName))
+ if _, err = os.Lstat(altPath); err == nil {
+ absDockerfile = altPath
+ }
+ }
+ }
+
+ // If not already an absolute path, the Dockerfile path should be joined to
+ // the base directory.
+ if !filepath.IsAbs(absDockerfile) {
+ absDockerfile = filepath.Join(absContextDir, absDockerfile)
+ }
+
+ // Evaluate symlinks in the path to the Dockerfile too.
+ //
+ // FIXME. We use isUNC (always false on non-Windows platforms) to workaround
+ // an issue in golang. On Windows, EvalSymLinks does not work on UNC file
+ // paths (those starting with \\). This hack means that when using links
+ // on UNC paths, they will not be followed.
+ if !isUNC(absDockerfile) {
+ absDockerfile, err = filepath.EvalSymlinks(absDockerfile)
+ if err != nil {
+ return "", "", fmt.Errorf("unable to evaluate symlinks in Dockerfile path: %v", err)
+ }
+ }
+
+ if _, err := os.Lstat(absDockerfile); err != nil {
+ if os.IsNotExist(err) {
+ return "", "", fmt.Errorf("Cannot locate Dockerfile: %q", absDockerfile)
+ }
+ return "", "", fmt.Errorf("unable to stat Dockerfile: %v", err)
+ }
+
+ if relDockerfile, err = filepath.Rel(absContextDir, absDockerfile); err != nil {
+ return "", "", fmt.Errorf("unable to get relative Dockerfile path: %v", err)
+ }
+
+ if strings.HasPrefix(relDockerfile, ".."+string(filepath.Separator)) {
+ return "", "", fmt.Errorf("The Dockerfile (%s) must be within the build context (%s)", givenDockerfile, givenContextDir)
+ }
+
+ return absContextDir, relDockerfile, nil
+}
+
+// isUNC returns true if the path is UNC (one starting \\). It always returns
+// false on Linux.
+func isUNC(path string) bool {
+ return runtime.GOOS == "windows" && strings.HasPrefix(path, `\\`)
+}
diff --git a/components/engine/api/client/utils_unix.go b/components/engine/builder/context_unix.go
similarity index 90%
rename from components/engine/api/client/utils_unix.go
rename to components/engine/builder/context_unix.go
index ff10ddde9e..d1f72e0573 100644
--- a/components/engine/api/client/utils_unix.go
+++ b/components/engine/builder/context_unix.go
@@ -1,6 +1,6 @@
// +build !windows
-package client
+package builder
import (
"path/filepath"
diff --git a/components/engine/api/client/utils_windows.go b/components/engine/builder/context_windows.go
similarity index 94%
rename from components/engine/api/client/utils_windows.go
rename to components/engine/builder/context_windows.go
index 09c33dadd8..b8ba2ba231 100644
--- a/components/engine/api/client/utils_windows.go
+++ b/components/engine/builder/context_windows.go
@@ -1,6 +1,6 @@
// +build windows
-package client
+package builder
import (
"path/filepath"
diff --git a/components/engine/builder/dockerfile/dispatchers.go b/components/engine/builder/dockerfile/dispatchers.go
index f800acbe8e..cc2d177ccd 100644
--- a/components/engine/builder/dockerfile/dispatchers.go
+++ b/components/engine/builder/dockerfile/dispatchers.go
@@ -19,7 +19,6 @@ import (
"github.com/Sirupsen/logrus"
"github.com/docker/docker/api"
"github.com/docker/docker/builder"
- derr "github.com/docker/docker/errors"
"github.com/docker/docker/pkg/signal"
"github.com/docker/docker/pkg/system"
runconfigopts "github.com/docker/docker/runconfig/opts"
@@ -40,12 +39,12 @@ func nullDispatch(b *Builder, args []string, attributes map[string]bool, origina
//
func env(b *Builder, args []string, attributes map[string]bool, original string) error {
if len(args) == 0 {
- return derr.ErrorCodeAtLeastOneArg.WithArgs("ENV")
+ return errAtLeastOneArgument("ENV")
}
if len(args)%2 != 0 {
// should never get here, but just in case
- return derr.ErrorCodeTooManyArgs.WithArgs("ENV")
+ return errTooManyArguments("ENV")
}
if err := b.flags.Parse(); err != nil {
@@ -99,7 +98,7 @@ func env(b *Builder, args []string, attributes map[string]bool, original string)
// Sets the maintainer metadata.
func maintainer(b *Builder, args []string, attributes map[string]bool, original string) error {
if len(args) != 1 {
- return derr.ErrorCodeExactlyOneArg.WithArgs("MAINTAINER")
+ return errExactlyOneArgument("MAINTAINER")
}
if err := b.flags.Parse(); err != nil {
@@ -116,11 +115,11 @@ func maintainer(b *Builder, args []string, attributes map[string]bool, original
//
func label(b *Builder, args []string, attributes map[string]bool, original string) error {
if len(args) == 0 {
- return derr.ErrorCodeAtLeastOneArg.WithArgs("LABEL")
+ return errAtLeastOneArgument("LABEL")
}
if len(args)%2 != 0 {
// should never get here, but just in case
- return derr.ErrorCodeTooManyArgs.WithArgs("LABEL")
+ return errTooManyArguments("LABEL")
}
if err := b.flags.Parse(); err != nil {
@@ -152,7 +151,7 @@ func label(b *Builder, args []string, attributes map[string]bool, original strin
//
func add(b *Builder, args []string, attributes map[string]bool, original string) error {
if len(args) < 2 {
- return derr.ErrorCodeAtLeastTwoArgs.WithArgs("ADD")
+ return errAtLeastOneArgument("ADD")
}
if err := b.flags.Parse(); err != nil {
@@ -168,7 +167,7 @@ func add(b *Builder, args []string, attributes map[string]bool, original string)
//
func dispatchCopy(b *Builder, args []string, attributes map[string]bool, original string) error {
if len(args) < 2 {
- return derr.ErrorCodeAtLeastTwoArgs.WithArgs("COPY")
+ return errAtLeastOneArgument("COPY")
}
if err := b.flags.Parse(); err != nil {
@@ -184,7 +183,7 @@ func dispatchCopy(b *Builder, args []string, attributes map[string]bool, origina
//
func from(b *Builder, args []string, attributes map[string]bool, original string) error {
if len(args) != 1 {
- return derr.ErrorCodeExactlyOneArg.WithArgs("FROM")
+ return errExactlyOneArgument("FROM")
}
if err := b.flags.Parse(); err != nil {
@@ -233,7 +232,7 @@ func from(b *Builder, args []string, attributes map[string]bool, original string
//
func onbuild(b *Builder, args []string, attributes map[string]bool, original string) error {
if len(args) == 0 {
- return derr.ErrorCodeAtLeastOneArg.WithArgs("ONBUILD")
+ return errAtLeastOneArgument("ONBUILD")
}
if err := b.flags.Parse(); err != nil {
@@ -243,9 +242,9 @@ func onbuild(b *Builder, args []string, attributes map[string]bool, original str
triggerInstruction := strings.ToUpper(strings.TrimSpace(args[0]))
switch triggerInstruction {
case "ONBUILD":
- return derr.ErrorCodeChainOnBuild
+ return fmt.Errorf("Chaining ONBUILD via `ONBUILD ONBUILD` isn't allowed")
case "MAINTAINER", "FROM":
- return derr.ErrorCodeBadOnBuildCmd.WithArgs(triggerInstruction)
+ return fmt.Errorf("%s isn't allowed as an ONBUILD trigger", triggerInstruction)
}
original = regexp.MustCompile(`(?i)^\s*ONBUILD\s*`).ReplaceAllString(original, "")
@@ -260,7 +259,7 @@ func onbuild(b *Builder, args []string, attributes map[string]bool, original str
//
func workdir(b *Builder, args []string, attributes map[string]bool, original string) error {
if len(args) != 1 {
- return derr.ErrorCodeExactlyOneArg.WithArgs("WORKDIR")
+ return errExactlyOneArgument("WORKDIR")
}
if err := b.flags.Parse(); err != nil {
@@ -293,7 +292,7 @@ func workdir(b *Builder, args []string, attributes map[string]bool, original str
//
func run(b *Builder, args []string, attributes map[string]bool, original string) error {
if b.image == "" && !b.noBaseImage {
- return derr.ErrorCodeMissingFrom
+ return fmt.Errorf("Please provide a source image with `from` prior to run")
}
if err := b.flags.Parse(); err != nil {
@@ -311,20 +310,20 @@ func run(b *Builder, args []string, attributes map[string]bool, original string)
}
config := &container.Config{
- Cmd: strslice.New(args...),
+ Cmd: strslice.StrSlice(args),
Image: b.image,
}
// stash the cmd
cmd := b.runConfig.Cmd
- if b.runConfig.Entrypoint.Len() == 0 && b.runConfig.Cmd.Len() == 0 {
+ if len(b.runConfig.Entrypoint) == 0 && len(b.runConfig.Cmd) == 0 {
b.runConfig.Cmd = config.Cmd
}
// stash the config environment
env := b.runConfig.Env
- defer func(cmd *strslice.StrSlice) { b.runConfig.Cmd = cmd }(cmd)
+ defer func(cmd strslice.StrSlice) { b.runConfig.Cmd = cmd }(cmd)
defer func(env []string) { b.runConfig.Env = env }(env)
// derive the net build-time environment for this run. We let config
@@ -367,7 +366,7 @@ func run(b *Builder, args []string, attributes map[string]bool, original string)
if len(cmdBuildEnv) > 0 {
sort.Strings(cmdBuildEnv)
tmpEnv := append([]string{fmt.Sprintf("|%d", len(cmdBuildEnv))}, cmdBuildEnv...)
- saveCmd = strslice.New(append(tmpEnv, saveCmd.Slice()...)...)
+ saveCmd = strslice.StrSlice(append(tmpEnv, saveCmd...))
}
b.runConfig.Cmd = saveCmd
@@ -425,7 +424,7 @@ func cmd(b *Builder, args []string, attributes map[string]bool, original string)
}
}
- b.runConfig.Cmd = strslice.New(cmdSlice...)
+ b.runConfig.Cmd = strslice.StrSlice(cmdSlice)
if err := b.commit("", b.runConfig.Cmd, fmt.Sprintf("CMD %q", cmdSlice)); err != nil {
return err
@@ -456,16 +455,16 @@ func entrypoint(b *Builder, args []string, attributes map[string]bool, original
switch {
case attributes["json"]:
// ENTRYPOINT ["echo", "hi"]
- b.runConfig.Entrypoint = strslice.New(parsed...)
+ b.runConfig.Entrypoint = strslice.StrSlice(parsed)
case len(parsed) == 0:
// ENTRYPOINT []
b.runConfig.Entrypoint = nil
default:
// ENTRYPOINT echo hi
if runtime.GOOS != "windows" {
- b.runConfig.Entrypoint = strslice.New("/bin/sh", "-c", parsed[0])
+ b.runConfig.Entrypoint = strslice.StrSlice{"/bin/sh", "-c", parsed[0]}
} else {
- b.runConfig.Entrypoint = strslice.New("cmd", "/S", "/C", parsed[0])
+ b.runConfig.Entrypoint = strslice.StrSlice{"cmd", "/S", "/C", parsed[0]}
}
}
@@ -491,7 +490,7 @@ func expose(b *Builder, args []string, attributes map[string]bool, original stri
portsTab := args
if len(args) == 0 {
- return derr.ErrorCodeAtLeastOneArg.WithArgs("EXPOSE")
+ return errAtLeastOneArgument("EXPOSE")
}
if err := b.flags.Parse(); err != nil {
@@ -530,7 +529,7 @@ func expose(b *Builder, args []string, attributes map[string]bool, original stri
//
func user(b *Builder, args []string, attributes map[string]bool, original string) error {
if len(args) != 1 {
- return derr.ErrorCodeExactlyOneArg.WithArgs("USER")
+ return errExactlyOneArgument("USER")
}
if err := b.flags.Parse(); err != nil {
@@ -547,7 +546,7 @@ func user(b *Builder, args []string, attributes map[string]bool, original string
//
func volume(b *Builder, args []string, attributes map[string]bool, original string) error {
if len(args) == 0 {
- return derr.ErrorCodeAtLeastOneArg.WithArgs("VOLUME")
+ return errAtLeastOneArgument("VOLUME")
}
if err := b.flags.Parse(); err != nil {
@@ -560,7 +559,7 @@ func volume(b *Builder, args []string, attributes map[string]bool, original stri
for _, v := range args {
v = strings.TrimSpace(v)
if v == "" {
- return derr.ErrorCodeVolumeEmpty
+ return fmt.Errorf("Volume specified can not be an empty string")
}
b.runConfig.Volumes[v] = struct{}{}
}
@@ -631,3 +630,15 @@ func arg(b *Builder, args []string, attributes map[string]bool, original string)
return b.commit("", b.runConfig.Cmd, fmt.Sprintf("ARG %s", arg))
}
+
+func errAtLeastOneArgument(command string) error {
+ return fmt.Errorf("%s requires at least one argument", command)
+}
+
+func errExactlyOneArgument(command string) error {
+ return fmt.Errorf("%s requires exactly one argument", command)
+}
+
+func errTooManyArguments(command string) error {
+ return fmt.Errorf("Bad input to %s, too many arguments", command)
+}
diff --git a/components/engine/builder/dockerfile/internals.go b/components/engine/builder/dockerfile/internals.go
index bd352ae036..ff3e1a25d2 100644
--- a/components/engine/builder/dockerfile/internals.go
+++ b/components/engine/builder/dockerfile/internals.go
@@ -19,7 +19,6 @@ import (
"time"
"github.com/Sirupsen/logrus"
- "github.com/docker/docker/api"
"github.com/docker/docker/builder"
"github.com/docker/docker/builder/dockerfile/parser"
"github.com/docker/docker/pkg/archive"
@@ -38,7 +37,7 @@ import (
"github.com/docker/engine-api/types/strslice"
)
-func (b *Builder) commit(id string, autoCmd *strslice.StrSlice, comment string) error {
+func (b *Builder) commit(id string, autoCmd strslice.StrSlice, comment string) error {
if b.disableCommit {
return nil
}
@@ -49,11 +48,11 @@ func (b *Builder) commit(id string, autoCmd *strslice.StrSlice, comment string)
if id == "" {
cmd := b.runConfig.Cmd
if runtime.GOOS != "windows" {
- b.runConfig.Cmd = strslice.New("/bin/sh", "-c", "#(nop) "+comment)
+ b.runConfig.Cmd = strslice.StrSlice{"/bin/sh", "-c", "#(nop) " + comment}
} else {
- b.runConfig.Cmd = strslice.New("cmd", "/S /C", "REM (nop) "+comment)
+ b.runConfig.Cmd = strslice.StrSlice{"cmd", "/S /C", "REM (nop) " + comment}
}
- defer func(cmd *strslice.StrSlice) { b.runConfig.Cmd = cmd }(cmd)
+ defer func(cmd strslice.StrSlice) { b.runConfig.Cmd = cmd }(cmd)
hit, err := b.probeCache()
if err != nil {
@@ -172,11 +171,11 @@ func (b *Builder) runContextCommand(args []string, allowRemote bool, allowLocalD
cmd := b.runConfig.Cmd
if runtime.GOOS != "windows" {
- b.runConfig.Cmd = strslice.New("/bin/sh", "-c", fmt.Sprintf("#(nop) %s %s in %s", cmdName, srcHash, dest))
+ b.runConfig.Cmd = strslice.StrSlice{"/bin/sh", "-c", fmt.Sprintf("#(nop) %s %s in %s", cmdName, srcHash, dest)}
} else {
- b.runConfig.Cmd = strslice.New("cmd", "/S", "/C", fmt.Sprintf("REM (nop) %s %s in %s", cmdName, srcHash, dest))
+ b.runConfig.Cmd = strslice.StrSlice{"cmd", "/S", "/C", fmt.Sprintf("REM (nop) %s %s in %s", cmdName, srcHash, dest)}
}
- defer func(cmd *strslice.StrSlice) { b.runConfig.Cmd = cmd }(cmd)
+ defer func(cmd strslice.StrSlice) { b.runConfig.Cmd = cmd }(cmd)
if hit, err := b.probeCache(); err != nil {
return err
@@ -506,7 +505,7 @@ func (b *Builder) create() (string, error) {
// TODO: why not embed a hostconfig in builder?
hostConfig := &container.HostConfig{
- Isolation: b.options.IsolationLevel,
+ Isolation: b.options.Isolation,
ShmSize: b.options.ShmSize,
Resources: resources,
}
@@ -528,9 +527,9 @@ func (b *Builder) create() (string, error) {
b.tmpContainers[c.ID] = struct{}{}
fmt.Fprintf(b.Stdout, " ---> Running in %s\n", stringid.TruncateID(c.ID))
- if config.Cmd.Len() > 0 {
+ if len(config.Cmd) > 0 {
// override the entry point that may have been picked up from the base image
- if err := b.docker.ContainerUpdateCmdOnBuild(c.ID, config.Cmd.Slice()); err != nil {
+ if err := b.docker.ContainerUpdateCmdOnBuild(c.ID, config.Cmd); err != nil {
return "", err
}
}
@@ -568,7 +567,7 @@ func (b *Builder) run(cID string) (err error) {
if ret, _ := b.docker.ContainerWait(cID, -1); ret != 0 {
// TODO: change error type, because jsonmessage.JSONError assumes HTTP
return &jsonmessage.JSONError{
- Message: fmt.Sprintf("The command '%s' returned a non-zero code: %d", b.runConfig.Cmd.ToString(), ret),
+ Message: fmt.Sprintf("The command '%s' returned a non-zero code: %d", strings.Join(b.runConfig.Cmd, " "), ret),
Code: ret,
}
}
@@ -604,7 +603,7 @@ func (b *Builder) readDockerfile() error {
// that then look for 'dockerfile'. If neither are found then default
// back to 'Dockerfile' and use that in the error message.
if b.options.Dockerfile == "" {
- b.options.Dockerfile = api.DefaultDockerfileName
+ b.options.Dockerfile = builder.DefaultDockerfileName
if _, _, err := b.context.Stat(b.options.Dockerfile); os.IsNotExist(err) {
lowercase := strings.ToLower(b.options.Dockerfile)
if _, _, err := b.context.Stat(lowercase); err == nil {
diff --git a/components/engine/builder/dockerfile/parser/line_parsers.go b/components/engine/builder/dockerfile/parser/line_parsers.go
index b8792708d5..8cfd39bb2f 100644
--- a/components/engine/builder/dockerfile/parser/line_parsers.go
+++ b/components/engine/builder/dockerfile/parser/line_parsers.go
@@ -71,7 +71,7 @@ func parseWords(rest string) []string {
if unicode.IsSpace(ch) { // skip spaces
continue
}
- phase = inWord // found it, fall thru
+ phase = inWord // found it, fall through
}
if (phase == inWord || phase == inQuote) && (pos == len(rest)) {
if blankOK || len(word) > 0 {
diff --git a/components/engine/builder/dockerfile/parser/utils.go b/components/engine/builder/dockerfile/parser/utils.go
index 352d7a7e1c..b21eb62ae0 100644
--- a/components/engine/builder/dockerfile/parser/utils.go
+++ b/components/engine/builder/dockerfile/parser/utils.go
@@ -118,7 +118,7 @@ func extractBuilderFlags(line string) (string, []string, error) {
return line[pos:], words, nil
}
- phase = inWord // found someting with "--", fall thru
+ phase = inWord // found someting with "--", fall through
}
if (phase == inWord || phase == inQuote) && (pos == len(line)) {
if word != "--" && (blankOK || len(word) > 0) {
diff --git a/components/engine/builder/remote.go b/components/engine/builder/remote.go
index 3ab5923109..12f34c7b60 100644
--- a/components/engine/builder/remote.go
+++ b/components/engine/builder/remote.go
@@ -8,7 +8,6 @@ import (
"io/ioutil"
"regexp"
- "github.com/docker/docker/api"
"github.com/docker/docker/pkg/archive"
"github.com/docker/docker/pkg/httputils"
"github.com/docker/docker/pkg/urlutil"
@@ -87,7 +86,7 @@ func DetectContextFromRemoteURL(r io.ReadCloser, remoteURL string, createProgres
// dockerfileName is set to signal that the remote was interpreted as a single Dockerfile, in which case the caller
// should use dockerfileName as the new name for the Dockerfile, irrespective of any other user input.
- dockerfileName = api.DefaultDockerfileName
+ dockerfileName = DefaultDockerfileName
// TODO: return a context without tarsum
return archive.Generate(dockerfileName, string(dockerfile))
diff --git a/components/engine/cli/common.go b/components/engine/cli/common.go
index 1ece1fb616..df6a6ec115 100644
--- a/components/engine/cli/common.go
+++ b/components/engine/cli/common.go
@@ -19,7 +19,7 @@ type CommonFlags struct {
TrustKey string
}
-// Command is the struct contains command name and description
+// Command is the struct containing the command name and description
type Command struct {
Name string
Description string
@@ -42,7 +42,7 @@ var dockerCommands = []Command{
{"inspect", "Return low-level information on a container or image"},
{"kill", "Kill a running container"},
{"load", "Load an image from a tar archive or STDIN"},
- {"login", "Register or log in to a Docker registry"},
+ {"login", "Log in to a Docker registry"},
{"logout", "Log out from a Docker registry"},
{"logs", "Fetch the logs of a container"},
{"network", "Manage Docker networks"},
@@ -64,7 +64,7 @@ var dockerCommands = []Command{
{"tag", "Tag an image into a repository"},
{"top", "Display the running processes of a container"},
{"unpause", "Unpause all processes within a container"},
- {"update", "Update resources of one or more containers"},
+ {"update", "Update configuration of one or more containers"},
{"version", "Show the Docker version information"},
{"volume", "Manage Docker volumes"},
{"wait", "Block until a container stops, then print its exit code"},
diff --git a/components/engine/cliconfig/config.go b/components/engine/cliconfig/config.go
index f5b2be8a40..54e0ea387d 100644
--- a/components/engine/cliconfig/config.go
+++ b/components/engine/cliconfig/config.go
@@ -17,6 +17,7 @@ import (
const (
// ConfigFileName is the name of config file
ConfigFileName = "config.json"
+ configFileDir = ".docker"
oldConfigfile = ".dockercfg"
// This constant is only used for really old config files when the
@@ -31,7 +32,7 @@ var (
func init() {
if configDir == "" {
- configDir = filepath.Join(homedir.Get(), ".docker")
+ configDir = filepath.Join(homedir.Get(), configFileDir)
}
}
@@ -47,12 +48,13 @@ func SetConfigDir(dir string) {
// ConfigFile ~/.docker/config.json file info
type ConfigFile struct {
- AuthConfigs map[string]types.AuthConfig `json:"auths"`
- HTTPHeaders map[string]string `json:"HttpHeaders,omitempty"`
- PsFormat string `json:"psFormat,omitempty"`
- ImagesFormat string `json:"imagesFormat,omitempty"`
- DetachKeys string `json:"detachKeys,omitempty"`
- filename string // Note: not serialized - for internal use only
+ AuthConfigs map[string]types.AuthConfig `json:"auths"`
+ HTTPHeaders map[string]string `json:"HttpHeaders,omitempty"`
+ PsFormat string `json:"psFormat,omitempty"`
+ ImagesFormat string `json:"imagesFormat,omitempty"`
+ DetachKeys string `json:"detachKeys,omitempty"`
+ CredentialsStore string `json:"credsStore,omitempty"`
+ filename string // Note: not serialized - for internal use only
}
// NewConfigFile initializes an empty configuration file for the given filename 'fn'
@@ -86,11 +88,6 @@ func (configFile *ConfigFile) LegacyLoadFromReader(configData io.Reader) error {
if err != nil {
return err
}
- origEmail := strings.Split(arr[1], " = ")
- if len(origEmail) != 2 {
- return fmt.Errorf("Invalid Auth config file")
- }
- authConfig.Email = origEmail[1]
authConfig.ServerAddress = defaultIndexserver
configFile.AuthConfigs[defaultIndexserver] = authConfig
} else {
@@ -126,6 +123,13 @@ func (configFile *ConfigFile) LoadFromReader(configData io.Reader) error {
return nil
}
+// ContainsAuth returns whether there is authentication configured
+// in this file or not.
+func (configFile *ConfigFile) ContainsAuth() bool {
+ return configFile.CredentialsStore != "" ||
+ (configFile.AuthConfigs != nil && len(configFile.AuthConfigs) > 0)
+}
+
// LegacyLoadFromReader is a convenience function that creates a ConfigFile object from
// a non-nested reader
func LegacyLoadFromReader(configData io.Reader) (*ConfigFile, error) {
@@ -249,6 +253,10 @@ func (configFile *ConfigFile) Filename() string {
// encodeAuth creates a base64 encoded string to containing authorization information
func encodeAuth(authConfig *types.AuthConfig) string {
+ if authConfig.Username == "" && authConfig.Password == "" {
+ return ""
+ }
+
authStr := authConfig.Username + ":" + authConfig.Password
msg := []byte(authStr)
encoded := make([]byte, base64.StdEncoding.EncodedLen(len(msg)))
@@ -258,6 +266,10 @@ func encodeAuth(authConfig *types.AuthConfig) string {
// decodeAuth decodes a base64 encoded string and returns username and password
func decodeAuth(authStr string) (string, string, error) {
+ if authStr == "" {
+ return "", "", nil
+ }
+
decLen := base64.StdEncoding.DecodedLen(len(authStr))
decoded := make([]byte, decLen)
authByte := []byte(authStr)
diff --git a/components/engine/cliconfig/config_test.go b/components/engine/cliconfig/config_test.go
index 17f9fd673b..5ea6f4071c 100644
--- a/components/engine/cliconfig/config_test.go
+++ b/components/engine/cliconfig/config_test.go
@@ -111,12 +111,9 @@ func TestOldInvalidsAuth(t *testing.T) {
invalids := map[string]string{
`username = test`: "The Auth config file is empty",
`username
-password
-email`: "Invalid Auth config file",
+password`: "Invalid Auth config file",
`username = test
email`: "Invalid auth configuration file",
- `username = am9lam9lOmhlbGxv
-email`: "Invalid Auth config file",
}
tmpHome, err := ioutil.TempDir("", "config-test")
@@ -164,7 +161,7 @@ func TestOldValidAuth(t *testing.T) {
fn := filepath.Join(tmpHome, oldConfigfile)
js := `username = am9lam9lOmhlbGxv
-email = user@example.com`
+ email = user@example.com`
if err := ioutil.WriteFile(fn, []byte(js), 0600); err != nil {
t.Fatal(err)
}
@@ -176,15 +173,23 @@ email = user@example.com`
// defaultIndexserver is https://index.docker.io/v1/
ac := config.AuthConfigs["https://index.docker.io/v1/"]
- if ac.Email != "user@example.com" || ac.Username != "joejoe" || ac.Password != "hello" {
+ if ac.Username != "joejoe" || ac.Password != "hello" {
t.Fatalf("Missing data from parsing:\n%q", config)
}
// Now save it and make sure it shows up in new form
configStr := saveConfigAndValidateNewFormat(t, config, tmpHome)
- if !strings.Contains(configStr, "user@example.com") {
- t.Fatalf("Should have save in new form: %s", configStr)
+ expConfStr := `{
+ "auths": {
+ "https://index.docker.io/v1/": {
+ "auth": "am9lam9lOmhlbGxv"
+ }
+ }
+}`
+
+ if configStr != expConfStr {
+ t.Fatalf("Should have save in new form: \n%s\n not \n%s", configStr, expConfStr)
}
}
@@ -239,15 +244,24 @@ func TestOldJson(t *testing.T) {
}
ac := config.AuthConfigs["https://index.docker.io/v1/"]
- if ac.Email != "user@example.com" || ac.Username != "joejoe" || ac.Password != "hello" {
+ if ac.Username != "joejoe" || ac.Password != "hello" {
t.Fatalf("Missing data from parsing:\n%q", config)
}
// Now save it and make sure it shows up in new form
configStr := saveConfigAndValidateNewFormat(t, config, tmpHome)
- if !strings.Contains(configStr, "user@example.com") {
- t.Fatalf("Should have save in new form: %s", configStr)
+ expConfStr := `{
+ "auths": {
+ "https://index.docker.io/v1/": {
+ "auth": "am9lam9lOmhlbGxv",
+ "email": "user@example.com"
+ }
+ }
+}`
+
+ if configStr != expConfStr {
+ t.Fatalf("Should have save in new form: \n'%s'\n not \n'%s'\n", configStr, expConfStr)
}
}
@@ -259,7 +273,7 @@ func TestNewJson(t *testing.T) {
defer os.RemoveAll(tmpHome)
fn := filepath.Join(tmpHome, ConfigFileName)
- js := ` { "auths": { "https://index.docker.io/v1/": { "auth": "am9lam9lOmhlbGxv", "email": "user@example.com" } } }`
+ js := ` { "auths": { "https://index.docker.io/v1/": { "auth": "am9lam9lOmhlbGxv" } } }`
if err := ioutil.WriteFile(fn, []byte(js), 0600); err != nil {
t.Fatal(err)
}
@@ -270,15 +284,62 @@ func TestNewJson(t *testing.T) {
}
ac := config.AuthConfigs["https://index.docker.io/v1/"]
- if ac.Email != "user@example.com" || ac.Username != "joejoe" || ac.Password != "hello" {
+ if ac.Username != "joejoe" || ac.Password != "hello" {
t.Fatalf("Missing data from parsing:\n%q", config)
}
// Now save it and make sure it shows up in new form
configStr := saveConfigAndValidateNewFormat(t, config, tmpHome)
- if !strings.Contains(configStr, "user@example.com") {
- t.Fatalf("Should have save in new form: %s", configStr)
+ expConfStr := `{
+ "auths": {
+ "https://index.docker.io/v1/": {
+ "auth": "am9lam9lOmhlbGxv"
+ }
+ }
+}`
+
+ if configStr != expConfStr {
+ t.Fatalf("Should have save in new form: \n%s\n not \n%s", configStr, expConfStr)
+ }
+}
+
+func TestNewJsonNoEmail(t *testing.T) {
+ tmpHome, err := ioutil.TempDir("", "config-test")
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer os.RemoveAll(tmpHome)
+
+ fn := filepath.Join(tmpHome, ConfigFileName)
+ js := ` { "auths": { "https://index.docker.io/v1/": { "auth": "am9lam9lOmhlbGxv" } } }`
+ if err := ioutil.WriteFile(fn, []byte(js), 0600); err != nil {
+ t.Fatal(err)
+ }
+
+ config, err := Load(tmpHome)
+ if err != nil {
+ t.Fatalf("Failed loading on empty json file: %q", err)
+ }
+
+ ac := config.AuthConfigs["https://index.docker.io/v1/"]
+ if ac.Username != "joejoe" || ac.Password != "hello" {
+ t.Fatalf("Missing data from parsing:\n%q", config)
+ }
+
+ // Now save it and make sure it shows up in new form
+ configStr := saveConfigAndValidateNewFormat(t, config, tmpHome)
+
+ expConfStr := `{
+ "auths": {
+ "https://index.docker.io/v1/": {
+ "auth": "am9lam9lOmhlbGxv"
+ }
+ }
+}`
+
+ if configStr != expConfStr {
+ t.Fatalf("Should have save in new form: \n%s\n not \n%s", configStr, expConfStr)
}
}
@@ -366,7 +427,7 @@ func TestJsonReaderNoFile(t *testing.T) {
}
ac := config.AuthConfigs["https://index.docker.io/v1/"]
- if ac.Email != "user@example.com" || ac.Username != "joejoe" || ac.Password != "hello" {
+ if ac.Username != "joejoe" || ac.Password != "hello" {
t.Fatalf("Missing data from parsing:\n%q", config)
}
@@ -381,7 +442,7 @@ func TestOldJsonReaderNoFile(t *testing.T) {
}
ac := config.AuthConfigs["https://index.docker.io/v1/"]
- if ac.Email != "user@example.com" || ac.Username != "joejoe" || ac.Password != "hello" {
+ if ac.Username != "joejoe" || ac.Password != "hello" {
t.Fatalf("Missing data from parsing:\n%q", config)
}
}
@@ -404,7 +465,7 @@ func TestJsonWithPsFormatNoFile(t *testing.T) {
func TestJsonSaveWithNoFile(t *testing.T) {
js := `{
- "auths": { "https://index.docker.io/v1/": { "auth": "am9lam9lOmhlbGxv", "email": "user@example.com" } },
+ "auths": { "https://index.docker.io/v1/": { "auth": "am9lam9lOmhlbGxv" } },
"psFormat": "table {{.ID}}\\t{{.Label \"com.docker.label.cpu\"}}"
}`
config, err := LoadFromReader(strings.NewReader(js))
@@ -426,9 +487,16 @@ func TestJsonSaveWithNoFile(t *testing.T) {
t.Fatalf("Failed saving to file: %q", err)
}
buf, err := ioutil.ReadFile(filepath.Join(tmpHome, ConfigFileName))
- if !strings.Contains(string(buf), `"auths":`) ||
- !strings.Contains(string(buf), "user@example.com") {
- t.Fatalf("Should have save in new form: %s", string(buf))
+ expConfStr := `{
+ "auths": {
+ "https://index.docker.io/v1/": {
+ "auth": "am9lam9lOmhlbGxv"
+ }
+ },
+ "psFormat": "table {{.ID}}\\t{{.Label \"com.docker.label.cpu\"}}"
+}`
+ if string(buf) != expConfStr {
+ t.Fatalf("Should have save in new form: \n%s\nnot \n%s", string(buf), expConfStr)
}
}
@@ -454,14 +522,23 @@ func TestLegacyJsonSaveWithNoFile(t *testing.T) {
t.Fatalf("Failed saving to file: %q", err)
}
buf, err := ioutil.ReadFile(filepath.Join(tmpHome, ConfigFileName))
- if !strings.Contains(string(buf), `"auths":`) ||
- !strings.Contains(string(buf), "user@example.com") {
- t.Fatalf("Should have save in new form: %s", string(buf))
+
+ expConfStr := `{
+ "auths": {
+ "https://index.docker.io/v1/": {
+ "auth": "am9lam9lOmhlbGxv",
+ "email": "user@example.com"
+ }
+ }
+}`
+
+ if string(buf) != expConfStr {
+ t.Fatalf("Should have save in new form: \n%s\n not \n%s", string(buf), expConfStr)
}
}
func TestEncodeAuth(t *testing.T) {
- newAuthConfig := &types.AuthConfig{Username: "ken", Password: "test", Email: "test@example.com"}
+ newAuthConfig := &types.AuthConfig{Username: "ken", Password: "test"}
authStr := encodeAuth(newAuthConfig)
decAuthConfig := &types.AuthConfig{}
var err error
diff --git a/components/engine/cliconfig/credentials/credentials.go b/components/engine/cliconfig/credentials/credentials.go
new file mode 100644
index 0000000000..510cf8cf0e
--- /dev/null
+++ b/components/engine/cliconfig/credentials/credentials.go
@@ -0,0 +1,17 @@
+package credentials
+
+import (
+ "github.com/docker/engine-api/types"
+)
+
+// Store is the interface that any credentials store must implement.
+type Store interface {
+ // Erase removes credentials from the store for a given server.
+ Erase(serverAddress string) error
+ // Get retrieves credentials from the store for a given server.
+ Get(serverAddress string) (types.AuthConfig, error)
+ // GetAll retrieves all the credentials from the store.
+ GetAll() (map[string]types.AuthConfig, error)
+ // Store saves credentials in the store.
+ Store(authConfig types.AuthConfig) error
+}
diff --git a/components/engine/cliconfig/credentials/default_store.go b/components/engine/cliconfig/credentials/default_store.go
new file mode 100644
index 0000000000..b5fc47ccb3
--- /dev/null
+++ b/components/engine/cliconfig/credentials/default_store.go
@@ -0,0 +1,22 @@
+package credentials
+
+import (
+ "os/exec"
+
+ "github.com/docker/docker/cliconfig"
+)
+
+// DetectDefaultStore sets the default credentials store
+// if the host includes the default store helper program.
+func DetectDefaultStore(c *cliconfig.ConfigFile) {
+ if c.CredentialsStore != "" {
+ // user defined
+ return
+ }
+
+ if defaultCredentialsStore != "" {
+ if _, err := exec.LookPath(remoteCredentialsPrefix + defaultCredentialsStore); err == nil {
+ c.CredentialsStore = defaultCredentialsStore
+ }
+ }
+}
diff --git a/components/engine/cliconfig/credentials/default_store_darwin.go b/components/engine/cliconfig/credentials/default_store_darwin.go
new file mode 100644
index 0000000000..63e8ed4010
--- /dev/null
+++ b/components/engine/cliconfig/credentials/default_store_darwin.go
@@ -0,0 +1,3 @@
+package credentials
+
+const defaultCredentialsStore = "osxkeychain"
diff --git a/components/engine/cliconfig/credentials/default_store_linux.go b/components/engine/cliconfig/credentials/default_store_linux.go
new file mode 100644
index 0000000000..864c540f6c
--- /dev/null
+++ b/components/engine/cliconfig/credentials/default_store_linux.go
@@ -0,0 +1,3 @@
+package credentials
+
+const defaultCredentialsStore = "secretservice"
diff --git a/components/engine/cliconfig/credentials/default_store_unsupported.go b/components/engine/cliconfig/credentials/default_store_unsupported.go
new file mode 100644
index 0000000000..519ef53dcd
--- /dev/null
+++ b/components/engine/cliconfig/credentials/default_store_unsupported.go
@@ -0,0 +1,5 @@
+// +build !windows,!darwin,!linux
+
+package credentials
+
+const defaultCredentialsStore = ""
diff --git a/components/engine/cliconfig/credentials/default_store_windows.go b/components/engine/cliconfig/credentials/default_store_windows.go
new file mode 100644
index 0000000000..fb6a9745cf
--- /dev/null
+++ b/components/engine/cliconfig/credentials/default_store_windows.go
@@ -0,0 +1,3 @@
+package credentials
+
+const defaultCredentialsStore = "wincred"
diff --git a/components/engine/cliconfig/credentials/file_store.go b/components/engine/cliconfig/credentials/file_store.go
new file mode 100644
index 0000000000..8e7edd624a
--- /dev/null
+++ b/components/engine/cliconfig/credentials/file_store.go
@@ -0,0 +1,67 @@
+package credentials
+
+import (
+ "strings"
+
+ "github.com/docker/docker/cliconfig"
+ "github.com/docker/engine-api/types"
+)
+
+// fileStore implements a credentials store using
+// the docker configuration file to keep the credentials in plain text.
+type fileStore struct {
+ file *cliconfig.ConfigFile
+}
+
+// NewFileStore creates a new file credentials store.
+func NewFileStore(file *cliconfig.ConfigFile) Store {
+ return &fileStore{
+ file: file,
+ }
+}
+
+// Erase removes the given credentials from the file store.
+func (c *fileStore) Erase(serverAddress string) error {
+ delete(c.file.AuthConfigs, serverAddress)
+ return c.file.Save()
+}
+
+// Get retrieves credentials for a specific server from the file store.
+func (c *fileStore) Get(serverAddress string) (types.AuthConfig, error) {
+ authConfig, ok := c.file.AuthConfigs[serverAddress]
+ if !ok {
+ // Maybe they have a legacy config file, we will iterate the keys converting
+ // them to the new format and testing
+ for registry, ac := range c.file.AuthConfigs {
+ if serverAddress == convertToHostname(registry) {
+ return ac, nil
+ }
+ }
+
+ authConfig = types.AuthConfig{}
+ }
+ return authConfig, nil
+}
+
+func (c *fileStore) GetAll() (map[string]types.AuthConfig, error) {
+ return c.file.AuthConfigs, nil
+}
+
+// Store saves the given credentials in the file store.
+func (c *fileStore) Store(authConfig types.AuthConfig) error {
+ c.file.AuthConfigs[authConfig.ServerAddress] = authConfig
+ return c.file.Save()
+}
+
+func convertToHostname(url string) string {
+ stripped := url
+ if strings.HasPrefix(url, "http://") {
+ stripped = strings.Replace(url, "http://", "", 1)
+ } else if strings.HasPrefix(url, "https://") {
+ stripped = strings.Replace(url, "https://", "", 1)
+ }
+
+ nameParts := strings.SplitN(stripped, "/", 2)
+
+ return nameParts[0]
+}
diff --git a/components/engine/cliconfig/credentials/file_store_test.go b/components/engine/cliconfig/credentials/file_store_test.go
new file mode 100644
index 0000000000..668b6f097d
--- /dev/null
+++ b/components/engine/cliconfig/credentials/file_store_test.go
@@ -0,0 +1,138 @@
+package credentials
+
+import (
+ "io/ioutil"
+ "testing"
+
+ "github.com/docker/docker/cliconfig"
+ "github.com/docker/engine-api/types"
+)
+
+func newConfigFile(auths map[string]types.AuthConfig) *cliconfig.ConfigFile {
+ tmp, _ := ioutil.TempFile("", "docker-test")
+ name := tmp.Name()
+ tmp.Close()
+
+ c := cliconfig.NewConfigFile(name)
+ c.AuthConfigs = auths
+ return c
+}
+
+func TestFileStoreAddCredentials(t *testing.T) {
+ f := newConfigFile(make(map[string]types.AuthConfig))
+
+ s := NewFileStore(f)
+ err := s.Store(types.AuthConfig{
+ Auth: "super_secret_token",
+ Email: "foo@example.com",
+ ServerAddress: "https://example.com",
+ })
+
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ if len(f.AuthConfigs) != 1 {
+ t.Fatalf("expected 1 auth config, got %d", len(f.AuthConfigs))
+ }
+
+ a, ok := f.AuthConfigs["https://example.com"]
+ if !ok {
+ t.Fatalf("expected auth for https://example.com, got %v", f.AuthConfigs)
+ }
+ if a.Auth != "super_secret_token" {
+ t.Fatalf("expected auth `super_secret_token`, got %s", a.Auth)
+ }
+ if a.Email != "foo@example.com" {
+ t.Fatalf("expected email `foo@example.com`, got %s", a.Email)
+ }
+}
+
+func TestFileStoreGet(t *testing.T) {
+ f := newConfigFile(map[string]types.AuthConfig{
+ "https://example.com": {
+ Auth: "super_secret_token",
+ Email: "foo@example.com",
+ ServerAddress: "https://example.com",
+ },
+ })
+
+ s := NewFileStore(f)
+ a, err := s.Get("https://example.com")
+ if err != nil {
+ t.Fatal(err)
+ }
+ if a.Auth != "super_secret_token" {
+ t.Fatalf("expected auth `super_secret_token`, got %s", a.Auth)
+ }
+ if a.Email != "foo@example.com" {
+ t.Fatalf("expected email `foo@example.com`, got %s", a.Email)
+ }
+}
+
+func TestFileStoreGetAll(t *testing.T) {
+ s1 := "https://example.com"
+ s2 := "https://example2.com"
+ f := newConfigFile(map[string]types.AuthConfig{
+ s1: {
+ Auth: "super_secret_token",
+ Email: "foo@example.com",
+ ServerAddress: "https://example.com",
+ },
+ s2: {
+ Auth: "super_secret_token2",
+ Email: "foo@example2.com",
+ ServerAddress: "https://example2.com",
+ },
+ })
+
+ s := NewFileStore(f)
+ as, err := s.GetAll()
+ if err != nil {
+ t.Fatal(err)
+ }
+ if len(as) != 2 {
+ t.Fatalf("wanted 2, got %d", len(as))
+ }
+ if as[s1].Auth != "super_secret_token" {
+ t.Fatalf("expected auth `super_secret_token`, got %s", as[s1].Auth)
+ }
+ if as[s1].Email != "foo@example.com" {
+ t.Fatalf("expected email `foo@example.com`, got %s", as[s1].Email)
+ }
+ if as[s2].Auth != "super_secret_token2" {
+ t.Fatalf("expected auth `super_secret_token2`, got %s", as[s2].Auth)
+ }
+ if as[s2].Email != "foo@example2.com" {
+ t.Fatalf("expected email `foo@example2.com`, got %s", as[s2].Email)
+ }
+}
+
+func TestFileStoreErase(t *testing.T) {
+ f := newConfigFile(map[string]types.AuthConfig{
+ "https://example.com": {
+ Auth: "super_secret_token",
+ Email: "foo@example.com",
+ ServerAddress: "https://example.com",
+ },
+ })
+
+ s := NewFileStore(f)
+ err := s.Erase("https://example.com")
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ // file store never returns errors, check that the auth config is empty
+ a, err := s.Get("https://example.com")
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ if a.Auth != "" {
+ t.Fatalf("expected empty auth token, got %s", a.Auth)
+ }
+ if a.Email != "" {
+ t.Fatalf("expected empty email, got %s", a.Email)
+ }
+}
diff --git a/components/engine/cliconfig/credentials/native_store.go b/components/engine/cliconfig/credentials/native_store.go
new file mode 100644
index 0000000000..2da041d4b2
--- /dev/null
+++ b/components/engine/cliconfig/credentials/native_store.go
@@ -0,0 +1,180 @@
+package credentials
+
+import (
+ "bytes"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "io"
+ "strings"
+
+ "github.com/Sirupsen/logrus"
+ "github.com/docker/docker/cliconfig"
+ "github.com/docker/engine-api/types"
+)
+
+const remoteCredentialsPrefix = "docker-credential-"
+
+// Standarize the not found error, so every helper returns
+// the same message and docker can handle it properly.
+var errCredentialsNotFound = errors.New("credentials not found in native keychain")
+
+// command is an interface that remote executed commands implement.
+type command interface {
+ Output() ([]byte, error)
+ Input(in io.Reader)
+}
+
+// credentialsRequest holds information shared between docker and a remote credential store.
+type credentialsRequest struct {
+ ServerURL string
+ Username string
+ Password string
+}
+
+// credentialsGetResponse is the information serialized from a remote store
+// when the plugin sends requests to get the user credentials.
+type credentialsGetResponse struct {
+ Username string
+ Password string
+}
+
+// nativeStore implements a credentials store
+// using native keychain to keep credentials secure.
+// It piggybacks into a file store to keep users' emails.
+type nativeStore struct {
+ commandFn func(args ...string) command
+ fileStore Store
+}
+
+// NewNativeStore creates a new native store that
+// uses a remote helper program to manage credentials.
+func NewNativeStore(file *cliconfig.ConfigFile) Store {
+ return &nativeStore{
+ commandFn: shellCommandFn(file.CredentialsStore),
+ fileStore: NewFileStore(file),
+ }
+}
+
+// Erase removes the given credentials from the native store.
+func (c *nativeStore) Erase(serverAddress string) error {
+ if err := c.eraseCredentialsFromStore(serverAddress); err != nil {
+ return err
+ }
+
+ // Fallback to plain text store to remove email
+ return c.fileStore.Erase(serverAddress)
+}
+
+// Get retrieves credentials for a specific server from the native store.
+func (c *nativeStore) Get(serverAddress string) (types.AuthConfig, error) {
+ // load user email if it exist or an empty auth config.
+ auth, _ := c.fileStore.Get(serverAddress)
+
+ creds, err := c.getCredentialsFromStore(serverAddress)
+ if err != nil {
+ return auth, err
+ }
+ auth.Username = creds.Username
+ auth.Password = creds.Password
+
+ return auth, nil
+}
+
+// GetAll retrieves all the credentials from the native store.
+func (c *nativeStore) GetAll() (map[string]types.AuthConfig, error) {
+ auths, _ := c.fileStore.GetAll()
+
+ for s, ac := range auths {
+ creds, _ := c.getCredentialsFromStore(s)
+ ac.Username = creds.Username
+ ac.Password = creds.Password
+ auths[s] = ac
+ }
+
+ return auths, nil
+}
+
+// Store saves the given credentials in the file store.
+func (c *nativeStore) Store(authConfig types.AuthConfig) error {
+ if err := c.storeCredentialsInStore(authConfig); err != nil {
+ return err
+ }
+ authConfig.Username = ""
+ authConfig.Password = ""
+
+ // Fallback to old credential in plain text to save only the email
+ return c.fileStore.Store(authConfig)
+}
+
+// storeCredentialsInStore executes the command to store the credentials in the native store.
+func (c *nativeStore) storeCredentialsInStore(config types.AuthConfig) error {
+ cmd := c.commandFn("store")
+ creds := &credentialsRequest{
+ ServerURL: config.ServerAddress,
+ Username: config.Username,
+ Password: config.Password,
+ }
+
+ buffer := new(bytes.Buffer)
+ if err := json.NewEncoder(buffer).Encode(creds); err != nil {
+ return err
+ }
+ cmd.Input(buffer)
+
+ out, err := cmd.Output()
+ if err != nil {
+ t := strings.TrimSpace(string(out))
+ logrus.Debugf("error adding credentials - err: %v, out: `%s`", err, t)
+ return fmt.Errorf(t)
+ }
+
+ return nil
+}
+
+// getCredentialsFromStore executes the command to get the credentials from the native store.
+func (c *nativeStore) getCredentialsFromStore(serverAddress string) (types.AuthConfig, error) {
+ var ret types.AuthConfig
+
+ cmd := c.commandFn("get")
+ cmd.Input(strings.NewReader(serverAddress))
+
+ out, err := cmd.Output()
+ if err != nil {
+ t := strings.TrimSpace(string(out))
+
+ // do not return an error if the credentials are not
+ // in the keyckain. Let docker ask for new credentials.
+ if t == errCredentialsNotFound.Error() {
+ return ret, nil
+ }
+
+ logrus.Debugf("error getting credentials - err: %v, out: `%s`", err, t)
+ return ret, fmt.Errorf(t)
+ }
+
+ var resp credentialsGetResponse
+ if err := json.NewDecoder(bytes.NewReader(out)).Decode(&resp); err != nil {
+ return ret, err
+ }
+
+ ret.Username = resp.Username
+ ret.Password = resp.Password
+ ret.ServerAddress = serverAddress
+ return ret, nil
+}
+
+// eraseCredentialsFromStore executes the command to remove the server redentails from the native store.
+func (c *nativeStore) eraseCredentialsFromStore(serverURL string) error {
+ cmd := c.commandFn("erase")
+ cmd.Input(strings.NewReader(serverURL))
+
+ out, err := cmd.Output()
+ if err != nil {
+ t := strings.TrimSpace(string(out))
+ logrus.Debugf("error erasing credentials - err: %v, out: `%s`", err, t)
+ return fmt.Errorf(t)
+ }
+
+ return nil
+}
diff --git a/components/engine/cliconfig/credentials/native_store_test.go b/components/engine/cliconfig/credentials/native_store_test.go
new file mode 100644
index 0000000000..454fd0bd91
--- /dev/null
+++ b/components/engine/cliconfig/credentials/native_store_test.go
@@ -0,0 +1,309 @@
+package credentials
+
+import (
+ "encoding/json"
+ "fmt"
+ "io"
+ "io/ioutil"
+ "strings"
+ "testing"
+
+ "github.com/docker/engine-api/types"
+)
+
+const (
+ validServerAddress = "https://index.docker.io/v1"
+ validServerAddress2 = "https://example.com:5002"
+ invalidServerAddress = "https://foobar.example.com"
+ missingCredsAddress = "https://missing.docker.io/v1"
+)
+
+var errCommandExited = fmt.Errorf("exited 1")
+
+// mockCommand simulates interactions between the docker client and a remote
+// credentials helper.
+// Unit tests inject this mocked command into the remote to control execution.
+type mockCommand struct {
+ arg string
+ input io.Reader
+}
+
+// Output returns responses from the remote credentials helper.
+// It mocks those reponses based in the input in the mock.
+func (m *mockCommand) Output() ([]byte, error) {
+ in, err := ioutil.ReadAll(m.input)
+ if err != nil {
+ return nil, err
+ }
+ inS := string(in)
+
+ switch m.arg {
+ case "erase":
+ switch inS {
+ case validServerAddress:
+ return nil, nil
+ default:
+ return []byte("error erasing credentials"), errCommandExited
+ }
+ case "get":
+ switch inS {
+ case validServerAddress, validServerAddress2:
+ return []byte(`{"Username": "foo", "Password": "bar"}`), nil
+ case missingCredsAddress:
+ return []byte(errCredentialsNotFound.Error()), errCommandExited
+ case invalidServerAddress:
+ return []byte("error getting credentials"), errCommandExited
+ }
+ case "store":
+ var c credentialsRequest
+ err := json.NewDecoder(strings.NewReader(inS)).Decode(&c)
+ if err != nil {
+ return []byte("error storing credentials"), errCommandExited
+ }
+ switch c.ServerURL {
+ case validServerAddress:
+ return nil, nil
+ default:
+ return []byte("error storing credentials"), errCommandExited
+ }
+ }
+
+ return []byte(fmt.Sprintf("unknown argument %q with %q", m.arg, inS)), errCommandExited
+}
+
+// Input sets the input to send to a remote credentials helper.
+func (m *mockCommand) Input(in io.Reader) {
+ m.input = in
+}
+
+func mockCommandFn(args ...string) command {
+ return &mockCommand{
+ arg: args[0],
+ }
+}
+
+func TestNativeStoreAddCredentials(t *testing.T) {
+ f := newConfigFile(make(map[string]types.AuthConfig))
+ f.CredentialsStore = "mock"
+
+ s := &nativeStore{
+ commandFn: mockCommandFn,
+ fileStore: NewFileStore(f),
+ }
+ err := s.Store(types.AuthConfig{
+ Username: "foo",
+ Password: "bar",
+ Email: "foo@example.com",
+ ServerAddress: validServerAddress,
+ })
+
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ if len(f.AuthConfigs) != 1 {
+ t.Fatalf("expected 1 auth config, got %d", len(f.AuthConfigs))
+ }
+
+ a, ok := f.AuthConfigs[validServerAddress]
+ if !ok {
+ t.Fatalf("expected auth for %s, got %v", validServerAddress, f.AuthConfigs)
+ }
+ if a.Auth != "" {
+ t.Fatalf("expected auth to be empty, got %s", a.Auth)
+ }
+ if a.Username != "" {
+ t.Fatalf("expected username to be empty, got %s", a.Username)
+ }
+ if a.Password != "" {
+ t.Fatalf("expected password to be empty, got %s", a.Password)
+ }
+ if a.Email != "foo@example.com" {
+ t.Fatalf("expected email `foo@example.com`, got %s", a.Email)
+ }
+}
+
+func TestNativeStoreAddInvalidCredentials(t *testing.T) {
+ f := newConfigFile(make(map[string]types.AuthConfig))
+ f.CredentialsStore = "mock"
+
+ s := &nativeStore{
+ commandFn: mockCommandFn,
+ fileStore: NewFileStore(f),
+ }
+ err := s.Store(types.AuthConfig{
+ Username: "foo",
+ Password: "bar",
+ Email: "foo@example.com",
+ ServerAddress: invalidServerAddress,
+ })
+
+ if err == nil {
+ t.Fatal("expected error, got nil")
+ }
+
+ if err.Error() != "error storing credentials" {
+ t.Fatalf("expected `error storing credentials`, got %v", err)
+ }
+
+ if len(f.AuthConfigs) != 0 {
+ t.Fatalf("expected 0 auth config, got %d", len(f.AuthConfigs))
+ }
+}
+
+func TestNativeStoreGet(t *testing.T) {
+ f := newConfigFile(map[string]types.AuthConfig{
+ validServerAddress: {
+ Email: "foo@example.com",
+ },
+ })
+ f.CredentialsStore = "mock"
+
+ s := &nativeStore{
+ commandFn: mockCommandFn,
+ fileStore: NewFileStore(f),
+ }
+ a, err := s.Get(validServerAddress)
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ if a.Username != "foo" {
+ t.Fatalf("expected username `foo`, got %s", a.Username)
+ }
+ if a.Password != "bar" {
+ t.Fatalf("expected password `bar`, got %s", a.Password)
+ }
+ if a.Email != "foo@example.com" {
+ t.Fatalf("expected email `foo@example.com`, got %s", a.Email)
+ }
+}
+
+func TestNativeStoreGetAll(t *testing.T) {
+ f := newConfigFile(map[string]types.AuthConfig{
+ validServerAddress: {
+ Email: "foo@example.com",
+ },
+ validServerAddress2: {
+ Email: "foo@example2.com",
+ },
+ })
+ f.CredentialsStore = "mock"
+
+ s := &nativeStore{
+ commandFn: mockCommandFn,
+ fileStore: NewFileStore(f),
+ }
+ as, err := s.GetAll()
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ if len(as) != 2 {
+ t.Fatalf("wanted 2, got %d", len(as))
+ }
+
+ if as[validServerAddress].Username != "foo" {
+ t.Fatalf("expected username `foo` for %s, got %s", validServerAddress, as[validServerAddress].Username)
+ }
+ if as[validServerAddress].Password != "bar" {
+ t.Fatalf("expected password `bar` for %s, got %s", validServerAddress, as[validServerAddress].Password)
+ }
+ if as[validServerAddress].Email != "foo@example.com" {
+ t.Fatalf("expected email `foo@example.com` for %s, got %s", validServerAddress, as[validServerAddress].Email)
+ }
+ if as[validServerAddress2].Username != "foo" {
+ t.Fatalf("expected username `foo` for %s, got %s", validServerAddress2, as[validServerAddress2].Username)
+ }
+ if as[validServerAddress2].Password != "bar" {
+ t.Fatalf("expected password `bar` for %s, got %s", validServerAddress2, as[validServerAddress2].Password)
+ }
+ if as[validServerAddress2].Email != "foo@example2.com" {
+ t.Fatalf("expected email `foo@example2.com` for %s, got %s", validServerAddress2, as[validServerAddress2].Email)
+ }
+}
+
+func TestNativeStoreGetMissingCredentials(t *testing.T) {
+ f := newConfigFile(map[string]types.AuthConfig{
+ validServerAddress: {
+ Email: "foo@example.com",
+ },
+ })
+ f.CredentialsStore = "mock"
+
+ s := &nativeStore{
+ commandFn: mockCommandFn,
+ fileStore: NewFileStore(f),
+ }
+ _, err := s.Get(missingCredsAddress)
+ if err != nil {
+ // missing credentials do not produce an error
+ t.Fatal(err)
+ }
+}
+
+func TestNativeStoreGetInvalidAddress(t *testing.T) {
+ f := newConfigFile(map[string]types.AuthConfig{
+ validServerAddress: {
+ Email: "foo@example.com",
+ },
+ })
+ f.CredentialsStore = "mock"
+
+ s := &nativeStore{
+ commandFn: mockCommandFn,
+ fileStore: NewFileStore(f),
+ }
+ _, err := s.Get(invalidServerAddress)
+ if err == nil {
+ t.Fatal("expected error, got nil")
+ }
+
+ if err.Error() != "error getting credentials" {
+ t.Fatalf("expected `error getting credentials`, got %v", err)
+ }
+}
+
+func TestNativeStoreErase(t *testing.T) {
+ f := newConfigFile(map[string]types.AuthConfig{
+ validServerAddress: {
+ Email: "foo@example.com",
+ },
+ })
+ f.CredentialsStore = "mock"
+
+ s := &nativeStore{
+ commandFn: mockCommandFn,
+ fileStore: NewFileStore(f),
+ }
+ err := s.Erase(validServerAddress)
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ if len(f.AuthConfigs) != 0 {
+ t.Fatalf("expected 0 auth configs, got %d", len(f.AuthConfigs))
+ }
+}
+
+func TestNativeStoreEraseInvalidAddress(t *testing.T) {
+ f := newConfigFile(map[string]types.AuthConfig{
+ validServerAddress: {
+ Email: "foo@example.com",
+ },
+ })
+ f.CredentialsStore = "mock"
+
+ s := &nativeStore{
+ commandFn: mockCommandFn,
+ fileStore: NewFileStore(f),
+ }
+ err := s.Erase(invalidServerAddress)
+ if err == nil {
+ t.Fatal("expected error, got nil")
+ }
+
+ if err.Error() != "error erasing credentials" {
+ t.Fatalf("expected `error erasing credentials`, got %v", err)
+ }
+}
diff --git a/components/engine/cliconfig/credentials/shell_command.go b/components/engine/cliconfig/credentials/shell_command.go
new file mode 100644
index 0000000000..fa481b195d
--- /dev/null
+++ b/components/engine/cliconfig/credentials/shell_command.go
@@ -0,0 +1,28 @@
+package credentials
+
+import (
+ "io"
+ "os/exec"
+)
+
+func shellCommandFn(storeName string) func(args ...string) command {
+ name := remoteCredentialsPrefix + storeName
+ return func(args ...string) command {
+ return &shell{cmd: exec.Command(name, args...)}
+ }
+}
+
+// shell invokes shell commands to talk with a remote credentials helper.
+type shell struct {
+ cmd *exec.Cmd
+}
+
+// Output returns responses from the remote credentials helper.
+func (s *shell) Output() ([]byte, error) {
+ return s.cmd.Output()
+}
+
+// Input sets the input to send to a remote credentials helper.
+func (s *shell) Input(in io.Reader) {
+ s.cmd.Stdin = in
+}
diff --git a/components/engine/container/container.go b/components/engine/container/container.go
index c92e3de4d1..42ce6db8f4 100644
--- a/components/engine/container/container.go
+++ b/components/engine/container/container.go
@@ -16,13 +16,12 @@ import (
"github.com/docker/docker/daemon/logger"
"github.com/docker/docker/daemon/logger/jsonfilelog"
"github.com/docker/docker/daemon/network"
- derr "github.com/docker/docker/errors"
"github.com/docker/docker/image"
"github.com/docker/docker/layer"
+ "github.com/docker/docker/pkg/idtools"
"github.com/docker/docker/pkg/promise"
"github.com/docker/docker/pkg/signal"
"github.com/docker/docker/pkg/symlink"
- "github.com/docker/docker/pkg/system"
"github.com/docker/docker/runconfig"
"github.com/docker/docker/volume"
containertypes "github.com/docker/engine-api/types/container"
@@ -185,10 +184,17 @@ func (container *Container) WriteHostConfig() error {
}
// SetupWorkingDirectory sets up the container's working directory as set in container.Config.WorkingDir
-func (container *Container) SetupWorkingDirectory() error {
+func (container *Container) SetupWorkingDirectory(rootUID, rootGID int) error {
if container.Config.WorkingDir == "" {
return nil
}
+
+ // If can't mount container FS at this point (eg Hyper-V Containers on
+ // Windows) bail out now with no action.
+ if !container.canMountFS() {
+ return nil
+ }
+
container.Config.WorkingDir = filepath.Clean(container.Config.WorkingDir)
pth, err := container.GetResourcePath(container.Config.WorkingDir)
@@ -196,10 +202,10 @@ func (container *Container) SetupWorkingDirectory() error {
return err
}
- if err := system.MkdirAll(pth, 0755); err != nil {
+ if err := idtools.MkdirAllNewAs(pth, 0755, rootUID, rootGID); err != nil {
pthInfo, err2 := os.Stat(pth)
if err2 == nil && pthInfo != nil && !pthInfo.IsDir() {
- return derr.ErrorCodeNotADir.WithArgs(container.Config.WorkingDir)
+ return fmt.Errorf("Cannot mkdir: %s is not a directory", container.Config.WorkingDir)
}
return err
@@ -277,37 +283,17 @@ func (container *Container) ConfigPath() (string, error) {
return container.GetRootResourcePath(configFileName)
}
-func validateID(id string) error {
- if id == "" {
- return derr.ErrorCodeEmptyID
- }
- return nil
-}
-
// Returns true if the container exposes a certain port
func (container *Container) exposes(p nat.Port) bool {
_, exists := container.Config.ExposedPorts[p]
return exists
}
-// GetLogConfig returns the log configuration for the container.
-func (container *Container) GetLogConfig(defaultConfig containertypes.LogConfig) containertypes.LogConfig {
- cfg := container.HostConfig.LogConfig
- if cfg.Type != "" || len(cfg.Config) > 0 { // container has log driver configured
- if cfg.Type == "" {
- cfg.Type = jsonfilelog.Name
- }
- return cfg
- }
- // Use daemon's default log config for containers
- return defaultConfig
-}
-
// StartLogger starts a new logger driver for the container.
func (container *Container) StartLogger(cfg containertypes.LogConfig) (logger.Logger, error) {
c, err := logger.GetLogDriver(cfg.Type)
if err != nil {
- return nil, derr.ErrorCodeLoggingFactory.WithArgs(err)
+ return nil, fmt.Errorf("Failed to get logging factory: %v", err)
}
ctx := logger.Context{
Config: cfg.Config,
@@ -594,3 +580,20 @@ func (container *Container) InitDNSHostConfig() {
container.HostConfig.DNSOptions = make([]string, 0)
}
}
+
+// UpdateMonitor updates monitor configure for running container
+func (container *Container) UpdateMonitor(restartPolicy containertypes.RestartPolicy) {
+ monitor := container.monitor
+ // No need to update monitor if container hasn't got one
+ // monitor will be generated correctly according to container
+ if monitor == nil {
+ return
+ }
+
+ monitor.mux.Lock()
+ // to check whether restart policy has changed.
+ if restartPolicy.Name != "" && !monitor.restartPolicy.IsSame(&restartPolicy) {
+ monitor.restartPolicy = restartPolicy
+ }
+ monitor.mux.Unlock()
+}
diff --git a/components/engine/container/container_unix.go b/components/engine/container/container_unix.go
index 3d85e32e48..3cffb8a1d0 100644
--- a/components/engine/container/container_unix.go
+++ b/components/engine/container/container_unix.go
@@ -14,7 +14,6 @@ import (
"github.com/Sirupsen/logrus"
"github.com/docker/docker/daemon/execdriver"
- derr "github.com/docker/docker/errors"
"github.com/docker/docker/pkg/chrootarchive"
"github.com/docker/docker/pkg/symlink"
"github.com/docker/docker/pkg/system"
@@ -34,6 +33,11 @@ import (
// DefaultSHMSize is the default size (64MB) of the SHM which will be mounted in the container
const DefaultSHMSize int64 = 67108864
+var (
+ errInvalidEndpoint = fmt.Errorf("invalid endpoint while building port map info")
+ errInvalidNetwork = fmt.Errorf("invalid network settings while building port map info")
+)
+
// Container holds the fields specific to unixen implementations.
// See CommonContainer for standard fields common to all containers.
type Container struct {
@@ -46,6 +50,7 @@ type Container struct {
ShmPath string
ResolvConfPath string
SeccompProfile string
+ NoNewPrivileges bool
}
// CreateDaemonEnvironment returns the list of all environment variables given the list of
@@ -116,12 +121,12 @@ func (container *Container) GetEndpointInNetwork(n libnetwork.Network) (libnetwo
func (container *Container) buildPortMapInfo(ep libnetwork.Endpoint) error {
if ep == nil {
- return derr.ErrorCodeEmptyEndpoint
+ return errInvalidEndpoint
}
networkSettings := container.NetworkSettings
if networkSettings == nil {
- return derr.ErrorCodeEmptyNetwork
+ return errInvalidNetwork
}
if len(networkSettings.Ports) == 0 {
@@ -151,7 +156,7 @@ func getEndpointPortMapInfo(ep libnetwork.Endpoint) (nat.PortMap, error) {
for _, tp := range exposedPorts {
natPort, err := nat.NewPort(tp.Proto.String(), strconv.Itoa(int(tp.Port)))
if err != nil {
- return pm, derr.ErrorCodeParsingPort.WithArgs(tp.Port, err)
+ return pm, fmt.Errorf("Error parsing Port value(%v):%v", tp.Port, err)
}
pm[natPort] = nil
}
@@ -195,12 +200,12 @@ func getSandboxPortMapInfo(sb libnetwork.Sandbox) nat.PortMap {
// BuildEndpointInfo sets endpoint-related fields on container.NetworkSettings based on the provided network and endpoint.
func (container *Container) BuildEndpointInfo(n libnetwork.Network, ep libnetwork.Endpoint) error {
if ep == nil {
- return derr.ErrorCodeEmptyEndpoint
+ return errInvalidEndpoint
}
networkSettings := container.NetworkSettings
if networkSettings == nil {
- return derr.ErrorCodeEmptyNetwork
+ return errInvalidNetwork
}
epInfo := ep.Info()
@@ -285,7 +290,6 @@ func (container *Container) BuildJoinOptions(n libnetwork.Network) ([]libnetwork
// BuildCreateEndpointOptions builds endpoint options from a given network.
func (container *Container) BuildCreateEndpointOptions(n libnetwork.Network, epConfig *network.EndpointSettings, sb libnetwork.Sandbox) ([]libnetwork.EndpointOption, error) {
var (
- portSpecs = make(nat.PortSet)
bindings = make(nat.PortMap)
pbList []types.PortBinding
exposeList []types.TransportPort
@@ -338,10 +342,6 @@ func (container *Container) BuildCreateEndpointOptions(n libnetwork.Network, epC
return createOptions, nil
}
- if container.Config.ExposedPorts != nil {
- portSpecs = container.Config.ExposedPorts
- }
-
if container.HostConfig.PortBindings != nil {
for p, b := range container.HostConfig.PortBindings {
bindings[p] = []nat.PortBinding{}
@@ -354,6 +354,7 @@ func (container *Container) BuildCreateEndpointOptions(n libnetwork.Network, epC
}
}
+ portSpecs := container.Config.ExposedPorts
ports := make([]nat.Port, len(portSpecs))
var i int
for p := range portSpecs {
@@ -377,7 +378,7 @@ func (container *Container) BuildCreateEndpointOptions(n libnetwork.Network, epC
portStart, portEnd, err = newP.Range()
}
if err != nil {
- return nil, derr.ErrorCodeHostPort.WithArgs(binding[i].HostPort, err)
+ return nil, fmt.Errorf("Error parsing HostPort value(%s):%v", binding[i].HostPort, err)
}
pbCopy.HostPort = uint16(portStart)
pbCopy.HostPortEnd = uint16(portEnd)
@@ -498,11 +499,6 @@ func (container *Container) ShmResourcePath() (string, error) {
return container.GetRootResourcePath("shm")
}
-// MqueueResourcePath returns path to mqueue
-func (container *Container) MqueueResourcePath() (string, error) {
- return container.GetRootResourcePath("mqueue")
-}
-
// HasMountFor checks if path is a mountpoint
func (container *Container) HasMountFor(path string) bool {
_, exists := container.MountPoints[path]
@@ -564,10 +560,11 @@ func updateCommand(c *execdriver.Command, resources containertypes.Resources) {
c.Resources.KernelMemory = resources.KernelMemory
}
-// UpdateContainer updates resources of a container.
+// UpdateContainer updates configuration of a container.
func (container *Container) UpdateContainer(hostConfig *containertypes.HostConfig) error {
container.Lock()
+ // update resources of container
resources := hostConfig.Resources
cResources := &container.HostConfig.Resources
if resources.BlkioWeight != 0 {
@@ -600,6 +597,11 @@ func (container *Container) UpdateContainer(hostConfig *containertypes.HostConfi
if resources.KernelMemory != 0 {
cResources.KernelMemory = resources.KernelMemory
}
+
+ // update HostConfig of container
+ if hostConfig.RestartPolicy.Name != "" {
+ container.HostConfig.RestartPolicy = hostConfig.RestartPolicy
+ }
container.Unlock()
// If container is not running, update hostConfig struct is enough,
@@ -722,3 +724,9 @@ func (container *Container) TmpfsMounts() []execdriver.Mount {
func cleanResourcePath(path string) string {
return filepath.Join(string(os.PathSeparator), path)
}
+
+// canMountFS determines if the file system for the container
+// can be mounted locally. A no-op on non-Windows platforms
+func (container *Container) canMountFS() bool {
+ return true
+}
diff --git a/components/engine/container/container_windows.go b/components/engine/container/container_windows.go
index 61a8994244..18c7e0b0ff 100644
--- a/components/engine/container/container_windows.go
+++ b/components/engine/container/container_windows.go
@@ -3,12 +3,13 @@
package container
import (
+ "fmt"
"os"
"path/filepath"
"github.com/docker/docker/daemon/execdriver"
"github.com/docker/docker/volume"
- "github.com/docker/engine-api/types/container"
+ containertypes "github.com/docker/engine-api/types/container"
)
// Container holds fields specific to the Windows implementation. See
@@ -45,8 +46,22 @@ func (container *Container) TmpfsMounts() []execdriver.Mount {
return nil
}
-// UpdateContainer updates resources of a container
-func (container *Container) UpdateContainer(hostConfig *container.HostConfig) error {
+// UpdateContainer updates configuration of a container
+func (container *Container) UpdateContainer(hostConfig *containertypes.HostConfig) error {
+ container.Lock()
+ defer container.Unlock()
+ resources := hostConfig.Resources
+ if resources.BlkioWeight != 0 || resources.CPUShares != 0 ||
+ resources.CPUPeriod != 0 || resources.CPUQuota != 0 ||
+ resources.CpusetCpus != "" || resources.CpusetMems != "" ||
+ resources.Memory != 0 || resources.MemorySwap != 0 ||
+ resources.MemoryReservation != 0 || resources.KernelMemory != 0 {
+ return fmt.Errorf("Resource updating isn't supported on Windows")
+ }
+ // update HostConfig of container
+ if hostConfig.RestartPolicy.Name != "" {
+ container.HostConfig.RestartPolicy = hostConfig.RestartPolicy
+ }
return nil
}
@@ -68,3 +83,10 @@ func cleanResourcePath(path string) string {
}
return filepath.Join(string(os.PathSeparator), path)
}
+
+// canMountFS determines if the file system for the container
+// can be mounted locally. In the case of Windows, this is not possible
+// for Hyper-V containers during WORKDIR execution for example.
+func (container *Container) canMountFS() bool {
+ return !containertypes.Isolation.IsHyperV(container.HostConfig.Isolation)
+}
diff --git a/components/engine/container/monitor.go b/components/engine/container/monitor.go
index a292d8ce3f..afea01fcc9 100644
--- a/components/engine/container/monitor.go
+++ b/components/engine/container/monitor.go
@@ -1,6 +1,7 @@
package container
import (
+ "fmt"
"io"
"os/exec"
"strings"
@@ -10,10 +11,8 @@ import (
"github.com/Sirupsen/logrus"
"github.com/docker/docker/daemon/execdriver"
- derr "github.com/docker/docker/errors"
"github.com/docker/docker/pkg/promise"
"github.com/docker/docker/pkg/stringid"
- "github.com/docker/docker/utils"
"github.com/docker/engine-api/types/container"
)
@@ -79,11 +78,11 @@ type containerMonitor struct {
// StartMonitor initializes a containerMonitor for this container with the provided supervisor and restart policy
// and starts the container's process.
-func (container *Container) StartMonitor(s supervisor, policy container.RestartPolicy) error {
+func (container *Container) StartMonitor(s supervisor) error {
container.monitor = &containerMonitor{
supervisor: s,
container: container,
- restartPolicy: policy,
+ restartPolicy: container.HostConfig.RestartPolicy,
timeIncrement: defaultTimeIncrement,
stopChan: make(chan struct{}),
startSignal: make(chan struct{}),
@@ -126,9 +125,6 @@ func (m *containerMonitor) Close() error {
// Cleanup networking and mounts
m.supervisor.Cleanup(m.container)
- // FIXME: here is race condition between two RUN instructions in Dockerfile
- // because they share same runconfig and change image. Must be fixed
- // in builder/builder.go
if err := m.container.ToDisk(); err != nil {
logrus.Errorf("Error dumping container %s state to disk: %s", m.container.ID, err)
@@ -190,7 +186,7 @@ func (m *containerMonitor) start() error {
if m.container.RestartCount == 0 {
m.container.ExitCode = 127
m.resetContainer(false)
- return derr.ErrorCodeCmdNotFound
+ return fmt.Errorf("Container command not found or does not exist.")
}
}
// set to 126 for container cmd can't be invoked errors
@@ -198,7 +194,7 @@ func (m *containerMonitor) start() error {
if m.container.RestartCount == 0 {
m.container.ExitCode = 126
m.resetContainer(false)
- return derr.ErrorCodeCmdCouldNotBeInvoked
+ return fmt.Errorf("Container command could not be invoked.")
}
}
@@ -206,7 +202,7 @@ func (m *containerMonitor) start() error {
m.container.ExitCode = -1
m.resetContainer(false)
- return derr.ErrorCodeCantStart.WithArgs(m.container.ID, utils.GetErrorMessage(err))
+ return fmt.Errorf("Cannot start container %s: %v", m.container.ID, err)
}
logrus.Errorf("Error running container: %s", err)
diff --git a/components/engine/container/state.go b/components/engine/container/state.go
index 4a923aa968..7173c7632f 100644
--- a/components/engine/container/state.go
+++ b/components/engine/container/state.go
@@ -6,7 +6,6 @@ import (
"time"
"github.com/docker/docker/daemon/execdriver"
- derr "github.com/docker/docker/errors"
"github.com/docker/go-units"
)
@@ -113,17 +112,17 @@ func wait(waitChan <-chan struct{}, timeout time.Duration) error {
}
select {
case <-time.After(timeout):
- return derr.ErrorCodeTimedOut.WithArgs(timeout)
+ return fmt.Errorf("Timed out: %v", timeout)
case <-waitChan:
return nil
}
}
-// waitRunning waits until state is running. If state is already
+// WaitRunning waits until state is running. If state is already
// running it returns immediately. If you want wait forever you must
// supply negative timeout. Returns pid, that was passed to
// SetRunning.
-func (s *State) waitRunning(timeout time.Duration) (int, error) {
+func (s *State) WaitRunning(timeout time.Duration) (int, error) {
s.Lock()
if s.Running {
pid := s.Pid
@@ -256,14 +255,15 @@ func (s *State) IsRestarting() bool {
}
// SetRemovalInProgress sets the container state as being removed.
-func (s *State) SetRemovalInProgress() error {
+// It returns true if the container was already in that state.
+func (s *State) SetRemovalInProgress() bool {
s.Lock()
defer s.Unlock()
if s.RemovalInProgress {
- return derr.ErrorCodeAlreadyRemoving
+ return true
}
s.RemovalInProgress = true
- return nil
+ return false
}
// ResetRemovalInProgress make the RemovalInProgress state to false.
diff --git a/components/engine/container/state_test.go b/components/engine/container/state_test.go
index 00c45f1324..75028168d4 100644
--- a/components/engine/container/state_test.go
+++ b/components/engine/container/state_test.go
@@ -14,7 +14,7 @@ func TestStateRunStop(t *testing.T) {
started := make(chan struct{})
var pid int64
go func() {
- runPid, _ := s.waitRunning(-1 * time.Second)
+ runPid, _ := s.WaitRunning(-1 * time.Second)
atomic.StoreInt64(&pid, int64(runPid))
close(started)
}()
@@ -41,8 +41,8 @@ func TestStateRunStop(t *testing.T) {
if runPid != i+100 {
t.Fatalf("Pid %v, expected %v", runPid, i+100)
}
- if pid, err := s.waitRunning(-1 * time.Second); err != nil || pid != i+100 {
- t.Fatalf("waitRunning returned pid: %v, err: %v, expected pid: %v, err: %v", pid, err, i+100, nil)
+ if pid, err := s.WaitRunning(-1 * time.Second); err != nil || pid != i+100 {
+ t.Fatalf("WaitRunning returned pid: %v, err: %v, expected pid: %v, err: %v", pid, err, i+100, nil)
}
stopped := make(chan struct{})
@@ -82,7 +82,7 @@ func TestStateTimeoutWait(t *testing.T) {
s := NewState()
started := make(chan struct{})
go func() {
- s.waitRunning(100 * time.Millisecond)
+ s.WaitRunning(100 * time.Millisecond)
close(started)
}()
select {
@@ -98,7 +98,7 @@ func TestStateTimeoutWait(t *testing.T) {
stopped := make(chan struct{})
go func() {
- s.waitRunning(100 * time.Millisecond)
+ s.WaitRunning(100 * time.Millisecond)
close(stopped)
}()
select {
diff --git a/components/engine/contrib/README b/components/engine/contrib/README.md
similarity index 100%
rename from components/engine/contrib/README
rename to components/engine/contrib/README.md
diff --git a/components/engine/contrib/apparmor/main.go b/components/engine/contrib/apparmor/main.go
index 25f6e8c480..f4a2978b86 100644
--- a/components/engine/contrib/apparmor/main.go
+++ b/components/engine/contrib/apparmor/main.go
@@ -11,8 +11,7 @@ import (
)
type profileData struct {
- MajorVersion int
- MinorVersion int
+ Version int
}
func main() {
@@ -23,13 +22,12 @@ func main() {
// parse the arg
apparmorProfilePath := os.Args[1]
- majorVersion, minorVersion, err := aaparser.GetVersion()
+ version, err := aaparser.GetVersion()
if err != nil {
log.Fatal(err)
}
data := profileData{
- MajorVersion: majorVersion,
- MinorVersion: minorVersion,
+ Version: version,
}
fmt.Printf("apparmor_parser is of version %+v\n", data)
diff --git a/components/engine/contrib/apparmor/template.go b/components/engine/contrib/apparmor/template.go
index ea9c706d11..e5e1c8bed6 100644
--- a/components/engine/contrib/apparmor/template.go
+++ b/components/engine/contrib/apparmor/template.go
@@ -20,11 +20,11 @@ profile /usr/bin/docker (attach_disconnected, complain) {
umount,
pivot_root,
-{{if ge .MajorVersion 2}}{{if ge .MinorVersion 9}}
+{{if ge .Version 209000}}
signal (receive) peer=@{profile_name},
signal (receive) peer=unconfined,
signal (send),
-{{end}}{{end}}
+{{end}}
network,
capability,
owner /** rw,
@@ -46,12 +46,12 @@ profile /usr/bin/docker (attach_disconnected, complain) {
/etc/ld.so.cache r,
/etc/passwd r,
-{{if ge .MajorVersion 2}}{{if ge .MinorVersion 9}}
+{{if ge .Version 209000}}
ptrace peer=@{profile_name},
ptrace (read) peer=docker-default,
deny ptrace (trace) peer=docker-default,
deny ptrace peer=/usr/bin/docker///bin/ps,
-{{end}}{{end}}
+{{end}}
/usr/lib/** rm,
/lib/** rm,
@@ -72,11 +72,11 @@ profile /usr/bin/docker (attach_disconnected, complain) {
/sbin/zfs rCx,
/sbin/apparmor_parser rCx,
-{{if ge .MajorVersion 2}}{{if ge .MinorVersion 9}}
+{{if ge .Version 209000}}
# Transitions
change_profile -> docker-*,
change_profile -> unconfined,
-{{end}}{{end}}
+{{end}}
profile /bin/cat (complain) {
/etc/ld.so.cache r,
@@ -98,10 +98,10 @@ profile /usr/bin/docker (attach_disconnected, complain) {
/dev/null rw,
/bin/ps mr,
-{{if ge .MajorVersion 2}}{{if ge .MinorVersion 9}}
+{{if ge .Version 209000}}
# We don't need ptrace so we'll deny and ignore the error.
deny ptrace (read, trace),
-{{end}}{{end}}
+{{end}}
# Quiet dac_override denials
deny capability dac_override,
@@ -119,15 +119,15 @@ profile /usr/bin/docker (attach_disconnected, complain) {
/proc/tty/drivers r,
}
profile /sbin/iptables (complain) {
-{{if ge .MajorVersion 2}}{{if ge .MinorVersion 9}}
+{{if ge .Version 209000}}
signal (receive) peer=/usr/bin/docker,
-{{end}}{{end}}
+{{end}}
capability net_admin,
}
profile /sbin/auplink flags=(attach_disconnected, complain) {
-{{if ge .MajorVersion 2}}{{if ge .MinorVersion 9}}
+{{if ge .Version 209000}}
signal (receive) peer=/usr/bin/docker,
-{{end}}{{end}}
+{{end}}
capability sys_admin,
capability dac_override,
@@ -146,9 +146,9 @@ profile /usr/bin/docker (attach_disconnected, complain) {
/proc/[0-9]*/mounts rw,
}
profile /sbin/modprobe /bin/kmod (complain) {
-{{if ge .MajorVersion 2}}{{if ge .MinorVersion 9}}
+{{if ge .Version 209000}}
signal (receive) peer=/usr/bin/docker,
-{{end}}{{end}}
+{{end}}
capability sys_module,
/etc/ld.so.cache r,
/lib/** rm,
@@ -162,9 +162,9 @@ profile /usr/bin/docker (attach_disconnected, complain) {
}
# xz works via pipes, so we do not need access to the filesystem.
profile /usr/bin/xz (complain) {
-{{if ge .MajorVersion 2}}{{if ge .MinorVersion 9}}
+{{if ge .Version 209000}}
signal (receive) peer=/usr/bin/docker,
-{{end}}{{end}}
+{{end}}
/etc/ld.so.cache r,
/lib/** rm,
/usr/bin/xz rm,
diff --git a/components/engine/contrib/check-config.sh b/components/engine/contrib/check-config.sh
index 3a043658e7..bcc90d4a1f 100755
--- a/components/engine/contrib/check-config.sh
+++ b/components/engine/contrib/check-config.sh
@@ -115,6 +115,17 @@ check_device() {
fi
}
+check_distro_userns() {
+ source /etc/os-release 2>/dev/null || /bin/true
+ if [[ "${ID}" =~ ^(centos|rhel)$ && "${VERSION_ID}" =~ ^7 ]]; then
+ # this is a CentOS7 or RHEL7 system
+ grep -q "user_namespace.enable=1" /proc/cmdline || {
+ # no user namespace support enabled
+ wrap_bad " (RHEL7/CentOS7" "User namespaces disabled; add 'user_namespace.enable=1' to boot command line)"
+ }
+ fi
+}
+
if [ ! -e "$CONFIG" ]; then
wrap_warning "warning: $CONFIG does not exist, searching other paths for kernel config ..."
for tryConfig in "${possibleConfigs[@]}"; do
@@ -171,6 +182,7 @@ flags=(
NAMESPACES {NET,PID,IPC,UTS}_NS
DEVPTS_MULTIPLE_INSTANCES
CGROUPS CGROUP_CPUACCT CGROUP_DEVICE CGROUP_FREEZER CGROUP_SCHED CPUSETS MEMCG
+ KEYS
MACVLAN VETH BRIDGE BRIDGE_NETFILTER
NF_NAT_IPV4 IP_NF_FILTER IP_NF_TARGET_MASQUERADE
NETFILTER_XT_MATCH_{ADDRTYPE,CONNTRACK}
@@ -185,10 +197,14 @@ echo
echo 'Optional Features:'
{
check_flags USER_NS
+ check_distro_userns
}
{
check_flags SECCOMP
}
+{
+ check_flags CGROUP_PIDS
+}
{
check_flags MEMCG_KMEM MEMCG_SWAP MEMCG_SWAP_ENABLED
if is_set MEMCG_SWAP && ! is_set MEMCG_SWAP_ENABLED; then
diff --git a/components/engine/contrib/completion/bash/docker b/components/engine/contrib/completion/bash/docker
index 7588e33355..343e1dc717 100644
--- a/components/engine/contrib/completion/bash/docker
+++ b/components/engine/contrib/completion/bash/docker
@@ -395,7 +395,9 @@ __docker_complete_isolation() {
__docker_complete_log_drivers() {
COMPREPLY=( $( compgen -W "
awslogs
+ etwlogs
fluentd
+ gcplogs
gelf
journald
json-file
@@ -409,13 +411,14 @@ __docker_complete_log_options() {
# see docs/reference/logging/index.md
local awslogs_options="awslogs-region awslogs-group awslogs-stream"
local fluentd_options="env fluentd-address labels tag"
+ local gcplogs_options="env gcp-log-cmd gcp-project labels"
local gelf_options="env gelf-address labels tag"
local journald_options="env labels tag"
local json_file_options="env labels max-file max-size"
local syslog_options="syslog-address syslog-tls-ca-cert syslog-tls-cert syslog-tls-key syslog-tls-skip-verify syslog-facility tag"
local splunk_options="env labels splunk-caname splunk-capath splunk-index splunk-insecureskipverify splunk-source splunk-sourcetype splunk-token splunk-url tag"
- local all_options="$fluentd_options $gelf_options $journald_options $json_file_options $syslog_options $splunk_options"
+ local all_options="$fluentd_options $gcplogs_options $gelf_options $journald_options $json_file_options $syslog_options $splunk_options"
case $(__docker_value_of_option --log-driver) in
'')
@@ -427,6 +430,9 @@ __docker_complete_log_options() {
fluentd)
COMPREPLY=( $( compgen -W "$fluentd_options" -S = -- "$cur" ) )
;;
+ gcplogs)
+ COMPREPLY=( $( compgen -W "$gcplogs_options" -S = -- "$cur" ) )
+ ;;
gelf)
COMPREPLY=( $( compgen -W "$gelf_options" -S = -- "$cur" ) )
;;
@@ -515,6 +521,22 @@ __docker_complete_log_levels() {
COMPREPLY=( $( compgen -W "debug info warn error fatal" -- "$cur" ) )
}
+__docker_complete_restart() {
+ case "$prev" in
+ --restart)
+ case "$cur" in
+ on-failure:*)
+ ;;
+ *)
+ COMPREPLY=( $( compgen -W "always no on-failure on-failure: unless-stopped" -- "$cur") )
+ ;;
+ esac
+ return
+ ;;
+ esac
+ return 1
+}
+
# a selection of the available signals that is most likely of interest in the
# context of docker containers.
__docker_complete_signals() {
@@ -794,7 +816,7 @@ _docker_daemon() {
return
;;
esac
-
+
local key=$(__docker_map_key_of_current_option '--storage-opt')
case "$key" in
dm.@(blkdiscard|override_udev_sync_check|use_deferred_@(removal|deletion)))
@@ -1188,14 +1210,14 @@ _docker_load() {
_docker_login() {
case "$prev" in
- --email|-e|--password|-p|--username|-u)
+ --password|-p|--username|-u)
return
;;
esac
case "$cur" in
-*)
- COMPREPLY=( $( compgen -W "--email -e --help --password -p --username -u" -- "$cur" ) )
+ COMPREPLY=( $( compgen -W "--help --password -p --username -u" -- "$cur" ) )
;;
esac
}
@@ -1275,7 +1297,7 @@ _docker_network_connect() {
_docker_network_create() {
case "$prev" in
- --aux-address|--gateway|--ip-range|--ipam-opt|--opt|-o|--subnet)
+ --aux-address|--gateway|--internal|--ip-range|--ipam-opt|--ipv6|--opt|-o|--subnet)
return
;;
--ipam-driver)
@@ -1294,7 +1316,7 @@ _docker_network_create() {
case "$cur" in
-*)
- COMPREPLY=( $( compgen -W "--aux-address --driver -d --gateway --help --internal --ip-range --ipam-driver --ipam-opt --opt -o --subnet" -- "$cur" ) )
+ COMPREPLY=( $( compgen -W "--aux-address --driver -d --gateway --help --internal --ip-range --ipam-driver --ipam-opt --ipv6 --opt -o --subnet" -- "$cur" ) )
;;
esac
}
@@ -1615,6 +1637,7 @@ _docker_run() {
--net-alias
--oom-score-adj
--pid
+ --pids-limit
--publish -p
--restart
--security-opt
@@ -1657,6 +1680,7 @@ _docker_run() {
__docker_complete_log_driver_options && return
+ __docker_complete_restart && return
case "$prev" in
--add-host)
@@ -1754,16 +1778,6 @@ _docker_run() {
esac
return
;;
- --restart)
- case "$cur" in
- on-failure:*)
- ;;
- *)
- COMPREPLY=( $( compgen -W "always no on-failure on-failure: unless-stopped" -- "$cur") )
- ;;
- esac
- return
- ;;
--security-opt)
case "$cur" in
label:*:*)
@@ -1938,6 +1952,7 @@ _docker_update() {
--memory -m
--memory-reservation
--memory-swap
+ --restart
"
local boolean_options="
@@ -1946,6 +1961,8 @@ _docker_update() {
local all_options="$options_with_args $boolean_options"
+ __docker_complete_restart && return
+
case "$prev" in
$(__docker_to_extglob "$options_with_args") )
return
diff --git a/components/engine/contrib/completion/fish/docker.fish b/components/engine/contrib/completion/fish/docker.fish
index 17af1c0171..1e734c7d3a 100644
--- a/components/engine/contrib/completion/fish/docker.fish
+++ b/components/engine/contrib/completion/fish/docker.fish
@@ -221,8 +221,7 @@ complete -c docker -A -f -n '__fish_seen_subcommand_from load' -l help -d 'Print
complete -c docker -A -f -n '__fish_seen_subcommand_from load' -s i -l input -d 'Read from a tar archive file, instead of STDIN'
# login
-complete -c docker -f -n '__fish_docker_no_subcommand' -a login -d 'Register or log in to a Docker registry server'
-complete -c docker -A -f -n '__fish_seen_subcommand_from login' -s e -l email -d 'Email'
+complete -c docker -f -n '__fish_docker_no_subcommand' -a login -d 'Log in to a Docker registry server'
complete -c docker -A -f -n '__fish_seen_subcommand_from login' -l help -d 'Print usage'
complete -c docker -A -f -n '__fish_seen_subcommand_from login' -s p -l password -d 'Password'
complete -c docker -A -f -n '__fish_seen_subcommand_from login' -s u -l username -d 'Username'
@@ -290,6 +289,7 @@ complete -c docker -A -f -n '__fish_seen_subcommand_from rm' -l help -d 'Print u
complete -c docker -A -f -n '__fish_seen_subcommand_from rm' -s l -l link -d 'Remove the specified link and not the underlying container'
complete -c docker -A -f -n '__fish_seen_subcommand_from rm' -s v -l volumes -d 'Remove the volumes associated with the container'
complete -c docker -A -f -n '__fish_seen_subcommand_from rm' -a '(__fish_print_docker_containers stopped)' -d "Container"
+complete -c docker -A -f -n '__fish_seen_subcommand_from rm' -s f -l force -a '(__fish_print_docker_containers all)' -d "Container"
# rmi
complete -c docker -f -n '__fish_docker_no_subcommand' -a rmi -d 'Remove one or more images'
@@ -398,5 +398,3 @@ complete -c docker -f -n '__fish_docker_no_subcommand' -a version -d 'Show the D
complete -c docker -f -n '__fish_docker_no_subcommand' -a wait -d 'Block until a container stops, then print its exit code'
complete -c docker -A -f -n '__fish_seen_subcommand_from wait' -l help -d 'Print usage'
complete -c docker -A -f -n '__fish_seen_subcommand_from wait' -a '(__fish_print_docker_containers running)' -d "Container"
-
-
diff --git a/components/engine/contrib/completion/zsh/_docker b/components/engine/contrib/completion/zsh/_docker
index b77435a156..0f2a361a6b 100644
--- a/components/engine/contrib/completion/zsh/_docker
+++ b/components/engine/contrib/completion/zsh/_docker
@@ -201,6 +201,7 @@ __docker_get_log_options() {
awslogs_options=("awslogs-region" "awslogs-group" "awslogs-stream")
fluentd_options=("env" "fluentd-address" "labels" "tag")
+ gcplogs_options=("env" "gcp-log-cmd" "gcp-project" "labels")
gelf_options=("env" "gelf-address" "labels" "tag")
journald_options=("env" "labels")
json_file_options=("env" "labels" "max-file" "max-size")
@@ -209,6 +210,7 @@ __docker_get_log_options() {
[[ $log_driver = (awslogs|all) ]] && _describe -t awslogs-options "awslogs options" awslogs_options "$@" && ret=0
[[ $log_driver = (fluentd|all) ]] && _describe -t fluentd-options "fluentd options" fluentd_options "$@" && ret=0
+ [[ $log_driver = (gcplogs|all) ]] && _describe -t gcplogs-options "gcplogs options" gcplogs_options "$@" && ret=0
[[ $log_driver = (gelf|all) ]] && _describe -t gelf-options "gelf options" gelf_options "$@" && ret=0
[[ $log_driver = (journald|all) ]] && _describe -t journald-options "journald options" journald_options "$@" && ret=0
[[ $log_driver = (json-file|all) ]] && _describe -t json-file-options "json-file options" json_file_options "$@" && ret=0
@@ -325,14 +327,15 @@ __docker_network_subcommand() {
(create)
_arguments $(__docker_arguments) -A '-*' \
$opts_help \
- "($help)*--aux-address[Auxiliary ipv4 or ipv6 addresses used by network driver]:key=IP: " \
+ "($help)*--aux-address[Auxiliary IPv4 or IPv6 addresses used by network driver]:key=IP: " \
"($help -d --driver)"{-d=,--driver=}"[Driver to manage the Network]:driver:(null host bridge overlay)" \
- "($help)*--gateway=[ipv4 or ipv6 Gateway for the master subnet]:IP: " \
+ "($help)*--gateway=[IPv4 or IPv6 Gateway for the master subnet]:IP: " \
"($help)--internal[Restricts external access to the network]" \
"($help)*--ip-range=[Allocate container ip from a sub-range]:IP/mask: " \
"($help)--ipam-driver=[IP Address Management Driver]:driver:(default)" \
- "($help)*--ipam-opt=[Set custom IPAM plugin options]:opt=value: " \
- "($help)*"{-o=,--opt=}"[Set driver specific options]:opt=value: " \
+ "($help)*--ipam-opt=[Custom IPAM plugin options]:opt=value: " \
+ "($help)--ipv6[Enable IPv6 networking]" \
+ "($help)*"{-o=,--opt=}"[Driver specific options]:opt=value: " \
"($help)*--subnet=[Subnet in CIDR format that represents a network segment]:IP/mask: " \
"($help -)1:Network Name: " && ret=0
;;
@@ -421,9 +424,9 @@ __docker_volume_subcommand() {
(create)
_arguments $(__docker_arguments) \
$opts_help \
- "($help -d --driver)"{-d=,--driver=}"[Specify volume driver name]:Driver name:(local)" \
- "($help)--name=[Specify volume name]" \
- "($help)*"{-o=,--opt=}"[Set driver specific options]:Driver option: " && ret=0
+ "($help -d --driver)"{-d=,--driver=}"[Volume driver name]:Driver name:(local)" \
+ "($help)--name=[Volume name]" \
+ "($help)*"{-o=,--opt=}"[Driver specific options]:Driver option: " && ret=0
;;
(inspect)
_arguments $(__docker_arguments) \
@@ -483,8 +486,8 @@ __docker_subcommand() {
opts_help=("(: -)--help[Print usage]")
opts_build_create_run=(
"($help)--cgroup-parent=[Parent cgroup for the container]:cgroup: "
- "($help)--isolation=[]:isolation:(default hyperv process)"
- "($help)*--shm-size=[Size of '/dev/shm'. The format is ''. Default is '64m'.]:shm size: "
+ "($help)--isolation=[Container isolation technology]:isolation:(default hyperv process)"
+ "($help)*--shm-size=[Size of '/dev/shm' (format is '')]:shm size: "
"($help)*--ulimit=[ulimit options]:ulimit: "
)
opts_build_create_run_update=(
@@ -508,10 +511,10 @@ __docker_subcommand() {
"($help)*--device-read-iops=[Limit the read rate (IO per second) from a device]:device:IO rate: "
"($help)*--device-write-bps=[Limit the write rate (bytes per second) to a device]:device:IO rate: "
"($help)*--device-write-iops=[Limit the write rate (IO per second) to a device]:device:IO rate: "
- "($help)*--dns=[Set custom DNS servers]:DNS server: "
- "($help)*--dns-opt=[Set custom DNS options]:DNS option: "
- "($help)*--dns-search=[Set custom DNS search domains]:DNS domains: "
- "($help)*"{-e=,--env=}"[Set environment variables]:environment variable: "
+ "($help)*--dns=[Custom DNS servers]:DNS server: "
+ "($help)*--dns-opt=[Custom DNS options]:DNS option: "
+ "($help)*--dns-search=[Custom DNS search domains]:DNS domains: "
+ "($help)*"{-e=,--env=}"[Environment variables]:environment variable: "
"($help)--entrypoint=[Overwrite the default entrypoint of the image]:entry point: "
"($help)*--env-file=[Read environment variables from a file]:environment file:_files"
"($help)*--expose=[Expose a port from the container without publishing it]: "
@@ -522,7 +525,7 @@ __docker_subcommand() {
"($help)--ip6=[Container IPv6 address]:IPv6: "
"($help)--ipc=[IPC namespace to use]:IPC namespace: "
"($help)*--link=[Add link to another container]:link:->link"
- "($help)*"{-l=,--label=}"[Set meta data on a container]:label: "
+ "($help)*"{-l=,--label=}"[Container metadata]:label: "
"($help)--log-driver=[Default driver for container logs]:Logging driver:(json-file syslog journald gelf fluentd awslogs splunk none)"
"($help)*--log-opt=[Log driver specific options]:log driver options:__docker_log_options"
"($help)--mac-address=[Container MAC address]:MAC address: "
@@ -531,6 +534,7 @@ __docker_subcommand() {
"($help)*--net-alias=[Add network-scoped alias for the container]:alias: "
"($help)--oom-kill-disable[Disable OOM Killer]"
"($help)--oom-score-adj[Tune the host's OOM preferences for containers (accepts -1000 to 1000)]"
+ "($help)--pids-limit[Tune container pids limit (set -1 for unlimited)]"
"($help -P --publish-all)"{-P,--publish-all}"[Publish all exposed ports]"
"($help)*"{-p=,--publish=}"[Expose a container's port to the host]:port:_ports"
"($help)--pid=[PID namespace to use]:PID: "
@@ -548,11 +552,11 @@ __docker_subcommand() {
)
opts_create_run_update=(
"($help)--blkio-weight=[Block IO (relative weight), between 10 and 1000]:Block IO weight:(10 100 500 1000)"
- "($help)--kernel-memory=[Kernel memory limit in bytes.]:Memory limit: "
+ "($help)--kernel-memory=[Kernel memory limit in bytes]:Memory limit: "
"($help)--memory-reservation=[Memory soft limit]:Memory limit: "
)
opts_attach_exec_run_start=(
- "($help)--detach-keys=[Specify the escape key sequence used to detach a container]:sequence:__docker_complete_detach_keys"
+ "($help)--detach-keys=[Escape key sequence used to detach a container]:sequence:__docker_complete_detach_keys"
)
case "$words[1]" in
@@ -569,7 +573,7 @@ __docker_subcommand() {
$opts_help \
$opts_build_create_run \
$opts_build_create_run_update \
- "($help)*--build-arg[Set build-time variables]:=: " \
+ "($help)*--build-arg[Build-time variables]:=: " \
"($help -f --file)"{-f=,--file=}"[Name of the Dockerfile]:Dockerfile:_files" \
"($help)--force-rm[Always remove intermediate containers]" \
"($help)--no-cache[Do not use cache when building the image]" \
@@ -592,7 +596,7 @@ __docker_subcommand() {
(cp)
_arguments $(__docker_arguments) \
$opts_help \
- "($help -L --follow-link)"{-L,--follow-link}"[Always follow symbol link in SRC_PATH]" \
+ "($help -L --follow-link)"{-L,--follow-link}"[Always follow symbol link]" \
"($help -)1:container:->container" \
"($help -)2:hostpath:_files" && ret=0
case $state in
@@ -630,23 +634,23 @@ __docker_subcommand() {
(daemon)
_arguments $(__docker_arguments) \
$opts_help \
- "($help)--api-cors-header=[Set CORS headers in the remote API]:CORS headers: " \
- "($help)*--authorization-plugin=[Set authorization plugins to load]" \
+ "($help)--api-cors-header=[CORS headers in the remote API]:CORS headers: " \
+ "($help)*--authorization-plugin=[Authorization plugins to load]" \
"($help -b --bridge)"{-b=,--bridge=}"[Attach containers to a network bridge]:bridge:_net_interfaces" \
- "($help)--bip=[Specify network bridge IP]" \
- "($help)--cgroup-parent=[Set parent cgroup for all containers]:cgroup: " \
+ "($help)--bip=[Network bridge IP]:IP address: " \
+ "($help)--cgroup-parent=[Parent cgroup for all containers]:cgroup: " \
"($help -D --debug)"{-D,--debug}"[Enable debug mode]" \
"($help)--default-gateway[Container default gateway IPv4 address]:IPv4 address: " \
"($help)--default-gateway-v6[Container default gateway IPv6 address]:IPv6 address: " \
"($help)--cluster-store=[URL of the distributed storage backend]:Cluster Store:->cluster-store" \
"($help)--cluster-advertise=[Address of the daemon instance to advertise]:Instance to advertise (host\:port): " \
- "($help)*--cluster-store-opt=[Set cluster options]:Cluster options:->cluster-store-options" \
+ "($help)*--cluster-store-opt=[Cluster options]:Cluster options:->cluster-store-options" \
"($help)*--dns=[DNS server to use]:DNS: " \
"($help)*--dns-search=[DNS search domains to use]:DNS search: " \
"($help)*--dns-opt=[DNS options to use]:DNS option: " \
- "($help)*--default-ulimit=[Set default ulimit settings for containers]:ulimit: " \
+ "($help)*--default-ulimit=[Default ulimit settings for containers]:ulimit: " \
"($help)--disable-legacy-registry[Do not contact legacy registries]" \
- "($help)*--exec-opt=[Set exec driver options]:exec driver options: " \
+ "($help)*--exec-opt=[Exec driver options]:exec driver options: " \
"($help)--exec-root=[Root of the Docker execdriver]:path:_directories" \
"($help)--fixed-cidr=[IPv4 subnet for fixed IPs]:IPv4 subnet: " \
"($help)--fixed-cidr-v6=[IPv6 subnet for fixed IPs]:IPv6 subnet: " \
@@ -660,17 +664,17 @@ __docker_subcommand() {
"($help)--ip-masq[Enable IP masquerading]" \
"($help)--iptables[Enable addition of iptables rules]" \
"($help)--ipv6[Enable IPv6 networking]" \
- "($help -l --log-level)"{-l=,--log-level=}"[Set the logging level]:level:(debug info warn error fatal)" \
- "($help)*--label=[Set key=value labels to the daemon]:label: " \
+ "($help -l --log-level)"{-l=,--log-level=}"[Logging level]:level:(debug info warn error fatal)" \
+ "($help)*--label=[Key=value labels]:label: " \
"($help)--log-driver=[Default driver for container logs]:Logging driver:(json-file syslog journald gelf fluentd awslogs splunk none)" \
"($help)*--log-opt=[Log driver specific options]:log driver options:__docker_log_options" \
- "($help)--mtu=[Set the containers network MTU]:mtu:(0 576 1420 1500 9000)" \
+ "($help)--mtu=[Network MTU]:mtu:(0 576 1420 1500 9000)" \
"($help -p --pidfile)"{-p=,--pidfile=}"[Path to use for daemon PID file]:PID file:_files" \
"($help)--raw-logs[Full timestamps without ANSI coloring]" \
"($help)*--registry-mirror=[Preferred Docker registry mirror]:registry mirror: " \
"($help -s --storage-driver)"{-s=,--storage-driver=}"[Storage driver to use]:driver:(aufs devicemapper btrfs zfs overlay)" \
"($help)--selinux-enabled[Enable selinux support]" \
- "($help)*--storage-opt=[Set storage driver options]:storage driver options: " \
+ "($help)*--storage-opt=[Storage driver options]:storage driver options: " \
"($help)--tls[Use TLS]" \
"($help)--tlscacert=[Trust certs signed only by this CA]:PEM file:_files -g "*.(pem|crt)"" \
"($help)--tlscert=[Path to TLS certificate file]:PEM file:_files -g "*.(pem|crt)"" \
@@ -768,7 +772,7 @@ __docker_subcommand() {
_arguments $(__docker_arguments) \
$opts_help \
"($help)*"{-c=,--change=}"[Apply Dockerfile instruction to the created image]:Dockerfile:_files" \
- "($help -m --message)"{-m=,--message=}"[Set commit message for imported image]:message: " \
+ "($help -m --message)"{-m=,--message=}"[Commit message for imported image]:message: " \
"($help -):URL:(- http:// file://)" \
"($help -): :__docker_repositories_with_tags" && ret=0
;;
@@ -811,7 +815,6 @@ __docker_subcommand() {
(login)
_arguments $(__docker_arguments) \
$opts_help \
- "($help -e --email)"{-e=,--email=}"[Email]:email: " \
"($help -p --password)"{-p=,--password=}"[Password]:password: " \
"($help -u --user)"{-u=,--user=}"[Username]:username: " \
"($help -)1:server: " && ret=0
@@ -1048,7 +1051,7 @@ _docker() {
"($help)--config[Location of client config files]:path:_directories" \
"($help -D --debug)"{-D,--debug}"[Enable debug mode]" \
"($help -H --host)"{-H=,--host=}"[tcp://host:port to bind/connect to]:host: " \
- "($help -l --log-level)"{-l=,--log-level=}"[Set the logging level]:level:(debug info warn error fatal)" \
+ "($help -l --log-level)"{-l=,--log-level=}"[Logging level]:level:(debug info warn error fatal)" \
"($help)--tls[Use TLS]" \
"($help)--tlscacert=[Trust certs signed only by this CA]:PEM file:_files -g "*.(pem|crt)"" \
"($help)--tlscert=[Path to TLS certificate file]:PEM file:_files -g "*.(pem|crt)"" \
diff --git a/components/engine/contrib/download-frozen-image-v2.sh b/components/engine/contrib/download-frozen-image-v2.sh
index 81b047561e..111e3fa2ba 100755
--- a/components/engine/contrib/download-frozen-image-v2.sh
+++ b/components/engine/contrib/download-frozen-image-v2.sh
@@ -95,6 +95,7 @@ while [ $# -gt 0 ]; do
echo "skipping existing ${imageId:0:12}"
continue
fi
+ token="$(curl -sSL "https://auth.docker.io/token?service=registry.docker.io&scope=repository:$image:pull" | jq --raw-output .token)"
curl -SL --progress -H "Authorization: Bearer $token" "https://registry-1.docker.io/v2/$image/blobs/$imageLayer" -o "$dir/$imageId/layer.tar" # -C -
done
echo
diff --git a/components/engine/contrib/init/systemd/docker.service b/components/engine/contrib/init/systemd/docker.service
index 6015b7441f..75cb68c8b3 100644
--- a/components/engine/contrib/init/systemd/docker.service
+++ b/components/engine/contrib/init/systemd/docker.service
@@ -6,12 +6,17 @@ Requires=docker.socket
[Service]
Type=notify
+# the default is not to use systemd for cgroups because the delegate issues still
+# exists and systemd currently does not support the cgroup feature set required
+# for containers run by docker
ExecStart=/usr/bin/docker daemon -H fd://
MountFlags=slave
LimitNOFILE=1048576
LimitNPROC=1048576
LimitCORE=infinity
TimeoutStartSec=0
+# set delegate yes so that systemd does not reset the cgroups of docker containers
+Delegate=yes
[Install]
WantedBy=multi-user.target
diff --git a/components/engine/contrib/nnp-test/Dockerfile b/components/engine/contrib/nnp-test/Dockerfile
new file mode 100644
index 0000000000..026d86954f
--- /dev/null
+++ b/components/engine/contrib/nnp-test/Dockerfile
@@ -0,0 +1,9 @@
+FROM buildpack-deps:jessie
+
+COPY . /usr/src/
+
+WORKDIR /usr/src/
+
+RUN gcc -g -Wall -static nnp-test.c -o /usr/bin/nnp-test
+
+RUN chmod +s /usr/bin/nnp-test
diff --git a/components/engine/contrib/nnp-test/nnp-test.c b/components/engine/contrib/nnp-test/nnp-test.c
new file mode 100644
index 0000000000..b767da7e1a
--- /dev/null
+++ b/components/engine/contrib/nnp-test/nnp-test.c
@@ -0,0 +1,10 @@
+#include
+#include
+#include
+
+int main(int argc, char *argv[])
+{
+ printf("EUID=%d\n", geteuid());
+ return 0;
+}
+
diff --git a/components/engine/contrib/report-issue.sh b/components/engine/contrib/report-issue.sh
old mode 100644
new mode 100755
diff --git a/components/engine/daemon/archive.go b/components/engine/daemon/archive.go
index 5d0e9e0b52..5cf6985210 100644
--- a/components/engine/daemon/archive.go
+++ b/components/engine/daemon/archive.go
@@ -250,13 +250,13 @@ func (daemon *Daemon) containerExtractToDir(container *container.Container, path
return ErrRootFSReadOnly
}
+ uid, gid := daemon.GetRemappedUIDGID()
options := &archive.TarOptions{
- ChownOpts: &archive.TarChownOptions{
- UID: 0, GID: 0, // TODO: use config.User? Remap to userns root?
- },
NoOverwriteDirNonDir: noOverwriteDirNonDir,
+ ChownOpts: &archive.TarChownOptions{
+ UID: uid, GID: gid, // TODO: should all ownership be set to root (either real or remapped)?
+ },
}
-
if err := chrootarchive.Untar(content, resolvedPath, options); err != nil {
return err
}
diff --git a/components/engine/daemon/attach.go b/components/engine/daemon/attach.go
index 1beedbf138..79e9cd51da 100644
--- a/components/engine/daemon/attach.go
+++ b/components/engine/daemon/attach.go
@@ -9,7 +9,7 @@ import (
"github.com/docker/docker/api/types/backend"
"github.com/docker/docker/container"
"github.com/docker/docker/daemon/logger"
- derr "github.com/docker/docker/errors"
+ "github.com/docker/docker/errors"
"github.com/docker/docker/pkg/stdcopy"
)
@@ -17,10 +17,11 @@ import (
func (daemon *Daemon) ContainerAttach(prefixOrName string, c *backend.ContainerAttachConfig) error {
container, err := daemon.GetContainer(prefixOrName)
if err != nil {
- return derr.ErrorCodeNoSuchContainer.WithArgs(prefixOrName)
+ return err
}
if container.IsPaused() {
- return derr.ErrorCodePausedContainer.WithArgs(prefixOrName)
+ err := fmt.Errorf("Container %s is paused. Unpause the container before attach", prefixOrName)
+ return errors.NewRequestConflictError(err)
}
inStream, outStream, errStream, err := c.GetStreams()
diff --git a/components/engine/daemon/commit.go b/components/engine/daemon/commit.go
index 6f2b38d52d..7bc7b6f25d 100644
--- a/components/engine/daemon/commit.go
+++ b/components/engine/daemon/commit.go
@@ -70,8 +70,8 @@ func merge(userConf, imageConf *containertypes.Config) error {
userConf.Labels = imageConf.Labels
}
- if userConf.Entrypoint.Len() == 0 {
- if userConf.Cmd.Len() == 0 {
+ if len(userConf.Entrypoint) == 0 {
+ if len(userConf.Cmd) == 0 {
userConf.Cmd = imageConf.Cmd
}
@@ -89,6 +89,10 @@ func merge(userConf, imageConf *containertypes.Config) error {
userConf.Volumes[k] = v
}
}
+
+ if userConf.StopSignal == "" {
+ userConf.StopSignal = imageConf.StopSignal
+ }
return nil
}
@@ -147,7 +151,7 @@ func (daemon *Daemon) Commit(name string, c *types.ContainerCommitConfig) (strin
h := image.History{
Author: c.Author,
Created: time.Now().UTC(),
- CreatedBy: strings.Join(container.Config.Cmd.Slice(), " "),
+ CreatedBy: strings.Join(container.Config.Cmd, " "),
Comment: c.Comment,
EmptyLayer: true,
}
diff --git a/components/engine/daemon/config.go b/components/engine/daemon/config.go
index fb46f26667..50e814ba4e 100644
--- a/components/engine/daemon/config.go
+++ b/components/engine/daemon/config.go
@@ -154,14 +154,20 @@ func parseClusterAdvertiseSettings(clusterStore, clusterAdvertise string) (strin
}
// ReloadConfiguration reads the configuration in the host and reloads the daemon and server.
-func ReloadConfiguration(configFile string, flags *flag.FlagSet, reload func(*Config)) {
+func ReloadConfiguration(configFile string, flags *flag.FlagSet, reload func(*Config)) error {
logrus.Infof("Got signal to reload configuration, reloading from: %s", configFile)
newConfig, err := getConflictFreeConfiguration(configFile, flags)
if err != nil {
- logrus.Error(err)
- } else {
- reload(newConfig)
+ return err
}
+ reload(newConfig)
+ return nil
+}
+
+// boolValue is an interface that boolean value flags implement
+// to tell the command line how to make -name equivalent to -name=true.
+type boolValue interface {
+ IsBoolFlag() bool
}
// MergeDaemonConfigurations reads a configuration file,
@@ -206,6 +212,36 @@ func getConflictFreeConfiguration(configFile string, flags *flag.FlagSet) (*Conf
return nil, err
}
+ // Override flag values to make sure the values set in the config file with nullable values, like `false`,
+ // are not overriden by default truthy values from the flags that were not explicitly set.
+ // See https://github.com/docker/docker/issues/20289 for an example.
+ //
+ // TODO: Rewrite configuration logic to avoid same issue with other nullable values, like numbers.
+ namedOptions := make(map[string]interface{})
+ for key, value := range configSet {
+ f := flags.Lookup("-" + key)
+ if f == nil { // ignore named flags that don't match
+ namedOptions[key] = value
+ continue
+ }
+
+ if _, ok := f.Value.(boolValue); ok {
+ f.Value.Set(fmt.Sprintf("%v", value))
+ }
+ }
+ if len(namedOptions) > 0 {
+ // set also default for mergeVal flags that are boolValue at the same time.
+ flags.VisitAll(func(f *flag.Flag) {
+ if opt, named := f.Value.(opts.NamedOption); named {
+ v, set := namedOptions[opt.Name()]
+ _, boolean := f.Value.(boolValue)
+ if set && boolean {
+ f.Value.Set(fmt.Sprintf("%v", v))
+ }
+ }
+ })
+ }
+
config.valuesSet = configSet
}
@@ -245,14 +281,16 @@ func findConfigurationConflicts(config map[string]interface{}, flags *flag.FlagS
// 2. Discard values that implement NamedOption.
// Their configuration name differs from their flag name, like `labels` and `label`.
- unknownNamedConflicts := func(f *flag.Flag) {
- if namedOption, ok := f.Value.(opts.NamedOption); ok {
- if _, valid := unknownKeys[namedOption.Name()]; valid {
- delete(unknownKeys, namedOption.Name())
+ if len(unknownKeys) > 0 {
+ unknownNamedConflicts := func(f *flag.Flag) {
+ if namedOption, ok := f.Value.(opts.NamedOption); ok {
+ if _, valid := unknownKeys[namedOption.Name()]; valid {
+ delete(unknownKeys, namedOption.Name())
+ }
}
}
+ flags.VisitAll(unknownNamedConflicts)
}
- flags.VisitAll(unknownNamedConflicts)
if len(unknownKeys) > 0 {
var unknown []string
diff --git a/components/engine/daemon/container_operations_unix.go b/components/engine/daemon/container_operations_unix.go
index 4db5b4d62f..e93f2dad4c 100644
--- a/components/engine/daemon/container_operations_unix.go
+++ b/components/engine/daemon/container_operations_unix.go
@@ -4,6 +4,7 @@ package daemon
import (
"fmt"
+ "net"
"os"
"path"
"path/filepath"
@@ -17,7 +18,7 @@ import (
"github.com/docker/docker/daemon/execdriver"
"github.com/docker/docker/daemon/links"
"github.com/docker/docker/daemon/network"
- derr "github.com/docker/docker/errors"
+ "github.com/docker/docker/errors"
"github.com/docker/docker/pkg/fileutils"
"github.com/docker/docker/pkg/idtools"
"github.com/docker/docker/pkg/mount"
@@ -25,10 +26,12 @@ import (
"github.com/docker/docker/runconfig"
containertypes "github.com/docker/engine-api/types/container"
networktypes "github.com/docker/engine-api/types/network"
+ "github.com/docker/go-connections/nat"
"github.com/docker/go-units"
"github.com/docker/libnetwork"
"github.com/docker/libnetwork/netlabel"
"github.com/docker/libnetwork/options"
+ "github.com/docker/libnetwork/types"
"github.com/opencontainers/runc/libcontainer/configs"
"github.com/opencontainers/runc/libcontainer/devices"
"github.com/opencontainers/runc/libcontainer/label"
@@ -45,7 +48,7 @@ func (daemon *Daemon) setupLinkedContainers(container *container.Container) ([]s
for linkAlias, child := range children {
if !child.IsRunning() {
- return nil, derr.ErrorCodeLinkNotRunning.WithArgs(child.Name, linkAlias)
+ return nil, fmt.Errorf("Cannot link to a non running container: %s AS %s", child.Name, linkAlias)
}
childBridgeSettings := child.NetworkSettings.Networks["bridge"]
@@ -198,6 +201,7 @@ func (daemon *Daemon) populateCommand(c *container.Container, env []string) erro
BlkioThrottleWriteBpsDevice: writeBpsDevice,
BlkioThrottleReadIOpsDevice: readIOpsDevice,
BlkioThrottleWriteIOpsDevice: writeIOpsDevice,
+ PidsLimit: c.HostConfig.PidsLimit,
MemorySwappiness: -1,
}
@@ -257,8 +261,8 @@ func (daemon *Daemon) populateCommand(c *container.Container, env []string) erro
AllowedDevices: allowedDevices,
AppArmorProfile: c.AppArmorProfile,
AutoCreatedDevices: autoCreatedDevices,
- CapAdd: c.HostConfig.CapAdd.Slice(),
- CapDrop: c.HostConfig.CapDrop.Slice(),
+ CapAdd: c.HostConfig.CapAdd,
+ CapDrop: c.HostConfig.CapDrop,
CgroupParent: defaultCgroupParent,
GIDMapping: gidMap,
GroupAdd: c.HostConfig.GroupAdd,
@@ -270,6 +274,7 @@ func (daemon *Daemon) populateCommand(c *container.Container, env []string) erro
SeccompProfile: c.SeccompProfile,
UIDMapping: uidMap,
UTS: uts,
+ NoNewPrivileges: c.NoNewPrivileges,
}
if c.HostConfig.CgroupParent != "" {
c.Command.CgroupParent = c.HostConfig.CgroupParent
@@ -318,6 +323,9 @@ func (daemon *Daemon) buildSandboxOptions(container *container.Container, n libn
dns []string
dnsSearch []string
dnsOptions []string
+ bindings = make(nat.PortMap)
+ pbList []types.PortBinding
+ exposeList []types.TransportPort
)
sboxOptions = append(sboxOptions, libnetwork.OptionHostname(container.Config.Hostname),
@@ -392,6 +400,59 @@ func (daemon *Daemon) buildSandboxOptions(container *container.Container, n libn
sboxOptions = append(sboxOptions, libnetwork.OptionExtraHost(parts[0], parts[1]))
}
+ if container.HostConfig.PortBindings != nil {
+ for p, b := range container.HostConfig.PortBindings {
+ bindings[p] = []nat.PortBinding{}
+ for _, bb := range b {
+ bindings[p] = append(bindings[p], nat.PortBinding{
+ HostIP: bb.HostIP,
+ HostPort: bb.HostPort,
+ })
+ }
+ }
+ }
+
+ portSpecs := container.Config.ExposedPorts
+ ports := make([]nat.Port, len(portSpecs))
+ var i int
+ for p := range portSpecs {
+ ports[i] = p
+ i++
+ }
+ nat.SortPortMap(ports, bindings)
+ for _, port := range ports {
+ expose := types.TransportPort{}
+ expose.Proto = types.ParseProtocol(port.Proto())
+ expose.Port = uint16(port.Int())
+ exposeList = append(exposeList, expose)
+
+ pb := types.PortBinding{Port: expose.Port, Proto: expose.Proto}
+ binding := bindings[port]
+ for i := 0; i < len(binding); i++ {
+ pbCopy := pb.GetCopy()
+ newP, err := nat.NewPort(nat.SplitProtoPort(binding[i].HostPort))
+ var portStart, portEnd int
+ if err == nil {
+ portStart, portEnd, err = newP.Range()
+ }
+ if err != nil {
+ return nil, fmt.Errorf("Error parsing HostPort value(%s):%v", binding[i].HostPort, err)
+ }
+ pbCopy.HostPort = uint16(portStart)
+ pbCopy.HostPortEnd = uint16(portEnd)
+ pbCopy.HostIP = net.ParseIP(binding[i].HostIP)
+ pbList = append(pbList, pbCopy)
+ }
+
+ if container.HostConfig.PublishAllPorts && len(binding) == 0 {
+ pbList = append(pbList, pb)
+ }
+ }
+
+ sboxOptions = append(sboxOptions,
+ libnetwork.OptionPortMapping(pbList),
+ libnetwork.OptionExposedPorts(exposeList))
+
// Link feature is supported only for the default bridge network.
// return if this call to build join options is not for default bridge network
if n.Name() != "bridge" {
@@ -509,7 +570,7 @@ func (daemon *Daemon) updateNetwork(container *container.Container) error {
sb, err := ctrl.SandboxByID(sid)
if err != nil {
- return derr.ErrorCodeNoSandbox.WithArgs(sid, err)
+ return fmt.Errorf("error locating sandbox id %s: %v", sid, err)
}
// Find if container is connected to the default bridge network
@@ -532,11 +593,11 @@ func (daemon *Daemon) updateNetwork(container *container.Container) error {
options, err := daemon.buildSandboxOptions(container, n)
if err != nil {
- return derr.ErrorCodeNetworkUpdate.WithArgs(err)
+ return fmt.Errorf("Update network failed: %v", err)
}
if err := sb.Refresh(options...); err != nil {
- return derr.ErrorCodeNetworkRefresh.WithArgs(sid, err)
+ return fmt.Errorf("Update network failed: Failure in refresh sandbox %s: %v", sid, err)
}
return nil
@@ -730,7 +791,7 @@ func (daemon *Daemon) updateNetworkConfig(container *container.Container, idOrNa
func (daemon *Daemon) ConnectToNetwork(container *container.Container, idOrName string, endpointConfig *networktypes.EndpointSettings) error {
if !container.Running {
if container.RemovalInProgress || container.Dead {
- return derr.ErrorCodeRemovalContainer.WithArgs(container.ID)
+ return errRemovalContainer(container.ID)
}
if _, err := daemon.updateNetworkConfig(container, idOrName, endpointConfig, true); err != nil {
return err
@@ -810,7 +871,7 @@ func (daemon *Daemon) connectToNetwork(container *container.Container, idOrName
}
if err := container.UpdateJoinInfo(n, ep); err != nil {
- return derr.ErrorCodeJoinInfo.WithArgs(err)
+ return fmt.Errorf("Updating join info failed: %v", err)
}
daemon.LogNetworkEventWithAttributes(n, "connect", map[string]string{"container": container.ID})
@@ -833,7 +894,7 @@ func (daemon *Daemon) DisconnectFromNetwork(container *container.Container, n li
}
if !container.Running {
if container.RemovalInProgress || container.Dead {
- return derr.ErrorCodeRemovalContainer.WithArgs(container.ID)
+ return errRemovalContainer(container.ID)
}
if _, ok := container.NetworkSettings.Networks[n.Name()]; ok {
delete(container.NetworkSettings.Networks, n.Name())
@@ -950,7 +1011,7 @@ func (daemon *Daemon) setNetworkNamespaceKey(containerID string, pid int) error
search := libnetwork.SandboxContainerWalker(&sandbox, containerID)
daemon.netController.WalkSandboxes(search)
if sandbox == nil {
- return derr.ErrorCodeNoSandbox.WithArgs(containerID, "no sandbox found")
+ return fmt.Errorf("error locating sandbox id %s: no sandbox found", containerID)
}
return sandbox.SetKey(path)
@@ -963,10 +1024,10 @@ func (daemon *Daemon) getIpcContainer(container *container.Container) (*containe
return nil, err
}
if !c.IsRunning() {
- return nil, derr.ErrorCodeIPCRunning.WithArgs(containerID)
+ return nil, fmt.Errorf("cannot join IPC of a non running container: %s", containerID)
}
if c.IsRestarting() {
- return nil, derr.ErrorCodeContainerRestarting.WithArgs(containerID)
+ return nil, errContainerIsRestarting(container.ID)
}
return c, nil
}
@@ -977,13 +1038,14 @@ func (daemon *Daemon) getNetworkedContainer(containerID, connectedContainerID st
return nil, err
}
if containerID == nc.ID {
- return nil, derr.ErrorCodeJoinSelf
+ return nil, fmt.Errorf("cannot join own network")
}
if !nc.IsRunning() {
- return nil, derr.ErrorCodeJoinRunning.WithArgs(connectedContainerID)
+ err := fmt.Errorf("cannot join network of a non running container: %s", connectedContainerID)
+ return nil, errors.NewRequestConflictError(err)
}
if nc.IsRestarting() {
- return nil, derr.ErrorCodeContainerRestarting.WithArgs(connectedContainerID)
+ return nil, errContainerIsRestarting(connectedContainerID)
}
return nc, nil
}
@@ -1098,7 +1160,9 @@ func killProcessDirectly(container *container.Container) error {
if err != syscall.ESRCH {
return err
}
- logrus.Debugf("Cannot kill process (pid=%d) with signal 9: no such process.", pid)
+ e := errNoSuchProcess{pid, 9}
+ logrus.Debug(e)
+ return e
}
}
}
@@ -1106,7 +1170,16 @@ func killProcessDirectly(container *container.Container) error {
}
func getDevicesFromPath(deviceMapping containertypes.DeviceMapping) (devs []*configs.Device, err error) {
- device, err := devices.DeviceFromPath(deviceMapping.PathOnHost, deviceMapping.CgroupPermissions)
+ resolvedPathOnHost := deviceMapping.PathOnHost
+
+ // check if it is a symbolic link
+ if src, e := os.Lstat(deviceMapping.PathOnHost); e == nil && src.Mode()&os.ModeSymlink == os.ModeSymlink {
+ if linkedPathOnHost, e := os.Readlink(deviceMapping.PathOnHost); e == nil {
+ resolvedPathOnHost = linkedPathOnHost
+ }
+ }
+
+ device, err := devices.DeviceFromPath(resolvedPathOnHost, deviceMapping.CgroupPermissions)
// if there was no error, return the device
if err == nil {
device.Path = deviceMapping.PathInContainer
@@ -1118,10 +1191,10 @@ func getDevicesFromPath(deviceMapping containertypes.DeviceMapping) (devs []*con
if err == devices.ErrNotADevice {
// check if it is a directory
- if src, e := os.Stat(deviceMapping.PathOnHost); e == nil && src.IsDir() {
+ if src, e := os.Stat(resolvedPathOnHost); e == nil && src.IsDir() {
// mount the internal devices recursively
- filepath.Walk(deviceMapping.PathOnHost, func(dpath string, f os.FileInfo, e error) error {
+ filepath.Walk(resolvedPathOnHost, func(dpath string, f os.FileInfo, e error) error {
childDevice, e := devices.DeviceFromPath(dpath, deviceMapping.CgroupPermissions)
if e != nil {
// ignore the device
@@ -1129,7 +1202,7 @@ func getDevicesFromPath(deviceMapping containertypes.DeviceMapping) (devs []*con
}
// add the device to userSpecified devices
- childDevice.Path = strings.Replace(dpath, deviceMapping.PathOnHost, deviceMapping.PathInContainer, 1)
+ childDevice.Path = strings.Replace(dpath, resolvedPathOnHost, deviceMapping.PathInContainer, 1)
devs = append(devs, childDevice)
return nil
@@ -1141,7 +1214,7 @@ func getDevicesFromPath(deviceMapping containertypes.DeviceMapping) (devs []*con
return devs, nil
}
- return devs, derr.ErrorCodeDeviceInfo.WithArgs(deviceMapping.PathOnHost, err)
+ return devs, fmt.Errorf("error gathering device information while adding custom device %q: %s", deviceMapping.PathOnHost, err)
}
func mergeDevices(defaultDevices, userDevices []*configs.Device) []*configs.Device {
@@ -1172,3 +1245,7 @@ func isLinkable(child *container.Container) bool {
_, ok := child.NetworkSettings.Networks["bridge"]
return ok
}
+
+func errRemovalContainer(containerID string) error {
+ return fmt.Errorf("Container %s is marked for removal and cannot be connected or disconnected to the network", containerID)
+}
diff --git a/components/engine/daemon/container_operations_windows.go b/components/engine/daemon/container_operations_windows.go
index b812bfc5f4..02173d09f1 100644
--- a/components/engine/daemon/container_operations_windows.go
+++ b/components/engine/daemon/container_operations_windows.go
@@ -3,12 +3,12 @@
package daemon
import (
+ "fmt"
"strings"
"github.com/docker/docker/container"
"github.com/docker/docker/daemon/execdriver"
"github.com/docker/docker/daemon/execdriver/windows"
- derr "github.com/docker/docker/errors"
"github.com/docker/docker/layer"
networktypes "github.com/docker/engine-api/types/network"
"github.com/docker/libnetwork"
@@ -64,7 +64,7 @@ func (daemon *Daemon) populateCommand(c *container.Container, env []string) erro
}
}
default:
- return derr.ErrorCodeInvalidNetworkMode.WithArgs(c.HostConfig.NetworkMode)
+ return fmt.Errorf("invalid network mode: %s", c.HostConfig.NetworkMode)
}
// TODO Windows. More resource controls to be implemented later.
@@ -88,7 +88,7 @@ func (daemon *Daemon) populateCommand(c *container.Container, env []string) erro
var layerPaths []string
img, err := daemon.imageStore.Get(c.ImageID)
if err != nil {
- return derr.ErrorCodeGetGraph.WithArgs(c.ImageID, err)
+ return fmt.Errorf("Failed to graph.Get on ImageID %s - %s", c.ImageID, err)
}
if img.RootFS != nil && img.RootFS.Type == "layers+base" {
@@ -97,7 +97,7 @@ func (daemon *Daemon) populateCommand(c *container.Container, env []string) erro
img.RootFS.DiffIDs = img.RootFS.DiffIDs[:i]
path, err := layer.GetLayerPath(daemon.layerStore, img.RootFS.ChainID())
if err != nil {
- return derr.ErrorCodeGetLayer.WithArgs(err)
+ return fmt.Errorf("Failed to get layer path from graphdriver %s for ImageID %s - %s", daemon.layerStore, img.RootFS.ChainID(), err)
}
// Reverse order, expecting parent most first
layerPaths = append([]string{path}, layerPaths...)
@@ -106,7 +106,7 @@ func (daemon *Daemon) populateCommand(c *container.Container, env []string) erro
m, err := c.RWLayer.Metadata()
if err != nil {
- return derr.ErrorCodeGetLayerMetadata.WithArgs(err)
+ return fmt.Errorf("Failed to get layer metadata - %s", err)
}
layerFolder := m["dir"]
diff --git a/components/engine/daemon/create.go b/components/engine/daemon/create.go
index 166af3bf73..425c4344bb 100644
--- a/components/engine/daemon/create.go
+++ b/components/engine/daemon/create.go
@@ -1,9 +1,10 @@
package daemon
import (
+ "fmt"
+
"github.com/Sirupsen/logrus"
"github.com/docker/docker/container"
- derr "github.com/docker/docker/errors"
"github.com/docker/docker/image"
"github.com/docker/docker/layer"
"github.com/docker/docker/pkg/idtools"
@@ -18,10 +19,10 @@ import (
// ContainerCreate creates a container.
func (daemon *Daemon) ContainerCreate(params types.ContainerCreateConfig) (types.ContainerCreateResponse, error) {
if params.Config == nil {
- return types.ContainerCreateResponse{}, derr.ErrorCodeEmptyConfig
+ return types.ContainerCreateResponse{}, fmt.Errorf("Config cannot be empty in order to create a container")
}
- warnings, err := daemon.verifyContainerSettings(params.HostConfig, params.Config)
+ warnings, err := daemon.verifyContainerSettings(params.HostConfig, params.Config, false)
if err != nil {
return types.ContainerCreateResponse{Warnings: warnings}, err
}
@@ -174,7 +175,7 @@ func (daemon *Daemon) VolumeCreate(name, driverName string, opts map[string]stri
v, err := daemon.volumes.Create(name, driverName, opts)
if err != nil {
if volumestore.IsNameConflict(err) {
- return nil, derr.ErrorVolumeNameTaken.WithArgs(name)
+ return nil, fmt.Errorf("A volume named %s already exists. Choose a different volume name.", name)
}
return nil, err
}
diff --git a/components/engine/daemon/create_unix.go b/components/engine/daemon/create_unix.go
index 8eca648deb..583ca13b76 100644
--- a/components/engine/daemon/create_unix.go
+++ b/components/engine/daemon/create_unix.go
@@ -3,12 +3,12 @@
package daemon
import (
+ "fmt"
"os"
"path/filepath"
"github.com/Sirupsen/logrus"
"github.com/docker/docker/container"
- derr "github.com/docker/docker/errors"
"github.com/docker/docker/pkg/stringid"
containertypes "github.com/docker/engine-api/types/container"
"github.com/opencontainers/runc/libcontainer/label"
@@ -21,7 +21,8 @@ func (daemon *Daemon) createContainerPlatformSpecificSettings(container *contain
}
defer daemon.Unmount(container)
- if err := container.SetupWorkingDirectory(); err != nil {
+ rootUID, rootGID := daemon.GetRemappedUIDGID()
+ if err := container.SetupWorkingDirectory(rootUID, rootGID); err != nil {
return err
}
@@ -41,7 +42,7 @@ func (daemon *Daemon) createContainerPlatformSpecificSettings(container *contain
stat, err := os.Stat(path)
if err == nil && !stat.IsDir() {
- return derr.ErrorCodeMountOverFile.WithArgs(path)
+ return fmt.Errorf("cannot mount volume over existing file, file exists %s", path)
}
v, err := daemon.volumes.CreateWithRef(name, hostConfig.VolumeDriver, container.ID, nil)
diff --git a/components/engine/daemon/daemon.go b/components/engine/daemon/daemon.go
index 33c3f10caf..892e83dac0 100644
--- a/components/engine/daemon/daemon.go
+++ b/components/engine/daemon/daemon.go
@@ -6,7 +6,6 @@
package daemon
import (
- "errors"
"fmt"
"io"
"io/ioutil"
@@ -15,6 +14,7 @@ import (
"path"
"path/filepath"
"runtime"
+ "strings"
"sync"
"syscall"
"time"
@@ -28,6 +28,7 @@ import (
"github.com/docker/docker/daemon/exec"
"github.com/docker/docker/daemon/execdriver"
"github.com/docker/docker/daemon/execdriver/execdrivers"
+ "github.com/docker/docker/errors"
"github.com/docker/engine-api/types"
containertypes "github.com/docker/engine-api/types/container"
eventtypes "github.com/docker/engine-api/types/events"
@@ -43,7 +44,6 @@ import (
dmetadata "github.com/docker/docker/distribution/metadata"
"github.com/docker/docker/distribution/xfer"
"github.com/docker/docker/dockerversion"
- derr "github.com/docker/docker/errors"
"github.com/docker/docker/image"
"github.com/docker/docker/image/tarexport"
"github.com/docker/docker/layer"
@@ -90,7 +90,7 @@ var (
validContainerNameChars = utils.RestrictedNameChars
validContainerNamePattern = utils.RestrictedNamePattern
- errSystemNotSupported = errors.New("The Docker daemon is not supported on this platform.")
+ errSystemNotSupported = fmt.Errorf("The Docker daemon is not supported on this platform.")
)
// ErrImageDoesNotExist is error returned when no image can be found for a reference.
@@ -157,7 +157,8 @@ func (daemon *Daemon) GetContainer(prefixOrName string) (*container.Container, e
if indexError != nil {
// When truncindex defines an error type, use that instead
if indexError == truncindex.ErrNotExist {
- return nil, derr.ErrorCodeNoSuchContainer.WithArgs(prefixOrName)
+ err := fmt.Errorf("No such container: %s", prefixOrName)
+ return nil, errors.NewRequestNotFoundError(err)
}
return nil, indexError
}
@@ -411,7 +412,7 @@ func (daemon *Daemon) mergeAndVerifyConfig(config *containertypes.Config, img *i
return err
}
}
- if config.Entrypoint.Len() == 0 && config.Cmd.Len() == 0 {
+ if len(config.Entrypoint) == 0 && len(config.Cmd) == 0 {
return fmt.Errorf("No command specified")
}
return nil
@@ -494,13 +495,11 @@ func (daemon *Daemon) generateHostname(id string, config *containertypes.Config)
}
}
-func (daemon *Daemon) getEntrypointAndArgs(configEntrypoint *strslice.StrSlice, configCmd *strslice.StrSlice) (string, []string) {
- cmdSlice := configCmd.Slice()
- if configEntrypoint.Len() != 0 {
- eSlice := configEntrypoint.Slice()
- return eSlice[0], append(eSlice[1:], cmdSlice...)
+func (daemon *Daemon) getEntrypointAndArgs(configEntrypoint strslice.StrSlice, configCmd strslice.StrSlice) (string, []string) {
+ if len(configEntrypoint) != 0 {
+ return configEntrypoint[0], append(configEntrypoint[1:], configCmd...)
}
- return cmdSlice[0], cmdSlice[1:]
+ return configCmd[0], configCmd[1:]
}
func (daemon *Daemon) newContainer(name string, config *containertypes.Config, imgID image.ID) (*container.Container, error) {
@@ -760,7 +759,7 @@ func NewDaemon(config *Config, registryService *registry.Service) (daemon *Daemo
migrationStart := time.Now()
if err := v1.Migrate(config.Root, graphDriver, d.layerStore, d.imageStore, referenceStore, distributionMetadataStore); err != nil {
- return nil, err
+ logrus.Errorf("Graph migration failed: %q. Your old graph data was found to be too inconsistent for upgrading to content-addressable storage. Some of the old data was probably not upgraded. We recommend starting over with a clean storage directory if possible.", err)
}
logrus.Infof("Graph migration to content-addressability took %.2f seconds", time.Since(migrationStart).Seconds())
@@ -1211,7 +1210,7 @@ func (daemon *Daemon) ImageHistory(name string) ([]*types.ImageHistory, error) {
if !h.EmptyLayer {
if len(img.RootFS.DiffIDs) <= layerCounter {
- return nil, errors.New("too many non-empty layers in History section")
+ return nil, fmt.Errorf("too many non-empty layers in History section")
}
rootFS.Append(img.RootFS.DiffIDs[layerCounter])
@@ -1450,7 +1449,7 @@ func setDefaultMtu(config *Config) {
// verifyContainerSettings performs validation of the hostconfig and config
// structures.
-func (daemon *Daemon) verifyContainerSettings(hostConfig *containertypes.HostConfig, config *containertypes.Config) ([]string, error) {
+func (daemon *Daemon) verifyContainerSettings(hostConfig *containertypes.HostConfig, config *containertypes.Config, update bool) ([]string, error) {
// First perform verification of settings common across all platforms.
if config != nil {
@@ -1473,6 +1472,11 @@ func (daemon *Daemon) verifyContainerSettings(hostConfig *containertypes.HostCon
return nil, nil
}
+ logCfg := daemon.getLogConfig(hostConfig.LogConfig)
+ if err := logger.ValidateLogOpts(logCfg.Type, logCfg.Config); err != nil {
+ return nil, err
+ }
+
for port := range hostConfig.PortBindings {
_, portStr := nat.SplitProtoPort(string(port))
if _, err := nat.ParsePort(portStr); err != nil {
@@ -1487,7 +1491,7 @@ func (daemon *Daemon) verifyContainerSettings(hostConfig *containertypes.HostCon
}
// Now do platform-specific verification
- return verifyPlatformContainerSettings(daemon, hostConfig, config)
+ return verifyPlatformContainerSettings(daemon, hostConfig, config, update)
}
// Checks if the client set configurations for more than one network while creating a container
@@ -1499,7 +1503,8 @@ func (daemon *Daemon) verifyNetworkingConfig(nwConfig *networktypes.NetworkingCo
for k := range nwConfig.EndpointsConfig {
l = append(l, k)
}
- return derr.ErrorCodeMultipleNetworkConnect.WithArgs(fmt.Sprintf("%v", l))
+ err := fmt.Errorf("Container cannot be connected to network endpoints: %s", strings.Join(l, ", "))
+ return errors.NewBadRequestError(err)
}
func configureVolumes(config *Config, rootUID, rootGID int) (*store.VolumeStore, error) {
@@ -1598,28 +1603,41 @@ func (daemon *Daemon) initDiscovery(config *Config) error {
// daemon according to those changes.
// This are the settings that Reload changes:
// - Daemon labels.
+// - Cluster discovery (reconfigure and restart).
func (daemon *Daemon) Reload(config *Config) error {
daemon.configStore.reloadLock.Lock()
- daemon.configStore.Labels = config.Labels
- daemon.configStore.reloadLock.Unlock()
-
- return nil
+ defer daemon.configStore.reloadLock.Unlock()
+ if config.IsValueSet("label") {
+ daemon.configStore.Labels = config.Labels
+ }
+ if config.IsValueSet("debug") {
+ daemon.configStore.Debug = config.Debug
+ }
+ return daemon.reloadClusterDiscovery(config)
}
func (daemon *Daemon) reloadClusterDiscovery(config *Config) error {
- newAdvertise, err := parseClusterAdvertiseSettings(config.ClusterStore, config.ClusterAdvertise)
- if err != nil && err != errDiscoveryDisabled {
- return err
+ var err error
+ newAdvertise := daemon.configStore.ClusterAdvertise
+ newClusterStore := daemon.configStore.ClusterStore
+ if config.IsValueSet("cluster-advertise") {
+ if config.IsValueSet("cluster-store") {
+ newClusterStore = config.ClusterStore
+ }
+ newAdvertise, err = parseClusterAdvertiseSettings(newClusterStore, config.ClusterAdvertise)
+ if err != nil && err != errDiscoveryDisabled {
+ return err
+ }
}
// check discovery modifications
- if !modifiedDiscoverySettings(daemon.configStore, newAdvertise, config.ClusterStore, config.ClusterOpts) {
+ if !modifiedDiscoverySettings(daemon.configStore, newAdvertise, newClusterStore, config.ClusterOpts) {
return nil
}
// enable discovery for the first time if it was not previously enabled
if daemon.discoveryWatcher == nil {
- discoveryWatcher, err := initDiscovery(config.ClusterStore, newAdvertise, config.ClusterOpts)
+ discoveryWatcher, err := initDiscovery(newClusterStore, newAdvertise, config.ClusterOpts)
if err != nil {
return fmt.Errorf("discovery initialization failed (%v)", err)
}
@@ -1636,10 +1654,23 @@ func (daemon *Daemon) reloadClusterDiscovery(config *Config) error {
}
}
- daemon.configStore.ClusterStore = config.ClusterStore
+ daemon.configStore.ClusterStore = newClusterStore
daemon.configStore.ClusterOpts = config.ClusterOpts
daemon.configStore.ClusterAdvertise = newAdvertise
+ if daemon.netController == nil {
+ return nil
+ }
+ netOptions, err := daemon.networkOptions(daemon.configStore)
+ if err != nil {
+ logrus.Warnf("Failed to reload configuration with network controller: %v", err)
+ return nil
+ }
+ err = daemon.netController.ReloadConfiguration(netOptions...)
+ if err != nil {
+ logrus.Warnf("Failed to reload configuration with network controller: %v", err)
+ }
+
return nil
}
@@ -1658,7 +1689,7 @@ func convertLnNetworkStats(name string, stats *lntypes.InterfaceStatistics) *lib
func validateID(id string) error {
if id == "" {
- return derr.ErrorCodeEmptyID
+ return fmt.Errorf("Invalid empty id")
}
return nil
}
diff --git a/components/engine/daemon/daemon_test.go b/components/engine/daemon/daemon_test.go
index 4a78bf2875..1c34d3ae4c 100644
--- a/components/engine/daemon/daemon_test.go
+++ b/components/engine/daemon/daemon_test.go
@@ -315,9 +315,12 @@ func TestDaemonReloadLabels(t *testing.T) {
},
}
+ valuesSets := make(map[string]interface{})
+ valuesSets["label"] = "foo:baz"
newConfig := &Config{
CommonConfig: CommonConfig{
- Labels: []string{"foo:baz"},
+ Labels: []string{"foo:baz"},
+ valuesSet: valuesSets,
},
}
@@ -328,6 +331,35 @@ func TestDaemonReloadLabels(t *testing.T) {
}
}
+func TestDaemonReloadNotAffectOthers(t *testing.T) {
+ daemon := &Daemon{}
+ daemon.configStore = &Config{
+ CommonConfig: CommonConfig{
+ Labels: []string{"foo:bar"},
+ Debug: true,
+ },
+ }
+
+ valuesSets := make(map[string]interface{})
+ valuesSets["label"] = "foo:baz"
+ newConfig := &Config{
+ CommonConfig: CommonConfig{
+ Labels: []string{"foo:baz"},
+ valuesSet: valuesSets,
+ },
+ }
+
+ daemon.Reload(newConfig)
+ label := daemon.configStore.Labels[0]
+ if label != "foo:baz" {
+ t.Fatalf("Expected daemon label `foo:baz`, got %s", label)
+ }
+ debug := daemon.configStore.Debug
+ if !debug {
+ t.Fatalf("Expected debug 'enabled', got 'disabled'")
+ }
+}
+
func TestDaemonDiscoveryReload(t *testing.T) {
daemon := &Daemon{}
daemon.configStore = &Config{
@@ -360,10 +392,14 @@ func TestDaemonDiscoveryReload(t *testing.T) {
t.Fatal(e)
}
+ valuesSets := make(map[string]interface{})
+ valuesSets["cluster-store"] = "memory://127.0.0.1:2222"
+ valuesSets["cluster-advertise"] = "127.0.0.1:5555"
newConfig := &Config{
CommonConfig: CommonConfig{
ClusterStore: "memory://127.0.0.1:2222",
ClusterAdvertise: "127.0.0.1:5555",
+ valuesSet: valuesSets,
},
}
@@ -371,7 +407,7 @@ func TestDaemonDiscoveryReload(t *testing.T) {
&discovery.Entry{Host: "127.0.0.1", Port: "5555"},
}
- if err := daemon.reloadClusterDiscovery(newConfig); err != nil {
+ if err := daemon.Reload(newConfig); err != nil {
t.Fatal(err)
}
ch, errCh = daemon.discoveryWatcher.Watch(stopCh)
@@ -392,10 +428,14 @@ func TestDaemonDiscoveryReloadFromEmptyDiscovery(t *testing.T) {
daemon := &Daemon{}
daemon.configStore = &Config{}
+ valuesSet := make(map[string]interface{})
+ valuesSet["cluster-store"] = "memory://127.0.0.1:2222"
+ valuesSet["cluster-advertise"] = "127.0.0.1:5555"
newConfig := &Config{
CommonConfig: CommonConfig{
ClusterStore: "memory://127.0.0.1:2222",
ClusterAdvertise: "127.0.0.1:5555",
+ valuesSet: valuesSet,
},
}
@@ -403,7 +443,7 @@ func TestDaemonDiscoveryReloadFromEmptyDiscovery(t *testing.T) {
&discovery.Entry{Host: "127.0.0.1", Port: "5555"},
}
- if err := daemon.reloadClusterDiscovery(newConfig); err != nil {
+ if err := daemon.Reload(newConfig); err != nil {
t.Fatal(err)
}
stopCh := make(chan struct{})
@@ -421,3 +461,42 @@ func TestDaemonDiscoveryReloadFromEmptyDiscovery(t *testing.T) {
t.Fatal(e)
}
}
+
+func TestDaemonDiscoveryReloadOnlyClusterAdvertise(t *testing.T) {
+ daemon := &Daemon{}
+ daemon.configStore = &Config{
+ CommonConfig: CommonConfig{
+ ClusterStore: "memory://127.0.0.1",
+ },
+ }
+ valuesSets := make(map[string]interface{})
+ valuesSets["cluster-advertise"] = "127.0.0.1:5555"
+ newConfig := &Config{
+ CommonConfig: CommonConfig{
+ ClusterAdvertise: "127.0.0.1:5555",
+ valuesSet: valuesSets,
+ },
+ }
+ expected := discovery.Entries{
+ &discovery.Entry{Host: "127.0.0.1", Port: "5555"},
+ }
+
+ if err := daemon.Reload(newConfig); err != nil {
+ t.Fatal(err)
+ }
+ stopCh := make(chan struct{})
+ defer close(stopCh)
+ ch, errCh := daemon.discoveryWatcher.Watch(stopCh)
+
+ select {
+ case <-time.After(1 * time.Second):
+ t.Fatal("failed to get discovery advertisements in time")
+ case e := <-ch:
+ if !reflect.DeepEqual(e, expected) {
+ t.Fatalf("expected %v, got %v\n", expected, e)
+ }
+ case e := <-errCh:
+ t.Fatal(e)
+ }
+
+}
diff --git a/components/engine/daemon/daemon_unix.go b/components/engine/daemon/daemon_unix.go
index faa6c0ee68..21923fb0b2 100644
--- a/components/engine/daemon/daemon_unix.go
+++ b/components/engine/daemon/daemon_unix.go
@@ -16,7 +16,6 @@ import (
"github.com/Sirupsen/logrus"
"github.com/docker/docker/container"
- derr "github.com/docker/docker/errors"
"github.com/docker/docker/image"
"github.com/docker/docker/layer"
"github.com/docker/docker/pkg/idtools"
@@ -76,17 +75,23 @@ func parseSecurityOpt(container *container.Container, config *containertypes.Hos
for _, opt := range config.SecurityOpt {
con := strings.SplitN(opt, ":", 2)
if len(con) == 1 {
- return fmt.Errorf("Invalid --security-opt: %q", opt)
- }
- switch con[0] {
- case "label":
- labelOpts = append(labelOpts, con[1])
- case "apparmor":
- container.AppArmorProfile = con[1]
- case "seccomp":
- container.SeccompProfile = con[1]
- default:
- return fmt.Errorf("Invalid --security-opt: %q", opt)
+ switch con[0] {
+ case "no-new-privileges":
+ container.NoNewPrivileges = true
+ default:
+ return fmt.Errorf("Invalid --security-opt 1: %q", opt)
+ }
+ } else {
+ switch con[0] {
+ case "label":
+ labelOpts = append(labelOpts, con[1])
+ case "apparmor":
+ container.AppArmorProfile = con[1]
+ case "seccomp":
+ container.SeccompProfile = con[1]
+ default:
+ return fmt.Errorf("Invalid --security-opt 2: %q", opt)
+ }
}
}
@@ -221,7 +226,7 @@ func (daemon *Daemon) adaptContainerSettings(hostConfig *containertypes.HostConf
return nil
}
-func verifyContainerResources(resources *containertypes.Resources, sysInfo *sysinfo.SysInfo) ([]string, error) {
+func verifyContainerResources(resources *containertypes.Resources, sysInfo *sysinfo.SysInfo, update bool) ([]string, error) {
warnings := []string{}
// memory subsystem checks and adjustments
@@ -242,7 +247,7 @@ func verifyContainerResources(resources *containertypes.Resources, sysInfo *sysi
if resources.Memory > 0 && resources.MemorySwap > 0 && resources.MemorySwap < resources.Memory {
return warnings, fmt.Errorf("Minimum memoryswap limit should be larger than memory limit, see usage.")
}
- if resources.Memory == 0 && resources.MemorySwap > 0 {
+ if resources.Memory == 0 && resources.MemorySwap > 0 && !update {
return warnings, fmt.Errorf("You should always set the Memory limit when using Memoryswap limit, see usage.")
}
if resources.MemorySwappiness != nil && *resources.MemorySwappiness != -1 && !sysInfo.MemorySwappiness {
@@ -286,6 +291,12 @@ func verifyContainerResources(resources *containertypes.Resources, sysInfo *sysi
resources.OomKillDisable = nil
}
+ if resources.PidsLimit != 0 && !sysInfo.PidsLimit {
+ warnings = append(warnings, "Your kernel does not support pids limit capabilities, pids limit discarded.")
+ logrus.Warnf("Your kernel does not support pids limit capabilities, pids limit discarded.")
+ resources.PidsLimit = 0
+ }
+
// cpu subsystem checks and adjustments
if resources.CPUShares > 0 && !sysInfo.CPUShares {
warnings = append(warnings, "Your kernel does not support CPU shares. Shares discarded.")
@@ -312,17 +323,17 @@ func verifyContainerResources(resources *containertypes.Resources, sysInfo *sysi
}
cpusAvailable, err := sysInfo.IsCpusetCpusAvailable(resources.CpusetCpus)
if err != nil {
- return warnings, derr.ErrorCodeInvalidCpusetCpus.WithArgs(resources.CpusetCpus)
+ return warnings, fmt.Errorf("Invalid value %s for cpuset cpus.", resources.CpusetCpus)
}
if !cpusAvailable {
- return warnings, derr.ErrorCodeNotAvailableCpusetCpus.WithArgs(resources.CpusetCpus, sysInfo.Cpus)
+ return warnings, fmt.Errorf("Requested CPUs are not available - requested %s, available: %s.", resources.CpusetCpus, sysInfo.Cpus)
}
memsAvailable, err := sysInfo.IsCpusetMemsAvailable(resources.CpusetMems)
if err != nil {
- return warnings, derr.ErrorCodeInvalidCpusetMems.WithArgs(resources.CpusetMems)
+ return warnings, fmt.Errorf("Invalid value %s for cpuset mems.", resources.CpusetMems)
}
if !memsAvailable {
- return warnings, derr.ErrorCodeNotAvailableCpusetMems.WithArgs(resources.CpusetMems, sysInfo.Mems)
+ return warnings, fmt.Errorf("Requested memory nodes are not available - requested %s, available: %s.", resources.CpusetMems, sysInfo.Mems)
}
// blkio subsystem checks and adjustments
@@ -363,6 +374,14 @@ func verifyContainerResources(resources *containertypes.Resources, sysInfo *sysi
return warnings, nil
}
+func (daemon *Daemon) getCgroupDriver() string {
+ cgroupDriver := "cgroupfs"
+ if daemon.usingSystemd() {
+ cgroupDriver = "systemd"
+ }
+ return cgroupDriver
+}
+
func usingSystemd(config *Config) bool {
for _, option := range config.ExecOptions {
key, val, err := parsers.ParseKeyValueOpt(option)
@@ -383,7 +402,7 @@ func (daemon *Daemon) usingSystemd() bool {
// verifyPlatformContainerSettings performs platform-specific validation of the
// hostconfig and config structures.
-func verifyPlatformContainerSettings(daemon *Daemon, hostConfig *containertypes.HostConfig, config *containertypes.Config) ([]string, error) {
+func verifyPlatformContainerSettings(daemon *Daemon, hostConfig *containertypes.HostConfig, config *containertypes.Config, update bool) ([]string, error) {
warnings := []string{}
sysInfo := sysinfo.New(true)
@@ -392,7 +411,7 @@ func verifyPlatformContainerSettings(daemon *Daemon, hostConfig *containertypes.
return warnings, err
}
- w, err := verifyContainerResources(&hostConfig.Resources, sysInfo)
+ w, err := verifyContainerResources(&hostConfig.Resources, sysInfo, update)
if err != nil {
return warnings, err
}
@@ -694,10 +713,8 @@ func initBridgeDriver(controller libnetwork.NetworkController, config *Config) e
}
// Initialize default network on "bridge" with the same name
_, err = controller.NewNetwork("bridge", "bridge",
- libnetwork.NetworkOptionGeneric(options.Generic{
- netlabel.GenericData: netOption,
- netlabel.EnableIPv6: config.bridgeConfig.EnableIPv6,
- }),
+ libnetwork.NetworkOptionEnableIPv6(config.bridgeConfig.EnableIPv6),
+ libnetwork.NetworkOptionDriverOpts(netOption),
libnetwork.NetworkOptionIpam("default", "", v4Conf, v6Conf, nil),
libnetwork.NetworkOptionDeferIPv6Alloc(deferIPv6Alloc))
if err != nil {
diff --git a/components/engine/daemon/daemon_unix_test.go b/components/engine/daemon/daemon_unix_test.go
index 62f870fbd8..26eb93fcd3 100644
--- a/components/engine/daemon/daemon_unix_test.go
+++ b/components/engine/daemon/daemon_unix_test.go
@@ -142,7 +142,7 @@ func TestNetworkOptions(t *testing.T) {
}
if _, err := daemon.networkOptions(dconfigCorrect); err != nil {
- t.Fatalf("Expect networkOptions sucess, got error: %v", err)
+ t.Fatalf("Expect networkOptions success, got error: %v", err)
}
dconfigWrong := &Config{
diff --git a/components/engine/daemon/daemon_windows.go b/components/engine/daemon/daemon_windows.go
index 2491caf28d..b9aadc6b00 100644
--- a/components/engine/daemon/daemon_windows.go
+++ b/components/engine/daemon/daemon_windows.go
@@ -22,6 +22,7 @@ import (
"github.com/docker/docker/pkg/idtools"
"github.com/docker/docker/pkg/system"
"github.com/docker/libnetwork"
+ nwconfig "github.com/docker/libnetwork/config"
blkiodev "github.com/opencontainers/runc/libcontainer/configs"
)
@@ -64,6 +65,10 @@ func checkKernel() error {
return nil
}
+func (daemon *Daemon) getCgroupDriver() string {
+ return ""
+}
+
// adaptContainerSettings is called during container creation to modify any
// settings necessary in the HostConfig structure.
func (daemon *Daemon) adaptContainerSettings(hostConfig *containertypes.HostConfig, adjustCPUShares bool) error {
@@ -84,7 +89,7 @@ func (daemon *Daemon) adaptContainerSettings(hostConfig *containertypes.HostConf
// verifyPlatformContainerSettings performs platform-specific validation of the
// hostconfig and config structures.
-func verifyPlatformContainerSettings(daemon *Daemon, hostConfig *containertypes.HostConfig, config *containertypes.Config) ([]string, error) {
+func verifyPlatformContainerSettings(daemon *Daemon, hostConfig *containertypes.HostConfig, config *containertypes.Config, update bool) ([]string, error) {
return nil, nil
}
@@ -251,3 +256,7 @@ func restoreCustomImage(is image.Store, ls layer.Store, rs reference.Store) erro
}
return nil
}
+
+func (daemon *Daemon) networkOptions(dconfig *Config) ([]nwconfig.Option, error) {
+ return nil, fmt.Errorf("Network controller config reload not aavailable on Windows yet")
+}
diff --git a/components/engine/daemon/delete.go b/components/engine/daemon/delete.go
index 6c56fd7ba1..75af4c01a1 100644
--- a/components/engine/daemon/delete.go
+++ b/components/engine/daemon/delete.go
@@ -8,7 +8,7 @@ import (
"github.com/Sirupsen/logrus"
"github.com/docker/docker/container"
- derr "github.com/docker/docker/errors"
+ "github.com/docker/docker/errors"
"github.com/docker/docker/layer"
volumestore "github.com/docker/docker/volume/store"
"github.com/docker/engine-api/types"
@@ -25,12 +25,8 @@ func (daemon *Daemon) ContainerRm(name string, config *types.ContainerRmConfig)
}
// Container state RemovalInProgress should be used to avoid races.
- if err = container.SetRemovalInProgress(); err != nil {
- if err == derr.ErrorCodeAlreadyRemoving {
- // do not fail when the removal is in progress started by other request.
- return nil
- }
- return derr.ErrorCodeRmState.WithArgs(container.ID, err)
+ if inProgress := container.SetRemovalInProgress(); inProgress {
+ return nil
}
defer container.ResetRemovalInProgress()
@@ -84,10 +80,11 @@ func (daemon *Daemon) rmLink(container *container.Container, name string) error
func (daemon *Daemon) cleanupContainer(container *container.Container, forceRemove bool) (err error) {
if container.IsRunning() {
if !forceRemove {
- return derr.ErrorCodeRmRunning.WithArgs(container.ID)
+ err := fmt.Errorf("You cannot remove a running container %s. Stop the container before attempting removal or use -f", container.ID)
+ return errors.NewRequestConflictError(err)
}
if err := daemon.Kill(container); err != nil {
- return derr.ErrorCodeRmFailed.WithArgs(container.ID, err)
+ return fmt.Errorf("Could not kill running container %s, cannot remove - %v", container.ID, err)
}
}
@@ -123,17 +120,17 @@ func (daemon *Daemon) cleanupContainer(container *container.Container, forceRemo
}()
if err = os.RemoveAll(container.Root); err != nil {
- return derr.ErrorCodeRmFS.WithArgs(container.ID, err)
+ return fmt.Errorf("Unable to remove filesystem for %v: %v", container.ID, err)
}
metadata, err := daemon.layerStore.ReleaseRWLayer(container.RWLayer)
layer.LogReleaseMetadata(metadata)
if err != nil && err != layer.ErrMountDoesNotExist {
- return derr.ErrorCodeRmDriverFS.WithArgs(daemon.GraphDriverName(), container.ID, err)
+ return fmt.Errorf("Driver %s failed to remove root filesystem %s: %s", daemon.GraphDriverName(), container.ID, err)
}
if err = daemon.execDriver.Clean(container.ID); err != nil {
- return derr.ErrorCodeRmExecDriver.WithArgs(container.ID, err)
+ return fmt.Errorf("Unable to remove execdriver data for %s: %s", container.ID, err)
}
return nil
}
@@ -149,9 +146,10 @@ func (daemon *Daemon) VolumeRm(name string) error {
if err := daemon.volumes.Remove(v); err != nil {
if volumestore.IsInUse(err) {
- return derr.ErrorCodeRmVolumeInUse.WithArgs(err)
+ err := fmt.Errorf("Unable to remove volume, volume still in use: %v", err)
+ return errors.NewRequestConflictError(err)
}
- return derr.ErrorCodeRmVolume.WithArgs(name, err)
+ return fmt.Errorf("Error while removing volume %s: %v", name, err)
}
daemon.LogVolumeEvent(v.Name(), "destroy", map[string]string{"driver": v.DriverName()})
return nil
diff --git a/components/engine/daemon/delete_test.go b/components/engine/daemon/delete_test.go
index 0d39b4d68f..adce2eb8c5 100644
--- a/components/engine/daemon/delete_test.go
+++ b/components/engine/daemon/delete_test.go
@@ -32,9 +32,7 @@ func TestContainerDoubleDelete(t *testing.T) {
daemon.containers.Add(container.ID, container)
// Mark the container as having a delete in progress
- if err := container.SetRemovalInProgress(); err != nil {
- t.Fatal(err)
- }
+ container.SetRemovalInProgress()
// Try to remove the container when it's start is removalInProgress.
// It should ignore the container and not return an error.
diff --git a/components/engine/daemon/errors.go b/components/engine/daemon/errors.go
index 8fff66b9ff..131c9a1e22 100644
--- a/components/engine/daemon/errors.go
+++ b/components/engine/daemon/errors.go
@@ -1,26 +1,57 @@
package daemon
import (
+ "fmt"
"strings"
- derr "github.com/docker/docker/errors"
+ "github.com/docker/docker/errors"
"github.com/docker/docker/reference"
)
func (d *Daemon) imageNotExistToErrcode(err error) error {
if dne, isDNE := err.(ErrImageDoesNotExist); isDNE {
if strings.Contains(dne.RefOrID, "@") {
- return derr.ErrorCodeNoSuchImageHash.WithArgs(dne.RefOrID)
+ e := fmt.Errorf("No such image: %s", dne.RefOrID)
+ return errors.NewRequestNotFoundError(e)
}
tag := reference.DefaultTag
ref, err := reference.ParseNamed(dne.RefOrID)
if err != nil {
- return derr.ErrorCodeNoSuchImageTag.WithArgs(dne.RefOrID, tag)
+ e := fmt.Errorf("No such image: %s:%s", dne.RefOrID, tag)
+ return errors.NewRequestNotFoundError(e)
}
if tagged, isTagged := ref.(reference.NamedTagged); isTagged {
tag = tagged.Tag()
}
- return derr.ErrorCodeNoSuchImageTag.WithArgs(ref.Name(), tag)
+ e := fmt.Errorf("No such image: %s:%s", ref.Name(), tag)
+ return errors.NewRequestNotFoundError(e)
}
return err
}
+
+type errNotRunning struct {
+ containerID string
+}
+
+func (e errNotRunning) Error() string {
+ return fmt.Sprintf("Container %s is not running", e.containerID)
+}
+
+func (e errNotRunning) ContainerIsRunning() bool {
+ return false
+}
+
+func errContainerIsRestarting(containerID string) error {
+ err := fmt.Errorf("Container %s is restarting, wait until the container is running", containerID)
+ return errors.NewRequestConflictError(err)
+}
+
+func errExecNotFound(id string) error {
+ err := fmt.Errorf("No such exec instance '%s' found in daemon", id)
+ return errors.NewRequestNotFoundError(err)
+}
+
+func errExecPaused(id string) error {
+ err := fmt.Errorf("Container %s is paused, unpause the container before exec", id)
+ return errors.NewRequestConflictError(err)
+}
diff --git a/components/engine/daemon/events/events.go b/components/engine/daemon/events/events.go
index a8cd66fd99..ac1c98cd46 100644
--- a/components/engine/daemon/events/events.go
+++ b/components/engine/daemon/events/events.go
@@ -50,33 +50,23 @@ func (e *Events) Subscribe() ([]eventtypes.Message, chan interface{}, func()) {
// of interface{}, so you need type assertion).
func (e *Events) SubscribeTopic(since, sinceNano int64, ef *Filter) ([]eventtypes.Message, chan interface{}) {
e.mu.Lock()
- defer e.mu.Unlock()
- var buffered []eventtypes.Message
- topic := func(m interface{}) bool {
- return ef.Include(m.(eventtypes.Message))
+ var topic func(m interface{}) bool
+ if ef != nil && ef.filter.Len() > 0 {
+ topic = func(m interface{}) bool { return ef.Include(m.(eventtypes.Message)) }
}
- if since != -1 {
- for i := len(e.events) - 1; i >= 0; i-- {
- ev := e.events[i]
- if ev.Time < since || ((ev.Time == since) && (ev.TimeNano < sinceNano)) {
- break
- }
- if ef.filter.Len() == 0 || topic(ev) {
- buffered = append([]eventtypes.Message{ev}, buffered...)
- }
- }
- }
+ buffered := e.loadBufferedEvents(since, sinceNano, topic)
var ch chan interface{}
- if ef.filter.Len() > 0 {
+ if topic != nil {
ch = e.pub.SubscribeTopic(topic)
} else {
// Subscribe to all events if there are no filters
ch = e.pub.Subscribe()
}
+ e.mu.Unlock()
return buffered, ch
}
@@ -124,3 +114,29 @@ func (e *Events) Log(action, eventType string, actor eventtypes.Actor) {
func (e *Events) SubscribersCount() int {
return e.pub.Len()
}
+
+// loadBufferedEvents iterates over the cached events in the buffer
+// and returns those that were emitted before a specific date.
+// The date is splitted in two values:
+// - the `since` argument is a date timestamp without nanoseconds, or -1 to return an empty slice.
+// - the `sinceNano` argument is the nanoseconds offset from the timestamp.
+// It uses `time.Unix(seconds, nanoseconds)` to generate a valid date with those two first arguments.
+// It filters those buffered messages with a topic function if it's not nil, otherwise it adds all messages.
+func (e *Events) loadBufferedEvents(since, sinceNano int64, topic func(interface{}) bool) []eventtypes.Message {
+ var buffered []eventtypes.Message
+ if since == -1 {
+ return buffered
+ }
+
+ sinceNanoUnix := time.Unix(since, sinceNano).UnixNano()
+ for i := len(e.events) - 1; i >= 0; i-- {
+ ev := e.events[i]
+ if ev.TimeNano < sinceNanoUnix {
+ break
+ }
+ if topic == nil || topic(ev) {
+ buffered = append([]eventtypes.Message{ev}, buffered...)
+ }
+ }
+ return buffered
+}
diff --git a/components/engine/daemon/events/events_test.go b/components/engine/daemon/events/events_test.go
index fc3b84bb85..5fd577b992 100644
--- a/components/engine/daemon/events/events_test.go
+++ b/components/engine/daemon/events/events_test.go
@@ -5,7 +5,9 @@ import (
"testing"
"time"
+ "github.com/docker/docker/daemon/events/testutils"
"github.com/docker/engine-api/types/events"
+ timetypes "github.com/docker/engine-api/types/time"
)
func TestEventsLog(t *testing.T) {
@@ -150,3 +152,45 @@ func TestLogEvents(t *testing.T) {
t.Fatalf("Last action is %s, must be action_89", lastC.Status)
}
}
+
+// https://github.com/docker/docker/issues/20999
+// Fixtures:
+//
+//2016-03-07T17:28:03.022433271+02:00 container die 0b863f2a26c18557fc6cdadda007c459f9ec81b874780808138aea78a3595079 (image=ubuntu, name=small_hoover)
+//2016-03-07T17:28:03.091719377+02:00 network disconnect 19c5ed41acb798f26b751e0035cd7821741ab79e2bbd59a66b5fd8abf954eaa0 (type=bridge, container=0b863f2a26c18557fc6cdadda007c459f9ec81b874780808138aea78a3595079, name=bridge)
+//2016-03-07T17:28:03.129014751+02:00 container destroy 0b863f2a26c18557fc6cdadda007c459f9ec81b874780808138aea78a3595079 (image=ubuntu, name=small_hoover)
+func TestLoadBufferedEvents(t *testing.T) {
+ now := time.Now()
+ f, err := timetypes.GetTimestamp("2016-03-07T17:28:03.100000000+02:00", now)
+ if err != nil {
+ t.Fatal(err)
+ }
+ since, sinceNano, err := timetypes.ParseTimestamps(f, -1)
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ m1, err := eventstestutils.Scan("2016-03-07T17:28:03.022433271+02:00 container die 0b863f2a26c18557fc6cdadda007c459f9ec81b874780808138aea78a3595079 (image=ubuntu, name=small_hoover)")
+ if err != nil {
+ t.Fatal(err)
+ }
+ m2, err := eventstestutils.Scan("2016-03-07T17:28:03.091719377+02:00 network disconnect 19c5ed41acb798f26b751e0035cd7821741ab79e2bbd59a66b5fd8abf954eaa0 (type=bridge, container=0b863f2a26c18557fc6cdadda007c459f9ec81b874780808138aea78a3595079, name=bridge)")
+ if err != nil {
+ t.Fatal(err)
+ }
+ m3, err := eventstestutils.Scan("2016-03-07T17:28:03.129014751+02:00 container destroy 0b863f2a26c18557fc6cdadda007c459f9ec81b874780808138aea78a3595079 (image=ubuntu, name=small_hoover)")
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ buffered := []events.Message{*m1, *m2, *m3}
+
+ events := &Events{
+ events: buffered,
+ }
+
+ out := events.loadBufferedEvents(since, sinceNano, nil)
+ if len(out) != 1 {
+ t.Fatalf("expected 1 message, got %d: %v", len(out), out)
+ }
+}
diff --git a/components/engine/daemon/events/testutils/testutils.go b/components/engine/daemon/events/testutils/testutils.go
new file mode 100644
index 0000000000..c84418a9e7
--- /dev/null
+++ b/components/engine/daemon/events/testutils/testutils.go
@@ -0,0 +1,76 @@
+package eventstestutils
+
+import (
+ "fmt"
+ "regexp"
+ "strings"
+ "time"
+
+ "github.com/docker/engine-api/types/events"
+ timetypes "github.com/docker/engine-api/types/time"
+)
+
+var (
+ reTimestamp = `(?P\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}.\d{9}(:?(:?(:?-|\+)\d{2}:\d{2})|Z))`
+ reEventType = `(?P\w+)`
+ reAction = `(?P\w+)`
+ reID = `(?P[^\s]+)`
+ reAttributes = `(\s\((?P[^\)]+)\))?`
+ reString = fmt.Sprintf(`\A%s\s%s\s%s\s%s%s\z`, reTimestamp, reEventType, reAction, reID, reAttributes)
+
+ // eventCliRegexp is a regular expression that matches all possible event outputs in the cli
+ eventCliRegexp = regexp.MustCompile(reString)
+)
+
+// ScanMap turns an event string like the default ones formatted in the cli output
+// and turns it into map.
+func ScanMap(text string) map[string]string {
+ matches := eventCliRegexp.FindAllStringSubmatch(text, -1)
+ md := map[string]string{}
+ if len(matches) == 0 {
+ return md
+ }
+
+ names := eventCliRegexp.SubexpNames()
+ for i, n := range matches[0] {
+ md[names[i]] = n
+ }
+ return md
+}
+
+// Scan turns an event string like the default ones formatted in the cli output
+// and turns it into an event message.
+func Scan(text string) (*events.Message, error) {
+ md := ScanMap(text)
+ if len(md) == 0 {
+ return nil, fmt.Errorf("text is not an event: %s", text)
+ }
+
+ f, err := timetypes.GetTimestamp(md["timestamp"], time.Now())
+ if err != nil {
+ return nil, err
+ }
+
+ t, tn, err := timetypes.ParseTimestamps(f, -1)
+ if err != nil {
+ return nil, err
+ }
+
+ attrs := make(map[string]string)
+ for _, a := range strings.SplitN(md["attributes"], ", ", -1) {
+ kv := strings.SplitN(a, "=", 2)
+ attrs[kv[0]] = kv[1]
+ }
+
+ tu := time.Unix(t, tn)
+ return &events.Message{
+ Time: t,
+ TimeNano: tu.UnixNano(),
+ Type: md["eventType"],
+ Action: md["action"],
+ Actor: events.Actor{
+ ID: md["id"],
+ Attributes: attrs,
+ },
+ }, nil
+}
diff --git a/components/engine/daemon/exec.go b/components/engine/daemon/exec.go
index 1151a77c7b..56798a5979 100644
--- a/components/engine/daemon/exec.go
+++ b/components/engine/daemon/exec.go
@@ -1,6 +1,7 @@
package daemon
import (
+ "fmt"
"io"
"strings"
"time"
@@ -9,7 +10,7 @@ import (
"github.com/docker/docker/container"
"github.com/docker/docker/daemon/exec"
"github.com/docker/docker/daemon/execdriver"
- derr "github.com/docker/docker/errors"
+ "github.com/docker/docker/errors"
"github.com/docker/docker/pkg/pools"
"github.com/docker/docker/pkg/promise"
"github.com/docker/docker/pkg/term"
@@ -47,19 +48,19 @@ func (d *Daemon) getExecConfig(name string) (*exec.Config, error) {
if ec != nil {
if container := d.containers.Get(ec.ContainerID); container != nil {
if !container.IsRunning() {
- return nil, derr.ErrorCodeContainerNotRunning.WithArgs(container.ID, container.State.String())
+ return nil, fmt.Errorf("Container %s is not running: %s", container.ID, container.State.String())
}
if container.IsPaused() {
- return nil, derr.ErrorCodeExecPaused.WithArgs(container.ID)
+ return nil, errExecPaused(container.ID)
}
if container.IsRestarting() {
- return nil, derr.ErrorCodeContainerRestarting.WithArgs(container.ID)
+ return nil, errContainerIsRestarting(container.ID)
}
return ec, nil
}
}
- return nil, derr.ErrorCodeNoExecID.WithArgs(name)
+ return nil, errExecNotFound(name)
}
func (d *Daemon) unregisterExecCommand(container *container.Container, execConfig *exec.Config) {
@@ -74,13 +75,13 @@ func (d *Daemon) getActiveContainer(name string) (*container.Container, error) {
}
if !container.IsRunning() {
- return nil, derr.ErrorCodeNotRunning.WithArgs(name)
+ return nil, errNotRunning{container.ID}
}
if container.IsPaused() {
- return nil, derr.ErrorCodeExecPaused.WithArgs(name)
+ return nil, errExecPaused(name)
}
if container.IsRestarting() {
- return nil, derr.ErrorCodeContainerRestarting.WithArgs(name)
+ return nil, errContainerIsRestarting(container.ID)
}
return container, nil
}
@@ -92,8 +93,8 @@ func (d *Daemon) ContainerExecCreate(config *types.ExecConfig) (string, error) {
return "", err
}
- cmd := strslice.New(config.Cmd...)
- entrypoint, args := d.getEntrypointAndArgs(strslice.New(), cmd)
+ cmd := strslice.StrSlice(config.Cmd)
+ entrypoint, args := d.getEntrypointAndArgs(strslice.StrSlice{}, cmd)
keys := []byte{}
if config.DetachKeys != "" {
@@ -137,18 +138,19 @@ func (d *Daemon) ContainerExecStart(name string, stdin io.ReadCloser, stdout io.
ec, err := d.getExecConfig(name)
if err != nil {
- return derr.ErrorCodeNoExecID.WithArgs(name)
+ return errExecNotFound(name)
}
ec.Lock()
if ec.ExitCode != nil {
ec.Unlock()
- return derr.ErrorCodeExecExited.WithArgs(ec.ID)
+ err := fmt.Errorf("Error: Exec command %s has already run", ec.ID)
+ return errors.NewRequestConflictError(err)
}
if ec.Running {
ec.Unlock()
- return derr.ErrorCodeExecRunning.WithArgs(ec.ID)
+ return fmt.Errorf("Error: Exec command %s is already running", ec.ID)
}
ec.Running = true
ec.Unlock()
@@ -157,7 +159,7 @@ func (d *Daemon) ContainerExecStart(name string, stdin io.ReadCloser, stdout io.
logrus.Debugf("starting exec command %s in container %s", ec.ID, c.ID)
d.LogContainerEvent(c, "exec_start: "+ec.ProcessConfig.Entrypoint+" "+strings.Join(ec.ProcessConfig.Arguments, " "))
- if ec.OpenStdin {
+ if ec.OpenStdin && stdin != nil {
r, w := io.Pipe()
go func() {
defer w.Close()
@@ -194,12 +196,12 @@ func (d *Daemon) ContainerExecStart(name string, stdin io.ReadCloser, stdout io.
select {
case err := <-attachErr:
if err != nil {
- return derr.ErrorCodeExecAttach.WithArgs(err)
+ return fmt.Errorf("attach failed with error: %v", err)
}
return nil
case err := <-execErr:
if aErr := <-attachErr; aErr != nil && err == nil {
- return derr.ErrorCodeExecAttach.WithArgs(aErr)
+ return fmt.Errorf("attach failed with error: %v", aErr)
}
if err == nil {
return nil
@@ -207,9 +209,9 @@ func (d *Daemon) ContainerExecStart(name string, stdin io.ReadCloser, stdout io.
// Maybe the container stopped while we were trying to exec
if !c.IsRunning() {
- return derr.ErrorCodeExecContainerStopped
+ return fmt.Errorf("container stopped while running exec: %s", c.ID)
}
- return derr.ErrorCodeExecCantRun.WithArgs(ec.ID, c.ID, err)
+ return fmt.Errorf("Cannot run exec command %s in container %s: %s", ec.ID, c.ID, err)
}
}
diff --git a/components/engine/daemon/exec/exec.go b/components/engine/daemon/exec/exec.go
index 6941cde689..2efb20ee9a 100644
--- a/components/engine/daemon/exec/exec.go
+++ b/components/engine/daemon/exec/exec.go
@@ -1,11 +1,11 @@
package exec
import (
+ "fmt"
"sync"
"time"
"github.com/docker/docker/daemon/execdriver"
- derr "github.com/docker/docker/errors"
"github.com/docker/docker/pkg/stringid"
"github.com/docker/docker/runconfig"
)
@@ -116,7 +116,7 @@ func (c *Config) Resize(h, w int) error {
select {
case <-c.waitStart:
case <-time.After(time.Second):
- return derr.ErrorCodeExecResize.WithArgs(c.ID)
+ return fmt.Errorf("Exec %s is not running, so it can not be resized.", c.ID)
}
return c.ProcessConfig.Terminal.Resize(h, w)
}
diff --git a/components/engine/daemon/execdriver/driver_unix.go b/components/engine/daemon/execdriver/driver_unix.go
index 19550b3419..988737df19 100644
--- a/components/engine/daemon/execdriver/driver_unix.go
+++ b/components/engine/daemon/execdriver/driver_unix.go
@@ -50,6 +50,7 @@ type Resources struct {
CPUPeriod int64 `json:"cpu_period"`
Rlimits []*units.Rlimit `json:"rlimits"`
OomKillDisable bool `json:"oom_kill_disable"`
+ PidsLimit int64 `json:"pids_limit"`
MemorySwappiness int64 `json:"memory_swappiness"`
}
@@ -124,6 +125,7 @@ type Command struct {
SeccompProfile string `json:"seccomp_profile"`
UIDMapping []idtools.IDMap `json:"uidmapping"`
UTS *UTS `json:"uts"`
+ NoNewPrivileges bool `json:"no_new_privileges"`
}
// SetRootPropagation sets the root mount propagation mode.
@@ -140,7 +142,7 @@ func InitContainer(c *Command) *configs.Config {
container.Hostname = getEnv("HOSTNAME", c.ProcessConfig.Env)
container.Cgroups.Name = c.ID
container.Cgroups.Resources.AllowedDevices = c.AllowedDevices
- container.Devices = c.AutoCreatedDevices
+ container.Devices = filterDevices(c.AutoCreatedDevices, (c.RemappedRoot.UID != 0))
container.Rootfs = c.Rootfs
container.Readonlyfs = c.ReadonlyRootfs
// This can be overridden later by driver during mount setup based
@@ -154,6 +156,24 @@ func InitContainer(c *Command) *configs.Config {
return container
}
+func filterDevices(devices []*configs.Device, userNamespacesEnabled bool) []*configs.Device {
+ if !userNamespacesEnabled {
+ return devices
+ }
+
+ filtered := []*configs.Device{}
+ // if we have user namespaces enabled, these devices will not be created
+ // because of the mknod limitation in the kernel for an unprivileged process.
+ // Rather, they will be bind-mounted, which will only work if they exist;
+ // check for existence and remove non-existent entries from the list
+ for _, device := range devices {
+ if _, err := os.Stat(device.Path); err == nil {
+ filtered = append(filtered, device)
+ }
+ }
+ return filtered
+}
+
func getEnv(key string, env []string) string {
for _, pair := range env {
parts := strings.SplitN(pair, "=", 2)
@@ -183,6 +203,7 @@ func SetupCgroups(container *configs.Config, c *Command) error {
container.Cgroups.Resources.BlkioThrottleReadIOPSDevice = c.Resources.BlkioThrottleReadIOpsDevice
container.Cgroups.Resources.BlkioThrottleWriteIOPSDevice = c.Resources.BlkioThrottleWriteIOpsDevice
container.Cgroups.Resources.OomKillDisable = c.Resources.OomKillDisable
+ container.Cgroups.Resources.PidsLimit = c.Resources.PidsLimit
container.Cgroups.Resources.MemorySwappiness = c.Resources.MemorySwappiness
}
diff --git a/components/engine/daemon/execdriver/driver_windows.go b/components/engine/daemon/execdriver/driver_windows.go
index ec482cd30f..2fdb533729 100644
--- a/components/engine/daemon/execdriver/driver_windows.go
+++ b/components/engine/daemon/execdriver/driver_windows.go
@@ -49,11 +49,11 @@ type Command struct {
// Fields below here are platform specific
- FirstStart bool `json:"first_start"` // Optimisation for first boot of Windows
+ FirstStart bool `json:"first_start"` // Optimization for first boot of Windows
Hostname string `json:"hostname"` // Windows sets the hostname in the execdriver
LayerFolder string `json:"layer_folder"` // Layer folder for a command
LayerPaths []string `json:"layer_paths"` // Layer paths for a command
- Isolation string `json:"isolation"` // Isolation level for the container
+ Isolation string `json:"isolation"` // Isolation technology for the container
ArgsEscaped bool `json:"args_escaped"` // True if args are already escaped
HvPartition bool `json:"hv_partition"` // True if it's an hypervisor partition
}
diff --git a/components/engine/daemon/execdriver/native/create.go b/components/engine/daemon/execdriver/native/create.go
index 14df913638..d85898f673 100644
--- a/components/engine/daemon/execdriver/native/create.go
+++ b/components/engine/daemon/execdriver/native/create.go
@@ -9,7 +9,6 @@ import (
"syscall"
"github.com/docker/docker/daemon/execdriver"
- derr "github.com/docker/docker/errors"
"github.com/docker/docker/pkg/mount"
"github.com/docker/docker/profiles/seccomp"
@@ -72,7 +71,10 @@ func (d *Driver) createContainer(c *execdriver.Command, hooks execdriver.Hooks)
}
if c.SeccompProfile == "" {
- container.Seccomp = seccomp.GetDefaultProfile()
+ container.Seccomp, err = seccomp.GetDefaultProfile()
+ if err != nil {
+ return nil, err
+ }
}
}
// add CAP_ prefix to all caps for new libcontainer update to match
@@ -104,7 +106,7 @@ func (d *Driver) createContainer(c *execdriver.Command, hooks execdriver.Hooks)
if container.Readonlyfs {
for i := range container.Mounts {
switch container.Mounts[i].Destination {
- case "/proc", "/dev", "/dev/pts":
+ case "/proc", "/dev", "/dev/pts", "/dev/mqueue":
continue
}
container.Mounts[i].Flags |= syscall.MS_RDONLY
@@ -120,6 +122,8 @@ func (d *Driver) createContainer(c *execdriver.Command, hooks execdriver.Hooks)
d.setupLabels(container, c)
d.setupRlimits(container, c)
+
+ container.NoNewPrivileges = c.NoNewPrivileges
return container, nil
}
@@ -427,7 +431,7 @@ func (d *Driver) setupMounts(container *configs.Config, c *execdriver.Command) e
for _, m := range c.Mounts {
for _, cm := range container.Mounts {
if cm.Destination == m.Destination {
- return derr.ErrorCodeMountDup.WithArgs(m.Destination)
+ return fmt.Errorf("Duplicate mount point '%s'", m.Destination)
}
}
diff --git a/components/engine/daemon/execdriver/native/driver.go b/components/engine/daemon/execdriver/native/driver.go
index 93ad481d17..98d64a32cf 100644
--- a/components/engine/daemon/execdriver/native/driver.go
+++ b/components/engine/daemon/execdriver/native/driver.go
@@ -123,11 +123,6 @@ func NewDriver(root string, options []string) (*Driver, error) {
}, nil
}
-type execOutput struct {
- exitCode int
- err error
-}
-
// Run implements the exec driver Driver interface,
// it calls libcontainer APIs to run a container.
func (d *Driver) Run(c *execdriver.Command, pipes *execdriver.Pipes, hooks execdriver.Hooks) (execdriver.ExitStatus, error) {
@@ -152,7 +147,9 @@ func (d *Driver) Run(c *execdriver.Command, pipes *execdriver.Pipes, hooks execd
User: c.ProcessConfig.User,
}
- if err := setupPipes(container, &c.ProcessConfig, p, pipes); err != nil {
+ wg := sync.WaitGroup{}
+ writers, err := setupPipes(container, &c.ProcessConfig, p, pipes, &wg)
+ if err != nil {
return execdriver.ExitStatus{ExitCode: -1}, err
}
@@ -160,6 +157,10 @@ func (d *Driver) Run(c *execdriver.Command, pipes *execdriver.Pipes, hooks execd
if err != nil {
return execdriver.ExitStatus{ExitCode: -1}, err
}
+
+ if err := cont.Start(p); err != nil {
+ return execdriver.ExitStatus{ExitCode: -1}, err
+ }
d.Lock()
d.activeContainers[c.ID] = cont
d.Unlock()
@@ -170,10 +171,10 @@ func (d *Driver) Run(c *execdriver.Command, pipes *execdriver.Pipes, hooks execd
d.cleanContainer(c.ID)
}()
- if err := cont.Start(p); err != nil {
- return execdriver.ExitStatus{ExitCode: -1}, err
+ //close the write end of any opened pipes now that they are dup'ed into the container
+ for _, writer := range writers {
+ writer.Close()
}
-
// 'oom' is used to emit 'oom' events to the eventstream, 'oomKilled' is used
// to set the 'OOMKilled' flag in state
oom := notifyOnOOM(cont)
@@ -202,6 +203,9 @@ func (d *Driver) Run(c *execdriver.Command, pipes *execdriver.Pipes, hooks execd
}
ps = execErr.ProcessState
}
+ // wait for all IO goroutine copiers to finish
+ wg.Wait()
+
cont.Destroy()
destroyed = true
// oomKilled will have an oom event if any process within the container was
@@ -298,6 +302,9 @@ func (d *Driver) Kill(c *execdriver.Command, sig int) error {
if err != nil {
return err
}
+ if state.InitProcessPid == -1 {
+ return fmt.Errorf("avoid sending signal %d to container %s with pid -1", sig, c.ID)
+ }
return syscall.Kill(state.InitProcessPid, syscall.Signal(sig))
}
@@ -480,24 +487,26 @@ func (t *TtyConsole) Close() error {
return t.console.Close()
}
-func setupPipes(container *configs.Config, processConfig *execdriver.ProcessConfig, p *libcontainer.Process, pipes *execdriver.Pipes) error {
+func setupPipes(container *configs.Config, processConfig *execdriver.ProcessConfig, p *libcontainer.Process, pipes *execdriver.Pipes, wg *sync.WaitGroup) ([]io.WriteCloser, error) {
+
+ writers := []io.WriteCloser{}
rootuid, err := container.HostUID()
if err != nil {
- return err
+ return writers, err
}
if processConfig.Tty {
cons, err := p.NewConsole(rootuid)
if err != nil {
- return err
+ return writers, err
}
term, err := NewTtyConsole(cons, pipes)
if err != nil {
- return err
+ return writers, err
}
processConfig.Terminal = term
- return nil
+ return writers, nil
}
// not a tty--set up stdio pipes
term := &execdriver.StdConsole{}
@@ -512,7 +521,7 @@ func setupPipes(container *configs.Config, processConfig *execdriver.ProcessConf
r, w, err := os.Pipe()
if err != nil {
- return err
+ return writers, err
}
if pipes.Stdin != nil {
go func() {
@@ -521,23 +530,32 @@ func setupPipes(container *configs.Config, processConfig *execdriver.ProcessConf
}()
p.Stdin = r
}
- return nil
+ return writers, nil
}
// if we have user namespaces enabled (rootuid != 0), we will set
// up os pipes for stderr, stdout, stdin so we can chown them to
// the proper ownership to allow for proper access to the underlying
// fds
- var fds []int
+ var fds []uintptr
+
+ copyPipes := func(out io.Writer, in io.ReadCloser) {
+ defer wg.Done()
+ io.Copy(out, in)
+ in.Close()
+ }
//setup stdout
r, w, err := os.Pipe()
if err != nil {
- return err
+ w.Close()
+ return writers, err
}
- fds = append(fds, int(r.Fd()), int(w.Fd()))
+ writers = append(writers, w)
+ fds = append(fds, r.Fd(), w.Fd())
if pipes.Stdout != nil {
- go io.Copy(pipes.Stdout, r)
+ wg.Add(1)
+ go copyPipes(pipes.Stdout, r)
}
term.Closers = append(term.Closers, r)
p.Stdout = w
@@ -545,11 +563,14 @@ func setupPipes(container *configs.Config, processConfig *execdriver.ProcessConf
//setup stderr
r, w, err = os.Pipe()
if err != nil {
- return err
+ w.Close()
+ return writers, err
}
- fds = append(fds, int(r.Fd()), int(w.Fd()))
+ writers = append(writers, w)
+ fds = append(fds, r.Fd(), w.Fd())
if pipes.Stderr != nil {
- go io.Copy(pipes.Stderr, r)
+ wg.Add(1)
+ go copyPipes(pipes.Stderr, r)
}
term.Closers = append(term.Closers, r)
p.Stderr = w
@@ -557,9 +578,10 @@ func setupPipes(container *configs.Config, processConfig *execdriver.ProcessConf
//setup stdin
r, w, err = os.Pipe()
if err != nil {
- return err
+ r.Close()
+ return writers, err
}
- fds = append(fds, int(r.Fd()), int(w.Fd()))
+ fds = append(fds, r.Fd(), w.Fd())
if pipes.Stdin != nil {
go func() {
io.Copy(w, pipes.Stdin)
@@ -568,11 +590,11 @@ func setupPipes(container *configs.Config, processConfig *execdriver.ProcessConf
p.Stdin = r
}
for _, fd := range fds {
- if err := syscall.Fchown(fd, rootuid, rootuid); err != nil {
- return fmt.Errorf("Failed to chown pipes fd: %v", err)
+ if err := syscall.Fchown(int(fd), rootuid, rootuid); err != nil {
+ return writers, fmt.Errorf("Failed to chown pipes fd: %v", err)
}
}
- return nil
+ return writers, nil
}
// SupportsHooks implements the execdriver Driver interface.
diff --git a/components/engine/daemon/execdriver/native/exec.go b/components/engine/daemon/execdriver/native/exec.go
index 0af5670239..d62fe5f405 100644
--- a/components/engine/daemon/execdriver/native/exec.go
+++ b/components/engine/daemon/execdriver/native/exec.go
@@ -7,6 +7,7 @@ import (
"os"
"os/exec"
"strings"
+ "sync"
"syscall"
"github.com/docker/docker/daemon/execdriver"
@@ -52,13 +53,19 @@ func (d *Driver) Exec(c *execdriver.Command, processConfig *execdriver.ProcessCo
}
config := active.Config()
- if err := setupPipes(&config, processConfig, p, pipes); err != nil {
+ wg := sync.WaitGroup{}
+ writers, err := setupPipes(&config, processConfig, p, pipes, &wg)
+ if err != nil {
return -1, err
}
if err := active.Start(p); err != nil {
return -1, err
}
+ //close the write end of any opened pipes now that they are dup'ed into the container
+ for _, writer := range writers {
+ writer.Close()
+ }
if hooks.Start != nil {
pid, err := p.Pid()
@@ -83,5 +90,7 @@ func (d *Driver) Exec(c *execdriver.Command, processConfig *execdriver.ProcessCo
}
ps = exitErr.ProcessState
}
+ // wait for all IO goroutine copiers to finish
+ wg.Wait()
return utils.ExitStatus(ps.Sys().(syscall.WaitStatus)), nil
}
diff --git a/components/engine/daemon/execdriver/windows/exec.go b/components/engine/daemon/execdriver/windows/exec.go
index 1b6c06bd14..c9129a8bbd 100644
--- a/components/engine/daemon/execdriver/windows/exec.go
+++ b/components/engine/daemon/execdriver/windows/exec.go
@@ -4,6 +4,7 @@ package windows
import (
"fmt"
+ "syscall"
"github.com/Microsoft/hcsshim"
"github.com/Sirupsen/logrus"
@@ -17,7 +18,6 @@ func (d *Driver) Exec(c *execdriver.Command, processConfig *execdriver.ProcessCo
term execdriver.Terminal
err error
exitCode int32
- errno uint32
)
active := d.activeContainers[c.ID]
@@ -41,13 +41,13 @@ func (d *Driver) Exec(c *execdriver.Command, processConfig *execdriver.ProcessCo
}
// Start the command running in the container.
- pid, stdin, stdout, stderr, rc, err := hcsshim.CreateProcessInComputeSystem(c.ID, pipes.Stdin != nil, true, !processConfig.Tty, createProcessParms)
+ pid, stdin, stdout, stderr, err := hcsshim.CreateProcessInComputeSystem(c.ID, pipes.Stdin != nil, true, !processConfig.Tty, createProcessParms)
if err != nil {
// TODO Windows: TP4 Workaround. In Hyper-V containers, there is a limitation
// of one exec per container. This should be fixed post TP4. CreateProcessInComputeSystem
// will return a specific error which we handle here to give a good error message
// back to the user instead of an inactionable "An invalid argument was supplied"
- if rc == hcsshim.Win32InvalidArgument {
+ if herr, ok := err.(*hcsshim.HcsError); ok && herr.Err == hcsshim.WSAEINVAL {
return -1, fmt.Errorf("The limit of docker execs per Hyper-V container has been exceeded")
}
logrus.Errorf("CreateProcessInComputeSystem() failed %s", err)
@@ -75,12 +75,12 @@ func (d *Driver) Exec(c *execdriver.Command, processConfig *execdriver.ProcessCo
hooks.Start(&c.ProcessConfig, int(pid), chOOM)
}
- if exitCode, errno, err = hcsshim.WaitForProcessInComputeSystem(c.ID, pid, hcsshim.TimeoutInfinite); err != nil {
- if errno == hcsshim.Win32PipeHasBeenEnded {
- logrus.Debugf("Exiting Run() after WaitForProcessInComputeSystem failed with recognised error 0x%X", errno)
+ if exitCode, err = hcsshim.WaitForProcessInComputeSystem(c.ID, pid, hcsshim.TimeoutInfinite); err != nil {
+ if herr, ok := err.(*hcsshim.HcsError); ok && herr.Err == syscall.ERROR_BROKEN_PIPE {
+ logrus.Debugf("Exiting Run() after WaitForProcessInComputeSystem failed with recognised error %s", err)
return hcsshim.WaitErrExecFailed, nil
}
- logrus.Warnf("WaitForProcessInComputeSystem failed (container may have been killed): 0x%X %s", errno, err)
+ logrus.Warnf("WaitForProcessInComputeSystem failed (container may have been killed): %s", err)
return -1, err
}
diff --git a/components/engine/daemon/execdriver/windows/run.go b/components/engine/daemon/execdriver/windows/run.go
index 0837ba85ee..b02fa747d9 100644
--- a/components/engine/daemon/execdriver/windows/run.go
+++ b/components/engine/daemon/execdriver/windows/run.go
@@ -9,6 +9,7 @@ import (
"path/filepath"
"strconv"
"strings"
+ "syscall"
"time"
"github.com/Microsoft/hcsshim"
@@ -20,6 +21,16 @@ import (
// preconfigured on the server.
const defaultContainerNAT = "ContainerNAT"
+// Win32 error codes that are used for various workarounds
+// These really should be ALL_CAPS to match golangs syscall library and standard
+// Win32 error conventions, but golint insists on CamelCase.
+const (
+ CoEClassstring = syscall.Errno(0x800401F3) // Invalid class string
+ ErrorNoNetwork = syscall.Errno(1222) // The network is not present or not started
+ ErrorBadPathname = syscall.Errno(161) // The specified path is invalid
+ ErrorInvalidObject = syscall.Errno(0x800710D8) // The object identifier does not represent a valid object
+)
+
type layer struct {
ID string
Path string
@@ -70,10 +81,10 @@ type containerInit struct {
IsDummy bool // Used for development purposes.
VolumePath string // Windows volume path for scratch space
Devices []device // Devices used by the container
- IgnoreFlushesDuringBoot bool // Optimisation hint for container startup in Windows
+ IgnoreFlushesDuringBoot bool // Optimization hint for container startup in Windows
LayerFolderPath string // Where the layer folders are located
Layers []layer // List of storage layers
- ProcessorWeight int64 `json:",omitempty"` // CPU Shares 0..10000 on Windows; where 0 will be ommited and HCS will default.
+ ProcessorWeight int64 `json:",omitempty"` // CPU Shares 0..10000 on Windows; where 0 will be omitted and HCS will default.
HostName string // Hostname
MappedDirectories []mappedDir // List of mapped directories (volumes/mounts)
SandboxPath string // Location of unmounted sandbox (used for Hyper-V containers, not Windows Server containers)
@@ -227,30 +238,30 @@ func (d *Driver) Run(c *execdriver.Command, pipes *execdriver.Pipes, hooks execd
// TODO Windows TP5 timeframe. Remove when TP4 is no longer supported.
// The following a workaround for Windows TP4 which has a networking
// bug which fairly frequently returns an error. Back off and retry.
- maxAttempts := 1
- if TP4RetryHack {
- maxAttempts = 5
- }
- i := 0
- for i < maxAttempts {
- i++
+ maxAttempts := 5
+ for i := 0; i < maxAttempts; i++ {
err = hcsshim.CreateComputeSystem(c.ID, configuration)
- if err != nil {
- if TP4RetryHack {
- if !strings.Contains(err.Error(), `Win32 API call returned error r1=0x800401f3`) && // Invalid class string
- !strings.Contains(err.Error(), `Win32 API call returned error r1=0x80070490`) && // Element not found
- !strings.Contains(err.Error(), `Win32 API call returned error r1=0x80070002`) && // The system cannot find the file specified
- !strings.Contains(err.Error(), `Win32 API call returned error r1=0x800704c6`) && // The network is not present or not started
- !strings.Contains(err.Error(), `Win32 API call returned error r1=0x800700a1`) { // The specified path is invalid
- logrus.Debugln("Failed to create temporary container ", err)
- return execdriver.ExitStatus{ExitCode: -1}, err
- }
- logrus.Warnf("Invoking Windows TP4 retry hack (%d of %d)", i, maxAttempts-1)
- time.Sleep(50 * time.Millisecond)
- }
- } else {
+ if err == nil {
break
}
+
+ if !TP4RetryHack {
+ return execdriver.ExitStatus{ExitCode: -1}, err
+ }
+
+ if herr, ok := err.(*hcsshim.HcsError); ok {
+ if herr.Err != syscall.ERROR_NOT_FOUND && // Element not found
+ herr.Err != syscall.ERROR_FILE_NOT_FOUND && // The system cannot find the file specified
+ herr.Err != ErrorNoNetwork && // The network is not present or not started
+ herr.Err != ErrorBadPathname && // The specified path is invalid
+ herr.Err != CoEClassstring && // Invalid class string
+ herr.Err != ErrorInvalidObject { // The object identifier does not represent a valid object
+ logrus.Debugln("Failed to create temporary container ", err)
+ return execdriver.ExitStatus{ExitCode: -1}, err
+ }
+ logrus.Warnf("Invoking Windows TP4 retry hack (%d of %d)", i, maxAttempts-1)
+ time.Sleep(50 * time.Millisecond)
+ }
}
// Start the container
@@ -264,16 +275,17 @@ func (d *Driver) Run(c *execdriver.Command, pipes *execdriver.Pipes, hooks execd
// Stop the container
if forceKill {
logrus.Debugf("Forcibly terminating container %s", c.ID)
- if errno, err := hcsshim.TerminateComputeSystem(c.ID, hcsshim.TimeoutInfinite, "exec-run-defer"); err != nil {
- logrus.Warnf("Ignoring error from TerminateComputeSystem 0x%X %s", errno, err)
+ if err := hcsshim.TerminateComputeSystem(c.ID, hcsshim.TimeoutInfinite, "exec-run-defer"); err != nil {
+ logrus.Warnf("Ignoring error from TerminateComputeSystem %s", err)
}
} else {
logrus.Debugf("Shutting down container %s", c.ID)
- if errno, err := hcsshim.ShutdownComputeSystem(c.ID, hcsshim.TimeoutInfinite, "exec-run-defer"); err != nil {
- if errno != hcsshim.Win32SystemShutdownIsInProgress &&
- errno != hcsshim.Win32SpecifiedPathInvalid &&
- errno != hcsshim.Win32SystemCannotFindThePathSpecified {
- logrus.Warnf("Ignoring error from ShutdownComputeSystem 0x%X %s", errno, err)
+ if err := hcsshim.ShutdownComputeSystem(c.ID, hcsshim.TimeoutInfinite, "exec-run-defer"); err != nil {
+ if herr, ok := err.(*hcsshim.HcsError); !ok ||
+ (herr.Err != hcsshim.ERROR_SHUTDOWN_IN_PROGRESS &&
+ herr.Err != ErrorBadPathname &&
+ herr.Err != syscall.ERROR_PATH_NOT_FOUND) {
+ logrus.Warnf("Ignoring error from ShutdownComputeSystem %s", err)
}
}
}
@@ -295,7 +307,7 @@ func (d *Driver) Run(c *execdriver.Command, pipes *execdriver.Pipes, hooks execd
}
// Start the command running in the container.
- pid, stdin, stdout, stderr, _, err := hcsshim.CreateProcessInComputeSystem(c.ID, pipes.Stdin != nil, true, !c.ProcessConfig.Tty, createProcessParms)
+ pid, stdin, stdout, stderr, err := hcsshim.CreateProcessInComputeSystem(c.ID, pipes.Stdin != nil, true, !c.ProcessConfig.Tty, createProcessParms)
if err != nil {
logrus.Errorf("CreateProcessInComputeSystem() failed %s", err)
return execdriver.ExitStatus{ExitCode: -1}, err
@@ -332,13 +344,9 @@ func (d *Driver) Run(c *execdriver.Command, pipes *execdriver.Pipes, hooks execd
hooks.Start(&c.ProcessConfig, int(pid), chOOM)
}
- var (
- exitCode int32
- errno uint32
- )
- exitCode, errno, err = hcsshim.WaitForProcessInComputeSystem(c.ID, pid, hcsshim.TimeoutInfinite)
+ exitCode, err := hcsshim.WaitForProcessInComputeSystem(c.ID, pid, hcsshim.TimeoutInfinite)
if err != nil {
- if errno != hcsshim.Win32PipeHasBeenEnded {
+ if herr, ok := err.(*hcsshim.HcsError); ok && herr.Err != syscall.ERROR_BROKEN_PIPE {
logrus.Warnf("WaitForProcessInComputeSystem failed (container may have been killed): %s", err)
}
// Do NOT return err here as the container would have
diff --git a/components/engine/daemon/execdriver/windows/terminatekill.go b/components/engine/daemon/execdriver/windows/terminatekill.go
index 7068a68fa9..d20b0e2b9d 100644
--- a/components/engine/daemon/execdriver/windows/terminatekill.go
+++ b/components/engine/daemon/execdriver/windows/terminatekill.go
@@ -28,8 +28,8 @@ func kill(id string, pid int, sig syscall.Signal) error {
if sig == syscall.SIGKILL || forceKill {
// Terminate the compute system
- if errno, err := hcsshim.TerminateComputeSystem(id, hcsshim.TimeoutInfinite, context); err != nil {
- logrus.Errorf("Failed to terminate %s - 0x%X %q", id, errno, err)
+ if err := hcsshim.TerminateComputeSystem(id, hcsshim.TimeoutInfinite, context); err != nil {
+ logrus.Errorf("Failed to terminate %s - %q", id, err)
}
} else {
@@ -41,8 +41,8 @@ func kill(id string, pid int, sig syscall.Signal) error {
}
// Shutdown the compute system
- if errno, err := hcsshim.ShutdownComputeSystem(id, hcsshim.TimeoutInfinite, context); err != nil {
- logrus.Errorf("Failed to shutdown %s - 0x%X %q", id, errno, err)
+ if err := hcsshim.ShutdownComputeSystem(id, hcsshim.TimeoutInfinite, context); err != nil {
+ logrus.Errorf("Failed to shutdown %s - %q", id, err)
}
}
return err
diff --git a/components/engine/daemon/execdriver/windows/update.go b/components/engine/daemon/execdriver/windows/update.go
index 33c0b9ef1a..a4c42a6b08 100644
--- a/components/engine/daemon/execdriver/windows/update.go
+++ b/components/engine/daemon/execdriver/windows/update.go
@@ -3,12 +3,12 @@
package windows
import (
- "fmt"
-
"github.com/docker/docker/daemon/execdriver"
)
// Update updates resource configs for a container.
func (d *Driver) Update(c *execdriver.Command) error {
- return fmt.Errorf("Windows: Update not implemented")
+ // Updating resource isn't supported on Windows
+ // but we should return nil for enabling updating container
+ return nil
}
diff --git a/components/engine/daemon/execdriver/windows/windows.go b/components/engine/daemon/execdriver/windows/windows.go
index 0be7c0b02d..e6c8f10b91 100644
--- a/components/engine/daemon/execdriver/windows/windows.go
+++ b/components/engine/daemon/execdriver/windows/windows.go
@@ -4,16 +4,15 @@ package windows
import (
"fmt"
- "strconv"
"strings"
"sync"
+ "github.com/Microsoft/hcsshim"
"github.com/Sirupsen/logrus"
"github.com/docker/docker/daemon/execdriver"
"github.com/docker/docker/dockerversion"
"github.com/docker/docker/pkg/parsers"
"github.com/docker/engine-api/types/container"
- "golang.org/x/sys/windows/registry"
)
// TP4RetryHack is a hack to retry CreateComputeSystem if it fails with
@@ -28,11 +27,11 @@ var dummyMode bool
// This allows the daemon to force kill (HCS terminate) rather than shutdown
var forceKill bool
-// DefaultIsolation allows users to specify a default isolation mode for
+// DefaultIsolation allows users to specify a default isolation technology for
// when running a container on Windows. For example docker daemon -D
// --exec-opt isolation=hyperv will cause Windows to always run containers
// as Hyper-V containers unless otherwise specified.
-var DefaultIsolation container.IsolationLevel = "process"
+var DefaultIsolation container.Isolation = "process"
// Define name and version for windows
var (
@@ -83,13 +82,13 @@ func NewDriver(root string, options []string) (*Driver, error) {
}
case "isolation":
- if !container.IsolationLevel(val).IsValid() {
+ if !container.Isolation(val).IsValid() {
return nil, fmt.Errorf("Unrecognised exec driver option 'isolation':'%s'", val)
}
- if container.IsolationLevel(val).IsHyperV() {
+ if container.Isolation(val).IsHyperV() {
DefaultIsolation = "hyperv"
}
- logrus.Infof("Windows default isolation level: '%s'", val)
+ logrus.Infof("Windows default isolation: '%s'", val)
default:
return nil, fmt.Errorf("Unrecognised exec driver option %s\n", key)
}
@@ -98,33 +97,11 @@ func NewDriver(root string, options []string) (*Driver, error) {
// TODO Windows TP5 timeframe. Remove this next block of code once TP4
// is no longer supported. Also remove the workaround in run.go.
//
- // Hack for TP4 - determine the version of Windows from the registry.
+ // Hack for TP4.
// This overcomes an issue on TP4 which causes CreateComputeSystem to
// intermittently fail. It's predominantly here to make Windows to Windows
// CI more reliable.
- TP4RetryHack = false
- k, err := registry.OpenKey(registry.LOCAL_MACHINE, `SOFTWARE\Microsoft\Windows NT\CurrentVersion`, registry.QUERY_VALUE)
- if err != nil {
- return &Driver{}, err
- }
- defer k.Close()
-
- s, _, err := k.GetStringValue("BuildLab")
- if err != nil {
- return &Driver{}, err
- }
- parts := strings.Split(s, ".")
- if len(parts) < 1 {
- return &Driver{}, err
- }
- var val int
- if val, err = strconv.Atoi(parts[0]); err != nil {
- return &Driver{}, err
- }
- if val < 14250 {
- TP4RetryHack = true
- }
- // End of Windows TP4 hack
+ TP4RetryHack = hcsshim.IsTP4()
return &Driver{
root: root,
diff --git a/components/engine/daemon/export.go b/components/engine/daemon/export.go
index 9e3d8da340..80d7dbb2e1 100644
--- a/components/engine/daemon/export.go
+++ b/components/engine/daemon/export.go
@@ -1,10 +1,10 @@
package daemon
import (
+ "fmt"
"io"
"github.com/docker/docker/container"
- derr "github.com/docker/docker/errors"
"github.com/docker/docker/pkg/archive"
"github.com/docker/docker/pkg/ioutils"
)
@@ -19,13 +19,13 @@ func (daemon *Daemon) ContainerExport(name string, out io.Writer) error {
data, err := daemon.containerExport(container)
if err != nil {
- return derr.ErrorCodeExportFailed.WithArgs(name, err)
+ return fmt.Errorf("Error exporting container %s: %v", name, err)
}
defer data.Close()
// Stream the entire contents of the container (basically a volatile snapshot)
if _, err := io.Copy(out, data); err != nil {
- return derr.ErrorCodeExportFailed.WithArgs(name, err)
+ return fmt.Errorf("Error exporting container %s: %v", name, err)
}
return nil
}
diff --git a/components/engine/daemon/graphdriver/aufs/aufs.go b/components/engine/daemon/graphdriver/aufs/aufs.go
index 51054fa6ef..ac0bc5f483 100644
--- a/components/engine/daemon/graphdriver/aufs/aufs.go
+++ b/components/engine/daemon/graphdriver/aufs/aufs.go
@@ -34,6 +34,7 @@ import (
"syscall"
"github.com/Sirupsen/logrus"
+ "github.com/vbatts/tar-split/tar/storage"
"github.com/docker/docker/daemon/graphdriver"
"github.com/docker/docker/pkg/archive"
@@ -227,10 +228,14 @@ func (a *Driver) Create(id, parent, mountLabel string) error {
}
}
}
+ a.Lock()
a.active[id] = &data{}
+ a.Unlock()
return nil
}
+// createDirsFor creates two directories for the given id.
+// mnt and diff
func (a *Driver) createDirsFor(id string) error {
paths := []string{
"mnt",
@@ -241,6 +246,9 @@ func (a *Driver) createDirsFor(id string) error {
if err != nil {
return err
}
+ // Directory permission is 0755.
+ // The path of directories are /mnt/
+ // and /diff/
for _, p := range paths {
if err := idtools.MkdirAllAs(path.Join(a.rootPath(), p, id), 0755, rootUID, rootGID); err != nil {
return err
@@ -285,20 +293,15 @@ func (a *Driver) Remove(id string) error {
if err := os.Remove(path.Join(a.rootPath(), "layers", id)); err != nil && !os.IsNotExist(err) {
return err
}
+ if m != nil {
+ delete(a.active, id)
+ }
return nil
}
// Get returns the rootfs path for the id.
// This will mount the dir at it's given path
func (a *Driver) Get(id, mountLabel string) (string, error) {
- ids, err := getParentIds(a.rootPath(), id)
- if err != nil {
- if !os.IsNotExist(err) {
- return "", err
- }
- ids = []string{}
- }
-
// Protect the a.active from concurrent access
a.Lock()
defer a.Unlock()
@@ -309,13 +312,18 @@ func (a *Driver) Get(id, mountLabel string) (string, error) {
a.active[id] = m
}
+ parents, err := a.getParentLayerPaths(id)
+ if err != nil && !os.IsNotExist(err) {
+ return "", err
+ }
+
// If a dir does not have a parent ( no layers )do not try to mount
// just return the diff path to the data
m.path = path.Join(a.rootPath(), "diff", id)
- if len(ids) > 0 {
+ if len(parents) > 0 {
m.path = path.Join(a.rootPath(), "mnt", id)
if m.referenceCount == 0 {
- if err := a.mount(id, m, mountLabel); err != nil {
+ if err := a.mount(id, m, mountLabel, parents); err != nil {
return "", err
}
}
@@ -367,10 +375,19 @@ func (a *Driver) Diff(id, parent string) (archive.Archive, error) {
})
}
-// DiffPath returns path to the directory that contains files for the layer
-// differences. Used for direct access for tar-split.
-func (a *Driver) DiffPath(id string) (string, func() error, error) {
- return path.Join(a.rootPath(), "diff", id), func() error { return nil }, nil
+type fileGetNilCloser struct {
+ storage.FileGetter
+}
+
+func (f fileGetNilCloser) Close() error {
+ return nil
+}
+
+// DiffGetter returns a FileGetCloser that can read files from the directory that
+// contains files for the layer differences. Used for direct access for tar-split.
+func (a *Driver) DiffGetter(id string) (graphdriver.FileGetCloser, error) {
+ p := path.Join(a.rootPath(), "diff", id)
+ return fileGetNilCloser{storage.NewPathFileGetter(p)}, nil
}
func (a *Driver) applyDiff(id string, diff archive.Reader) error {
@@ -426,7 +443,7 @@ func (a *Driver) getParentLayerPaths(id string) ([]string, error) {
return layers, nil
}
-func (a *Driver) mount(id string, m *data, mountLabel string) error {
+func (a *Driver) mount(id string, m *data, mountLabel string, layers []string) error {
// If the id is mounted or we get an error return
if mounted, err := a.mounted(m); err != nil || mounted {
return err
@@ -437,11 +454,6 @@ func (a *Driver) mount(id string, m *data, mountLabel string) error {
rw = path.Join(a.rootPath(), "diff", id)
)
- layers, err := a.getParentLayerPaths(id)
- if err != nil {
- return err
- }
-
if err := a.aufsMount(layers, rw, target, mountLabel); err != nil {
return fmt.Errorf("error creating aufs mount to %s: %v", target, err)
}
@@ -456,7 +468,11 @@ func (a *Driver) unmount(m *data) error {
}
func (a *Driver) mounted(m *data) (bool, error) {
- return mountpk.Mounted(m.path)
+ var buf syscall.Statfs_t
+ if err := syscall.Statfs(m.path, &buf); err != nil {
+ return false, nil
+ }
+ return graphdriver.FsMagic(buf.Type) == graphdriver.FsMagicAufs, nil
}
// Cleanup aufs and unmount all mountpoints
diff --git a/components/engine/daemon/graphdriver/aufs/aufs_test.go b/components/engine/daemon/graphdriver/aufs/aufs_test.go
index 761b5b6872..0f6d59d054 100644
--- a/components/engine/daemon/graphdriver/aufs/aufs_test.go
+++ b/components/engine/daemon/graphdriver/aufs/aufs_test.go
@@ -9,11 +9,13 @@ import (
"io/ioutil"
"os"
"path"
+ "sync"
"testing"
"github.com/docker/docker/daemon/graphdriver"
"github.com/docker/docker/pkg/archive"
"github.com/docker/docker/pkg/reexec"
+ "github.com/docker/docker/pkg/stringid"
)
var (
@@ -25,7 +27,7 @@ func init() {
reexec.Init()
}
-func testInit(dir string, t *testing.T) graphdriver.Driver {
+func testInit(dir string, t testing.TB) graphdriver.Driver {
d, err := Init(dir, nil, nil, nil)
if err != nil {
if err == graphdriver.ErrNotSupported {
@@ -37,7 +39,7 @@ func testInit(dir string, t *testing.T) graphdriver.Driver {
return d
}
-func newDriver(t *testing.T) *Driver {
+func newDriver(t testing.TB) *Driver {
if err := os.MkdirAll(tmp, 0755); err != nil {
t.Fatal(err)
}
@@ -732,3 +734,68 @@ func TestMountMoreThan42LayersMatchingPathLength(t *testing.T) {
zeroes += "0"
}
}
+
+func BenchmarkConcurrentAccess(b *testing.B) {
+ b.StopTimer()
+ b.ResetTimer()
+
+ d := newDriver(b)
+ defer os.RemoveAll(tmp)
+ defer d.Cleanup()
+
+ numConcurent := 256
+ // create a bunch of ids
+ var ids []string
+ for i := 0; i < numConcurent; i++ {
+ ids = append(ids, stringid.GenerateNonCryptoID())
+ }
+
+ if err := d.Create(ids[0], "", ""); err != nil {
+ b.Fatal(err)
+ }
+
+ if err := d.Create(ids[1], ids[0], ""); err != nil {
+ b.Fatal(err)
+ }
+
+ parent := ids[1]
+ ids = append(ids[2:])
+
+ chErr := make(chan error, numConcurent)
+ var outerGroup sync.WaitGroup
+ outerGroup.Add(len(ids))
+ b.StartTimer()
+
+ // here's the actual bench
+ for _, id := range ids {
+ go func(id string) {
+ defer outerGroup.Done()
+ if err := d.Create(id, parent, ""); err != nil {
+ b.Logf("Create %s failed", id)
+ chErr <- err
+ return
+ }
+ var innerGroup sync.WaitGroup
+ for i := 0; i < b.N; i++ {
+ innerGroup.Add(1)
+ go func() {
+ d.Get(id, "")
+ d.Put(id)
+ innerGroup.Done()
+ }()
+ }
+ innerGroup.Wait()
+ d.Remove(id)
+ }(id)
+ }
+
+ outerGroup.Wait()
+ b.StopTimer()
+ close(chErr)
+ for err := range chErr {
+ if err != nil {
+ b.Log(err)
+ b.Fail()
+ }
+ }
+}
diff --git a/components/engine/daemon/graphdriver/aufs/mount.go b/components/engine/daemon/graphdriver/aufs/mount.go
index d7e9bf9fd7..36fa62e41b 100644
--- a/components/engine/daemon/graphdriver/aufs/mount.go
+++ b/components/engine/daemon/graphdriver/aufs/mount.go
@@ -12,7 +12,7 @@ import (
// Unmount the target specified.
func Unmount(target string) error {
if err := exec.Command("auplink", target, "flush").Run(); err != nil {
- logrus.Errorf("Couldn't run auplink before unmount: %s", err)
+ logrus.Errorf("Couldn't run auplink before unmount %s: %s", target, err)
}
if err := syscall.Unmount(target, 0); err != nil {
return err
diff --git a/components/engine/daemon/graphdriver/btrfs/btrfs.go b/components/engine/daemon/graphdriver/btrfs/btrfs.go
index 48388f72fe..5ca86a5b6f 100644
--- a/components/engine/daemon/graphdriver/btrfs/btrfs.go
+++ b/components/engine/daemon/graphdriver/btrfs/btrfs.go
@@ -262,7 +262,7 @@ func (d *Driver) Create(id, parent, mountLabel string) error {
return err
}
if !st.IsDir() {
- return fmt.Errorf("%s: not a direcotory", parentDir)
+ return fmt.Errorf("%s: not a directory", parentDir)
}
if err := subvolSnapshot(parentDir, subvolumes, id); err != nil {
return err
diff --git a/components/engine/daemon/graphdriver/devmapper/deviceset.go b/components/engine/daemon/graphdriver/devmapper/deviceset.go
index d8522349b8..7748413c13 100644
--- a/components/engine/daemon/graphdriver/devmapper/deviceset.go
+++ b/components/engine/daemon/graphdriver/devmapper/deviceset.go
@@ -573,7 +573,7 @@ func determineDefaultFS() string {
return "xfs"
}
- logrus.Warn("devmapper: XFS is not supported in your system. Either the kernel doesnt support it or mkfs.xfs is not in your PATH. Defaulting to ext4 filesystem")
+ logrus.Warn("devmapper: XFS is not supported in your system. Either the kernel doesn't support it or mkfs.xfs is not in your PATH. Defaulting to ext4 filesystem")
return "ext4"
}
diff --git a/components/engine/daemon/graphdriver/driver.go b/components/engine/daemon/graphdriver/driver.go
index d9ab839c2e..abc400083d 100644
--- a/components/engine/daemon/graphdriver/driver.go
+++ b/components/engine/daemon/graphdriver/driver.go
@@ -8,6 +8,7 @@ import (
"strings"
"github.com/Sirupsen/logrus"
+ "github.com/vbatts/tar-split/tar/storage"
"github.com/docker/docker/pkg/archive"
"github.com/docker/docker/pkg/idtools"
@@ -92,6 +93,23 @@ type Driver interface {
DiffSize(id, parent string) (size int64, err error)
}
+// DiffGetterDriver is the interface for layered file system drivers that
+// provide a specialized function for getting file contents for tar-split.
+type DiffGetterDriver interface {
+ Driver
+ // DiffGetter returns an interface to efficiently retrieve the contents
+ // of files in a layer.
+ DiffGetter(id string) (FileGetCloser, error)
+}
+
+// FileGetCloser extends the storage.FileGetter interface with a Close method
+// for cleaning up.
+type FileGetCloser interface {
+ storage.FileGetter
+ // Close cleans up any resources associated with the FileGetCloser.
+ Close() error
+}
+
func init() {
drivers = make(map[string]InitFunc)
}
diff --git a/components/engine/daemon/graphdriver/windows/windows.go b/components/engine/daemon/graphdriver/windows/windows.go
index 77f4f1b774..2b5b549e20 100644
--- a/components/engine/daemon/graphdriver/windows/windows.go
+++ b/components/engine/daemon/graphdriver/windows/windows.go
@@ -3,17 +3,23 @@
package windows
import (
+ "bufio"
"crypto/sha512"
"encoding/json"
"fmt"
+ "io"
"io/ioutil"
"os"
+ "path"
"path/filepath"
- "strconv"
"strings"
"sync"
+ "syscall"
"time"
+ "github.com/Microsoft/go-winio"
+ "github.com/Microsoft/go-winio/archive/tar"
+ "github.com/Microsoft/go-winio/backuptar"
"github.com/Microsoft/hcsshim"
"github.com/Sirupsen/logrus"
"github.com/docker/docker/daemon/graphdriver"
@@ -21,7 +27,7 @@ import (
"github.com/docker/docker/pkg/chrootarchive"
"github.com/docker/docker/pkg/idtools"
"github.com/docker/docker/pkg/ioutils"
- "github.com/docker/docker/pkg/random"
+ "github.com/vbatts/tar-split/tar/storage"
)
// init registers the windows graph drivers to the register.
@@ -47,6 +53,8 @@ type Driver struct {
active map[string]int
}
+var _ graphdriver.DiffGetterDriver = &Driver{}
+
// InitFilter returns a new Windows storage filter driver.
func InitFilter(home string, options []string, uidMaps, gidMaps []idtools.IDMap) (graphdriver.Driver, error) {
logrus.Debugf("WindowsGraphDriver InitFilter at %s", home)
@@ -262,7 +270,7 @@ func (d *Driver) Cleanup() error {
// Diff produces an archive of the changes between the specified
// layer and its parent layer which may be "".
-func (d *Driver) Diff(id, parent string) (arch archive.Archive, err error) {
+func (d *Driver) Diff(id, parent string) (_ archive.Archive, err error) {
rID, err := d.resolveID(id)
if err != nil {
return
@@ -274,6 +282,8 @@ func (d *Driver) Diff(id, parent string) (arch archive.Archive, err error) {
return
}
+ var undo func()
+
d.Lock()
// To support export, a layer must be activated but not prepared.
@@ -283,6 +293,56 @@ func (d *Driver) Diff(id, parent string) (arch archive.Archive, err error) {
d.Unlock()
return
}
+ undo = func() {
+ if err := hcsshim.DeactivateLayer(d.info, rID); err != nil {
+ logrus.Warnf("Failed to Deactivate %s: %s", rID, err)
+ }
+ }
+ } else {
+ if err = hcsshim.UnprepareLayer(d.info, rID); err != nil {
+ d.Unlock()
+ return
+ }
+ undo = func() {
+ if err := hcsshim.PrepareLayer(d.info, rID, layerChain); err != nil {
+ logrus.Warnf("Failed to re-PrepareLayer %s: %s", rID, err)
+ }
+ }
+ }
+ }
+
+ d.Unlock()
+
+ arch, err := d.exportLayer(rID, layerChain)
+ if err != nil {
+ undo()
+ return
+ }
+ return ioutils.NewReadCloserWrapper(arch, func() error {
+ defer undo()
+ return arch.Close()
+ }), nil
+}
+
+// Changes produces a list of changes between the specified layer
+// and its parent layer. If parent is "", then all changes will be ADD changes.
+func (d *Driver) Changes(id, parent string) ([]archive.Change, error) {
+ rID, err := d.resolveID(id)
+ if err != nil {
+ return nil, err
+ }
+ parentChain, err := d.getLayerChain(rID)
+ if err != nil {
+ return nil, err
+ }
+
+ d.Lock()
+ if d.info.Flavour == filterDriver {
+ if d.active[rID] == 0 {
+ if err = hcsshim.ActivateLayer(d.info, rID); err != nil {
+ d.Unlock()
+ return nil, err
+ }
defer func() {
if err := hcsshim.DeactivateLayer(d.info, rID); err != nil {
logrus.Warnf("Failed to Deactivate %s: %s", rID, err)
@@ -291,25 +351,41 @@ func (d *Driver) Diff(id, parent string) (arch archive.Archive, err error) {
} else {
if err = hcsshim.UnprepareLayer(d.info, rID); err != nil {
d.Unlock()
- return
+ return nil, err
}
defer func() {
- if err := hcsshim.PrepareLayer(d.info, rID, layerChain); err != nil {
+ if err := hcsshim.PrepareLayer(d.info, rID, parentChain); err != nil {
logrus.Warnf("Failed to re-PrepareLayer %s: %s", rID, err)
}
}()
}
}
-
d.Unlock()
- return d.exportLayer(rID, layerChain)
-}
+ r, err := hcsshim.NewLayerReader(d.info, id, parentChain)
+ if err != nil {
+ return nil, err
+ }
+ defer r.Close()
-// Changes produces a list of changes between the specified layer
-// and its parent layer. If parent is "", then all changes will be ADD changes.
-func (d *Driver) Changes(id, parent string) ([]archive.Change, error) {
- return nil, fmt.Errorf("The Windows graphdriver does not support Changes()")
+ var changes []archive.Change
+ for {
+ name, _, fileInfo, err := r.Next()
+ if err == io.EOF {
+ break
+ }
+ if err != nil {
+ return nil, err
+ }
+ name = filepath.ToSlash(name)
+ if fileInfo == nil {
+ changes = append(changes, archive.Change{name, archive.ChangeDelete})
+ } else {
+ // Currently there is no way to tell between an add and a modify.
+ changes = append(changes, archive.Change{name, archive.ChangeModify})
+ }
+ }
+ return changes, nil
}
// ApplyDiff extracts the changeset from the given diff into the
@@ -441,71 +517,162 @@ func (d *Driver) GetMetadata(id string) (map[string]string, error) {
return m, nil
}
-// exportLayer generates an archive from a layer based on the given ID.
-func (d *Driver) exportLayer(id string, parentLayerPaths []string) (arch archive.Archive, err error) {
- layerFolder := d.dir(id)
-
- tempFolder := layerFolder + "-" + strconv.FormatUint(uint64(random.Rand.Uint32()), 10)
- if err = os.MkdirAll(tempFolder, 0755); err != nil {
- logrus.Errorf("Could not create %s %s", tempFolder, err)
- return
- }
- defer func() {
+func writeTarFromLayer(r hcsshim.LayerReader, w io.Writer) error {
+ t := tar.NewWriter(w)
+ for {
+ name, size, fileInfo, err := r.Next()
+ if err == io.EOF {
+ break
+ }
if err != nil {
- _, folderName := filepath.Split(tempFolder)
- if err2 := hcsshim.DestroyLayer(d.info, folderName); err2 != nil {
- logrus.Warnf("Couldn't clean-up tempFolder: %s %s", tempFolder, err2)
+ return err
+ }
+ if fileInfo == nil {
+ // Write a whiteout file.
+ hdr := &tar.Header{
+ Name: filepath.ToSlash(filepath.Join(filepath.Dir(name), archive.WhiteoutPrefix+filepath.Base(name))),
+ }
+ err := t.WriteHeader(hdr)
+ if err != nil {
+ return err
+ }
+ } else {
+ err = backuptar.WriteTarFileFromBackupStream(t, r, name, size, fileInfo)
+ if err != nil {
+ return err
}
}
+ }
+ return t.Close()
+}
+
+// exportLayer generates an archive from a layer based on the given ID.
+func (d *Driver) exportLayer(id string, parentLayerPaths []string) (archive.Archive, error) {
+ if hcsshim.IsTP4() {
+ // Export in TP4 format to maintain compatibility with existing images and
+ // because ExportLayer is somewhat broken on TP4 and can't work with the new
+ // scheme.
+ tempFolder, err := ioutil.TempDir("", "hcs")
+ if err != nil {
+ return nil, err
+ }
+ defer func() {
+ if err != nil {
+ os.RemoveAll(tempFolder)
+ }
+ }()
+
+ if err = hcsshim.ExportLayer(d.info, id, tempFolder, parentLayerPaths); err != nil {
+ return nil, err
+ }
+ archive, err := archive.Tar(tempFolder, archive.Uncompressed)
+ if err != nil {
+ return nil, err
+ }
+ return ioutils.NewReadCloserWrapper(archive, func() error {
+ err := archive.Close()
+ os.RemoveAll(tempFolder)
+ return err
+ }), nil
+ }
+
+ var r hcsshim.LayerReader
+ r, err := hcsshim.NewLayerReader(d.info, id, parentLayerPaths)
+ if err != nil {
+ return nil, err
+ }
+
+ archive, w := io.Pipe()
+ go func() {
+ err := writeTarFromLayer(r, w)
+ cerr := r.Close()
+ if err == nil {
+ err = cerr
+ }
+ w.CloseWithError(err)
}()
- if err = hcsshim.ExportLayer(d.info, id, tempFolder, parentLayerPaths); err != nil {
- return
- }
+ return archive, nil
+}
- archive, err := archive.Tar(tempFolder, archive.Uncompressed)
- if err != nil {
- return
- }
- return ioutils.NewReadCloserWrapper(archive, func() error {
- err := archive.Close()
- d.Put(id)
- _, folderName := filepath.Split(tempFolder)
- if err2 := hcsshim.DestroyLayer(d.info, folderName); err2 != nil {
- logrus.Warnf("Couldn't clean-up tempFolder: %s %s", tempFolder, err2)
+func writeLayerFromTar(r archive.Reader, w hcsshim.LayerWriter) (int64, error) {
+ t := tar.NewReader(r)
+ hdr, err := t.Next()
+ totalSize := int64(0)
+ buf := bufio.NewWriter(nil)
+ for err == nil {
+ base := path.Base(hdr.Name)
+ if strings.HasPrefix(base, archive.WhiteoutPrefix) {
+ name := path.Join(path.Dir(hdr.Name), base[len(archive.WhiteoutPrefix):])
+ err = w.Remove(filepath.FromSlash(name))
+ if err != nil {
+ return 0, err
+ }
+ hdr, err = t.Next()
+ } else {
+ var (
+ name string
+ size int64
+ fileInfo *winio.FileBasicInfo
+ )
+ name, size, fileInfo, err = backuptar.FileInfoFromHeader(hdr)
+ if err != nil {
+ return 0, err
+ }
+ err = w.Add(filepath.FromSlash(name), fileInfo)
+ if err != nil {
+ return 0, err
+ }
+ buf.Reset(w)
+ hdr, err = backuptar.WriteBackupStreamFromTarFile(buf, t, hdr)
+ ferr := buf.Flush()
+ if ferr != nil {
+ err = ferr
+ }
+ totalSize += size
}
- return err
- }), nil
-
+ }
+ if err != io.EOF {
+ return 0, err
+ }
+ return totalSize, nil
}
// importLayer adds a new layer to the tag and graph store based on the given data.
func (d *Driver) importLayer(id string, layerData archive.Reader, parentLayerPaths []string) (size int64, err error) {
- layerFolder := d.dir(id)
-
- tempFolder := layerFolder + "-" + strconv.FormatUint(uint64(random.Rand.Uint32()), 10)
- if err = os.MkdirAll(tempFolder, 0755); err != nil {
- logrus.Errorf("Could not create %s %s", tempFolder, err)
- return
- }
- defer func() {
- _, folderName := filepath.Split(tempFolder)
- if err2 := hcsshim.DestroyLayer(d.info, folderName); err2 != nil {
- logrus.Warnf("Couldn't clean-up tempFolder: %s %s", tempFolder, err2)
+ if hcsshim.IsTP4() {
+ // Import from TP4 format to maintain compatibility with existing images.
+ var tempFolder string
+ tempFolder, err = ioutil.TempDir("", "hcs")
+ if err != nil {
+ return
}
- }()
+ defer os.RemoveAll(tempFolder)
- start := time.Now().UTC()
- logrus.Debugf("Start untar layer")
- if size, err = chrootarchive.ApplyLayer(tempFolder, layerData); err != nil {
- return
- }
- logrus.Debugf("Untar time: %vs", time.Now().UTC().Sub(start).Seconds())
-
- if err = hcsshim.ImportLayer(d.info, id, tempFolder, parentLayerPaths); err != nil {
+ if size, err = chrootarchive.ApplyLayer(tempFolder, layerData); err != nil {
+ return
+ }
+ if err = hcsshim.ImportLayer(d.info, id, tempFolder, parentLayerPaths); err != nil {
+ return
+ }
return
}
+ var w hcsshim.LayerWriter
+ w, err = hcsshim.NewLayerWriter(d.info, id, parentLayerPaths)
+ if err != nil {
+ return
+ }
+
+ size, err = writeLayerFromTar(layerData, w)
+ if err != nil {
+ w.Close()
+ return
+ }
+ err = w.Close()
+ if err != nil {
+ return
+ }
return
}
@@ -564,42 +731,78 @@ func (d *Driver) setLayerChain(id string, chain []string) error {
return nil
}
-// DiffPath returns a directory that contains files needed to construct layer diff.
-func (d *Driver) DiffPath(id string) (path string, release func() error, err error) {
- id, err = d.resolveID(id)
- if err != nil {
- return
- }
+type fileGetCloserWithBackupPrivileges struct {
+ path string
+}
+
+func (fg *fileGetCloserWithBackupPrivileges) Get(filename string) (io.ReadCloser, error) {
+ var f *os.File
+ // Open the file while holding the Windows backup privilege. This ensures that the
+ // file can be opened even if the caller does not actually have access to it according
+ // to the security descriptor.
+ err := winio.RunWithPrivilege(winio.SeBackupPrivilege, func() error {
+ path := filepath.Join(fg.path, filename)
+ p, err := syscall.UTF16FromString(path)
+ if err != nil {
+ return err
+ }
+ h, err := syscall.CreateFile(&p[0], syscall.GENERIC_READ, syscall.FILE_SHARE_READ, nil, syscall.OPEN_EXISTING, syscall.FILE_FLAG_BACKUP_SEMANTICS, 0)
+ if err != nil {
+ return &os.PathError{Op: "open", Path: path, Err: err}
+ }
+ f = os.NewFile(uintptr(h), path)
+ return nil
+ })
+ return f, err
+}
+
+func (fg *fileGetCloserWithBackupPrivileges) Close() error {
+ return nil
+}
+
+type fileGetDestroyCloser struct {
+ storage.FileGetter
+ path string
+}
+
+func (f *fileGetDestroyCloser) Close() error {
+ // TODO: activate layers and release here?
+ return os.RemoveAll(f.path)
+}
- // Getting the layer paths must be done outside of the lock.
- layerChain, err := d.getLayerChain(id)
+// DiffGetter returns a FileGetCloser that can read files from the directory that
+// contains files for the layer differences. Used for direct access for tar-split.
+func (d *Driver) DiffGetter(id string) (graphdriver.FileGetCloser, error) {
+ id, err := d.resolveID(id)
if err != nil {
- return
+ return nil, err
}
- layerFolder := d.dir(id)
- tempFolder := layerFolder + "-" + strconv.FormatUint(uint64(random.Rand.Uint32()), 10)
- if err = os.MkdirAll(tempFolder, 0755); err != nil {
- logrus.Errorf("Could not create %s %s", tempFolder, err)
- return
- }
+ if hcsshim.IsTP4() {
+ // The export format for TP4 is different from the contents of the layer, so
+ // fall back to exporting the layer and getting file contents from there.
+ layerChain, err := d.getLayerChain(id)
+ if err != nil {
+ return nil, err
+ }
- defer func() {
+ var tempFolder string
+ tempFolder, err = ioutil.TempDir("", "hcs")
if err != nil {
- _, folderName := filepath.Split(tempFolder)
- if err2 := hcsshim.DestroyLayer(d.info, folderName); err2 != nil {
- logrus.Warnf("Couldn't clean-up tempFolder: %s %s", tempFolder, err2)
+ return nil, err
+ }
+ defer func() {
+ if err != nil {
+ os.RemoveAll(tempFolder)
}
+ }()
+
+ if err = hcsshim.ExportLayer(d.info, id, tempFolder, layerChain); err != nil {
+ return nil, err
}
- }()
- if err = hcsshim.ExportLayer(d.info, id, tempFolder, layerChain); err != nil {
- return
+ return &fileGetDestroyCloser{storage.NewPathFileGetter(tempFolder), tempFolder}, nil
}
- return tempFolder, func() error {
- // TODO: activate layers and release here?
- _, folderName := filepath.Split(tempFolder)
- return hcsshim.DestroyLayer(d.info, folderName)
- }, nil
+ return &fileGetCloserWithBackupPrivileges{d.dir(id)}, nil
}
diff --git a/components/engine/daemon/graphdriver/zfs/zfs.go b/components/engine/daemon/graphdriver/zfs/zfs.go
index 5cc10d2e26..28a94dd0b5 100644
--- a/components/engine/daemon/graphdriver/zfs/zfs.go
+++ b/components/engine/daemon/graphdriver/zfs/zfs.go
@@ -22,6 +22,12 @@ import (
"github.com/opencontainers/runc/libcontainer/label"
)
+type activeMount struct {
+ count int
+ path string
+ mounted bool
+}
+
type zfsOptions struct {
fsName string
mountPath string
@@ -103,6 +109,7 @@ func Init(base string, opt []string, uidMaps, gidMaps []idtools.IDMap) (graphdri
dataset: rootDataset,
options: options,
filesystemsCache: filesystemsCache,
+ active: make(map[string]*activeMount),
uidMaps: uidMaps,
gidMaps: gidMaps,
}
@@ -159,6 +166,7 @@ type Driver struct {
options zfsOptions
sync.Mutex // protects filesystem cache against concurrent access
filesystemsCache map[string]bool
+ active map[string]*activeMount
uidMaps []idtools.IDMap
gidMaps []idtools.IDMap
}
@@ -294,6 +302,17 @@ func (d *Driver) Remove(id string) error {
// Get returns the mountpoint for the given id after creating the target directories if necessary.
func (d *Driver) Get(id, mountLabel string) (string, error) {
+ d.Lock()
+ defer d.Unlock()
+
+ mnt := d.active[id]
+ if mnt != nil {
+ mnt.count++
+ return mnt.path, nil
+ }
+
+ mnt = &activeMount{count: 1}
+
mountpoint := d.mountPath(id)
filesystem := d.zfsPath(id)
options := label.FormatMountLabel("", mountLabel)
@@ -316,17 +335,43 @@ func (d *Driver) Get(id, mountLabel string) (string, error) {
if err := os.Chown(mountpoint, rootUID, rootGID); err != nil {
return "", fmt.Errorf("error modifying zfs mountpoint (%s) directory ownership: %v", mountpoint, err)
}
+ mnt.path = mountpoint
+ mnt.mounted = true
+ d.active[id] = mnt
return mountpoint, nil
}
// Put removes the existing mountpoint for the given id if it exists.
func (d *Driver) Put(id string) error {
- mountpoint := d.mountPath(id)
- logrus.Debugf(`[zfs] unmount("%s")`, mountpoint)
+ d.Lock()
+ defer d.Unlock()
- if err := mount.Unmount(mountpoint); err != nil {
- return fmt.Errorf("error unmounting to %s: %v", mountpoint, err)
+ mnt := d.active[id]
+ if mnt == nil {
+ logrus.Debugf("[zfs] Put on a non-mounted device %s", id)
+ // but it might be still here
+ if d.Exists(id) {
+ err := mount.Unmount(d.mountPath(id))
+ if err != nil {
+ logrus.Debugf("[zfs] Failed to unmount %s zfs fs: %v", id, err)
+ }
+ }
+ return nil
+ }
+
+ mnt.count--
+ if mnt.count > 0 {
+ return nil
+ }
+
+ defer delete(d.active, id)
+ if mnt.mounted {
+ logrus.Debugf(`[zfs] unmount("%s")`, mnt.path)
+
+ if err := mount.Unmount(mnt.path); err != nil {
+ return fmt.Errorf("error unmounting to %s: %v", mnt.path, err)
+ }
}
return nil
}
diff --git a/components/engine/daemon/image_delete.go b/components/engine/daemon/image_delete.go
index 0f32a903e6..7c6329a669 100644
--- a/components/engine/daemon/image_delete.go
+++ b/components/engine/daemon/image_delete.go
@@ -5,7 +5,7 @@ import (
"strings"
"github.com/docker/docker/container"
- derr "github.com/docker/docker/errors"
+ "github.com/docker/docker/errors"
"github.com/docker/docker/image"
"github.com/docker/docker/pkg/stringid"
"github.com/docker/docker/reference"
@@ -43,7 +43,7 @@ const (
//
// Hard Conflict:
// - a pull or build using the image.
-// - any descendent image.
+// - any descendant image.
// - any running container using the image.
//
// Soft Conflict:
@@ -82,7 +82,8 @@ func (daemon *Daemon) ImageDelete(imageRef string, force, prune bool) ([]types.I
// this image would remain "dangling" and since
// we really want to avoid that the client must
// explicitly force its removal.
- return nil, derr.ErrorCodeImgDelUsed.WithArgs(imageRef, stringid.TruncateID(container.ID), stringid.TruncateID(imgID.String()))
+ err := fmt.Errorf("conflict: unable to remove repository reference %q (must force) - container %s is using its referenced image %s", imageRef, stringid.TruncateID(container.ID), stringid.TruncateID(imgID.String()))
+ return nil, errors.NewRequestConflictError(err)
}
}
@@ -313,7 +314,7 @@ func (daemon *Daemon) imageDeleteHelper(imgID image.ID, records *[]types.ImageDe
// image or any stopped container using the image. If ignoreSoftConflicts is
// true, this function will not check for soft conflict conditions.
func (daemon *Daemon) checkImageDeleteConflict(imgID image.ID, mask conflictType) *imageDeleteConflict {
- // Check if the image has any descendent images.
+ // Check if the image has any descendant images.
if mask&conflictDependentChild != 0 && len(daemon.imageStore.Children(imgID)) > 0 {
return &imageDeleteConflict{
hard: true,
diff --git a/components/engine/daemon/import.go b/components/engine/daemon/import.go
index c04e8a38f2..4961a30fd9 100644
--- a/components/engine/daemon/import.go
+++ b/components/engine/daemon/import.go
@@ -11,6 +11,7 @@ import (
"github.com/docker/docker/dockerversion"
"github.com/docker/docker/image"
"github.com/docker/docker/layer"
+ "github.com/docker/docker/pkg/archive"
"github.com/docker/docker/pkg/httputils"
"github.com/docker/docker/pkg/progress"
"github.com/docker/docker/pkg/streamformatter"
@@ -24,13 +25,13 @@ import (
// the repo and tag arguments, respectively.
func (daemon *Daemon) ImportImage(src string, newRef reference.Named, msg string, inConfig io.ReadCloser, outStream io.Writer, config *container.Config) error {
var (
- sf = streamformatter.NewJSONStreamFormatter()
- archive io.ReadCloser
- resp *http.Response
+ sf = streamformatter.NewJSONStreamFormatter()
+ rc io.ReadCloser
+ resp *http.Response
)
if src == "-" {
- archive = inConfig
+ rc = inConfig
} else {
inConfig.Close()
u, err := url.Parse(src)
@@ -48,15 +49,20 @@ func (daemon *Daemon) ImportImage(src string, newRef reference.Named, msg string
return err
}
progressOutput := sf.NewProgressOutput(outStream, true)
- archive = progress.NewProgressReader(resp.Body, progressOutput, resp.ContentLength, "", "Importing")
+ rc = progress.NewProgressReader(resp.Body, progressOutput, resp.ContentLength, "", "Importing")
}
- defer archive.Close()
+ defer rc.Close()
if len(msg) == 0 {
msg = "Imported from " + src
}
+
+ inflatedLayerData, err := archive.DecompressStream(rc)
+ if err != nil {
+ return err
+ }
// TODO: support windows baselayer?
- l, err := daemon.layerStore.Register(archive, "")
+ l, err := daemon.layerStore.Register(inflatedLayerData, "")
if err != nil {
return err
}
diff --git a/components/engine/daemon/info.go b/components/engine/daemon/info.go
index 20d8356d44..1ee367add1 100644
--- a/components/engine/daemon/info.go
+++ b/components/engine/daemon/info.go
@@ -20,6 +20,7 @@ import (
"github.com/docker/docker/utils"
"github.com/docker/docker/volume/drivers"
"github.com/docker/engine-api/types"
+ "github.com/docker/go-connections/sockets"
)
// SystemInfo returns information about the host server the daemon is running on.
@@ -74,14 +75,15 @@ func (daemon *Daemon) SystemInfo() (*types.Info, error) {
DriverStatus: daemon.layerStore.DriverStatus(),
Plugins: daemon.showPluginsInfo(),
IPv4Forwarding: !sysInfo.IPv4ForwardingDisabled,
- BridgeNfIptables: !sysInfo.BridgeNfCallIptablesDisabled,
- BridgeNfIP6tables: !sysInfo.BridgeNfCallIP6tablesDisabled,
+ BridgeNfIptables: !sysInfo.BridgeNFCallIPTablesDisabled,
+ BridgeNfIP6tables: !sysInfo.BridgeNFCallIP6TablesDisabled,
Debug: utils.IsDebugEnabled(),
NFd: fileutils.GetTotalUsedFds(),
NGoroutines: runtime.NumGoroutine(),
SystemTime: time.Now().Format(time.RFC3339Nano),
ExecutionDriver: daemon.ExecutionDriver().Name(),
LoggingDriver: daemon.defaultLogConfig.Type,
+ CgroupDriver: daemon.getCgroupDriver(),
NEventsListener: daemon.EventsService.SubscribersCount(),
KernelVersion: kernelVersion,
OperatingSystem: operatingSystem,
@@ -97,9 +99,9 @@ func (daemon *Daemon) SystemInfo() (*types.Info, error) {
ServerVersion: dockerversion.Version,
ClusterStore: daemon.configStore.ClusterStore,
ClusterAdvertise: daemon.configStore.ClusterAdvertise,
- HTTPProxy: getProxyEnv("http_proxy"),
- HTTPSProxy: getProxyEnv("https_proxy"),
- NoProxy: getProxyEnv("no_proxy"),
+ HTTPProxy: sockets.GetProxyEnv("http_proxy"),
+ HTTPSProxy: sockets.GetProxyEnv("https_proxy"),
+ NoProxy: sockets.GetProxyEnv("no_proxy"),
}
// TODO Windows. Refactor this more once sysinfo is refactored into
@@ -109,6 +111,7 @@ func (daemon *Daemon) SystemInfo() (*types.Info, error) {
if runtime.GOOS != "windows" {
v.MemoryLimit = sysInfo.MemoryLimit
v.SwapLimit = sysInfo.SwapLimit
+ v.KernelMemory = sysInfo.KernelMemory
v.OomKillDisable = sysInfo.OomKillDisable
v.CPUCfsPeriod = sysInfo.CPUCfsPeriod
v.CPUCfsQuota = sysInfo.CPUCfsQuota
diff --git a/components/engine/daemon/inspect.go b/components/engine/daemon/inspect.go
index feb7de28f1..511454f9c7 100644
--- a/components/engine/daemon/inspect.go
+++ b/components/engine/daemon/inspect.go
@@ -4,8 +4,8 @@ import (
"fmt"
"time"
+ "github.com/docker/docker/api/types/backend"
"github.com/docker/docker/container"
- "github.com/docker/docker/daemon/exec"
"github.com/docker/docker/daemon/network"
"github.com/docker/docker/pkg/version"
"github.com/docker/engine-api/types"
@@ -175,12 +175,26 @@ func (daemon *Daemon) getInspectData(container *container.Container, size bool)
// ContainerExecInspect returns low-level information about the exec
// command. An error is returned if the exec cannot be found.
-func (daemon *Daemon) ContainerExecInspect(id string) (*exec.Config, error) {
- eConfig, err := daemon.getExecConfig(id)
+func (daemon *Daemon) ContainerExecInspect(id string) (*backend.ExecInspect, error) {
+ e, err := daemon.getExecConfig(id)
if err != nil {
return nil, err
}
- return eConfig, nil
+
+ pc := inspectExecProcessConfig(e)
+
+ return &backend.ExecInspect{
+ ID: e.ID,
+ Running: e.Running,
+ ExitCode: e.ExitCode,
+ ProcessConfig: pc,
+ OpenStdin: e.OpenStdin,
+ OpenStdout: e.OpenStdout,
+ OpenStderr: e.OpenStderr,
+ CanRemove: e.CanRemove,
+ ContainerID: e.ContainerID,
+ DetachKeys: e.DetachKeys,
+ }, nil
}
// VolumeInspect looks up a volume by name. An error is returned if
diff --git a/components/engine/daemon/inspect_unix.go b/components/engine/daemon/inspect_unix.go
index b9321f34c3..bb224c8796 100644
--- a/components/engine/daemon/inspect_unix.go
+++ b/components/engine/daemon/inspect_unix.go
@@ -3,7 +3,9 @@
package daemon
import (
+ "github.com/docker/docker/api/types/backend"
"github.com/docker/docker/container"
+ "github.com/docker/docker/daemon/exec"
"github.com/docker/engine-api/types"
"github.com/docker/engine-api/types/versions/v1p19"
)
@@ -77,3 +79,13 @@ func addMountPoints(container *container.Container) []types.MountPoint {
}
return mountPoints
}
+
+func inspectExecProcessConfig(e *exec.Config) *backend.ExecProcessConfig {
+ return &backend.ExecProcessConfig{
+ Tty: e.ProcessConfig.Tty,
+ Entrypoint: e.ProcessConfig.Entrypoint,
+ Arguments: e.ProcessConfig.Arguments,
+ Privileged: &e.ProcessConfig.Privileged,
+ User: e.ProcessConfig.User,
+ }
+}
diff --git a/components/engine/daemon/inspect_windows.go b/components/engine/daemon/inspect_windows.go
index e42a61dadc..f20571d052 100644
--- a/components/engine/daemon/inspect_windows.go
+++ b/components/engine/daemon/inspect_windows.go
@@ -1,7 +1,9 @@
package daemon
import (
+ "github.com/docker/docker/api/types/backend"
"github.com/docker/docker/container"
+ "github.com/docker/docker/daemon/exec"
"github.com/docker/engine-api/types"
)
@@ -28,3 +30,11 @@ func addMountPoints(container *container.Container) []types.MountPoint {
func (daemon *Daemon) containerInspectPre120(name string) (*types.ContainerJSON, error) {
return daemon.containerInspectCurrent(name, false)
}
+
+func inspectExecProcessConfig(e *exec.Config) *backend.ExecProcessConfig {
+ return &backend.ExecProcessConfig{
+ Tty: e.ProcessConfig.Tty,
+ Entrypoint: e.ProcessConfig.Entrypoint,
+ Arguments: e.ProcessConfig.Arguments,
+ }
+}
diff --git a/components/engine/daemon/kill.go b/components/engine/daemon/kill.go
index 2bb695699c..4d29346d2f 100644
--- a/components/engine/daemon/kill.go
+++ b/components/engine/daemon/kill.go
@@ -8,10 +8,25 @@ import (
"github.com/Sirupsen/logrus"
"github.com/docker/docker/container"
- derr "github.com/docker/docker/errors"
"github.com/docker/docker/pkg/signal"
)
+type errNoSuchProcess struct {
+ pid int
+ signal int
+}
+
+func (e errNoSuchProcess) Error() string {
+ return fmt.Sprintf("Cannot kill process (pid=%d) with signal %d: no such process.", e.pid, e.signal)
+}
+
+// isErrNoSuchProcess returns true if the error
+// is an instance of errNoSuchProcess.
+func isErrNoSuchProcess(err error) bool {
+ _, ok := err.(errNoSuchProcess)
+ return ok
+}
+
// ContainerKill send signal to the container
// If no signal is given (sig 0), then Kill with SIGKILL and wait
// for the container to exit.
@@ -45,11 +60,11 @@ func (daemon *Daemon) killWithSignal(container *container.Container, sig int) er
// We could unpause the container for them rather than returning this error
if container.Paused {
- return derr.ErrorCodeUnpauseContainer.WithArgs(container.ID)
+ return fmt.Errorf("Container %s is paused. Unpause the container before stopping", container.ID)
}
if !container.Running {
- return derr.ErrorCodeNotRunning.WithArgs(container.ID)
+ return errNotRunning{container.ID}
}
container.ExitOnNext()
@@ -62,7 +77,7 @@ func (daemon *Daemon) killWithSignal(container *container.Container, sig int) er
}
if err := daemon.kill(container, sig); err != nil {
- return derr.ErrorCodeCantKill.WithArgs(container.ID, err)
+ return fmt.Errorf("Cannot kill container %s: %s", container.ID, err)
}
attributes := map[string]string{
@@ -75,7 +90,7 @@ func (daemon *Daemon) killWithSignal(container *container.Container, sig int) er
// Kill forcefully terminates a container.
func (daemon *Daemon) Kill(container *container.Container) error {
if !container.IsRunning() {
- return derr.ErrorCodeNotRunning.WithArgs(container.ID)
+ return errNotRunning{container.ID}
}
// 1. Send SIGKILL
@@ -90,6 +105,9 @@ func (daemon *Daemon) Kill(container *container.Container) error {
// So, instead we'll give it up to 2 more seconds to complete and if
// by that time the container is still running, then the error
// we got is probably valid and so we return it to the caller.
+ if isErrNoSuchProcess(err) {
+ return nil
+ }
if container.IsRunning() {
container.WaitStop(2 * time.Second)
@@ -101,6 +119,9 @@ func (daemon *Daemon) Kill(container *container.Container) error {
// 2. Wait for the process to die, in last resort, try to kill the process directly
if err := killProcessDirectly(container); err != nil {
+ if isErrNoSuchProcess(err) {
+ return nil
+ }
return err
}
@@ -112,8 +133,9 @@ func (daemon *Daemon) Kill(container *container.Container) error {
func (daemon *Daemon) killPossiblyDeadProcess(container *container.Container, sig int) error {
err := daemon.killWithSignal(container, sig)
if err == syscall.ESRCH {
- logrus.Debugf("Cannot kill process (pid=%d) with signal %d: no such process.", container.GetPID(), sig)
- return nil
+ e := errNoSuchProcess{container.GetPID(), sig}
+ logrus.Debug(e)
+ return e
}
return err
}
diff --git a/components/engine/daemon/list.go b/components/engine/daemon/list.go
index 472aa8894f..e84353419f 100644
--- a/components/engine/daemon/list.go
+++ b/components/engine/daemon/list.go
@@ -9,6 +9,7 @@ import (
"github.com/Sirupsen/logrus"
"github.com/docker/docker/container"
"github.com/docker/docker/image"
+ "github.com/docker/docker/volume"
"github.com/docker/engine-api/types"
"github.com/docker/engine-api/types/filters"
networktypes "github.com/docker/engine-api/types/network"
@@ -225,8 +226,24 @@ func (daemon *Daemon) foldFilter(config *types.ContainerListOptions) (*listConte
// includeContainerInList decides whether a containers should be include in the output or not based in the filter.
// It also decides if the iteration should be stopped or not.
func includeContainerInList(container *container.Container, ctx *listContext) iterationAction {
+ // Do not include container if it's in the list before the filter container.
+ // Set the filter container to nil to include the rest of containers after this one.
+ if ctx.beforeFilter != nil {
+ if container.ID == ctx.beforeFilter.ID {
+ ctx.beforeFilter = nil
+ }
+ return excludeContainer
+ }
+
+ // Stop iteration when the container arrives to the filter container
+ if ctx.sinceFilter != nil {
+ if container.ID == ctx.sinceFilter.ID {
+ return stopIteration
+ }
+ }
+
// Do not include container if it's stopped and we're not filters
- // FIXME remove the ctx.beforContainer part of the condition for 1.12 as --since and --before are deprecated
+ // FIXME remove the ctx.beforContainer and ctx.sinceContainer part of the condition for 1.12 as --since and --before are deprecated
if !container.Running && !ctx.All && ctx.Limit <= 0 && ctx.beforeContainer == nil && ctx.sinceContainer == nil {
return excludeContainer
}
@@ -246,7 +263,7 @@ func includeContainerInList(container *container.Container, ctx *listContext) it
return excludeContainer
}
- // Do not include container if the isolation mode doesn't match
+ // Do not include container if isolation doesn't match
if excludeContainer == excludeByIsolation(container, ctx) {
return excludeContainer
}
@@ -266,22 +283,6 @@ func includeContainerInList(container *container.Container, ctx *listContext) it
}
}
- // Do not include container if it's in the list before the filter container.
- // Set the filter container to nil to include the rest of containers after this one.
- if ctx.beforeFilter != nil {
- if container.ID == ctx.beforeFilter.ID {
- ctx.beforeFilter = nil
- }
- return excludeContainer
- }
-
- // Stop iteration when the container arrives to the filter container
- if ctx.sinceFilter != nil {
- if container.ID == ctx.sinceFilter.ID {
- return stopIteration
- }
- }
-
// Stop iteration when the index is over the limit
if ctx.Limit > 0 && ctx.idx == ctx.Limit {
return stopIteration
@@ -306,6 +307,27 @@ func includeContainerInList(container *container.Container, ctx *listContext) it
return excludeContainer
}
+ if ctx.filters.Include("volume") {
+ volumesByName := make(map[string]*volume.MountPoint)
+ for _, m := range container.MountPoints {
+ volumesByName[m.Name] = m
+ }
+
+ volumeExist := fmt.Errorf("volume mounted in container")
+ err := ctx.filters.WalkValues("volume", func(value string) error {
+ if _, exist := container.MountPoints[value]; exist {
+ return volumeExist
+ }
+ if _, exist := volumesByName[value]; exist {
+ return volumeExist
+ }
+ return nil
+ })
+ if err != volumeExist {
+ return excludeContainer
+ }
+ }
+
if ctx.ancestorFilter {
if len(ctx.images) == 0 {
return excludeContainer
@@ -419,6 +441,7 @@ func (daemon *Daemon) transformContainer(container *container.Container, ctx *li
newC.SizeRootFs = sizeRootFs
}
newC.Labels = container.Config.Labels
+ newC.Mounts = addMountPoints(container)
return newC, nil
}
diff --git a/components/engine/daemon/logdrivers_linux.go b/components/engine/daemon/logdrivers_linux.go
index 0abc6269de..89fe49a858 100644
--- a/components/engine/daemon/logdrivers_linux.go
+++ b/components/engine/daemon/logdrivers_linux.go
@@ -5,6 +5,7 @@ import (
// therefore they register themselves to the logdriver factory.
_ "github.com/docker/docker/daemon/logger/awslogs"
_ "github.com/docker/docker/daemon/logger/fluentd"
+ _ "github.com/docker/docker/daemon/logger/gcplogs"
_ "github.com/docker/docker/daemon/logger/gelf"
_ "github.com/docker/docker/daemon/logger/journald"
_ "github.com/docker/docker/daemon/logger/jsonfilelog"
diff --git a/components/engine/daemon/logdrivers_windows.go b/components/engine/daemon/logdrivers_windows.go
index d3710ec174..129b06650b 100644
--- a/components/engine/daemon/logdrivers_windows.go
+++ b/components/engine/daemon/logdrivers_windows.go
@@ -4,6 +4,7 @@ import (
// Importing packages here only to make sure their init gets called and
// therefore they register themselves to the logdriver factory.
_ "github.com/docker/docker/daemon/logger/awslogs"
+ _ "github.com/docker/docker/daemon/logger/etwlogs"
_ "github.com/docker/docker/daemon/logger/jsonfilelog"
_ "github.com/docker/docker/daemon/logger/splunk"
)
diff --git a/components/engine/daemon/logger/etwlogs/etwlogs_windows.go b/components/engine/daemon/logger/etwlogs/etwlogs_windows.go
new file mode 100644
index 0000000000..de128a2e93
--- /dev/null
+++ b/components/engine/daemon/logger/etwlogs/etwlogs_windows.go
@@ -0,0 +1,183 @@
+// Package etwlogs provides a log driver for forwarding container logs
+// as ETW events.(ETW stands for Event Tracing for Windows)
+// A client can then create an ETW listener to listen for events that are sent
+// by the ETW provider that we register, using the provider's GUID "a3693192-9ed6-46d2-a981-f8226c8363bd".
+// Here is an example of how to do this using the logman utility:
+// 1. logman start -ets DockerContainerLogs -p {a3693192-9ed6-46d2-a981-f8226c8363bd} 0 0 -o trace.etl
+// 2. Run container(s) and generate log messages
+// 3. logman stop -ets DockerContainerLogs
+// 4. You can then convert the etl log file to XML using: tracerpt -y trace.etl
+//
+// Each container log message generates a ETW event that also contains:
+// the container name and ID, the timestamp, and the stream type.
+package etwlogs
+
+import (
+ "errors"
+ "fmt"
+ "sync"
+ "syscall"
+ "unsafe"
+
+ "github.com/Sirupsen/logrus"
+ "github.com/docker/docker/daemon/logger"
+)
+
+type etwLogs struct {
+ containerName string
+ imageName string
+ containerID string
+ imageID string
+}
+
+const (
+ name = "etwlogs"
+ win32CallSuccess = 0
+)
+
+var win32Lib *syscall.DLL
+var providerHandle syscall.Handle
+var refCount int
+var mu sync.Mutex
+
+func init() {
+ providerHandle = syscall.InvalidHandle
+ if err := logger.RegisterLogDriver(name, New); err != nil {
+ logrus.Fatal(err)
+ }
+}
+
+// New creates a new etwLogs logger for the given container and registers the EWT provider.
+func New(ctx logger.Context) (logger.Logger, error) {
+ if err := registerETWProvider(); err != nil {
+ return nil, err
+ }
+ logrus.Debugf("logging driver etwLogs configured for container: %s.", ctx.ContainerID)
+
+ return &etwLogs{
+ containerName: fixContainerName(ctx.ContainerName),
+ imageName: ctx.ContainerImageName,
+ containerID: ctx.ContainerID,
+ imageID: ctx.ContainerImageID,
+ }, nil
+}
+
+// Log logs the message to the ETW stream.
+func (etwLogger *etwLogs) Log(msg *logger.Message) error {
+ if providerHandle == syscall.InvalidHandle {
+ // This should never be hit, if it is, it indicates a programming error.
+ errorMessage := "ETWLogs cannot log the message, because the event provider has not been registered."
+ logrus.Error(errorMessage)
+ return errors.New(errorMessage)
+ }
+ return callEventWriteString(createLogMessage(etwLogger, msg))
+}
+
+// Close closes the logger by unregistering the ETW provider.
+func (etwLogger *etwLogs) Close() error {
+ unregisterETWProvider()
+ return nil
+}
+
+func (etwLogger *etwLogs) Name() string {
+ return name
+}
+
+func createLogMessage(etwLogger *etwLogs, msg *logger.Message) string {
+ return fmt.Sprintf("container_name: %s, image_name: %s, container_id: %s, image_id: %s, source: %s, log: %s",
+ etwLogger.containerName,
+ etwLogger.imageName,
+ etwLogger.containerID,
+ etwLogger.imageID,
+ msg.Source,
+ msg.Line)
+}
+
+// fixContainerName removes the initial '/' from the container name.
+func fixContainerName(cntName string) string {
+ if len(cntName) > 0 && cntName[0] == '/' {
+ cntName = cntName[1:]
+ }
+ return cntName
+}
+
+func registerETWProvider() error {
+ mu.Lock()
+ defer mu.Unlock()
+ if refCount == 0 {
+ var err error
+ if win32Lib, err = syscall.LoadDLL("Advapi32.dll"); err != nil {
+ return err
+ }
+ if err = callEventRegister(); err != nil {
+ win32Lib.Release()
+ win32Lib = nil
+ return err
+ }
+ }
+
+ refCount++
+ return nil
+}
+
+func unregisterETWProvider() {
+ mu.Lock()
+ defer mu.Unlock()
+ if refCount == 1 {
+ if callEventUnregister() {
+ refCount--
+ providerHandle = syscall.InvalidHandle
+ win32Lib.Release()
+ win32Lib = nil
+ }
+ // Not returning an error if EventUnregister fails, because etwLogs will continue to work
+ } else {
+ refCount--
+ }
+}
+
+func callEventRegister() error {
+ proc, err := win32Lib.FindProc("EventRegister")
+ if err != nil {
+ return err
+ }
+ // The provider's GUID is {a3693192-9ed6-46d2-a981-f8226c8363bd}
+ guid := syscall.GUID{
+ 0xa3693192, 0x9ed6, 0x46d2,
+ [8]byte{0xa9, 0x81, 0xf8, 0x22, 0x6c, 0x83, 0x63, 0xbd},
+ }
+
+ ret, _, _ := proc.Call(uintptr(unsafe.Pointer(&guid)), 0, 0, uintptr(unsafe.Pointer(&providerHandle)))
+ if ret != win32CallSuccess {
+ errorMessage := fmt.Sprintf("Failed to register ETW provider. Error: %d", ret)
+ logrus.Error(errorMessage)
+ return errors.New(errorMessage)
+ }
+ return nil
+}
+
+func callEventWriteString(message string) error {
+ proc, err := win32Lib.FindProc("EventWriteString")
+ if err != nil {
+ return err
+ }
+ ret, _, _ := proc.Call(uintptr(providerHandle), 0, 0, uintptr(unsafe.Pointer(syscall.StringToUTF16Ptr(message))))
+ if ret != win32CallSuccess {
+ errorMessage := fmt.Sprintf("ETWLogs provider failed to log message. Error: %d", ret)
+ logrus.Error(errorMessage)
+ return errors.New(errorMessage)
+ }
+ return nil
+}
+
+func callEventUnregister() bool {
+ proc, err := win32Lib.FindProc("EventUnregister")
+ if err != nil {
+ return false
+ }
+ ret, _, _ := proc.Call(uintptr(providerHandle))
+ if ret != win32CallSuccess {
+ return false
+ }
+ return true
+}
diff --git a/components/engine/daemon/logger/gcplogs/gcplogging.go b/components/engine/daemon/logger/gcplogs/gcplogging.go
new file mode 100644
index 0000000000..b9b8af5871
--- /dev/null
+++ b/components/engine/daemon/logger/gcplogs/gcplogging.go
@@ -0,0 +1,181 @@
+package gcplogs
+
+import (
+ "fmt"
+ "sync/atomic"
+ "time"
+
+ "github.com/docker/docker/daemon/logger"
+
+ "github.com/Sirupsen/logrus"
+ "golang.org/x/net/context"
+ "google.golang.org/cloud/compute/metadata"
+ "google.golang.org/cloud/logging"
+)
+
+const (
+ name = "gcplogs"
+
+ projectOptKey = "gcp-project"
+ logLabelsKey = "labels"
+ logEnvKey = "env"
+ logCmdKey = "gcp-log-cmd"
+)
+
+var (
+ // The number of logs the gcplogs driver has dropped.
+ droppedLogs uint64
+
+ onGCE = metadata.OnGCE()
+
+ // instance metadata populated from the metadata server if available
+ projectID string
+ zone string
+ instanceName string
+ instanceID string
+)
+
+func init() {
+ if onGCE {
+ // These will fail on instances if the metadata service is
+ // down or the client is compiled with an API version that
+ // has been removed. Since these are not vital, let's ignore
+ // them and make their fields in the dockeLogEntry ,omitempty
+ projectID, _ = metadata.ProjectID()
+ zone, _ = metadata.Zone()
+ instanceName, _ = metadata.InstanceName()
+ instanceID, _ = metadata.InstanceID()
+ }
+
+ if err := logger.RegisterLogDriver(name, New); err != nil {
+ logrus.Fatal(err)
+ }
+
+ if err := logger.RegisterLogOptValidator(name, ValidateLogOpts); err != nil {
+ logrus.Fatal(err)
+ }
+}
+
+type gcplogs struct {
+ client *logging.Client
+ instance *instanceInfo
+ container *containerInfo
+}
+
+type dockerLogEntry struct {
+ Instance *instanceInfo `json:"instance,omitempty"`
+ Container *containerInfo `json:"container,omitempty"`
+ Data string `json:"data,omitempty"`
+}
+
+type instanceInfo struct {
+ Zone string `json:"zone,omitempty"`
+ Name string `json:"name,omitempty"`
+ ID string `json:"id,omitempty"`
+}
+
+type containerInfo struct {
+ Name string `json:"name,omitempty"`
+ ID string `json:"id,omitempty"`
+ ImageName string `json:"imageName,omitempty"`
+ ImageID string `json:"imageId,omitempty"`
+ Created time.Time `json:"created,omitempty"`
+ Command string `json:"command,omitempty"`
+ Metadata map[string]string `json:"metadata,omitempty"`
+}
+
+// New creates a new logger that logs to Google Cloud Logging using the application
+// default credentials.
+//
+// See https://developers.google.com/identity/protocols/application-default-credentials
+func New(ctx logger.Context) (logger.Logger, error) {
+
+ var project string
+ if projectID != "" {
+ project = projectID
+ }
+ if projectID, found := ctx.Config[projectOptKey]; found {
+ project = projectID
+ }
+ if project == "" {
+ return nil, fmt.Errorf("No project was specified and couldn't read project from the meatadata server. Please specify a project")
+ }
+
+ c, err := logging.NewClient(context.Background(), project, "gcplogs-docker-driver")
+ if err != nil {
+ return nil, err
+ }
+
+ if err := c.Ping(); err != nil {
+ return nil, fmt.Errorf("unable to connect or authenticate with Google Cloud Logging: %v", err)
+ }
+
+ l := &gcplogs{
+ client: c,
+ container: &containerInfo{
+ Name: ctx.ContainerName,
+ ID: ctx.ContainerID,
+ ImageName: ctx.ContainerImageName,
+ ImageID: ctx.ContainerImageID,
+ Created: ctx.ContainerCreated,
+ Metadata: ctx.ExtraAttributes(nil),
+ },
+ }
+
+ if ctx.Config[logCmdKey] == "true" {
+ l.container.Command = ctx.Command()
+ }
+
+ if onGCE {
+ l.instance = &instanceInfo{
+ Zone: zone,
+ Name: instanceName,
+ ID: instanceID,
+ }
+ }
+
+ // The logger "overflows" at a rate of 10,000 logs per second and this
+ // overflow func is called. We want to surface the error to the user
+ // without overly spamming /var/log/docker.log so we log the first time
+ // we overflow and every 1000th time after.
+ c.Overflow = func(_ *logging.Client, _ logging.Entry) error {
+ if i := atomic.AddUint64(&droppedLogs, 1); i%1000 == 1 {
+ logrus.Errorf("gcplogs driver has dropped %v logs", i)
+ }
+ return nil
+ }
+
+ return l, nil
+}
+
+// ValidateLogOpts validates the opts passed to the gcplogs driver. Currently, the gcplogs
+// driver doesn't take any arguments.
+func ValidateLogOpts(cfg map[string]string) error {
+ for k := range cfg {
+ switch k {
+ case projectOptKey, logLabelsKey, logEnvKey, logCmdKey:
+ default:
+ return fmt.Errorf("%q is not a valid option for the gcplogs driver", k)
+ }
+ }
+ return nil
+}
+
+func (l *gcplogs) Log(m *logger.Message) error {
+ return l.client.Log(logging.Entry{
+ Time: m.Timestamp,
+ Payload: &dockerLogEntry{
+ Instance: l.instance,
+ Container: l.container,
+ Data: string(m.Line),
+ },
+ })
+}
+
+func (l *gcplogs) Close() error {
+ return l.client.Flush()
+}
+
+func (l *gcplogs) Name() string {
+ return name
+}
diff --git a/components/engine/daemon/logger/journald/read.go b/components/engine/daemon/logger/journald/read.go
index 80c1fbda7a..8d94c302fb 100644
--- a/components/engine/daemon/logger/journald/read.go
+++ b/components/engine/daemon/logger/journald/read.go
@@ -2,7 +2,6 @@
package journald
-// #cgo pkg-config: libsystemd-journal
// #include
// #include
// #include
diff --git a/components/engine/daemon/logger/journald/read_native.go b/components/engine/daemon/logger/journald/read_native.go
new file mode 100644
index 0000000000..bba6de55be
--- /dev/null
+++ b/components/engine/daemon/logger/journald/read_native.go
@@ -0,0 +1,6 @@
+// +build linux,cgo,!static_build,journald,!journald_compat
+
+package journald
+
+// #cgo pkg-config: libsystemd
+import "C"
diff --git a/components/engine/daemon/logger/journald/read_native_compat.go b/components/engine/daemon/logger/journald/read_native_compat.go
new file mode 100644
index 0000000000..3f7a43c59e
--- /dev/null
+++ b/components/engine/daemon/logger/journald/read_native_compat.go
@@ -0,0 +1,6 @@
+// +build linux,cgo,!static_build,journald,journald_compat
+
+package journald
+
+// #cgo pkg-config: libsystemd-journal
+import "C"
diff --git a/components/engine/daemon/logger/jsonfilelog/jsonfilelog.go b/components/engine/daemon/logger/jsonfilelog/jsonfilelog.go
index 86baa316b9..9faa4e02db 100644
--- a/components/engine/daemon/logger/jsonfilelog/jsonfilelog.go
+++ b/components/engine/daemon/logger/jsonfilelog/jsonfilelog.go
@@ -25,7 +25,6 @@ type JSONFileLogger struct {
buf *bytes.Buffer
writer *loggerutils.RotateFileWriter
mu sync.Mutex
- ctx logger.Context
readers map[*logger.LogWatcher]struct{} // stores the active log followers
extra []byte // json-encoded extra attributes
}
@@ -91,7 +90,6 @@ func (l *JSONFileLogger) Log(msg *logger.Message) error {
return err
}
l.mu.Lock()
- defer l.mu.Unlock()
err = (&jsonlog.JSONLogs{
Log: append(msg.Line, '\n'),
Stream: msg.Source,
@@ -99,12 +97,14 @@ func (l *JSONFileLogger) Log(msg *logger.Message) error {
RawAttrs: l.extra,
}).MarshalJSONBuf(l.buf)
if err != nil {
+ l.mu.Unlock()
return err
}
l.buf.WriteByte('\n')
_, err = l.writer.Write(l.buf.Bytes())
l.buf.Reset()
+ l.mu.Unlock()
return err
}
diff --git a/components/engine/daemon/logger/jsonfilelog/jsonfilelog_test.go b/components/engine/daemon/logger/jsonfilelog/jsonfilelog_test.go
index 162c685b24..ef840531a1 100644
--- a/components/engine/daemon/logger/jsonfilelog/jsonfilelog_test.go
+++ b/components/engine/daemon/logger/jsonfilelog/jsonfilelog_test.go
@@ -199,3 +199,50 @@ func TestJSONFileLoggerWithLabelsEnv(t *testing.T) {
t.Fatalf("Wrong log attrs: %q, expected %q", extra, expected)
}
}
+
+func BenchmarkJSONFileLoggerWithReader(b *testing.B) {
+ b.StopTimer()
+ b.ResetTimer()
+ cid := "a7317399f3f857173c6179d44823594f8294678dea9999662e5c625b5a1c7657"
+ dir, err := ioutil.TempDir("", "json-logger-bench")
+ if err != nil {
+ b.Fatal(err)
+ }
+ defer os.RemoveAll(dir)
+
+ l, err := New(logger.Context{
+ ContainerID: cid,
+ LogPath: filepath.Join(dir, "container.log"),
+ })
+ if err != nil {
+ b.Fatal(err)
+ }
+ defer l.Close()
+ msg := &logger.Message{ContainerID: cid, Line: []byte("line"), Source: "src1"}
+ jsonlog, err := (&jsonlog.JSONLog{Log: string(msg.Line) + "\n", Stream: msg.Source, Created: msg.Timestamp}).MarshalJSON()
+ if err != nil {
+ b.Fatal(err)
+ }
+ b.SetBytes(int64(len(jsonlog)+1) * 30)
+
+ b.StartTimer()
+
+ go func() {
+ for i := 0; i < b.N; i++ {
+ for j := 0; j < 30; j++ {
+ l.Log(msg)
+ }
+ }
+ l.Close()
+ }()
+
+ lw := l.(logger.LogReader).ReadLogs(logger.ReadConfig{Follow: true})
+ watchClose := lw.WatchClose()
+ for {
+ select {
+ case <-lw.Msg:
+ case <-watchClose:
+ return
+ }
+ }
+}
diff --git a/components/engine/daemon/logger/loggerutils/rotatefilewriter.go b/components/engine/daemon/logger/loggerutils/rotatefilewriter.go
index 0d2553a13f..de7112896c 100644
--- a/components/engine/daemon/logger/loggerutils/rotatefilewriter.go
+++ b/components/engine/daemon/logger/loggerutils/rotatefilewriter.go
@@ -13,6 +13,7 @@ type RotateFileWriter struct {
f *os.File // store for closing
mu sync.Mutex
capacity int64 //maximum size of each file
+ currentSize int64 // current size of the latest file
maxFiles int //maximum number of files
notifyRotate *pubsub.Publisher
}
@@ -21,12 +22,18 @@ type RotateFileWriter struct {
func NewRotateFileWriter(logPath string, capacity int64, maxFiles int) (*RotateFileWriter, error) {
log, err := os.OpenFile(logPath, os.O_WRONLY|os.O_APPEND|os.O_CREATE, 0640)
if err != nil {
- return &RotateFileWriter{}, err
+ return nil, err
+ }
+
+ size, err := log.Seek(0, os.SEEK_END)
+ if err != nil {
+ return nil, err
}
return &RotateFileWriter{
f: log,
capacity: capacity,
+ currentSize: size,
maxFiles: maxFiles,
notifyRotate: pubsub.NewPublisher(0, 1),
}, nil
@@ -35,12 +42,17 @@ func NewRotateFileWriter(logPath string, capacity int64, maxFiles int) (*RotateF
//WriteLog write log message to File
func (w *RotateFileWriter) Write(message []byte) (int, error) {
w.mu.Lock()
- defer w.mu.Unlock()
if err := w.checkCapacityAndRotate(); err != nil {
+ w.mu.Unlock()
return -1, err
}
- return w.f.Write(message)
+ n, err := w.f.Write(message)
+ if err == nil {
+ w.currentSize += int64(n)
+ }
+ w.mu.Unlock()
+ return n, err
}
func (w *RotateFileWriter) checkCapacityAndRotate() error {
@@ -48,12 +60,7 @@ func (w *RotateFileWriter) checkCapacityAndRotate() error {
return nil
}
- meta, err := w.f.Stat()
- if err != nil {
- return err
- }
-
- if meta.Size() >= w.capacity {
+ if w.currentSize >= w.capacity {
name := w.f.Name()
if err := w.f.Close(); err != nil {
return err
@@ -66,6 +73,7 @@ func (w *RotateFileWriter) checkCapacityAndRotate() error {
return err
}
w.f = file
+ w.currentSize = 0
w.notifyRotate.Publish(struct{}{})
}
diff --git a/components/engine/daemon/logs.go b/components/engine/daemon/logs.go
index eb6fa54ae6..8172df175c 100644
--- a/components/engine/daemon/logs.go
+++ b/components/engine/daemon/logs.go
@@ -1,6 +1,7 @@
package daemon
import (
+ "fmt"
"io"
"strconv"
"time"
@@ -10,9 +11,9 @@ import (
"github.com/docker/docker/container"
"github.com/docker/docker/daemon/logger"
"github.com/docker/docker/daemon/logger/jsonfilelog"
- derr "github.com/docker/docker/errors"
"github.com/docker/docker/pkg/ioutils"
"github.com/docker/docker/pkg/stdcopy"
+ containertypes "github.com/docker/engine-api/types/container"
timetypes "github.com/docker/engine-api/types/time"
)
@@ -21,11 +22,11 @@ import (
func (daemon *Daemon) ContainerLogs(containerName string, config *backend.ContainerLogsConfig, started chan struct{}) error {
container, err := daemon.GetContainer(containerName)
if err != nil {
- return derr.ErrorCodeNoSuchContainer.WithArgs(containerName)
+ return err
}
if !(config.ShowStdout || config.ShowStderr) {
- return derr.ErrorCodeNeedStream
+ return fmt.Errorf("You must choose at least one stream")
}
cLog, err := daemon.getLogger(container)
@@ -103,7 +104,7 @@ func (daemon *Daemon) getLogger(container *container.Container) (logger.Logger,
if container.LogDriver != nil && container.IsRunning() {
return container.LogDriver, nil
}
- cfg := container.GetLogConfig(daemon.defaultLogConfig)
+ cfg := daemon.getLogConfig(container.HostConfig.LogConfig)
if err := logger.ValidateLogOpts(cfg.Type, cfg.Config); err != nil {
return nil, err
}
@@ -112,7 +113,7 @@ func (daemon *Daemon) getLogger(container *container.Container) (logger.Logger,
// StartLogging initializes and starts the container logging stream.
func (daemon *Daemon) StartLogging(container *container.Container) error {
- cfg := container.GetLogConfig(daemon.defaultLogConfig)
+ cfg := daemon.getLogConfig(container.HostConfig.LogConfig)
if cfg.Type == "none" {
return nil // do not start logging routines
}
@@ -122,7 +123,7 @@ func (daemon *Daemon) StartLogging(container *container.Container) error {
}
l, err := container.StartLogger(cfg)
if err != nil {
- return derr.ErrorCodeInitLogger.WithArgs(err)
+ return fmt.Errorf("Failed to initialize logging driver: %v", err)
}
copier := logger.NewCopier(container.ID, map[string]io.Reader{"stdout": container.StdoutPipe(), "stderr": container.StderrPipe()}, l)
@@ -137,3 +138,16 @@ func (daemon *Daemon) StartLogging(container *container.Container) error {
return nil
}
+
+// getLogConfig returns the log configuration for the container.
+func (daemon *Daemon) getLogConfig(cfg containertypes.LogConfig) containertypes.LogConfig {
+ if cfg.Type != "" || len(cfg.Config) > 0 { // container has log driver configured
+ if cfg.Type == "" {
+ cfg.Type = jsonfilelog.Name
+ }
+ return cfg
+ }
+
+ // Use daemon's default log config for containers
+ return daemon.defaultLogConfig
+}
diff --git a/components/engine/daemon/mounts.go b/components/engine/daemon/mounts.go
index 276301d130..d4f24b2812 100644
--- a/components/engine/daemon/mounts.go
+++ b/components/engine/daemon/mounts.go
@@ -1,10 +1,10 @@
package daemon
import (
+ "fmt"
"strings"
"github.com/docker/docker/container"
- derr "github.com/docker/docker/errors"
volumestore "github.com/docker/docker/volume/store"
)
@@ -42,7 +42,7 @@ func (daemon *Daemon) removeMountPoints(container *container.Container, rm bool)
}
}
if len(rmErrors) > 0 {
- return derr.ErrorCodeRemovingVolume.WithArgs(strings.Join(rmErrors, "\n"))
+ return fmt.Errorf("Error removing volumes:\n%v", strings.Join(rmErrors, "\n"))
}
return nil
}
diff --git a/components/engine/daemon/network.go b/components/engine/daemon/network.go
index 07d0771f7c..e937391b83 100644
--- a/components/engine/daemon/network.go
+++ b/components/engine/daemon/network.go
@@ -3,9 +3,10 @@ package daemon
import (
"fmt"
"net"
+ "net/http"
"strings"
- derr "github.com/docker/docker/errors"
+ "github.com/docker/docker/errors"
"github.com/docker/docker/runconfig"
"github.com/docker/engine-api/types/network"
"github.com/docker/libnetwork"
@@ -90,7 +91,7 @@ func (daemon *Daemon) GetAllNetworks() []libnetwork.Network {
}
// CreateNetwork creates a network with the given name, driver and other optional parameters
-func (daemon *Daemon) CreateNetwork(name, driver string, ipam network.IPAM, options map[string]string, internal bool) (libnetwork.Network, error) {
+func (daemon *Daemon) CreateNetwork(name, driver string, ipam network.IPAM, netOption map[string]string, internal bool, enableIPv6 bool) (libnetwork.Network, error) {
c := daemon.netController
if driver == "" {
driver = c.Config().Daemon.DefaultDriver
@@ -104,7 +105,8 @@ func (daemon *Daemon) CreateNetwork(name, driver string, ipam network.IPAM, opti
}
nwOptions = append(nwOptions, libnetwork.NetworkOptionIpam(ipam.Driver, "", v4Conf, v6Conf, ipam.Options))
- nwOptions = append(nwOptions, libnetwork.NetworkOptionDriverOpts(options))
+ nwOptions = append(nwOptions, libnetwork.NetworkOptionEnableIPv6(enableIPv6))
+ nwOptions = append(nwOptions, libnetwork.NetworkOptionDriverOpts(netOption))
if internal {
nwOptions = append(nwOptions, libnetwork.NetworkOptionInternalNetwork())
}
@@ -190,7 +192,8 @@ func (daemon *Daemon) DeleteNetwork(networkID string) error {
}
if runconfig.IsPreDefinedNetwork(nw.Name()) {
- return derr.ErrorCodeCantDeletePredefinedNetwork.WithArgs(nw.Name())
+ err := fmt.Errorf("%s is a pre-defined network and cannot be removed", nw.Name())
+ return errors.NewErrorWithStatusCode(err, http.StatusForbidden)
}
if err := nw.Delete(); err != nil {
diff --git a/components/engine/daemon/pause.go b/components/engine/daemon/pause.go
index a8ce013d4f..2ec0df7030 100644
--- a/components/engine/daemon/pause.go
+++ b/components/engine/daemon/pause.go
@@ -1,8 +1,9 @@
package daemon
import (
+ "fmt"
+
"github.com/docker/docker/container"
- derr "github.com/docker/docker/errors"
)
// ContainerPause pauses a container
@@ -27,21 +28,21 @@ func (daemon *Daemon) containerPause(container *container.Container) error {
// We cannot Pause the container which is not running
if !container.Running {
- return derr.ErrorCodeNotRunning.WithArgs(container.ID)
+ return errNotRunning{container.ID}
}
// We cannot Pause the container which is already paused
if container.Paused {
- return derr.ErrorCodeAlreadyPaused.WithArgs(container.ID)
+ return fmt.Errorf("Container %s is already paused", container.ID)
}
// We cannot Pause the container which is restarting
if container.Restarting {
- return derr.ErrorCodeContainerRestarting.WithArgs(container.ID)
+ return errContainerIsRestarting(container.ID)
}
if err := daemon.execDriver.Pause(container.Command); err != nil {
- return derr.ErrorCodeCantPause.WithArgs(container.ID, err)
+ return fmt.Errorf("Cannot pause container %s: %s", container.ID, err)
}
container.Paused = true
daemon.LogContainerEvent(container, "pause")
diff --git a/components/engine/daemon/rename.go b/components/engine/daemon/rename.go
index 2f903d9324..363a7f8bf4 100644
--- a/components/engine/daemon/rename.go
+++ b/components/engine/daemon/rename.go
@@ -1,10 +1,10 @@
package daemon
import (
+ "fmt"
"strings"
"github.com/Sirupsen/logrus"
- derr "github.com/docker/docker/errors"
"github.com/docker/libnetwork"
)
@@ -18,7 +18,7 @@ func (daemon *Daemon) ContainerRename(oldName, newName string) error {
)
if oldName == "" || newName == "" {
- return derr.ErrorCodeEmptyRename
+ return fmt.Errorf("Neither old nor new names may be empty")
}
container, err := daemon.GetContainer(oldName)
@@ -31,7 +31,7 @@ func (daemon *Daemon) ContainerRename(oldName, newName string) error {
container.Lock()
defer container.Unlock()
if newName, err = daemon.reserveName(container.ID, newName); err != nil {
- return derr.ErrorCodeRenameTaken.WithArgs(err)
+ return fmt.Errorf("Error when allocating new name: %v", err)
}
container.Name = newName
diff --git a/components/engine/daemon/resize.go b/components/engine/daemon/resize.go
index c326248c92..d7bb105b36 100644
--- a/components/engine/daemon/resize.go
+++ b/components/engine/daemon/resize.go
@@ -1,10 +1,6 @@
package daemon
-import (
- "fmt"
-
- derr "github.com/docker/docker/errors"
-)
+import "fmt"
// ContainerResize changes the size of the TTY of the process running
// in the container with the given name to the given height and width.
@@ -15,7 +11,7 @@ func (daemon *Daemon) ContainerResize(name string, height, width int) error {
}
if !container.IsRunning() {
- return derr.ErrorCodeNotRunning.WithArgs(container.ID)
+ return errNotRunning{container.ID}
}
if err = container.Resize(height, width); err == nil {
diff --git a/components/engine/daemon/restart.go b/components/engine/daemon/restart.go
index 8ee16918cb..3779116cfa 100644
--- a/components/engine/daemon/restart.go
+++ b/components/engine/daemon/restart.go
@@ -1,8 +1,9 @@
package daemon
import (
+ "fmt"
+
"github.com/docker/docker/container"
- derr "github.com/docker/docker/errors"
)
// ContainerRestart stops and starts a container. It attempts to
@@ -17,7 +18,7 @@ func (daemon *Daemon) ContainerRestart(name string, seconds int) error {
return err
}
if err := daemon.containerRestart(container, seconds); err != nil {
- return derr.ErrorCodeCantRestart.WithArgs(name, err)
+ return fmt.Errorf("Cannot restart container %s: %v", name, err)
}
return nil
}
diff --git a/components/engine/daemon/start.go b/components/engine/daemon/start.go
index 883e9d06d9..09842e3ae3 100644
--- a/components/engine/daemon/start.go
+++ b/components/engine/daemon/start.go
@@ -2,11 +2,12 @@ package daemon
import (
"fmt"
+ "net/http"
"runtime"
"github.com/Sirupsen/logrus"
"github.com/docker/docker/container"
- derr "github.com/docker/docker/errors"
+ "github.com/docker/docker/errors"
"github.com/docker/docker/runconfig"
containertypes "github.com/docker/engine-api/types/container"
)
@@ -19,11 +20,12 @@ func (daemon *Daemon) ContainerStart(name string, hostConfig *containertypes.Hos
}
if container.IsPaused() {
- return derr.ErrorCodeStartPaused
+ return fmt.Errorf("Cannot start a paused container, try unpause instead.")
}
if container.IsRunning() {
- return derr.ErrorCodeAlreadyStarted
+ err := fmt.Errorf("Container already started")
+ return errors.NewErrorWithStatusCode(err, http.StatusNotModified)
}
// Windows does not have the backwards compatibility issue here.
@@ -52,13 +54,13 @@ func (daemon *Daemon) ContainerStart(name string, hostConfig *containertypes.Hos
}
} else {
if hostConfig != nil {
- return derr.ErrorCodeHostConfigStart
+ return fmt.Errorf("Supplying a hostconfig on start is not supported. It should be supplied on create")
}
}
// check if hostConfig is in line with the current system settings.
// It may happen cgroups are umounted or the like.
- if _, err = daemon.verifyContainerSettings(container.HostConfig, nil); err != nil {
+ if _, err = daemon.verifyContainerSettings(container.HostConfig, nil, false); err != nil {
return err
}
// Adapt for old containers in case we have updates in this function and
@@ -88,7 +90,7 @@ func (daemon *Daemon) containerStart(container *container.Container) (err error)
}
if container.RemovalInProgress || container.Dead {
- return derr.ErrorCodeContainerBeingRemoved
+ return fmt.Errorf("Container is marked for removal and cannot be started.")
}
// if we encounter an error during start we need to ensure that any other
@@ -124,7 +126,8 @@ func (daemon *Daemon) containerStart(container *container.Container) (err error)
if err != nil {
return err
}
- if err := container.SetupWorkingDirectory(); err != nil {
+ rootUID, rootGID := daemon.GetRemappedUIDGID()
+ if err := container.SetupWorkingDirectory(rootUID, rootGID); err != nil {
return err
}
env := container.CreateDaemonEnvironment(linkedEnv)
@@ -154,7 +157,7 @@ func (daemon *Daemon) containerStart(container *container.Container) (err error)
}
func (daemon *Daemon) waitForStart(container *container.Container) error {
- return container.StartMonitor(daemon, container.HostConfig.RestartPolicy)
+ return container.StartMonitor(daemon)
}
// Cleanup releases any network resources allocated to the container along with any rules
diff --git a/components/engine/daemon/stats_collector_unix.go b/components/engine/daemon/stats_collector_unix.go
index 2fd368cd34..a8de5a2062 100644
--- a/components/engine/daemon/stats_collector_unix.go
+++ b/components/engine/daemon/stats_collector_unix.go
@@ -4,6 +4,7 @@ package daemon
import (
"bufio"
+ "fmt"
"os"
"strconv"
"strings"
@@ -13,7 +14,6 @@ import (
"github.com/Sirupsen/logrus"
"github.com/docker/docker/container"
"github.com/docker/docker/daemon/execdriver"
- derr "github.com/docker/docker/errors"
"github.com/docker/docker/pkg/pubsub"
"github.com/opencontainers/runc/libcontainer/system"
)
@@ -163,13 +163,13 @@ func (s *statsCollector) getSystemCPUUsage() (uint64, error) {
switch parts[0] {
case "cpu":
if len(parts) < 8 {
- return 0, derr.ErrorCodeBadCPUFields
+ return 0, fmt.Errorf("invalid number of cpu fields")
}
var totalClockTicks uint64
for _, i := range parts[1:8] {
v, err := strconv.ParseUint(i, 10, 64)
if err != nil {
- return 0, derr.ErrorCodeBadCPUInt.WithArgs(i, err)
+ return 0, fmt.Errorf("Unable to convert value %s to int: %s", i, err)
}
totalClockTicks += v
}
@@ -177,5 +177,5 @@ func (s *statsCollector) getSystemCPUUsage() (uint64, error) {
s.clockTicksPerSecond, nil
}
}
- return 0, derr.ErrorCodeBadStatFormat
+ return 0, fmt.Errorf("invalid stat format. Error trying to parse the '/proc/stat' file")
}
diff --git a/components/engine/daemon/stats_linux.go b/components/engine/daemon/stats_linux.go
index 201552a4e1..1a907e015a 100644
--- a/components/engine/daemon/stats_linux.go
+++ b/components/engine/daemon/stats_linux.go
@@ -61,6 +61,10 @@ func convertStatsToAPITypes(ls *libcontainer.Stats) *types.StatsJSON {
Stats: mem.Stats,
Failcnt: mem.Usage.Failcnt,
}
+ pids := cs.PidsStats
+ s.PidsStats = types.PidsStats{
+ Current: pids.Current,
+ }
}
return s
diff --git a/components/engine/daemon/stop.go b/components/engine/daemon/stop.go
index 55e3787751..701743008a 100644
--- a/components/engine/daemon/stop.go
+++ b/components/engine/daemon/stop.go
@@ -1,11 +1,13 @@
package daemon
import (
+ "fmt"
+ "net/http"
"time"
"github.com/Sirupsen/logrus"
"github.com/docker/docker/container"
- derr "github.com/docker/docker/errors"
+ "github.com/docker/docker/errors"
)
// ContainerStop looks for the given container and terminates it,
@@ -20,10 +22,11 @@ func (daemon *Daemon) ContainerStop(name string, seconds int) error {
return err
}
if !container.IsRunning() {
- return derr.ErrorCodeStopped.WithArgs(name)
+ err := fmt.Errorf("Container %s is already stopped", name)
+ return errors.NewErrorWithStatusCode(err, http.StatusNotModified)
}
if err := daemon.containerStop(container, seconds); err != nil {
- return derr.ErrorCodeCantStop.WithArgs(name, err)
+ return fmt.Errorf("Cannot stop container %s: %v", name, err)
}
return nil
}
diff --git a/components/engine/daemon/top_unix.go b/components/engine/daemon/top_unix.go
index 6d92592eb6..1f8ab07f04 100644
--- a/components/engine/daemon/top_unix.go
+++ b/components/engine/daemon/top_unix.go
@@ -3,11 +3,11 @@
package daemon
import (
+ "fmt"
"os/exec"
"strconv"
"strings"
- derr "github.com/docker/docker/errors"
"github.com/docker/engine-api/types"
)
@@ -27,11 +27,11 @@ func (daemon *Daemon) ContainerTop(name string, psArgs string) (*types.Container
}
if !container.IsRunning() {
- return nil, derr.ErrorCodeNotRunning.WithArgs(name)
+ return nil, errNotRunning{container.ID}
}
if container.IsRestarting() {
- return nil, derr.ErrorCodeContainerRestarting.WithArgs(name)
+ return nil, errContainerIsRestarting(container.ID)
}
pids, err := daemon.ExecutionDriver().GetPidsForContainer(container.ID)
if err != nil {
@@ -40,7 +40,7 @@ func (daemon *Daemon) ContainerTop(name string, psArgs string) (*types.Container
output, err := exec.Command("ps", strings.Split(psArgs, " ")...).Output()
if err != nil {
- return nil, derr.ErrorCodePSError.WithArgs(err)
+ return nil, fmt.Errorf("Error running ps: %v", err)
}
procList := &types.ContainerProcessList{}
@@ -55,7 +55,7 @@ func (daemon *Daemon) ContainerTop(name string, psArgs string) (*types.Container
}
}
if pidIndex == -1 {
- return nil, derr.ErrorCodeNoPID
+ return nil, fmt.Errorf("Couldn't find PID field in ps output")
}
// loop through the output and extract the PID from each line
@@ -66,7 +66,7 @@ func (daemon *Daemon) ContainerTop(name string, psArgs string) (*types.Container
fields := strings.Fields(line)
p, err := strconv.Atoi(fields[pidIndex])
if err != nil {
- return nil, derr.ErrorCodeBadPID.WithArgs(fields[pidIndex], err)
+ return nil, fmt.Errorf("Unexpected pid '%s': %s", fields[pidIndex], err)
}
for _, pid := range pids {
diff --git a/components/engine/daemon/top_windows.go b/components/engine/daemon/top_windows.go
index dc4cace65d..8b4fb2c6f0 100644
--- a/components/engine/daemon/top_windows.go
+++ b/components/engine/daemon/top_windows.go
@@ -1,11 +1,12 @@
package daemon
import (
- derr "github.com/docker/docker/errors"
+ "fmt"
+
"github.com/docker/engine-api/types"
)
// ContainerTop is not supported on Windows and returns an error.
func (daemon *Daemon) ContainerTop(name string, psArgs string) (*types.ContainerProcessList, error) {
- return nil, derr.ErrorCodeNoTop
+ return nil, fmt.Errorf("Top is not supported on Windows")
}
diff --git a/components/engine/daemon/unpause.go b/components/engine/daemon/unpause.go
index ace52593d7..4af6f11222 100644
--- a/components/engine/daemon/unpause.go
+++ b/components/engine/daemon/unpause.go
@@ -1,8 +1,9 @@
package daemon
import (
+ "fmt"
+
"github.com/docker/docker/container"
- derr "github.com/docker/docker/errors"
)
// ContainerUnpause unpauses a container
@@ -26,16 +27,16 @@ func (daemon *Daemon) containerUnpause(container *container.Container) error {
// We cannot unpause the container which is not running
if !container.Running {
- return derr.ErrorCodeNotRunning.WithArgs(container.ID)
+ return errNotRunning{container.ID}
}
// We cannot unpause the container which is not paused
if !container.Paused {
- return derr.ErrorCodeNotPaused.WithArgs(container.ID)
+ return fmt.Errorf("Container %s is not paused", container.ID)
}
if err := daemon.execDriver.Unpause(container.Command); err != nil {
- return derr.ErrorCodeCantUnpause.WithArgs(container.ID, err)
+ return fmt.Errorf("Cannot unpause container %s: %s", container.ID, err)
}
container.Paused = false
diff --git a/components/engine/daemon/update.go b/components/engine/daemon/update.go
index 181b399c96..ffdcc852bd 100644
--- a/components/engine/daemon/update.go
+++ b/components/engine/daemon/update.go
@@ -2,16 +2,16 @@ package daemon
import (
"fmt"
+ "time"
- derr "github.com/docker/docker/errors"
"github.com/docker/engine-api/types/container"
)
-// ContainerUpdate updates resources of the container
+// ContainerUpdate updates configuration of the container
func (daemon *Daemon) ContainerUpdate(name string, hostConfig *container.HostConfig) ([]string, error) {
var warnings []string
- warnings, err := daemon.verifyContainerSettings(hostConfig, nil)
+ warnings, err := daemon.verifyContainerSettings(hostConfig, nil, true)
if err != nil {
return warnings, err
}
@@ -44,27 +44,46 @@ func (daemon *Daemon) update(name string, hostConfig *container.HostConfig) erro
return err
}
+ restoreConfig := false
+ backupHostConfig := *container.HostConfig
+ defer func() {
+ if restoreConfig {
+ container.Lock()
+ container.HostConfig = &backupHostConfig
+ container.ToDisk()
+ container.Unlock()
+ }
+ }()
+
if container.RemovalInProgress || container.Dead {
- errMsg := fmt.Errorf("Container is marked for removal and cannot be \"update\".")
- return derr.ErrorCodeCantUpdate.WithArgs(container.ID, errMsg)
+ return errCannotUpdate(container.ID, fmt.Errorf("Container is marked for removal and cannot be \"update\"."))
}
if container.IsRunning() && hostConfig.KernelMemory != 0 {
- errMsg := fmt.Errorf("Can not update kernel memory to a running container, please stop it first.")
- return derr.ErrorCodeCantUpdate.WithArgs(container.ID, errMsg)
+ return errCannotUpdate(container.ID, fmt.Errorf("Can not update kernel memory to a running container, please stop it first."))
}
if err := container.UpdateContainer(hostConfig); err != nil {
- return derr.ErrorCodeCantUpdate.WithArgs(container.ID, err.Error())
+ restoreConfig = true
+ return errCannotUpdate(container.ID, err)
+ }
+
+ // if Restart Policy changed, we need to update container monitor
+ container.UpdateMonitor(hostConfig.RestartPolicy)
+
+ // if container is restarting, wait 5 seconds until it's running
+ if container.IsRestarting() {
+ container.WaitRunning(5 * time.Second)
}
// If container is not running, update hostConfig struct is enough,
// resources will be updated when the container is started again.
// If container is running (including paused), we need to update configs
// to the real world.
- if container.IsRunning() {
+ if container.IsRunning() && !container.IsRestarting() {
if err := daemon.execDriver.Update(container.Command); err != nil {
- return derr.ErrorCodeCantUpdate.WithArgs(container.ID, err.Error())
+ restoreConfig = true
+ return errCannotUpdate(container.ID, err)
}
}
@@ -72,3 +91,7 @@ func (daemon *Daemon) update(name string, hostConfig *container.HostConfig) erro
return nil
}
+
+func errCannotUpdate(containerID string, err error) error {
+ return fmt.Errorf("Cannot update container %s: %v", containerID, err)
+}
diff --git a/components/engine/daemon/volumes.go b/components/engine/daemon/volumes.go
index 7e2c417b2b..d32715997d 100644
--- a/components/engine/daemon/volumes.go
+++ b/components/engine/daemon/volumes.go
@@ -2,13 +2,13 @@ package daemon
import (
"errors"
+ "fmt"
"os"
"path/filepath"
"strings"
"github.com/docker/docker/container"
"github.com/docker/docker/daemon/execdriver"
- derr "github.com/docker/docker/errors"
"github.com/docker/docker/volume"
"github.com/docker/engine-api/types"
containertypes "github.com/docker/engine-api/types/container"
@@ -114,10 +114,10 @@ func (daemon *Daemon) registerMountPoints(container *container.Container, hostCo
}
if binds[bind.Destination] {
- return derr.ErrorCodeMountDup.WithArgs(bind.Destination)
+ return fmt.Errorf("Duplicate mount point '%s'", bind.Destination)
}
- if len(bind.Name) > 0 && len(bind.Driver) > 0 {
+ if len(bind.Name) > 0 {
// create the volume
v, err := daemon.volumes.CreateWithRef(bind.Name, bind.Driver, container.ID, nil)
if err != nil {
@@ -128,7 +128,9 @@ func (daemon *Daemon) registerMountPoints(container *container.Container, hostCo
// bind.Name is an already existing volume, we need to use that here
bind.Driver = v.DriverName()
bind.Named = true
- bind = setBindModeIfNull(bind)
+ if bind.Driver == "local" {
+ bind = setBindModeIfNull(bind)
+ }
}
if label.RelabelNeeded(bind.Mode) {
if err := label.Relabel(bind.Source, container.MountLabel, label.IsShared(bind.Mode)); err != nil {
diff --git a/components/engine/daemon/volumes_windows.go b/components/engine/daemon/volumes_windows.go
index 05a45c385d..23c6a3b5e3 100644
--- a/components/engine/daemon/volumes_windows.go
+++ b/components/engine/daemon/volumes_windows.go
@@ -3,11 +3,11 @@
package daemon
import (
+ "fmt"
"sort"
"github.com/docker/docker/container"
"github.com/docker/docker/daemon/execdriver"
- derr "github.com/docker/docker/errors"
"github.com/docker/docker/volume"
)
@@ -27,7 +27,7 @@ func (daemon *Daemon) setupMounts(container *container.Container) ([]execdriver.
s = mount.Volume.Path()
}
if s == "" {
- return nil, derr.ErrorCodeVolumeNoSourceForMount.WithArgs(mount.Name, mount.Driver, mount.Destination)
+ return nil, fmt.Errorf("No source for mount name '%s' driver %q destination '%s'", mount.Name, mount.Driver, mount.Destination)
}
mnts = append(mnts, execdriver.Mount{
Source: s,
diff --git a/components/engine/distribution/errors.go b/components/engine/distribution/errors.go
new file mode 100644
index 0000000000..1cb34fdd51
--- /dev/null
+++ b/components/engine/distribution/errors.go
@@ -0,0 +1,106 @@
+package distribution
+
+import (
+ "net/url"
+ "strings"
+ "syscall"
+
+ "github.com/docker/distribution/registry/api/errcode"
+ "github.com/docker/distribution/registry/api/v2"
+ "github.com/docker/distribution/registry/client"
+ "github.com/docker/docker/distribution/xfer"
+)
+
+// ErrNoSupport is an error type used for errors indicating that an operation
+// is not supported. It encapsulates a more specific error.
+type ErrNoSupport struct{ Err error }
+
+func (e ErrNoSupport) Error() string {
+ if e.Err == nil {
+ return "not supported"
+ }
+ return e.Err.Error()
+}
+
+// fallbackError wraps an error that can possibly allow fallback to a different
+// endpoint.
+type fallbackError struct {
+ // err is the error being wrapped.
+ err error
+ // confirmedV2 is set to true if it was confirmed that the registry
+ // supports the v2 protocol. This is used to limit fallbacks to the v1
+ // protocol.
+ confirmedV2 bool
+ // transportOK is set to true if we managed to speak HTTP with the
+ // registry. This confirms that we're using appropriate TLS settings
+ // (or lack of TLS).
+ transportOK bool
+}
+
+// Error renders the FallbackError as a string.
+func (f fallbackError) Error() string {
+ return f.err.Error()
+}
+
+// shouldV2Fallback returns true if this error is a reason to fall back to v1.
+func shouldV2Fallback(err errcode.Error) bool {
+ switch err.Code {
+ case errcode.ErrorCodeUnauthorized, v2.ErrorCodeManifestUnknown, v2.ErrorCodeNameUnknown:
+ return true
+ }
+ return false
+}
+
+// continueOnError returns true if we should fallback to the next endpoint
+// as a result of this error.
+func continueOnError(err error) bool {
+ switch v := err.(type) {
+ case errcode.Errors:
+ if len(v) == 0 {
+ return true
+ }
+ return continueOnError(v[0])
+ case ErrNoSupport:
+ return continueOnError(v.Err)
+ case errcode.Error:
+ return shouldV2Fallback(v)
+ case *client.UnexpectedHTTPResponseError:
+ return true
+ case ImageConfigPullError:
+ return false
+ case error:
+ return !strings.Contains(err.Error(), strings.ToLower(syscall.ENOSPC.Error()))
+ }
+ // let's be nice and fallback if the error is a completely
+ // unexpected one.
+ // If new errors have to be handled in some way, please
+ // add them to the switch above.
+ return true
+}
+
+// retryOnError wraps the error in xfer.DoNotRetry if we should not retry the
+// operation after this error.
+func retryOnError(err error) error {
+ switch v := err.(type) {
+ case errcode.Errors:
+ return retryOnError(v[0])
+ case errcode.Error:
+ switch v.Code {
+ case errcode.ErrorCodeUnauthorized, errcode.ErrorCodeUnsupported, errcode.ErrorCodeDenied:
+ return xfer.DoNotRetry{Err: err}
+ }
+ case *url.Error:
+ return retryOnError(v.Err)
+ case *client.UnexpectedHTTPResponseError:
+ return xfer.DoNotRetry{Err: err}
+ case error:
+ if strings.Contains(err.Error(), strings.ToLower(syscall.ENOSPC.Error())) {
+ return xfer.DoNotRetry{Err: err}
+ }
+ }
+ // let's be nice and fallback if the error is a completely
+ // unexpected one.
+ // If new errors have to be handled in some way, please
+ // add them to the switch above.
+ return err
+}
diff --git a/components/engine/distribution/pull.go b/components/engine/distribution/pull.go
index 603d5de5be..4b42371b90 100644
--- a/components/engine/distribution/pull.go
+++ b/components/engine/distribution/pull.go
@@ -2,7 +2,6 @@ package distribution
import (
"fmt"
- "os"
"github.com/Sirupsen/logrus"
"github.com/docker/docker/api"
@@ -89,7 +88,7 @@ func Pull(ctx context.Context, ref reference.Named, imagePullConfig *ImagePullCo
return err
}
- endpoints, err := imagePullConfig.RegistryService.LookupPullEndpoints(repoInfo)
+ endpoints, err := imagePullConfig.RegistryService.LookupPullEndpoints(repoInfo.Hostname())
if err != nil {
return err
}
@@ -110,12 +109,25 @@ func Pull(ctx context.Context, ref reference.Named, imagePullConfig *ImagePullCo
// confirm that it was talking to a v2 registry. This will
// prevent fallback to the v1 protocol.
confirmedV2 bool
+
+ // confirmedTLSRegistries is a map indicating which registries
+ // are known to be using TLS. There should never be a plaintext
+ // retry for any of these.
+ confirmedTLSRegistries = make(map[string]struct{})
)
for _, endpoint := range endpoints {
if confirmedV2 && endpoint.Version == registry.APIVersion1 {
logrus.Debugf("Skipping v1 endpoint %s because v2 registry was detected", endpoint.URL)
continue
}
+
+ if endpoint.URL.Scheme != "https" {
+ if _, confirmedTLS := confirmedTLSRegistries[endpoint.URL.Host]; confirmedTLS {
+ logrus.Debugf("Skipping non-TLS endpoint %s for host/port that appears to use TLS", endpoint.URL)
+ continue
+ }
+ }
+
logrus.Debugf("Trying to pull %s from %s %s", repoInfo.Name(), endpoint.URL, endpoint.Version)
puller, err := newPuller(endpoint, repoInfo, imagePullConfig)
@@ -133,11 +145,14 @@ func Pull(ctx context.Context, ref reference.Named, imagePullConfig *ImagePullCo
if fallbackErr, ok := err.(fallbackError); ok {
fallback = true
confirmedV2 = confirmedV2 || fallbackErr.confirmedV2
+ if fallbackErr.transportOK && endpoint.URL.Scheme == "https" {
+ confirmedTLSRegistries[endpoint.URL.Host] = struct{}{}
+ }
err = fallbackErr.err
}
}
if fallback {
- if _, ok := err.(registry.ErrNoSupport); !ok {
+ if _, ok := err.(ErrNoSupport); !ok {
// Because we found an error that's not ErrNoSupport, discard all subsequent ErrNoSupport errors.
discardNoSupportErrors = true
// append subsequent errors
@@ -148,9 +163,10 @@ func Pull(ctx context.Context, ref reference.Named, imagePullConfig *ImagePullCo
// append subsequent errors
lastErr = err
}
+ logrus.Errorf("Attempting next endpoint for pull after error: %v", err)
continue
}
- logrus.Debugf("Not continuing with error: %v", err)
+ logrus.Errorf("Not continuing with pull after error: %v", err)
return err
}
@@ -187,16 +203,3 @@ func validateRepoName(name string) error {
}
return nil
}
-
-// tmpFileClose creates a closer function for a temporary file that closes the file
-// and also deletes it.
-func tmpFileCloser(tmpFile *os.File) func() error {
- return func() error {
- tmpFile.Close()
- if err := os.RemoveAll(tmpFile.Name()); err != nil {
- logrus.Errorf("Failed to remove temp file: %s", tmpFile.Name())
- }
-
- return nil
- }
-}
diff --git a/components/engine/distribution/pull_v1.go b/components/engine/distribution/pull_v1.go
index 312f7e30e2..3e0cbdb46c 100644
--- a/components/engine/distribution/pull_v1.go
+++ b/components/engine/distribution/pull_v1.go
@@ -7,6 +7,7 @@ import (
"io/ioutil"
"net"
"net/url"
+ "os"
"strings"
"time"
@@ -37,7 +38,7 @@ type v1Puller struct {
func (p *v1Puller) Pull(ctx context.Context, ref reference.Named) error {
if _, isCanonical := ref.(reference.Canonical); isCanonical {
// Allowing fallback, because HTTPS v1 is before HTTP v2
- return fallbackError{err: registry.ErrNoSupport{Err: errors.New("Cannot pull by digest with v1 registry")}}
+ return fallbackError{err: ErrNoSupport{Err: errors.New("Cannot pull by digest with v1 registry")}}
}
tlsConfig, err := p.config.RegistryService.TLSConfig(p.repoInfo.Index.Name)
@@ -279,6 +280,7 @@ type v1LayerDescriptor struct {
layersDownloaded *bool
layerSize int64
session *registry.Session
+ tmpFile *os.File
}
func (ld *v1LayerDescriptor) Key() string {
@@ -308,7 +310,7 @@ func (ld *v1LayerDescriptor) Download(ctx context.Context, progressOutput progre
}
*ld.layersDownloaded = true
- tmpFile, err := ioutil.TempFile("", "GetImageBlob")
+ ld.tmpFile, err = ioutil.TempFile("", "GetImageBlob")
if err != nil {
layerReader.Close()
return nil, 0, err
@@ -317,17 +319,28 @@ func (ld *v1LayerDescriptor) Download(ctx context.Context, progressOutput progre
reader := progress.NewProgressReader(ioutils.NewCancelReadCloser(ctx, layerReader), progressOutput, ld.layerSize, ld.ID(), "Downloading")
defer reader.Close()
- _, err = io.Copy(tmpFile, reader)
+ _, err = io.Copy(ld.tmpFile, reader)
if err != nil {
+ ld.Close()
return nil, 0, err
}
progress.Update(progressOutput, ld.ID(), "Download complete")
- logrus.Debugf("Downloaded %s to tempfile %s", ld.ID(), tmpFile.Name())
+ logrus.Debugf("Downloaded %s to tempfile %s", ld.ID(), ld.tmpFile.Name())
- tmpFile.Seek(0, 0)
- return ioutils.NewReadCloserWrapper(tmpFile, tmpFileCloser(tmpFile)), ld.layerSize, nil
+ ld.tmpFile.Seek(0, 0)
+ return ld.tmpFile, ld.layerSize, nil
+}
+
+func (ld *v1LayerDescriptor) Close() {
+ if ld.tmpFile != nil {
+ ld.tmpFile.Close()
+ if err := os.RemoveAll(ld.tmpFile.Name()); err != nil {
+ logrus.Errorf("Failed to remove temp file: %s", ld.tmpFile.Name())
+ }
+ ld.tmpFile = nil
+ }
}
func (ld *v1LayerDescriptor) Registered(diffID layer.DiffID) {
diff --git a/components/engine/distribution/pull_v2.go b/components/engine/distribution/pull_v2.go
index 04d05e02f4..596d1c1321 100644
--- a/components/engine/distribution/pull_v2.go
+++ b/components/engine/distribution/pull_v2.go
@@ -6,6 +6,7 @@ import (
"fmt"
"io"
"io/ioutil"
+ "net/url"
"os"
"runtime"
@@ -17,6 +18,8 @@ import (
"github.com/docker/distribution/manifest/schema2"
"github.com/docker/distribution/registry/api/errcode"
"github.com/docker/distribution/registry/client"
+ "github.com/docker/distribution/registry/client/auth"
+ "github.com/docker/distribution/registry/client/transport"
"github.com/docker/docker/distribution/metadata"
"github.com/docker/docker/distribution/xfer"
"github.com/docker/docker/image"
@@ -32,6 +35,17 @@ import (
var errRootFSMismatch = errors.New("layers from manifest don't match image configuration")
+// ImageConfigPullError is an error pulling the image config blob
+// (only applies to schema2).
+type ImageConfigPullError struct {
+ Err error
+}
+
+// Error returns the error string for ImageConfigPullError.
+func (e ImageConfigPullError) Error() string {
+ return "error pulling image configuration: " + e.Err.Error()
+}
+
type v2Puller struct {
V2MetadataService *metadata.V2MetadataService
endpoint registry.APIEndpoint
@@ -48,16 +62,20 @@ func (p *v2Puller) Pull(ctx context.Context, ref reference.Named) (err error) {
p.repo, p.confirmedV2, err = NewV2Repository(ctx, p.repoInfo, p.endpoint, p.config.MetaHeaders, p.config.AuthConfig, "pull")
if err != nil {
logrus.Warnf("Error getting v2 registry: %v", err)
- return fallbackError{err: err, confirmedV2: p.confirmedV2}
+ return err
}
if err = p.pullV2Repository(ctx, ref); err != nil {
if _, ok := err.(fallbackError); ok {
return err
}
- if registry.ContinueOnError(err) {
- logrus.Debugf("Error trying v2 registry: %v", err)
- return fallbackError{err: err, confirmedV2: p.confirmedV2}
+ if continueOnError(err) {
+ logrus.Errorf("Error trying v2 registry: %v", err)
+ return fallbackError{
+ err: err,
+ confirmedV2: p.confirmedV2,
+ transportOK: true,
+ }
}
}
return err
@@ -114,6 +132,8 @@ type v2LayerDescriptor struct {
repoInfo *registry.RepositoryInfo
repo distribution.Repository
V2MetadataService *metadata.V2MetadataService
+ tmpFile *os.File
+ verifier digest.Verifier
}
func (ld *v2LayerDescriptor) Key() string {
@@ -131,17 +151,56 @@ func (ld *v2LayerDescriptor) DiffID() (layer.DiffID, error) {
func (ld *v2LayerDescriptor) Download(ctx context.Context, progressOutput progress.Output) (io.ReadCloser, int64, error) {
logrus.Debugf("pulling blob %q", ld.digest)
+ var (
+ err error
+ offset int64
+ )
+
+ if ld.tmpFile == nil {
+ ld.tmpFile, err = createDownloadFile()
+ if err != nil {
+ return nil, 0, xfer.DoNotRetry{Err: err}
+ }
+ } else {
+ offset, err = ld.tmpFile.Seek(0, os.SEEK_END)
+ if err != nil {
+ logrus.Debugf("error seeking to end of download file: %v", err)
+ offset = 0
+
+ ld.tmpFile.Close()
+ if err := os.Remove(ld.tmpFile.Name()); err != nil {
+ logrus.Errorf("Failed to remove temp file: %s", ld.tmpFile.Name())
+ }
+ ld.tmpFile, err = createDownloadFile()
+ if err != nil {
+ return nil, 0, xfer.DoNotRetry{Err: err}
+ }
+ } else if offset != 0 {
+ logrus.Debugf("attempting to resume download of %q from %d bytes", ld.digest, offset)
+ }
+ }
+
+ tmpFile := ld.tmpFile
blobs := ld.repo.Blobs(ctx)
layerDownload, err := blobs.Open(ctx, ld.digest)
if err != nil {
- logrus.Debugf("Error statting layer: %v", err)
+ logrus.Errorf("Error initiating layer download: %v", err)
if err == distribution.ErrBlobUnknown {
return nil, 0, xfer.DoNotRetry{Err: err}
}
return nil, 0, retryOnError(err)
}
+ if offset != 0 {
+ _, err := layerDownload.Seek(offset, os.SEEK_SET)
+ if err != nil {
+ if err := ld.truncateDownloadFile(); err != nil {
+ return nil, 0, xfer.DoNotRetry{Err: err}
+ }
+ return nil, 0, err
+ }
+ }
size, err := layerDownload.Seek(0, os.SEEK_END)
if err != nil {
// Seek failed, perhaps because there was no Content-Length
@@ -149,46 +208,59 @@ func (ld *v2LayerDescriptor) Download(ctx context.Context, progressOutput progre
// still continue without a progress bar.
size = 0
} else {
- // Restore the seek offset at the beginning of the stream.
- _, err = layerDownload.Seek(0, os.SEEK_SET)
+ if size != 0 && offset > size {
+ logrus.Debugf("Partial download is larger than full blob. Starting over")
+ offset = 0
+ if err := ld.truncateDownloadFile(); err != nil {
+ return nil, 0, xfer.DoNotRetry{Err: err}
+ }
+ }
+
+ // Restore the seek offset either at the beginning of the
+ // stream, or just after the last byte we have from previous
+ // attempts.
+ _, err = layerDownload.Seek(offset, os.SEEK_SET)
if err != nil {
return nil, 0, err
}
}
- reader := progress.NewProgressReader(ioutils.NewCancelReadCloser(ctx, layerDownload), progressOutput, size, ld.ID(), "Downloading")
+ reader := progress.NewProgressReader(ioutils.NewCancelReadCloser(ctx, layerDownload), progressOutput, size-offset, ld.ID(), "Downloading")
defer reader.Close()
- verifier, err := digest.NewDigestVerifier(ld.digest)
- if err != nil {
- return nil, 0, xfer.DoNotRetry{Err: err}
+ if ld.verifier == nil {
+ ld.verifier, err = digest.NewDigestVerifier(ld.digest)
+ if err != nil {
+ return nil, 0, xfer.DoNotRetry{Err: err}
+ }
}
- tmpFile, err := ioutil.TempFile("", "GetImageBlob")
+ _, err = io.Copy(tmpFile, io.TeeReader(reader, ld.verifier))
if err != nil {
- return nil, 0, xfer.DoNotRetry{Err: err}
- }
-
- _, err = io.Copy(tmpFile, io.TeeReader(reader, verifier))
- if err != nil {
- tmpFile.Close()
- if err := os.Remove(tmpFile.Name()); err != nil {
- logrus.Errorf("Failed to remove temp file: %s", tmpFile.Name())
+ if err == transport.ErrWrongCodeForByteRange {
+ if err := ld.truncateDownloadFile(); err != nil {
+ return nil, 0, xfer.DoNotRetry{Err: err}
+ }
+ return nil, 0, err
}
return nil, 0, retryOnError(err)
}
progress.Update(progressOutput, ld.ID(), "Verifying Checksum")
- if !verifier.Verified() {
+ if !ld.verifier.Verified() {
err = fmt.Errorf("filesystem layer verification failed for digest %s", ld.digest)
logrus.Error(err)
- tmpFile.Close()
- if err := os.Remove(tmpFile.Name()); err != nil {
- logrus.Errorf("Failed to remove temp file: %s", tmpFile.Name())
- }
+ // Allow a retry if this digest verification error happened
+ // after a resumed download.
+ if offset != 0 {
+ if err := ld.truncateDownloadFile(); err != nil {
+ return nil, 0, xfer.DoNotRetry{Err: err}
+ }
+ return nil, 0, err
+ }
return nil, 0, xfer.DoNotRetry{Err: err}
}
@@ -202,9 +274,37 @@ func (ld *v2LayerDescriptor) Download(ctx context.Context, progressOutput progre
if err := os.Remove(tmpFile.Name()); err != nil {
logrus.Errorf("Failed to remove temp file: %s", tmpFile.Name())
}
+ ld.tmpFile = nil
+ ld.verifier = nil
return nil, 0, xfer.DoNotRetry{Err: err}
}
- return ioutils.NewReadCloserWrapper(tmpFile, tmpFileCloser(tmpFile)), size, nil
+ return tmpFile, size, nil
+}
+
+func (ld *v2LayerDescriptor) Close() {
+ if ld.tmpFile != nil {
+ ld.tmpFile.Close()
+ if err := os.RemoveAll(ld.tmpFile.Name()); err != nil {
+ logrus.Errorf("Failed to remove temp file: %s", ld.tmpFile.Name())
+ }
+ }
+}
+
+func (ld *v2LayerDescriptor) truncateDownloadFile() error {
+ // Need a new hash context since we will be redoing the download
+ ld.verifier = nil
+
+ if _, err := ld.tmpFile.Seek(0, os.SEEK_SET); err != nil {
+ logrus.Errorf("error seeking to beginning of download file: %v", err)
+ return err
+ }
+
+ if err := ld.tmpFile.Truncate(0); err != nil {
+ logrus.Errorf("error truncating download file: %v", err)
+ return err
+ }
+
+ return nil
}
func (ld *v2LayerDescriptor) Registered(diffID layer.DiffID) {
@@ -399,7 +499,7 @@ func (p *v2Puller) pullSchema2(ctx context.Context, ref reference.Named, mfst *s
go func() {
configJSON, err := p.pullSchema2ImageConfig(ctx, target.Digest)
if err != nil {
- errChan <- err
+ errChan <- ImageConfigPullError{Err: err}
cancel()
return
}
@@ -619,12 +719,24 @@ func allowV1Fallback(err error) error {
switch v := err.(type) {
case errcode.Errors:
if len(v) != 0 {
- if v0, ok := v[0].(errcode.Error); ok && registry.ShouldV2Fallback(v0) {
- return fallbackError{err: err, confirmedV2: false}
+ if v0, ok := v[0].(errcode.Error); ok && shouldV2Fallback(v0) {
+ return fallbackError{
+ err: err,
+ confirmedV2: false,
+ transportOK: true,
+ }
}
}
case errcode.Error:
- if registry.ShouldV2Fallback(v) {
+ if shouldV2Fallback(v) {
+ return fallbackError{
+ err: err,
+ confirmedV2: false,
+ transportOK: true,
+ }
+ }
+ case *url.Error:
+ if v.Err == auth.ErrNoBasicAuthCredentials {
return fallbackError{err: err, confirmedV2: false}
}
}
@@ -711,3 +823,7 @@ func fixManifestLayers(m *schema1.Manifest) error {
return nil
}
+
+func createDownloadFile() (*os.File, error) {
+ return ioutil.TempFile("", "GetImageBlob")
+}
diff --git a/components/engine/distribution/pull_v2_test.go b/components/engine/distribution/pull_v2_test.go
index 53995bf663..8555c81e6b 100644
--- a/components/engine/distribution/pull_v2_test.go
+++ b/components/engine/distribution/pull_v2_test.go
@@ -4,6 +4,7 @@ import (
"encoding/json"
"io/ioutil"
"reflect"
+ "runtime"
"strings"
"testing"
@@ -62,6 +63,10 @@ func TestFixManifestLayers(t *testing.T) {
// TestFixManifestLayersBaseLayerParent makes sure that fixManifestLayers fails
// if the base layer configuration specifies a parent.
func TestFixManifestLayersBaseLayerParent(t *testing.T) {
+ // TODO Windows: Fix this unit text
+ if runtime.GOOS == "windows" {
+ t.Skip("Needs fixing on Windows")
+ }
duplicateLayerManifest := schema1.Manifest{
FSLayers: []schema1.FSLayer{
{BlobSum: digest.Digest("sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4")},
@@ -104,6 +109,10 @@ func TestFixManifestLayersBadParent(t *testing.T) {
// TestValidateManifest verifies the validateManifest function
func TestValidateManifest(t *testing.T) {
+ // TODO Windows: Fix this unit text
+ if runtime.GOOS == "windows" {
+ t.Skip("Needs fixing on Windows")
+ }
expectedDigest, err := reference.ParseNamed("repo@sha256:02fee8c3220ba806531f606525eceb83f4feb654f62b207191b1c9209188dedd")
if err != nil {
t.Fatal("could not parse reference")
diff --git a/components/engine/distribution/push.go b/components/engine/distribution/push.go
index d0622f82c9..52ee8e77e2 100644
--- a/components/engine/distribution/push.go
+++ b/components/engine/distribution/push.go
@@ -100,7 +100,7 @@ func Push(ctx context.Context, ref reference.Named, imagePushConfig *ImagePushCo
return err
}
- endpoints, err := imagePushConfig.RegistryService.LookupPushEndpoints(repoInfo)
+ endpoints, err := imagePushConfig.RegistryService.LookupPushEndpoints(repoInfo.Hostname())
if err != nil {
return err
}
@@ -119,6 +119,11 @@ func Push(ctx context.Context, ref reference.Named, imagePushConfig *ImagePushCo
// confirm that it was talking to a v2 registry. This will
// prevent fallback to the v1 protocol.
confirmedV2 bool
+
+ // confirmedTLSRegistries is a map indicating which registries
+ // are known to be using TLS. There should never be a plaintext
+ // retry for any of these.
+ confirmedTLSRegistries = make(map[string]struct{})
)
for _, endpoint := range endpoints {
@@ -127,6 +132,13 @@ func Push(ctx context.Context, ref reference.Named, imagePushConfig *ImagePushCo
continue
}
+ if endpoint.URL.Scheme != "https" {
+ if _, confirmedTLS := confirmedTLSRegistries[endpoint.URL.Host]; confirmedTLS {
+ logrus.Debugf("Skipping non-TLS endpoint %s for host/port that appears to use TLS", endpoint.URL)
+ continue
+ }
+ }
+
logrus.Debugf("Trying to push %s to %s %s", repoInfo.FullName(), endpoint.URL, endpoint.Version)
pusher, err := NewPusher(ref, endpoint, repoInfo, imagePushConfig)
@@ -142,13 +154,17 @@ func Push(ctx context.Context, ref reference.Named, imagePushConfig *ImagePushCo
default:
if fallbackErr, ok := err.(fallbackError); ok {
confirmedV2 = confirmedV2 || fallbackErr.confirmedV2
+ if fallbackErr.transportOK && endpoint.URL.Scheme == "https" {
+ confirmedTLSRegistries[endpoint.URL.Host] = struct{}{}
+ }
err = fallbackErr.err
lastErr = err
+ logrus.Errorf("Attempting next endpoint for push after error: %v", err)
continue
}
}
- logrus.Debugf("Not continuing with error: %v", err)
+ logrus.Errorf("Not continuing with push after error: %v", err)
return err
}
diff --git a/components/engine/distribution/push_v1.go b/components/engine/distribution/push_v1.go
index 8be1df97f1..e9b1065f7a 100644
--- a/components/engine/distribution/push_v1.go
+++ b/components/engine/distribution/push_v1.go
@@ -21,7 +21,6 @@ import (
)
type v1Pusher struct {
- ctx context.Context
v1IDService *metadata.V1IDService
endpoint registry.APIEndpoint
ref reference.Named
diff --git a/components/engine/distribution/push_v2.go b/components/engine/distribution/push_v2.go
index ac2017a8d0..e812c1da8c 100644
--- a/components/engine/distribution/push_v2.go
+++ b/components/engine/distribution/push_v2.go
@@ -42,7 +42,7 @@ type v2Pusher struct {
config *ImagePushConfig
repo distribution.Repository
- // pushState is state built by the Download functions.
+ // pushState is state built by the Upload functions.
pushState pushState
}
@@ -64,12 +64,16 @@ func (p *v2Pusher) Push(ctx context.Context) (err error) {
p.repo, p.pushState.confirmedV2, err = NewV2Repository(ctx, p.repoInfo, p.endpoint, p.config.MetaHeaders, p.config.AuthConfig, "push", "pull")
if err != nil {
logrus.Debugf("Error getting v2 registry: %v", err)
- return fallbackError{err: err, confirmedV2: p.pushState.confirmedV2}
+ return err
}
if err = p.pushV2Repository(ctx); err != nil {
- if registry.ContinueOnError(err) {
- return fallbackError{err: err, confirmedV2: p.pushState.confirmedV2}
+ if continueOnError(err) {
+ return fallbackError{
+ err: err,
+ confirmedV2: p.pushState.confirmedV2,
+ transportOK: true,
+ }
}
}
return err
@@ -220,6 +224,7 @@ type v2PushDescriptor struct {
repoInfo reference.Named
repo distribution.Repository
pushState *pushState
+ remoteDescriptor distribution.Descriptor
}
func (pd *v2PushDescriptor) Key() string {
@@ -234,16 +239,16 @@ func (pd *v2PushDescriptor) DiffID() layer.DiffID {
return pd.layer.DiffID()
}
-func (pd *v2PushDescriptor) Upload(ctx context.Context, progressOutput progress.Output) error {
+func (pd *v2PushDescriptor) Upload(ctx context.Context, progressOutput progress.Output) (distribution.Descriptor, error) {
diffID := pd.DiffID()
pd.pushState.Lock()
- if _, ok := pd.pushState.remoteLayers[diffID]; ok {
+ if descriptor, ok := pd.pushState.remoteLayers[diffID]; ok {
// it is already known that the push is not needed and
// therefore doing a stat is unnecessary
pd.pushState.Unlock()
progress.Update(progressOutput, pd.ID(), "Layer already exists")
- return nil
+ return descriptor, nil
}
pd.pushState.Unlock()
@@ -253,14 +258,14 @@ func (pd *v2PushDescriptor) Upload(ctx context.Context, progressOutput progress.
descriptor, exists, err := layerAlreadyExists(ctx, v2Metadata, pd.repoInfo, pd.repo, pd.pushState)
if err != nil {
progress.Update(progressOutput, pd.ID(), "Image push failed")
- return retryOnError(err)
+ return distribution.Descriptor{}, retryOnError(err)
}
if exists {
progress.Update(progressOutput, pd.ID(), "Layer already exists")
pd.pushState.Lock()
pd.pushState.remoteLayers[diffID] = descriptor
pd.pushState.Unlock()
- return nil
+ return descriptor, nil
}
}
@@ -270,27 +275,29 @@ func (pd *v2PushDescriptor) Upload(ctx context.Context, progressOutput progress.
// then push the blob.
bs := pd.repo.Blobs(ctx)
- var mountFrom metadata.V2Metadata
+ var layerUpload distribution.BlobWriter
+ mountAttemptsRemaining := 3
- // Attempt to find another repository in the same registry to mount the layer from to avoid an unnecessary upload
- for _, metadata := range v2Metadata {
- sourceRepo, err := reference.ParseNamed(metadata.SourceRepository)
+ // Attempt to find another repository in the same registry to mount the layer
+ // from to avoid an unnecessary upload.
+ // Note: metadata is stored from oldest to newest, so we iterate through this
+ // slice in reverse to maximize our chances of the blob still existing in the
+ // remote repository.
+ for i := len(v2Metadata) - 1; i >= 0 && mountAttemptsRemaining > 0; i-- {
+ mountFrom := v2Metadata[i]
+
+ sourceRepo, err := reference.ParseNamed(mountFrom.SourceRepository)
if err != nil {
continue
}
- if pd.repoInfo.Hostname() == sourceRepo.Hostname() {
- logrus.Debugf("attempting to mount layer %s (%s) from %s", diffID, metadata.Digest, sourceRepo.FullName())
- mountFrom = metadata
- break
+ if pd.repoInfo.Hostname() != sourceRepo.Hostname() {
+ // don't mount blobs from another registry
+ continue
}
- }
- var createOpts []distribution.BlobCreateOption
-
- if mountFrom.SourceRepository != "" {
namedRef, err := reference.WithName(mountFrom.SourceRepository)
if err != nil {
- return err
+ continue
}
// TODO (brianbland): We need to construct a reference where the Name is
@@ -298,51 +305,55 @@ func (pd *v2PushDescriptor) Upload(ctx context.Context, progressOutput progress.
// richer reference package
remoteRef, err := distreference.WithName(namedRef.RemoteName())
if err != nil {
- return err
+ continue
}
canonicalRef, err := distreference.WithDigest(remoteRef, mountFrom.Digest)
if err != nil {
- return err
+ continue
}
- createOpts = append(createOpts, client.WithMountFrom(canonicalRef))
- }
+ logrus.Debugf("attempting to mount layer %s (%s) from %s", diffID, mountFrom.Digest, sourceRepo.FullName())
- // Send the layer
- layerUpload, err := bs.Create(ctx, createOpts...)
- switch err := err.(type) {
- case distribution.ErrBlobMounted:
- progress.Updatef(progressOutput, pd.ID(), "Mounted from %s", err.From.Name())
+ layerUpload, err = bs.Create(ctx, client.WithMountFrom(canonicalRef))
+ switch err := err.(type) {
+ case distribution.ErrBlobMounted:
+ progress.Updatef(progressOutput, pd.ID(), "Mounted from %s", err.From.Name())
- err.Descriptor.MediaType = schema2.MediaTypeLayer
+ err.Descriptor.MediaType = schema2.MediaTypeLayer
- pd.pushState.Lock()
- pd.pushState.confirmedV2 = true
- pd.pushState.remoteLayers[diffID] = err.Descriptor
- pd.pushState.Unlock()
+ pd.pushState.Lock()
+ pd.pushState.confirmedV2 = true
+ pd.pushState.remoteLayers[diffID] = err.Descriptor
+ pd.pushState.Unlock()
- // Cache mapping from this layer's DiffID to the blobsum
- if err := pd.v2MetadataService.Add(diffID, metadata.V2Metadata{Digest: mountFrom.Digest, SourceRepository: pd.repoInfo.FullName()}); err != nil {
- return xfer.DoNotRetry{Err: err}
+ // Cache mapping from this layer's DiffID to the blobsum
+ if err := pd.v2MetadataService.Add(diffID, metadata.V2Metadata{Digest: mountFrom.Digest, SourceRepository: pd.repoInfo.FullName()}); err != nil {
+ return distribution.Descriptor{}, xfer.DoNotRetry{Err: err}
+ }
+ return err.Descriptor, nil
+ case nil:
+ // blob upload session created successfully, so begin the upload
+ mountAttemptsRemaining = 0
+ default:
+ // unable to mount layer from this repository, so this source mapping is no longer valid
+ logrus.Debugf("unassociating layer %s (%s) with %s", diffID, mountFrom.Digest, mountFrom.SourceRepository)
+ pd.v2MetadataService.Remove(mountFrom)
+ mountAttemptsRemaining--
}
-
- return nil
- }
- if mountFrom.SourceRepository != "" {
- // unable to mount layer from this repository, so this source mapping is no longer valid
- logrus.Debugf("unassociating layer %s (%s) with %s", diffID, mountFrom.Digest, mountFrom.SourceRepository)
- pd.v2MetadataService.Remove(mountFrom)
}
- if err != nil {
- return retryOnError(err)
+ if layerUpload == nil {
+ layerUpload, err = bs.Create(ctx)
+ if err != nil {
+ return distribution.Descriptor{}, retryOnError(err)
+ }
}
defer layerUpload.Close()
arch, err := pd.layer.TarStream()
if err != nil {
- return xfer.DoNotRetry{Err: err}
+ return distribution.Descriptor{}, xfer.DoNotRetry{Err: err}
}
// don't care if this fails; best effort
@@ -361,12 +372,12 @@ func (pd *v2PushDescriptor) Upload(ctx context.Context, progressOutput progress.
nn, err := layerUpload.ReadFrom(tee)
compressedReader.Close()
if err != nil {
- return retryOnError(err)
+ return distribution.Descriptor{}, retryOnError(err)
}
pushDigest := digester.Digest()
if _, err := layerUpload.Commit(ctx, distribution.Descriptor{Digest: pushDigest}); err != nil {
- return retryOnError(err)
+ return distribution.Descriptor{}, retryOnError(err)
}
logrus.Debugf("uploaded layer %s (%s), %d bytes", diffID, pushDigest, nn)
@@ -374,32 +385,33 @@ func (pd *v2PushDescriptor) Upload(ctx context.Context, progressOutput progress.
// Cache mapping from this layer's DiffID to the blobsum
if err := pd.v2MetadataService.Add(diffID, metadata.V2Metadata{Digest: pushDigest, SourceRepository: pd.repoInfo.FullName()}); err != nil {
- return xfer.DoNotRetry{Err: err}
+ return distribution.Descriptor{}, xfer.DoNotRetry{Err: err}
}
pd.pushState.Lock()
- // If Commit succeded, that's an indication that the remote registry
+ // If Commit succeeded, that's an indication that the remote registry
// speaks the v2 protocol.
pd.pushState.confirmedV2 = true
- pd.pushState.remoteLayers[diffID] = distribution.Descriptor{
+ descriptor := distribution.Descriptor{
Digest: pushDigest,
MediaType: schema2.MediaTypeLayer,
Size: nn,
}
+ pd.pushState.remoteLayers[diffID] = descriptor
pd.pushState.Unlock()
- return nil
+ return descriptor, nil
+}
+
+func (pd *v2PushDescriptor) SetRemoteDescriptor(descriptor distribution.Descriptor) {
+ pd.remoteDescriptor = descriptor
}
func (pd *v2PushDescriptor) Descriptor() distribution.Descriptor {
- // Not necessary to lock pushStatus because this is always
- // called after all the mutation in pushStatus.
- // By the time this function is called, every layer will have
- // an entry in remoteLayers.
- return pd.pushState.remoteLayers[pd.DiffID()]
+ return pd.remoteDescriptor
}
// layerAlreadyExists checks if the registry already know about any of the
diff --git a/components/engine/distribution/registry.go b/components/engine/distribution/registry.go
index 4a8988f13f..4946c0b573 100644
--- a/components/engine/distribution/registry.go
+++ b/components/engine/distribution/registry.go
@@ -5,39 +5,19 @@ import (
"net"
"net/http"
"net/url"
- "strings"
- "syscall"
"time"
"github.com/docker/distribution"
distreference "github.com/docker/distribution/reference"
- "github.com/docker/distribution/registry/api/errcode"
"github.com/docker/distribution/registry/client"
"github.com/docker/distribution/registry/client/auth"
"github.com/docker/distribution/registry/client/transport"
- "github.com/docker/docker/distribution/xfer"
"github.com/docker/docker/dockerversion"
"github.com/docker/docker/registry"
"github.com/docker/engine-api/types"
"golang.org/x/net/context"
)
-// fallbackError wraps an error that can possibly allow fallback to a different
-// endpoint.
-type fallbackError struct {
- // err is the error being wrapped.
- err error
- // confirmedV2 is set to true if it was confirmed that the registry
- // supports the v2 protocol. This is used to limit fallbacks to the v1
- // protocol.
- confirmedV2 bool
-}
-
-// Error renders the FallbackError as a string.
-func (f fallbackError) Error() string {
- return f.err.Error()
-}
-
type dumbCredentialStore struct {
auth *types.AuthConfig
}
@@ -72,41 +52,19 @@ func NewV2Repository(ctx context.Context, repoInfo *registry.RepositoryInfo, end
modifiers := registry.DockerHeaders(dockerversion.DockerUserAgent(), metaHeaders)
authTransport := transport.NewTransport(base, modifiers...)
- pingClient := &http.Client{
- Transport: authTransport,
- Timeout: 15 * time.Second,
- }
- endpointStr := strings.TrimRight(endpoint.URL, "/") + "/v2/"
- req, err := http.NewRequest("GET", endpointStr, nil)
+
+ challengeManager, foundVersion, err := registry.PingV2Registry(endpoint, authTransport)
if err != nil {
- return nil, false, err
- }
- resp, err := pingClient.Do(req)
- if err != nil {
- return nil, false, err
- }
- defer resp.Body.Close()
-
- v2Version := auth.APIVersion{
- Type: "registry",
- Version: "2.0",
- }
-
- versions := auth.APIVersions(resp, registry.DefaultRegistryVersionHeader)
- for _, pingVersion := range versions {
- if pingVersion == v2Version {
- // The version header indicates we're definitely
- // talking to a v2 registry. So don't allow future
- // fallbacks to the v1 protocol.
-
- foundVersion = true
- break
+ transportOK := false
+ if responseErr, ok := err.(registry.PingResponseError); ok {
+ transportOK = true
+ err = responseErr.Err
+ }
+ return nil, foundVersion, fallbackError{
+ err: err,
+ confirmedV2: foundVersion,
+ transportOK: transportOK,
}
- }
-
- challengeManager := auth.NewSimpleChallengeManager()
- if err := challengeManager.AddResponse(resp); err != nil {
- return nil, foundVersion, err
}
if authConfig.RegistryToken != "" {
@@ -122,11 +80,22 @@ func NewV2Repository(ctx context.Context, repoInfo *registry.RepositoryInfo, end
repoNameRef, err := distreference.ParseNamed(repoName)
if err != nil {
- return nil, foundVersion, err
+ return nil, foundVersion, fallbackError{
+ err: err,
+ confirmedV2: foundVersion,
+ transportOK: true,
+ }
}
- repo, err = client.NewRepository(ctx, repoNameRef, endpoint.URL, tr)
- return repo, foundVersion, err
+ repo, err = client.NewRepository(ctx, repoNameRef, endpoint.URL.String(), tr)
+ if err != nil {
+ err = fallbackError{
+ err: err,
+ confirmedV2: foundVersion,
+ transportOK: true,
+ }
+ }
+ return
}
type existingTokenHandler struct {
@@ -141,30 +110,3 @@ func (th *existingTokenHandler) AuthorizeRequest(req *http.Request, params map[s
req.Header.Set("Authorization", fmt.Sprintf("Bearer %s", th.token))
return nil
}
-
-// retryOnError wraps the error in xfer.DoNotRetry if we should not retry the
-// operation after this error.
-func retryOnError(err error) error {
- switch v := err.(type) {
- case errcode.Errors:
- return retryOnError(v[0])
- case errcode.Error:
- switch v.Code {
- case errcode.ErrorCodeUnauthorized, errcode.ErrorCodeUnsupported, errcode.ErrorCodeDenied:
- return xfer.DoNotRetry{Err: err}
- }
- case *url.Error:
- return retryOnError(v.Err)
- case *client.UnexpectedHTTPResponseError:
- return xfer.DoNotRetry{Err: err}
- case error:
- if strings.Contains(err.Error(), strings.ToLower(syscall.ENOSPC.Error())) {
- return xfer.DoNotRetry{Err: err}
- }
- }
- // let's be nice and fallback if the error is a completely
- // unexpected one.
- // If new errors have to be handled in some way, please
- // add them to the switch above.
- return err
-}
diff --git a/components/engine/distribution/registry_unit_test.go b/components/engine/distribution/registry_unit_test.go
index 0702232943..b60a465d78 100644
--- a/components/engine/distribution/registry_unit_test.go
+++ b/components/engine/distribution/registry_unit_test.go
@@ -3,6 +3,7 @@ package distribution
import (
"net/http"
"net/http/httptest"
+ "net/url"
"os"
"strings"
"testing"
@@ -43,9 +44,14 @@ func testTokenPassThru(t *testing.T, ts *httptest.Server) {
}
defer os.RemoveAll(tmp)
+ uri, err := url.Parse(ts.URL)
+ if err != nil {
+ t.Fatalf("could not parse url from test server: %v", err)
+ }
+
endpoint := registry.APIEndpoint{
Mirror: false,
- URL: ts.URL,
+ URL: uri,
Version: 2,
Official: false,
TrimHostname: false,
diff --git a/components/engine/distribution/xfer/download.go b/components/engine/distribution/xfer/download.go
index 69c8bad031..2536f1dd23 100644
--- a/components/engine/distribution/xfer/download.go
+++ b/components/engine/distribution/xfer/download.go
@@ -59,6 +59,10 @@ type DownloadDescriptor interface {
DiffID() (layer.DiffID, error)
// Download is called to perform the download.
Download(ctx context.Context, progressOutput progress.Output) (io.ReadCloser, int64, error)
+ // Close is called when the download manager is finished with this
+ // descriptor and will not call Download again or read from the reader
+ // that Download returned.
+ Close()
}
// DownloadDescriptorWithRegistered is a DownloadDescriptor that has an
@@ -229,6 +233,8 @@ func (ldm *LayerDownloadManager) makeDownloadFunc(descriptor DownloadDescriptor,
retries int
)
+ defer descriptor.Close()
+
for {
downloadReader, size, err = descriptor.Download(d.Transfer.Context(), progressOutput)
if err == nil {
diff --git a/components/engine/distribution/xfer/download_test.go b/components/engine/distribution/xfer/download_test.go
index 6dc6708531..9be9a24a3b 100644
--- a/components/engine/distribution/xfer/download_test.go
+++ b/components/engine/distribution/xfer/download_test.go
@@ -5,6 +5,7 @@ import (
"errors"
"io"
"io/ioutil"
+ "runtime"
"sync/atomic"
"testing"
"time"
@@ -199,6 +200,9 @@ func (d *mockDownloadDescriptor) Download(ctx context.Context, progressOutput pr
return d.mockTarStream(), 0, nil
}
+func (d *mockDownloadDescriptor) Close() {
+}
+
func downloadDescriptors(currentDownloads *int32) []DownloadDescriptor {
return []DownloadDescriptor{
&mockDownloadDescriptor{
@@ -236,6 +240,10 @@ func downloadDescriptors(currentDownloads *int32) []DownloadDescriptor {
}
func TestSuccessfulDownload(t *testing.T) {
+ // TODO Windows: Fix this unit text
+ if runtime.GOOS == "windows" {
+ t.Skip("Needs fixing on Windows")
+ }
layerStore := &mockLayerStore{make(map[layer.ChainID]*mockLayer)}
ldm := NewLayerDownloadManager(layerStore, maxDownloadConcurrency)
diff --git a/components/engine/distribution/xfer/upload.go b/components/engine/distribution/xfer/upload.go
index 8da6a89e39..20fe045ac6 100644
--- a/components/engine/distribution/xfer/upload.go
+++ b/components/engine/distribution/xfer/upload.go
@@ -5,6 +5,7 @@ import (
"time"
"github.com/Sirupsen/logrus"
+ "github.com/docker/distribution"
"github.com/docker/docker/layer"
"github.com/docker/docker/pkg/progress"
"golang.org/x/net/context"
@@ -28,8 +29,8 @@ func NewLayerUploadManager(concurrencyLimit int) *LayerUploadManager {
type uploadTransfer struct {
Transfer
- diffID layer.DiffID
- err error
+ remoteDescriptor distribution.Descriptor
+ err error
}
// An UploadDescriptor references a layer that may need to be uploaded.
@@ -41,7 +42,12 @@ type UploadDescriptor interface {
// DiffID should return the DiffID for this layer.
DiffID() layer.DiffID
// Upload is called to perform the Upload.
- Upload(ctx context.Context, progressOutput progress.Output) error
+ Upload(ctx context.Context, progressOutput progress.Output) (distribution.Descriptor, error)
+ // SetRemoteDescriptor provides the distribution.Descriptor that was
+ // returned by Upload. This descriptor is not to be confused with
+ // the UploadDescriptor interface, which is used for internally
+ // identifying layers that are being uploaded.
+ SetRemoteDescriptor(descriptor distribution.Descriptor)
}
// Upload is a blocking function which ensures the listed layers are present on
@@ -50,7 +56,7 @@ type UploadDescriptor interface {
func (lum *LayerUploadManager) Upload(ctx context.Context, layers []UploadDescriptor, progressOutput progress.Output) error {
var (
uploads []*uploadTransfer
- dedupDescriptors = make(map[string]struct{})
+ dedupDescriptors = make(map[string]*uploadTransfer)
)
for _, descriptor := range layers {
@@ -60,12 +66,12 @@ func (lum *LayerUploadManager) Upload(ctx context.Context, layers []UploadDescri
if _, present := dedupDescriptors[key]; present {
continue
}
- dedupDescriptors[key] = struct{}{}
xferFunc := lum.makeUploadFunc(descriptor)
upload, watcher := lum.tm.Transfer(descriptor.Key(), xferFunc, progressOutput)
defer upload.Release(watcher)
uploads = append(uploads, upload.(*uploadTransfer))
+ dedupDescriptors[key] = upload.(*uploadTransfer)
}
for _, upload := range uploads {
@@ -78,6 +84,9 @@ func (lum *LayerUploadManager) Upload(ctx context.Context, layers []UploadDescri
}
}
}
+ for _, l := range layers {
+ l.SetRemoteDescriptor(dedupDescriptors[l.Key()].remoteDescriptor)
+ }
return nil
}
@@ -86,7 +95,6 @@ func (lum *LayerUploadManager) makeUploadFunc(descriptor UploadDescriptor) DoFun
return func(progressChan chan<- progress.Progress, start <-chan struct{}, inactive chan<- struct{}) Transfer {
u := &uploadTransfer{
Transfer: NewTransfer(),
- diffID: descriptor.DiffID(),
}
go func() {
@@ -105,8 +113,9 @@ func (lum *LayerUploadManager) makeUploadFunc(descriptor UploadDescriptor) DoFun
retries := 0
for {
- err := descriptor.Upload(u.Transfer.Context(), progressOutput)
+ remoteDescriptor, err := descriptor.Upload(u.Transfer.Context(), progressOutput)
if err == nil {
+ u.remoteDescriptor = remoteDescriptor
break
}
diff --git a/components/engine/distribution/xfer/upload_test.go b/components/engine/distribution/xfer/upload_test.go
index d87dfcaa25..275d24268d 100644
--- a/components/engine/distribution/xfer/upload_test.go
+++ b/components/engine/distribution/xfer/upload_test.go
@@ -6,6 +6,7 @@ import (
"testing"
"time"
+ "github.com/docker/distribution"
"github.com/docker/distribution/digest"
"github.com/docker/docker/layer"
"github.com/docker/docker/pkg/progress"
@@ -35,13 +36,17 @@ func (u *mockUploadDescriptor) DiffID() layer.DiffID {
return u.diffID
}
+// SetRemoteDescriptor is not used in the mock.
+func (u *mockUploadDescriptor) SetRemoteDescriptor(remoteDescriptor distribution.Descriptor) {
+}
+
// Upload is called to perform the upload.
-func (u *mockUploadDescriptor) Upload(ctx context.Context, progressOutput progress.Output) error {
+func (u *mockUploadDescriptor) Upload(ctx context.Context, progressOutput progress.Output) (distribution.Descriptor, error) {
if u.currentUploads != nil {
defer atomic.AddInt32(u.currentUploads, -1)
if atomic.AddInt32(u.currentUploads, 1) > maxUploadConcurrency {
- return errors.New("concurrency limit exceeded")
+ return distribution.Descriptor{}, errors.New("concurrency limit exceeded")
}
}
@@ -49,7 +54,7 @@ func (u *mockUploadDescriptor) Upload(ctx context.Context, progressOutput progre
for i := int64(0); i <= 10; i++ {
select {
case <-ctx.Done():
- return ctx.Err()
+ return distribution.Descriptor{}, ctx.Err()
case <-time.After(10 * time.Millisecond):
progressOutput.WriteProgress(progress.Progress{ID: u.ID(), Current: i, Total: 10})
}
@@ -57,10 +62,10 @@ func (u *mockUploadDescriptor) Upload(ctx context.Context, progressOutput progre
if u.simulateRetries != 0 {
u.simulateRetries--
- return errors.New("simulating retry")
+ return distribution.Descriptor{}, errors.New("simulating retry")
}
- return nil
+ return distribution.Descriptor{}, nil
}
func uploadDescriptors(currentUploads *int32) []UploadDescriptor {
diff --git a/components/engine/docker/daemon.go b/components/engine/docker/daemon.go
index 821b3f993b..efb459f6f9 100644
--- a/components/engine/docker/daemon.go
+++ b/components/engine/docker/daemon.go
@@ -14,10 +14,19 @@ import (
"github.com/Sirupsen/logrus"
"github.com/docker/distribution/uuid"
apiserver "github.com/docker/docker/api/server"
+ "github.com/docker/docker/api/server/router"
+ "github.com/docker/docker/api/server/router/build"
+ "github.com/docker/docker/api/server/router/container"
+ "github.com/docker/docker/api/server/router/image"
+ "github.com/docker/docker/api/server/router/network"
+ systemrouter "github.com/docker/docker/api/server/router/system"
+ "github.com/docker/docker/api/server/router/volume"
+ "github.com/docker/docker/builder/dockerfile"
"github.com/docker/docker/cli"
"github.com/docker/docker/cliconfig"
"github.com/docker/docker/daemon"
"github.com/docker/docker/daemon/logger"
+ "github.com/docker/docker/docker/listeners"
"github.com/docker/docker/dockerversion"
"github.com/docker/docker/opts"
"github.com/docker/docker/pkg/jsonlog"
@@ -226,6 +235,9 @@ func (cli *DaemonCli) CmdDaemon(args ...string) error {
if len(cli.Config.Hosts) == 0 {
cli.Config.Hosts = make([]string, 1)
}
+
+ api := apiserver.New(serverConfig)
+
for i := 0; i < len(cli.Config.Hosts); i++ {
var err error
if cli.Config.Hosts[i], err = opts.ParseHost(cli.Config.TLS, cli.Config.Hosts[i]); err != nil {
@@ -237,12 +249,13 @@ func (cli *DaemonCli) CmdDaemon(args ...string) error {
if len(protoAddrParts) != 2 {
logrus.Fatalf("bad format %s, expected PROTO://ADDR", protoAddr)
}
- serverConfig.Addrs = append(serverConfig.Addrs, apiserver.Addr{Proto: protoAddrParts[0], Addr: protoAddrParts[1]})
- }
+ l, err := listeners.Init(protoAddrParts[0], protoAddrParts[1], serverConfig.SocketGroup, serverConfig.TLSConfig)
+ if err != nil {
+ logrus.Fatal(err)
+ }
- api, err := apiserver.New(serverConfig)
- if err != nil {
- logrus.Fatal(err)
+ logrus.Debugf("Listener created for HTTP on %s (%s)", protoAddrParts[0], protoAddrParts[1])
+ api.Accept(protoAddrParts[1], l...)
}
if err := migrateKey(); err != nil {
@@ -270,14 +283,25 @@ func (cli *DaemonCli) CmdDaemon(args ...string) error {
"graphdriver": d.GraphDriverName(),
}).Info("Docker daemon")
- api.InitRouters(d)
+ initRouter(api, d)
reload := func(config *daemon.Config) {
if err := d.Reload(config); err != nil {
logrus.Errorf("Error reconfiguring the daemon: %v", err)
return
}
- api.Reload(config)
+ if config.IsValueSet("debug") {
+ debugEnabled := utils.IsDebugEnabled()
+ switch {
+ case debugEnabled && !config.Debug: // disable debug
+ utils.DisableDebug()
+ api.DisableProfiler()
+ case config.Debug && !debugEnabled: // enable debug
+ utils.EnableDebug()
+ api.EnableProfiler()
+ }
+
+ }
}
setupConfigReloadTrap(*configFile, cli.flags, reload)
@@ -373,3 +397,18 @@ func loadDaemonCliConfig(config *daemon.Config, daemonFlags *flag.FlagSet, commo
return config, nil
}
+
+func initRouter(s *apiserver.Server, d *daemon.Daemon) {
+ routers := []router.Router{
+ container.NewRouter(d),
+ image.NewRouter(d),
+ systemrouter.NewRouter(d),
+ volume.NewRouter(d),
+ build.NewRouter(dockerfile.NewBuildManager(d)),
+ }
+ if d.NetworkControllerEnabled() {
+ routers = append(routers, network.NewRouter(d))
+ }
+
+ s.InitRouter(utils.IsDebugEnabled(), routers...)
+}
diff --git a/components/engine/docker/daemon_test.go b/components/engine/docker/daemon_test.go
index 5afdfb3bde..322e0b7604 100644
--- a/components/engine/docker/daemon_test.go
+++ b/components/engine/docker/daemon_test.go
@@ -247,47 +247,3 @@ func TestLoadDaemonConfigWithEmbeddedOptions(t *testing.T) {
t.Fatalf("expected LogConfig type syslog, got %v", loadedConfig.LogConfig.Type)
}
}
-
-func TestLoadDaemonConfigWithMapOptions(t *testing.T) {
- c := &daemon.Config{}
- common := &cli.CommonFlags{}
- flags := mflag.NewFlagSet("test", mflag.ContinueOnError)
-
- flags.Var(opts.NewNamedMapOpts("cluster-store-opts", c.ClusterOpts, nil), []string{"-cluster-store-opt"}, "")
- flags.Var(opts.NewNamedMapOpts("log-opts", c.LogConfig.Config, nil), []string{"-log-opt"}, "")
-
- f, err := ioutil.TempFile("", "docker-config-")
- if err != nil {
- t.Fatal(err)
- }
-
- configFile := f.Name()
- f.Write([]byte(`{
- "cluster-store-opts": {"kv.cacertfile": "/var/lib/docker/discovery_certs/ca.pem"},
- "log-opts": {"tag": "test"}
-}`))
- f.Close()
-
- loadedConfig, err := loadDaemonCliConfig(c, flags, common, configFile)
- if err != nil {
- t.Fatal(err)
- }
- if loadedConfig == nil {
- t.Fatal("expected configuration, got nil")
- }
- if loadedConfig.ClusterOpts == nil {
- t.Fatal("expected cluster options, got nil")
- }
-
- expectedPath := "/var/lib/docker/discovery_certs/ca.pem"
- if caPath := loadedConfig.ClusterOpts["kv.cacertfile"]; caPath != expectedPath {
- t.Fatalf("expected %s, got %s", expectedPath, caPath)
- }
-
- if loadedConfig.LogConfig.Config == nil {
- t.Fatal("expected log config options, got nil")
- }
- if tag := loadedConfig.LogConfig.Config["tag"]; tag != "test" {
- t.Fatalf("expected log tag `test`, got %s", tag)
- }
-}
diff --git a/components/engine/docker/daemon_unix.go b/components/engine/docker/daemon_unix.go
index a89bdc73bb..c76700f014 100644
--- a/components/engine/docker/daemon_unix.go
+++ b/components/engine/docker/daemon_unix.go
@@ -8,6 +8,7 @@ import (
"os/signal"
"syscall"
+ "github.com/Sirupsen/logrus"
apiserver "github.com/docker/docker/api/server"
"github.com/docker/docker/daemon"
"github.com/docker/docker/pkg/mflag"
@@ -58,7 +59,9 @@ func setupConfigReloadTrap(configFile string, flags *mflag.FlagSet, reload func(
signal.Notify(c, syscall.SIGHUP)
go func() {
for range c {
- daemon.ReloadConfiguration(configFile, flags, reload)
+ if err := daemon.ReloadConfiguration(configFile, flags, reload); err != nil {
+ logrus.Error(err)
+ }
}
}()
}
diff --git a/components/engine/docker/daemon_unix_test.go b/components/engine/docker/daemon_unix_test.go
index 889482b007..58b692b532 100644
--- a/components/engine/docker/daemon_unix_test.go
+++ b/components/engine/docker/daemon_unix_test.go
@@ -8,6 +8,7 @@ import (
"github.com/docker/docker/cli"
"github.com/docker/docker/daemon"
+ "github.com/docker/docker/opts"
"github.com/docker/docker/pkg/mflag"
)
@@ -41,3 +42,124 @@ func TestLoadDaemonConfigWithNetwork(t *testing.T) {
t.Fatalf("expected DefaultIP 127.0.0.1, got %s", loadedConfig.DefaultIP)
}
}
+
+func TestLoadDaemonConfigWithMapOptions(t *testing.T) {
+ c := &daemon.Config{}
+ common := &cli.CommonFlags{}
+ flags := mflag.NewFlagSet("test", mflag.ContinueOnError)
+
+ flags.Var(opts.NewNamedMapOpts("cluster-store-opts", c.ClusterOpts, nil), []string{"-cluster-store-opt"}, "")
+ flags.Var(opts.NewNamedMapOpts("log-opts", c.LogConfig.Config, nil), []string{"-log-opt"}, "")
+
+ f, err := ioutil.TempFile("", "docker-config-")
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ configFile := f.Name()
+ f.Write([]byte(`{
+ "cluster-store-opts": {"kv.cacertfile": "/var/lib/docker/discovery_certs/ca.pem"},
+ "log-opts": {"tag": "test"}
+}`))
+ f.Close()
+
+ loadedConfig, err := loadDaemonCliConfig(c, flags, common, configFile)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if loadedConfig == nil {
+ t.Fatal("expected configuration, got nil")
+ }
+ if loadedConfig.ClusterOpts == nil {
+ t.Fatal("expected cluster options, got nil")
+ }
+
+ expectedPath := "/var/lib/docker/discovery_certs/ca.pem"
+ if caPath := loadedConfig.ClusterOpts["kv.cacertfile"]; caPath != expectedPath {
+ t.Fatalf("expected %s, got %s", expectedPath, caPath)
+ }
+
+ if loadedConfig.LogConfig.Config == nil {
+ t.Fatal("expected log config options, got nil")
+ }
+ if tag := loadedConfig.LogConfig.Config["tag"]; tag != "test" {
+ t.Fatalf("expected log tag `test`, got %s", tag)
+ }
+}
+
+func TestLoadDaemonConfigWithTrueDefaultValues(t *testing.T) {
+ c := &daemon.Config{}
+ common := &cli.CommonFlags{}
+ flags := mflag.NewFlagSet("test", mflag.ContinueOnError)
+ flags.BoolVar(&c.EnableUserlandProxy, []string{"-userland-proxy"}, true, "")
+
+ f, err := ioutil.TempFile("", "docker-config-")
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ if err := flags.ParseFlags([]string{}, false); err != nil {
+ t.Fatal(err)
+ }
+
+ configFile := f.Name()
+ f.Write([]byte(`{
+ "userland-proxy": false
+}`))
+ f.Close()
+
+ loadedConfig, err := loadDaemonCliConfig(c, flags, common, configFile)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if loadedConfig == nil {
+ t.Fatal("expected configuration, got nil")
+ }
+
+ if loadedConfig.EnableUserlandProxy {
+ t.Fatal("expected userland proxy to be disabled, got enabled")
+ }
+
+ // make sure reloading doesn't generate configuration
+ // conflicts after normalizing boolean values.
+ err = daemon.ReloadConfiguration(configFile, flags, func(reloadedConfig *daemon.Config) {
+ if reloadedConfig.EnableUserlandProxy {
+ t.Fatal("expected userland proxy to be disabled, got enabled")
+ }
+ })
+ if err != nil {
+ t.Fatal(err)
+ }
+}
+
+func TestLoadDaemonConfigWithTrueDefaultValuesLeaveDefaults(t *testing.T) {
+ c := &daemon.Config{}
+ common := &cli.CommonFlags{}
+ flags := mflag.NewFlagSet("test", mflag.ContinueOnError)
+ flags.BoolVar(&c.EnableUserlandProxy, []string{"-userland-proxy"}, true, "")
+
+ f, err := ioutil.TempFile("", "docker-config-")
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ if err := flags.ParseFlags([]string{}, false); err != nil {
+ t.Fatal(err)
+ }
+
+ configFile := f.Name()
+ f.Write([]byte(`{}`))
+ f.Close()
+
+ loadedConfig, err := loadDaemonCliConfig(c, flags, common, configFile)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if loadedConfig == nil {
+ t.Fatal("expected configuration, got nil")
+ }
+
+ if !loadedConfig.EnableUserlandProxy {
+ t.Fatal("expected userland proxy to be enabled, got disabled")
+ }
+}
diff --git a/components/engine/docker/daemon_windows.go b/components/engine/docker/daemon_windows.go
index 307bbcc39b..52649daf0b 100644
--- a/components/engine/docker/daemon_windows.go
+++ b/components/engine/docker/daemon_windows.go
@@ -50,7 +50,9 @@ func setupConfigReloadTrap(configFile string, flags *mflag.FlagSet, reload func(
logrus.Debugf("Config reload - waiting signal at %s", ev)
for {
syscall.WaitForSingleObject(h, syscall.INFINITE)
- daemon.ReloadConfiguration(configFile, flags, reload)
+ if err := daemon.ReloadConfiguration(configFile, flags, reload); err != nil {
+ logrus.Error(err)
+ }
}
}
}()
diff --git a/components/engine/docker/listeners/listeners.go b/components/engine/docker/listeners/listeners.go
new file mode 100644
index 0000000000..8150ba0c23
--- /dev/null
+++ b/components/engine/docker/listeners/listeners.go
@@ -0,0 +1,22 @@
+package listeners
+
+import (
+ "crypto/tls"
+ "net"
+
+ "github.com/Sirupsen/logrus"
+ "github.com/docker/go-connections/sockets"
+)
+
+func initTCPSocket(addr string, tlsConfig *tls.Config) (l net.Listener, err error) {
+ if tlsConfig == nil || tlsConfig.ClientAuth != tls.RequireAndVerifyClientCert {
+ logrus.Warn("/!\\ DON'T BIND ON ANY IP ADDRESS WITHOUT setting -tlsverify IF YOU DON'T KNOW WHAT YOU'RE DOING /!\\")
+ }
+ if l, err = sockets.NewTCPSocket(addr, tlsConfig); err != nil {
+ return nil, err
+ }
+ if err := allocateDaemonPort(addr); err != nil {
+ return nil, err
+ }
+ return
+}
diff --git a/components/engine/api/server/server_unix.go b/components/engine/docker/listeners/listeners_unix.go
similarity index 73%
rename from components/engine/api/server/server_unix.go
rename to components/engine/docker/listeners/listeners_unix.go
index a4fc639575..732565a386 100644
--- a/components/engine/api/server/server_unix.go
+++ b/components/engine/docker/listeners/listeners_unix.go
@@ -1,42 +1,35 @@
-// +build freebsd linux
+// +build !windows
-package server
+package listeners
import (
"crypto/tls"
"fmt"
"net"
- "net/http"
"strconv"
"github.com/Sirupsen/logrus"
+ "github.com/coreos/go-systemd/activation"
"github.com/docker/go-connections/sockets"
"github.com/docker/libnetwork/portallocator"
-
- systemdActivation "github.com/coreos/go-systemd/activation"
)
-// newServer sets up the required HTTPServers and does protocol specific checking.
-// newServer does not set any muxers, you should set it later to Handler field
-func (s *Server) newServer(proto, addr string) ([]*HTTPServer, error) {
- var (
- err error
- ls []net.Listener
- )
+// Init creates new listeners for the server.
+func Init(proto, addr, socketGroup string, tlsConfig *tls.Config) (ls []net.Listener, err error) {
switch proto {
case "fd":
- ls, err = listenFD(addr, s.cfg.TLSConfig)
+ ls, err = listenFD(addr, tlsConfig)
if err != nil {
return nil, err
}
case "tcp":
- l, err := s.initTCPSocket(addr)
+ l, err := initTCPSocket(addr, tlsConfig)
if err != nil {
return nil, err
}
ls = append(ls, l)
case "unix":
- l, err := sockets.NewUnixSocket(addr, s.cfg.SocketGroup)
+ l, err := sockets.NewUnixSocket(addr, socketGroup)
if err != nil {
return nil, fmt.Errorf("can't create unix socket %s: %v", addr, err)
}
@@ -44,43 +37,8 @@ func (s *Server) newServer(proto, addr string) ([]*HTTPServer, error) {
default:
return nil, fmt.Errorf("Invalid protocol format: %q", proto)
}
- var res []*HTTPServer
- for _, l := range ls {
- res = append(res, &HTTPServer{
- &http.Server{
- Addr: addr,
- },
- l,
- })
- }
- return res, nil
-}
-func allocateDaemonPort(addr string) error {
- host, port, err := net.SplitHostPort(addr)
- if err != nil {
- return err
- }
-
- intPort, err := strconv.Atoi(port)
- if err != nil {
- return err
- }
-
- var hostIPs []net.IP
- if parsedIP := net.ParseIP(host); parsedIP != nil {
- hostIPs = append(hostIPs, parsedIP)
- } else if hostIPs, err = net.LookupIP(host); err != nil {
- return fmt.Errorf("failed to lookup %s address in host specification", host)
- }
-
- pa := portallocator.Get()
- for _, hostIP := range hostIPs {
- if _, err := pa.RequestPort(hostIP, "tcp", intPort); err != nil {
- return fmt.Errorf("failed to allocate daemon listening port %d (err: %v)", intPort, err)
- }
- }
- return nil
+ return
}
// listenFD returns the specified socket activated files as a slice of
@@ -92,16 +50,16 @@ func listenFD(addr string, tlsConfig *tls.Config) ([]net.Listener, error) {
)
// socket activation
if tlsConfig != nil {
- listeners, err = systemdActivation.TLSListeners(false, tlsConfig)
+ listeners, err = activation.TLSListeners(false, tlsConfig)
} else {
- listeners, err = systemdActivation.Listeners(false)
+ listeners, err = activation.Listeners(false)
}
if err != nil {
return nil, err
}
if len(listeners) == 0 {
- return nil, fmt.Errorf("No sockets found")
+ return nil, fmt.Errorf("No sockets found. Make sure the docker daemon was started by systemd.")
}
// default to all fds just like unix:// and tcp://
@@ -130,3 +88,32 @@ func listenFD(addr string, tlsConfig *tls.Config) ([]net.Listener, error) {
}
return []net.Listener{listeners[fdOffset]}, nil
}
+
+// allocateDaemonPort ensures that there are no containers
+// that try to use any port allocated for the docker server.
+func allocateDaemonPort(addr string) error {
+ host, port, err := net.SplitHostPort(addr)
+ if err != nil {
+ return err
+ }
+
+ intPort, err := strconv.Atoi(port)
+ if err != nil {
+ return err
+ }
+
+ var hostIPs []net.IP
+ if parsedIP := net.ParseIP(host); parsedIP != nil {
+ hostIPs = append(hostIPs, parsedIP)
+ } else if hostIPs, err = net.LookupIP(host); err != nil {
+ return fmt.Errorf("failed to lookup %s address in host specification", host)
+ }
+
+ pa := portallocator.Get()
+ for _, hostIP := range hostIPs {
+ if _, err := pa.RequestPort(hostIP, "tcp", intPort); err != nil {
+ return fmt.Errorf("failed to allocate daemon listening port %d (err: %v)", intPort, err)
+ }
+ }
+ return nil
+}
diff --git a/components/engine/api/server/server_windows.go b/components/engine/docker/listeners/listeners_windows.go
similarity index 59%
rename from components/engine/api/server/server_windows.go
rename to components/engine/docker/listeners/listeners_windows.go
index 613d185522..282b285256 100644
--- a/components/engine/api/server/server_windows.go
+++ b/components/engine/docker/listeners/listeners_windows.go
@@ -1,24 +1,20 @@
-// +build windows
-
-package server
+package listeners
import (
+ "crypto/tls"
"errors"
"fmt"
- "github.com/Microsoft/go-winio"
"net"
- "net/http"
"strings"
+
+ "github.com/Microsoft/go-winio"
)
-// NewServer sets up the required Server and does protocol specific checking.
-func (s *Server) newServer(proto, addr string) ([]*HTTPServer, error) {
- var (
- ls []net.Listener
- )
+// Init creates new listeners for the server.
+func Init(proto, addr, socketGroup string, tlsConfig *tls.Config) (ls []net.Listener, err error) {
switch proto {
case "tcp":
- l, err := s.initTCPSocket(addr)
+ l, err := initTCPSocket(addr, tlsConfig)
if err != nil {
return nil, err
}
@@ -27,8 +23,8 @@ func (s *Server) newServer(proto, addr string) ([]*HTTPServer, error) {
case "npipe":
// allow Administrators and SYSTEM, plus whatever additional users or groups were specified
sddl := "D:P(A;;GA;;;BA)(A;;GA;;;SY)"
- if s.cfg.SocketGroup != "" {
- for _, g := range strings.Split(s.cfg.SocketGroup, ",") {
+ if socketGroup != "" {
+ for _, g := range strings.Split(socketGroup, ",") {
sid, err := winio.LookupSidByName(g)
if err != nil {
return nil, err
@@ -46,19 +42,11 @@ func (s *Server) newServer(proto, addr string) ([]*HTTPServer, error) {
return nil, errors.New("Invalid protocol format. Windows only supports tcp and npipe.")
}
- var res []*HTTPServer
- for _, l := range ls {
- res = append(res, &HTTPServer{
- &http.Server{
- Addr: addr,
- },
- l,
- })
- }
- return res, nil
-
+ return
}
+// allocateDaemonPort ensures that there are no containers
+// that try to use any port allocated for the docker server.
func allocateDaemonPort(addr string) error {
return nil
}
diff --git a/components/engine/docs/Dockerfile b/components/engine/docs/Dockerfile
index 2f29b214eb..3690d1572a 100644
--- a/components/engine/docs/Dockerfile
+++ b/components/engine/docs/Dockerfile
@@ -5,8 +5,9 @@ RUN svn checkout https://github.com/docker/compose/trunk/docs /docs/content/comp
RUN svn checkout https://github.com/docker/swarm/trunk/docs /docs/content/swarm
RUN svn checkout https://github.com/docker/machine/trunk/docs /docs/content/machine
RUN svn checkout https://github.com/docker/distribution/trunk/docs /docs/content/registry
-RUN svn checkout https://github.com/kitematic/kitematic/trunk/docs /docs/content/kitematic
-RUN svn checkout https://github.com/docker/tutorials/trunk/docs /docs/content/
+RUN svn checkout https://github.com/docker/notary/trunk/docs /docs/content/notary
+RUN svn checkout https://github.com/docker/kitematic/trunk/docs /docs/content/kitematic
+RUN svn checkout https://github.com/docker/toolbox/trunk/docs /docs/content/toolbox
RUN svn checkout https://github.com/docker/opensource/trunk/docs /docs/content/opensource
ENV PROJECT=engine
diff --git a/components/engine/docs/admin/logging/etwlogs.md b/components/engine/docs/admin/logging/etwlogs.md
new file mode 100644
index 0000000000..5b98fd5495
--- /dev/null
+++ b/components/engine/docs/admin/logging/etwlogs.md
@@ -0,0 +1,69 @@
+
+
+
+# ETW logging driver
+
+The ETW logging driver forwards container logs as ETW events.
+ETW stands for Event Tracing in Windows, and is the common framework
+for tracing applications in Windows. Each ETW event contains a message
+with both the log and its context information. A client can then create
+an ETW listener to listen to these events.
+
+The ETW provider that this logging driver registers with Windows, has the
+GUID identifier of: `{a3693192-9ed6-46d2-a981-f8226c8363bd}`. A client creates an
+ETW listener and registers to listen to events from the logging driver's provider.
+It does not matter the order in which the provider and listener are created.
+A client can create their ETW listener and start listening for events from the provider,
+before the provider has been registered with the system.
+
+## Usage
+
+Here is an example of how to listen to these events using the logman utility program
+included in most installations of Windows:
+
+ 1. `logman start -ets DockerContainerLogs -p {a3693192-9ed6-46d2-a981-f8226c8363bd} 0 0 -o trace.etl`
+ 2. Run your container(s) with the etwlogs driver, by adding `--log-driver=etwlogs`
+ to the Docker run command, and generate log messages.
+ 3. `logman stop -ets DockerContainerLogs`
+ 4. This will generate an etl file that contains the events. One way to convert this file into
+ human-readable form is to run: `tracerpt -y trace.etl`.
+
+Each ETW event will contain a structured message string in this format:
+
+ container_name: %s, image_name: %s, container_id: %s, image_id: %s, source: [stdout | stderr], log: %s
+
+Details on each item in the message can be found below:
+
+| Field | Description |
+-----------------------|-------------------------------------------------|
+| `container_name` | The container name at the time it was started. |
+| `image_name` | The name of the container's image. |
+| `container_id` | The full 64-character container ID. |
+| `image_id` | The full ID of the container's image. |
+| `source` | `stdout` or `stderr`. |
+| `log` | The container log message. |
+
+Here is an example event message:
+
+ container_name: backstabbing_spence,
+ image_name: windowsservercore,
+ container_id: f14bb55aa862d7596b03a33251c1be7dbbec8056bbdead1da8ec5ecebbe29731,
+ image_id: sha256:2f9e19bd998d3565b4f345ac9aaf6e3fc555406239a4fb1b1ba879673713824b,
+ source: stdout,
+ log: Hello world!
+
+A client can parse this message string to get both the log message, as well as its
+context information. Note that the time stamp is also available within the ETW event.
+
+**Note** This ETW provider emits only a message string, and not a specially
+structured ETW event. Therefore, it is not required to register a manifest file
+with the system to read and interpret its ETW events.
diff --git a/components/engine/docs/admin/logging/fluentd.md b/components/engine/docs/admin/logging/fluentd.md
index a87b1dca69..86bddecb72 100644
--- a/components/engine/docs/admin/logging/fluentd.md
+++ b/components/engine/docs/admin/logging/fluentd.md
@@ -101,7 +101,7 @@ and [its documents](http://docs.fluentd.org/).
To use this logging driver, start the `fluentd` daemon on a host. We recommend
that you use [the Fluentd docker
-image](https://registry.hub.docker.com/u/fluent/fluentd/). This image is
+image](https://hub.docker.com/r/fluent/fluentd/). This image is
especially useful if you want to aggregate multiple container logs on a each
host then, later, transfer the logs to another Fluentd node to create an
aggregate store.
diff --git a/components/engine/docs/admin/logging/gcplogs.md b/components/engine/docs/admin/logging/gcplogs.md
new file mode 100644
index 0000000000..08fd858da0
--- /dev/null
+++ b/components/engine/docs/admin/logging/gcplogs.md
@@ -0,0 +1,70 @@
+
+
+# Google Cloud Logging driver
+
+The Google Cloud Logging driver sends container logs to Google Cloud
+Logging.
+
+## Usage
+
+You can configure the default logging driver by passing the `--log-driver`
+option to the Docker daemon:
+
+ docker daemon --log-driver=gcplogs
+
+You can set the logging driver for a specific container by using the
+`--log-driver` option to `docker run`:
+
+ docker run --log-driver=gcplogs ...
+
+This log driver does not implement a reader so it is incompatible with
+`docker logs`.
+
+If Docker detects that it is running in a Google Cloud Project, it will discover configuration
+from the instance metadata service.
+Otherwise, the user must specify which project to log to using the `--gcp-project`
+log option and Docker will attempt to obtain credentials from the
+Google Application Default Credential.
+The `--gcp-project` takes precedence over information discovered from the metadata server
+so a Docker daemon running in a Google Cloud Project can be overriden to log to a different
+Google Cloud Project using `--gcp-project`.
+
+## gcplogs options
+
+You can use the `--log-opt NAME=VALUE` flag to specify these additional Google
+Cloud Logging driver options:
+
+| Option | Required | Description |
+|-----------------------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------|
+| `gcp-project` | optional | Which GCP project to log to. Defaults to discovering this value from the GCE metadata service. |
+| `gcp-log-cmd` | optional | Whether to log the command that the container was started with. Defaults to false. |
+| `labels` | optional | Comma-separated list of keys of labels, which should be included in message, if these labels are specified for container. |
+| `env` | optional | Comma-separated list of keys of environment variables, which should be included in message, if these variables are specified for container. |
+
+If there is collision between `label` and `env` keys, the value of the `env`
+takes precedence. Both options add additional fields to the attributes of a
+logging message.
+
+Below is an example of the logging options required to log to the default
+logging destination which is discovered by querying the GCE metadata server.
+
+ docker run --log-driver=gcplogs \
+ --log-opt labels=location
+ --log-opt env=TEST
+ --log-opt gcp-log-cmd=true
+ --env "TEST=false"
+ --label location=west
+ your/application
+
+This configuration also directs the driver to include in the payload the label
+`location`, the environment variable `ENV`, and the command used to start the
+container.
diff --git a/components/engine/docs/admin/logging/index.md b/components/engine/docs/admin/logging/index.md
index 64f9c526aa..4300565e8f 100644
--- a/components/engine/docs/admin/logging/index.md
+++ b/components/engine/docs/admin/logging/index.md
@@ -20,3 +20,4 @@ weight=8
* [Journald logging driver](journald.md)
* [Amazon CloudWatch Logs logging driver](awslogs.md)
* [Splunk logging driver](splunk.md)
+* [ETW logging driver](etwlogs.md)
diff --git a/components/engine/docs/admin/logging/overview.md b/components/engine/docs/admin/logging/overview.md
index 531b338ed0..e3d3d11256 100644
--- a/components/engine/docs/admin/logging/overview.md
+++ b/components/engine/docs/admin/logging/overview.md
@@ -26,6 +26,8 @@ container's logging driver. The following options are supported:
| `fluentd` | Fluentd logging driver for Docker. Writes log messages to `fluentd` (forward input). |
| `awslogs` | Amazon CloudWatch Logs logging driver for Docker. Writes log messages to Amazon CloudWatch Logs. |
| `splunk` | Splunk logging driver for Docker. Writes log messages to `splunk` using HTTP Event Collector. |
+| `etwlogs` | ETW logging driver for Docker on Windows. Writes log messages as ETW events. |
+| `gcplogs` | Google Cloud Logging driver for Docker. Writes log messages to Google Cloud Logging. |
The `docker logs`command is available only for the `json-file` and `journald`
logging drivers.
@@ -204,3 +206,22 @@ The Splunk logging driver requires the following options:
For detailed information about working with this logging driver, see the [Splunk logging driver](splunk.md)
reference documentation.
+
+## ETW logging driver options
+
+The etwlogs logging driver does not require any options to be specified. This logging driver will forward each log message
+as an ETW event. An ETW listener can then be created to listen for these events.
+
+For detailed information on working with this logging driver, see [the ETW logging driver](etwlogs.md) reference documentation.
+
+## Google Cloud Logging
+
+The Google Cloud Logging driver supports the following options:
+
+ --log-opt gcp-project=
+ --log-opt labels=,
+ --log-opt env=,
+ --log-opt log-cmd=true
+
+For detailed information about working with this logging driver, see the [Google Cloud Logging driver](gcplogs.md).
+reference documentation.
diff --git a/components/engine/docs/deprecated.md b/components/engine/docs/deprecated.md
index 4ed127f953..7a351ac176 100644
--- a/components/engine/docs/deprecated.md
+++ b/components/engine/docs/deprecated.md
@@ -14,6 +14,13 @@ weight=80
The following list of features are deprecated in Engine.
+### `-e` and `--email` flags on `docker login`
+**Deprecated In Release: v1.11**
+
+**Target For Removal In Release: v1.13**
+
+The docker login command is removing the ability to automatically register for an account with the target registry if the given username doesn't exist. Due to this change, the email flag is no longer required, and will be deprecated.
+
### Ambiguous event fields in API
**Deprecated In Release: v1.10**
diff --git a/components/engine/docs/examples/mongodb.md b/components/engine/docs/examples/mongodb.md
index f6498e0cc5..3173aa1b7e 100644
--- a/components/engine/docs/examples/mongodb.md
+++ b/components/engine/docs/examples/mongodb.md
@@ -17,7 +17,7 @@ MongoDB pre-installed. We'll also see how to `push` that image to the
[Docker Hub registry](https://hub.docker.com) and share it with others!
> **Note:** This guide will show the mechanics of building a MongoDB container, but
-> you will probably want to use the official image on [Docker Hub]( https://registry.hub.docker.com/_/mongo/)
+> you will probably want to use the official image on [Docker Hub]( https://hub.docker.com/_/mongo/)
Using Docker and containers for deploying [MongoDB](https://www.mongodb.org/)
instances will bring several benefits, such as:
@@ -49,7 +49,7 @@ Although optional, it is handy to have comments at the beginning of a
> the *parent* of your *Dockerized MongoDB* image.
We will build our image using the latest version of Ubuntu from the
-[Docker Hub Ubuntu](https://registry.hub.docker.com/_/ubuntu/) repository.
+[Docker Hub Ubuntu](https://hub.docker.com/_/ubuntu/) repository.
# Format: FROM repository[:version]
FROM ubuntu:latest
diff --git a/components/engine/docs/examples/nodejs_web_app.md b/components/engine/docs/examples/nodejs_web_app.md
index 55425c0672..149f5b47e4 100644
--- a/components/engine/docs/examples/nodejs_web_app.md
+++ b/components/engine/docs/examples/nodejs_web_app.md
@@ -67,7 +67,7 @@ Open the `Dockerfile` in your favorite text editor
Define the parent image you want to use to build your own image on
top of. Here, we'll use
-[CentOS](https://registry.hub.docker.com/_/centos/) (tag: `centos6`)
+[CentOS](https://hub.docker.com/_/centos/) (tag: `centos6`)
available on the [Docker Hub](https://hub.docker.com/):
FROM centos:centos6
@@ -89,7 +89,7 @@ Install your app dependencies using the `npm` binary:
# Install app dependencies
COPY package.json /src/package.json
- RUN cd /src; npm install
+ RUN cd /src; npm install --production
To bundle your app's source code inside the Docker image, use the `COPY`
instruction:
@@ -119,7 +119,7 @@ Your `Dockerfile` should now look like this:
# Install app dependencies
COPY package.json /src/package.json
- RUN cd /src; npm install
+ RUN cd /src; npm install --production
# Bundle app source
COPY . /src
diff --git a/components/engine/docs/examples/running_riak_service.md b/components/engine/docs/examples/running_riak_service.md
index a6c3d3f4d4..f17969fe48 100644
--- a/components/engine/docs/examples/running_riak_service.md
+++ b/components/engine/docs/examples/running_riak_service.md
@@ -20,7 +20,7 @@ Create an empty file called `Dockerfile`:
$ touch Dockerfile
Next, define the parent image you want to use to build your image on top
-of. We'll use [Ubuntu](https://registry.hub.docker.com/_/ubuntu/) (tag:
+of. We'll use [Ubuntu](https://hub.docker.com/_/ubuntu/) (tag:
`trusty`), which is available on [Docker Hub](https://hub.docker.com):
# Riak
diff --git a/components/engine/docs/extend/index.md b/components/engine/docs/extend/index.md
index f491926e9a..8a061e4e29 100644
--- a/components/engine/docs/extend/index.md
+++ b/components/engine/docs/extend/index.md
@@ -18,5 +18,5 @@ Currently, you can extend Docker Engine by adding a plugin. This section contain
* [Understand Docker plugins](plugins.md)
* [Write a volume plugin](plugins_volume.md)
* [Write a network plugin](plugins_network.md)
-* [Write an authorization plugin](authorization.md)
+* [Write an authorization plugin](plugins_authorization.md)
* [Docker plugin API](plugin_api.md)
diff --git a/components/engine/docs/extend/plugin_api.md b/components/engine/docs/extend/plugin_api.md
index c6793a3dda..1a4237e2c7 100644
--- a/components/engine/docs/extend/plugin_api.md
+++ b/components/engine/docs/extend/plugin_api.md
@@ -96,6 +96,43 @@ directory and activates it with a handshake. See Handshake API below.
Plugins are *not* activated automatically at Docker daemon startup. Rather,
they are activated only lazily, or on-demand, when they are needed.
+## Systemd socket activation
+
+Plugins may also be socket activated by `systemd`. The official [Plugins helpers](https://github.com/docker/go-plugins-helpers)
+natively supports socket activation. In order for a plugin to be socket activated it needs
+a `service` file and a `socket` file.
+
+The `service` file (for example `/lib/systemd/system/your-plugin.service`):
+
+```
+[Unit]
+Description=Your plugin
+Before=docker.service
+After=network.target your-plugin.socket
+Requires=your-plugin.socket docker.service
+
+[Service]
+ExecStart=/usr/lib/docker/your-plugin
+
+[Install]
+WantedBy=multi-user.target
+```
+The `socket` file (for example `/lib/systemd/system/your-plugin.socket`):
+```
+[Unit]
+Description=Your plugin
+
+[Socket]
+ListenStream=/run/docker/plugins/your-plugin.sock
+
+[Install]
+WantedBy=sockets.target
+```
+
+This will allow plugins to be actually started when the Docker daemon connects to
+the sockets they're listening on (for instance the first time the daemon uses them
+or if one of the plugin goes down accidentally).
+
## API design
The Plugin API is RPC-style JSON over HTTP, much like webhooks.
@@ -127,9 +164,20 @@ Plugins are activated via the following "handshake" API call.
Responds with a list of Docker subsystems which this plugin implements.
After activation, the plugin will then be sent events from this subsystem.
+Possible values are:
+ - [`authz`](plugins_authorization.md)
+ - [`NetworkDriver`](plugins_network.md)
+ - [`VolumeDriver`](plugins_volume.md)
+
+
## Plugin retries
Attempts to call a method on a plugin are retried with an exponential backoff
for up to 30 seconds. This may help when packaging plugins as containers, since
it gives plugin containers a chance to start up before failing any user
containers which depend on them.
+
+## Plugins helpers
+
+To ease plugins development, we're providing an `sdk` for each kind of plugins
+currently supported by Docker at [docker/go-plugins-helpers](https://github.com/docker/go-plugins-helpers).
diff --git a/components/engine/docs/extend/plugins.md b/components/engine/docs/extend/plugins.md
index 997c9dfbc5..da46a16ed7 100644
--- a/components/engine/docs/extend/plugins.md
+++ b/components/engine/docs/extend/plugins.md
@@ -99,6 +99,11 @@ The following plugins exist:
Neutron, the OpenStack networking service. It includes an IPAM
driver as well.
+* The [Local Persist Plugin](https://github.com/CWSpear/local-persist)
+ extends the default `local` driver's functionality by allowing you specify
+ a mountpoint anywhere on the host, which enables the files to *always persist*,
+ even if the volume is removed via `docker volume rm`.
+
## Troubleshooting a plugin
If you are having problems with Docker after loading a plugin, ask the authors
diff --git a/components/engine/docs/extend/authorization.md b/components/engine/docs/extend/plugins_authorization.md
similarity index 99%
rename from components/engine/docs/extend/authorization.md
rename to components/engine/docs/extend/plugins_authorization.md
index 3512c56ccd..7db31b85c4 100644
--- a/components/engine/docs/extend/authorization.md
+++ b/components/engine/docs/extend/plugins_authorization.md
@@ -3,6 +3,7 @@
title = "Access authorization plugin"
description = "How to create authorization plugins to manage access control to your Docker daemon."
keywords = ["security, authorization, authentication, docker, documentation, plugin, extend"]
+aliases = ["/engine/extend/authorization/"]
[menu.main]
parent = "engine_extend"
weight = -1
diff --git a/components/engine/docs/installation/cloud/cloud-ex-aws.md b/components/engine/docs/installation/cloud/cloud-ex-aws.md
index 3163865ad6..0484f2481d 100644
--- a/components/engine/docs/installation/cloud/cloud-ex-aws.md
+++ b/components/engine/docs/installation/cloud/cloud-ex-aws.md
@@ -1,6 +1,6 @@
-# Example: Manual install on a cloud provider
+# Example: Manual install on cloud provider
You can install Docker Engine directly to servers you have on cloud providers. This example shows how to create an Amazon Web Services (AWS) EC2 instance, and install Docker Engine on it.
@@ -197,8 +197,12 @@ For Ubuntu Trusty (and some other versions), it’s recommended to install the `
## Where to go next
-* Would you like a quicker way to do Docker cloud installs? See [Digital Ocean Example: Use Docker Machine to provision Docker on cloud hosts](cloud-ex-aws.md).
+_Looking for a quicker way to do Docker cloud installs and provision multiple hosts?_ You can use [Docker Machine](https://docs.docker.com/machine/overview/) to provision hosts.
-* To learn more about options for installing Docker Engine on cloud providers, see [Understand cloud install options and choose one](cloud.md).
+ * [Use Docker Machine to provision hosts on cloud providers](https://docs.docker.com/machine/get-started-cloud/)
-* To get started with Docker, see Docker User Guide .
+ * [Docker Machine driver reference](https://docs.docker.com/machine/drivers/)
+
+* [Install Docker Engine](../index.md)
+
+* [Docker User Guide](../../userguide/intro.md)
diff --git a/components/engine/docs/installation/cloud/cloud-ex-machine-ocean.md b/components/engine/docs/installation/cloud/cloud-ex-machine-ocean.md
index 2164f4bf22..ac00a84c2a 100644
--- a/components/engine/docs/installation/cloud/cloud-ex-machine-ocean.md
+++ b/components/engine/docs/installation/cloud/cloud-ex-machine-ocean.md
@@ -14,7 +14,7 @@ Docker Machine driver plugins are available for many cloud platforms, so you can
You'll need to install and run Docker Machine, and create an account with the cloud provider.
-Then you provide account verification, security credentials, and configuration options for the providers as flags to `docker-machine create`. The flags are unique for each cloud-specific driver. For instance, to pass a Digital Ocean access token you use the `--digitalocean-access-token` flag.
+Then you provide account verification, security credentials, and configuration options for the providers as flags to `docker-machine create`. The flags are unique for each cloud-specific driver. For instance, to pass a Digital Ocean access token, you use the `--digitalocean-access-token` flag.
As an example, let's take a look at how to create a Dockerized Digital Ocean _Droplet_ (cloud server).
@@ -44,7 +44,7 @@ To generate your access token:
This is the personal access token you'll use in the next step to create your cloud server.
-### Step 3. Start Docker Machine
+### Step 3. Install Docker Machine
1. If you have not done so already, install Docker Machine on your local host.
@@ -56,30 +56,6 @@ To generate your access token:
2. At a command terminal, use `docker-machine ls` to get a list of Docker Machines and their status.
- $ docker-machine ls
- NAME ACTIVE DRIVER STATE URL SWARM
- default - virtualbox Stopped
-
-3. If Machine is stopped, start it.
-
- $ docker-machine start default
- (default) OUT | Starting VM...
- Started machines may have new IP addresses. You may need to re-run the `docker-machine env` command.
-
-4. Set environment variables to connect your shell to the local VM.
-
- $ docker-machine env default
- export DOCKER_TLS_VERIFY="1"
- export DOCKER_HOST="tcp://xxx.xxx.xx.xxx:xxxx"
- export DOCKER_CERT_PATH="/Users/londoncalling/.docker/machine/machines/default"
- export DOCKER_MACHINE_NAME="default"
- # Run this command to configure your shell:
- # eval "$(docker-machine env default)"
-
- eval "$(docker-machine env default)"
-
-5. Re-run `docker-machine ls` to check that it's now running.
-
$ docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM
default * virtualbox Running tcp:////xxx.xxx.xx.xxx:xxxx
@@ -92,28 +68,15 @@ To generate your access token:
Hello from Docker.
This message shows that your installation appears to be working correctly.
+ ...
- To generate this message, Docker took the following steps:
- 1. The Docker client contacted the Docker daemon.
- 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
- 3. The Docker daemon created a new container from that image which runs the executable that produces the output you are currently reading.
- 4. The Docker daemon streamed that output to the Docker client, which sent it to your terminal.
-
- To try something more ambitious, you can run an Ubuntu container with:
- $ docker run -it ubuntu bash
-
- Share images, automate workflows, and more with a free Docker Hub account: https://hub.docker.com
-
- For more examples and ideas, visit:
- https://docs.docker.com/userguide/
-
-### Step 4. Use Docker Machine to Create the Droplet
+### Step 4. Use Machine to Create the Droplet
1. Run `docker-machine create` with the `digitalocean` driver and pass your key to the `--digitalocean-access-token` flag, along with a name for the new cloud server.
For this example, we'll call our new Droplet "docker-sandbox".
- $ docker-machine create --driver digitalocean --digitalocean-access-token 455275108641c7716462d6f35d08b76b246b6b6151a816cf75de63c5ef918872 docker-sandbox
+ $ docker-machine create --driver digitalocean --digitalocean-access-token xxxxx docker-sandbox
Running pre-create checks...
Creating machine...
(docker-sandbox) OUT | Creating SSH key...
@@ -163,45 +126,52 @@ To generate your access token:
default - virtualbox Running tcp://192.168.99.100:2376
docker-sandbox * digitalocean Running tcp://45.55.222.72:2376
-6. Log in to the Droplet with the `docker-machine ssh` command.
+6. Run some `docker-machine` commands to inspect the remote host. For example, `docker-machine ip ` gets the host IP adddress and `docker-machine inspect ` lists all the details.
- $ docker-machine ssh docker-sandbox
- Welcome to Ubuntu 14.04.3 LTS (GNU/Linux 3.13.0-71-generic x86_64)
+ $ docker-machine ip docker-sandbox
+ 104.131.43.236
- * Documentation: https://help.ubuntu.com/
+ $ docker-machine inspect docker-sandbox
+ {
+ "ConfigVersion": 3,
+ "Driver": {
+ "IPAddress": "104.131.43.236",
+ "MachineName": "docker-sandbox",
+ "SSHUser": "root",
+ "SSHPort": 22,
+ "SSHKeyPath": "/Users/samanthastevens/.docker/machine/machines/docker-sandbox/id_rsa",
+ "StorePath": "/Users/samanthastevens/.docker/machine",
+ "SwarmMaster": false,
+ "SwarmHost": "tcp://0.0.0.0:3376",
+ "SwarmDiscovery": "",
+ ...
- System information as of Mon Dec 21 21:38:53 EST 2015
+7. Verify Docker Engine is installed correctly by running `docker` commands.
- System load: 0.77 Processes: 70
- Usage of /: 11.4% of 19.56GB Users logged in: 0
- Memory usage: 15% IP address for eth0: 45.55.139.48
- Swap usage: 0% IP address for docker0: 172.17.0.1
+ Start with something basic like `docker run hello-world`, or for a more interesting test, run a Dockerized webserver on your new remote machine.
- Graph this data and manage this system at:
- https://landscape.canonical.com/
+ In this example, the `-p` option is used to expose port 80 from the `nginx` container and make it accessible on port `8000` of the `docker-sandbox` host.
-7. Verify Docker Engine is installed correctly by running `docker run hello-world`.
+ $ docker run -d -p 8000:80 --name webserver kitematic/hello-world-nginx
+ Unable to find image 'kitematic/hello-world-nginx:latest' locally
+ latest: Pulling from kitematic/hello-world-nginx
+ a285d7f063ea: Pull complete
+ 2d7baf27389b: Pull complete
+ ...
+ Digest: sha256:ec0ca6dcb034916784c988b4f2432716e2e92b995ac606e080c7a54b52b87066
+ Status: Downloaded newer image for kitematic/hello-world-nginx:latest
+ 942dfb4a0eaae75bf26c9785ade4ff47ceb2ec2a152be82b9d7960e8b5777e65
- ubuntu@ip-172-31-0-151:~$ sudo docker run hello-world
- Unable to find image 'hello-world:latest' locally
- latest: Pulling from library/hello-world
- b901d36b6f2f: Pull complete
- 0a6ba66e537a: Pull complete
- Digest: sha256:8be990ef2aeb16dbcb9271ddfe2610fa6658d13f6dfb8bc72074cc1ca36966a7
- Status: Downloaded newer image for hello-world:latest
+ In a web browser, go to `http://:8000` to bring up the webserver home page. You got the `` from the output of the `docker-machine ip ` command you ran in a previous step. Use the port you exposed in the `docker run` command.
- Hello from Docker.
- This message shows that your installation appears to be working correctly.
- . . .
-
- You can type keyboard command Control-D or `exit` to log out of the remote server.
+ 
#### Understand the defaults and options on the create command
For convenience, `docker-machine` will use sensible defaults for choosing settings such as the image that the server is based on, but you override the defaults using the respective flags (e.g. `--digitalocean-image`). This is useful if, for example, you want to create a cloud server with a lot of memory and CPUs (by default `docker-machine` creates a small server). For a full list of the flags/settings available and their defaults, see the output of `docker-machine create -h` at the command line. See also Driver options and operating system defaults and information about the create command in the Docker Machine documentation.
-### Step 5. Use Docker Machine to remove the Droplet
+### Step 5. Use Machine to remove the Droplet
To remove a host and all of its containers and images, first stop the machine, then use `docker-machine rm`:
@@ -220,8 +190,12 @@ If you create a host with Docker Machine, but remove it through the cloud provid
## Where to go next
-* To learn more about options for installing Docker Engine on cloud providers, see [Understand cloud install options and choose one](cloud.md).
+* [Docker Machine driver reference](https://docs.docker.com/machine/drivers/)
-* To learn more about using Docker Machine to provision cloud hosts, see Using Docker Machine with a cloud provider.
+* [Docker Machine Overview](https://docs.docker.com/machine/overview/)
-* To get started with Docker, see Docker User Guide.
+* [Use Docker Machine to provision hosts on cloud providers](https://docs.docker.com/machine/get-started-cloud/)
+
+* [Install Docker Engine](../../installation/index.md)
+
+* [Docker User Guide](../../userguide/intro.md)
diff --git a/components/engine/docs/installation/cloud/index.md b/components/engine/docs/installation/cloud/index.md
index 96589c4608..c7a83d31d1 100644
--- a/components/engine/docs/installation/cloud/index.md
+++ b/components/engine/docs/installation/cloud/index.md
@@ -8,7 +8,7 @@ aliases = [
"/engine/installation/rackspace/",
"/engine/installation/joyent/"
]
-title = "In the cloud"
+title = "On cloud providers"
description = "Cloud Installations"
keywords = ["Docker install "]
[menu.main]
@@ -20,6 +20,6 @@ weight="-60"
# Install Engine in the cloud
-* [Understand cloud install options and choose one](cloud.md)
-* [Example: Use Docker Machine to provision cloud hosts](cloud-ex-machine-ocean.md)
+* [Understand cloud install options and choose one](overview.md)
+* [Example: Use Machine to provision cloud hosts](cloud-ex-machine-ocean.md)
* [Example: Manual install on a cloud provider](cloud-ex-aws.md)
diff --git a/components/engine/docs/installation/cloud/cloud.md b/components/engine/docs/installation/cloud/overview.md
similarity index 72%
rename from components/engine/docs/installation/cloud/cloud.md
rename to components/engine/docs/installation/cloud/overview.md
index d5ba411b9c..e8b3bb7e94 100644
--- a/components/engine/docs/installation/cloud/cloud.md
+++ b/components/engine/docs/installation/cloud/overview.md
@@ -1,5 +1,8 @@
-# Understand cloud install options and choose one
+# Choose how to install
You can install Docker Engine on any cloud platform that runs an operating system (OS) that Docker supports. This includes many flavors and versions of Linux, along with Mac and Windows.
@@ -26,11 +29,11 @@ To install on a cloud provider:
2. Decide which OS you want to run on the cloud host.
-3. Understand the Docker prerequisites and install process for the chosen OS. See [Install Docker Engine](index.md) for a list of supported systems and links to the install guides.
+3. Understand the Docker prerequisites and install process for the chosen OS. See [Install Docker Engine](../index.md) for a list of supported systems and links to the install guides.
4. Create a host with a Docker supported OS, and install Docker per the instructions for that OS.
-[Example: Manual install on a cloud provider](cloud-ex-aws.md) shows how to create an Amazon Web Services (AWS) EC2 instance, and install Docker Engine on it.
+[Example (AWS): Manual install on a cloud provider](cloud-ex-aws.md) shows how to create an Amazon Web Services (AWS) EC2 instance, and install Docker Engine on it.
## Use Docker Machine to provision cloud hosts
@@ -41,13 +44,13 @@ With Docker Machine, you can use the same interface to create cloud hosts with D
To do this, you use the `docker-machine create` command with the driver for the cloud provider, and provider-specific flags for account verification, security credentials, and other configuration details.
-[Example: Use Docker Machine to provision cloud hosts](cloud-ex-machine-ocean.md) walks you through the steps to set up Docker Machine and provision a Dockerized host on [Digital Ocean](https://www.digitalocean.com/).
+[Example: Use Docker Machine to provision cloud hosts](cloud-ex-machine-ocean.md) walks you through the steps to set up Docker Machine and provision a Dockerized host on Digital Ocean).
## Where to go next
* [Example: Manual install on a cloud provider](cloud-ex-aws.md) (AWS EC2)
* [Example: Use Docker Machine to provision cloud hosts](cloud-ex-machine-ocean.md) (Digital Ocean)
-* [Using Docker Machine with a cloud provider](https://docs.docker.com/machine/get-started-cloud/)
+* For supported platforms, see [Install Docker Engine](../index.md).
-* Docker User Guide (after your install is complete, get started using Docker)
+* To get started with Docker post-install, see [Docker User Guide](../../userguide/intro.md).
diff --git a/components/engine/docs/installation/images/nginx-webserver.png b/components/engine/docs/installation/images/nginx-webserver.png
new file mode 100644
index 0000000000..941fdaaf63
Binary files /dev/null and b/components/engine/docs/installation/images/nginx-webserver.png differ
diff --git a/components/engine/docs/installation/index.md b/components/engine/docs/installation/index.md
index 54d85c107c..f4c530a10e 100644
--- a/components/engine/docs/installation/index.md
+++ b/components/engine/docs/installation/index.md
@@ -30,7 +30,7 @@ Docker Engine is supported on Linux, Cloud, Windows, and OS X. Installation inst
If your linux distribution is not listed above, don't give up yet. To try out Docker on a distribution that is not listed above, go here: [Installation from binaries](binaries.md).
## On Cloud
-* [Choose how to Install](cloud/cloud.md)
+* [Choose how to Install](cloud/overview.md)
* [Example: Manual install on a cloud provider](cloud/cloud-ex-aws.md)
* [Example: Use Docker Machine to provision cloud hosts](cloud/cloud-ex-machine-ocean.md)
diff --git a/components/engine/docs/installation/linux/centos.md b/components/engine/docs/installation/linux/centos.md
index d0569f61bb..1647f76611 100644
--- a/components/engine/docs/installation/linux/centos.md
+++ b/components/engine/docs/installation/linux/centos.md
@@ -32,7 +32,7 @@ display your kernel version:
$ uname -r
3.10.0-229.el7.x86_64
-Finally, is it recommended that you fully update your system. Please keep in
+Finally, it is recommended that you fully update your system. Please keep in
mind that your system should be fully patched to fix any potential kernel bugs.
Any reported kernel bugs may have already been fixed on the latest kernel
packages.
@@ -141,15 +141,19 @@ To create the `docker` group and add your user:
1. Log into Centos as a user with `sudo` privileges.
-2. Create the `docker` group and add your user.
+2. Create the `docker` group.
+
+ `sudo groupadd docker`
+
+3. Add your user to `docker` group.
`sudo usermod -aG docker your_username`
-3. Log out and log back in.
+4. Log out and log back in.
This ensures your user is running with the correct permissions.
-4. Verify your work by running `docker` without `sudo`.
+5. Verify your work by running `docker` without `sudo`.
$ docker run hello-world
diff --git a/components/engine/docs/installation/linux/cruxlinux.md b/components/engine/docs/installation/linux/cruxlinux.md
index 583f3f45ba..6c95110b40 100644
--- a/components/engine/docs/installation/linux/cruxlinux.md
+++ b/components/engine/docs/installation/linux/cruxlinux.md
@@ -64,7 +64,7 @@ or use it as part of your `FROM` line in your `Dockerfile(s)`.
$ docker pull crux
$ docker run -i -t crux
-There are also user contributed [CRUX based image(s)](https://registry.hub.docker.com/repos/crux/) on the Docker Hub.
+There are also user contributed [CRUX based image(s)](https://hub.docker.com/_/crux/) on the Docker Hub.
## Uninstallation
diff --git a/components/engine/docs/installation/linux/fedora.md b/components/engine/docs/installation/linux/fedora.md
index 3fd46e9a5a..782adc6735 100644
--- a/components/engine/docs/installation/linux/fedora.md
+++ b/components/engine/docs/installation/linux/fedora.md
@@ -135,15 +135,19 @@ To create the `docker` group and add your user:
1. Log into your system as a user with `sudo` privileges.
-2. Create the `docker` group and add your user.
+2. Create the `docker` group.
+
+ `sudo groupadd docker`
+
+3. Add your user to `docker` group.
`sudo usermod -aG docker your_username`
-3. Log out and log back in.
+4. Log out and log back in.
This ensures your user is running with the correct permissions.
-4. Verify your work by running `docker` without `sudo`.
+5. Verify your work by running `docker` without `sudo`.
$ docker run hello-world
diff --git a/components/engine/docs/installation/linux/gentoolinux.md b/components/engine/docs/installation/linux/gentoolinux.md
index dac0497465..3b33ee453b 100644
--- a/components/engine/docs/installation/linux/gentoolinux.md
+++ b/components/engine/docs/installation/linux/gentoolinux.md
@@ -76,6 +76,7 @@ To use Docker, the `docker` daemon must be running as **root**.
To use Docker as a **non-root** user, add yourself to the **docker**
group by running the following command:
+ $ sudo groupadd docker
$ sudo usermod -a -G docker user
### OpenRC
diff --git a/components/engine/docs/installation/linux/oracle.md b/components/engine/docs/installation/linux/oracle.md
index a154346494..9513f8b85d 100644
--- a/components/engine/docs/installation/linux/oracle.md
+++ b/components/engine/docs/installation/linux/oracle.md
@@ -113,15 +113,19 @@ To create the `docker` group and add your user:
1. Log into Oracle Linux as a user with `sudo` privileges.
-2. Create the `docker` group and add your user.
+2. Create the `docker` group.
+
+ sudo groupadd docker
+
+3. Add your user to `docker` group.
sudo usermod -aG docker username
-3. Log out and log back in.
+4. Log out and log back in.
This ensures your user is running with the correct permissions.
-4. Verify your work by running `docker` without `sudo`.
+5. Verify your work by running `docker` without `sudo`.
$ docker run hello-world
diff --git a/components/engine/docs/installation/linux/rhel.md b/components/engine/docs/installation/linux/rhel.md
index d35d09a1b3..abf7b30ba8 100644
--- a/components/engine/docs/installation/linux/rhel.md
+++ b/components/engine/docs/installation/linux/rhel.md
@@ -133,15 +133,19 @@ To create the `docker` group and add your user:
1. Log into your machine as a user with `sudo` or `root` privileges.
-2. Create the `docker` group and add your user.
+2. Create the `docker` group.
+
+ `sudo groupadd docker`
+
+3. Add your user to `docker` group.
`sudo usermod -aG docker your_username`
-3. Log out and log back in.
+4. Log out and log back in.
This ensures your user is running with the correct permissions.
-4. Verify your work by running `docker` without `sudo`.
+5. Verify your work by running `docker` without `sudo`.
$ docker run hello-world
diff --git a/components/engine/docs/installation/linux/ubuntulinux.md b/components/engine/docs/installation/linux/ubuntulinux.md
index 8db5e3f931..4a9fa71dee 100644
--- a/components/engine/docs/installation/linux/ubuntulinux.md
+++ b/components/engine/docs/installation/linux/ubuntulinux.md
@@ -85,7 +85,8 @@ packages from the new repository:
deb https://apt.dockerproject.org/repo ubuntu-wily main
- > **Note**: Docker does not provide packages for all architectures. To install docker on
+ > **Note**: Docker does not provide packages for all architectures. You can find
+ > nightly built binaries in https://master.dockerproject.org. To install docker on
> a multi-architecture system, add an `[arch=...]` clause to the entry. Refer to the
> [Debian Multiarch wiki](https://wiki.debian.org/Multiarch/HOWTO#Setting_up_apt_sources)
> for details.
@@ -109,10 +110,9 @@ packages from the new repository:
### Prerequisites by Ubuntu Version
- Ubuntu Wily 15.10
-- Ubuntu Vivid 15.04
- Ubuntu Trusty 14.04 (LTS)
-For Ubuntu Trusty, Vivid, and Wily, it's recommended to install the
+For Ubuntu Trusty and Wily, it's recommended to install the
`linux-image-extra` kernel package. The `linux-image-extra` package
allows you use the `aufs` storage driver.
@@ -239,15 +239,19 @@ To create the `docker` group and add your user:
This procedure assumes you log in as the `ubuntu` user.
-3. Create the `docker` group and add your user.
+2. Create the `docker` group.
+
+ $ sudo groupadd docker
+
+3. Add your user to `docker` group.
$ sudo usermod -aG docker ubuntu
-3. Log out and log back in.
+4. Log out and log back in.
This ensures your user is running with the correct permissions.
-4. Verify your work by running `docker` without `sudo`.
+5. Verify your work by running `docker` without `sudo`.
$ docker run hello-world
diff --git a/components/engine/docs/installation/mac.md b/components/engine/docs/installation/mac.md
index 67b56b15ce..3d520b3781 100644
--- a/components/engine/docs/installation/mac.md
+++ b/components/engine/docs/installation/mac.md
@@ -11,10 +11,6 @@ weight="-90"
# Mac OS X
-> **Note**: This release of Docker deprecates the Boot2Docker command line in
-> favor of Docker Machine. Use the Docker Toolbox to install Docker Machine as
-> well as the other Docker tools.
-
You install Docker using Docker Toolbox. Docker Toolbox includes the following Docker tools:
* Docker Machine for running the `docker-machine` binary
@@ -67,7 +63,7 @@ installer.
1. Go to the [Docker Toolbox](https://www.docker.com/toolbox) page.
-2. Click the installer link to download.
+2. Click the Download link.
3. Install Docker Toolbox by double-clicking the package or by right-clicking
and choosing "Open" from the pop-up menu.
@@ -89,7 +85,7 @@ and choosing "Open" from the pop-up menu.
* makes these binaries available to all users
* installs VirtualBox; or updates any existing installation
- Change these defaults by pressing "Customize" or "Change
+ To change these defaults, press "Customize" or "Change
Install Location."
5. Press "Install" to perform the standard installation.
@@ -107,16 +103,15 @@ and choosing "Open" from the pop-up menu.
7. Press "Close" to exit.
-
## Running a Docker Container
To run a Docker container, you:
-* create a new (or start an existing) virtual machine that runs Docker.
-* switch your environment to your new VM
-* use the `docker` client to create, load, and manage containers
+* Create a new (or start an existing) virtual machine
+* Switch your environment to your new VM
+* Use the `docker` client to create, load, and manage containers
-Once you create a machine, you can reuse it as often as you like. Like any
+You can reuse this virtual machine as often as you like. Like any
VirtualBox VM, it maintains its configuration between uses.
There are two ways to use the installed tools, from the Docker Quickstart Terminal or
@@ -130,9 +125,9 @@ There are two ways to use the installed tools, from the Docker Quickstart Termin
The application:
- * opens a terminal window
- * creates a `default` VM if it doesn't exists, and starts the VM after
- * points the terminal environment to this VM
+ * Opens a terminal window
+ * Creates a `default` VM if it doesn't exists, and starts the VM after
+ * Points the terminal environment to this VM
Once the launch completes, the Docker Quickstart Terminal reports:
@@ -222,7 +217,6 @@ different shell such as C Shell but the commands are the same.
$ docker run hello-world
-
## Learn about your Toolbox installation
Toolbox installs the Docker Engine binary, the Docker binary on your system. When you
@@ -271,7 +265,7 @@ and what it does:
| upgrade | upgrade | Upgrades a machine's Docker client to the latest stable release. |
-## Example of Docker on Mac OS X
+## Examples on Mac OS X
Work through this section to try some practical container tasks on a VM. At this
point, you should have a VM running and be connected to it through your shell.
@@ -361,7 +355,7 @@ The next exercise demonstrates how to do this.
$ docker run -d -P -v $HOME/site:/usr/share/nginx/html \
--name mysite nginx
-6. Get the `mysite` container's port.
+6. View the `mysite` container's port.
$ docker port mysite
80/tcp -> 0.0.0.0:49166
@@ -371,7 +365,7 @@ The next exercise demonstrates how to do this.

-8. Try adding a page to your `$HOME/site` in real time.
+8. Add a page to your `$HOME/site` in real time.
$ echo "This is cool" > cool.html
@@ -391,7 +385,7 @@ The next exercise demonstrates how to do this.
## Upgrade Docker Toolbox
-To upgrade Docker Toolbox, download and re-run [the Docker Toolbox
+To upgrade Docker Toolbox, download and re-run the [Docker Toolbox
installer](https://docker.com/toolbox/).
@@ -427,9 +421,9 @@ To uninstall, do the following:
## Learning more
Use `docker-machine help` to list the full command line reference for Docker Machine. For more
-information about using SSH or SCP to access a VM, see [the Docker Machine
+information about using SSH or SCP to access a VM, see the [Docker Machine
documentation](https://docs.docker.com/machine/).
-You can continue with the [Docker User Guide](../userguide/index.md). If you are
+You can continue with the [Docker Engine User Guide](../userguide/index.md). If you are
interested in using the Kitematic GUI, see the [Kitematic user
guide](https://docs.docker.com/kitematic/userguide/).
diff --git a/components/engine/docs/installation/windows.md b/components/engine/docs/installation/windows.md
index dfb650f8e9..696ee3394a 100644
--- a/components/engine/docs/installation/windows.md
+++ b/components/engine/docs/installation/windows.md
@@ -144,9 +144,9 @@ installer.
To run a Docker container, you:
-* create a new (or start an existing) Docker virtual machine
-* switch your environment to your new VM
-* use the `docker` client to create, load, and manage containers
+* Create a new (or start an existing) Docker virtual machine
+* Switch your environment to your new VM
+* Use the `docker` client to create, load, and manage containers
Once you create a machine, you can reuse it as often as you like. Like any
VirtualBox VM, it maintains its configuration between uses.
@@ -160,9 +160,9 @@ There are several ways to use the installed tools, from the Docker Quickstart Te
The application:
- * opens a terminal window
- * creates a `default` VM if it doesn't exist, and starts the VM after
- * points the terminal environment to this VM
+ * Opens a terminal window
+ * Creates a `default` VM if it doesn't exist, and starts the VM after
+ * Points the terminal environment to this VM
Once the launch completes, you can run `docker` commands.
@@ -353,9 +353,9 @@ DHCP implementation.
## Login with PUTTY instead of using the CMD
Docker Machine generates and uses the public/private key pair in your
-`%USERPROFILE%\.ssh` directory so to log in you need to use the private key from
-this same directory. The private key needs to be converted into the format PuTTY
-uses. You can do this with
+`%USERPROFILE%\.docker\machine\machines\` directory. To
+log in you need to use the private key from this same directory. The private key
+needs to be converted into the format PuTTY uses. You can do this with
[puttygen](http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html):
1. Open `puttygen.exe` and load ("File"->"Load" menu) the private key from (you may need to change to the `All Files (*.*)` filter)
@@ -374,6 +374,6 @@ delete that file yourself.
## Learn more
-You can continue with the [Docker User Guide](../userguide/index.md). If you are
+You can continue with the [Docker Engine User Guide](../userguide/index.md). If you are
interested in using the Kitematic GUI, see the [Kitematic user
guide](https://docs.docker.com/kitematic/userguide/).
diff --git a/components/engine/docs/quickstart.md b/components/engine/docs/quickstart.md
index d8a93227f2..4f282b75dc 100644
--- a/components/engine/docs/quickstart.md
+++ b/components/engine/docs/quickstart.md
@@ -1,7 +1,7 @@
-# Quickstart Docker Engine
+# Docker Engine Quickstart
-This quickstart assumes you have a working installation of Docker Engine. To verify Engine is installed, use the following command:
+This quickstart assumes you have a working installation of Docker Engine. To verify Engine is installed and configured, use the following command:
# Check that you have a working install
$ docker info
-If you get `docker: command not found` or something like
+If you have a successful install, the system information appears. If you get `docker: command not found` or something like
`/var/lib/docker/repositories: permission denied` you may have an
incomplete Docker installation or insufficient privileges to access
Engine on your machine. With the default installation of Engine `docker`
@@ -25,9 +25,9 @@ commands need to be run by a user that is in the `docker` group or by the
`root` user.
Depending on your Engine system configuration, you may be required
-to preface each `docker` command with `sudo`. One way to avoid having to use
-`sudo` with the `docker` commands is to create a Unix group called `docker` and
-add users that will be entering `docker` commands to the 'docker' group.
+to preface each `docker` command with `sudo`. If you want to run without using
+`sudo` with the `docker` commands, then create a Unix group called `docker` and
+add the user to the 'docker' group.
For more information about installing Docker Engine or `sudo` configuration, refer to
the [installation](installation/index.md) instructions for your operating system.
@@ -35,34 +35,40 @@ the [installation](installation/index.md) instructions for your operating system
## Download a pre-built image
+To pull an `ubuntu` image, run:
+
# Download an ubuntu image
$ docker pull ubuntu
-This will find the `ubuntu` image by name on
-[*Docker Hub*](userguide/containers/dockerrepos.md#searching-for-images)
-and download it from [Docker Hub](https://hub.docker.com) to a local
-image cache.
+This downloads the `ubuntu` image by name from [Docker Hub](https://hub.docker.com) to a local
+image cache. To search for an image, run `docker search`. For more information, go to:
+[Searching images](userguide/containers/dockerrepos.md#searching-for-images)
+
> **Note**:
> When the image is successfully downloaded, you see a 12 character
> hash `539c0211cd76: Download complete` which is the
-> short form of the image ID. These short image IDs are the first 12
-> characters of the full image ID - which can be found using
+> short form of the Image ID. These short Image IDs are the first 12
+> characters of the full Image ID. To view this information, run
> `docker inspect` or `docker images --no-trunc=true`.
+To display a list of downloaded images, run `docker images`.
+
## Running an interactive shell
To run an interactive shell in the Ubuntu image:
$ docker run -i -t ubuntu /bin/bash
-The `-i` flag starts an interactive container. The `-t` flag creates a
-pseudo-TTY that attaches `stdin` and `stdout`.
+The `-i` flag starts an interactive container.
+The `-t` flag creates a pseudo-TTY that attaches `stdin` and `stdout`.
+The image is `ubuntu`.
+The command `/bin/bash` starts a shell you can log in.
To detach the `tty` without exiting the shell, use the escape sequence
-`Ctrl-p` + `Ctrl-q`. The container will continue to exist in a stopped state
-once exited. To list all containers, stopped and running, use the `docker ps -a`
-command.
+`Ctrl-p` + `Ctrl-q`. The container continues to exist in a stopped state
+once exited. To list all running containers, run `docker ps`. To view stopped and running containers,
+run `docker ps -a`.
## Bind Docker to another host/port or a Unix socket
@@ -179,16 +185,14 @@ TCP and a Unix socket
## Committing (saving) a container state
-Save your containers state to an image, so the state can be
-re-used.
+To save the current state of a container as an image:
-When you commit your container, Docker only stores the diff (difference) between
-the source image and the current state of the container's image. To list images
-you already have, use the `docker images` command.
-
- # Commit your container to a new named image
$ docker commit
+When you commit your container, Docker Engine only stores the diff (difference) between
+the source image and the current state of the container's image. To list images
+you already have, run:
+
# List your images
$ docker images
@@ -196,6 +200,6 @@ You now have an image state from which you can create new instances.
## Where to go next
-* Work your way through the [Docker User Guide](userguide/index.md)
-* Read more about [*Share Images via Repositories*](userguide/containers/dockerrepos.md)
-* Review [*Command Line*](reference/commandline/cli.md)
+* Work your way through the [Docker Engine User Guide](userguide/index.md)
+* Read more about [Store Images on Docker Hub](userguide/containers/dockerrepos.md)
+* Review [Command Line](reference/commandline/cli.md)
diff --git a/components/engine/docs/reference/api/docker_remote_api.md b/components/engine/docs/reference/api/docker_remote_api.md
index debf9f0454..75edd928bf 100644
--- a/components/engine/docs/reference/api/docker_remote_api.md
+++ b/components/engine/docs/reference/api/docker_remote_api.md
@@ -24,6 +24,17 @@ client must have `root` access to interact with the daemon. If a group named
`docker` exists on your system, `docker` applies ownership of the socket to the
group.
+To connect to the Docker daemon with cURL you need to use cURL 7.40 or
+later, as these versions have the `--unix-socket` flag available. To
+run `curl` against the daemon on the default socket, use the
+following:
+
+ curl --unix-socket /var/run/docker.sock http://containers/json
+
+If you have bound the Docker daemon to a different socket path or TCP
+port, you would reference that in your cURL rather than the
+default.
+
The current version of the API is v1.23 which means calling `/info` is the same
as calling `/v1.23/info`. To call an older version of the API use
`/v1.22/info`.
@@ -94,7 +105,7 @@ Some container-related events are not affected by container state, so they are n
Running `docker rmi` emits an **untag** event when removing an image name. The `rmi` command may also emit **delete** events when images are deleted by ID directly or by deleting the last tag referring to the image.
-> **Acknowledgement**: This diagram and the accompanying text were used with the permission of Matt Good and Gilder Labs. See Matt's original blog post [Docker Events Explained](http://gliderlabs.com/blog/2015/04/14/docker-events-explained/).
+> **Acknowledgment**: This diagram and the accompanying text were used with the permission of Matt Good and Gilder Labs. See Matt's original blog post [Docker Events Explained](https://gliderlabs.com/blog/2015/04/14/docker-events-explained/).
## Version history
@@ -105,8 +116,15 @@ This section lists each version from latest to oldest. Each listing includes a
[Docker Remote API v1.23](docker_remote_api_v1.23.md) documentation
* `GET /containers/json` returns the state of the container, one of `created`, `restarting`, `running`, `paused`, `exited` or `dead`.
+* `GET /containers/json` returns the mount points for the container.
* `GET /networks/(name)` now returns an `Internal` field showing whether the network is internal or not.
-
+* `GET /networks/(name)` now returns an `EnableIPv6` field showing whether the network has ipv6 enabled or not.
+* `POST /containers/(name)/update` now supports updating container's restart policy.
+* `POST /networks/create` now supports enabling ipv6 on the network by setting the `EnableIPv6` field (doing this with a label will no longer work).
+* `GET /info` now returns `CgroupDriver` field showing what cgroup driver the daemon is using; `cgroupfs` or `systemd`.
+* `GET /info` now returns `KernelMemory` field, showing if "kernel memory limit" is supported.
+* `POST /containers/create` now takes `PidsLimit` field, if the kernel is >= 4.3 and the pids cgroup is supported.
+* `GET /containers/(id or name)/stats` now returns `pids_stats`, if the kernel is >= 4.3 and the pids cgroup is supported.
### v1.22 API changes
@@ -131,7 +149,7 @@ This section lists each version from latest to oldest. Each listing includes a
* `POST /containers/create` now allows you to set the static IPv4 and/or IPv6 address for the container.
* `POST /networks/(id)/connect` now allows you to set the static IPv4 and/or IPv6 address for the container.
* `GET /info` now includes the number of containers running, stopped, and paused.
-* `POST /networks/create` now supports restricting external access to the network by setting the `internal` field.
+* `POST /networks/create` now supports restricting external access to the network by setting the `Internal` field.
* `POST /networks/(id)/disconnect` now includes a `Force` option to forcefully disconnect a container from network
* `GET /containers/(id)/json` now returns the `NetworkID` of containers.
* `POST /networks/create` Now supports an options field in the IPAM config that provides options
diff --git a/components/engine/docs/reference/api/docker_remote_api_v1.14.md b/components/engine/docs/reference/api/docker_remote_api_v1.14.md
index 3b8c9030b0..8aa7871c30 100644
--- a/components/engine/docs/reference/api/docker_remote_api_v1.14.md
+++ b/components/engine/docs/reference/api/docker_remote_api_v1.14.md
@@ -180,7 +180,7 @@ Status Codes:
### Inspect a container
-`GET /containers/(id)/json`
+`GET /containers/(id or name)/json`
Return low-level information on the container `id`
@@ -267,7 +267,7 @@ Status Codes:
### List processes running inside a container
-`GET /containers/(id)/top`
+`GET /containers/(id or name)/top`
List processes running inside the container `id`. On Unix systems this
is done by running the `ps` command. This endpoint is not
@@ -331,7 +331,7 @@ Status Codes:
### Get container logs
-`GET /containers/(id)/logs`
+`GET /containers/(id or name)/logs`
Get stdout and stderr logs from the container ``id``
@@ -364,7 +364,7 @@ Status Codes:
### Inspect changes on a container's filesystem
-`GET /containers/(id)/changes`
+`GET /containers/(id or name)/changes`
Inspect changes on container `id`'s filesystem
@@ -400,7 +400,7 @@ Status Codes:
### Export a container
-`GET /containers/(id)/export`
+`GET /containers/(id or name)/export`
Export the contents of container `id`
@@ -423,13 +423,13 @@ Status Codes:
### Start a container
-`POST /containers/(id)/start`
+`POST /containers/(id or name)/start`
Start the container `id`
**Example request**:
- POST /containers/(id)/start HTTP/1.1
+ POST /containers/e90e34656806/start HTTP/1.1
Content-Type: application/json
{
@@ -462,7 +462,7 @@ Status Codes:
### Stop a container
-`POST /containers/(id)/stop`
+`POST /containers/(id or name)/stop`
Stop the container `id`
@@ -487,7 +487,7 @@ Status Codes:
### Restart a container
-`POST /containers/(id)/restart`
+`POST /containers/(id or name)/restart`
Restart the container `id`
@@ -511,7 +511,7 @@ Status Codes:
### Kill a container
-`POST /containers/(id)/kill`
+`POST /containers/(id or name)/kill`
Kill the container `id`
@@ -536,7 +536,7 @@ Status Codes:
### Pause a container
-`POST /containers/(id)/pause`
+`POST /containers/(id or name)/pause`
Pause the container `id`
@@ -556,7 +556,7 @@ Status Codes:
### Unpause a container
-`POST /containers/(id)/unpause`
+`POST /containers/(id or name)/unpause`
Unpause the container `id`
@@ -576,7 +576,7 @@ Status Codes:
### Attach to a container
-`POST /containers/(id)/attach`
+`POST /containers/(id or name)/attach`
Attach to the container `id`
@@ -654,7 +654,7 @@ Status Codes:
### Attach to a container (websocket)
-`GET /containers/(id)/attach/ws`
+`GET /containers/(id or name)/attach/ws`
Attach to the container `id` via websocket
@@ -689,7 +689,7 @@ Status Codes:
### Wait a container
-`POST /containers/(id)/wait`
+`POST /containers/(id or name)/wait`
Block until container `id` stops, then returns the exit code
@@ -712,7 +712,7 @@ Status Codes:
### Remove a container
-`DELETE /containers/(id)`
+`DELETE /containers/(id or name)`
Remove the container `id` from the filesystem
@@ -740,7 +740,7 @@ Status Codes:
### Copy files or folders from a container
-`POST /containers/(id)/copy`
+`POST /containers/(id or name)/copy`
Copy files or folders of container `id`
diff --git a/components/engine/docs/reference/api/docker_remote_api_v1.15.md b/components/engine/docs/reference/api/docker_remote_api_v1.15.md
index d7c7abd076..9988aafef1 100644
--- a/components/engine/docs/reference/api/docker_remote_api_v1.15.md
+++ b/components/engine/docs/reference/api/docker_remote_api_v1.15.md
@@ -249,7 +249,7 @@ Json Parameters:
An ever increasing delay (double the previous delay, starting at 100mS)
is added before each restart to prevent flooding the server.
- **NetworkMode** - Sets the networking mode for the container. Supported
- values are: `bridge`, `host`, and `container:`
+ values are: `bridge`, `host`, `none`, and `container:`
- **Devices** - A list of devices to add to the container specified in the
form
`{ "PathOnHost": "/dev/deviceName", "PathInContainer": "/dev/deviceName", "CgroupPermissions": "mrw"}`
@@ -268,7 +268,7 @@ Status Codes:
### Inspect a container
-`GET /containers/(id)/json`
+`GET /containers/(id or name)/json`
Return low-level information on the container `id`
@@ -355,7 +355,7 @@ Status Codes:
### List processes running inside a container
-`GET /containers/(id)/top`
+`GET /containers/(id or name)/top`
List processes running inside the container `id`. On Unix systems this
is done by running the `ps` command. This endpoint is not
@@ -419,7 +419,7 @@ Status Codes:
### Get container logs
-`GET /containers/(id)/logs`
+`GET /containers/(id or name)/logs`
Get stdout and stderr logs from the container ``id``
@@ -451,7 +451,7 @@ Status Codes:
### Inspect changes on a container's filesystem
-`GET /containers/(id)/changes`
+`GET /containers/(id or name)/changes`
Inspect changes on container `id`'s filesystem
@@ -487,7 +487,7 @@ Status Codes:
### Export a container
-`GET /containers/(id)/export`
+`GET /containers/(id or name)/export`
Export the contents of container `id`
@@ -510,7 +510,7 @@ Status Codes:
### Resize a container TTY
-`GET /containers/(id)/resize?h=&w=`
+`GET /containers/(id or name)/resize?h=&w=`
Resize the TTY of container `id`
@@ -532,13 +532,13 @@ Status Codes:
### Start a container
-`POST /containers/(id)/start`
+`POST /containers/(id or name)/start`
Start the container `id`
**Example request**:
- POST /containers/(id)/start HTTP/1.1
+ POST /containers/e90e34656806/start HTTP/1.1
Content-Type: application/json
{
@@ -596,7 +596,7 @@ Json Parameters:
An ever increasing delay (double the previous delay, starting at 100mS)
is added before each restart to prevent flooding the server.
- **NetworkMode** - Sets the networking mode for the container. Supported
- values are: `bridge`, `host`, and `container:`
+ values are: `bridge`, `host`, `none`, and `container:`
- **Devices** - A list of devices to add to the container specified in the
form
`{ "PathOnHost": "/dev/deviceName", "PathInContainer": "/dev/deviceName", "CgroupPermissions": "mrw"}`
@@ -610,7 +610,7 @@ Status Codes:
### Stop a container
-`POST /containers/(id)/stop`
+`POST /containers/(id or name)/stop`
Stop the container `id`
@@ -635,7 +635,7 @@ Status Codes:
### Restart a container
-`POST /containers/(id)/restart`
+`POST /containers/(id or name)/restart`
Restart the container `id`
@@ -659,7 +659,7 @@ Status Codes:
### Kill a container
-`POST /containers/(id)/kill`
+`POST /containers/(id or name)/kill`
Kill the container `id`
@@ -684,7 +684,7 @@ Status Codes:
### Pause a container
-`POST /containers/(id)/pause`
+`POST /containers/(id or name)/pause`
Pause the container `id`
@@ -704,7 +704,7 @@ Status Codes:
### Unpause a container
-`POST /containers/(id)/unpause`
+`POST /containers/(id or name)/unpause`
Unpause the container `id`
@@ -724,7 +724,7 @@ Status Codes:
### Attach to a container
-`POST /containers/(id)/attach`
+`POST /containers/(id or name)/attach`
Attach to the container `id`
@@ -803,7 +803,7 @@ Status Codes:
### Attach to a container (websocket)
-`GET /containers/(id)/attach/ws`
+`GET /containers/(id or name)/attach/ws`
Attach to the container `id` via websocket
@@ -838,7 +838,7 @@ Status Codes:
### Wait a container
-`POST /containers/(id)/wait`
+`POST /containers/(id or name)/wait`
Block until container `id` stops, then returns the exit code
@@ -861,7 +861,7 @@ Status Codes:
### Remove a container
-`DELETE /containers/(id)`
+`DELETE /containers/(id or name)`
Remove the container `id` from the filesystem
@@ -889,7 +889,7 @@ Status Codes:
### Copy files or folders from a container
-`POST /containers/(id)/copy`
+`POST /containers/(id or name)/copy`
Copy files or folders of container `id`
@@ -1623,7 +1623,7 @@ the root that contains a list of repository and tag names mapped to layer IDs.
### Exec Create
-`POST /containers/(id)/exec`
+`POST /containers/(id or name)/exec`
Sets up an exec instance in a running container `id`
@@ -1701,7 +1701,7 @@ Status Codes:
- **404** – no such exec instance
**Stream details**:
- Similar to the stream behavior of `POST /container/(id)/attach` API
+ Similar to the stream behavior of `POST /containers/(id or name)/attach` API
### Exec Resize
diff --git a/components/engine/docs/reference/api/docker_remote_api_v1.16.md b/components/engine/docs/reference/api/docker_remote_api_v1.16.md
index df94a7320a..ad88c5eac5 100644
--- a/components/engine/docs/reference/api/docker_remote_api_v1.16.md
+++ b/components/engine/docs/reference/api/docker_remote_api_v1.16.md
@@ -249,7 +249,7 @@ Json Parameters:
An ever increasing delay (double the previous delay, starting at 100mS)
is added before each restart to prevent flooding the server.
- **NetworkMode** - Sets the networking mode for the container. Supported
- values are: `bridge`, `host`, and `container:`
+ values are: `bridge`, `host`, `none`, and `container:`
- **Devices** - A list of devices to add to the container specified in the
form
`{ "PathOnHost": "/dev/deviceName", "PathInContainer": "/dev/deviceName", "CgroupPermissions": "mrw"}`
@@ -268,7 +268,7 @@ Status Codes:
### Inspect a container
-`GET /containers/(id)/json`
+`GET /containers/(id or name)/json`
Return low-level information on the container `id`
@@ -355,7 +355,7 @@ Status Codes:
### List processes running inside a container
-`GET /containers/(id)/top`
+`GET /containers/(id or name)/top`
List processes running inside the container `id`. On Unix systems this
is done by running the `ps` command. This endpoint is not
@@ -419,7 +419,7 @@ Status Codes:
### Get container logs
-`GET /containers/(id)/logs`
+`GET /containers/(id or name)/logs`
Get stdout and stderr logs from the container ``id``
@@ -451,7 +451,7 @@ Status Codes:
### Inspect changes on a container's filesystem
-`GET /containers/(id)/changes`
+`GET /containers/(id or name)/changes`
Inspect changes on container `id`'s filesystem
@@ -487,7 +487,7 @@ Status Codes:
### Export a container
-`GET /containers/(id)/export`
+`GET /containers/(id or name)/export`
Export the contents of container `id`
@@ -510,7 +510,7 @@ Status Codes:
### Resize a container TTY
-`POST /containers/(id)/resize?h=&w=`
+`POST /containers/(id or name)/resize?h=&w=`
Resize the TTY for container with `id`. The container must be restarted for the resize to take effect.
@@ -532,7 +532,7 @@ Status Codes:
### Start a container
-`POST /containers/(id)/start`
+`POST /containers/(id or name)/start`
Start the container `id`
@@ -542,7 +542,7 @@ Start the container `id`
**Example request**:
- POST /containers/(id)/start HTTP/1.1
+ POST /containers/e90e34656806/start HTTP/1.1
**Example response**:
@@ -557,7 +557,7 @@ Status Codes:
### Stop a container
-`POST /containers/(id)/stop`
+`POST /containers/(id or name)/stop`
Stop the container `id`
@@ -582,7 +582,7 @@ Status Codes:
### Restart a container
-`POST /containers/(id)/restart`
+`POST /containers/(id or name)/restart`
Restart the container `id`
@@ -606,7 +606,7 @@ Status Codes:
### Kill a container
-`POST /containers/(id)/kill`
+`POST /containers/(id or name)/kill`
Kill the container `id`
@@ -631,7 +631,7 @@ Status Codes:
### Pause a container
-`POST /containers/(id)/pause`
+`POST /containers/(id or name)/pause`
Pause the container `id`
@@ -651,7 +651,7 @@ Status Codes:
### Unpause a container
-`POST /containers/(id)/unpause`
+`POST /containers/(id or name)/unpause`
Unpause the container `id`
@@ -671,7 +671,7 @@ Status Codes:
### Attach to a container
-`POST /containers/(id)/attach`
+`POST /containers/(id or name)/attach`
Attach to the container `id`
@@ -751,7 +751,7 @@ Status Codes:
### Attach to a container (websocket)
-`GET /containers/(id)/attach/ws`
+`GET /containers/(id or name)/attach/ws`
Attach to the container `id` via websocket
@@ -786,7 +786,7 @@ Status Codes:
### Wait a container
-`POST /containers/(id)/wait`
+`POST /containers/(id or name)/wait`
Block until container `id` stops, then returns the exit code
@@ -809,7 +809,7 @@ Status Codes:
### Remove a container
-`DELETE /containers/(id)`
+`DELETE /containers/(id or name)`
Remove the container `id` from the filesystem
@@ -837,7 +837,7 @@ Status Codes:
### Copy files or folders from a container
-`POST /containers/(id)/copy`
+`POST /containers/(id or name)/copy`
Copy files or folders of container `id`
@@ -1585,7 +1585,7 @@ the root that contains a list of repository and tag names mapped to layer IDs.
### Exec Create
-`POST /containers/(id)/exec`
+`POST /containers/(id or name)/exec`
Sets up an exec instance in a running container `id`
@@ -1663,7 +1663,7 @@ Status Codes:
- **404** – no such exec instance
**Stream details**:
- Similar to the stream behavior of `POST /container/(id)/attach` API
+ Similar to the stream behavior of `POST /containers/(id or name)/attach` API
### Exec Resize
diff --git a/components/engine/docs/reference/api/docker_remote_api_v1.17.md b/components/engine/docs/reference/api/docker_remote_api_v1.17.md
index 6f2cfe5f03..5a85ba09b8 100644
--- a/components/engine/docs/reference/api/docker_remote_api_v1.17.md
+++ b/components/engine/docs/reference/api/docker_remote_api_v1.17.md
@@ -251,7 +251,7 @@ Json Parameters:
An ever increasing delay (double the previous delay, starting at 100mS)
is added before each restart to prevent flooding the server.
- **NetworkMode** - Sets the networking mode for the container. Supported
- values are: `bridge`, `host`, and `container:`
+ values are: `bridge`, `host`, `none`, and `container:`
- **Devices** - A list of devices to add to the container specified in the
form
`{ "PathOnHost": "/dev/deviceName", "PathInContainer": "/dev/deviceName", "CgroupPermissions": "mrw"}`
@@ -272,7 +272,7 @@ Status Codes:
### Inspect a container
-`GET /containers/(id)/json`
+`GET /containers/(id or name)/json`
Return low-level information on the container `id`
@@ -394,7 +394,7 @@ Status Codes:
### List processes running inside a container
-`GET /containers/(id)/top`
+`GET /containers/(id or name)/top`
List processes running inside the container `id`. On Unix systems this
is done by running the `ps` command. This endpoint is not
@@ -458,7 +458,7 @@ Status Codes:
### Get container logs
-`GET /containers/(id)/logs`
+`GET /containers/(id or name)/logs`
Get stdout and stderr logs from the container ``id``
@@ -493,7 +493,7 @@ Status Codes:
### Inspect changes on a container's filesystem
-`GET /containers/(id)/changes`
+`GET /containers/(id or name)/changes`
Inspect changes on container `id`'s filesystem
@@ -529,7 +529,7 @@ Status Codes:
### Export a container
-`GET /containers/(id)/export`
+`GET /containers/(id or name)/export`
Export the contents of container `id`
@@ -552,7 +552,7 @@ Status Codes:
### Get container stats based on resource usage
-`GET /containers/(id)/stats`
+`GET /containers/(id or name)/stats`
This endpoint returns a live stream of a container's resource usage statistics.
@@ -640,7 +640,7 @@ Status Codes:
### Resize a container TTY
-`POST /containers/(id)/resize?h=&w=`
+`POST /containers/(id or name)/resize?h=&w=`
Resize the TTY for container with `id`. The container must be restarted for the resize to take effect.
@@ -662,7 +662,7 @@ Status Codes:
### Start a container
-`POST /containers/(id)/start`
+`POST /containers/(id or name)/start`
Start the container `id`
@@ -672,7 +672,7 @@ Start the container `id`
**Example request**:
- POST /containers/(id)/start HTTP/1.1
+ POST /containers/e90e34656806/start HTTP/1.1
**Example response**:
@@ -687,7 +687,7 @@ Status Codes:
### Stop a container
-`POST /containers/(id)/stop`
+`POST /containers/(id or name)/stop`
Stop the container `id`
@@ -712,7 +712,7 @@ Status Codes:
### Restart a container
-`POST /containers/(id)/restart`
+`POST /containers/(id or name)/restart`
Restart the container `id`
@@ -736,7 +736,7 @@ Status Codes:
### Kill a container
-`POST /containers/(id)/kill`
+`POST /containers/(id or name)/kill`
Kill the container `id`
@@ -761,7 +761,7 @@ Status Codes:
### Rename a container
-`POST /containers/(id)/rename`
+`POST /containers/(id or name)/rename`
Rename the container `id` to a `new_name`
@@ -786,7 +786,7 @@ Status Codes:
### Pause a container
-`POST /containers/(id)/pause`
+`POST /containers/(id or name)/pause`
Pause the container `id`
@@ -806,7 +806,7 @@ Status Codes:
### Unpause a container
-`POST /containers/(id)/unpause`
+`POST /containers/(id or name)/unpause`
Unpause the container `id`
@@ -826,7 +826,7 @@ Status Codes:
### Attach to a container
-`POST /containers/(id)/attach`
+`POST /containers/(id or name)/attach`
Attach to the container `id`
@@ -909,7 +909,7 @@ Status Codes:
### Attach to a container (websocket)
-`GET /containers/(id)/attach/ws`
+`GET /containers/(id or name)/attach/ws`
Attach to the container `id` via websocket
@@ -944,7 +944,7 @@ Status Codes:
### Wait a container
-`POST /containers/(id)/wait`
+`POST /containers/(id or name)/wait`
Block until container `id` stops, then returns the exit code
@@ -967,7 +967,7 @@ Status Codes:
### Remove a container
-`DELETE /containers/(id)`
+`DELETE /containers/(id or name)`
Remove the container `id` from the filesystem
@@ -995,7 +995,7 @@ Status Codes:
### Copy files or folders from a container
-`POST /containers/(id)/copy`
+`POST /containers/(id or name)/copy`
Copy files or folders of container `id`
@@ -1748,7 +1748,7 @@ the root that contains a list of repository and tag names mapped to layer IDs.
### Exec Create
-`POST /containers/(id)/exec`
+`POST /containers/(id or name)/exec`
Sets up an exec instance in a running container `id`
@@ -1826,7 +1826,7 @@ Status Codes:
- **404** – no such exec instance
**Stream details**:
- Similar to the stream behavior of `POST /container/(id)/attach` API
+ Similar to the stream behavior of `POST /containers/(id or name)/attach` API
### Exec Resize
@@ -2005,4 +2005,4 @@ This might change in the future.
To set cross origin requests to the remote api, please add flag "--api-enable-cors"
when running docker in daemon mode.
- $ docker -d -H="192.168.1.9:2375" --api-enable-cors
+ $ docker -d -H="192.168.1.9:2375" --api-enable-cors
diff --git a/components/engine/docs/reference/api/docker_remote_api_v1.18.md b/components/engine/docs/reference/api/docker_remote_api_v1.18.md
index f117a471a6..09d8ae27f8 100644
--- a/components/engine/docs/reference/api/docker_remote_api_v1.18.md
+++ b/components/engine/docs/reference/api/docker_remote_api_v1.18.md
@@ -270,7 +270,7 @@ Json Parameters:
An ever increasing delay (double the previous delay, starting at 100mS)
is added before each restart to prevent flooding the server.
- **NetworkMode** - Sets the networking mode for the container. Supported
- values are: `bridge`, `host`, and `container:`
+ values are: `bridge`, `host`, `none`, and `container:`
- **Devices** - A list of devices to add to the container specified in the
form
`{ "PathOnHost": "/dev/deviceName", "PathInContainer": "/dev/deviceName", "CgroupPermissions": "mrw"}`
@@ -299,7 +299,7 @@ Status Codes:
### Inspect a container
-`GET /containers/(id)/json`
+`GET /containers/(id or name)/json`
Return low-level information on the container `id`
@@ -432,7 +432,7 @@ Status Codes:
### List processes running inside a container
-`GET /containers/(id)/top`
+`GET /containers/(id or name)/top`
List processes running inside the container `id`. On Unix systems this
is done by running the `ps` command. This endpoint is not
@@ -496,7 +496,7 @@ Status Codes:
### Get container logs
-`GET /containers/(id)/logs`
+`GET /containers/(id or name)/logs`
Get stdout and stderr logs from the container ``id``
@@ -534,7 +534,7 @@ Status Codes:
### Inspect changes on a container's filesystem
-`GET /containers/(id)/changes`
+`GET /containers/(id or name)/changes`
Inspect changes on container `id`'s filesystem
@@ -576,7 +576,7 @@ Status Codes:
### Export a container
-`GET /containers/(id)/export`
+`GET /containers/(id or name)/export`
Export the contents of container `id`
@@ -599,7 +599,7 @@ Status Codes:
### Get container stats based on resource usage
-`GET /containers/(id)/stats`
+`GET /containers/(id or name)/stats`
This endpoint returns a live stream of a container's resource usage statistics.
@@ -687,7 +687,7 @@ Status Codes:
### Resize a container TTY
-`POST /containers/(id)/resize?h=&w=`
+`POST /containers/(id or name)/resize?h=&w=`
Resize the TTY for container with `id`. The container must be restarted for the resize to take effect.
@@ -709,7 +709,7 @@ Status Codes:
### Start a container
-`POST /containers/(id)/start`
+`POST /containers/(id or name)/start`
Start the container `id`
@@ -719,7 +719,7 @@ Start the container `id`
**Example request**:
- POST /containers/(id)/start HTTP/1.1
+ POST /containers/e90e34656806/start HTTP/1.1
**Example response**:
@@ -734,7 +734,7 @@ Status Codes:
### Stop a container
-`POST /containers/(id)/stop`
+`POST /containers/(id or name)/stop`
Stop the container `id`
@@ -759,7 +759,7 @@ Status Codes:
### Restart a container
-`POST /containers/(id)/restart`
+`POST /containers/(id or name)/restart`
Restart the container `id`
@@ -783,7 +783,7 @@ Status Codes:
### Kill a container
-`POST /containers/(id)/kill`
+`POST /containers/(id or name)/kill`
Kill the container `id`
@@ -808,7 +808,7 @@ Status Codes:
### Rename a container
-`POST /containers/(id)/rename`
+`POST /containers/(id or name)/rename`
Rename the container `id` to a `new_name`
@@ -833,7 +833,7 @@ Status Codes:
### Pause a container
-`POST /containers/(id)/pause`
+`POST /containers/(id or name)/pause`
Pause the container `id`
@@ -853,7 +853,7 @@ Status Codes:
### Unpause a container
-`POST /containers/(id)/unpause`
+`POST /containers/(id or name)/unpause`
Unpause the container `id`
@@ -873,7 +873,7 @@ Status Codes:
### Attach to a container
-`POST /containers/(id)/attach`
+`POST /containers/(id or name)/attach`
Attach to the container `id`
@@ -956,7 +956,7 @@ Status Codes:
### Attach to a container (websocket)
-`GET /containers/(id)/attach/ws`
+`GET /containers/(id or name)/attach/ws`
Attach to the container `id` via websocket
@@ -991,7 +991,7 @@ Status Codes:
### Wait a container
-`POST /containers/(id)/wait`
+`POST /containers/(id or name)/wait`
Block until container `id` stops, then returns the exit code
@@ -1014,7 +1014,7 @@ Status Codes:
### Remove a container
-`DELETE /containers/(id)`
+`DELETE /containers/(id or name)`
Remove the container `id` from the filesystem
@@ -1042,7 +1042,7 @@ Status Codes:
### Copy files or folders from a container
-`POST /containers/(id)/copy`
+`POST /containers/(id or name)/copy`
Copy files or folders of container `id`
@@ -1194,12 +1194,12 @@ or being killed.
Query Parameters:
-- **dockerfile** - path within the build context to the Dockerfile. This is
+- **dockerfile** - path within the build context to the Dockerfile. This is
ignored if `remote` is specified and points to an individual filename.
- **t** – repository name (and optionally a tag) to be applied to
the resulting image in case of success
-- **remote** – A Git repository URI or HTTP/HTTPS URI build source. If the
- URI specifies a filename, the file's contents are placed into a file
+- **remote** – A Git repository URI or HTTP/HTTPS URI build source. If the
+ URI specifies a filename, the file's contents are placed into a file
called `Dockerfile`.
- **q** – suppress verbose build output
- **nocache** – do not use the cache when building the image
@@ -1599,7 +1599,7 @@ Display system-wide information
"SwapLimit": 0,
"SystemTime": "2015-03-10T11:11:23.730591467-07:00"
}
-
+
Status Codes:
- **200** – no error
@@ -1866,7 +1866,7 @@ the root that contains a list of repository and tag names mapped to layer IDs.
### Exec Create
-`POST /containers/(id)/exec`
+`POST /containers/(id or name)/exec`
Sets up an exec instance in a running container `id`
@@ -1945,7 +1945,7 @@ Status Codes:
- **404** – no such exec instance
**Stream details**:
- Similar to the stream behavior of `POST /container/(id)/attach` API
+ Similar to the stream behavior of `POST /containers/(id or name)/attach` API
### Exec Resize
@@ -2118,7 +2118,7 @@ This might change in the future.
## 3.3 CORS Requests
-To set cross origin requests to the remote api please give values to
+To set cross origin requests to the remote api please give values to
"--api-cors-header" when running docker in daemon mode. Set * will allow all,
default or blank means CORS disabled
diff --git a/components/engine/docs/reference/api/docker_remote_api_v1.19.md b/components/engine/docs/reference/api/docker_remote_api_v1.19.md
index 196d9f75c6..0175148ab8 100644
--- a/components/engine/docs/reference/api/docker_remote_api_v1.19.md
+++ b/components/engine/docs/reference/api/docker_remote_api_v1.19.md
@@ -220,7 +220,7 @@ Json Parameters:
(ie. the relative weight vs other containers).
- **CpuPeriod** - The length of a CPU period in microseconds.
- **CpuQuota** - Microseconds of CPU time that the container can get in a CPU period.
-- **Cpuset** - Deprecated please don't use. Use `CpusetCpus` instead.
+- **Cpuset** - Deprecated please don't use. Use `CpusetCpus` instead.
- **CpusetCpus** - String value containing the `cgroups CpusetCpus` to use.
- **CpusetMems** - Memory nodes (MEMs) in which to allow execution (0-3, 0,1). Only effective on NUMA systems.
- **BlkioWeight** - Block IO weight (relative weight) accepts a weight value between 10 and 1000.
@@ -281,7 +281,7 @@ Json Parameters:
An ever increasing delay (double the previous delay, starting at 100mS)
is added before each restart to prevent flooding the server.
- **NetworkMode** - Sets the networking mode for the container. Supported
- values are: `bridge`, `host`, and `container:`
+ values are: `bridge`, `host`, `none`, and `container:`
- **Devices** - A list of devices to add to the container specified as a JSON object in the
form
`{ "PathOnHost": "/dev/deviceName", "PathInContainer": "/dev/deviceName", "CgroupPermissions": "mrw"}`
@@ -310,7 +310,7 @@ Status Codes:
### Inspect a container
-`GET /containers/(id)/json`
+`GET /containers/(id or name)/json`
Return low-level information on the container `id`
@@ -447,7 +447,7 @@ Status Codes:
### List processes running inside a container
-`GET /containers/(id)/top`
+`GET /containers/(id or name)/top`
List processes running inside the container `id`. On Unix systems this
is done by running the `ps` command. This endpoint is not
@@ -511,7 +511,7 @@ Status Codes:
### Get container logs
-`GET /containers/(id)/logs`
+`GET /containers/(id or name)/logs`
Get `stdout` and `stderr` logs from the container ``id``
@@ -551,7 +551,7 @@ Status Codes:
### Inspect changes on a container's filesystem
-`GET /containers/(id)/changes`
+`GET /containers/(id or name)/changes`
Inspect changes on container `id`'s filesystem
@@ -593,7 +593,7 @@ Status Codes:
### Export a container
-`GET /containers/(id)/export`
+`GET /containers/(id or name)/export`
Export the contents of container `id`
@@ -616,7 +616,7 @@ Status Codes:
### Get container stats based on resource usage
-`GET /containers/(id)/stats`
+`GET /containers/(id or name)/stats`
This endpoint returns a live stream of a container's resource usage statistics.
@@ -682,20 +682,37 @@ This endpoint returns a live stream of a container's resource usage statistics.
"cpu_stats" : {
"cpu_usage" : {
"percpu_usage" : [
- 16970827,
- 1839451,
- 7107380,
- 10571290
+ 8646879,
+ 24472255,
+ 36438778,
+ 30657443
],
- "usage_in_usermode" : 10000000,
- "total_usage" : 36488948,
- "usage_in_kernelmode" : 20000000
+ "usage_in_usermode" : 50000000,
+ "total_usage" : 100215355,
+ "usage_in_kernelmode" : 30000000
},
- "system_cpu_usage" : 20091722000000000,
- "throttling_data" : {}
+ "system_cpu_usage" : 739306590000000,
+ "throttling_data" : {"periods":0,"throttled_periods":0,"throttled_time":0}
+ },
+ "precpu_stats" : {
+ "cpu_usage" : {
+ "percpu_usage" : [
+ 8646879,
+ 24350896,
+ 36438778,
+ 30657443
+ ],
+ "usage_in_usermode" : 50000000,
+ "total_usage" : 100093996,
+ "usage_in_kernelmode" : 30000000
+ },
+ "system_cpu_usage" : 9492140000000,
+ "throttling_data" : {"periods":0,"throttled_periods":0,"throttled_time":0}
}
}
+The precpu_stats is the cpu statistic of last read, which is used for calculating the cpu usage percent. It is not the exact copy of the “cpu_stats” field.
+
Query Parameters:
- **stream** – 1/True/true or 0/False/false, pull stats once then disconnect. Default `true`.
@@ -708,7 +725,7 @@ Status Codes:
### Resize a container TTY
-`POST /containers/(id)/resize?h=&w=`
+`POST /containers/(id or name)/resize?h=&w=`
Resize the TTY for container with `id`. You must restart the container for the resize to take effect.
@@ -730,7 +747,7 @@ Status Codes:
### Start a container
-`POST /containers/(id)/start`
+`POST /containers/(id or name)/start`
Start the container `id`
@@ -740,7 +757,7 @@ Start the container `id`
**Example request**:
- POST /containers/(id)/start HTTP/1.1
+ POST /containers/e90e34656806/start HTTP/1.1
**Example response**:
@@ -755,7 +772,7 @@ Status Codes:
### Stop a container
-`POST /containers/(id)/stop`
+`POST /containers/(id or name)/stop`
Stop the container `id`
@@ -780,7 +797,7 @@ Status Codes:
### Restart a container
-`POST /containers/(id)/restart`
+`POST /containers/(id or name)/restart`
Restart the container `id`
@@ -804,7 +821,7 @@ Status Codes:
### Kill a container
-`POST /containers/(id)/kill`
+`POST /containers/(id or name)/kill`
Kill the container `id`
@@ -829,7 +846,7 @@ Status Codes:
### Rename a container
-`POST /containers/(id)/rename`
+`POST /containers/(id or name)/rename`
Rename the container `id` to a `new_name`
@@ -854,7 +871,7 @@ Status Codes:
### Pause a container
-`POST /containers/(id)/pause`
+`POST /containers/(id or name)/pause`
Pause the container `id`
@@ -874,7 +891,7 @@ Status Codes:
### Unpause a container
-`POST /containers/(id)/unpause`
+`POST /containers/(id or name)/unpause`
Unpause the container `id`
@@ -894,7 +911,7 @@ Status Codes:
### Attach to a container
-`POST /containers/(id)/attach`
+`POST /containers/(id or name)/attach`
Attach to the container `id`
@@ -977,7 +994,7 @@ Status Codes:
### Attach to a container (websocket)
-`GET /containers/(id)/attach/ws`
+`GET /containers/(id or name)/attach/ws`
Attach to the container `id` via websocket
@@ -1012,7 +1029,7 @@ Status Codes:
### Wait a container
-`POST /containers/(id)/wait`
+`POST /containers/(id or name)/wait`
Block until container `id` stops, then returns the exit code
@@ -1035,7 +1052,7 @@ Status Codes:
### Remove a container
-`DELETE /containers/(id)`
+`DELETE /containers/(id or name)`
Remove the container `id` from the filesystem
@@ -1063,7 +1080,7 @@ Status Codes:
### Copy files or folders from a container
-`POST /containers/(id)/copy`
+`POST /containers/(id or name)/copy`
Copy files or folders of container `id`
@@ -1365,37 +1382,37 @@ Return the history of the image `name`
HTTP/1.1 200 OK
Content-Type: application/json
- [
- {
+ [
+ {
"Id": "3db9c44f45209632d6050b35958829c3a2aa256d81b9a7be45b362ff85c54710",
"Created": 1398108230,
"CreatedBy": "/bin/sh -c #(nop) ADD file:eb15dbd63394e063b805a3c32ca7bf0266ef64676d5a6fab4801f2e81e2a5148 in /",
"Tags": [
"ubuntu:lucid",
"ubuntu:10.04"
- ],
+ ],
"Size": 182964289,
"Comment": ""
- },
- {
+ },
+ {
"Id": "6cfa4d1f33fb861d4d114f43b25abd0ac737509268065cdfd69d544a59c85ab8",
"Created": 1398108222,
"CreatedBy": "/bin/sh -c #(nop) MAINTAINER Tianon Gravi - mkimage-debootstrap.sh -i iproute,iputils-ping,ubuntu-minimal -t lucid.tar.xz lucid http://archive.ubuntu.com/ubuntu/",
"Tags": null,
"Size": 0,
"Comment": ""
- },
- {
+ },
+ {
"Id": "511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158",
"Created": 1371157430,
"CreatedBy": "",
"Tags": [
"scratch12:latest",
"scratch:latest"
- ],
+ ],
"Size": 0,
"Comment": "Imported from -"
- }
+ }
]
Status Codes:
@@ -1932,7 +1949,7 @@ the root that contains a list of repository and tag names mapped to layer IDs.
### Exec Create
-`POST /containers/(id)/exec`
+`POST /containers/(id or name)/exec`
Sets up an exec instance in a running container `id`
@@ -2011,7 +2028,7 @@ Status Codes:
- **404** – no such exec instance
**Stream details**:
- Similar to the stream behavior of `POST /container/(id)/attach` API
+ Similar to the stream behavior of `POST /containers/(id or name)/attach` API
### Exec Resize
@@ -2182,7 +2199,7 @@ from **200 OK** to **101 UPGRADED** and resends the same headers.
## 3.3 CORS Requests
-To set cross origin requests to the remote api please give values to
+To set cross origin requests to the remote api please give values to
`--api-cors-header` when running Docker in daemon mode. Set * (asterisk) allows all,
default or blank means CORS disabled
diff --git a/components/engine/docs/reference/api/docker_remote_api_v1.20.md b/components/engine/docs/reference/api/docker_remote_api_v1.20.md
index f8ab5823d1..4ada5c26ac 100644
--- a/components/engine/docs/reference/api/docker_remote_api_v1.20.md
+++ b/components/engine/docs/reference/api/docker_remote_api_v1.20.md
@@ -289,7 +289,7 @@ Json Parameters:
An ever increasing delay (double the previous delay, starting at 100mS)
is added before each restart to prevent flooding the server.
- **NetworkMode** - Sets the networking mode for the container. Supported
- values are: `bridge`, `host`, and `container:`
+ values are: `bridge`, `host`, `none`, and `container:`
- **Devices** - A list of devices to add to the container specified as a JSON object in the
form
`{ "PathOnHost": "/dev/deviceName", "PathInContainer": "/dev/deviceName", "CgroupPermissions": "mrw"}`
@@ -318,7 +318,7 @@ Status Codes:
### Inspect a container
-`GET /containers/(id)/json`
+`GET /containers/(id or name)/json`
Return low-level information on the container `id`
@@ -460,7 +460,7 @@ Status Codes:
### List processes running inside a container
-`GET /containers/(id)/top`
+`GET /containers/(id or name)/top`
List processes running inside the container `id`. On Unix systems this
is done by running the `ps` command. This endpoint is not
@@ -524,7 +524,7 @@ Status Codes:
### Get container logs
-`GET /containers/(id)/logs`
+`GET /containers/(id or name)/logs`
Get `stdout` and `stderr` logs from the container ``id``
@@ -564,7 +564,7 @@ Status Codes:
### Inspect changes on a container's filesystem
-`GET /containers/(id)/changes`
+`GET /containers/(id or name)/changes`
Inspect changes on container `id`'s filesystem
@@ -606,7 +606,7 @@ Status Codes:
### Export a container
-`GET /containers/(id)/export`
+`GET /containers/(id or name)/export`
Export the contents of container `id`
@@ -629,7 +629,7 @@ Status Codes:
### Get container stats based on resource usage
-`GET /containers/(id)/stats`
+`GET /containers/(id or name)/stats`
This endpoint returns a live stream of a container's resource usage statistics.
@@ -695,20 +695,37 @@ This endpoint returns a live stream of a container's resource usage statistics.
"cpu_stats" : {
"cpu_usage" : {
"percpu_usage" : [
- 16970827,
- 1839451,
- 7107380,
- 10571290
+ 8646879,
+ 24472255,
+ 36438778,
+ 30657443
],
- "usage_in_usermode" : 10000000,
- "total_usage" : 36488948,
- "usage_in_kernelmode" : 20000000
+ "usage_in_usermode" : 50000000,
+ "total_usage" : 100215355,
+ "usage_in_kernelmode" : 30000000
},
- "system_cpu_usage" : 20091722000000000,
- "throttling_data" : {}
+ "system_cpu_usage" : 739306590000000,
+ "throttling_data" : {"periods":0,"throttled_periods":0,"throttled_time":0}
+ },
+ "precpu_stats" : {
+ "cpu_usage" : {
+ "percpu_usage" : [
+ 8646879,
+ 24350896,
+ 36438778,
+ 30657443
+ ],
+ "usage_in_usermode" : 50000000,
+ "total_usage" : 100093996,
+ "usage_in_kernelmode" : 30000000
+ },
+ "system_cpu_usage" : 9492140000000,
+ "throttling_data" : {"periods":0,"throttled_periods":0,"throttled_time":0}
}
}
+The precpu_stats is the cpu statistic of last read, which is used for calculating the cpu usage percent. It is not the exact copy of the “cpu_stats” field.
+
Query Parameters:
- **stream** – 1/True/true or 0/False/false, pull stats once then disconnect. Default `true`.
@@ -721,7 +738,7 @@ Status Codes:
### Resize a container TTY
-`POST /containers/(id)/resize?h=&w=`
+`POST /containers/(id or name)/resize?h=&w=`
Resize the TTY for container with `id`. You must restart the container for the resize to take effect.
@@ -743,7 +760,7 @@ Status Codes:
### Start a container
-`POST /containers/(id)/start`
+`POST /containers/(id or name)/start`
Start the container `id`
@@ -753,7 +770,7 @@ Start the container `id`
**Example request**:
- POST /containers/(id)/start HTTP/1.1
+ POST /containers/e90e34656806/start HTTP/1.1
**Example response**:
@@ -768,7 +785,7 @@ Status Codes:
### Stop a container
-`POST /containers/(id)/stop`
+`POST /containers/(id or name)/stop`
Stop the container `id`
@@ -793,7 +810,7 @@ Status Codes:
### Restart a container
-`POST /containers/(id)/restart`
+`POST /containers/(id or name)/restart`
Restart the container `id`
@@ -817,7 +834,7 @@ Status Codes:
### Kill a container
-`POST /containers/(id)/kill`
+`POST /containers/(id or name)/kill`
Kill the container `id`
@@ -842,7 +859,7 @@ Status Codes:
### Rename a container
-`POST /containers/(id)/rename`
+`POST /containers/(id or name)/rename`
Rename the container `id` to a `new_name`
@@ -867,7 +884,7 @@ Status Codes:
### Pause a container
-`POST /containers/(id)/pause`
+`POST /containers/(id or name)/pause`
Pause the container `id`
@@ -887,7 +904,7 @@ Status Codes:
### Unpause a container
-`POST /containers/(id)/unpause`
+`POST /containers/(id or name)/unpause`
Unpause the container `id`
@@ -907,7 +924,7 @@ Status Codes:
### Attach to a container
-`POST /containers/(id)/attach`
+`POST /containers/(id or name)/attach`
Attach to the container `id`
@@ -990,7 +1007,7 @@ Status Codes:
### Attach to a container (websocket)
-`GET /containers/(id)/attach/ws`
+`GET /containers/(id or name)/attach/ws`
Attach to the container `id` via websocket
@@ -1025,7 +1042,7 @@ Status Codes:
### Wait a container
-`POST /containers/(id)/wait`
+`POST /containers/(id or name)/wait`
Block until container `id` stops, then returns the exit code
@@ -1048,7 +1065,7 @@ Status Codes:
### Remove a container
-`DELETE /containers/(id)`
+`DELETE /containers/(id or name)`
Remove the container `id` from the filesystem
@@ -1076,7 +1093,7 @@ Status Codes:
### Copy files or folders from a container
-`POST /containers/(id)/copy`
+`POST /containers/(id or name)/copy`
Copy files or folders of container `id`
@@ -1106,14 +1123,14 @@ Status Codes:
### Retrieving information about files and folders in a container
-`HEAD /containers/(id)/archive`
+`HEAD /containers/(id or name)/archive`
See the description of the `X-Docker-Container-Path-Stat` header in the
following section.
### Get an archive of a filesystem resource in a container
-`GET /containers/(id)/archive`
+`GET /containers/(id or name)/archive`
Get an tar archive of a resource in the filesystem of container `id`.
@@ -1174,7 +1191,7 @@ Status Codes:
### Extract an archive of files or folders to a directory in a container
-`PUT /containers/(id)/archive`
+`PUT /containers/(id or name)/archive`
Upload a tar archive to be extracted to a path in the filesystem of container
`id`.
@@ -2078,7 +2095,7 @@ the root that contains a list of repository and tag names mapped to layer IDs.
### Exec Create
-`POST /containers/(id)/exec`
+`POST /containers/(id or name)/exec`
Sets up an exec instance in a running container `id`
@@ -2157,7 +2174,7 @@ Status Codes:
- **404** – no such exec instance
**Stream details**:
- Similar to the stream behavior of `POST /container/(id)/attach` API
+ Similar to the stream behavior of `POST /containers/(id or name)/attach` API
### Exec Resize
diff --git a/components/engine/docs/reference/api/docker_remote_api_v1.21.md b/components/engine/docs/reference/api/docker_remote_api_v1.21.md
index a1bcd62585..467ec32fdd 100644
--- a/components/engine/docs/reference/api/docker_remote_api_v1.21.md
+++ b/components/engine/docs/reference/api/docker_remote_api_v1.21.md
@@ -305,7 +305,8 @@ Json Parameters:
An ever increasing delay (double the previous delay, starting at 100mS)
is added before each restart to prevent flooding the server.
- **NetworkMode** - Sets the networking mode for the container. Supported
- values are: `bridge`, `host`, and `container:`
+ standard values are: `bridge`, `host`, `none`, and `container:`. Any other value is taken
+ as a custom network's name to which this container should connect to.
- **Devices** - A list of devices to add to the container specified as a JSON object in the
form
`{ "PathOnHost": "/dev/deviceName", "PathInContainer": "/dev/deviceName", "CgroupPermissions": "mrw"}`
@@ -335,7 +336,7 @@ Status Codes:
### Inspect a container
-`GET /containers/(id)/json`
+`GET /containers/(id or name)/json`
Return low-level information on the container `id`
@@ -525,7 +526,7 @@ Status Codes:
### List processes running inside a container
-`GET /containers/(id)/top`
+`GET /containers/(id or name)/top`
List processes running inside the container `id`. On Unix systems this
is done by running the `ps` command. This endpoint is not
@@ -589,7 +590,7 @@ Status Codes:
### Get container logs
-`GET /containers/(id)/logs`
+`GET /containers/(id or name)/logs`
Get `stdout` and `stderr` logs from the container ``id``
@@ -629,7 +630,7 @@ Status Codes:
### Inspect changes on a container's filesystem
-`GET /containers/(id)/changes`
+`GET /containers/(id or name)/changes`
Inspect changes on container `id`'s filesystem
@@ -671,7 +672,7 @@ Status Codes:
### Export a container
-`GET /containers/(id)/export`
+`GET /containers/(id or name)/export`
Export the contents of container `id`
@@ -694,7 +695,7 @@ Status Codes:
### Get container stats based on resource usage
-`GET /containers/(id)/stats`
+`GET /containers/(id or name)/stats`
This endpoint returns a live stream of a container's resource usage statistics.
@@ -772,20 +773,37 @@ This endpoint returns a live stream of a container's resource usage statistics.
"cpu_stats" : {
"cpu_usage" : {
"percpu_usage" : [
- 16970827,
- 1839451,
- 7107380,
- 10571290
+ 8646879,
+ 24472255,
+ 36438778,
+ 30657443
],
- "usage_in_usermode" : 10000000,
- "total_usage" : 36488948,
- "usage_in_kernelmode" : 20000000
+ "usage_in_usermode" : 50000000,
+ "total_usage" : 100215355,
+ "usage_in_kernelmode" : 30000000
},
- "system_cpu_usage" : 20091722000000000,
- "throttling_data" : {}
+ "system_cpu_usage" : 739306590000000,
+ "throttling_data" : {"periods":0,"throttled_periods":0,"throttled_time":0}
+ },
+ "precpu_stats" : {
+ "cpu_usage" : {
+ "percpu_usage" : [
+ 8646879,
+ 24350896,
+ 36438778,
+ 30657443
+ ],
+ "usage_in_usermode" : 50000000,
+ "total_usage" : 100093996,
+ "usage_in_kernelmode" : 30000000
+ },
+ "system_cpu_usage" : 9492140000000,
+ "throttling_data" : {"periods":0,"throttled_periods":0,"throttled_time":0}
}
}
+The precpu_stats is the cpu statistic of last read, which is used for calculating the cpu usage percent. It is not the exact copy of the “cpu_stats” field.
+
Query Parameters:
- **stream** – 1/True/true or 0/False/false, pull stats once then disconnect. Default `true`.
@@ -798,7 +816,7 @@ Status Codes:
### Resize a container TTY
-`POST /containers/(id)/resize`
+`POST /containers/(id or name)/resize`
Resize the TTY for container with `id`. The unit is number of characters. You must restart the container for the resize to take effect.
@@ -825,7 +843,7 @@ Status Codes:
### Start a container
-`POST /containers/(id)/start`
+`POST /containers/(id or name)/start`
Start the container `id`
@@ -835,7 +853,7 @@ Start the container `id`
**Example request**:
- POST /containers/(id)/start HTTP/1.1
+ POST /containers/e90e34656806/start HTTP/1.1
**Example response**:
@@ -850,7 +868,7 @@ Status Codes:
### Stop a container
-`POST /containers/(id)/stop`
+`POST /containers/(id or name)/stop`
Stop the container `id`
@@ -875,7 +893,7 @@ Status Codes:
### Restart a container
-`POST /containers/(id)/restart`
+`POST /containers/(id or name)/restart`
Restart the container `id`
@@ -899,7 +917,7 @@ Status Codes:
### Kill a container
-`POST /containers/(id)/kill`
+`POST /containers/(id or name)/kill`
Kill the container `id`
@@ -924,7 +942,7 @@ Status Codes:
### Rename a container
-`POST /containers/(id)/rename`
+`POST /containers/(id or name)/rename`
Rename the container `id` to a `new_name`
@@ -949,7 +967,7 @@ Status Codes:
### Pause a container
-`POST /containers/(id)/pause`
+`POST /containers/(id or name)/pause`
Pause the container `id`
@@ -969,7 +987,7 @@ Status Codes:
### Unpause a container
-`POST /containers/(id)/unpause`
+`POST /containers/(id or name)/unpause`
Unpause the container `id`
@@ -989,7 +1007,7 @@ Status Codes:
### Attach to a container
-`POST /containers/(id)/attach`
+`POST /containers/(id or name)/attach`
Attach to the container `id`
@@ -1072,7 +1090,7 @@ Status Codes:
### Attach to a container (websocket)
-`GET /containers/(id)/attach/ws`
+`GET /containers/(id or name)/attach/ws`
Attach to the container `id` via websocket
@@ -1107,7 +1125,7 @@ Status Codes:
### Wait a container
-`POST /containers/(id)/wait`
+`POST /containers/(id or name)/wait`
Block until container `id` stops, then returns the exit code
@@ -1130,7 +1148,7 @@ Status Codes:
### Remove a container
-`DELETE /containers/(id)`
+`DELETE /containers/(id or name)`
Remove the container `id` from the filesystem
@@ -1158,7 +1176,7 @@ Status Codes:
### Copy files or folders from a container
-`POST /containers/(id)/copy`
+`POST /containers/(id or name)/copy`
Copy files or folders of container `id`
@@ -1188,14 +1206,14 @@ Status Codes:
### Retrieving information about files and folders in a container
-`HEAD /containers/(id)/archive`
+`HEAD /containers/(id or name)/archive`
See the description of the `X-Docker-Container-Path-Stat` header in the
following section.
### Get an archive of a filesystem resource in a container
-`GET /containers/(id)/archive`
+`GET /containers/(id or name)/archive`
Get an tar archive of a resource in the filesystem of container `id`.
@@ -1256,7 +1274,7 @@ Status Codes:
### Extract an archive of files or folders to a directory in a container
-`PUT /containers/(id)/archive`
+`PUT /containers/(id or name)/archive`
Upload a tar archive to be extracted to a path in the filesystem of container
`id`.
@@ -2231,7 +2249,7 @@ the root that contains a list of repository and tag names mapped to layer IDs.
### Exec Create
-`POST /containers/(id)/exec`
+`POST /containers/(id or name)/exec`
Sets up an exec instance in a running container `id`
@@ -2313,7 +2331,7 @@ Status Codes:
- **409** - container is paused
**Stream details**:
- Similar to the stream behavior of `POST /container/(id)/attach` API
+ Similar to the stream behavior of `POST /containers/(id or name)/attach` API
### Exec Resize
diff --git a/components/engine/docs/reference/api/docker_remote_api_v1.22.md b/components/engine/docs/reference/api/docker_remote_api_v1.22.md
index 22edf39c20..3e4da1058b 100644
--- a/components/engine/docs/reference/api/docker_remote_api_v1.22.md
+++ b/components/engine/docs/reference/api/docker_remote_api_v1.22.md
@@ -58,9 +58,15 @@ List containers
},
"SizeRw": 12288,
"SizeRootFs": 0,
+ "HostConfig": {
+ "NetworkMode": "default"
+ },
"NetworkSettings": {
"Networks": {
"bridge": {
+ "IPAMConfig": null,
+ "Links": null,
+ "Aliases": null,
"NetworkID": "7ea29fc1412292a2d7bba362f9253545fecdfa8ce9a6e37dd10ba8bee7129812",
"EndpointID": "2cdc4edb1ded3631c81f57966563e5c8525b81121bb3706a9a9a3ae102711f3f",
"Gateway": "172.17.0.1",
@@ -86,9 +92,15 @@ List containers
"Labels": {},
"SizeRw": 12288,
"SizeRootFs": 0,
+ "HostConfig": {
+ "NetworkMode": "default"
+ },
"NetworkSettings": {
"Networks": {
"bridge": {
+ "IPAMConfig": null,
+ "Links": null,
+ "Aliases": null,
"NetworkID": "7ea29fc1412292a2d7bba362f9253545fecdfa8ce9a6e37dd10ba8bee7129812",
"EndpointID": "88eaed7b37b38c2a3f0c4bc796494fdf51b270c2d22656412a2ca5d559a64d7a",
"Gateway": "172.17.0.1",
@@ -115,9 +127,15 @@ List containers
"Labels": {},
"SizeRw":12288,
"SizeRootFs":0,
+ "HostConfig": {
+ "NetworkMode": "default"
+ },
"NetworkSettings": {
"Networks": {
"bridge": {
+ "IPAMConfig": null,
+ "Links": null,
+ "Aliases": null,
"NetworkID": "7ea29fc1412292a2d7bba362f9253545fecdfa8ce9a6e37dd10ba8bee7129812",
"EndpointID": "8b27c041c30326d59cd6e6f510d4f8d1d570a228466f956edf7815508f78e30d",
"Gateway": "172.17.0.1",
@@ -144,9 +162,15 @@ List containers
"Labels": {},
"SizeRw": 12288,
"SizeRootFs": 0,
+ "HostConfig": {
+ "NetworkMode": "default"
+ },
"NetworkSettings": {
"Networks": {
"bridge": {
+ "IPAMConfig": null,
+ "Links": null,
+ "Aliases": null,
"NetworkID": "7ea29fc1412292a2d7bba362f9253545fecdfa8ce9a6e37dd10ba8bee7129812",
"EndpointID": "d91c7b2f0644403d7ef3095985ea0e2370325cd2332ff3a3225c4247328e66e9",
"Gateway": "172.17.0.1",
@@ -386,7 +410,8 @@ Json Parameters:
An ever increasing delay (double the previous delay, starting at 100mS)
is added before each restart to prevent flooding the server.
- **NetworkMode** - Sets the networking mode for the container. Supported
- values are: `bridge`, `host`, and `container:`
+ standard values are: `bridge`, `host`, `none`, and `container:`. Any other value is taken
+ as a custom network's name to which this container should connect to.
- **Devices** - A list of devices to add to the container specified as a JSON object in the
form
`{ "PathOnHost": "/dev/deviceName", "PathInContainer": "/dev/deviceName", "CgroupPermissions": "mrw"}`
@@ -417,7 +442,7 @@ Status Codes:
### Inspect a container
-`GET /containers/(id)/json`
+`GET /containers/(id or name)/json`
Return low-level information on the container `id`
@@ -619,7 +644,7 @@ Status Codes:
### List processes running inside a container
-`GET /containers/(id)/top`
+`GET /containers/(id or name)/top`
List processes running inside the container `id`. On Unix systems this
is done by running the `ps` command. This endpoint is not
@@ -683,7 +708,7 @@ Status Codes:
### Get container logs
-`GET /containers/(id)/logs`
+`GET /containers/(id or name)/logs`
Get `stdout` and `stderr` logs from the container ``id``
@@ -723,7 +748,7 @@ Status Codes:
### Inspect changes on a container's filesystem
-`GET /containers/(id)/changes`
+`GET /containers/(id or name)/changes`
Inspect changes on container `id`'s filesystem
@@ -765,7 +790,7 @@ Status Codes:
### Export a container
-`GET /containers/(id)/export`
+`GET /containers/(id or name)/export`
Export the contents of container `id`
@@ -788,7 +813,7 @@ Status Codes:
### Get container stats based on resource usage
-`GET /containers/(id)/stats`
+`GET /containers/(id or name)/stats`
This endpoint returns a live stream of a container's resource usage statistics.
@@ -866,20 +891,37 @@ This endpoint returns a live stream of a container's resource usage statistics.
"cpu_stats" : {
"cpu_usage" : {
"percpu_usage" : [
- 16970827,
- 1839451,
- 7107380,
- 10571290
+ 8646879,
+ 24472255,
+ 36438778,
+ 30657443
],
- "usage_in_usermode" : 10000000,
- "total_usage" : 36488948,
- "usage_in_kernelmode" : 20000000
+ "usage_in_usermode" : 50000000,
+ "total_usage" : 100215355,
+ "usage_in_kernelmode" : 30000000
},
- "system_cpu_usage" : 20091722000000000,
- "throttling_data" : {}
+ "system_cpu_usage" : 739306590000000,
+ "throttling_data" : {"periods":0,"throttled_periods":0,"throttled_time":0}
+ },
+ "precpu_stats" : {
+ "cpu_usage" : {
+ "percpu_usage" : [
+ 8646879,
+ 24350896,
+ 36438778,
+ 30657443
+ ],
+ "usage_in_usermode" : 50000000,
+ "total_usage" : 100093996,
+ "usage_in_kernelmode" : 30000000
+ },
+ "system_cpu_usage" : 9492140000000,
+ "throttling_data" : {"periods":0,"throttled_periods":0,"throttled_time":0}
}
}
+The precpu_stats is the cpu statistic of last read, which is used for calculating the cpu usage percent. It is not the exact copy of the “cpu_stats” field.
+
Query Parameters:
- **stream** – 1/True/true or 0/False/false, pull stats once then disconnect. Default `true`.
@@ -892,7 +934,7 @@ Status Codes:
### Resize a container TTY
-`POST /containers/(id)/resize`
+`POST /containers/(id or name)/resize`
Resize the TTY for container with `id`. The unit is number of characters. You must restart the container for the resize to take effect.
@@ -919,7 +961,7 @@ Status Codes:
### Start a container
-`POST /containers/(id)/start`
+`POST /containers/(id or name)/start`
Start the container `id`
@@ -929,7 +971,7 @@ Start the container `id`
**Example request**:
- POST /containers/(id)/start HTTP/1.1
+ POST /containers/e90e34656806/start HTTP/1.1
**Example response**:
@@ -950,7 +992,7 @@ Status Codes:
### Stop a container
-`POST /containers/(id)/stop`
+`POST /containers/(id or name)/stop`
Stop the container `id`
@@ -975,7 +1017,7 @@ Status Codes:
### Restart a container
-`POST /containers/(id)/restart`
+`POST /containers/(id or name)/restart`
Restart the container `id`
@@ -999,7 +1041,7 @@ Status Codes:
### Kill a container
-`POST /containers/(id)/kill`
+`POST /containers/(id or name)/kill`
Kill the container `id`
@@ -1024,13 +1066,13 @@ Status Codes:
### Update a container
-`POST /containers/(id)/update`
+`POST /containers/(id or name)/update`
Update resource configs of one or more containers.
**Example request**:
- POST /containers/(id)/update HTTP/1.1
+ POST /containers/e90e34656806/update HTTP/1.1
Content-Type: application/json
{
@@ -1064,7 +1106,7 @@ Status Codes:
### Rename a container
-`POST /containers/(id)/rename`
+`POST /containers/(id or name)/rename`
Rename the container `id` to a `new_name`
@@ -1089,7 +1131,7 @@ Status Codes:
### Pause a container
-`POST /containers/(id)/pause`
+`POST /containers/(id or name)/pause`
Pause the container `id`
@@ -1109,7 +1151,7 @@ Status Codes:
### Unpause a container
-`POST /containers/(id)/unpause`
+`POST /containers/(id or name)/unpause`
Unpause the container `id`
@@ -1129,7 +1171,7 @@ Status Codes:
### Attach to a container
-`POST /containers/(id)/attach`
+`POST /containers/(id or name)/attach`
Attach to the container `id`
@@ -1215,7 +1257,7 @@ Status Codes:
### Attach to a container (websocket)
-`GET /containers/(id)/attach/ws`
+`GET /containers/(id or name)/attach/ws`
Attach to the container `id` via websocket
@@ -1253,7 +1295,7 @@ Status Codes:
### Wait a container
-`POST /containers/(id)/wait`
+`POST /containers/(id or name)/wait`
Block until container `id` stops, then returns the exit code
@@ -1276,7 +1318,7 @@ Status Codes:
### Remove a container
-`DELETE /containers/(id)`
+`DELETE /containers/(id or name)`
Remove the container `id` from the filesystem
@@ -1304,7 +1346,7 @@ Status Codes:
### Copy files or folders from a container
-`POST /containers/(id)/copy`
+`POST /containers/(id or name)/copy`
Copy files or folders of container `id`
@@ -1334,14 +1376,14 @@ Status Codes:
### Retrieving information about files and folders in a container
-`HEAD /containers/(id)/archive`
+`HEAD /containers/(id or name)/archive`
See the description of the `X-Docker-Container-Path-Stat` header in the
following section.
### Get an archive of a filesystem resource in a container
-`GET /containers/(id)/archive`
+`GET /containers/(id or name)/archive`
Get an tar archive of a resource in the filesystem of container `id`.
@@ -1402,7 +1444,7 @@ Status Codes:
### Extract an archive of files or folders to a directory in a container
-`PUT /containers/(id)/archive`
+`PUT /containers/(id or name)/archive`
Upload a tar archive to be extracted to a path in the filesystem of container
`id`.
@@ -2480,7 +2522,7 @@ the root that contains a list of repository and tag names mapped to layer IDs.
### Exec Create
-`POST /containers/(id)/exec`
+`POST /containers/(id or name)/exec`
Sets up an exec instance in a running container `id`
@@ -2566,7 +2608,7 @@ Status Codes:
- **409** - container is paused
**Stream details**:
- Similar to the stream behavior of `POST /container/(id)/attach` API
+ Similar to the stream behavior of `POST /containers/(id or name)/attach` API
### Exec Resize
@@ -3054,7 +3096,7 @@ Content-Type: application/json
{
"Container":"3613f73ba0e4",
"EndpointConfig": {
- "test_nw": {
+ "IPAMConfig": {
"IPv4Address":"172.24.56.89",
"IPv6Address":"2001:db8::5689"
}
diff --git a/components/engine/docs/reference/api/docker_remote_api_v1.23.md b/components/engine/docs/reference/api/docker_remote_api_v1.23.md
index 25ebb1fc5b..8ff2652368 100644
--- a/components/engine/docs/reference/api/docker_remote_api_v1.23.md
+++ b/components/engine/docs/reference/api/docker_remote_api_v1.23.md
@@ -59,9 +59,15 @@ List containers
},
"SizeRw": 12288,
"SizeRootFs": 0,
+ "HostConfig": {
+ "NetworkMode": "default"
+ },
"NetworkSettings": {
"Networks": {
"bridge": {
+ "IPAMConfig": null,
+ "Links": null,
+ "Aliases": null,
"NetworkID": "7ea29fc1412292a2d7bba362f9253545fecdfa8ce9a6e37dd10ba8bee7129812",
"EndpointID": "2cdc4edb1ded3631c81f57966563e5c8525b81121bb3706a9a9a3ae102711f3f",
"Gateway": "172.17.0.1",
@@ -73,7 +79,18 @@ List containers
"MacAddress": "02:42:ac:11:00:02"
}
}
- }
+ },
+ "Mounts": [
+ {
+ "Name": "fac362...80535",
+ "Source": "/data",
+ "Destination": "/data",
+ "Driver": "local",
+ "Mode": "ro,Z",
+ "RW": false,
+ "Propagation": ""
+ }
+ ]
},
{
"Id": "9cd87474be90",
@@ -88,9 +105,15 @@ List containers
"Labels": {},
"SizeRw": 12288,
"SizeRootFs": 0,
+ "HostConfig": {
+ "NetworkMode": "default"
+ },
"NetworkSettings": {
"Networks": {
"bridge": {
+ "IPAMConfig": null,
+ "Links": null,
+ "Aliases": null,
"NetworkID": "7ea29fc1412292a2d7bba362f9253545fecdfa8ce9a6e37dd10ba8bee7129812",
"EndpointID": "88eaed7b37b38c2a3f0c4bc796494fdf51b270c2d22656412a2ca5d559a64d7a",
"Gateway": "172.17.0.1",
@@ -102,8 +125,8 @@ List containers
"MacAddress": "02:42:ac:11:00:08"
}
}
- }
-
+ },
+ "Mounts": []
},
{
"Id": "3176a2479c92",
@@ -118,9 +141,15 @@ List containers
"Labels": {},
"SizeRw":12288,
"SizeRootFs":0,
+ "HostConfig": {
+ "NetworkMode": "default"
+ },
"NetworkSettings": {
"Networks": {
"bridge": {
+ "IPAMConfig": null,
+ "Links": null,
+ "Aliases": null,
"NetworkID": "7ea29fc1412292a2d7bba362f9253545fecdfa8ce9a6e37dd10ba8bee7129812",
"EndpointID": "8b27c041c30326d59cd6e6f510d4f8d1d570a228466f956edf7815508f78e30d",
"Gateway": "172.17.0.1",
@@ -132,8 +161,8 @@ List containers
"MacAddress": "02:42:ac:11:00:06"
}
}
- }
-
+ },
+ "Mounts": []
},
{
"Id": "4cb07b47f9fb",
@@ -148,9 +177,15 @@ List containers
"Labels": {},
"SizeRw": 12288,
"SizeRootFs": 0,
+ "HostConfig": {
+ "NetworkMode": "default"
+ },
"NetworkSettings": {
"Networks": {
"bridge": {
+ "IPAMConfig": null,
+ "Links": null,
+ "Aliases": null,
"NetworkID": "7ea29fc1412292a2d7bba362f9253545fecdfa8ce9a6e37dd10ba8bee7129812",
"EndpointID": "d91c7b2f0644403d7ef3095985ea0e2370325cd2332ff3a3225c4247328e66e9",
"Gateway": "172.17.0.1",
@@ -162,8 +197,8 @@ List containers
"MacAddress": "02:42:ac:11:00:05"
}
}
- }
-
+ },
+ "Mounts": []
}
]
@@ -184,6 +219,10 @@ Query Parameters:
- `status=`(`created`|`restarting`|`running`|`paused`|`exited`|`dead`)
- `label=key` or `label="key=value"` of a container label
- `isolation=`(`default`|`process`|`hyperv`) (Windows daemon only)
+ - `ancestor`=(`[:]`, `` or ``)
+ - `before`=(`` or ``)
+ - `since`=(`` or ``)
+ - `volume`=(`` or ``)
Status Codes:
@@ -331,6 +370,7 @@ Json Parameters:
- **MemorySwappiness** - Tune a container's memory swappiness behavior. Accepts an integer between 0 and 100.
- **OomKillDisable** - Boolean value, whether to disable OOM Killer for the container or not.
- **OomScoreAdj** - An integer value containing the score given to the container in order to tune OOM killer preferences.
+- **PidsLimit** - Tune a container's pids limit. Set -1 for unlimited.
- **AttachStdin** - Boolean value, attaches to `stdin`.
- **AttachStdout** - Boolean value, attaches to `stdout`.
- **AttachStderr** - Boolean value, attaches to `stderr`.
@@ -390,7 +430,8 @@ Json Parameters:
An ever increasing delay (double the previous delay, starting at 100mS)
is added before each restart to prevent flooding the server.
- **NetworkMode** - Sets the networking mode for the container. Supported
- values are: `bridge`, `host`, and `container:`
+ standard values are: `bridge`, `host`, `none`, and `container:`. Any other value is taken
+ as a custom network's name to which this container should connect to.
- **Devices** - A list of devices to add to the container specified as a JSON object in the
form
`{ "PathOnHost": "/dev/deviceName", "PathInContainer": "/dev/deviceName", "CgroupPermissions": "mrw"}`
@@ -401,7 +442,7 @@ Json Parameters:
systems, such as SELinux.
- **LogConfig** - Log configuration for the container, specified as a JSON object in the form
`{ "Type": "", "Config": {"key1": "val1"}}`.
- Available types: `json-file`, `syslog`, `journald`, `gelf`, `awslogs`, `splunk`, `none`.
+ Available types: `json-file`, `syslog`, `journald`, `gelf`, `fluentd`, `awslogs`, `splunk`, `etwlogs`, `none`.
`json-file` logging driver.
- **CgroupParent** - Path to `cgroups` under which the container's `cgroup` is created. If the path is not absolute, the path is considered to be relative to the `cgroups` path of the init process. Cgroups are created if they do not already exist.
- **VolumeDriver** - Driver that this container users to mount volumes.
@@ -421,7 +462,7 @@ Status Codes:
### Inspect a container
-`GET /containers/(id)/json`
+`GET /containers/(id or name)/json`
Return low-level information on the container `id`
@@ -623,7 +664,7 @@ Status Codes:
### List processes running inside a container
-`GET /containers/(id)/top`
+`GET /containers/(id or name)/top`
List processes running inside the container `id`. On Unix systems this
is done by running the `ps` command. This endpoint is not
@@ -687,7 +728,7 @@ Status Codes:
### Get container logs
-`GET /containers/(id)/logs`
+`GET /containers/(id or name)/logs`
Get `stdout` and `stderr` logs from the container ``id``
@@ -727,7 +768,7 @@ Status Codes:
### Inspect changes on a container's filesystem
-`GET /containers/(id)/changes`
+`GET /containers/(id or name)/changes`
Inspect changes on container `id`'s filesystem
@@ -769,7 +810,7 @@ Status Codes:
### Export a container
-`GET /containers/(id)/export`
+`GET /containers/(id or name)/export`
Export the contents of container `id`
@@ -792,7 +833,7 @@ Status Codes:
### Get container stats based on resource usage
-`GET /containers/(id)/stats`
+`GET /containers/(id or name)/stats`
This endpoint returns a live stream of a container's resource usage statistics.
@@ -807,6 +848,9 @@ This endpoint returns a live stream of a container's resource usage statistics.
{
"read" : "2015-01-08T22:57:31.547920715Z",
+ "pids_stats": {
+ "current": 3
+ },
"networks": {
"eth0": {
"rx_bytes": 5338,
@@ -870,20 +914,37 @@ This endpoint returns a live stream of a container's resource usage statistics.
"cpu_stats" : {
"cpu_usage" : {
"percpu_usage" : [
- 16970827,
- 1839451,
- 7107380,
- 10571290
+ 8646879,
+ 24472255,
+ 36438778,
+ 30657443
],
- "usage_in_usermode" : 10000000,
- "total_usage" : 36488948,
- "usage_in_kernelmode" : 20000000
+ "usage_in_usermode" : 50000000,
+ "total_usage" : 100215355,
+ "usage_in_kernelmode" : 30000000
},
- "system_cpu_usage" : 20091722000000000,
- "throttling_data" : {}
+ "system_cpu_usage" : 739306590000000,
+ "throttling_data" : {"periods":0,"throttled_periods":0,"throttled_time":0}
+ },
+ "precpu_stats" : {
+ "cpu_usage" : {
+ "percpu_usage" : [
+ 8646879,
+ 24350896,
+ 36438778,
+ 30657443
+ ],
+ "usage_in_usermode" : 50000000,
+ "total_usage" : 100093996,
+ "usage_in_kernelmode" : 30000000
+ },
+ "system_cpu_usage" : 9492140000000,
+ "throttling_data" : {"periods":0,"throttled_periods":0,"throttled_time":0}
}
}
+The precpu_stats is the cpu statistic of last read, which is used for calculating the cpu usage percent. It is not the exact copy of the “cpu_stats” field.
+
Query Parameters:
- **stream** – 1/True/true or 0/False/false, pull stats once then disconnect. Default `true`.
@@ -896,7 +957,7 @@ Status Codes:
### Resize a container TTY
-`POST /containers/(id)/resize`
+`POST /containers/(id or name)/resize`
Resize the TTY for container with `id`. The unit is number of characters. You must restart the container for the resize to take effect.
@@ -923,7 +984,7 @@ Status Codes:
### Start a container
-`POST /containers/(id)/start`
+`POST /containers/(id or name)/start`
Start the container `id`
@@ -933,7 +994,7 @@ Start the container `id`
**Example request**:
- POST /containers/(id)/start HTTP/1.1
+ POST /containers/e90e34656806/start HTTP/1.1
**Example response**:
@@ -954,7 +1015,7 @@ Status Codes:
### Stop a container
-`POST /containers/(id)/stop`
+`POST /containers/(id or name)/stop`
Stop the container `id`
@@ -979,7 +1040,7 @@ Status Codes:
### Restart a container
-`POST /containers/(id)/restart`
+`POST /containers/(id or name)/restart`
Restart the container `id`
@@ -1003,7 +1064,7 @@ Status Codes:
### Kill a container
-`POST /containers/(id)/kill`
+`POST /containers/(id or name)/kill`
Kill the container `id`
@@ -1028,13 +1089,13 @@ Status Codes:
### Update a container
-`POST /containers/(id)/update`
+`POST /containers/(id or name)/update`
-Update resource configs of one or more containers.
+Update configuration of one or more containers.
**Example request**:
- POST /containers/(id)/update HTTP/1.1
+ POST /containers/e90e34656806/update HTTP/1.1
Content-Type: application/json
{
@@ -1048,6 +1109,10 @@ Update resource configs of one or more containers.
"MemorySwap": 514288000,
"MemoryReservation": 209715200,
"KernelMemory": 52428800,
+ "RestartPolicy": {
+ "MaximumRetryCount": 4,
+ "Name": "on-failure"
+ },
}
**Example response**:
@@ -1068,7 +1133,7 @@ Status Codes:
### Rename a container
-`POST /containers/(id)/rename`
+`POST /containers/(id or name)/rename`
Rename the container `id` to a `new_name`
@@ -1093,7 +1158,7 @@ Status Codes:
### Pause a container
-`POST /containers/(id)/pause`
+`POST /containers/(id or name)/pause`
Pause the container `id`
@@ -1113,7 +1178,7 @@ Status Codes:
### Unpause a container
-`POST /containers/(id)/unpause`
+`POST /containers/(id or name)/unpause`
Unpause the container `id`
@@ -1133,7 +1198,7 @@ Status Codes:
### Attach to a container
-`POST /containers/(id)/attach`
+`POST /containers/(id or name)/attach`
Attach to the container `id`
@@ -1219,7 +1284,7 @@ Status Codes:
### Attach to a container (websocket)
-`GET /containers/(id)/attach/ws`
+`GET /containers/(id or name)/attach/ws`
Attach to the container `id` via websocket
@@ -1257,7 +1322,7 @@ Status Codes:
### Wait a container
-`POST /containers/(id)/wait`
+`POST /containers/(id or name)/wait`
Block until container `id` stops, then returns the exit code
@@ -1280,7 +1345,7 @@ Status Codes:
### Remove a container
-`DELETE /containers/(id)`
+`DELETE /containers/(id or name)`
Remove the container `id` from the filesystem
@@ -1308,7 +1373,7 @@ Status Codes:
### Copy files or folders from a container
-`POST /containers/(id)/copy`
+`POST /containers/(id or name)/copy`
Copy files or folders of container `id`
@@ -1338,14 +1403,14 @@ Status Codes:
### Retrieving information about files and folders in a container
-`HEAD /containers/(id)/archive`
+`HEAD /containers/(id or name)/archive`
See the description of the `X-Docker-Container-Path-Stat` header in the
following section.
### Get an archive of a filesystem resource in a container
-`GET /containers/(id)/archive`
+`GET /containers/(id or name)/archive`
Get an tar archive of a resource in the filesystem of container `id`.
@@ -1406,7 +1471,7 @@ Status Codes:
### Extract an archive of files or folders to a directory in a container
-`PUT /containers/(id)/archive`
+`PUT /containers/(id or name)/archive`
Upload a tar archive to be extracted to a path in the filesystem of container
`id`.
@@ -2091,6 +2156,7 @@ Display system-wide information
{
"Architecture": "x86_64",
+ "CgroupDriver": "cgroupfs",
"Containers": 11,
"ContainersRunning": 7,
"ContainersStopped": 3,
@@ -2123,6 +2189,7 @@ Display system-wide information
"IndexServerAddress": "https://index.docker.io/v1/",
"InitPath": "/usr/bin/docker",
"InitSha1": "",
+ "KernelMemory": true,
"KernelVersion": "3.12.0-1-amd64",
"Labels": [
"storage=ssd"
@@ -2484,7 +2551,7 @@ the root that contains a list of repository and tag names mapped to layer IDs.
### Exec Create
-`POST /containers/(id)/exec`
+`POST /containers/(id or name)/exec`
Sets up an exec instance in a running container `id`
@@ -2570,7 +2637,7 @@ Status Codes:
- **409** - container is paused
**Stream details**:
- Similar to the stream behavior of `POST /container/(id)/attach` API
+ Similar to the stream behavior of `POST /containers/(id or name)/attach` API
### Exec Resize
@@ -2868,6 +2935,8 @@ Content-Type: application/json
"Id": "f2de39df4171b0dc801e8002d1d999b77256983dfc63041c0f34030aa3977566",
"Scope": "local",
"Driver": "bridge",
+ "EnableIPv6": false,
+ "Internal": false,
"IPAM": {
"Driver": "default",
"Config": [
@@ -2898,6 +2967,8 @@ Content-Type: application/json
"Id": "e086a3893b05ab69242d3c44e49483a3bbbd3a26b46baa8f61ab797c1088d794",
"Scope": "local",
"Driver": "null",
+ "EnableIPv6": false,
+ "Internal": false,
"IPAM": {
"Driver": "default",
"Config": []
@@ -2910,6 +2981,8 @@ Content-Type: application/json
"Id": "13e871235c677f196c4e1ecebb9dc733b9b2d2ab589e30c539efeda84a24215e",
"Scope": "local",
"Driver": "host",
+ "EnableIPv6": false,
+ "Internal": false,
"IPAM": {
"Driver": "default",
"Config": []
@@ -2951,6 +3024,7 @@ Content-Type: application/json
"Id": "7d86d31b1478e7cca9ebed7e73aa0fdeec46c5ca29497431d3007d2d9e15ed99",
"Scope": "local",
"Driver": "bridge",
+ "EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Config": [
@@ -3004,6 +3078,7 @@ Content-Type: application/json
{
"Name":"isolated_nw",
"Driver":"bridge",
+ "EnableIPv6": false,
"IPAM":{
"Config":[{
"Subnet":"172.20.0.0/16",
@@ -3040,7 +3115,9 @@ JSON Parameters:
- **Name** - The new network's name. this is a mandatory field
- **Driver** - Name of the network driver plugin to use. Defaults to `bridge` driver
+- **Internal** - Restrict external access to the network
- **IPAM** - Optional custom IP scheme for the network
+- **EnableIPv6** - Enable IPv6 on the network
- **Options** - Network specific options to be used by the drivers
- **CheckDuplicate** - Requests daemon to check for networks with same name
@@ -3059,7 +3136,7 @@ Content-Type: application/json
{
"Container":"3613f73ba0e4",
"EndpointConfig": {
- "test_nw": {
+ "IPAMConfig": {
"IPv4Address":"172.24.56.89",
"IPv6Address":"2001:db8::5689"
}
diff --git a/components/engine/docs/reference/builder.md b/components/engine/docs/reference/builder.md
index 3b9a6d5ea4..8a01d41d52 100644
--- a/components/engine/docs/reference/builder.md
+++ b/components/engine/docs/reference/builder.md
@@ -516,7 +516,7 @@ feature](../userguide/networking/index.md)).
ENV = ...
The `ENV` instruction sets the environment variable `` to the value
-``. This value will be in the environment of all "descendent"
+``. This value will be in the environment of all "descendant"
`Dockerfile` commands and can be [replaced inline](#environment-replacement) in
many as well.
@@ -579,7 +579,7 @@ The `` is an absolute path, or a path relative to `WORKDIR`, into which
the source will be copied inside the destination container.
ADD test relativeDir/ # adds "test" to `WORKDIR`/relativeDir/
- ADD test /absoluteDir # adds "test" to /absoluteDir
+ ADD test /absoluteDir/ # adds "test" to /absoluteDir/
All new files and directories are created with a UID and GID of 0.
@@ -646,7 +646,7 @@ guide](../userguide/eng-image/dockerfile_best-practices.md#build-cache) for more
> **Note**:
> Whether a file is identified as a recognized compression format or not
- > is done soley based on the contents of the file, not the name of the file.
+ > is done solely based on the contents of the file, not the name of the file.
> For example, if an empty file happens to end with `.tar.gz` this will not
> be recognized as a compressed file and **will not** generate any kind of
> decompression error message, rather the file will simply be copied to the
@@ -691,7 +691,7 @@ The `` is an absolute path, or a path relative to `WORKDIR`, into which
the source will be copied inside the destination container.
COPY test relativeDir/ # adds "test" to `WORKDIR`/relativeDir/
- COPY test /absoluteDir # adds "test" to /absoluteDir
+ COPY test /absoluteDir/ # adds "test" to /absoluteDir/
All new files and directories are created with a UID and GID of 0.
@@ -950,6 +950,29 @@ If you then run `docker stop test`, the container will not exit cleanly - the
user 0m 0.04s
sys 0m 0.03s
+### Understand how CMD and ENTRYPOINT interact
+
+Both `CMD` and `ENTRYPOINT` instructions define what command gets executed when running a container.
+There are few rules that describe their co-operation.
+
+1. Dockerfile should specify at least one of `CMD` or `ENTRYPOINT` commands.
+
+2. `ENTRYPOINT` should be defined when using the container as an executable.
+
+3. `CMD` should be used as a way of defining default arguments for an `ENTRYPOINT` command
+or for executing an ad-hoc command in a container.
+
+4. `CMD` will be overridden when running the container with alternative arguments.
+
+The table below shows what command is executed for different `ENTRYPOINT` / `CMD` combinations:
+
+| | No ENTRYPOINT | ENTRYPOINT exec_entry p1_entry | ENTRYPOINT ["exec_entry", "p1_entry"] |
+|--------------------------------|----------------------------|-----------------------------------------------------------|------------------------------------------------|
+| **No CMD** | *error, not allowed* | /bin/sh -c exec_entry p1_entry | exec_entry p1_entry |
+| **CMD ["exec_cmd", "p1_cmd"]** | exec_cmd p1_cmd | /bin/sh -c exec_entry p1_entry exec_cmd p1_cmd | exec_entry p1_entry exec_cmd p1_cmd |
+| **CMD ["p1_cmd", "p2_cmd"]** | p1_cmd p2_cmd | /bin/sh -c exec_entry p1_entry p1_cmd p2_cmd | exec_entry p1_entry p1_cmd p2_cmd |
+| **CMD exec_cmd p1_cmd** | /bin/sh -c exec_cmd p1_cmd | /bin/sh -c exec_entry p1_entry /bin/sh -c exec_cmd p1_cmd | exec_entry p1_entry /bin/sh -c exec_cmd p1_cmd |
+
## VOLUME
VOLUME ["/data"]
diff --git a/components/engine/docs/reference/commandline/commit.md b/components/engine/docs/reference/commandline/commit.md
index 13dd3340b0..df64e957ac 100644
--- a/components/engine/docs/reference/commandline/commit.md
+++ b/components/engine/docs/reference/commandline/commit.md
@@ -31,7 +31,7 @@ volumes mounted inside the container.
By default, the container being committed and its processes will be paused
while the image is committed. This reduces the likelihood of encountering data
corruption during the process of creating the commit. If this behavior is
-undesired, set the 'p' option to false.
+undesired, set the `--pause` option to false.
The `--change` option will apply `Dockerfile` instructions to the image that is
created. Supported `Dockerfile` instructions:
diff --git a/components/engine/docs/reference/commandline/cp.md b/components/engine/docs/reference/commandline/cp.md
index 50179fb4cf..841aeb36e0 100644
--- a/components/engine/docs/reference/commandline/cp.md
+++ b/components/engine/docs/reference/commandline/cp.md
@@ -23,7 +23,7 @@ You can copy from the container's file system to the local machine or the
reverse, from the local filesystem to the container. If `-` is specified for
either the `SRC_PATH` or `DEST_PATH`, you can also stream a tar archive from
`STDIN` or to `STDOUT`. The `CONTAINER` can be a running or stopped container.
-The `SRC_PATH` or `DEST_PATH` be a file or directory.
+The `SRC_PATH` or `DEST_PATH` can be a file or directory.
The `docker cp` command assumes container paths are relative to the container's
`/` (root) directory. This means supplying the initial forward slash is optional;
@@ -39,7 +39,8 @@ the user and primary group at the destination. For example, files copied to a
container are created with `UID:GID` of the root user. Files copied to the local
machine are created with the `UID:GID` of the user which invoked the `docker cp`
command. If you specify the `-L` option, `docker cp` follows any symbolic link
-in the `SRC_PATH`.
+in the `SRC_PATH`. `docker cp` does *not* create parent directories for
+`DEST_PATH` if they do not exist.
Assuming a path separator of `/`, a first argument of `SRC_PATH` and second
argument of `DEST_PATH`, the behavior is as follows:
@@ -85,4 +86,4 @@ It is not possible to copy certain system files such as resources under
Using `-` as the `SRC_PATH` streams the contents of `STDIN` as a tar archive.
The command extracts the content of the tar to the `DEST_PATH` in container's
filesystem. In this case, `DEST_PATH` must specify a directory. Using `-` as
-`DEST_PATH` streams the contents of the resource as a tar archive to `STDOUT`.
+the `DEST_PATH` streams the contents of the resource as a tar archive to `STDOUT`.
diff --git a/components/engine/docs/reference/commandline/create.md b/components/engine/docs/reference/commandline/create.md
index ad23995ac1..fa68b0feb1 100644
--- a/components/engine/docs/reference/commandline/create.md
+++ b/components/engine/docs/reference/commandline/create.md
@@ -74,6 +74,7 @@ Creates a new container.
-P, --publish-all Publish all exposed ports to random ports
-p, --publish=[] Publish a container's port(s) to the host
--pid="" PID namespace to use
+ --pids-limit=-1 Tune container pids limit (set -1 for unlimited), kernel >= 4.3
--privileged Give extended privileges to this container
--read-only Mount the container's root filesystem as read only
--restart="no" Restart policy (no, on-failure[:max-retry], always, unless-stopped)
diff --git a/components/engine/docs/reference/commandline/daemon.md b/components/engine/docs/reference/commandline/daemon.md
index 1986295414..aef28d576b 100644
--- a/components/engine/docs/reference/commandline/daemon.md
+++ b/components/engine/docs/reference/commandline/daemon.md
@@ -644,7 +644,7 @@ multiple plugins installed, at least one must allow the request for it to
complete.
For information about how to create an authorization plugin, see [authorization
-plugin](../../extend/authorization.md) section in the Docker extend section of this documentation.
+plugin](../../extend/plugins_authorization.md) section in the Docker extend section of this documentation.
## Daemon user namespace options
@@ -696,11 +696,17 @@ these resources are name-based, not id-based. If the numeric ID information
provided does not exist as entries in `/etc/passwd` or `/etc/group`, daemon
startup will fail with an error message.
+> **Note:** On Fedora 22, you have to `touch` the `/etc/subuid` and `/etc/subgid`
+> files to have ranges assigned when users are created. This must be done
+> *before* the `--userns-remap` option is enabled. Once these files exist, the
+> daemon can be (re)started and range assignment on user creation works properly.
+
*Example: starting with default Docker user management:*
+```bash
+$ docker daemon --userns-remap=default
```
- $ docker daemon --userns-remap=default
-```
+
When `default` is provided, Docker will create - or find the existing - user and group
named `dockremap`. If the user is created, and the Linux distribution has
appropriate support, the `/etc/subuid` and `/etc/subgid` files will be populated
@@ -709,16 +715,11 @@ at an offset based on prior entries in those files. For example, Ubuntu will
create the following range, based on an existing user named `user1` already owning
the first 65536 range:
+```bash
+$ cat /etc/subuid
+user1:100000:65536
+dockremap:165536:65536
```
- $ cat /etc/subuid
- user1:100000:65536
- dockremap:165536:65536
-```
-
-> **Note:** On a fresh Fedora install, we had to `touch` the
-> `/etc/subuid` and `/etc/subgid` files to have ranges assigned when users
-> were created. Once these files existed, range assignment on user creation
-> worked properly.
If you have a preferred/self-managed user with subordinate ID mappings already
configured, you can provide that username or uid to the `--userns-remap` flag.
@@ -802,9 +803,17 @@ cgroup.
Assuming the daemon is running in cgroup `daemoncgroup`,
`--cgroup-parent=/foobar` creates a cgroup in
-`/sys/fs/cgroup/memory/foobar`, wheras using `--cgroup-parent=foobar`
+`/sys/fs/cgroup/memory/foobar`, whereas using `--cgroup-parent=foobar`
creates the cgroup in `/sys/fs/cgroup/memory/daemoncgroup/foobar`
+The systemd cgroup driver has different rules for `--cgroup-parent`. Systemd
+represents hierarchy by slice and the name of the slice encodes the location in
+the tree. So `--cgroup-parent` for systemd cgroups should be a slice name. A
+name can consist of a dash-separated series of names, which describes the path
+to the slice from the root slice. For example, `--cgroup-parent=user-a-b.slice`
+means the memory cgroup for the container is created in
+`/sys/fs/cgroup/memory/user.slice/user-a.slice/user-a-b.slice/docker-.scope`.
+
This setting can also be set per container, using the `--cgroup-parent`
option on `docker create` and `docker run`, and takes precedence over
the `--cgroup-parent` option on the daemon.
@@ -890,4 +899,15 @@ if there are conflicts, but it won't stop execution.
The list of currently supported options that can be reconfigured is this:
- `debug`: it changes the daemon to debug mode when set to true.
+- `cluster-store`: it reloads the discovery store with the new address.
+- `cluster-store-opts`: it uses the new options to reload the discovery store.
+- `cluster-advertise`: it modifies the address advertised after reloading.
- `labels`: it replaces the daemon labels with a new set of labels.
+
+Updating and reloading the cluster configurations such as `--cluster-store`,
+`--cluster-advertise` and `--cluster-store-opts` will take effect only if
+these configurations were not previously configured. If `--cluster-store`
+has been provided in flags and `cluster-advertise` not, `cluster-advertise`
+can be added in the configuration file without accompanied by `--cluster-store`
+Configuration reload will log a warning message if it detects a change in
+previously configured cluster configurations.
diff --git a/components/engine/docs/reference/commandline/events.md b/components/engine/docs/reference/commandline/events.md
index 30eae105d9..22e94609d2 100644
--- a/components/engine/docs/reference/commandline/events.md
+++ b/components/engine/docs/reference/commandline/events.md
@@ -37,7 +37,7 @@ Docker networks report the following events:
The `--since` and `--until` parameters can be Unix timestamps, date formatted
timestamps, or Go duration strings (e.g. `10m`, `1h30m`) computed
-relative to the client machine’s time. If you do not provide the --since option,
+relative to the client machine’s time. If you do not provide the `--since` option,
the command returns only new and/or live events. Supported formats for date
formatted time stamps include RFC3339Nano, RFC3339, `2006-01-02T15:04:05`,
`2006-01-02T15:04:05.999999999`, `2006-01-02Z07:00`, and `2006-01-02`. The local
diff --git a/components/engine/docs/reference/commandline/import.md b/components/engine/docs/reference/commandline/import.md
index 3b36a52bad..d4ca8d5775 100644
--- a/components/engine/docs/reference/commandline/import.md
+++ b/components/engine/docs/reference/commandline/import.md
@@ -47,7 +47,7 @@ Import to docker via pipe and `STDIN`.
$ cat exampleimage.tgz | docker import - exampleimagelocal:new
-Import with a commit message
+Import with a commit message.
$ cat exampleimage.tgz | docker import --message "New image imported from tarball" - exampleimagelocal:new
diff --git a/components/engine/docs/reference/commandline/info.md b/components/engine/docs/reference/commandline/info.md
index 5bb6fb299f..06f7848d5a 100644
--- a/components/engine/docs/reference/commandline/info.md
+++ b/components/engine/docs/reference/commandline/info.md
@@ -33,6 +33,7 @@ For example:
Dirperm1 Supported: true
Execution Driver: native-0.2
Logging Driver: json-file
+ Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge null host
@@ -44,6 +45,7 @@ For example:
Total Memory: 62.86 GiB
Name: docker
ID: I54V:OLXT:HVMM:TPKO:JPHQ:CQCD:JNLC:O3BZ:4ZVJ:43XJ:PFHZ:6N2S
+ Docker Root Dir: /var/lib/docker
Debug mode (client): true
Debug mode (server): true
File Descriptors: 59
diff --git a/components/engine/docs/reference/commandline/login.md b/components/engine/docs/reference/commandline/login.md
index faf3615a00..34a7228427 100644
--- a/components/engine/docs/reference/commandline/login.md
+++ b/components/engine/docs/reference/commandline/login.md
@@ -12,10 +12,9 @@ parent = "smn_cli"
Usage: docker login [OPTIONS] [SERVER]
- Register or log in to a Docker registry server, if no server is
+ Log in to a Docker registry server, if no server is
specified "https://index.docker.io/v1/" is the default.
- -e, --email="" Email
--help Print usage
-p, --password="" Password
-u, --username="" Username
@@ -27,10 +26,10 @@ adding the server name.
$ docker login localhost:8080
-`docker login` requires user to use `sudo` or be `root`, except when:
+`docker login` requires user to use `sudo` or be `root`, except when:
1. connecting to a remote daemon, such as a `docker-machine` provisioned `docker engine`.
-2. user is added to the `docker` group. This will impact the security of your system; the `docker` group is `root` equivalent. See [Docker Daemon Attack Surface](https://docs.docker.com/security/security/#docker-daemon-attack-surface) for details.
+2. user is added to the `docker` group. This will impact the security of your system; the `docker` group is `root` equivalent. See [Docker Daemon Attack Surface](https://docs.docker.com/security/security/#docker-daemon-attack-surface) for details.
You can log into any public or private repository for which you have
credentials. When you log in, the command stores encoded credentials in
@@ -38,3 +37,78 @@ credentials. When you log in, the command stores encoded credentials in
> **Note**: When running `sudo docker login` credentials are saved in `/root/.docker/config.json`.
>
+
+## Credentials store
+
+The Docker Engine can keep user credentials in an external credentials store,
+such as the native keychain of the operating system. Using an external store
+is more secure than storing credentials in the Docker configuration file.
+
+To use a credentials store, you need an external helper program to interact
+with a specific keychain or external store. Docker requires the helper
+program to be in the client's host `$PATH`.
+
+This is the list of currently available credentials helpers and where
+you can download them from:
+
+- D-Bus Secret Service: https://github.com/docker/docker-credential-helpers/releases
+- Apple OS X keychain: https://github.com/docker/docker-credential-helpers/releases
+- Microsoft Windows Credential Manager: https://github.com/docker/docker-credential-helpers/releases
+
+### Usage
+
+You need to speficy the credentials store in `HOME/.docker/config.json`
+to tell the docker engine to use it:
+
+```json
+{
+ "credsStore": "osxkeychain"
+}
+```
+
+If you are currently logged in, run `docker logout` to remove
+the credentials from the file and run `docker login` again.
+
+### Protocol
+
+Credential helpers can be any program or script that follows a very simple protocol.
+This protocol is heavily inspired by Git, but it differs in the information shared.
+
+The helpers always use the first argument in the command to identify the action.
+There are only three possible values for that argument: `store`, `get`, and `erase`.
+
+The `store` command takes a JSON payload from the standard input. That payload carries
+the server address, to identify the credential, the user name and the password.
+This is an example of that payload:
+
+```json
+{
+ "ServerURL": "https://index.docker.io/v1",
+ "Username": "david",
+ "Password": "passw0rd1"
+}
+```
+
+The `store` command can write error messages to `STDOUT` that the docker engine
+will show if there was an issue.
+
+The `get` command takes a string payload from the standard input. That payload carries
+the server address that the docker engine needs credentials for. This is
+an example of that payload: `https://index.docker.io/v1`.
+
+The `get` command writes a JSON payload to `STDOUT`. Docker reads the user name
+and password from this payload:
+
+```json
+{
+ "Username": "david",
+ "Password": "passw0rd1"
+}
+```
+
+The `erase` command takes a string payload from `STDIN`. That payload carries
+the server address that the docker engine wants to remove credentials for. This is
+an example of that payload: `https://index.docker.io/v1`.
+
+The `erase` command can write error messages to `STDOUT` that the docker engine
+will show if there was an issue.
diff --git a/components/engine/docs/reference/commandline/network_connect.md b/components/engine/docs/reference/commandline/network_connect.md
index 49f0b3d7f4..a815ca38ec 100644
--- a/components/engine/docs/reference/commandline/network_connect.md
+++ b/components/engine/docs/reference/commandline/network_connect.md
@@ -40,7 +40,7 @@ You can specify the IP address you want to be assigned to the container's interf
$ docker network connect --ip 10.10.36.122 multi-host-network container2
```
-You can use `--link` option to link another container with a prefered alias
+You can use `--link` option to link another container with a preferred alias
```bash
$ docker network connect --link container1:c1 multi-host-network container2
diff --git a/components/engine/docs/reference/commandline/network_create.md b/components/engine/docs/reference/commandline/network_create.md
index bb85cc0462..6ae18e6e73 100644
--- a/components/engine/docs/reference/commandline/network_create.md
+++ b/components/engine/docs/reference/commandline/network_create.md
@@ -22,6 +22,7 @@ parent = "smn_cli"
--ip-range=[] Allocate container ip from a sub-range
--ipam-driver=default IP Address Management Driver
--ipam-opt=map[] Set custom IPAM driver specific options
+ --ipv6 Enable IPv6 networking
-o --opt=map[] Set custom driver specific options
--subnet=[] Subnet in CIDR format that represents a network segment
@@ -134,7 +135,17 @@ The following are those options and the equivalent docker daemon flags used for
| `com.docker.network.bridge.enable_icc` | `--icc` | Enable or Disable Inter Container Connectivity |
| `com.docker.network.bridge.host_binding_ipv4` | `--ip` | Default IP when binding container ports |
| `com.docker.network.mtu` | `--mtu` | Set the containers network MTU |
-| `com.docker.network.enable_ipv6` | `--ipv6` | Enable IPv6 networking |
+
+The following arguments can be passed to `docker network create` for any network driver, again with their approximate
+equivalents to `docker daemon`.
+
+| Argument | Equivalent | Description |
+|--------------|----------------|--------------------------------------------|
+| `--gateway` | - | ipv4 or ipv6 Gateway for the master subnet |
+| `--ip-range` | `--fixed-cidr` | Allocate IPs from a range |
+| `--internal` | - | Restricts external access to the network |
+| `--ipv6` | `--ipv6` | Enable IPv6 networking |
+| `--subnet` | `--bip` | Subnet for network |
For example, let's use `-o` or `--opt` options to specify an IP address binding when publishing ports:
diff --git a/components/engine/docs/reference/commandline/network_ls.md b/components/engine/docs/reference/commandline/network_ls.md
index 06733cd3ca..b12957a3a4 100644
--- a/components/engine/docs/reference/commandline/network_ls.md
+++ b/components/engine/docs/reference/commandline/network_ls.md
@@ -94,7 +94,7 @@ NETWORK ID NAME DRIVER
You can also filter for a substring in a name as this shows:
```bash
-$ docker ps --filter name=foo
+$ docker network ls --filter name=foo
NETWORK ID NAME DRIVER
95e74588f40d foo bridge
06e7eef0a170 foobar bridge
@@ -104,8 +104,8 @@ NETWORK ID NAME DRIVER
The `id` filter matches on all or part of a network's ID.
-The following filter matches all networks with a name containing the
-`06e7eef01700` string.
+The following filter matches all networks with an ID containing the
+`63d1ff1f77b0...` string.
```bash
$ docker network ls --filter id=63d1ff1f77b07ca51070a8c227e962238358bd310bde1529cf62e6c307ade161
@@ -113,14 +113,14 @@ NETWORK ID NAME DRIVER
63d1ff1f77b0 dev bridge
```
-You can also filter for a substring in a ID as this shows:
+You can also filter for a substring in an ID as this shows:
```bash
-$ docker ps --filter id=95e74588f40d
+$ docker network ls --filter id=95e74588f40d
NETWORK ID NAME DRIVER
95e74588f40d foo bridge
-$ docker ps --filter id=95e
+$ docker network ls --filter id=95e
NETWORK ID NAME DRIVER
95e74588f40d foo bridge
```
diff --git a/components/engine/docs/reference/commandline/network_rm.md b/components/engine/docs/reference/commandline/network_rm.md
index 516eb4ecfc..0653458f9d 100644
--- a/components/engine/docs/reference/commandline/network_rm.md
+++ b/components/engine/docs/reference/commandline/network_rm.md
@@ -25,7 +25,7 @@ To remove the network named 'my-network':
```
To delete multiple networks in a single `docker network rm` command, provide
-multiple network names or id's. The following example deletes a network with id
+multiple network names or ids. The following example deletes a network with id
`3695c422697f` and a network named `my-network`:
```bash
diff --git a/components/engine/docs/reference/commandline/ps.md b/components/engine/docs/reference/commandline/ps.md
index 328e674264..af82322e87 100644
--- a/components/engine/docs/reference/commandline/ps.md
+++ b/components/engine/docs/reference/commandline/ps.md
@@ -57,7 +57,10 @@ The currently supported filters are:
* exited (int - the code of exited containers. Only useful with `--all`)
* status (created|restarting|running|paused|exited|dead)
* ancestor (`[:]`, `` or ``) - filters containers that were created from the given image or a descendant.
+* before (container's id or name) - filters containers created before given id or name
+* since (container's id or name) - filters containers created since given id or name
* isolation (default|process|hyperv) (Windows daemon only)
+* volume (volume name or mount point) - filters containers that mount volumes.
#### Label
@@ -163,6 +166,46 @@ in it's layer stack.
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
82a598284012 ubuntu:12.04.5 "top" 3 minutes ago Up 3 minutes sleepy_bose
+#### Before
+
+The `before` filter shows only containers created before the container with given id or name. For example,
+having these containers created:
+
+ $ docker ps
+ CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
+ 9c3527ed70ce busybox "top" 14 seconds ago Up 15 seconds desperate_dubinsky
+ 4aace5031105 busybox "top" 48 seconds ago Up 49 seconds focused_hamilton
+ 6e63f6ff38b0 busybox "top" About a minute ago Up About a minute distracted_fermat
+
+Filtering with `before` would give:
+
+ $ docker ps -f before=9c3527ed70ce
+ CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
+ 4aace5031105 busybox "top" About a minute ago Up About a minute focused_hamilton
+ 6e63f6ff38b0 busybox "top" About a minute ago Up About a minute distracted_fermat
+
+#### Since
+
+The `since` filter shows only containers created since the container with given id or name. For example,
+with the same containers as in `before` filter:
+
+ $ docker ps -f since=6e63f6ff38b0
+ CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
+ 9c3527ed70ce busybox "top" 10 minutes ago Up 10 minutes desperate_dubinsky
+ 4aace5031105 busybox "top" 10 minutes ago Up 10 minutes focused_hamilton
+
+#### Volume
+
+The `volume` filter shows only containers that mount a specific volume or have a volume mounted in a specific path:
+
+ $ docker ps --filter volume=remote-volume --format "table {{.ID}}\t{{.Mounts}}"
+ CONTAINER ID MOUNTS
+ 9c3527ed70ce remote-volume
+
+ $ docker ps --filter volume=/data --format "table {{.ID}}\t{{.Mounts}}"
+ CONTAINER ID MOUNTS
+ 9c3527ed70ce remote-volume
+
## Formatting
@@ -183,6 +226,7 @@ Placeholder | Description
`.Names` | Container names.
`.Labels` | All labels assigned to the container.
`.Label` | Value of a specific label for this container. For example `{{.Label "com.docker.swarm.cpu"}}`
+`.Mounts` | Names of the volumes mounted in this container.
When using the `--format` option, the `ps` command will either output the data exactly as the template
declares or, when using the `table` directive, will include column headers as well.
diff --git a/components/engine/docs/reference/commandline/rm.md b/components/engine/docs/reference/commandline/rm.md
index 514b92c27e..bf615b55b8 100644
--- a/components/engine/docs/reference/commandline/rm.md
+++ b/components/engine/docs/reference/commandline/rm.md
@@ -46,15 +46,15 @@ This command will delete all stopped containers. The command
the `rm` command which will delete them. Any running containers will not be
deleted.
- $ docker rm -v redis
- redis
+ $ docker rm -v redis
+ redis
This command will remove the container and any volumes associated with it.
Note that if a volume was specified with a name, it will not be removed.
- $ docker create -v awesome:/foo -v /bar --name hello redis
- hello
- $ docker rm -v hello
+ $ docker create -v awesome:/foo -v /bar --name hello redis
+ hello
+ $ docker rm -v hello
In this example, the volume for `/foo` will remain intact, but the volume for
`/bar` will be removed. The same behavior holds for volumes inherited with
diff --git a/components/engine/docs/reference/commandline/run.md b/components/engine/docs/reference/commandline/run.md
index e9b913dff2..496ff4865d 100644
--- a/components/engine/docs/reference/commandline/run.md
+++ b/components/engine/docs/reference/commandline/run.md
@@ -74,6 +74,7 @@ parent = "smn_cli"
-P, --publish-all Publish all exposed ports to random ports
-p, --publish=[] Publish a container's port(s) to the host
--pid="" PID namespace to use
+ --pids-limit=-1 Tune container pids limit (set -1 for unlimited), kernel >= 4.3
--privileged Give extended privileges to this container
--read-only Mount the container's root filesystem as read only
--restart="no" Restart policy (no, on-failure[:max-retry], always, unless-stopped)
@@ -110,7 +111,7 @@ For information on connecting a container to a network, see the ["*Docker networ
## Examples
-### Assign name and allocate psuedo-TTY (--name, -it)
+### Assign name and allocate pseudo-TTY (--name, -it)
$ docker run --name test -it debian
root@d6c0fe130dba:/# exit 13
@@ -326,17 +327,17 @@ Guide.
### Connect a container to a network (--net)
When you start a container use the `--net` flag to connect it to a network.
-This adds the `busybox` container to the `mynet` network.
+This adds the `busybox` container to the `my-net` network.
```bash
-$ docker run -itd --net=my-multihost-network busybox
+$ docker run -itd --net=my-net busybox
```
You can also choose the IP addresses for the container with `--ip` and `--ip6`
flags when you start the container on a user-defined network.
```bash
-$ docker run -itd --net=my-multihost-network --ip=10.10.9.75 busybox
+$ docker run -itd --net=my-net --ip=10.10.9.75 busybox
```
If you want to add a running container to a network use the `docker network connect` subcommand.
diff --git a/components/engine/docs/reference/commandline/update.md b/components/engine/docs/reference/commandline/update.md
index bcbfab6ab9..24fb1f290d 100644
--- a/components/engine/docs/reference/commandline/update.md
+++ b/components/engine/docs/reference/commandline/update.md
@@ -12,7 +12,7 @@ parent = "smn_cli"
Usage: docker update [OPTIONS] CONTAINER [CONTAINER...]
- Updates container resource limits
+ Update configuration of one or more containers
--help=false Print usage
--blkio-weight=0 Block IO (relative weight), between 10 and 1000
@@ -25,11 +25,12 @@ parent = "smn_cli"
--memory-reservation="" Memory soft limit
--memory-swap="" A positive integer equal to memory plus swap. Specify -1 to enable unlimited swap
--kernel-memory="" Kernel memory limit: container must be stopped
+ --restart Restart policy to apply when a container exits
-The `docker update` command dynamically updates container resources. Use this
-command to prevent containers from consuming too many resources from their
-Docker host. With a single command, you can place limits on a single
-container or on many. To specify more than one container, provide
+The `docker update` command dynamically updates container configuration.
+You can use this command to prevent containers from consuming too many resources
+from their Docker host. With a single command, you can place limits on
+a single container or on many. To specify more than one container, provide
space-separated list of container names or IDs.
With the exception of the `--kernel-memory` value, you can specify these
@@ -38,6 +39,10 @@ options on a running or a stopped container. You can only update
stopped container, the next time you restart it, the container uses those
values.
+Another configuration you can change with this command is restart policy,
+new restart policy will take effect instantly after you run `docker update`
+on a container.
+
## EXAMPLES
The following sections illustrate ways to use this command.
@@ -59,3 +64,10 @@ To update multiple resource configurations for multiple containers:
```bash
$ docker update --cpu-shares 512 -m 300M abebf7571666 hopeful_morse
```
+
+### Update a container's restart policy
+
+To update restart policy for one or more containers:
+```bash
+$ docker update --restart=on-failure:3 abebf7571666 hopeful_morse
+```
diff --git a/components/engine/docs/reference/commandline/volume_create.md b/components/engine/docs/reference/commandline/volume_create.md
index 744cfd1ffa..da2c66de80 100644
--- a/components/engine/docs/reference/commandline/volume_create.md
+++ b/components/engine/docs/reference/commandline/volume_create.md
@@ -21,10 +21,12 @@ parent = "smn_cli"
Creates a new volume that containers can consume and store data in. If a name is not specified, Docker generates a random name. You create a volume and then configure the container to use it, for example:
- $ docker volume create --name hello
- hello
+```bash
+$ docker volume create --name hello
+hello
- $ docker run -d -v hello:/world busybox ls /world
+$ docker run -d -v hello:/world busybox ls /world
+```
The mount is created inside the container's `/world` directory. Docker does not support relative paths for mount points inside the container.
@@ -42,9 +44,32 @@ If you specify a volume name already in use on the current driver, Docker assume
Some volume drivers may take options to customize the volume creation. Use the `-o` or `--opt` flags to pass driver options:
- $ docker volume create --driver fake --opt tardis=blue --opt timey=wimey
+```bash
+$ docker volume create --driver fake --opt tardis=blue --opt timey=wimey
+```
These options are passed directly to the volume driver. Options for
different volume drivers may do different things (or nothing at all).
-*Note*: The built-in `local` volume driver does not currently accept any options.
+The built-in `local` driver on Windows does not support any options.
+
+The built-in `local` driver on Linux accepts options similar to the linux `mount`
+command:
+
+```bash
+$ docker volume create --driver local --opt type=tmpfs --opt device=tmpfs --opt o=size=100m,uid=1000
+```
+
+Another example:
+
+```bash
+$ docker volume create --driver local --opt type=btrfs --opt device=/dev/sda2
+```
+
+
+## Related information
+
+* [volume inspect](volume_inspect.md)
+* [volume ls](volume_ls.md)
+* [volume rm](volume_rm.md)
+* [Understand Data Volumes](../../userguide/containers/dockervolumes.md)
diff --git a/components/engine/docs/reference/commandline/volume_inspect.md b/components/engine/docs/reference/commandline/volume_inspect.md
index 519f80c23e..8fdd34d93b 100644
--- a/components/engine/docs/reference/commandline/volume_inspect.md
+++ b/components/engine/docs/reference/commandline/volume_inspect.md
@@ -12,7 +12,7 @@ parent = "smn_cli"
Usage: docker volume inspect [OPTIONS] VOLUME [VOLUME...]
- Inspect one or more volumes
+ Return low-level information on a volume
-f, --format= Format the output using the given go template.
--help Print usage
@@ -38,3 +38,10 @@ Example output:
$ docker volume inspect --format '{{ .Mountpoint }}' 85bffb0677236974f93955d8ecc4df55ef5070117b0e53333cc1b443777be24d
/var/lib/docker/volumes/85bffb0677236974f93955d8ecc4df55ef5070117b0e53333cc1b443777be24d/_data
+
+## Related information
+
+* [volume create](volume_create.md)
+* [volume ls](volume_ls.md)
+* [volume rm](volume_rm.md)
+* [Understand Data Volumes](../../userguide/containers/dockervolumes.md)
\ No newline at end of file
diff --git a/components/engine/docs/reference/commandline/volume_ls.md b/components/engine/docs/reference/commandline/volume_ls.md
index 3361959439..0388e8ae2d 100644
--- a/components/engine/docs/reference/commandline/volume_ls.md
+++ b/components/engine/docs/reference/commandline/volume_ls.md
@@ -32,3 +32,10 @@ Example output:
DRIVER VOLUME NAME
local rose
local tyler
+
+## Related information
+
+* [volume create](volume_create.md)
+* [volume inspect](volume_inspect.md)
+* [volume rm](volume_rm.md)
+* [Understand Data Volumes](../../userguide/containers/dockervolumes.md)
\ No newline at end of file
diff --git a/components/engine/docs/reference/commandline/volume_rm.md b/components/engine/docs/reference/commandline/volume_rm.md
index 495e746553..ff5ce24a4b 100644
--- a/components/engine/docs/reference/commandline/volume_rm.md
+++ b/components/engine/docs/reference/commandline/volume_rm.md
@@ -20,3 +20,10 @@ Removes one or more volumes. You cannot remove a volume that is in use by a cont
$ docker volume rm hello
hello
+
+## Related information
+
+* [volume create](volume_create.md)
+* [volume inspect](volume_inspect.md)
+* [volume ls](volume_ls.md)
+* [Understand Data Volumes](../../userguide/containers/dockervolumes.md)
\ No newline at end of file
diff --git a/components/engine/docs/reference/glossary.md b/components/engine/docs/reference/glossary.md
index ff4398c249..22c2d36d40 100644
--- a/components/engine/docs/reference/glossary.md
+++ b/components/engine/docs/reference/glossary.md
@@ -178,8 +178,8 @@ A repository is a set of Docker images. A repository can be shared by pushing it
to a [registry](#registry) server. The different images in the repository can be
labeled using [tags](#tag).
-Here is an example of the shared [nginx repository](https://registry.hub.docker.com/_/nginx/)
-and its [tags](https://registry.hub.docker.com/_/nginx/tags/manage/)
+Here is an example of the shared [nginx repository](https://hub.docker.com/_/nginx/)
+and its [tags](https://hub.docker.com/r/library/nginx/tags/)
## Swarm
diff --git a/components/engine/docs/reference/run.md b/components/engine/docs/reference/run.md
index e38fd1f18d..4be50a2d02 100644
--- a/components/engine/docs/reference/run.md
+++ b/components/engine/docs/reference/run.md
@@ -605,6 +605,8 @@ with the same logic -- if the original volume was specified with a name it will
--security-opt="label:disable" : Turn off label confinement for the container
--security-opt="apparmor:PROFILE" : Set the apparmor profile to be applied
to the container
+ --security-opt="no-new-privileges" : Disable container processes from gaining
+ new privileges
You can override the default labeling scheme for each container by specifying
the `--security-opt` flag. For example, you can specify the MCS/MLS level, a
@@ -631,6 +633,13 @@ command:
> **Note**: You would have to write policy defining a `svirt_apache_t` type.
+If you want to prevent your container processes from gaining additional
+privileges, you can execute the following command:
+
+ $ docker run --security-opt no-new-privileges -it centos bash
+
+For more details, see [kernel documentation](https://www.kernel.org/doc/Documentation/prctl/no_new_privs.txt).
+
## Specifying custom cgroups
Using the `--cgroup-parent` flag, you can pass a specific cgroup to run a
@@ -1049,8 +1058,8 @@ By default, the docker container process runs with the supplementary groups look
up for the specified user. If one wants to add more to that list of groups, then
one can use this flag:
- $ docker run -it --rm --group-add audio --group-add dbus --group-add 777 busybox id
- uid=0(root) gid=0(root) groups=10(wheel),29(audio),81(dbus),777
+ $ docker run --rm --group-add audio --group-add nogroup --group-add 777 busybox id
+ uid=0(root) gid=0(root) groups=10(wheel),29(audio),99(nogroup),777
## Runtime privilege and Linux capabilities
@@ -1059,6 +1068,14 @@ one can use this flag:
--privileged=false: Give extended privileges to this container
--device=[]: Allows you to run devices inside the container without the --privileged flag.
+> **Note:**
+> With Docker 1.10 and greater, the default seccomp profile will also block
+> syscalls, regardless of `--cap-add` passed to the container. We recommend in
+> these cases to create your own custom seccomp profile based off our
+> [default](https://github.com/docker/docker/blob/master/profiles/seccomp/default.json).
+> Or if you don't want to run with the default seccomp profile, you can pass
+> `--security-opt=seccomp:unconfined` on run.
+
By default, Docker containers are "unprivileged" and cannot, for
example, run a Docker daemon inside a Docker container. This is because
by default a container is not allowed to access any devices, but a
@@ -1429,7 +1446,10 @@ The developer can set a default user to run the first process with the
Dockerfile `USER` instruction. When starting a container, the operator can override
the `USER` instruction by passing the `-u` option.
- -u="": Username or UID
+ -u="", --user="": Sets the username or UID used and optionally the groupname or GID for the specified command.
+
+ The followings examples are all valid:
+ --user=[ user | user:group | uid | uid:gid | user:gid | uid:group ]
> **Note:** if you pass a numeric uid, it must be in the range of 0-2147483647.
diff --git a/components/engine/docs/security/https/README.md b/components/engine/docs/security/https/README.md
index 1369913ed7..ff5538911c 100644
--- a/components/engine/docs/security/https/README.md
+++ b/components/engine/docs/security/https/README.md
@@ -7,26 +7,27 @@ draft = true
This is an initial attempt to make it easier to test the examples in the https.md
-doc
+doc.
-at this point, it has to be a manual thing, and I've been running it in boot2docker
+At this point, it has to be a manual thing, and I've been running it in boot2docker.
-so my process is
+My process is as following:
+
+ $ boot2docker ssh
+ $$ git clone https://github.com/docker/docker
+ $$ cd docker/docs/articles/https
+ $$ make cert
-$ boot2docker ssh
-$$ git clone https://github.com/docker/docker
-$$ cd docker/docs/articles/https
-$$ make cert
lots of things to see and manually answer, as openssl wants to be interactive
+
**NOTE:** make sure you enter the hostname (`boot2docker` in my case) when prompted for `Computer Name`)
-$$ sudo make run
-start another terminal
+ $$ sudo make run
-$ boot2docker ssh
-$$ cd docker/docs/articles/https
-$$ make client
+Start another terminal:
-the last will connect first with `--tls` and then with `--tlsverify`
+ $ boot2docker ssh
+ $$ cd docker/docs/articles/https
+ $$ make client
-both should succeed
+The last will connect first with `--tls` and then with `--tlsverify`, both should succeed.
diff --git a/components/engine/docs/security/seccomp.md b/components/engine/docs/security/seccomp.md
index 5bfed0f647..196d93810d 100644
--- a/components/engine/docs/security/seccomp.md
+++ b/components/engine/docs/security/seccomp.md
@@ -16,48 +16,48 @@ restrict the actions available within the container. The `seccomp()` system
call operates on the seccomp state of the calling process. You can use this
feature to restrict your application's access.
-This feature is available only if the kernel is configured with `CONFIG_SECCOMP`
-enabled.
+This feature is available only if Docker has been built with seccomp and the
+kernel is configured with `CONFIG_SECCOMP` enabled. To check if your kernel
+supports seccomp:
-> **Note**: On Ubuntu 14.04, Debian Wheezy, and Debian Jessie, you must download
-> the [latest static Docker Linux binary](../installation/binaries.md) to use
-> seccomp.
+```bash
+$ cat /boot/config-`uname -r` | grep CONFIG_SECCOMP=
+CONFIG_SECCOMP=y
+```
+
+> **Note**: seccomp profiles require seccomp 2.2.1 and are only
+> available starting with Debian 9 "Stretch", Ubuntu 15.10 "Wily", and
+> Fedora 22. To use this feature on Ubuntu 14.04, Debian Wheezy, or
+> Debian Jessie, you must download the [latest static Docker Linux binary](../installation/binaries.md).
+> This feature is currently *not* available on other distributions.
## Passing a profile for a container
The default seccomp profile provides a sane default for running containers with
-seccomp. It is moderately protective while providing wide application
-compatibility. The default Docker profile has layout in the following form:
+seccomp and disables around 44 system calls out of 300+. It is moderately protective while providing wide application
+compatibility. The default Docker profile (found [here](https://github.com/docker/docker/blob/master/profiles/seccomp/default.json) has a JSON layout in the following form:
-```
+```json
{
- "defaultAction": "SCMP_ACT_ALLOW",
- "syscalls": [
- {
- "name": "getcwd",
- "action": "SCMP_ACT_ERRNO"
- },
- {
- "name": "mount",
- "action": "SCMP_ACT_ERRNO"
- },
- {
- "name": "setns",
- "action": "SCMP_ACT_ERRNO"
- },
- {
- "name": "create_module",
- "action": "SCMP_ACT_ERRNO"
- },
- {
- "name": "chown",
- "action": "SCMP_ACT_ERRNO"
- },
- {
- "name": "chmod",
- "action": "SCMP_ACT_ERRNO"
- }
- ]
+ "defaultAction": "SCMP_ACT_ERRNO",
+ "architectures": [
+ "SCMP_ARCH_X86_64",
+ "SCMP_ARCH_X86",
+ "SCMP_ARCH_X32"
+ ],
+ "syscalls": [
+ {
+ "name": "accept",
+ "action": "SCMP_ACT_ALLOW",
+ "args": []
+ },
+ {
+ "name": "accept4",
+ "action": "SCMP_ACT_ALLOW",
+ "args": []
+ },
+ ...
+ ]
}
```
@@ -69,7 +69,7 @@ specifies the default policy:
$ docker run --rm -it --security-opt seccomp:/path/to/seccomp/profile.json hello-world
```
-### Syscalls blocked by the default profile
+### Significant syscalls blocked by the default profile
Docker's default seccomp profile is a whitelist which specifies the calls that
are allowed. The table below lists the significant (but not all) syscalls that
diff --git a/components/engine/docs/security/security.md b/components/engine/docs/security/security.md
index ec24d879d8..9eea663788 100644
--- a/components/engine/docs/security/security.md
+++ b/components/engine/docs/security/security.md
@@ -243,26 +243,16 @@ with e.g., special network topologies or shared filesystems, you can
expect to see tools to harden existing Docker containers without
affecting Docker's core.
-Recent improvements in Linux namespaces will soon allow to run
-full-featured containers without root privileges, thanks to the new user
-namespace. This is covered in detail [here](
-http://s3hh.wordpress.com/2013/07/19/creating-and-using-containers-without-privilege/).
-Moreover, this will solve the problem caused by sharing filesystems
-between host and guest, since the user namespace allows users within
-containers (including the root user) to be mapped to other users in the
-host system.
+As of Docker 1.10 User Namespaces are supported directly by the docker
+daemon. This feature allows for the root user in a container to be mapped
+to a non uid-0 user outside the container, which can help to mitigate the
+risks of container breakout. This facility is available but not enabled
+by default.
-Today, Docker does not directly support user namespaces, but they
-may still be utilized by Docker containers on supported kernels,
-by directly using the clone syscall, or utilizing the 'unshare'
-utility. Using this, some users may find it possible to drop
-more capabilities from their process as user namespaces provide
-an artificial capabilities set. Likewise, however, this artificial
-capabilities set may require use of 'capsh' to restrict the
-user-namespace capabilities set when using 'unshare'.
-
-Eventually, it is expected that Docker will have direct, native support
-for user-namespaces, simplifying the process of hardening containers.
+Refer to the [daemon command](../reference/commandline/daemon.md#daemon-user-namespace-options)
+in the command line reference for more information on this feature.
+Additional information on the implementation of User Namespaces in Docker
+can be found in this blog post.
## Conclusions
diff --git a/components/engine/docs/understanding-docker.md b/components/engine/docs/understanding-docker.md
index 1278f3902c..1dba248e0b 100644
--- a/components/engine/docs/understanding-docker.md
+++ b/components/engine/docs/understanding-docker.md
@@ -11,7 +11,6 @@ weight = -82
# Understand the architecture
-**What is Docker?**
Docker is an open platform for developing, shipping, and running applications.
Docker is designed to deliver your applications faster. With Docker you can
@@ -32,11 +31,11 @@ your hardware.
Surrounding the container is tooling and a platform which can help you in
several ways:
-* getting your applications (and supporting components) into Docker containers
-* distributing and shipping those containers to your teams for further development
+* Get your applications (and supporting components) into Docker containers
+* Distribute and ship those containers to your teams for further development
and testing
-* deploying those applications to your production environment,
- whether it is in a local data center or the Cloud.
+* Deploy those applications to your production environment,
+ whether it is in a local data center or the Cloud
## What can I use Docker for?
@@ -75,7 +74,7 @@ out of the resources you have.
Docker has two major components:
-* Docker: the open source containerization platform.
+* Docker Engine: the open source containerization platform.
* [Docker Hub](https://hub.docker.com): our Software-as-a-Service
platform for sharing and managing Docker containers.
@@ -103,11 +102,11 @@ interface to Docker. It accepts commands from the user and communicates back and
forth with a Docker daemon.
### Inside Docker
-To understand Docker's internals, you need to know about three components:
+To understand Docker's internals, you need to know about three resources:
-* Docker images.
-* Docker registries.
-* Docker containers.
+* Docker images
+* Docker registries
+* Docker containers
#### Docker images
@@ -124,6 +123,8 @@ upload or download images. The public Docker registry is provided with the
images for your use. These can be images you create yourself or you can use
images that others have previously created. Docker registries are the
**distribution** component of Docker.
+For more information, go to [Docker Registry](https://docs.docker.com/registry/overview/) and
+[Docker Trusted Registry](https://docs.docker.com/docker-trusted-registry/overview/).
#### Docker containers
Docker containers are similar to a directory. A Docker container holds everything that
@@ -132,17 +133,6 @@ image. Docker containers can be run, started, stopped, moved, and deleted. Each
container is an isolated and secure application platform. Docker containers are the
**run** component of Docker.
-## So how does Docker work?
-So far, we've learned that:
-
-1. You can build Docker images that hold your applications.
-2. You can create Docker containers from those Docker images to run your
- applications.
-3. You can share those Docker images via
- [Docker Hub](https://hub.docker.com) or your own registry.
-
-Let's look at how these elements combine together to make Docker work.
-
### How does a Docker image work?
We've already seen that Docker images are read-only templates from which Docker
containers are launched. Each image consists of a series of layers. Docker
@@ -163,27 +153,27 @@ or `fedora`, a base Fedora image. You can also use images of your own as the
basis for a new image, for example if you have a base Apache image you could use
this as the base of all your web application images.
-> **Note:** Docker usually gets these base images from
-> [Docker Hub](https://hub.docker.com).
+> **Note:** [Docker Hub](https://hub.docker.com) is a public registry and stores
+images.
Docker images are then built from these base images using a simple, descriptive
set of steps we call *instructions*. Each instruction creates a new layer in our
image. Instructions include actions like:
-* Run a command.
-* Add a file or directory.
-* Create an environment variable.
-* What process to run when launching a container from this image.
+* Run a command
+* Add a file or directory
+* Create an environment variable
+* What process to run when launching a container from this image
-These instructions are stored in a file called a `Dockerfile`. Docker reads this
-`Dockerfile` when you request a build of an image, executes the instructions, and
-returns a final image.
+These instructions are stored in a file called a `Dockerfile`. A `Dockerfile` is
+a text based script that contains instructions and commands for building the image
+from the base image. Docker reads this `Dockerfile` when you request a build of
+an image, executes the instructions, and returns a final image.
### How does a Docker registry work?
The Docker registry is the store for your Docker images. Once you build a Docker
-image you can *push* it to a public registry such as the one provided by [Docker
-Hub](https://hub.docker.com) or to your own registry running behind your
-firewall.
+image you can *push* it to a public registry such as [Docker Hub](https://hub.docker.com)
+or to your own registry running behind your firewall.
Using the Docker client, you can search for already published images and then
pull them down to your Docker host to build containers from them.
@@ -209,25 +199,24 @@ daemon to run a container.
$ docker run -i -t ubuntu /bin/bash
-Let's break down this command. The Docker client is launched using the `docker`
-binary with the `run` option telling it to launch a new container. The bare
-minimum the Docker client needs to tell the Docker daemon to run the container
-is:
+The Docker Engine client is launched using the `docker` binary with the `run` option
+running a new container. The bare minimum the Docker client needs to tell the
+Docker daemon to run the container is:
-* What Docker image to build the container from, here `ubuntu`, a base Ubuntu
-image;
+* What Docker image to build the container from, for example, `ubuntu`
* The command you want to run inside the container when it is launched,
-here `/bin/bash`, to start the Bash shell inside the new container.
+for example,`/bin/bash`
So what happens under the hood when we run this command?
-In order, Docker does the following:
+In order, Docker Engine does the following:
-- **Pulls the `ubuntu` image:** Docker checks for the presence of the `ubuntu`
-image and, if it doesn't exist locally on the host, then Docker downloads it from
-[Docker Hub](https://hub.docker.com). If the image already exists, then Docker
+- **Pulls the `ubuntu` image:** Docker Engine checks for the presence of the `ubuntu`
+image. If the image already exists, then Docker Engine uses it for the new container.
+If it doesn't exist locally on the host, then Docker Engine pulls it from
+[Docker Hub](https://hub.docker.com). If the image already exists, then Docker Engine
uses it for the new container.
-- **Creates a new container:** Once Docker has the image, it uses it to create a
+- **Creates a new container:** Once Docker Engine has the image, it uses it to create a
container.
- **Allocates a filesystem and mounts a read-write _layer_:** The container is created in
the file system and a read-write layer is added to the image.
@@ -238,7 +227,7 @@ Docker container to talk to the local host.
- **Captures and provides application output:** Connects and logs standard input, outputs
and errors for you to see how your application is running.
-You now have a running container! From here you can manage your container, interact with
+You now have a running container! Now you can manage your container, interact with
your application and then, when finished, stop and remove your container.
## The underlying technology
@@ -253,40 +242,37 @@ creates a set of *namespaces* for that container.
This provides a layer of isolation: each aspect of a container runs in its own
namespace and does not have access outside it.
-Some of the namespaces that Docker uses on Linux are:
+Some of the namespaces that Docker Engine uses on Linux are:
- - **The `pid` namespace:** Used for process isolation (PID: Process ID).
- - **The `net` namespace:** Used for managing network interfaces (NET:
+ - **The `pid` namespace:** Process isolation (PID: Process ID).
+ - **The `net` namespace:** Managing network interfaces (NET:
Networking).
- - **The `ipc` namespace:** Used for managing access to IPC
+ - **The `ipc` namespace:** Managing access to IPC
resources (IPC: InterProcess Communication).
- - **The `mnt` namespace:** Used for managing mount-points (MNT: Mount).
- - **The `uts` namespace:** Used for isolating kernel and version identifiers. (UTS: Unix
+ - **The `mnt` namespace:** Managing mount-points (MNT: Mount).
+ - **The `uts` namespace:** Isolating kernel and version identifiers. (UTS: Unix
Timesharing System).
### Control groups
-Docker on Linux also makes use of another technology called `cgroups` or control groups.
+Docker Engine on Linux also makes use of another technology called `cgroups` or control groups.
A key to running applications in isolation is to have them only use the
resources you want. This ensures containers are good multi-tenant citizens on a
-host. Control groups allow Docker to share available hardware resources to
+host. Control groups allow Docker Engine to share available hardware resources to
containers and, if required, set up limits and constraints. For example,
limiting the memory available to a specific container.
### Union file systems
Union file systems, or UnionFS, are file systems that operate by creating layers,
-making them very lightweight and fast. Docker uses union file systems to provide
-the building blocks for containers. Docker can make use of several union file system variants
+making them very lightweight and fast. Docker Engine uses union file systems to provide
+the building blocks for containers. Docker Engine can make use of several union file system variants
including: AUFS, btrfs, vfs, and DeviceMapper.
### Container format
-Docker combines these components into a wrapper we call a container format. The
+Docker Engine combines these components into a wrapper we call a container format. The
default container format is called `libcontainer`. In the future, Docker may
support other container formats, for example, by integrating with BSD Jails
or Solaris Zones.
## Next steps
-### Installing Docker
-Visit the [installation section](installation/index.md#installation).
-
-### The Docker user guide
-[Learn Docker in depth](userguide/index.md).
+Read about [Installing Docker Engine](installation/index.md#installation).
+Learn about the [Docker Engine User Guide](userguide/index.md).
diff --git a/components/engine/docs/userguide/containers/dockerimages.md b/components/engine/docs/userguide/containers/dockerimages.md
index 74387a5166..7a8b96569e 100644
--- a/components/engine/docs/userguide/containers/dockerimages.md
+++ b/components/engine/docs/userguide/containers/dockerimages.md
@@ -19,14 +19,14 @@ used Docker images that already exist, for example the `ubuntu` image and the
You also discovered that Docker stores downloaded images on the Docker host. If
an image isn't already present on the host then it'll be downloaded from a
-registry: by default the [Docker Hub Registry](https://registry.hub.docker.com).
+registry: by default the [Docker Hub Registry](https://hub.docker.com).
In this section you're going to explore Docker images a bit more
including:
* Managing and working with images locally on your Docker host.
* Creating basic images.
-* Uploading images to [Docker Hub Registry](https://registry.hub.docker.com).
+* Uploading images to [Docker Hub Registry](https://hub.docker.com).
## Listing images on the host
@@ -455,7 +455,7 @@ step-by-step. You can see that each step creates a new container, runs
the instruction inside that container and then commits that change -
just like the `docker commit` work flow you saw earlier. When all the
instructions have executed you're left with the `97feabe5d2ed` image
-(also helpfully tagged as `ouruser/sinatra:v2`) and all intermediate
+(also helpfuly tagged as `ouruser/sinatra:v2`) and all intermediate
containers will get removed to clean things up.
> **Note:**
@@ -521,7 +521,7 @@ You can also reference by digest in `create`, `run`, and `rmi` commands, as well
Once you've built or created a new image you can push it to [Docker
Hub](https://hub.docker.com) using the `docker push` command. This
allows you to share it with others, either publicly, or push it into [a
-private repository](https://registry.hub.docker.com/plans/).
+private repository](https://hub.docker.com/account/billing-plans/).
$ docker push ouruser/sinatra
The push refers to a repository [ouruser/sinatra] (len: 1)
diff --git a/components/engine/docs/userguide/containers/dockerrepos.md b/components/engine/docs/userguide/containers/dockerrepos.md
index 257f87635b..9be9f53c98 100644
--- a/components/engine/docs/userguide/containers/dockerrepos.md
+++ b/components/engine/docs/userguide/containers/dockerrepos.md
@@ -33,15 +33,11 @@ Docker itself provides access to Docker Hub services via the `docker search`,
### Account creation and login
Typically, you'll want to start by creating an account on Docker Hub (if you haven't
already) and logging in. You can create your account directly on
-[Docker Hub](https://hub.docker.com/account/signup/), or by running:
+[Docker Hub](https://hub.docker.com/account/signup/).
$ docker login
-This will prompt you for a user name, which will become the public namespace for your
-public repositories.
-If your user name is available, Docker will prompt you to enter a password and your
-e-mail address. It will then automatically log you in. You can now commit and
-push your own images up to your repos on Docker Hub.
+You can now commit and push your own images up to your repos on Docker Hub.
> **Note:**
> Your authentication credentials will be stored in the `~/.docker/config.json`
@@ -124,7 +120,7 @@ information [here](https://docs.docker.com/docker-hub/).
Sometimes you have images you don't want to make public and share with
everyone. So Docker Hub allows you to have private repositories. You can
-sign up for a plan [here](https://registry.hub.docker.com/plans/).
+sign up for a plan [here](https://hub.docker.com/account/billing-plans/).
### Organizations and teams
@@ -132,7 +128,7 @@ One of the useful aspects of private repositories is that you can share
them only with members of your organization or team. Docker Hub lets you
create organizations where you can collaborate with your colleagues and
manage private repositories. You can learn how to create and manage an organization
-[here](https://registry.hub.docker.com/account/organizations/).
+[here](https://hub.docker.com/organizations/).
### Automated Builds
@@ -144,8 +140,8 @@ triggering a build and update when you push a commit.
#### To setup an Automated Build
1. Create a [Docker Hub account](https://hub.docker.com/) and login.
-2. Link your GitHub or Bitbucket account through the ["Link Accounts"](https://registry.hub.docker.com/account/accounts/) menu.
-3. [Configure an Automated Build](https://registry.hub.docker.com/builds/add/).
+2. Link your GitHub or Bitbucket account on the ["Linked Accounts & Services"](https://hub.docker.com/account/authorized-services/) page.
+3. Select "Create Automated Build" from the "Create" dropdown menu
4. Pick a GitHub or Bitbucket project that has a `Dockerfile` that you want to build.
5. Pick the branch you want to build (the default is the `master` branch).
6. Give the Automated Build a name.
diff --git a/components/engine/docs/userguide/containers/dockervolumes.md b/components/engine/docs/userguide/containers/dockervolumes.md
index 63d7e513d7..bdea82749b 100644
--- a/components/engine/docs/userguide/containers/dockervolumes.md
+++ b/components/engine/docs/userguide/containers/dockervolumes.md
@@ -67,7 +67,7 @@ The output will provide details on the container configurations including the
volumes. The output should look something similar to the following:
...
- Mounts": [
+ "Mounts": [
{
"Name": "fac362...80535",
"Source": "/var/lib/docker/volumes/fac362...80535/_data",
@@ -159,6 +159,48 @@ user with access to host and its mounted directory.
>should be portable. A host directory wouldn't be available on all potential
>hosts.
+### Mount a shared-storage volume as a data volume
+
+In addition to mounting a host directory in your container, some Docker
+[volume plugins](../../extend/plugins_volume.md) allow you to
+provision and mount shared storage, such as iSCSI, NFS, or FC.
+
+A benefit of using shared volumes is that they are host-independent. This
+means that a volume can be made available on any host that a container is
+started on as long as it has access to the shared storage backend, and has
+the plugin installed.
+
+One way to use volume drivers is through the `docker run` command.
+Volume drivers create volumes by name, instead of by path like in
+the other examples.
+
+The following command creates a named volume, called `my-named-volume`,
+using the `flocker` volume driver, and makes it available within the container
+at `/opt/webapp`:
+
+```bash
+$ docker run -d -P \
+ --volume-driver=flocker \
+ -v my-named-volume:/opt/webapp \
+ --name web training/webapp python app.py
+```
+
+You may also use the `docker volume create` command, to create a volume before
+using it in a container.
+
+The following example also creates the `my-named-volume` volume, this time
+using the `docker volume create` command.
+
+```bash
+$ docker volume create -d flocker --name my-named-volume -o size=20GB
+$ docker run -d -P \
+ -v my-named-volume:/opt/webapp \
+ --name web training/webapp python app.py
+```
+
+A list of available plugins, including volume plugins, is available
+[here](../../extend/plugins.md).
+
### Volume labels
Labeling systems like SELinux require that proper labels are placed on volume
diff --git a/components/engine/docs/userguide/eng-image/dockerfile_best-practices.md b/components/engine/docs/userguide/eng-image/dockerfile_best-practices.md
index 1c51af7037..ea89af49d3 100644
--- a/components/engine/docs/userguide/eng-image/dockerfile_best-practices.md
+++ b/components/engine/docs/userguide/eng-image/dockerfile_best-practices.md
@@ -130,7 +130,7 @@ various instructions available for use in a `Dockerfile`.
[Dockerfile reference for the FROM instruction](../../reference/builder.md#from)
Whenever possible, use current Official Repositories as the basis for your
-image. We recommend the [Debian image](https://registry.hub.docker.com/_/debian/)
+image. We recommend the [Debian image](https://hub.docker.com/_/debian/)
since it’s very tightly controlled and kept extremely minimal (currently under
100 mb), while still being a full distribution.
@@ -365,7 +365,7 @@ The `ENTRYPOINT` instruction can also be used in combination with a helper
script, allowing it to function in a similar way to the command above, even
when starting the tool may require more than one step.
-For example, the [Postgres Official Image](https://registry.hub.docker.com/_/postgres/)
+For example, the [Postgres Official Image](https://hub.docker.com/_/postgres/)
uses the following script as its `ENTRYPOINT`:
```bash
@@ -481,10 +481,10 @@ allowing the `Dockerfile` author to make a choice.
These Official Repositories have exemplary `Dockerfile`s:
-* [Go](https://registry.hub.docker.com/_/golang/)
-* [Perl](https://registry.hub.docker.com/_/perl/)
-* [Hy](https://registry.hub.docker.com/_/hylang/)
-* [Rails](https://registry.hub.docker.com/_/rails)
+* [Go](https://hub.docker.com/_/golang/)
+* [Perl](https://hub.docker.com/_/perl/)
+* [Hy](https://hub.docker.com/_/hylang/)
+* [Rails](https://hub.docker.com/_/rails)
## Additional resources:
diff --git a/components/engine/docs/userguide/index.md b/components/engine/docs/userguide/index.md
index 1bffafc4c8..2509997518 100644
--- a/components/engine/docs/userguide/index.md
+++ b/components/engine/docs/userguide/index.md
@@ -1,8 +1,8 @@
-# User guide
+# Docker Engine user guide
+
+This guide helps users learn how to use Docker Engine.
+
+- [Introduction to Engine user guide](intro.md)
+
+## Learn by example
+
+- [Hello world in a container](containers/dockerizing.md)
+- [Build your own images](containers/dockerimages.md)
+- [Network containers](containers/networkingcontainers.md)
+- [Run a simple application](containers/usingdocker.md)
+- [Manage data in containers](containers/dockervolumes.md)
+- [Store images on Docker Hub](containers/dockerrepos.md)
+
+## Work with images
+
+- [Best practices for writing Dockerfiles](eng-image/dockerfile_best-practices.md)
+- [Create a base image](eng-image/baseimages.md)
+- [Image management](eng-image/image_management.md)
+
+## Manage storage drivers
+
+- [Understand images, containers, and storage drivers](storagedriver/imagesandcontainers.md)
+- [Select a storage driver](storagedriver/selectadriver.md)
+- [AUFS storage in practice](storagedriver/aufs-driver.md)
+- [Btrfs storage in practice](storagedriver/btrfs-driver.md)
+- [Device Mapper storage in practice](storagedriver/device-mapper-driver.md)
+- [OverlayFS storage in practice](storagedriver/overlayfs-driver.md)
+- [ZFS storage in practice](storagedriver/zfs-driver.md)
+
+## Configure networks
+
+- [Understand Docker container networks](networking/dockernetworks.md)
+- [Embedded DNS server in user-defined networks](networking/configure-dns.md)
+- [Get started with multi-host networking](networking/get-started-overlay.md)
+- [Work with network commands](networking/work-with-networks.md)
+
+### Work with the default network
+
+- [Understand container communication](networking/default_network/container-communication.md)
+- [Legacy container links](networking/default_network/dockerlinks.md)
+- [Binding container ports to the host](networking/default_network/binding.md)
+- [Build your own bridge](networking/default_network/build-bridges.md)
+- [Configure container DNS](networking/default_network/configure-dns.md)
+- [Customize the docker0 bridge](networking/default_network/custom-docker0.md)
+- [IPv6 with Docker](networking/default_network/ipv6.md)
+
+## Misc
+
+- [Apply custom metadata](labels-custom-metadata.md)
diff --git a/components/engine/docs/userguide/networking/configure-dns.md b/components/engine/docs/userguide/networking/configure-dns.md
index b87436fada..d248f4294f 100644
--- a/components/engine/docs/userguide/networking/configure-dns.md
+++ b/components/engine/docs/userguide/networking/configure-dns.md
@@ -16,7 +16,7 @@ user-defined networks works differently compared to the containers connected
to `default bridge` network.
> **Note**: In order to maintain backward compatibility, the DNS configuration
-> in `default bridge` network is retained with no behaviorial change.
+> in `default bridge` network is retained with no behavioral change.
> Please refer to the [DNS in default bridge network](default_network/configure-dns.md)
> for more information on DNS configuration in the `default bridge` network.
@@ -75,7 +75,7 @@ Various container options that affect container domain name services.
of the container identified by CONTAINER_NAME. When using --link
the embedded DNS will guarantee that localized lookup result only on that
container where the --link is used. This lets processes inside the new container
- connect to container without without having to know its name or IP.
+ connect to container without having to know its name or IP.
diff --git a/components/engine/docs/userguide/networking/default_network/configure-dns.md b/components/engine/docs/userguide/networking/default_network/configure-dns.md
index ab87c82a79..2703aca1d0 100644
--- a/components/engine/docs/userguide/networking/default_network/configure-dns.md
+++ b/components/engine/docs/userguide/networking/default_network/configure-dns.md
@@ -61,7 +61,7 @@ Four different options affect container domain name services.
Using this option as you run a container gives the new
container's /etc/hosts an extra entry named
ALIAS that points to the IP address of the container
- identified by CONTAINER_NAME_or_ID. This lets processes
+ identified by CONTAINER_NAME_or_ID. This lets processes
inside the new container connect to the hostname ALIAS
without having to know its IP. The --link= option is
discussed in more detail below. Because Docker may assign a different IP
diff --git a/components/engine/docs/userguide/networking/default_network/options.md b/components/engine/docs/userguide/networking/default_network/options.md
deleted file mode 100644
index 612dffbc51..0000000000
--- a/components/engine/docs/userguide/networking/default_network/options.md
+++ /dev/null
@@ -1,141 +0,0 @@
-
-
-
-
-# Quick guide to the options
-Here is a quick list of the networking-related Docker command-line options, in case it helps you find the section below that you are looking for.
-
-Some networking command-line options can only be supplied to the Docker server when it starts up, and cannot be changed once it is running:
-- `-b BRIDGE` or `--bridge=BRIDGE` -- see
-
- [Building your own bridge](#bridge-building)
-
-- `--bip=CIDR` -- see
-
- [Customizing docker0](#docker0)
-
-- `--default-gateway=IP_ADDRESS` -- see
-
- [How Docker networks a container](#container-networking)
-
-- `--default-gateway-v6=IP_ADDRESS` -- see
-
- [IPv6](#ipv6)
-
-- `--fixed-cidr` -- see
-
- [Customizing docker0](#docker0)
-
-- `--fixed-cidr-v6` -- see
-
- [IPv6](#ipv6)
-
-- `-H SOCKET...` or `--host=SOCKET...` --
-
- This might sound like it would affect container networking,
-
- but it actually faces in the other direction:
-
- it tells the Docker server over what channels
-
- it should be willing to receive commands
-
- like "run container" and "stop container."
-
-- `--icc=true|false` -- see
-
- [Communication between containers](#between-containers)
-
-- `--ip=IP_ADDRESS` -- see
-
- [Binding container ports](#binding-ports)
-
-- `--ipv6=true|false` -- see
-
- [IPv6](#ipv6)
-
-- `--ip-forward=true|false` -- see
-
- [Communication between containers and the wider world](#the-world)
-
-- `--iptables=true|false` -- see
-
- [Communication between containers](#between-containers)
-
-- `--mtu=BYTES` -- see
-
- [Customizing docker0](#docker0)
-
-- `--userland-proxy=true|false` -- see
-
- [Binding container ports](#binding-ports)
-
-There are three networking options that can be supplied either at startup or when `docker run` is invoked. When provided at startup, set the default value that `docker run` will later use if the options are not specified:
-- `--dns=IP_ADDRESS...` -- see
-
- [Configuring DNS](#dns)
-
-- `--dns-search=DOMAIN...` -- see
-
- [Configuring DNS](#dns)
-
-- `--dns-opt=OPTION...` -- see
-
- [Configuring DNS](#dns)
-
-Finally, several networking options can only be provided when calling `docker run` because they specify something specific to one container:
-- `-h HOSTNAME` or `--hostname=HOSTNAME` -- see
-
- [Configuring DNS](#dns) and
-
- [How Docker networks a container](#container-networking)
-
-- `--link=CONTAINER_NAME_or_ID:ALIAS` -- see
-
- [Configuring DNS](#dns) and
-
- [Communication between containers](#between-containers)
-
-- `--net=bridge|none|container:NAME_or_ID|host` -- see
-
- [How Docker networks a container](#container-networking)
-
-- `--mac-address=MACADDRESS...` -- see
-
- [How Docker networks a container](#container-networking)
-
-- `-p SPEC` or `--publish=SPEC` -- see
-
- [Binding container ports](#binding-ports)
-
-- `-P` or `--publish-all=true|false` -- see
-
- [Binding container ports](#binding-ports)
-
-To supply networking options to the Docker server at startup, use the `DOCKER_OPTS` variable in the Docker upstart configuration file. For Ubuntu, edit the variable in `/etc/default/docker` or `/etc/sysconfig/docker` for CentOS.
-
-The following example illustrates how to configure Docker on Ubuntu to recognize a newly built bridge.
-
-Edit the `/etc/default/docker` file:
-
-```
-$ echo 'DOCKER_OPTS="-b=bridge0"' >> /etc/default/docker
-```
-
-Then restart the Docker server.
-
-```
-$ sudo service docker start
-```
-
-For additional information on bridges, see [building your own bridge](#building-your-own-bridge) later on this page.
diff --git a/components/engine/docs/userguide/networking/default_network/saveme.md b/components/engine/docs/userguide/networking/default_network/saveme.md
deleted file mode 100644
index f0ef85e8c0..0000000000
--- a/components/engine/docs/userguide/networking/default_network/saveme.md
+++ /dev/null
@@ -1,28 +0,0 @@
-
-
-
-
-
-## A Brief introduction to networking and docker
-When Docker starts, it creates a virtual interface named `docker0` on the host machine. It randomly chooses an address and subnet from the private range defined by [RFC 1918](http://tools.ietf.org/html/rfc1918) that are not in use on the host machine, and assigns it to `docker0`. Docker made the choice `172.17.42.1/16` when I started it a few minutes ago, for example -- a 16-bit netmask providing 65,534 addresses for the host machine and its containers. The MAC address is generated using the IP address allocated to the container to avoid ARP collisions, using a range from `02:42:ac:11:00:00` to `02:42:ac:11:ff:ff`.
-
-> **Note:** This document discusses advanced networking configuration and options for Docker. In most cases you won't need this information. If you're looking to get started with a simpler explanation of Docker networking and an introduction to the concept of container linking see the [Docker User Guide](dockerlinks.md).
-
-But `docker0` is no ordinary interface. It is a virtual _Ethernet bridge_ that automatically forwards packets between any other network interfaces that are attached to it. This lets containers communicate both with the host machine and with each other. Every time Docker creates a container, it creates a pair of "peer" interfaces that are like opposite ends of a pipe -- a packet sent on one will be received on the other. It gives one of the peers to the container to become its `eth0` interface and keeps the other peer, with a unique name like `vethAQI2QT`, out in the namespace of the host machine. By binding every `veth*` interface to the `docker0` bridge, Docker creates a virtual subnet shared between the host machine and every Docker container.
-
-The remaining sections of this document explain all of the ways that you can use Docker options and -- in advanced cases -- raw Linux networking commands to tweak, supplement, or entirely replace Docker's default networking configuration.
-
-## Editing networking config files
-Starting with Docker v.1.2.0, you can now edit `/etc/hosts`, `/etc/hostname` and `/etc/resolve.conf` in a running container. This is useful if you need to install bind or other services that might override one of those files.
-
-Note, however, that changes to these files will not be saved by `docker commit`, nor will they be saved during `docker run`. That means they won't be saved in the image, nor will they persist when a container is restarted; they will only "stick" in a running container.
diff --git a/components/engine/docs/userguide/networking/default_network/tools.md b/components/engine/docs/userguide/networking/default_network/tools.md
deleted file mode 100644
index 545c1e04c3..0000000000
--- a/components/engine/docs/userguide/networking/default_network/tools.md
+++ /dev/null
@@ -1,83 +0,0 @@
-
-
-
-
-# Tools and examples
-Before diving into the following sections on custom network topologies, you might be interested in glancing at a few external tools or examples of the same kinds of configuration. Here are two:
-- Jérôme Petazzoni has created a `pipework` shell script to help you
-
- connect together containers in arbitrarily complex scenarios:
-
- [https://github.com/jpetazzo/pipework](https://github.com/jpetazzo/pipework)
-
-- Brandon Rhodes has created a whole network topology of Docker
-
- containers for the next edition of Foundations of Python Network
-
- Programming that includes routing, NAT'd firewalls, and servers that
-
- offer HTTP, SMTP, POP, IMAP, Telnet, SSH, and FTP:
-
- [https://github.com/brandon-rhodes/fopnp/tree/m/playground](https://github.com/brandon-rhodes/fopnp/tree/m/playground)
-
-Both tools use networking commands very much like the ones you saw in the previous section, and will see in the following sections.
-
-# Building a point-to-point connection
-
-
-By default, Docker attaches all containers to the virtual subnet implemented by `docker0`. You can create containers that are each connected to some different virtual subnet by creating your own bridge as shown in [Building your own bridge](#bridge-building), starting each container with `docker run --net=none`, and then attaching the containers to your bridge with the shell commands shown in [How Docker networks a container](#container-networking).
-
-But sometimes you want two particular containers to be able to communicate directly without the added complexity of both being bound to a host-wide Ethernet bridge.
-
-The solution is simple: when you create your pair of peer interfaces, simply throw _both_ of them into containers, and configure them as classic point-to-point links. The two containers will then be able to communicate directly (provided you manage to tell each container the other's IP address, of course). You might adjust the instructions of the previous section to go something like this:
-
-```
-# Start up two containers in two terminal windows
-
-$ docker run -i -t --rm --net=none base /bin/bash
-root@1f1f4c1f931a:/#
-
-$ docker run -i -t --rm --net=none base /bin/bash
-root@12e343489d2f:/#
-
-# Learn the container process IDs
-# and create their namespace entries
-
-$ docker inspect -f '{{.State.Pid}}' 1f1f4c1f931a
-2989
-$ docker inspect -f '{{.State.Pid}}' 12e343489d2f
-3004
-$ sudo mkdir -p /var/run/netns
-$ sudo ln -s /proc/2989/ns/net /var/run/netns/2989
-$ sudo ln -s /proc/3004/ns/net /var/run/netns/3004
-
-# Create the "peer" interfaces and hand them out
-
-$ sudo ip link add A type veth peer name B
-
-$ sudo ip link set A netns 2989
-$ sudo ip netns exec 2989 ip addr add 10.1.1.1/32 dev A
-$ sudo ip netns exec 2989 ip link set A up
-$ sudo ip netns exec 2989 ip route add 10.1.1.2/32 dev A
-
-$ sudo ip link set B netns 3004
-$ sudo ip netns exec 3004 ip addr add 10.1.1.2/32 dev B
-$ sudo ip netns exec 3004 ip link set B up
-$ sudo ip netns exec 3004 ip route add 10.1.1.1/32 dev B
-```
-
-The two containers should now be able to ping each other and make connections successfully. Point-to-point links like this do not depend on a subnet nor a netmask, but on the bare assertion made by `ip route` that some other single IP address is connected to a particular network interface.
-
-Note that point-to-point links can be safely combined with other kinds of network connectivity -- there is no need to start the containers with `--net=none` if you want point-to-point links to be an addition to the container's normal networking instead of a replacement.
-
-A final permutation of this pattern is to create the point-to-point link between the Docker host and one container, which would allow the host to communicate with that one container on some single IP address and thus communicate "out-of-band" of the bridge that connects the other, more usual containers. But unless you have very specific networking needs that drive you to such a solution, it is probably far preferable to use `--icc=false` to lock down inter-container communication, as we explored earlier.
diff --git a/components/engine/docs/userguide/networking/dockernetworks.md b/components/engine/docs/userguide/networking/dockernetworks.md
index b9f1a63b44..1848e7a7a9 100644
--- a/components/engine/docs/userguide/networking/dockernetworks.md
+++ b/components/engine/docs/userguide/networking/dockernetworks.md
@@ -46,7 +46,7 @@ by default. You can see this bridge as part of a host's network stack by using
the `ifconfig` command on the host.
```
-ubuntu@ip-172-31-36-118:~$ ifconfig
+$ ifconfig
docker0 Link encap:Ethernet HWaddr 02:42:47:bc:3a:eb
inet addr:172.17.0.1 Bcast:0.0.0.0 Mask:255.255.0.0
inet6 addr: fe80::42:47ff:febc:3aeb/64 Scope:Link
@@ -60,16 +60,16 @@ docker0 Link encap:Ethernet HWaddr 02:42:47:bc:3a:eb
The `none` network adds a container to a container-specific network stack. That container lacks a network interface. Attaching to such a container and looking at it's stack you see this:
```
-ubuntu@ip-172-31-36-118:~$ docker attach nonenetcontainer
+$ docker attach nonenetcontainer
-/ # cat /etc/hosts
+root@0cb243cd1293:/# cat /etc/hosts
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
-/ # ifconfig
+root@0cb243cd1293:/# ifconfig
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
@@ -79,23 +79,24 @@ lo Link encap:Local Loopback
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
-/ #
+root@0cb243cd1293:/#
```
>**Note**: You can detach from the container and leave it running with `CTRL-p CTRL-q`.
The `host` network adds a container on the hosts network stack. You'll find the
network configuration inside the container is identical to the host.
-With the exception of the the `bridge` network, you really don't need to
+With the exception of the `bridge` network, you really don't need to
interact with these default networks. While you can list and inspect them, you
cannot remove them. They are required by your Docker installation. However, you
can add your own user-defined networks and these you can remove when you no
longer need them. Before you learn more about creating your own networks, it is
-worth looking at the `default` network a bit.
+worth looking at the default `bridge` network a bit.
### The default bridge network in detail
-The default bridge network is present on all Docker hosts. The `docker network inspect`
+The default `bridge` network is present on all Docker hosts. The `docker network inspect`
+command returns information about a network:
```
$ docker network inspect bridge
@@ -189,7 +190,7 @@ You can `attach` to a running `container` and investigate its configuration:
```
$ docker attach container1
-/ # ifconfig
+root@0cb243cd1293:/# ifconfig
ifconfig
eth0 Link encap:Ethernet HWaddr 02:42:AC:11:00:02
inet addr:172.17.0.2 Bcast:0.0.0.0 Mask:255.255.0.0
@@ -213,7 +214,7 @@ lo Link encap:Local Loopback
Then use `ping` for about 3 seconds to test the connectivity of the containers on this `bridge` network.
```
-/ # ping -w3 172.17.0.3
+root@0cb243cd1293:/# ping -w3 172.17.0.3
PING 172.17.0.3 (172.17.0.3): 56 data bytes
64 bytes from 172.17.0.3: seq=0 ttl=64 time=0.096 ms
64 bytes from 172.17.0.3: seq=1 ttl=64 time=0.080 ms
@@ -227,7 +228,7 @@ round-trip min/avg/max = 0.074/0.083/0.096 ms
Finally, use the `cat` command to check the `container1` network configuration:
```
-/ # cat /etc/hosts
+root@0cb243cd1293:/# cat /etc/hosts
172.17.0.2 3386a527aa08
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
@@ -241,7 +242,7 @@ To detach from a `container1` and leave it running use `CTRL-p CTRL-q`.Then, att
```
$ docker attach container2
-/ # ifconfig
+root@0cb243cd1293:/# ifconfig
eth0 Link encap:Ethernet HWaddr 02:42:AC:11:00:03
inet addr:172.17.0.3 Bcast:0.0.0.0 Mask:255.255.0.0
inet6 addr: fe80::42:acff:fe11:3/64 Scope:Link
@@ -260,7 +261,7 @@ lo Link encap:Local Loopback
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
-/ # ping -w3 172.17.0.2
+root@0cb243cd1293:/# ping -w3 172.17.0.2
PING 172.17.0.2 (172.17.0.2): 56 data bytes
64 bytes from 172.17.0.2: seq=0 ttl=64 time=0.067 ms
64 bytes from 172.17.0.2: seq=1 ttl=64 time=0.075 ms
@@ -279,7 +280,7 @@ ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
```
-The default `docker0` bridge network supports the use of port mapping and `docker run --link` to allow communications between containers in the `docker0` network. These techniques are cumbersome to set up and prone to error. While they are still available to you as techniques, it is better to avoid them and define your own bridge networks instead.
+The default `docker0` bridge network supports the use of port mapping and `docker run --link` to allow communications between containers in the `docker0` network. These techniques are cumbersome to set up and prone to error. While they are still available to you as techniques, it is better to avoid them and define your own bridge networks instead.
## User-defined networks
@@ -421,7 +422,7 @@ Once you have several machines provisioned, you can use Docker Swarm to quickly
form them into a swarm which includes a discovery service as well.
To create an overlay network, you configure options on the `daemon` on each
-Docker Engine for use with `overlay` network. There are two options to set:
+Docker Engine for use with `overlay` network. There are three options to set:
@@ -483,7 +484,7 @@ built-in network drivers. For example:
$ docker network create --driver weave mynet
-You can inspect it, add containers too and from it, and so forth. Of course,
+You can inspect it, add containers to and from it, and so forth. Of course,
different plugins may make use of different technologies or frameworks. Custom
networks can include features not present in Docker's default networks. For more
information on writing plugins, see [Extending Docker](../../extend/index.md) and
diff --git a/components/engine/docs/userguide/networking/get-started-overlay.md b/components/engine/docs/userguide/networking/get-started-overlay.md
index 39d7da9169..89d5b2ca59 100644
--- a/components/engine/docs/userguide/networking/get-started-overlay.md
+++ b/components/engine/docs/userguide/networking/get-started-overlay.md
@@ -19,6 +19,7 @@ some pre-existing conditions before you can create one. These conditions are:
* Access to a key-value store. Docker supports Consul, Etcd, and ZooKeeper (Distributed store) key-value stores.
* A cluster of hosts with connectivity to the key-value store.
* A properly configured Engine `daemon` on each host in the cluster.
+* Hosts within the cluster must have unique hostnames because the key-value store uses the hostnames to identify cluster members.
Though Docker Machine and Docker Swarm are not mandatory to experience Docker
multi-host networking, this example uses them to illustrate how they are
@@ -54,21 +55,20 @@ key-value stores. This example uses Consul.
instance using the [consul image from Docker
Hub](https://hub.docker.com/r/progrium/consul/). You'll do this in the next step.
-3. Start a `progrium/consul` container running on the `mh-keystore` machine.
+3. Set your local environment to the `mh-keystore` machine.
- $ docker $(docker-machine config mh-keystore) run -d \
+ $ eval "$(docker-machine env mh-keystore)"
+
+4. Start a `progrium/consul` container running on the `mh-keystore` machine.
+
+ $ docker run -d \
-p "8500:8500" \
-h "consul" \
progrium/consul -server -bootstrap
- A bash expansion `$(docker-machine config mh-keystore)` is used to pass the
- connection configuration to the `docker run` command. The client starts a
- `progrium/consul` image running in the `mh-keystore` machine. The server is
- called `consul` and is listening on port `8500`.
-
-4. Set your local environment to the `mh-keystore` machine.
-
- $ eval "$(docker-machine env mh-keystore)"
+ The client starts a `progrium/consul` image running in the
+ `mh-keystore` machine. The server is called `consul` and is
+ listening on port `8500`.
5. Run the `docker ps` command to see the `consul` container.
diff --git a/components/engine/docs/userguide/networking/work-with-networks.md b/components/engine/docs/userguide/networking/work-with-networks.md
index b668bc1c77..d2ac4693af 100644
--- a/components/engine/docs/userguide/networking/work-with-networks.md
+++ b/components/engine/docs/userguide/networking/work-with-networks.md
@@ -62,7 +62,7 @@ $ docker network inspect simple-network
Unlike `bridge` networks, `overlay` networks require some pre-existing conditions
before you can create one. These conditions are:
-* Access to a key-value store. Engine supports Consul Etcd, and ZooKeeper (Distributed store) key-value stores.
+* Access to a key-value store. Engine supports Consul, Etcd, and ZooKeeper (Distributed store) key-value stores.
* A cluster of hosts with connectivity to the key-value store.
* A properly configured Engine `daemon` on each host in the swarm.
@@ -111,7 +111,13 @@ The following are those options and the equivalent docker daemon flags used for
| `com.docker.network.bridge.enable_icc` | `--icc` | Enable or Disable Inter Container Connectivity |
| `com.docker.network.bridge.host_binding_ipv4` | `--ip` | Default IP when binding container ports |
| `com.docker.network.mtu` | `--mtu` | Set the containers network MTU |
-| `com.docker.network.enable_ipv6` | `--ipv6` | Enable IPv6 networking |
+
+The following arguments can be passed to `docker network create` for any network driver.
+
+| Argument | Equivalent | Description |
+|--------------|------------|------------------------------------------|
+| `--internal` | - | Restricts external access to the network |
+| `--ipv6` | `--ipv6` | Enable IPv6 networking |
For example, now let's use `-o` or `--opt` options to specify an IP address binding when publishing ports:
@@ -312,6 +318,7 @@ lo Link encap:Local Loopback
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
+```
On the `isolated_nw` which was user defined, the Docker embedded DNS server enables name resolution for other containers in the network. Inside of `container2` it is possible to ping `container3` by name.
@@ -376,7 +383,7 @@ You can connect both running and non-running containers to a network. However,
### Linking containers in user-defined networks
-In the above example, container_2 was able to resolve container_3's name automatically
+In the above example, `container2` was able to resolve `container3`'s name automatically
in the user defined network `isolated_nw`, but the name resolution did not succeed
automatically in the default `bridge` network. This is expected in order to maintain
backward compatibility with [legacy link](default_network/dockerlinks.md).
@@ -396,7 +403,7 @@ Comparing the above 4 functionalities with the non-default user-defined networks
* ability to dynamically attach and detach to multiple networks
* supports the `--link` option to provide name alias for the linked container
-Continuing with the above example, create another container `container_4` in `isolated_nw`
+Continuing with the above example, create another container `container4` in `isolated_nw`
with `--link` to provide additional name resolution using alias for other containers in
the same network.
@@ -405,26 +412,26 @@ $ docker run --net=isolated_nw -itd --name=container4 --link container5:c5 busyb
01b5df970834b77a9eadbaff39051f237957bd35c4c56f11193e0594cfd5117c
```
-With the help of `--link` container4 will be able to reach container5 using the
+With the help of `--link` `container4` will be able to reach `container5` using the
aliased name `c5` as well.
-Please note that while creating container4, we linked to a container named `container5`
+Please note that while creating `container4`, we linked to a container named `container5`
which is not created yet. That is one of the differences in behavior between the
-`legacy link` in default `bridge` network and the new `link` functionality in user defined
-networks. The `legacy link` is static in nature and it hard-binds the container with the
-alias and it doesnt tolerate linked container restarts. While the new `link` functionality
+*legacy link* in default `bridge` network and the new *link* functionality in user defined
+networks. The *legacy link* is static in nature and it hard-binds the container with the
+alias and it doesn't tolerate linked container restarts. While the new *link* functionality
in user defined networks are dynamic in nature and supports linked container restarts
including tolerating ip-address changes on the linked container.
-Now let us launch another container named `container5` linking container4 to c4.
+Now let us launch another container named `container5` linking `container4` to c4.
```bash
$ docker run --net=isolated_nw -itd --name=container5 --link container4:c4 busybox
72eccf2208336f31e9e33ba327734125af00d1e1d2657878e2ee8154fbb23c7a
```
-As expected, container4 will be able to reach container5 by both its container name and
-its alias c5 and container5 will be able to reach container4 by its container name and
+As expected, `container4` will be able to reach `container5` by both its container name and
+its alias c5 and `container5` will be able to reach `container4` by its container name and
its alias c4.
```bash
@@ -491,7 +498,7 @@ $ docker network create -d bridge --subnet 172.26.0.0/24 local_alias
76b7dc932e037589e6553f59f76008e5b76fa069638cd39776b890607f567aaa
```
-let us connect container4 and container5 to the new network `local_alias`
+let us connect `container4` and `container5` to the new network `local_alias`
```
$ docker network connect --link container5:foo local_alias container4
@@ -525,7 +532,7 @@ round-trip min/avg/max = 0.070/0.081/0.097 ms
```
Note that the ping succeeds for both the aliases but on different networks.
-Let us conclude this section by disconnecting container5 from the `isolated_nw`
+Let us conclude this section by disconnecting `container5` from the `isolated_nw`
and observe the results
```
@@ -550,9 +557,9 @@ round-trip min/avg/max = 0.070/0.081/0.097 ms
```
In conclusion, the new link functionality in user defined networks provides all the
-benefits of legacy links while avoiding most of the well-known issues with `legacy links`.
+benefits of legacy links while avoiding most of the well-known issues with *legacy links*.
-One notable missing functionality compared to `legacy links` is the injection of
+One notable missing functionality compared to *legacy links* is the injection of
environment variables. Though very useful, environment variable injection is static
in nature and must be injected when the container is started. One cannot inject
environment variables into a running container without significant effort and hence
@@ -561,10 +568,10 @@ disconnect containers to/from a network.
### Network-scoped alias
-While `links` provide private name resolution that is localized within a container,
+While *link*s provide private name resolution that is localized within a container,
the network-scoped alias provides a way for a container to be discovered by an
alternate name by any other container within the scope of a particular network.
-Unlike the `link` alias, which is defined by the consumer of a service, the
+Unlike the *link* alias, which is defined by the consumer of a service, the
network-scoped alias is defined by the container that is offering the service
to the network.
diff --git a/components/engine/docs/userguide/storagedriver/aufs-driver.md b/components/engine/docs/userguide/storagedriver/aufs-driver.md
index 76271cf0e1..4651e66e0c 100644
--- a/components/engine/docs/userguide/storagedriver/aufs-driver.md
+++ b/components/engine/docs/userguide/storagedriver/aufs-driver.md
@@ -148,7 +148,7 @@ layer IDs). Inside each file are the names of the directories that exist below
it in the stack
The command below shows the contents of a metadata file in
-`/var/lib/docker/aufs/layers/` that lists the the three directories that are
+`/var/lib/docker/aufs/layers/` that lists the three directories that are
stacked below it in the union mount. Remember, these directory names do no map
to image layer IDs with Docker 1.10 and higher.
diff --git a/components/engine/docs/userguide/storagedriver/imagesandcontainers.md b/components/engine/docs/userguide/storagedriver/imagesandcontainers.md
index e4293370f4..3f2594d3f6 100644
--- a/components/engine/docs/userguide/storagedriver/imagesandcontainers.md
+++ b/components/engine/docs/userguide/storagedriver/imagesandcontainers.md
@@ -110,7 +110,7 @@ single 8GB general purpose SSD EBS volume. The Docker data directory
centos latest c8a648134623 4 weeks ago 196.6 MB
ubuntu 15.04 c8be1ac8145a 7 weeks ago 131.3 MB
- $ du -hs /var/lib/docker
+ $ sudo du -hs /var/lib/docker
2.0G /var/lib/docker
$ time docker run --rm -v /var/lib/docker:/var/lib/docker docker/v1.10-migrator
@@ -275,12 +275,12 @@ image that you just pulled, make a change to it, and build a new image based on
command.
1. In an empty directory, create a simple `Dockerfile` that starts with the
-2. ubuntu:15.04 image.
+ ubuntu:15.04 image.
FROM ubuntu:15.04
2. Add a new file called "newfile" in the image's `/tmp` directory with the
-3. text "Hello world" in it.
+ text "Hello world" in it.
When you are done, the `Dockerfile` contains two lines:
@@ -291,7 +291,7 @@ command.
3. Save and close the file.
4. From a terminal in the same folder as your `Dockerfile`, run the following
-5. command:
+ command:
$ docker build -t changed-ubuntu .
Sending build context to Docker daemon 2.048 kB
@@ -310,14 +310,14 @@ command.
The output above shows a new image with image ID `94e6b7d2c720`.
5. Run the `docker images` command to verify the new `changed-ubuntu` image is
-6. in the Docker host's local storage area.
+ in the Docker host's local storage area.
REPOSITORY TAG IMAGE ID CREATED SIZE
changed-ubuntu latest 03b964f68d06 33 seconds ago 131.4 MB
ubuntu 15.04 013f3d01d247 6 weeks ago 131.3 MB
6. Run the `docker history` command to see which image layers were used to
-7. create the new `changed-ubuntu` image.
+ create the new `changed-ubuntu` image.
$ docker history changed-ubuntu
IMAGE CREATED CREATED BY SIZE COMMENT
diff --git a/components/engine/docs/userguide/storagedriver/index.md b/components/engine/docs/userguide/storagedriver/index.md
index 76671c7196..60d1255d77 100644
--- a/components/engine/docs/userguide/storagedriver/index.md
+++ b/components/engine/docs/userguide/storagedriver/index.md
@@ -25,7 +25,7 @@ Docker relies on driver technology to manage the storage and interactions associ
If you are new to Docker containers make sure you read ["Understand images, containers, and storage drivers"](imagesandcontainers.md) first. It explains key concepts and technologies that can help you when working with storage drivers.
-### Acknowledgement
+### Acknowledgment
The Docker storage driver material was created in large part by our guest author
Nigel Poulton with a bit of help from Docker's own Jérôme Petazzoni. In his
diff --git a/components/engine/docs/userguide/storagedriver/overlayfs-driver.md b/components/engine/docs/userguide/storagedriver/overlayfs-driver.md
index 9abc1dbe65..3f948f6bec 100644
--- a/components/engine/docs/userguide/storagedriver/overlayfs-driver.md
+++ b/components/engine/docs/userguide/storagedriver/overlayfs-driver.md
@@ -111,7 +111,7 @@ directories.
drwxr-xr-x 4 root root 4096 Oct 28 11:06 upper
drwx------ 3 root root 4096 Oct 28 11:06 work
-These four filesystem objects are all artefacts of OverlayFS. The "lower-id"
+These four filesystem objects are all artifacts of OverlayFS. The "lower-id"
file contains the ID of the top layer of the image the container is based on.
This is used by OverlayFS as the "lowerdir".
diff --git a/components/engine/docs/userguide/storagedriver/zfs-driver.md b/components/engine/docs/userguide/storagedriver/zfs-driver.md
index 3ecbb300d0..e55e7396f1 100644
--- a/components/engine/docs/userguide/storagedriver/zfs-driver.md
+++ b/components/engine/docs/userguide/storagedriver/zfs-driver.md
@@ -133,11 +133,11 @@ you should substitute your own values throughout the procedure.
1. If it is running, stop the Docker `daemon`.
-1. Install `the software-properties-common` package.
+1. Install the `software-properties-common` package.
This is required for the `add-apt-repository` command.
- $ sudo apt-get install software-properties-common
+ $ sudo apt-get install -y software-properties-common
Reading package lists... Done
Building dependency tree