This fix adds `t.Name()` to tests in integration/network
so that services created in those tests have unique names.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Upstream-commit: 76195f8047eb9a6be710d53ec493ab97bcf86e20
Component: engine
Begin to copy the data until the command to exit and any coping to
stdin or copy from stdout/stderr has completed.
Also adding defense code to trim the possible '\x00' null value.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Upstream-commit: 386e0f36c42593ef434517448ccfac5262f958d6
Component: engine
`${}`, `${:}` and so on are invalid because there's
no parameter within the brackets; fix detection for
this situation and add/update tests.
There were some existing test-cases that were testing
for the wrong behavior, which are now updated.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 334bf3ea76004d0abe02dd1698989f9eaf87a86a
Component: engine
The builder did not detect syntax errors in substitusions in the
Dockerfile, causing those values to be processed incorrectly instead of
producing an error.
Example 1: missing `}`
docker build --no-cache -<<'EOF'
FROM busybox
ARG var=${aaa:-bbb
RUN echo $var
EOF
Before:
Step 3/3 : RUN echo $var
---> Running in f06571e77146
bbb
After:
Step 2/3 : ARG var=${aaa:-bbb
failed to process "${aaa:-bbb": syntax error: missing '}'
Example 2: missing closing `}`, no default value
docker build --no-cache -<<'EOF'
FROM busybox
ARG var=${aaa
RUN echo $var
EOF
Before:
Step 2/3 : ARG var=${aaa
failed to process "${aaa": missing ':' in substitution
After:
Step 2/3 : ARG var=${aaa
failed to process "${aaa": syntax error: missing '}'
Example 3: double opening bracket (`{`)
docker build --no-cache -<<'EOF'
FROM busybox
ARG var=${{aaa:-bbb}
RUN echo $var
EOF
Before:
Step 2/3 : ARG var=${{aaa:-bbb}
failed to process "${{aaa:-bbb}": missing ':' in substitution
After:
Step 2/3 : ARG var=${{aaa:-bbb}
failed to process "${{aaa:-bbb}": syntax error: bad substitution
Example 4: double opening bracket (`{`), no default value
docker build --no-cache -<<'EOF'
FROM busybox
ARG var=${{aaa}
RUN echo $var
EOF
Before:
Step 2/3 : ARG var=${{aaa}
failed to process "${{aaa}": missing ':' in substitution
After:
Step 2/3 : ARG var=${{aaa}
failed to process "${{aaa}": syntax error: bad substitution
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 955a6ad95f7891a45692d975793abf1eeb07cdd5
Component: engine
If "ps" fails, in many cases it prints a meaningful error message
which a user can benefit from. Let's use it.
While at it, let's use errdefs.System to classify the error,
as well as errors.Wrap.
Before:
> $ docker top $CT <any bad ps options>
> Error response from daemon: Error running ps: exit status 1
After:
> $ docker top $CT auxm
> Error response from daemon: ps: error: thread display conflicts with forest display
or
> $ docker top $CT saur
> Error response from daemon: ps: error: conflicting format options
or, if there's no meaningful error on stderr, same as before:
> $ docker top $CT 1234
> Error response from daemon: ps: exit status 1
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: a41328d5704b8d1adbcd099fb4bb0697060df806
Component: engine
Current ContainerTop (a.k.a. docker top) implementation uses "ps"
to get the info about *all* running processes, then parses it, then
filters the results to only contain PIDs used by the container.
Collecting data only to throw most of it away is inefficient,
especially on a system running many containers (or processes).
For example, "docker top" on a container with a single process
can take up to 0.5 seconds to execute (on a mostly idle system)
which is noticeably slow.
Since the containers PIDs are known beforehand, let's use ps's
"-q" option to provide it with a list of PIDs we want info about.
The problem with this approach is, some ps options can't be used
with "-q" (the only one I'm aware of is "f" ("forest view") but
there might be more). As the list of such options is not known,
in case ps fails, it is executed again without "q" (retaining
the old behavior).
Next, the data produced by "ps" is filtered in the same way as before.
The difference here is, in case "-q" worked, the list is much shorter.
I ran some benchmarks on my laptop, with about 8000 "sleep" processes
running to amplify the savings.
The improvement in "docker top" execution times is 5x to 10x (roughly
0.05s vs 0.5s).
The improvement in ContainerTop() execution time is up to 100x
(roughly 3ms vs 300ms).
I haven't measured the memory or the CPU time savings, guess those
are not that critical.
NOTE that busybox ps does not implement -q so the fallback is always
used, but AFAIK it is not usable anyway and Docker expects a normal
ps to be on the system (say the list of fields produced by
"busybox ps -ef" differs from normal "ps -ef" etc.).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: a076badb8b33f1ecdc5d46f0a3701f10c0579f73
Component: engine
This fix was not yet included in Docker 17.05, so
API version v1.37 was not the right selector (Docker
18.03, 18.04 and 18.05 all support API v1.37).
We should change these checks for engine versions,
or use a different method to skip tests when running
against older engines.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: a061b1e2d8f6117b0524e44de7b6bc391245864e
Component: engine
remove unnescessary import aliases, brackets, and so on.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f23c00d8701e4bd0f2372a586dacbf66a26f9a51
Component: engine
Fixes an issue where `CopyDir` would truncate files larger than 2^32
bytes.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Upstream-commit: 616fa9446e49502da501a7d0f92b13191f5058dd
Component: engine
Sometimes docker-master CI fails on rhel4+selinux configuration,
like this:
--- FAIL: TestMount (0.12s)
--- FAIL: TestMount/none-remount,size=128k (0.01s)
mounter_linux_test.go:209: unexpected mount option "seclabel" expected "rw,size=128k"
--- FAIL: TestMount/none-remount,ro,size=128k (0.01s)
mounter_linux_test.go:209: unexpected mount option "seclabel" expected "ro,size=128k"
Earlier, commit 8bebd42df2 (PR #34965) fixed this failure,
but not entirely (i.e. the test is now flaky). It looks like
either selinux detection code is not always working (it won't
work in d-in-d), or the kernel might or might not add 'seclabel'
option).
As the subject of this test case is definitely not selinux,
it can just ignore the option added by it.
While at it, fix error messages:
- add missing commas;
- fix a typo;
- allow for clear distinction between mount
and vfs (per-superblock) options.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: d78e885326213e8ef89919c3cc6d16e712e852a8
Component: engine
The following failure is seen in CI from time to time:
> FAIL: docker_api_containers_test.go:435: DockerSuite.TestContainerAPITop
>
> docker_api_containers_test.go:453:
> c.Assert(top.Processes[0][10], checker.Equals, "/bin/sh -c top")
> ... obtained string = "top"
> ... expected string = "/bin/sh -c top"
The test case expects two processes in the output:
1. /bin/sh -c top
2. top
in the given order.
Now, "ps aux" output is sorted by PID*, and so since the "top" is a child
of "/bin/sh -c top" it has a higher PID and will come second as expected
by the test... unless the PIDs on the system are exhausted and PID rollover
happens, in which case PID of "top" will be lower than that of "/bin/sh".
Fix: sort output by process name.
* - in fact it is not sorted, but is being printed in the same order as
the kernel list PID entries in /proc directory, which appears to be
sorted by PID (see ls -1 -U /proc).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 0823ab70990fa4cbf520726013e75c264e6c5231
Component: engine
It does not make sense to keep looking for PID once
we found it, so let's give it a break.
The side effect of this patch is, if there's more than one column
titled "PID", the last (rightmost) column was used before, and now
the first (leftmost) column is used. Should make no practical
difference whatsoever.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 654a7625fc9f0b7b04da0e0e4d151af04a65cc7f
Component: engine
Adds functionality to parse and return network attachment spec
information. Network attachment tasks are phony tasks created in
swarmkit to deal with unmanaged containers attached to swarmkit. Before
this change, attempting `docker inspect` on the task id of a network
attachment task would result in an empty task object. After this change,
a full task object is returned
Fixes#26548 the correct way.
Signed-off-by: Drew Erny <drew.erny@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 5b69ff466e61fa168f24869710f2070c742a5565
Component: engine
This prevents the following test case failures "go test" is run
as non-root in the daemon/ directory:
> --- FAIL: TestContainerInitDNS (0.02s)
> daemon_test.go:209: chown /tmp/docker-container-test-054812199/volumes: operation not permitted
>
> --- FAIL: TestDaemonReloadNetworkDiagnosticPort (0.00s)
> reload_test.go:525: mkdir /var/lib/docker/network/files/: permission denied
> --- FAIL: TestRootMountCleanup (0.00s)
> daemon_linux_test.go:240: assertion failed: error is not nil: operation not permitted
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Upstream-commit: 16670ed4842b1ee4853ba39b6ebf2b771d28db9a
Component: engine
Arrays expect a type to be set for items in the array.
This patch adds the "string" type, adds a short description,
and some example values.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 8b6b55986644c093564497bfef51733a8d598baf
Component: engine
