Merge pull request #473 from seemethere/vbump

[18.03] bump version to 18.03.0-ce

CHANGELOG.md

@@ -1,71 +1,120 @@
-# Changelog
+# Changelog 
+For more information on the list of deprecated flags and APIs please have a look at
+https://docs.docker.com/engine/deprecated/ where you can find the target removal dates 
 
-For more information on the list of deprecated flags and APIs, have a look at
-https://docs.docker.com/engine/deprecated/ where you can find target removal dates.
-
-## 18.02.0-ce (2018-02-07)
+## 18.03.0-ce (2018-03-21)
 
 ### Builder
 
-- Gitutils: fix checking out submodules [moby/moby#35737](https://github.com/moby/moby/pull/35737)
+* Switch to -buildmode=pie [moby/moby#34369](https://github.com/moby/moby/pull/34369)
+* Allow Dockerfile to be outside of build-context [docker/cli#886](https://github.com/docker/cli/pull/886) 
+* Builder: fix wrong cache hits building from tars [moby/moby#36329](https://github.com/moby/moby/pull/36329)
+- Fixes files leaking to other images in a multi-stage build [moby/moby#36338](https://github.com/moby/moby/pull/36338)
 
 ### Client
 
-* Attach: Ensure attach exit code matches container's [docker/cli#696](https://github.com/docker/cli/pull/696)
-+ Added support for tmpfs-mode in compose file [docker/cli#808](https://github.com/docker/cli/pull/808)
-+ Adds a new compose file version 3.6 [docker/cli#808](https://github.com/docker/cli/pull/808)
-- Fix issue of filter in `docker ps` where `health=starting` returns nothing [moby/moby#35940](https://github.com/moby/moby/pull/35940)
-+ Improve presentation of published port ranges [docker/cli#581](https://github.com/docker/cli/pull/581)
-* Bump Go to 1.9.3 [docker/cli#827](https://github.com/docker/cli/pull/827)
+* Simplify the marshaling of compose types.Config [docker/cli#895](https://github.com/docker/cli/pull/895)
++ Add support for multiple composefile when deploying [docker/cli#569](https://github.com/docker/cli/pull/569)
 - Fix broken Kubernetes stack flags [docker/cli#831](https://github.com/docker/cli/pull/831)
-* Annotate "stack" commands to be "swarm" and "kubernetes" [docker/cli#804](https://github.com/docker/cli/pull/804)
-
-### Experimental
-
-+ Add manifest command [docker/cli#138](https://github.com/docker/cli/pull/138)
-* LCOW remotefs - return error in Read() implementation [moby/moby#36051](https://github.com/moby/moby/pull/36051)
-+ LCOW: Coalesce daemon stores, allow dual LCOW and WCOW mode [moby/moby#34859](https://github.com/moby/moby/pull/34859)
-- LCOW: Fix OpenFile parameters [moby/moby#36043](https://github.com/moby/moby/pull/36043)
-* LCOW: Raise minimum requirement to Windows RS3 RTM build (16299) [moby/moby#36065](https://github.com/moby/moby/pull/36065)
+- Fix stack marshaling for Kubernetes [docker/cli#890](https://github.com/docker/cli/pull/890)
+- Fix and simplify bash completion for service env, mounts and labels [docker/cli#682](https://github.com/docker/cli/pull/682)
+- Fix `before` and `since` filter for `docker ps` [moby/moby#35938](https://github.com/moby/moby/pull/35938)
+- Fix `--label-file` weird behavior [docker/cli#838](https://github.com/docker/cli/pull/838)
+- Fix compilation of defaultCredentialStore() on unsupported platforms [docker/cli#872](https://github.com/docker/cli/pull/872)
+* Improve and fix bash completion for images [docker/cli#717](https://github.com/docker/cli/pull/717)
++ Added check for empty source in bind mount [docker/cli#824](https://github.com/docker/cli/pull/824)
+- Fix TLS from environment variables in client [moby/moby#36270](https://github.com/moby/moby/pull/36270)
+* docker build now runs faster when registry-specific credential helper(s) are configured [docker/cli#840](https://github.com/docker/cli/pull/840)
+* Update event filter zsh completion with `disable`, `enable`, `install` and `remove` [docker/cli#372](https://github.com/docker/cli/pull/372)
+* Produce errors when empty ids are passed into inspect calls [moby/moby#36144](https://github.com/moby/moby/pull/36144)
+* Marshall version for the k8s controller [docker/cli#891](https://github.com/docker/cli/pull/891)
+* Set a non-zero timeout for HTTP client communication with plugin backend [docker/cli#883](https://github.com/docker/cli/pull/883)
++ Add DOCKER_TLS environment variable for --tls option [docker/cli#863](https://github.com/docker/cli/pull/863)
++ Add --template-driver option for secrets/configs [docker/cli#896](https://github.com/docker/cli/pull/896)
++ Move `docker trust` commands out of experimental [docker/cli#934](https://github.com/docker/cli/pull/934) [docker/cli#935](https://github.com/docker/cli/pull/935) [docker/cli#944](https://github.com/docker/cli/pull/944)
 
 ### Logging
 
-* Improve daemon config reload; log active configuration [moby/moby#36019](https://github.com/moby/moby/pull/36019)
-- Fixed error detection using IsErrNotFound and IsErrNotImplemented for the ContainerLogs method [moby/moby#36000](https://github.com/moby/moby/pull/36000)
-+ Add journald tag as SYSLOG_IDENTIFIER [moby/moby#35570](https://github.com/moby/moby/pull/35570)
-* Splunk: limit the reader size on error responses [moby/moby#35509](https://github.com/moby/moby/pull/35509)
-
+* AWS logs - don't add new lines to maximum sized events [moby/moby#36078](https://github.com/moby/moby/pull/36078)
+* Move log validator logic after plugins are loaded [moby/moby#36306](https://github.com/moby/moby/pull/36306)
+* Support a proxy in Splunk log driver [moby/moby#36220](https://github.com/moby/moby/pull/36220)
+- Fix log tail with empty logs [moby/moby#36305](https://github.com/moby/moby/pull/36305)
 
 ### Networking
 
-* Disable service on release network results in zero-downtime deployments with rolling upgrades [moby/moby#35960](https://github.com/moby/moby/pull/35960)
-- Fix services failing to start if multiple networks with the same name exist in different spaces [moby/moby#30897](https://github.com/moby/moby/pull/30897)
-- Fix duplicate networks being added with `docker service update --network-add` [docker/cli#780](https://github.com/docker/cli/pull/780)
-- Fixing ingress network when upgrading from 17.09 to 17.12. [moby/moby#36003](https://github.com/moby/moby/pull/36003)
-- Fix ndots configuration [docker/libnetwork#1995](https://github.com/docker/libnetwork/pull/1995)
-- Fix IPV6 networking being deconfigured if live-restore is enabled [docker/libnetwork#2043](https://github.com/docker/libnetwork/pull/2043)
-+ Add support for MX type DNS queries in the embedded DNS server [docker/libnetwork#2041](https://github.com/docker/libnetwork/pull/2041)
-
-### Packaging
-
-+ Added packaging for Fedora 26, Fedora 27, and Centos 7 on aarch64 [docker/docker-ce-packaging#71](https://github.com/docker/docker-ce-packaging/pull/71)
-- Removed support for Ubuntu Zesty [docker/docker-ce-packaging#73](https://github.com/docker/docker-ce-packaging/pull/73)
-- Removed support for Fedora 25 [docker/docker-ce-packaging#72](https://github.com/docker/docker-ce-packaging/pull/72)
-
+* Libnetwork revendoring [moby/moby#36137](https://github.com/moby/moby/pull/36137)
+- Fix for deadlock on exit with Memberlist revendor [docker/libnetwork#2040](https://github.com/docker/libnetwork/pull/2040)
+* Fix user specified ndots option [docker/libnetwork#2065](https://github.com/docker/libnetwork/pull/2065) 
+- Fix to use ContainerID for Windows instead of SandboxID [docker/libnetwork#2010](https://github.com/docker/libnetwork/pull/2010)
+* Verify NetworkingConfig to make sure EndpointSettings is not nil [moby/moby#36077](https://github.com/moby/moby/pull/36077)
+- Fix `DockerNetworkInternalMode` issue [moby/moby#36298](https://github.com/moby/moby/pull/36298)
+- Fix race in attachable network attachment [moby/moby#36191](https://github.com/moby/moby/pull/36191)
+- Fix timeout issue of `InspectNetwork` on AArch64 [moby/moby#36257](https://github.com/moby/moby/pull/36257)
+* Verbose info is missing for partial overlay ID [moby/moby#35989](https://github.com/moby/moby/pull/35989)
+* Update `FindNetwork` to address network name duplications [moby/moby#30897](https://github.com/moby/moby/pull/30897)
+* Disallow attaching ingress network [docker/swarmkit#2523](https://github.com/docker/swarmkit/pull/2523)
+- Prevent implicit removal of the ingress network [moby/moby#36538](https://github.com/moby/moby/pull/36538)
+- Fix stale HNS endpoints on Windows [moby/moby#36603](https://github.com/moby/moby/pull/36603)
+- IPAM fixes for duplicate IP addresses [docker/libnetwork#2104](https://github.com/docker/libnetwork/pull/2104) [docker/libnetwork#2105](https://github.com/docker/libnetwork/pull/2105)
 
 ### Runtime
 
-- Fixes unexpected Docker Daemon shutdown based on pipe error [moby/moby#35968](https://github.com/moby/moby/pull/35968)
-- Fix some occurrences of hcsshim::ImportLayer failed in Win32: The system cannot find the path specified [moby/moby#35924](https://github.com/moby/moby/pull/35924)
-* Windows: increase the maximum layer size during build to 127GB [moby/moby#35925](https://github.com/moby/moby/pull/35925)
-- Fix Devicemapper: Error running DeleteDevice dm_task_run failed [moby/moby#35919](https://github.com/moby/moby/pull/35919)
-+ Introduce « exec_die » event [moby/moby#35744](https://github.com/moby/moby/pull/35744)
-* Update API to version 1.36 [moby/moby#35744](https://github.com/moby/moby/pull/35744)
-- Fix `docker update` not updating cpu quota, and cpu-period of a running container [moby/moby#36030](https://github.com/moby/moby/pull/36030)
-* Make container shm parent unbindable [moby/moby#35830](https://github.com/moby/moby/pull/35830)
-+ Make image (layer) downloads faster by using pigz [moby/moby#35697](https://github.com/moby/moby/pull/35697)
-+ Protect the daemon from volume plugins that are slow or deadlocked [moby/moby#35441](https://github.com/moby/moby/pull/35441)
-- Fix `DOCKER_RAMDISK` environment variable not being honoured [moby/moby#35957](https://github.com/moby/moby/pull/35957)
-* Bump containerd to 1.0.1 (9b55aab90508bd389d7654c4baf173a981477d55) [moby/moby#35986](https://github.com/moby/moby/pull/35986)
-* Update runc to fix hang during start and exec [moby/moby#36097](https://github.com/moby/moby/pull/36097) 
+* Enable HotAdd for Windows [moby/moby#35414](https://github.com/moby/moby/pull/35414)
+* LCOW: Graphdriver fix deadlock in hotRemoveVHDs [moby/moby#36114](https://github.com/moby/moby/pull/36114)
+* LCOW: Regular mount if only one layer [moby/moby#36052](https://github.com/moby/moby/pull/36052)
+* Remove interim env var LCOW_API_PLATFORM_IF_OMITTED [moby/moby#36269](https://github.com/moby/moby/pull/36269)
+* Revendor Microsoft/opengcs @ v0.3.6 [moby/moby#36108](https://github.com/moby/moby/pull/36108)
+- Fix issue of ExitCode and PID not show up in Task.Status.ContainerStatus [moby/moby#36150](https://github.com/moby/moby/pull/36150)
+- Fix issue with plugin scanner going too deep [moby/moby#36119](https://github.com/moby/moby/pull/36119)
+* Do not make graphdriver homes private mounts [moby/moby#36047](https://github.com/moby/moby/pull/36047)
+* Do not recursive unmount on cleanup of zfs/btrfs [moby/moby#36237](https://github.com/moby/moby/pull/36237)
+* Don't restore image if layer does not exist [moby/moby#36304](https://github.com/moby/moby/pull/36304)
+* Adjust minimum API version for templated configs/secrets [moby/moby#36366](https://github.com/moby/moby/pull/36366)
+* Bump containerd to 1.0.2 (cfd04396dc68220d1cecbe686a6cc3aa5ce3667c) [moby/moby#36308](https://github.com/moby/moby/pull/36308)
+* Bump Golang to 1.9.4 [moby/moby#36243](https://github.com/moby/moby/pull/36243)
+* Ensure daemon root is unmounted on shutdown [moby/moby#36107](https://github.com/moby/moby/pull/36107)
+* Update runc to 6c55f98695e902427906eed2c799e566e3d3dfb5 [moby/moby#36222](https://github.com/moby/moby/pull/36222)
+- Fix container cleanup on daemon restart [moby/moby#36249](https://github.com/moby/moby/pull/36249)
+* Support SCTP port mapping (bump up API to v1.37) [moby/moby#33922](https://github.com/moby/moby/pull/33922)
+* Support SCTP port mapping [docker/cli#278](https://github.com/docker/cli/pull/278)
+- Fix Volumes property definition in ContainerConfig [moby/moby#35946](https://github.com/moby/moby/pull/35946)
+* Bump moby and dependencies [docker/cli#829](https://github.com/docker/cli/pull/829)
+* C.RWLayer: check for nil before use [moby/moby#36242](https://github.com/moby/moby/pull/36242)
++ Add `REMOVE` and `ORPHANED` to TaskState [moby/moby#36146](https://github.com/moby/moby/pull/36146)
+- Fixed error detection using `IsErrNotFound` and `IsErrNotImplemented` for `ContainerStatPath`, `CopyFromContainer`, and `CopyToContainer` methods [moby/moby#35979](https://github.com/moby/moby/pull/35979)
++ Add an integration/internal/container helper package [moby/moby#36266](https://github.com/moby/moby/pull/36266)
++ Add canonical import path [moby/moby#36194](https://github.com/moby/moby/pull/36194)
++ Add/use container.Exec() to integration [moby/moby#36326](https://github.com/moby/moby/pull/36326)
 - Fix "--node-generic-resource" singular/plural [moby/moby#36125](https://github.com/moby/moby/pull/36125)
+* Daemon.cleanupContainer: nullify container RWLayer upon release [moby/moby#36160](https://github.com/moby/moby/pull/36160)
+* Daemon: passdown the `--oom-kill-disable` option to containerd [moby/moby#36201](https://github.com/moby/moby/pull/36201)
+* Display a warn message when there is binding ports and net mode is host [moby/moby#35510](https://github.com/moby/moby/pull/35510)
+* Refresh containerd remotes on containerd restarted [moby/moby#36173](https://github.com/moby/moby/pull/36173)
+* Set daemon root to use shared propagation [moby/moby#36096](https://github.com/moby/moby/pull/36096)
+* Optimizations for recursive unmount [moby/moby#34379](https://github.com/moby/moby/pull/34379)
+* Perform plugin mounts in the runtime [moby/moby#35829](https://github.com/moby/moby/pull/35829)
+* Graphdriver: Fix RefCounter memory leak [moby/moby#36256](https://github.com/moby/moby/pull/36256)
+* Use continuity fs package for volume copy [moby/moby#36290](https://github.com/moby/moby/pull/36290)
+* Use proc/exe for reexec [moby/moby#36124](https://github.com/moby/moby/pull/36124)
++ Add API support for templated secrets and configs [moby/moby#33702](https://github.com/moby/moby/pull/33702) and [moby/moby#36366](https://github.com/moby/moby/pull/36366)
+* Use rslave propagation for mounts from daemon root [moby/moby#36055](https://github.com/moby/moby/pull/36055)
++ Add /proc/keys to masked paths [moby/moby#36368](https://github.com/moby/moby/pull/36368)
+* Bump Runc to 1.0.0-rc5 [moby/moby#36449](https://github.com/moby/moby/pull/36449)
+- Fixes `runc exec` on big-endian architectures [moby/moby#36449](https://github.com/moby/moby/pull/36449)
+* Use chroot when mount namespaces aren't provided [moby/moby#36449](https://github.com/moby/moby/pull/36449)
+- Fix systemd slice expansion so that it could be consumed by cAdvisor [moby/moby#36449](https://github.com/moby/moby/pull/36449)
+- Fix devices mounted with wrong uid/gid [moby/moby#36449](https://github.com/moby/moby/pull/36449)
+- Fix read-only containers with IPC private mounts `/dev/shm` read-only [moby/moby#36526](https://github.com/moby/moby/pull/36526)
+
+### Swarm Mode
+
+* Replace EC Private Key with PKCS#8 PEMs [docker/swarmkit#2246](https://github.com/docker/swarmkit/pull/2246)
+* Fix IP overlap with empty EndpointSpec [docker/swarmkit #2505](https://github.com/docker/swarmkit/pull/2505)
+* Add support for Support SCTP port mapping [docker/swarmkit#2298](https://github.com/docker/swarmkit/pull/2298)
+* Do not reschedule tasks if only placement constraints change and are satisfied by the assigned node [docker/swarmkit#2496](https://github.com/docker/swarmkit/pull/2496)
+* Ensure task reaper stopChan is closed no more than once [docker/swarmkit #2491](https://github.com/docker/swarmkit/pull/2491)
+* Synchronization fixes [docker/swarmkit#2495](https://github.com/docker/swarmkit/pull/2495)
+* Add log message to indicate message send retry if streaming unimplemented [docker/swarmkit#2483](https://github.com/docker/swarmkit/pull/2483)
+* Debug logs for session, node events on dispatcher, heartbeats [docker/swarmkit#2486](https://github.com/docker/swarmkit/pull/2486)
++ Add swarm types to bash completion event type filter [docker/cli#888](https://github.com/docker/cli/pull/888)
+- Fix issue where network inspect does not show Created time for networks in swarm scope [moby/moby#36095](https://github.com/moby/moby/pull/36095)

VERSION

@@ -1 +1 @@
-18.02.0-ce
+18.03.0-ce

components/cli/.gitignore

@@ -1,4 +1,12 @@
+# if you want to ignore files created by your editor/tools,
+# please consider a global .gitignore https://help.github.com/articles/ignoring-files
+*.exe
+*.exe~
+*.orig
+.*.swp
 .DS_Store
+Thumbs.db
+.editorconfig
 /build/
 cli/winresources/rsrc_386.syso
 cli/winresources/rsrc_amd64.syso

components/cli/.mailmap

@@ -232,6 +232,7 @@ Kai Qiang Wu (Kennan) <wkq5325@gmail.com>
 Kai Qiang Wu (Kennan) <wkq5325@gmail.com> <wkqwu@cn.ibm.com>
 Kamil Domański <kamil@domanski.co>
 Kamjar Gerami <kami.gerami@gmail.com>
+Kat Samperi <kat.samperi@gmail.com> <kizzie@users.noreply.github.com>
 Ken Cochrane <kencochrane@gmail.com> <KenCochrane@gmail.com>
 Ken Herner <kherner@progress.com> <chosenken@gmail.com>
 Kenfe-Mickaël Laventure <mickael.laventure@gmail.com>
@@ -281,6 +282,7 @@ Martin Redmond <redmond.martin@gmail.com> <xgithub@redmond5.com>
 Mary Anthony <mary.anthony@docker.com> <mary@docker.com>
 Mary Anthony <mary.anthony@docker.com> <moxieandmore@gmail.com>
 Mary Anthony <mary.anthony@docker.com> moxiegirl <mary@docker.com>
+Mateusz Major <apkd@users.noreply.github.com>
 Matt Bentley <matt.bentley@docker.com> <mbentley@mbentley.net>
 Matt Schurenko <matt.schurenko@gmail.com>
 Matt Williams <mattyw@me.com>
@@ -397,6 +399,7 @@ Thatcher Peskens <thatcher@docker.com>
 Thatcher Peskens <thatcher@docker.com> <thatcher@dotcloud.com>
 Thatcher Peskens <thatcher@docker.com> <thatcher@gmx.net>
 Thomas Gazagnaire <thomas@gazagnaire.org> <thomas@gazagnaire.com>
+Thomas Krzero <thomas.kovatchitch@gmail.com>
 Thomas Léveil <thomasleveil@gmail.com>
 Thomas Léveil <thomasleveil@gmail.com> <thomasleveil@users.noreply.github.com>
 Tibor Vass <teabee89@gmail.com> <tibor@docker.com>

components/cli/AUTHORS

@@ -17,6 +17,7 @@ Aidan Feldman <aidan.feldman@gmail.com>
 Aidan Hobson Sayers <aidanhs@cantab.net>
 AJ Bowen <aj@gandi.net>
 Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
+Akim Demaille <akim.demaille@docker.com>
 Alan Thompson <cloojure@gmail.com>
 Albert Callarisa <shark234@gmail.com>
 Aleksa Sarai <asarai@suse.de>
@@ -107,6 +108,7 @@ Christophe Robin <crobin@nekoo.com>
 Christophe Vidal <kriss@krizalys.com>
 Christopher Biscardi <biscarch@sketcht.com>
 Christopher Jones <tophj@linux.vnet.ibm.com>
+Christy Perez <christy@linux.vnet.ibm.com>
 Chun Chen <ramichen@tencent.com>
 Clinton Kitson <clintonskitson@gmail.com>
 Coenraad Loubser <coenraad@wish.org.za>
@@ -178,6 +180,7 @@ Eric-Olivier Lamey <eo@lamey.me>
 Erica Windisch <erica@windisch.us>
 Erik Hollensbe <github@hollensbe.org>
 Erik St. Martin <alakriti@gmail.com>
+Ethan Haynes <ethanhaynes@alumni.harvard.edu>
 Eugene Yakubovich <eugene.yakubovich@coreos.com>
 Evan Allrich <evan@unguku.com>
 Evan Hazlett <ejhazlett@gmail.com>
@@ -234,6 +237,7 @@ Ilya Khlopotov <ilya.khlopotov@gmail.com>
 Ilya Sotkov <ilya@sotkov.com>
 Isabel Jimenez <contact.isabeljimenez@gmail.com>
 Ivan Grcic <igrcic@gmail.com>
+Ivan Markin <sw@nogoegst.net>
 Jacob Atzen <jacob@jacobatzen.dk>
 Jacob Tomlinson <jacob@tom.linson.uk>
 Jaivish Kothari <janonymous.codevulture@gmail.com>
@@ -314,6 +318,7 @@ Kai Qiang Wu (Kennan) <wkq5325@gmail.com>
 Kara Alexandra <kalexandra@us.ibm.com>
 Kareem Khazem <karkhaz@karkhaz.com>
 Karthik Nayak <Karthik.188@gmail.com>
+Kat Samperi <kat.samperi@gmail.com>
 Katie McLaughlin <katie@glasnt.com>
 Ke Xu <leonhartx.k@gmail.com>
 Kei Ohmura <ohmura.kei@gmail.com>
@@ -380,6 +385,7 @@ Mark Oates <fl0yd@me.com>
 Martin Mosegaard Amdisen <martin.amdisen@praqma.com>
 Mary Anthony <mary.anthony@docker.com>
 Mason Malone <mason.malone@gmail.com>
+Mateusz Major <apkd@users.noreply.github.com>
 Matt Gucci <matt9ucci@gmail.com>
 Matt Robenolt <matt@ydekproductions.com>
 Matthew Heon <mheon@redhat.com>
@@ -420,6 +426,7 @@ Moorthy RS <rsmoorthy@gmail.com>
 Morgan Bauer <mbauer@us.ibm.com>
 Moysés Borges <moysesb@gmail.com>
 Mrunal Patel <mrunalp@gmail.com>
+muicoder <muicoder@gmail.com>
 Muthukumar R <muthur@gmail.com>
 Máximo Cuadros <mcuadros@gmail.com>
 Nace Oroz <orkica@gmail.com>
@@ -520,6 +527,7 @@ Shukui Yang <yangshukui@huawei.com>
 Sian Lerk Lau <kiawin@gmail.com>
 Sidhartha Mani <sidharthamn@gmail.com>
 sidharthamani <sid@rancher.com>
+Silvin Lubecki <silvin.lubecki@docker.com>
 Simei He <hesimei@zju.edu.cn>
 Simon Ferquel <simon.ferquel@docker.com>
 Sindhu S <sindhus@live.in>
@@ -538,6 +546,7 @@ Steve Durrheimer <s.durrheimer@gmail.com>
 Steven Burgess <steven.a.burgess@hotmail.com>
 Subhajit Ghosh <isubuz.g@gmail.com>
 Sun Jianbo <wonderflow.sun@gmail.com>
+Sungwon Han <sungwon.han@navercorp.com>
 Sven Dowideit <SvenDowideit@home.org.au>
 Sylvain Baubeau <sbaubeau@redhat.com>
 Sébastien HOUZÉ <cto@verylastroom.com>
@@ -546,6 +555,7 @@ TAGOMORI Satoshi <tagomoris@gmail.com>
 Taylor Jones <monitorjbl@gmail.com>
 Thatcher Peskens <thatcher@docker.com>
 Thomas Gazagnaire <thomas@gazagnaire.org>
+Thomas Krzero <thomas.kovatchitch@gmail.com>
 Thomas Leonard <thomas.leonard@docker.com>
 Thomas Léveil <thomasleveil@gmail.com>
 Thomas Riccardi <riccardi@systran.fr>
@@ -594,6 +604,7 @@ Wang Long <long.wanglong@huawei.com>
 Wang Ping <present.wp@icloud.com>
 Wang Xing <hzwangxing@corp.netease.com>
 Wang Yuexiao <wang.yuexiao@zte.com.cn>
+Wataru Ishida <ishida.wataru@lab.ntt.co.jp>
 Wayne Song <wsong@docker.com>
 Wen Cheng Ma <wenchma@cn.ibm.com>
 Wenzhi Liang <wenzhi.liang@gmail.com>

components/cli/MAINTAINERS

@@ -1,6 +1,6 @@
 # Docker maintainers file
 #
-# This file describes who runs the docker/docker project and how.
+# This file describes who runs the docker/cli project and how.
 # This is a living document - if you see something out of date or missing, speak up!
 #
 # It is structured to be consumable by both humans and programs.
@@ -21,23 +21,12 @@
 	# a subsystem, they are responsible for doing so and holding the
 	# subsystem maintainers accountable. If ownership is unclear, they are the de facto owners.
 
-	# For each release (including minor releases), a "release captain" is assigned from the
-	# pool of core maintainers. Rotation is encouraged across all maintainers, to ensure
-	# the release process is clear and up-to-date.
-
 		people = [
 			"aaronlehmann",
 			"albers",
-			"aluzzardi",
-			"anusha",
 			"cpuguy83",
-			"crosbymichael",
 			"dnephin",
-			"ehazlett",
-			"johnstep",
 			"justincormack",
-			"mavenugo",
-			"mlaventure",
 			"stevvooe",
 			"tibor",
 			"tonistiigi",
@@ -67,7 +56,6 @@
 	# - close an issue or pull request when it's inappropriate or off-topic
 
 		people = [
-			"ehazlett",
 			"programmerq",
 			"thajeztah"
 		]
@@ -90,41 +78,16 @@
 	Email = "github@albersweb.de"
 	GitHub = "albers"
 
-	[people.aluzzardi]
-	Name = "Andrea Luzzardi"
-	Email = "al@docker.com"
-	GitHub = "aluzzardi"
-
-	[people.anusha]
-	Name = "Anusha Ragunathan"
-	Email = "anusha@docker.com"
-	GitHub = "anusha-ragunathan"
-
 	[people.cpuguy83]
 	Name = "Brian Goff"
 	Email = "cpuguy83@gmail.com"
 	GitHub = "cpuguy83"
 
-	[people.crosbymichael]
-	Name = "Michael Crosby"
-	Email = "crosbymichael@gmail.com"
-	GitHub = "crosbymichael"
-
 	[people.dnephin]
 	Name = "Daniel Nephin"
 	Email = "dnephin@gmail.com"
 	GitHub = "dnephin"
 
-	[people.ehazlett]
-	Name = "Evan Hazlett"
-	Email = "ejhazlett@gmail.com"
-	GitHub = "ehazlett"
-
-	[people.johnstep]
-	Name = "John Stephens"
-	Email = "johnstep@docker.com"
-	GitHub = "johnstep"
-
 	[people.justincormack]
 	Name = "Justin Cormack"
 	Email = "justin.cormack@docker.com"
@@ -135,15 +98,10 @@
 	Email = "misty@docker.com"
 	GitHub = "mistyhacks"
 
-	[people.mlaventure]
-	Name = "Kenfe-Mickaël Laventure"
-	Email = "mickael.laventure@docker.com"
-	GitHub = "mlaventure"
-
-	[people.shykes]
-	Name = "Solomon Hykes"
-	Email = "solomon@docker.com"
-	GitHub = "shykes"
+	[people.programmerq]
+	Name = "Jeff Anderson"
+	Email = "jeff@docker.com"
+	GitHub = "programmerq"
 
 	[people.stevvooe]
 	Name = "Stephen Day"

components/cli/Makefile

@@ -51,7 +51,8 @@ watch: ## monitor file changes and run go test
 	./scripts/test/watch
 
 vendor: vendor.conf ## check that vendor matches vendor.conf
-	vndr 2> /dev/null
+	rm -rf vendor
+	bash -c 'vndr |& grep -v -i clone'
 	scripts/validate/check-git-diff vendor
 
 .PHONY: authors

components/cli/VERSION

@@ -1 +1 @@
-18.02.0-ce
+18.03.0-ce

components/cli/cli/command/cli.go

@@ -300,12 +300,12 @@ func newHTTPClient(host string, tlsOptions *tlsconfig.Options) (*http.Client, er
 			Timeout:   30 * time.Second,
 		}).DialContext,
 	}
-	proto, addr, _, err := client.ParseHost(host)
+	hostURL, err := client.ParseHostURL(host)
 	if err != nil {
 		return nil, err
 	}
 
-	sockets.ConfigureTransport(tr, proto, addr)
+	sockets.ConfigureTransport(tr, hostURL.Scheme, hostURL.Host)
 
 	return &http.Client{
 		Transport:     tr,

components/cli/cli/command/config/create.go

@@ -16,9 +16,10 @@ import (
 )
 
 type createOptions struct {
-	name   string
-	file   string
-	labels opts.ListOpts
+	name           string
+	templateDriver string
+	file           string
+	labels         opts.ListOpts
 }
 
 func newConfigCreateCommand(dockerCli command.Cli) *cobra.Command {
@@ -38,6 +39,8 @@ func newConfigCreateCommand(dockerCli command.Cli) *cobra.Command {
 	}
 	flags := cmd.Flags()
 	flags.VarP(&createOpts.labels, "label", "l", "Config labels")
+	flags.StringVar(&createOpts.templateDriver, "template-driver", "", "Template driver")
+	flags.SetAnnotation("driver", "version", []string{"1.37"})
 
 	return cmd
 }
@@ -68,7 +71,11 @@ func runConfigCreate(dockerCli command.Cli, options createOptions) error {
 		},
 		Data: configData,
 	}
-
+	if options.templateDriver != "" {
+		spec.Templating = &swarm.Driver{
+			Name: options.templateDriver,
+		}
+	}
 	r, err := client.ConfigCreate(ctx, spec)
 	if err != nil {
 		return err

components/cli/cli/command/config/create_test.go

@@ -82,14 +82,21 @@ func TestConfigCreateWithLabels(t *testing.T) {
 	}
 	name := "foo"
 
+	data, err := ioutil.ReadFile(filepath.Join("testdata", configDataFile))
+	assert.NoError(t, err)
+
+	expected := swarm.ConfigSpec{
+		Annotations: swarm.Annotations{
+			Name:   name,
+			Labels: expectedLabels,
+		},
+		Data: data,
+	}
+
 	cli := test.NewFakeCli(&fakeClient{
 		configCreateFunc: func(spec swarm.ConfigSpec) (types.ConfigCreateResponse, error) {
-			if spec.Name != name {
-				return types.ConfigCreateResponse{}, errors.Errorf("expected name %q, got %q", name, spec.Name)
-			}
-
-			if !reflect.DeepEqual(spec.Labels, expectedLabels) {
-				return types.ConfigCreateResponse{}, errors.Errorf("expected labels %v, got %v", expectedLabels, spec.Labels)
+			if !reflect.DeepEqual(spec, expected) {
+				return types.ConfigCreateResponse{}, errors.Errorf("expected %+v, got %+v", expected, spec)
 			}
 
 			return types.ConfigCreateResponse{
@@ -105,3 +112,32 @@ func TestConfigCreateWithLabels(t *testing.T) {
 	assert.NoError(t, cmd.Execute())
 	assert.Equal(t, "ID-"+name, strings.TrimSpace(cli.OutBuffer().String()))
 }
+
+func TestConfigCreateWithTemplatingDriver(t *testing.T) {
+	expectedDriver := &swarm.Driver{
+		Name: "template-driver",
+	}
+	name := "foo"
+
+	cli := test.NewFakeCli(&fakeClient{
+		configCreateFunc: func(spec swarm.ConfigSpec) (types.ConfigCreateResponse, error) {
+			if spec.Name != name {
+				return types.ConfigCreateResponse{}, errors.Errorf("expected name %q, got %q", name, spec.Name)
+			}
+
+			if spec.Templating.Name != expectedDriver.Name {
+				return types.ConfigCreateResponse{}, errors.Errorf("expected driver %v, got %v", expectedDriver, spec.Labels)
+			}
+
+			return types.ConfigCreateResponse{
+				ID: "ID-" + spec.Name,
+			}, nil
+		},
+	})
+
+	cmd := newConfigCreateCommand(cli)
+	cmd.SetArgs([]string{name, filepath.Join("testdata", configDataFile)})
+	cmd.Flags().Set("template-driver", expectedDriver.Name)
+	assert.NoError(t, cmd.Execute())
+	assert.Equal(t, "ID-"+name, strings.TrimSpace(cli.OutBuffer().String()))
+}

components/cli/cli/command/container/attach.go

@@ -1,12 +1,14 @@
 package container
 
 import (
+	"fmt"
 	"io"
 	"net/http/httputil"
 
 	"github.com/docker/cli/cli"
 	"github.com/docker/cli/cli/command"
 	"github.com/docker/docker/api/types"
+	"github.com/docker/docker/api/types/container"
 	"github.com/docker/docker/client"
 	"github.com/docker/docker/pkg/signal"
 	"github.com/pkg/errors"
@@ -66,6 +68,9 @@ func runAttach(dockerCli command.Cli, opts *attachOptions) error {
 	ctx := context.Background()
 	client := dockerCli.Client()
 
+	// request channel to wait for client
+	resultC, errC := client.ContainerWait(ctx, opts.container, "")
+
 	c, err := inspectContainerAndCheckState(ctx, client, opts.container)
 	if err != nil {
 		return err
@@ -140,7 +145,24 @@ func runAttach(dockerCli command.Cli, opts *attachOptions) error {
 	if errAttach != nil {
 		return errAttach
 	}
-	return getExitStatus(ctx, dockerCli.Client(), opts.container)
+
+	return getExitStatus(errC, resultC)
+}
+
+func getExitStatus(errC <-chan error, resultC <-chan container.ContainerWaitOKBody) error {
+	select {
+	case result := <-resultC:
+		if result.Error != nil {
+			return fmt.Errorf(result.Error.Message)
+		}
+		if result.StatusCode != 0 {
+			return cli.StatusError{StatusCode: int(result.StatusCode)}
+		}
+	case err := <-errC:
+		return err
+	}
+
+	return nil
 }
 
 func resizeTTY(ctx context.Context, dockerCli command.Cli, containerID string) {
@@ -157,19 +179,3 @@ func resizeTTY(ctx context.Context, dockerCli command.Cli, containerID string) {
 		logrus.Debugf("Error monitoring TTY size: %s", err)
 	}
 }
-
-func getExitStatus(ctx context.Context, apiclient client.ContainerAPIClient, containerID string) error {
-	container, err := apiclient.ContainerInspect(ctx, containerID)
-	if err != nil {
-		// If we can't connect, then the daemon probably died.
-		if !client.IsErrConnectionFailed(err) {
-			return err
-		}
-		return cli.StatusError{StatusCode: -1}
-	}
-	status := container.State.ExitCode
-	if status != 0 {
-		return cli.StatusError{StatusCode: status}
-	}
-	return nil
-}

components/cli/cli/command/container/attach_test.go

@@ -1,6 +1,7 @@
 package container
 
 import (
+	"fmt"
 	"io/ioutil"
 	"testing"
 
@@ -8,9 +9,9 @@ import (
 	"github.com/docker/cli/internal/test"
 	"github.com/docker/cli/internal/test/testutil"
 	"github.com/docker/docker/api/types"
+	"github.com/docker/docker/api/types/container"
 	"github.com/pkg/errors"
 	"github.com/stretchr/testify/assert"
-	"golang.org/x/net/context"
 )
 
 func TestNewAttachCommandErrors(t *testing.T) {
@@ -78,40 +79,50 @@ func TestNewAttachCommandErrors(t *testing.T) {
 }
 
 func TestGetExitStatus(t *testing.T) {
-	containerID := "the exec id"
-	expecatedErr := errors.New("unexpected error")
+	var (
+		expectedErr = fmt.Errorf("unexpected error")
+		errC        = make(chan error, 1)
+		resultC     = make(chan container.ContainerWaitOKBody, 1)
+	)
 
 	testcases := []struct {
-		inspectError  error
-		exitCode      int
+		result        *container.ContainerWaitOKBody
+		err           error
 		expectedError error
 	}{
 		{
-			inspectError: nil,
-			exitCode:     0,
+			result: &container.ContainerWaitOKBody{
+				StatusCode: 0,
+			},
 		},
 		{
-			inspectError:  expecatedErr,
-			expectedError: expecatedErr,
+			err:           expectedErr,
+			expectedError: expectedErr,
 		},
 		{
-			exitCode:      15,
+			result: &container.ContainerWaitOKBody{
+				Error: &container.ContainerWaitOKBodyError{
+					expectedErr.Error(),
+				},
+			},
+			expectedError: expectedErr,
+		},
+		{
+			result: &container.ContainerWaitOKBody{
+				StatusCode: 15,
+			},
 			expectedError: cli.StatusError{StatusCode: 15},
 		},
 	}
 
 	for _, testcase := range testcases {
-		client := &fakeClient{
-			inspectFunc: func(id string) (types.ContainerJSON, error) {
-				assert.Equal(t, containerID, id)
-				return types.ContainerJSON{
-					ContainerJSONBase: &types.ContainerJSONBase{
-						State: &types.ContainerState{ExitCode: testcase.exitCode},
-					},
-				}, testcase.inspectError
-			},
+		if testcase.err != nil {
+			errC <- testcase.err
 		}
-		err := getExitStatus(context.Background(), client, containerID)
+		if testcase.result != nil {
+			resultC <- *testcase.result
+		}
+		err := getExitStatus(errC, resultC)
 		assert.Equal(t, testcase.expectedError, err)
 	}
 }

components/cli/cli/command/container/client_test.go

@@ -16,15 +16,24 @@ type fakeClient struct {
 	execInspectFunc       func(execID string) (types.ContainerExecInspect, error)
 	execCreateFunc        func(container string, config types.ExecConfig) (types.IDResponse, error)
 	createContainerFunc   func(config *container.Config, hostConfig *container.HostConfig, networkingConfig *network.NetworkingConfig, containerName string) (container.ContainerCreateCreatedBody, error)
+	containerStartFunc    func(container string, options types.ContainerStartOptions) error
 	imageCreateFunc       func(parentReference string, options types.ImageCreateOptions) (io.ReadCloser, error)
 	infoFunc              func() (types.Info, error)
 	containerStatPathFunc func(container, path string) (types.ContainerPathStat, error)
 	containerCopyFromFunc func(container, srcPath string) (io.ReadCloser, types.ContainerPathStat, error)
 	logFunc               func(string, types.ContainerLogsOptions) (io.ReadCloser, error)
 	waitFunc              func(string) (<-chan container.ContainerWaitOKBody, <-chan error)
+	containerListFunc     func(types.ContainerListOptions) ([]types.Container, error)
 	Version               string
 }
 
+func (f *fakeClient) ContainerList(_ context.Context, options types.ContainerListOptions) ([]types.Container, error) {
+	if f.containerListFunc != nil {
+		return f.containerListFunc(options)
+	}
+	return []types.Container{}, nil
+}
+
 func (f *fakeClient) ContainerInspect(_ context.Context, containerID string) (types.ContainerJSON, error) {
 	if f.inspectFunc != nil {
 		return f.inspectFunc(containerID)
@@ -108,3 +117,10 @@ func (f *fakeClient) ContainerWait(_ context.Context, container string, _ contai
 	}
 	return nil, nil
 }
+
+func (f *fakeClient) ContainerStart(_ context.Context, container string, options types.ContainerStartOptions) error {
+	if f.containerStartFunc != nil {
+		return f.containerStartFunc(container, options)
+	}
+	return nil
+}

components/cli/cli/command/container/cp_test.go

@@ -3,6 +3,7 @@ package container
 import (
 	"io"
 	"io/ioutil"
+	"os"
 	"runtime"
 	"strings"
 	"testing"
@@ -111,6 +112,33 @@ func TestRunCopyFromContainerToFilesystemMissingDestinationDirectory(t *testing.
 	testutil.ErrorContains(t, err, destDir.Join("missing"))
 }
 
+func TestRunCopyToContainerFromFileWithTrailingSlash(t *testing.T) {
+	srcFile := fs.NewFile(t, t.Name())
+	defer srcFile.Remove()
+
+	options := copyOptions{
+		source:      srcFile.Path() + string(os.PathSeparator),
+		destination: "container:/path",
+	}
+	cli := test.NewFakeCli(&fakeClient{})
+	err := runCopy(cli, options)
+	testutil.ErrorContains(t, err, "not a directory")
+}
+
+func TestRunCopyToContainerSourceDoesNotExist(t *testing.T) {
+	options := copyOptions{
+		source:      "/does/not/exist",
+		destination: "container:/path",
+	}
+	cli := test.NewFakeCli(&fakeClient{})
+	err := runCopy(cli, options)
+	expected := "no such file or directory"
+	if runtime.GOOS == "windows" {
+		expected = "cannot find the file specified"
+	}
+	testutil.ErrorContains(t, err, expected)
+}
+
 func TestSplitCpArg(t *testing.T) {
 	var testcases = []struct {
 		doc               string

components/cli/cli/command/container/list_test.go

@@ -0,0 +1,165 @@
+package container
+
+import (
+	"fmt"
+	"io/ioutil"
+	"testing"
+
+	"github.com/docker/cli/cli/config/configfile"
+	"github.com/docker/cli/internal/test"
+	"github.com/docker/cli/internal/test/testutil"
+	"github.com/docker/docker/api/types"
+	"github.com/stretchr/testify/assert"
+	// Import builders to get the builder function as package function
+	. "github.com/docker/cli/internal/test/builders"
+	"github.com/gotestyourself/gotestyourself/golden"
+)
+
+func TestContainerListErrors(t *testing.T) {
+	testCases := []struct {
+		args              []string
+		flags             map[string]string
+		containerListFunc func(types.ContainerListOptions) ([]types.Container, error)
+		expectedError     string
+	}{
+		{
+			flags: map[string]string{
+				"format": "{{invalid}}",
+			},
+			expectedError: `function "invalid" not defined`,
+		},
+		{
+			flags: map[string]string{
+				"format": "{{join}}",
+			},
+			expectedError: `wrong number of args for join`,
+		},
+		{
+			containerListFunc: func(_ types.ContainerListOptions) ([]types.Container, error) {
+				return nil, fmt.Errorf("error listing containers")
+			},
+			expectedError: "error listing containers",
+		},
+	}
+	for _, tc := range testCases {
+		cmd := newListCommand(
+			test.NewFakeCli(&fakeClient{
+				containerListFunc: tc.containerListFunc,
+			}),
+		)
+		cmd.SetArgs(tc.args)
+		for key, value := range tc.flags {
+			cmd.Flags().Set(key, value)
+		}
+		cmd.SetOutput(ioutil.Discard)
+		testutil.ErrorContains(t, cmd.Execute(), tc.expectedError)
+	}
+}
+
+func TestContainerListWithoutFormat(t *testing.T) {
+	cli := test.NewFakeCli(&fakeClient{
+		containerListFunc: func(_ types.ContainerListOptions) ([]types.Container, error) {
+			return []types.Container{
+				*Container("c1"),
+				*Container("c2", WithName("foo")),
+				*Container("c3", WithPort(80, 80, TCP), WithPort(81, 81, TCP), WithPort(82, 82, TCP)),
+				*Container("c4", WithPort(81, 81, UDP)),
+				*Container("c5", WithPort(82, 82, IP("8.8.8.8"), TCP)),
+			}, nil
+		},
+	})
+	cmd := newListCommand(cli)
+	assert.NoError(t, cmd.Execute())
+	golden.Assert(t, cli.OutBuffer().String(), "container-list-without-format.golden")
+}
+
+func TestContainerListNoTrunc(t *testing.T) {
+	cli := test.NewFakeCli(&fakeClient{
+		containerListFunc: func(_ types.ContainerListOptions) ([]types.Container, error) {
+			return []types.Container{
+				*Container("c1"),
+				*Container("c2", WithName("foo/bar")),
+			}, nil
+		},
+	})
+	cmd := newListCommand(cli)
+	cmd.Flags().Set("no-trunc", "true")
+	assert.NoError(t, cmd.Execute())
+	golden.Assert(t, cli.OutBuffer().String(), "container-list-without-format-no-trunc.golden")
+}
+
+// Test for GitHub issue docker/docker#21772
+func TestContainerListNamesMultipleTime(t *testing.T) {
+	cli := test.NewFakeCli(&fakeClient{
+		containerListFunc: func(_ types.ContainerListOptions) ([]types.Container, error) {
+			return []types.Container{
+				*Container("c1"),
+				*Container("c2", WithName("foo/bar")),
+			}, nil
+		},
+	})
+	cmd := newListCommand(cli)
+	cmd.Flags().Set("format", "{{.Names}} {{.Names}}")
+	assert.NoError(t, cmd.Execute())
+	golden.Assert(t, cli.OutBuffer().String(), "container-list-format-name-name.golden")
+}
+
+// Test for GitHub issue docker/docker#30291
+func TestContainerListFormatTemplateWithArg(t *testing.T) {
+	cli := test.NewFakeCli(&fakeClient{
+		containerListFunc: func(_ types.ContainerListOptions) ([]types.Container, error) {
+			return []types.Container{
+				*Container("c1", WithLabel("some.label", "value")),
+				*Container("c2", WithName("foo/bar"), WithLabel("foo", "bar")),
+			}, nil
+		},
+	})
+	cmd := newListCommand(cli)
+	cmd.Flags().Set("format", `{{.Names}} {{.Label "some.label"}}`)
+	assert.NoError(t, cmd.Execute())
+	golden.Assert(t, cli.OutBuffer().String(), "container-list-format-with-arg.golden")
+}
+
+func TestContainerListFormatSizeSetsOption(t *testing.T) {
+	cli := test.NewFakeCli(&fakeClient{
+		containerListFunc: func(options types.ContainerListOptions) ([]types.Container, error) {
+			assert.True(t, options.Size)
+			return []types.Container{}, nil
+		},
+	})
+	cmd := newListCommand(cli)
+	cmd.Flags().Set("format", `{{.Size}}`)
+	assert.NoError(t, cmd.Execute())
+}
+
+func TestContainerListWithConfigFormat(t *testing.T) {
+	cli := test.NewFakeCli(&fakeClient{
+		containerListFunc: func(_ types.ContainerListOptions) ([]types.Container, error) {
+			return []types.Container{
+				*Container("c1", WithLabel("some.label", "value")),
+				*Container("c2", WithName("foo/bar"), WithLabel("foo", "bar")),
+			}, nil
+		},
+	})
+	cli.SetConfigFile(&configfile.ConfigFile{
+		PsFormat: "{{ .Names }} {{ .Image }} {{ .Labels }}",
+	})
+	cmd := newListCommand(cli)
+	assert.NoError(t, cmd.Execute())
+	golden.Assert(t, cli.OutBuffer().String(), "container-list-with-config-format.golden")
+}
+
+func TestContainerListWithFormat(t *testing.T) {
+	cli := test.NewFakeCli(&fakeClient{
+		containerListFunc: func(_ types.ContainerListOptions) ([]types.Container, error) {
+			return []types.Container{
+				*Container("c1", WithLabel("some.label", "value")),
+				*Container("c2", WithName("foo/bar"), WithLabel("foo", "bar")),
+			}, nil
+		},
+	})
+	cmd := newListCommand(cli)
+	cmd.Flags().Set("format", "{{ .Names }} {{ .Image }} {{ .Labels }}")
+	assert.NoError(t, cmd.Execute())
+	golden.Assert(t, cli.OutBuffer().String(), "container-list-with-format.golden")
+}

components/cli/cli/command/container/opts.go

@@ -145,7 +145,7 @@ func addFlags(flags *pflag.FlagSet) *containerOptions {
 		expose:            opts.NewListOpts(nil),
 		extraHosts:        opts.NewListOpts(opts.ValidateExtraHost),
 		groupAdd:          opts.NewListOpts(nil),
-		labels:            opts.NewListOpts(opts.ValidateEnv),
+		labels:            opts.NewListOpts(nil),
 		labelsFile:        opts.NewListOpts(nil),
 		linkLocalIPs:      opts.NewListOpts(nil),
 		links:             opts.NewListOpts(opts.ValidateLink),
@@ -410,7 +410,7 @@ func parse(flags *pflag.FlagSet, copts *containerOptions) (*containerConfig, err
 	}
 
 	// collect all the environment variables for the container
-	envVariables, err := opts.ReadKVStrings(copts.envFile.GetAll(), copts.env.GetAll())
+	envVariables, err := opts.ReadKVEnvStrings(copts.envFile.GetAll(), copts.env.GetAll())
 	if err != nil {
 		return nil, err
 	}

components/cli/cli/command/container/run_test.go

@@ -0,0 +1,25 @@
+package container
+
+import (
+	"testing"
+
+	"github.com/docker/cli/internal/test"
+	"github.com/docker/docker/api/types/container"
+	"github.com/docker/docker/api/types/network"
+	"github.com/stretchr/testify/assert"
+)
+
+func TestRunLabel(t *testing.T) {
+	cli := test.NewFakeCli(&fakeClient{
+		createContainerFunc: func(_ *container.Config, _ *container.HostConfig, _ *network.NetworkingConfig, _ string) (container.ContainerCreateCreatedBody, error) {
+			return container.ContainerCreateCreatedBody{
+				ID: "id",
+			}, nil
+		},
+		Version: "1.36",
+	})
+	cmd := NewRunCommand(cli)
+	cmd.Flags().Set("detach", "true")
+	cmd.SetArgs([]string{"--label", "foo", "busybox"})
+	assert.NoError(t, cmd.Execute())
+}

components/cli/cli/command/container/testdata/container-list-format-name-name.golden

@@ -0,0 +1,2 @@
+c1 c1
+c2 c2

components/cli/cli/command/container/testdata/container-list-format-with-arg.golden

@@ -0,0 +1,2 @@
+c1 value
+c2 

components/cli/cli/command/container/testdata/container-list-with-config-format.golden

@@ -0,0 +1,2 @@
+c1 busybox:latest some.label=value
+c2 busybox:latest foo=bar

components/cli/cli/command/container/testdata/container-list-with-format.golden

@@ -0,0 +1,2 @@
+c1 busybox:latest some.label=value
+c2 busybox:latest foo=bar

components/cli/cli/command/container/testdata/container-list-without-format-no-trunc.golden

@@ -0,0 +1,3 @@
+CONTAINER ID        IMAGE               COMMAND             CREATED                  STATUS              PORTS               NAMES
+container_id        busybox:latest      "top"               Less than a second ago   Up 1 second                             c1
+container_id        busybox:latest      "top"               Less than a second ago   Up 1 second                             c2,foo/bar

components/cli/cli/command/container/testdata/container-list-without-format.golden

@@ -0,0 +1,6 @@
+CONTAINER ID        IMAGE               COMMAND             CREATED                  STATUS              PORTS                NAMES
+container_id        busybox:latest      "top"               Less than a second ago   Up 1 second                              c1
+container_id        busybox:latest      "top"               Less than a second ago   Up 1 second                              c2
+container_id        busybox:latest      "top"               Less than a second ago   Up 1 second         80-82/tcp            c3
+container_id        busybox:latest      "top"               Less than a second ago   Up 1 second         81/udp               c4
+container_id        busybox:latest      "top"               Less than a second ago   Up 1 second         8.8.8.8:82->82/tcp   c5

components/cli/cli/command/formatter/container_test.go

@@ -642,9 +642,12 @@ func TestDisplayablePorts(t *testing.T) {
 					PublicPort:  1024,
 					PrivatePort: 80,
 					Type:        "udp",
+				}, {
+					PrivatePort: 12345,
+					Type:        "sctp",
 				},
 			},
-			"80/tcp, 80/udp, 1024/tcp, 1024/udp, 1.1.1.1:1024->80/tcp, 1.1.1.1:1024->80/udp, 2.1.1.1:1024->80/tcp, 2.1.1.1:1024->80/udp, 1.1.1.1:80->1024/tcp, 1.1.1.1:80->1024/udp, 2.1.1.1:80->1024/tcp, 2.1.1.1:80->1024/udp",
+			"80/tcp, 80/udp, 1024/tcp, 1024/udp, 12345/sctp, 1.1.1.1:1024->80/tcp, 1.1.1.1:1024->80/udp, 2.1.1.1:1024->80/tcp, 2.1.1.1:1024->80/udp, 1.1.1.1:80->1024/tcp, 1.1.1.1:80->1024/udp, 2.1.1.1:80->1024/tcp, 2.1.1.1:80->1024/udp",
 		},
 	}
 

components/cli/cli/command/formatter/node.go

@@ -14,7 +14,7 @@ import (
 )
 
 const (
-	defaultNodeTableFormat           = "table {{.ID}} {{if .Self}}*{{else}} {{ end }}\t{{.Hostname}}\t{{.Status}}\t{{.Availability}}\t{{.ManagerStatus}}"
+	defaultNodeTableFormat           = "table {{.ID}} {{if .Self}}*{{else}} {{ end }}\t{{.Hostname}}\t{{.Status}}\t{{.Availability}}\t{{.ManagerStatus}}\t{{.EngineVersion}}"
 	nodeInspectPrettyTemplate Format = `ID:			{{.ID}}
 {{- if .Name }}
 Name:			{{.Name}}
@@ -75,6 +75,7 @@ TLS Info:
 	hostnameHeader      = "HOSTNAME"
 	availabilityHeader  = "AVAILABILITY"
 	managerStatusHeader = "MANAGER STATUS"
+	engineVersionHeader = "ENGINE VERSION"
 	tlsStatusHeader     = "TLS STATUS"
 )
 
@@ -115,6 +116,7 @@ func NodeWrite(ctx Context, nodes []swarm.Node, info types.Info) error {
 		"Status":        statusHeader,
 		"Availability":  availabilityHeader,
 		"ManagerStatus": managerStatusHeader,
+		"EngineVersion": engineVersionHeader,
 		"TLSStatus":     tlsStatusHeader,
 	}
 	nodeCtx := nodeContext{}
@@ -176,6 +178,10 @@ func (c *nodeContext) TLSStatus() string {
 	return "Needs Rotation"
 }
 
+func (c *nodeContext) EngineVersion() string {
+	return c.n.Description.Engine.EngineVersion
+}
+
 // NodeInspectWrite renders the context for a list of nodes
 func NodeInspectWrite(ctx Context, refs []string, getRef inspect.GetRefFunc) error {
 	if ctx.Format != nodeInspectPrettyTemplate {

components/cli/cli/command/formatter/node_test.go

@@ -74,10 +74,10 @@ func TestNodeContextWrite(t *testing.T) {
 		// Table format
 		{
 			context: Context{Format: NewNodeFormat("table", false)},
-			expected: `ID                  HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS
-nodeID1             foobar_baz          Foo                 Drain               Leader
-nodeID2             foobar_bar          Bar                 Active              Reachable
-nodeID3             foobar_boo          Boo                 Active              ` + "\n", // (to preserve whitespace)
+			expected: `ID                  HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
+nodeID1             foobar_baz          Foo                 Drain               Leader              18.03.0-ce
+nodeID2             foobar_bar          Bar                 Active              Reachable           1.2.3
+nodeID3             foobar_boo          Boo                 Active                                  ` + "\n", // (to preserve whitespace)
 			clusterInfo: swarm.ClusterInfo{TLSInfo: swarm.TLSInfo{TrustRoot: "hi"}},
 		},
 		{
@@ -172,6 +172,7 @@ foobar_boo  Unknown
 				Description: swarm.NodeDescription{
 					Hostname: "foobar_baz",
 					TLSInfo:  swarm.TLSInfo{TrustRoot: "no"},
+					Engine:   swarm.EngineDescription{EngineVersion: "18.03.0-ce"},
 				},
 				Status:        swarm.NodeStatus{State: swarm.NodeState("foo")},
 				Spec:          swarm.NodeSpec{Availability: swarm.NodeAvailability("drain")},
@@ -182,6 +183,7 @@ foobar_boo  Unknown
 				Description: swarm.NodeDescription{
 					Hostname: "foobar_bar",
 					TLSInfo:  swarm.TLSInfo{TrustRoot: "hi"},
+					Engine:   swarm.EngineDescription{EngineVersion: "1.2.3"},
 				},
 				Status: swarm.NodeStatus{State: swarm.NodeState("bar")},
 				Spec:   swarm.NodeSpec{Availability: swarm.NodeAvailability("active")},
@@ -215,17 +217,17 @@ func TestNodeContextWriteJSON(t *testing.T) {
 	}{
 		{
 			expected: []map[string]interface{}{
-				{"Availability": "", "Hostname": "foobar_baz", "ID": "nodeID1", "ManagerStatus": "", "Status": "", "Self": false, "TLSStatus": "Unknown"},
-				{"Availability": "", "Hostname": "foobar_bar", "ID": "nodeID2", "ManagerStatus": "", "Status": "", "Self": false, "TLSStatus": "Unknown"},
-				{"Availability": "", "Hostname": "foobar_boo", "ID": "nodeID3", "ManagerStatus": "", "Status": "", "Self": false, "TLSStatus": "Unknown"},
+				{"Availability": "", "Hostname": "foobar_baz", "ID": "nodeID1", "ManagerStatus": "", "Status": "", "Self": false, "TLSStatus": "Unknown", "EngineVersion": "1.2.3"},
+				{"Availability": "", "Hostname": "foobar_bar", "ID": "nodeID2", "ManagerStatus": "", "Status": "", "Self": false, "TLSStatus": "Unknown", "EngineVersion": ""},
+				{"Availability": "", "Hostname": "foobar_boo", "ID": "nodeID3", "ManagerStatus": "", "Status": "", "Self": false, "TLSStatus": "Unknown", "EngineVersion": "18.03.0-ce"},
 			},
 			info: types.Info{},
 		},
 		{
 			expected: []map[string]interface{}{
-				{"Availability": "", "Hostname": "foobar_baz", "ID": "nodeID1", "ManagerStatus": "", "Status": "", "Self": false, "TLSStatus": "Ready"},
-				{"Availability": "", "Hostname": "foobar_bar", "ID": "nodeID2", "ManagerStatus": "", "Status": "", "Self": false, "TLSStatus": "Needs Rotation"},
-				{"Availability": "", "Hostname": "foobar_boo", "ID": "nodeID3", "ManagerStatus": "", "Status": "", "Self": false, "TLSStatus": "Unknown"},
+				{"Availability": "", "Hostname": "foobar_baz", "ID": "nodeID1", "ManagerStatus": "", "Status": "", "Self": false, "TLSStatus": "Ready", "EngineVersion": "1.2.3"},
+				{"Availability": "", "Hostname": "foobar_bar", "ID": "nodeID2", "ManagerStatus": "", "Status": "", "Self": false, "TLSStatus": "Needs Rotation", "EngineVersion": ""},
+				{"Availability": "", "Hostname": "foobar_boo", "ID": "nodeID3", "ManagerStatus": "", "Status": "", "Self": false, "TLSStatus": "Unknown", "EngineVersion": "18.03.0-ce"},
 			},
 			info: types.Info{
 				Swarm: swarm.Info{
@@ -240,9 +242,9 @@ func TestNodeContextWriteJSON(t *testing.T) {
 
 	for _, testcase := range cases {
 		nodes := []swarm.Node{
-			{ID: "nodeID1", Description: swarm.NodeDescription{Hostname: "foobar_baz", TLSInfo: swarm.TLSInfo{TrustRoot: "hi"}}},
+			{ID: "nodeID1", Description: swarm.NodeDescription{Hostname: "foobar_baz", TLSInfo: swarm.TLSInfo{TrustRoot: "hi"}, Engine: swarm.EngineDescription{EngineVersion: "1.2.3"}}},
 			{ID: "nodeID2", Description: swarm.NodeDescription{Hostname: "foobar_bar", TLSInfo: swarm.TLSInfo{TrustRoot: "no"}}},
-			{ID: "nodeID3", Description: swarm.NodeDescription{Hostname: "foobar_boo"}},
+			{ID: "nodeID3", Description: swarm.NodeDescription{Hostname: "foobar_boo", Engine: swarm.EngineDescription{EngineVersion: "18.03.0-ce"}}},
 		}
 		out := bytes.NewBufferString("")
 		err := NodeWrite(Context{Format: "{{json .}}", Output: out}, nodes, testcase.info)

components/cli/cli/command/formatter/service_test.go

@@ -338,10 +338,22 @@ func TestServiceContext_Ports(t *testing.T) {
 						PublishedPort: 62,
 						PublishMode:   "ingress",
 					},
+					{
+						Protocol:      "sctp",
+						TargetPort:    97,
+						PublishedPort: 97,
+						PublishMode:   "ingress",
+					},
+					{
+						Protocol:      "sctp",
+						TargetPort:    98,
+						PublishedPort: 98,
+						PublishMode:   "ingress",
+					},
 				},
 			},
 		},
 	}
 
-	assert.Equal(t, "*:60-61->60-61/tcp, *:62->61/tcp, *:80-81->80/tcp, *:90-95->90-95/tcp, *:90-96->90-96/udp", c.Ports())
+	assert.Equal(t, "*:97-98->97-98/sctp, *:60-61->60-61/tcp, *:62->61/tcp, *:80-81->80/tcp, *:90-95->90-95/tcp, *:90-96->90-96/udp", c.Ports())
 }

components/cli/cli/command/image/build.go

@@ -9,8 +9,10 @@ import (
 	"io"
 	"io/ioutil"
 	"os"
+	"path/filepath"
 	"regexp"
 	"runtime"
+	"strings"
 
 	"github.com/docker/cli/cli"
 	"github.com/docker/cli/cli/command"
@@ -206,6 +208,14 @@ func runBuild(dockerCli command.Cli, options buildOptions) error {
 		buildCtx, relDockerfile, err = build.GetContextFromReader(dockerCli.In(), options.dockerfileName)
 	case isLocalDir(specifiedContext):
 		contextDir, relDockerfile, err = build.GetContextFromLocalDir(specifiedContext, options.dockerfileName)
+		if err == nil && strings.HasPrefix(relDockerfile, ".."+string(filepath.Separator)) {
+			// Dockerfile is outside of build-context; read the Dockerfile and pass it as dockerfileCtx
+			dockerfileCtx, err = os.Open(options.dockerfileName)
+			if err != nil {
+				return errors.Errorf("unable to open Dockerfile: %v", err)
+			}
+			defer dockerfileCtx.Close()
+		}
 	case urlutil.IsGitURL(specifiedContext):
 		tempDir, relDockerfile, err = build.GetContextFromGitURL(specifiedContext, options.dockerfileName)
 	case urlutil.IsURL(specifiedContext):
@@ -253,7 +263,7 @@ func runBuild(dockerCli command.Cli, options buildOptions) error {
 		}
 	}
 
-	// replace Dockerfile if it was added from stdin and there is archive context
+	// replace Dockerfile if it was added from stdin or a file outside the build-context, and there is archive context
 	if dockerfileCtx != nil && buildCtx != nil {
 		buildCtx, relDockerfile, err = build.AddDockerfileToBuildContext(dockerfileCtx, buildCtx)
 		if err != nil {
@@ -261,7 +271,7 @@ func runBuild(dockerCli command.Cli, options buildOptions) error {
 		}
 	}
 
-	// if streaming and dockerfile was not from stdin then read from file
+	// if streaming and Dockerfile was not from stdin then read from file
 	// to the same reader that is usually stdin
 	if options.stream && dockerfileCtx == nil {
 		dockerfileCtx, err = os.Open(relDockerfile)

components/cli/cli/command/image/build/context.go

@@ -167,6 +167,10 @@ func GetContextFromGitURL(gitURL, dockerfileName string) (string, string, error)
 		return "", "", err
 	}
 	relDockerfile, err := getDockerfileRelPath(absContextDir, dockerfileName)
+	if err == nil && strings.HasPrefix(relDockerfile, ".."+string(filepath.Separator)) {
+		return "", "", errors.Errorf("the Dockerfile (%s) must be within the build context", dockerfileName)
+	}
+
 	return absContextDir, relDockerfile, err
 }
 
@@ -318,10 +322,6 @@ func getDockerfileRelPath(absContextDir, givenDockerfile string) (string, error)
 		return "", errors.Errorf("unable to get relative Dockerfile path: %v", err)
 	}
 
-	if strings.HasPrefix(relDockerfile, ".."+string(filepath.Separator)) {
-		return "", errors.Errorf("the Dockerfile (%s) must be within the build context", givenDockerfile)
-	}
-
 	return relDockerfile, nil
 }
 

components/cli/cli/command/image/build_test.go

@@ -108,6 +108,56 @@ func TestRunBuildDockerfileFromStdinWithCompress(t *testing.T) {
 	assert.Equal(t, []string{dockerfileName, ".dockerignore", "foo"}, actual)
 }
 
+func TestRunBuildDockerfileOutsideContext(t *testing.T) {
+	dir := fs.NewDir(t, t.Name(),
+		fs.WithFile("data", "data file"),
+	)
+	defer dir.Remove()
+
+	// Dockerfile outside of build-context
+	df := fs.NewFile(t, t.Name(),
+		fs.WithContent(`
+FROM FOOBAR
+COPY data /data
+		`),
+	)
+	defer df.Remove()
+
+	dest, err := ioutil.TempDir("", t.Name())
+	require.NoError(t, err)
+	defer os.RemoveAll(dest)
+
+	var dockerfileName string
+	fakeImageBuild := func(_ context.Context, context io.Reader, options types.ImageBuildOptions) (types.ImageBuildResponse, error) {
+		buffer := new(bytes.Buffer)
+		tee := io.TeeReader(context, buffer)
+
+		assert.NoError(t, archive.Untar(tee, dest, nil))
+		dockerfileName = options.Dockerfile
+
+		body := new(bytes.Buffer)
+		return types.ImageBuildResponse{Body: ioutil.NopCloser(body)}, nil
+	}
+
+	cli := test.NewFakeCli(&fakeClient{imageBuildFunc: fakeImageBuild})
+
+	options := newBuildOptions()
+	options.context = dir.Path()
+	options.dockerfileName = df.Path()
+
+	err = runBuild(cli, options)
+	require.NoError(t, err)
+
+	files, err := ioutil.ReadDir(dest)
+	require.NoError(t, err)
+	var actual []string
+	for _, fileInfo := range files {
+		actual = append(actual, fileInfo.Name())
+	}
+	sort.Strings(actual)
+	assert.Equal(t, []string{dockerfileName, ".dockerignore", "data"}, actual)
+}
+
 // TestRunBuildFromLocalGitHubDirNonExistingRepo tests that build contexts
 // starting with `github.com/` are special-cased, and the build command attempts
 // to clone the remote repo.

components/cli/cli/command/node/list_test.go

@@ -56,8 +56,8 @@ func TestNodeList(t *testing.T) {
 	cli := test.NewFakeCli(&fakeClient{
 		nodeListFunc: func() ([]swarm.Node, error) {
 			return []swarm.Node{
-				*Node(NodeID("nodeID1"), Hostname("node-2-foo"), Manager(Leader())),
-				*Node(NodeID("nodeID2"), Hostname("node-10-foo"), Manager()),
+				*Node(NodeID("nodeID1"), Hostname("node-2-foo"), Manager(Leader()), EngineVersion(".")),
+				*Node(NodeID("nodeID2"), Hostname("node-10-foo"), Manager(), EngineVersion("18.03.0-ce")),
 				*Node(NodeID("nodeID3"), Hostname("node-1-foo")),
 			}, nil
 		},

components/cli/cli/command/node/testdata/node-list-sort.golden

@@ -1,4 +1,4 @@
-ID                  HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS
-nodeID3             node-1-foo          Ready               Active              
-nodeID1 *           node-2-foo          Ready               Active              Leader
-nodeID2             node-10-foo         Ready               Active              Reachable
+ID                  HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
+nodeID3             node-1-foo          Ready               Active                                  1.13.0
+nodeID1 *           node-2-foo          Ready               Active              Leader              .
+nodeID2             node-10-foo         Ready               Active              Reachable           18.03.0-ce

components/cli/cli/command/plugin/client_test.go

@@ -0,0 +1,45 @@
+package plugin
+
+import (
+	"io"
+
+	"github.com/docker/docker/api/types"
+	"github.com/docker/docker/client"
+	"golang.org/x/net/context"
+)
+
+type fakeClient struct {
+	client.Client
+	pluginCreateFunc  func(createContext io.Reader, createOptions types.PluginCreateOptions) error
+	pluginDisableFunc func(name string, disableOptions types.PluginDisableOptions) error
+	pluginEnableFunc  func(name string, options types.PluginEnableOptions) error
+	pluginRemoveFunc  func(name string, options types.PluginRemoveOptions) error
+}
+
+func (c *fakeClient) PluginCreate(ctx context.Context, createContext io.Reader, createOptions types.PluginCreateOptions) error {
+	if c.pluginCreateFunc != nil {
+		return c.pluginCreateFunc(createContext, createOptions)
+	}
+	return nil
+}
+
+func (c *fakeClient) PluginEnable(ctx context.Context, name string, enableOptions types.PluginEnableOptions) error {
+	if c.pluginEnableFunc != nil {
+		return c.pluginEnableFunc(name, enableOptions)
+	}
+	return nil
+}
+
+func (c *fakeClient) PluginDisable(context context.Context, name string, disableOptions types.PluginDisableOptions) error {
+	if c.pluginDisableFunc != nil {
+		return c.pluginDisableFunc(name, disableOptions)
+	}
+	return nil
+}
+
+func (c *fakeClient) PluginRemove(context context.Context, name string, removeOptions types.PluginRemoveOptions) error {
+	if c.pluginRemoveFunc != nil {
+		return c.pluginRemoveFunc(name, removeOptions)
+	}
+	return nil
+}

components/cli/cli/command/plugin/create_test.go

@@ -0,0 +1,114 @@
+package plugin
+
+import (
+	"fmt"
+	"io"
+	"io/ioutil"
+	"testing"
+
+	"github.com/docker/cli/internal/test"
+	"github.com/docker/cli/internal/test/testutil"
+	"github.com/docker/docker/api/types"
+	"github.com/gotestyourself/gotestyourself/fs"
+	"github.com/stretchr/testify/assert"
+)
+
+func TestCreateErrors(t *testing.T) {
+
+	testCases := []struct {
+		args          []string
+		expectedError string
+	}{
+		{
+			args:          []string{},
+			expectedError: "requires at least 2 arguments",
+		},
+		{
+			args:          []string{"INVALID_TAG", "context-dir"},
+			expectedError: "invalid",
+		},
+		{
+			args:          []string{"plugin-foo", "nonexistent_context_dir"},
+			expectedError: "no such file or directory",
+		},
+	}
+
+	for _, tc := range testCases {
+		cli := test.NewFakeCli(&fakeClient{})
+		cmd := newCreateCommand(cli)
+		cmd.SetArgs(tc.args)
+		cmd.SetOutput(ioutil.Discard)
+		testutil.ErrorContains(t, cmd.Execute(), tc.expectedError)
+	}
+}
+
+func TestCreateErrorOnFileAsContextDir(t *testing.T) {
+	tmpFile := fs.NewFile(t, "file-as-context-dir")
+	defer tmpFile.Remove()
+
+	cli := test.NewFakeCli(&fakeClient{})
+	cmd := newCreateCommand(cli)
+	cmd.SetArgs([]string{"plugin-foo", tmpFile.Path()})
+	cmd.SetOutput(ioutil.Discard)
+	testutil.ErrorContains(t, cmd.Execute(), "context must be a directory")
+}
+
+func TestCreateErrorOnContextDirWithoutConfig(t *testing.T) {
+	tmpDir := fs.NewDir(t, "plugin-create-test")
+	defer tmpDir.Remove()
+
+	cli := test.NewFakeCli(&fakeClient{})
+	cmd := newCreateCommand(cli)
+	cmd.SetArgs([]string{"plugin-foo", tmpDir.Path()})
+	cmd.SetOutput(ioutil.Discard)
+	testutil.ErrorContains(t, cmd.Execute(), "config.json: no such file or directory")
+}
+
+func TestCreateErrorOnInvalidConfig(t *testing.T) {
+	tmpDir := fs.NewDir(t, "plugin-create-test",
+		fs.WithDir("rootfs"),
+		fs.WithFile("config.json", "invalid-config-contents"))
+	defer tmpDir.Remove()
+
+	cli := test.NewFakeCli(&fakeClient{})
+	cmd := newCreateCommand(cli)
+	cmd.SetArgs([]string{"plugin-foo", tmpDir.Path()})
+	cmd.SetOutput(ioutil.Discard)
+	testutil.ErrorContains(t, cmd.Execute(), "invalid")
+}
+
+func TestCreateErrorFromDaemon(t *testing.T) {
+	tmpDir := fs.NewDir(t, "plugin-create-test",
+		fs.WithDir("rootfs"),
+		fs.WithFile("config.json", `{ "Name": "plugin-foo" }`))
+	defer tmpDir.Remove()
+
+	cli := test.NewFakeCli(&fakeClient{
+		pluginCreateFunc: func(createContext io.Reader, createOptions types.PluginCreateOptions) error {
+			return fmt.Errorf("Error creating plugin")
+		},
+	})
+
+	cmd := newCreateCommand(cli)
+	cmd.SetArgs([]string{"plugin-foo", tmpDir.Path()})
+	cmd.SetOutput(ioutil.Discard)
+	testutil.ErrorContains(t, cmd.Execute(), "Error creating plugin")
+}
+
+func TestCreatePlugin(t *testing.T) {
+	tmpDir := fs.NewDir(t, "plugin-create-test",
+		fs.WithDir("rootfs"),
+		fs.WithFile("config.json", `{ "Name": "plugin-foo" }`))
+	defer tmpDir.Remove()
+
+	cli := test.NewFakeCli(&fakeClient{
+		pluginCreateFunc: func(createContext io.Reader, createOptions types.PluginCreateOptions) error {
+			return nil
+		},
+	})
+
+	cmd := newCreateCommand(cli)
+	cmd.SetArgs([]string{"plugin-foo", tmpDir.Path()})
+	assert.NoError(t, cmd.Execute())
+	assert.Equal(t, "plugin-foo\n", cli.OutBuffer().String())
+}

components/cli/cli/command/plugin/disable_test.go

@@ -0,0 +1,58 @@
+package plugin
+
+import (
+	"fmt"
+	"io/ioutil"
+	"testing"
+
+	"github.com/docker/cli/internal/test"
+	"github.com/docker/cli/internal/test/testutil"
+	"github.com/docker/docker/api/types"
+	"github.com/stretchr/testify/assert"
+)
+
+func TestPluginDisableErrors(t *testing.T) {
+	testCases := []struct {
+		args              []string
+		expectedError     string
+		pluginDisableFunc func(name string, disableOptions types.PluginDisableOptions) error
+	}{
+		{
+			args:          []string{},
+			expectedError: "requires exactly 1 argument",
+		},
+		{
+			args:          []string{"too", "many", "arguments"},
+			expectedError: "requires exactly 1 argument",
+		},
+		{
+			args:          []string{"plugin-foo"},
+			expectedError: "Error disabling plugin",
+			pluginDisableFunc: func(name string, disableOptions types.PluginDisableOptions) error {
+				return fmt.Errorf("Error disabling plugin")
+			},
+		},
+	}
+
+	for _, tc := range testCases {
+		cmd := newDisableCommand(
+			test.NewFakeCli(&fakeClient{
+				pluginDisableFunc: tc.pluginDisableFunc,
+			}))
+		cmd.SetArgs(tc.args)
+		cmd.SetOutput(ioutil.Discard)
+		testutil.ErrorContains(t, cmd.Execute(), tc.expectedError)
+	}
+}
+
+func TestPluginDisable(t *testing.T) {
+	cli := test.NewFakeCli(&fakeClient{
+		pluginDisableFunc: func(name string, disableOptions types.PluginDisableOptions) error {
+			return nil
+		},
+	})
+	cmd := newDisableCommand(cli)
+	cmd.SetArgs([]string{"plugin-foo"})
+	assert.NoError(t, cmd.Execute())
+	assert.Equal(t, "plugin-foo\n", cli.OutBuffer().String())
+}

components/cli/cli/command/plugin/enable.go

@@ -30,7 +30,7 @@ func newEnableCommand(dockerCli command.Cli) *cobra.Command {
 	}
 
 	flags := cmd.Flags()
-	flags.IntVar(&opts.timeout, "timeout", 0, "HTTP client timeout (in seconds)")
+	flags.IntVar(&opts.timeout, "timeout", 30, "HTTP client timeout (in seconds)")
 	return cmd
 }
 

components/cli/cli/command/plugin/enable_test.go

@@ -0,0 +1,70 @@
+package plugin
+
+import (
+	"fmt"
+	"io/ioutil"
+	"testing"
+
+	"github.com/docker/cli/internal/test"
+	"github.com/docker/cli/internal/test/testutil"
+	"github.com/docker/docker/api/types"
+	"github.com/stretchr/testify/assert"
+)
+
+func TestPluginEnableErrors(t *testing.T) {
+	testCases := []struct {
+		args             []string
+		flags            map[string]string
+		pluginEnableFunc func(name string, options types.PluginEnableOptions) error
+		expectedError    string
+	}{
+		{
+			args:          []string{},
+			expectedError: "requires exactly 1 argument",
+		},
+		{
+			args:          []string{"too-many", "arguments"},
+			expectedError: "requires exactly 1 argument",
+		},
+		{
+			args: []string{"plugin-foo"},
+			pluginEnableFunc: func(name string, options types.PluginEnableOptions) error {
+				return fmt.Errorf("failed to enable plugin")
+			},
+			expectedError: "failed to enable plugin",
+		},
+		{
+			args: []string{"plugin-foo"},
+			flags: map[string]string{
+				"timeout": "-1",
+			},
+			expectedError: "negative timeout -1 is invalid",
+		},
+	}
+
+	for _, tc := range testCases {
+		cmd := newEnableCommand(
+			test.NewFakeCli(&fakeClient{
+				pluginEnableFunc: tc.pluginEnableFunc,
+			}))
+		cmd.SetArgs(tc.args)
+		for key, value := range tc.flags {
+			cmd.Flags().Set(key, value)
+		}
+		cmd.SetOutput(ioutil.Discard)
+		testutil.ErrorContains(t, cmd.Execute(), tc.expectedError)
+	}
+}
+
+func TestPluginEnable(t *testing.T) {
+	cli := test.NewFakeCli(&fakeClient{
+		pluginEnableFunc: func(name string, options types.PluginEnableOptions) error {
+			return nil
+		},
+	})
+
+	cmd := newEnableCommand(cli)
+	cmd.SetArgs([]string{"plugin-foo"})
+	assert.NoError(t, cmd.Execute())
+	assert.Equal(t, "plugin-foo\n", cli.OutBuffer().String())
+}

components/cli/cli/command/plugin/remove_test.go

@@ -0,0 +1,71 @@
+package plugin
+
+import (
+	"fmt"
+	"io/ioutil"
+	"testing"
+
+	"github.com/docker/cli/internal/test"
+	"github.com/docker/cli/internal/test/testutil"
+	"github.com/docker/docker/api/types"
+	"github.com/stretchr/testify/assert"
+)
+
+func TestRemoveErrors(t *testing.T) {
+
+	testCases := []struct {
+		args             []string
+		pluginRemoveFunc func(name string, options types.PluginRemoveOptions) error
+		expectedError    string
+	}{
+		{
+			args:          []string{},
+			expectedError: "requires at least 1 argument",
+		},
+		{
+			args: []string{"plugin-foo"},
+			pluginRemoveFunc: func(name string, options types.PluginRemoveOptions) error {
+				return fmt.Errorf("Error removing plugin")
+			},
+			expectedError: "Error removing plugin",
+		},
+	}
+
+	for _, tc := range testCases {
+		cli := test.NewFakeCli(&fakeClient{
+			pluginRemoveFunc: tc.pluginRemoveFunc,
+		})
+		cmd := newRemoveCommand(cli)
+		cmd.SetArgs(tc.args)
+		cmd.SetOutput(ioutil.Discard)
+		testutil.ErrorContains(t, cmd.Execute(), tc.expectedError)
+	}
+}
+
+func TestRemove(t *testing.T) {
+	cli := test.NewFakeCli(&fakeClient{
+		pluginRemoveFunc: func(name string, options types.PluginRemoveOptions) error {
+			return nil
+		},
+	})
+	cmd := newRemoveCommand(cli)
+	cmd.SetArgs([]string{"plugin-foo"})
+	assert.NoError(t, cmd.Execute())
+	assert.Equal(t, "plugin-foo\n", cli.OutBuffer().String())
+}
+
+func TestRemoveWithForceOption(t *testing.T) {
+	force := false
+	cli := test.NewFakeCli(&fakeClient{
+		pluginRemoveFunc: func(name string, options types.PluginRemoveOptions) error {
+			force = options.Force
+			return nil
+		},
+	})
+	cmd := newRemoveCommand(cli)
+	cmd.SetArgs([]string{"plugin-foo"})
+	cmd.Flags().Set("force", "true")
+	assert.NoError(t, cmd.Execute())
+	assert.True(t, force)
+	assert.Equal(t, "plugin-foo\n", cli.OutBuffer().String())
+}

components/cli/cli/command/secret/create.go

@@ -16,10 +16,11 @@ import (
 )
 
 type createOptions struct {
-	name   string
-	driver string
-	file   string
-	labels opts.ListOpts
+	name           string
+	driver         string
+	templateDriver string
+	file           string
+	labels         opts.ListOpts
 }
 
 func newSecretCreateCommand(dockerCli command.Cli) *cobra.Command {
@@ -43,6 +44,8 @@ func newSecretCreateCommand(dockerCli command.Cli) *cobra.Command {
 	flags.VarP(&options.labels, "label", "l", "Secret labels")
 	flags.StringVarP(&options.driver, "driver", "d", "", "Secret driver")
 	flags.SetAnnotation("driver", "version", []string{"1.31"})
+	flags.StringVar(&options.templateDriver, "template-driver", "", "Template driver")
+	flags.SetAnnotation("driver", "version", []string{"1.37"})
 
 	return cmd
 }
@@ -71,7 +74,11 @@ func runSecretCreate(dockerCli command.Cli, options createOptions) error {
 			Name: options.driver,
 		}
 	}
-
+	if options.templateDriver != "" {
+		spec.Templating = &swarm.Driver{
+			Name: options.templateDriver,
+		}
+	}
 	r, err := client.SecretCreate(ctx, spec)
 	if err != nil {
 		return err

components/cli/cli/command/secret/create_test.go

@@ -11,7 +11,6 @@ import (
 	"github.com/docker/cli/internal/test/testutil"
 	"github.com/docker/docker/api/types"
 	"github.com/docker/docker/api/types/swarm"
-	"github.com/gotestyourself/gotestyourself/golden"
 	"github.com/pkg/errors"
 	"github.com/stretchr/testify/assert"
 )
@@ -52,15 +51,22 @@ func TestSecretCreateErrors(t *testing.T) {
 
 func TestSecretCreateWithName(t *testing.T) {
 	name := "foo"
-	var actual []byte
+	data, err := ioutil.ReadFile(filepath.Join("testdata", secretDataFile))
+	assert.NoError(t, err)
+
+	expected := swarm.SecretSpec{
+		Annotations: swarm.Annotations{
+			Name:   name,
+			Labels: make(map[string]string),
+		},
+		Data: data,
+	}
+
 	cli := test.NewFakeCli(&fakeClient{
 		secretCreateFunc: func(spec swarm.SecretSpec) (types.SecretCreateResponse, error) {
-			if spec.Name != name {
-				return types.SecretCreateResponse{}, errors.Errorf("expected name %q, got %q", name, spec.Name)
+			if !reflect.DeepEqual(spec, expected) {
+				return types.SecretCreateResponse{}, errors.Errorf("expected %+v, got %+v", expected, spec)
 			}
-
-			actual = spec.Data
-
 			return types.SecretCreateResponse{
 				ID: "ID-" + spec.Name,
 			}, nil
@@ -70,7 +76,6 @@ func TestSecretCreateWithName(t *testing.T) {
 	cmd := newSecretCreateCommand(cli)
 	cmd.SetArgs([]string{name, filepath.Join("testdata", secretDataFile)})
 	assert.NoError(t, cmd.Execute())
-	golden.Assert(t, string(actual), secretDataFile)
 	assert.Equal(t, "ID-"+name, strings.TrimSpace(cli.OutBuffer().String()))
 }
 
@@ -86,7 +91,7 @@ func TestSecretCreateWithDriver(t *testing.T) {
 				return types.SecretCreateResponse{}, errors.Errorf("expected name %q, got %q", name, spec.Name)
 			}
 
-			if !reflect.DeepEqual(spec.Driver.Name, expectedDriver.Name) {
+			if spec.Driver.Name != expectedDriver.Name {
 				return types.SecretCreateResponse{}, errors.Errorf("expected driver %v, got %v", expectedDriver, spec.Labels)
 			}
 
@@ -103,6 +108,35 @@ func TestSecretCreateWithDriver(t *testing.T) {
 	assert.Equal(t, "ID-"+name, strings.TrimSpace(cli.OutBuffer().String()))
 }
 
+func TestSecretCreateWithTemplatingDriver(t *testing.T) {
+	expectedDriver := &swarm.Driver{
+		Name: "template-driver",
+	}
+	name := "foo"
+
+	cli := test.NewFakeCli(&fakeClient{
+		secretCreateFunc: func(spec swarm.SecretSpec) (types.SecretCreateResponse, error) {
+			if spec.Name != name {
+				return types.SecretCreateResponse{}, errors.Errorf("expected name %q, got %q", name, spec.Name)
+			}
+
+			if spec.Templating.Name != expectedDriver.Name {
+				return types.SecretCreateResponse{}, errors.Errorf("expected driver %v, got %v", expectedDriver, spec.Labels)
+			}
+
+			return types.SecretCreateResponse{
+				ID: "ID-" + spec.Name,
+			}, nil
+		},
+	})
+
+	cmd := newSecretCreateCommand(cli)
+	cmd.SetArgs([]string{name})
+	cmd.Flags().Set("template-driver", expectedDriver.Name)
+	assert.NoError(t, cmd.Execute())
+	assert.Equal(t, "ID-"+name, strings.TrimSpace(cli.OutBuffer().String()))
+}
+
 func TestSecretCreateWithLabels(t *testing.T) {
 	expectedLabels := map[string]string{
 		"lbl1": "Label-foo",

components/cli/cli/command/service/opts.go

@@ -560,7 +560,7 @@ func (options *serviceOptions) ToStopGracePeriod(flags *pflag.FlagSet) *time.Dur
 func (options *serviceOptions) ToService(ctx context.Context, apiClient client.NetworkAPIClient, flags *pflag.FlagSet) (swarm.ServiceSpec, error) {
 	var service swarm.ServiceSpec
 
-	envVariables, err := opts.ReadKVStrings(options.envFile.GetAll(), options.env.GetAll())
+	envVariables, err := opts.ReadKVEnvStrings(options.envFile.GetAll(), options.env.GetAll())
 	if err != nil {
 		return service, err
 	}

components/cli/cli/command/stack/deploy.go

@@ -34,7 +34,7 @@ func newDeployCommand(dockerCli command.Cli) *cobra.Command {
 	flags.StringVar(&opts.Bundlefile, "bundle-file", "", "Path to a Distributed Application Bundle file")
 	flags.SetAnnotation("bundle-file", "experimental", nil)
 	flags.SetAnnotation("bundle-file", "swarm", nil)
-	flags.StringVarP(&opts.Composefile, "compose-file", "c", "", "Path to a Compose file")
+	flags.StringSliceVarP(&opts.Composefiles, "compose-file", "c", []string{}, "Path to a Compose file")
 	flags.SetAnnotation("compose-file", "version", []string{"1.25"})
 	flags.BoolVar(&opts.SendRegistryAuth, "with-registry-auth", false, "Send registry authentication details to Swarm agents")
 	flags.SetAnnotation("with-registry-auth", "swarm", nil)

components/cli/cli/command/stack/kubernetes/deploy.go

@@ -5,8 +5,9 @@ import (
 	"io/ioutil"
 	"path"
 
+	"github.com/docker/cli/cli/command/stack/loader"
 	"github.com/docker/cli/cli/command/stack/options"
-	composeTypes "github.com/docker/cli/cli/compose/types"
+	composetypes "github.com/docker/cli/cli/compose/types"
 	"github.com/pkg/errors"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	corev1 "k8s.io/client-go/kubernetes/typed/core/v1"
@@ -16,9 +17,20 @@ import (
 func RunDeploy(dockerCli *KubeCli, opts options.Deploy) error {
 	cmdOut := dockerCli.Out()
 	// Check arguments
-	if opts.Composefile == "" {
-		return errors.Errorf("Please specify a Compose file (with --compose-file).")
+	if len(opts.Composefiles) == 0 {
+		return errors.Errorf("Please specify only one compose file (with --compose-file).")
 	}
+
+	// Parse the compose file
+	cfg, err := loader.LoadComposefile(dockerCli, opts)
+	if err != nil {
+		return err
+	}
+	stack, err := LoadStack(opts.Namespace, *cfg)
+	if err != nil {
+		return err
+	}
+
 	// Initialize clients
 	stacks, err := dockerCli.stacks()
 	if err != nil {
@@ -36,12 +48,6 @@ func RunDeploy(dockerCli *KubeCli, opts options.Deploy) error {
 		Pods: pods,
 	}
 
-	// Parse the compose file
-	stack, cfg, err := LoadStack(opts.Namespace, opts.Composefile)
-	if err != nil {
-		return err
-	}
-
 	// FIXME(vdemeester) handle warnings server-side
 	if err = IsColliding(services, stack, cfg); err != nil {
 		return err
@@ -82,7 +88,7 @@ func RunDeploy(dockerCli *KubeCli, opts options.Deploy) error {
 }
 
 // createFileBasedConfigMaps creates a Kubernetes ConfigMap for each Compose global file-based config.
-func createFileBasedConfigMaps(stackName string, globalConfigs map[string]composeTypes.ConfigObjConfig, configMaps corev1.ConfigMapInterface) error {
+func createFileBasedConfigMaps(stackName string, globalConfigs map[string]composetypes.ConfigObjConfig, configMaps corev1.ConfigMapInterface) error {
 	for name, config := range globalConfigs {
 		if config.File == "" {
 			continue
@@ -102,7 +108,7 @@ func createFileBasedConfigMaps(stackName string, globalConfigs map[string]compos
 	return nil
 }
 
-func serviceNames(cfg *composeTypes.Config) []string {
+func serviceNames(cfg *composetypes.Config) []string {
 	names := []string{}
 
 	for _, service := range cfg.Services {
@@ -113,7 +119,7 @@ func serviceNames(cfg *composeTypes.Config) []string {
 }
 
 // createFileBasedSecrets creates a Kubernetes Secret for each Compose global file-based secret.
-func createFileBasedSecrets(stackName string, globalSecrets map[string]composeTypes.SecretConfig, secrets corev1.SecretInterface) error {
+func createFileBasedSecrets(stackName string, globalSecrets map[string]composetypes.SecretConfig, secrets corev1.SecretInterface) error {
 	for name, secret := range globalSecrets {
 		if secret.File == "" {
 			continue

components/cli/cli/command/stack/kubernetes/loader.go

@@ -1,169 +1,24 @@
 package kubernetes
 
 import (
-	"bufio"
-	"io/ioutil"
-	"os"
-	"path/filepath"
-	"strings"
-
-	"github.com/docker/cli/cli/compose/loader"
-	"github.com/docker/cli/cli/compose/template"
 	composetypes "github.com/docker/cli/cli/compose/types"
 	apiv1beta1 "github.com/docker/cli/kubernetes/compose/v1beta1"
-	"github.com/pkg/errors"
 	yaml "gopkg.in/yaml.v2"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 )
 
-// LoadStack loads a stack from a Compose file, with a given name.
-// FIXME(vdemeester) remove this and use cli/compose/loader for both swarm and kubernetes
-func LoadStack(name, composeFile string) (*apiv1beta1.Stack, *composetypes.Config, error) {
-	if composeFile == "" {
-		return nil, nil, errors.New("compose-file must be set")
-	}
-
-	workingDir, err := os.Getwd()
+// LoadStack loads a stack from a Compose config, with a given name.
+func LoadStack(name string, cfg composetypes.Config) (*apiv1beta1.Stack, error) {
+	res, err := yaml.Marshal(cfg)
 	if err != nil {
-		return nil, nil, err
+		return nil, err
 	}
-
-	composePath := composeFile
-	if !strings.HasPrefix(composePath, "/") {
-		composePath = filepath.Join(workingDir, composeFile)
-	}
-
-	if _, err := os.Stat(composePath); os.IsNotExist(err) {
-		return nil, nil, errors.Errorf("no compose file found in %s", filepath.Dir(composePath))
-	}
-
-	binary, err := ioutil.ReadFile(composePath)
-	if err != nil {
-		return nil, nil, errors.Wrap(err, "cannot read compose file")
-	}
-
-	env := env(workingDir)
-	return load(name, binary, workingDir, env)
-}
-
-func load(name string, binary []byte, workingDir string, env map[string]string) (*apiv1beta1.Stack, *composetypes.Config, error) {
-	processed, err := template.Substitute(string(binary), func(key string) (string, bool) { return env[key], true })
-	if err != nil {
-		return nil, nil, errors.Wrap(err, "cannot load compose file")
-	}
-
-	parsed, err := loader.ParseYAML([]byte(processed))
-	if err != nil {
-		return nil, nil, errors.Wrapf(err, "cannot load compose file")
-	}
-
-	cfg, err := loader.Load(composetypes.ConfigDetails{
-		WorkingDir: workingDir,
-		ConfigFiles: []composetypes.ConfigFile{
-			{
-				Config: parsed,
-			},
-		},
-	})
-	if err != nil {
-		return nil, nil, errors.Wrapf(err, "cannot load compose file")
-	}
-
-	result, err := processEnvFiles(processed, parsed, cfg)
-	if err != nil {
-		return nil, nil, errors.Wrapf(err, "cannot load compose file")
-	}
-
 	return &apiv1beta1.Stack{
 		ObjectMeta: metav1.ObjectMeta{
 			Name: name,
 		},
 		Spec: apiv1beta1.StackSpec{
-			ComposeFile: result,
+			ComposeFile: string(res),
 		},
-	}, cfg, nil
-}
-
-type iMap = map[string]interface{}
-
-func processEnvFiles(input string, parsed map[string]interface{}, config *composetypes.Config) (string, error) {
-	changed := false
-
-	for _, svc := range config.Services {
-		if len(svc.EnvFile) == 0 {
-			continue
-		}
-		// Load() processed the env_file for us, we just need to inject back into
-		// the intermediate representation
-		env := iMap{}
-		for k, v := range svc.Environment {
-			env[k] = v
-		}
-		parsed["services"].(iMap)[svc.Name].(iMap)["environment"] = env
-		delete(parsed["services"].(iMap)[svc.Name].(iMap), "env_file")
-		changed = true
-	}
-	if !changed {
-		return input, nil
-	}
-	res, err := yaml.Marshal(parsed)
-	if err != nil {
-		return "", err
-	}
-	return string(res), nil
-}
-
-func env(workingDir string) map[string]string {
-	// Apply .env file first
-	config := readEnvFile(filepath.Join(workingDir, ".env"))
-
-	// Apply env variables
-	for k, v := range envToMap(os.Environ()) {
-		config[k] = v
-	}
-
-	return config
-}
-
-func readEnvFile(path string) map[string]string {
-	config := map[string]string{}
-
-	file, err := os.Open(path)
-	if err != nil {
-		return config // Ignore
-	}
-	defer file.Close()
-
-	scanner := bufio.NewScanner(file)
-	for scanner.Scan() {
-		line := scanner.Text()
-		if strings.HasPrefix(strings.TrimSpace(line), "#") {
-			continue
-		}
-
-		parts := strings.SplitN(line, "=", 2)
-		if len(parts) == 2 {
-			key := parts[0]
-			value := parts[1]
-
-			config[key] = value
-		}
-	}
-
-	return config
-}
-
-func envToMap(env []string) map[string]string {
-	config := map[string]string{}
-
-	for _, value := range env {
-		parts := strings.SplitN(value, "=", 2)
-
-		key := parts[0]
-		value := parts[1]
-
-		config[key] = value
-	}
-
-	return config
+	}, nil
 }

components/cli/cli/command/stack/kubernetes/loader_test.go

@@ -3,32 +3,44 @@ package kubernetes
 import (
 	"testing"
 
-	"github.com/stretchr/testify/assert"
+	composetypes "github.com/docker/cli/cli/compose/types"
+	apiv1beta1 "github.com/docker/cli/kubernetes/compose/v1beta1"
 	"github.com/stretchr/testify/require"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 )
 
-func TestPlaceholders(t *testing.T) {
-	env := map[string]string{
-		"TAG": "_latest_",
-		"K1":  "V1",
-		"K2":  "V2",
-	}
-
-	prefix := "version: '3'\nvolumes:\n  data:\n    external:\n      name: "
-	var tests = []struct {
-		input          string
-		expectedOutput string
-	}{
-		{prefix + "BEFORE${TAG}AFTER", prefix + "BEFORE_latest_AFTER"},
-		{prefix + "BEFORE${K1}${K2}AFTER", prefix + "BEFOREV1V2AFTER"},
-		{prefix + "BEFORE$TAG AFTER", prefix + "BEFORE_latest_ AFTER"},
-		{prefix + "BEFORE$$TAG AFTER", prefix + "BEFORE$TAG AFTER"},
-		{prefix + "BEFORE $UNKNOWN AFTER", prefix + "BEFORE  AFTER"},
-	}
-
-	for _, test := range tests {
-		output, _, err := load("stack", []byte(test.input), ".", env)
-		require.NoError(t, err)
-		assert.Equal(t, test.expectedOutput, output.Spec.ComposeFile)
-	}
+func TestLoadStack(t *testing.T) {
+	s, err := LoadStack("foo", composetypes.Config{
+		Version:  "3.1",
+		Filename: "banana",
+		Services: []composetypes.ServiceConfig{
+			{
+				Name:  "foo",
+				Image: "foo",
+			},
+			{
+				Name:  "bar",
+				Image: "bar",
+			},
+		},
+	})
+	require.NoError(t, err)
+	require.Equal(t, &apiv1beta1.Stack{
+		ObjectMeta: metav1.ObjectMeta{
+			Name: "foo",
+		},
+		Spec: apiv1beta1.StackSpec{
+			ComposeFile: string(`version: "3.1"
+services:
+  bar:
+    image: bar
+  foo:
+    image: foo
+networks: {}
+volumes: {}
+secrets: {}
+configs: {}
+`),
+		},
+	}, s)
 }

components/cli/cli/command/stack/loader/loader.go

@@ -0,0 +1,152 @@
+package loader
+
+import (
+	"fmt"
+	"io"
+	"io/ioutil"
+	"os"
+	"path/filepath"
+	"sort"
+	"strings"
+
+	"github.com/docker/cli/cli/command"
+	"github.com/docker/cli/cli/command/stack/options"
+	"github.com/docker/cli/cli/compose/loader"
+	"github.com/docker/cli/cli/compose/schema"
+	composetypes "github.com/docker/cli/cli/compose/types"
+	"github.com/pkg/errors"
+)
+
+// LoadComposefile parse the composefile specified in the cli and returns its Config and version.
+func LoadComposefile(dockerCli command.Cli, opts options.Deploy) (*composetypes.Config, error) {
+	configDetails, err := getConfigDetails(opts.Composefiles, dockerCli.In())
+	if err != nil {
+		return nil, err
+	}
+
+	dicts := getDictsFrom(configDetails.ConfigFiles)
+	config, err := loader.Load(configDetails)
+	if err != nil {
+		if fpe, ok := err.(*loader.ForbiddenPropertiesError); ok {
+			return nil, errors.Errorf("Compose file contains unsupported options:\n\n%s\n",
+				propertyWarnings(fpe.Properties))
+		}
+
+		return nil, err
+	}
+
+	unsupportedProperties := loader.GetUnsupportedProperties(dicts...)
+	if len(unsupportedProperties) > 0 {
+		fmt.Fprintf(dockerCli.Err(), "Ignoring unsupported options: %s\n\n",
+			strings.Join(unsupportedProperties, ", "))
+	}
+
+	deprecatedProperties := loader.GetDeprecatedProperties(dicts...)
+	if len(deprecatedProperties) > 0 {
+		fmt.Fprintf(dockerCli.Err(), "Ignoring deprecated options:\n\n%s\n\n",
+			propertyWarnings(deprecatedProperties))
+	}
+	return config, nil
+}
+
+func getDictsFrom(configFiles []composetypes.ConfigFile) []map[string]interface{} {
+	dicts := []map[string]interface{}{}
+
+	for _, configFile := range configFiles {
+		dicts = append(dicts, configFile.Config)
+	}
+
+	return dicts
+}
+
+func propertyWarnings(properties map[string]string) string {
+	var msgs []string
+	for name, description := range properties {
+		msgs = append(msgs, fmt.Sprintf("%s: %s", name, description))
+	}
+	sort.Strings(msgs)
+	return strings.Join(msgs, "\n\n")
+}
+
+func getConfigDetails(composefiles []string, stdin io.Reader) (composetypes.ConfigDetails, error) {
+	var details composetypes.ConfigDetails
+
+	if len(composefiles) == 0 {
+		return details, errors.New("no composefile(s)")
+	}
+
+	if composefiles[0] == "-" && len(composefiles) == 1 {
+		workingDir, err := os.Getwd()
+		if err != nil {
+			return details, err
+		}
+		details.WorkingDir = workingDir
+	} else {
+		absPath, err := filepath.Abs(composefiles[0])
+		if err != nil {
+			return details, err
+		}
+		details.WorkingDir = filepath.Dir(absPath)
+	}
+
+	var err error
+	details.ConfigFiles, err = loadConfigFiles(composefiles, stdin)
+	if err != nil {
+		return details, err
+	}
+	// Take the first file version (2 files can't have different version)
+	details.Version = schema.Version(details.ConfigFiles[0].Config)
+	details.Environment, err = buildEnvironment(os.Environ())
+	return details, err
+}
+
+func buildEnvironment(env []string) (map[string]string, error) {
+	result := make(map[string]string, len(env))
+	for _, s := range env {
+		// if value is empty, s is like "K=", not "K".
+		if !strings.Contains(s, "=") {
+			return result, errors.Errorf("unexpected environment %q", s)
+		}
+		kv := strings.SplitN(s, "=", 2)
+		result[kv[0]] = kv[1]
+	}
+	return result, nil
+}
+
+func loadConfigFiles(filenames []string, stdin io.Reader) ([]composetypes.ConfigFile, error) {
+	var configFiles []composetypes.ConfigFile
+
+	for _, filename := range filenames {
+		configFile, err := loadConfigFile(filename, stdin)
+		if err != nil {
+			return configFiles, err
+		}
+		configFiles = append(configFiles, *configFile)
+	}
+
+	return configFiles, nil
+}
+
+func loadConfigFile(filename string, stdin io.Reader) (*composetypes.ConfigFile, error) {
+	var bytes []byte
+	var err error
+
+	if filename == "-" {
+		bytes, err = ioutil.ReadAll(stdin)
+	} else {
+		bytes, err = ioutil.ReadFile(filename)
+	}
+	if err != nil {
+		return nil, err
+	}
+
+	config, err := loader.ParseYAML(bytes)
+	if err != nil {
+		return nil, err
+	}
+
+	return &composetypes.ConfigFile{
+		Filename: filename,
+		Config:   config,
+	}, nil
+}

components/cli/cli/command/stack/loader/loader_test.go

@@ -0,0 +1,47 @@
+package loader
+
+import (
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+
+	"github.com/gotestyourself/gotestyourself/fs"
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+)
+
+func TestGetConfigDetails(t *testing.T) {
+	content := `
+version: "3.0"
+services:
+  foo:
+    image: alpine:3.5
+`
+	file := fs.NewFile(t, "test-get-config-details", fs.WithContent(content))
+	defer file.Remove()
+
+	details, err := getConfigDetails([]string{file.Path()}, nil)
+	require.NoError(t, err)
+	assert.Equal(t, filepath.Dir(file.Path()), details.WorkingDir)
+	require.Len(t, details.ConfigFiles, 1)
+	assert.Equal(t, "3.0", details.ConfigFiles[0].Config["version"])
+	assert.Len(t, details.Environment, len(os.Environ()))
+}
+
+func TestGetConfigDetailsStdin(t *testing.T) {
+	content := `
+version: "3.0"
+services:
+  foo:
+    image: alpine:3.5
+`
+	details, err := getConfigDetails([]string{"-"}, strings.NewReader(content))
+	require.NoError(t, err)
+	cwd, err := os.Getwd()
+	require.NoError(t, err)
+	assert.Equal(t, cwd, details.WorkingDir)
+	require.Len(t, details.ConfigFiles, 1)
+	assert.Equal(t, "3.0", details.ConfigFiles[0].Config["version"])
+	assert.Len(t, details.Environment, len(os.Environ()))
+}

components/cli/cli/command/stack/options/opts.go

@@ -5,7 +5,7 @@ import "github.com/docker/cli/opts"
 // Deploy holds docker stack deploy options
 type Deploy struct {
 	Bundlefile       string
-	Composefile      string
+	Composefiles     []string
 	Namespace        string
 	ResolveImage     string
 	SendRegistryAuth bool

components/cli/cli/command/stack/swarm/deploy.go

@@ -29,9 +29,9 @@ func RunDeploy(dockerCli command.Cli, opts options.Deploy) error {
 	}
 
 	switch {
-	case opts.Bundlefile == "" && opts.Composefile == "":
+	case opts.Bundlefile == "" && len(opts.Composefiles) == 0:
 		return errors.Errorf("Please specify either a bundle file (with --bundle-file) or a Compose file (with --compose-file).")
-	case opts.Bundlefile != "" && opts.Composefile != "":
+	case opts.Bundlefile != "" && len(opts.Composefiles) != 0:
 		return errors.Errorf("You cannot specify both a bundle file and a Compose file.")
 	case opts.Bundlefile != "":
 		return deployBundle(ctx, dockerCli, opts)

components/cli/cli/command/stack/swarm/deploy_composefile.go

@@ -2,17 +2,11 @@ package swarm
 
 import (
 	"fmt"
-	"io"
-	"io/ioutil"
-	"os"
-	"path/filepath"
-	"sort"
-	"strings"
 
 	"github.com/docker/cli/cli/command"
+	"github.com/docker/cli/cli/command/stack/loader"
 	"github.com/docker/cli/cli/command/stack/options"
 	"github.com/docker/cli/cli/compose/convert"
-	"github.com/docker/cli/cli/compose/loader"
 	composetypes "github.com/docker/cli/cli/compose/types"
 	"github.com/docker/docker/api/types"
 	"github.com/docker/docker/api/types/container"
@@ -24,33 +18,11 @@ import (
 )
 
 func deployCompose(ctx context.Context, dockerCli command.Cli, opts options.Deploy) error {
-	configDetails, err := getConfigDetails(opts.Composefile, dockerCli.In())
+	config, err := loader.LoadComposefile(dockerCli, opts)
 	if err != nil {
 		return err
 	}
 
-	config, err := loader.Load(configDetails)
-	if err != nil {
-		if fpe, ok := err.(*loader.ForbiddenPropertiesError); ok {
-			return errors.Errorf("Compose file contains unsupported options:\n\n%s\n",
-				propertyWarnings(fpe.Properties))
-		}
-
-		return err
-	}
-
-	unsupportedProperties := loader.GetUnsupportedProperties(configDetails)
-	if len(unsupportedProperties) > 0 {
-		fmt.Fprintf(dockerCli.Err(), "Ignoring unsupported options: %s\n\n",
-			strings.Join(unsupportedProperties, ", "))
-	}
-
-	deprecatedProperties := loader.GetDeprecatedProperties(configDetails)
-	if len(deprecatedProperties) > 0 {
-		fmt.Fprintf(dockerCli.Err(), "Ignoring deprecated options:\n\n%s\n\n",
-			propertyWarnings(deprecatedProperties))
-	}
-
 	if err := checkDaemonIsSwarmManager(ctx, dockerCli); err != nil {
 		return err
 	}
@@ -111,79 +83,6 @@ func getServicesDeclaredNetworks(serviceConfigs []composetypes.ServiceConfig) ma
 	return serviceNetworks
 }
 
-func propertyWarnings(properties map[string]string) string {
-	var msgs []string
-	for name, description := range properties {
-		msgs = append(msgs, fmt.Sprintf("%s: %s", name, description))
-	}
-	sort.Strings(msgs)
-	return strings.Join(msgs, "\n\n")
-}
-
-func getConfigDetails(composefile string, stdin io.Reader) (composetypes.ConfigDetails, error) {
-	var details composetypes.ConfigDetails
-
-	if composefile == "-" {
-		workingDir, err := os.Getwd()
-		if err != nil {
-			return details, err
-		}
-		details.WorkingDir = workingDir
-	} else {
-		absPath, err := filepath.Abs(composefile)
-		if err != nil {
-			return details, err
-		}
-		details.WorkingDir = filepath.Dir(absPath)
-	}
-
-	configFile, err := getConfigFile(composefile, stdin)
-	if err != nil {
-		return details, err
-	}
-	// TODO: support multiple files
-	details.ConfigFiles = []composetypes.ConfigFile{*configFile}
-	details.Environment, err = buildEnvironment(os.Environ())
-	return details, err
-}
-
-func buildEnvironment(env []string) (map[string]string, error) {
-	result := make(map[string]string, len(env))
-	for _, s := range env {
-		// if value is empty, s is like "K=", not "K".
-		if !strings.Contains(s, "=") {
-			return result, errors.Errorf("unexpected environment %q", s)
-		}
-		kv := strings.SplitN(s, "=", 2)
-		result[kv[0]] = kv[1]
-	}
-	return result, nil
-}
-
-func getConfigFile(filename string, stdin io.Reader) (*composetypes.ConfigFile, error) {
-	var bytes []byte
-	var err error
-
-	if filename == "-" {
-		bytes, err = ioutil.ReadAll(stdin)
-	} else {
-		bytes, err = ioutil.ReadFile(filename)
-	}
-	if err != nil {
-		return nil, err
-	}
-
-	config, err := loader.ParseYAML(bytes)
-	if err != nil {
-		return nil, err
-	}
-
-	return &composetypes.ConfigFile{
-		Filename: filename,
-		Config:   config,
-	}, nil
-}
-
 func validateExternalNetworks(
 	ctx context.Context,
 	client dockerclient.NetworkAPIClient,

components/cli/cli/command/stack/swarm/deploy_composefile_test.go

@@ -1,56 +1,16 @@
 package swarm
 
 import (
-	"os"
-	"path/filepath"
-	"strings"
 	"testing"
 
 	"github.com/docker/cli/internal/test/network"
 	"github.com/docker/cli/internal/test/testutil"
 	"github.com/docker/docker/api/types"
-	"github.com/gotestyourself/gotestyourself/fs"
 	"github.com/pkg/errors"
 	"github.com/stretchr/testify/assert"
-	"github.com/stretchr/testify/require"
 	"golang.org/x/net/context"
 )
 
-func TestGetConfigDetails(t *testing.T) {
-	content := `
-version: "3.0"
-services:
-  foo:
-    image: alpine:3.5
-`
-	file := fs.NewFile(t, "test-get-config-details", fs.WithContent(content))
-	defer file.Remove()
-
-	details, err := getConfigDetails(file.Path(), nil)
-	require.NoError(t, err)
-	assert.Equal(t, filepath.Dir(file.Path()), details.WorkingDir)
-	require.Len(t, details.ConfigFiles, 1)
-	assert.Equal(t, "3.0", details.ConfigFiles[0].Config["version"])
-	assert.Len(t, details.Environment, len(os.Environ()))
-}
-
-func TestGetConfigDetailsStdin(t *testing.T) {
-	content := `
-version: "3.0"
-services:
-  foo:
-    image: alpine:3.5
-`
-	details, err := getConfigDetails("-", strings.NewReader(content))
-	require.NoError(t, err)
-	cwd, err := os.Getwd()
-	require.NoError(t, err)
-	assert.Equal(t, cwd, details.WorkingDir)
-	require.Len(t, details.ConfigFiles, 1)
-	assert.Equal(t, "3.0", details.ConfigFiles[0].Config["version"])
-	assert.Len(t, details.Environment, len(os.Environ()))
-}
-
 type notFound struct {
 	error
 }

components/cli/cli/command/trust/cmd.go

@@ -9,14 +9,12 @@ import (
 // NewTrustCommand returns a cobra command for `trust` subcommands
 func NewTrustCommand(dockerCli command.Cli) *cobra.Command {
 	cmd := &cobra.Command{
-		Use:         "trust",
-		Short:       "Manage trust on Docker images (experimental)",
-		Args:        cli.NoArgs,
-		RunE:        command.ShowHelp(dockerCli.Err()),
-		Annotations: map[string]string{"experimentalCLI": ""},
+		Use:   "trust",
+		Short: "Manage trust on Docker images",
+		Args:  cli.NoArgs,
+		RunE:  command.ShowHelp(dockerCli.Err()),
 	}
 	cmd.AddCommand(
-		newViewCommand(dockerCli),
 		newRevokeCommand(dockerCli),
 		newSignCommand(dockerCli),
 		newTrustKeyCommand(dockerCli),

components/cli/cli/command/trust/inspect.go

@@ -2,6 +2,7 @@ package trust
 
 import (
 	"encoding/json"
+	"fmt"
 	"sort"
 
 	"github.com/docker/cli/cli"
@@ -11,24 +12,55 @@ import (
 	"github.com/theupdateframework/notary/tuf/data"
 )
 
+type inspectOptions struct {
+	remotes []string
+	// FIXME(n4ss): this is consistent with `docker service inspect` but we should provide
+	// a `--format` flag too. (format and pretty-print should be exclusive)
+	prettyPrint bool
+}
+
 func newInspectCommand(dockerCli command.Cli) *cobra.Command {
+	options := inspectOptions{}
 	cmd := &cobra.Command{
 		Use:   "inspect IMAGE[:TAG] [IMAGE[:TAG]...]",
 		Short: "Return low-level information about keys and signatures",
 		Args:  cli.RequiresMinArgs(1),
 		RunE: func(cmd *cobra.Command, args []string) error {
-			return runInspect(dockerCli, args)
+			options.remotes = args
+
+			return runInspect(dockerCli, options)
 		},
 	}
+
+	flags := cmd.Flags()
+	flags.BoolVar(&options.prettyPrint, "pretty", false, "Print the information in a human friendly format")
+
 	return cmd
 }
 
-func runInspect(dockerCli command.Cli, remotes []string) error {
+func runInspect(dockerCli command.Cli, opts inspectOptions) error {
+	if opts.prettyPrint {
+		var err error
+
+		for index, remote := range opts.remotes {
+			if err = prettyPrintTrustInfo(dockerCli, remote); err != nil {
+				return err
+			}
+
+			// Additional separator between the inspection output of each image
+			if index < len(opts.remotes)-1 {
+				fmt.Fprint(dockerCli.Out(), "\n\n")
+			}
+		}
+
+		return err
+	}
+
 	getRefFunc := func(ref string) (interface{}, []byte, error) {
 		i, err := getRepoTrustInfo(dockerCli, ref)
 		return nil, i, err
 	}
-	return inspect.Inspect(dockerCli.Out(), remotes, "", getRefFunc)
+	return inspect.Inspect(dockerCli.Out(), opts.remotes, "", getRefFunc)
 }
 
 func getRepoTrustInfo(cli command.Cli, remote string) ([]byte, error) {

components/cli/cli/command/trust/inspect_pretty.go

@@ -4,34 +4,21 @@ import (
 	"fmt"
 	"io"
 	"sort"
-	"strings"
 
-	"github.com/docker/cli/cli"
 	"github.com/docker/cli/cli/command"
 	"github.com/docker/cli/cli/command/formatter"
-	"github.com/spf13/cobra"
 	"github.com/theupdateframework/notary/client"
 )
 
-func newViewCommand(dockerCli command.Cli) *cobra.Command {
-	cmd := &cobra.Command{
-		Use:   "view IMAGE[:TAG]",
-		Short: "Display detailed information about keys and signatures",
-		Args:  cli.ExactArgs(1),
-		RunE: func(cmd *cobra.Command, args []string) error {
-			return viewTrustInfo(dockerCli, args[0])
-		},
-	}
-	return cmd
-}
-
-func viewTrustInfo(cli command.Cli, remote string) error {
+func prettyPrintTrustInfo(cli command.Cli, remote string) error {
 	signatureRows, adminRolesWithSigs, delegationRoles, err := lookupTrustInfo(cli, remote)
 	if err != nil {
 		return err
 	}
 
 	if len(signatureRows) > 0 {
+		fmt.Fprintf(cli.Out(), "\nSignatures for %s\n\n", remote)
+
 		if err := printSignatures(cli.Out(), signatureRows); err != nil {
 			return err
 		}
@@ -42,14 +29,14 @@ func viewTrustInfo(cli command.Cli, remote string) error {
 
 	// If we do not have additional signers, do not display
 	if len(signerRoleToKeyIDs) > 0 {
-		fmt.Fprintf(cli.Out(), "\nList of signers and their keys for %s:\n\n", strings.Split(remote, ":")[0])
+		fmt.Fprintf(cli.Out(), "\nList of signers and their keys for %s\n\n", remote)
 		if err := printSignerInfo(cli.Out(), signerRoleToKeyIDs); err != nil {
 			return err
 		}
 	}
 
 	// This will always have the root and targets information
-	fmt.Fprintf(cli.Out(), "\nAdministrative keys for %s:\n", strings.Split(remote, ":")[0])
+	fmt.Fprintf(cli.Out(), "\nAdministrative keys for %s\n\n", remote)
 	printSortedAdminKeys(cli.Out(), adminRolesWithSigs)
 	return nil
 }
@@ -57,7 +44,9 @@ func viewTrustInfo(cli command.Cli, remote string) error {
 func printSortedAdminKeys(out io.Writer, adminRoles []client.RoleWithSignatures) {
 	sort.Slice(adminRoles, func(i, j int) bool { return adminRoles[i].Name > adminRoles[j].Name })
 	for _, adminRole := range adminRoles {
-		fmt.Fprintf(out, "%s", formatAdminRole(adminRole))
+		if formattedAdminRole := formatAdminRole(adminRole); formattedAdminRole != "" {
+			fmt.Fprintf(out, "  %s", formattedAdminRole)
+		}
 	}
 }
 

components/cli/cli/command/trust/inspect_pretty_test.go

@@ -16,11 +16,13 @@ import (
 	"github.com/theupdateframework/notary/tuf/data"
 )
 
+// TODO(n4ss): remove common tests with the regular inspect command
+
 type fakeClient struct {
 	dockerClient.Client
 }
 
-func TestTrustViewCommandErrors(t *testing.T) {
+func TestTrustInspectPrettyCommandErrors(t *testing.T) {
 	testCases := []struct {
 		name          string
 		args          []string
@@ -28,12 +30,7 @@ func TestTrustViewCommandErrors(t *testing.T) {
 	}{
 		{
 			name:          "not-enough-args",
-			expectedError: "requires exactly 1 argument",
-		},
-		{
-			name:          "too-many-args",
-			args:          []string{"remote1", "remote2"},
-			expectedError: "requires exactly 1 argument",
+			expectedError: "requires at least 1 argument",
 		},
 		{
 			name:          "sha-reference",
@@ -47,104 +44,115 @@ func TestTrustViewCommandErrors(t *testing.T) {
 		},
 	}
 	for _, tc := range testCases {
-		cmd := newViewCommand(
+		cmd := newInspectCommand(
 			test.NewFakeCli(&fakeClient{}))
 		cmd.SetArgs(tc.args)
 		cmd.SetOutput(ioutil.Discard)
+		cmd.Flags().Set("pretty", "true")
 		testutil.ErrorContains(t, cmd.Execute(), tc.expectedError)
 	}
 }
 
-func TestTrustViewCommandOfflineErrors(t *testing.T) {
+func TestTrustInspectPrettyCommandOfflineErrors(t *testing.T) {
 	cli := test.NewFakeCli(&fakeClient{})
 	cli.SetNotaryClient(getOfflineNotaryRepository)
-	cmd := newViewCommand(cli)
+	cmd := newInspectCommand(cli)
+	cmd.Flags().Set("pretty", "true")
 	cmd.SetArgs([]string{"nonexistent-reg-name.io/image"})
 	cmd.SetOutput(ioutil.Discard)
 	testutil.ErrorContains(t, cmd.Execute(), "No signatures or cannot access nonexistent-reg-name.io/image")
 
 	cli = test.NewFakeCli(&fakeClient{})
 	cli.SetNotaryClient(getOfflineNotaryRepository)
-	cmd = newViewCommand(cli)
+	cmd = newInspectCommand(cli)
+	cmd.Flags().Set("pretty", "true")
 	cmd.SetArgs([]string{"nonexistent-reg-name.io/image:tag"})
 	cmd.SetOutput(ioutil.Discard)
 	testutil.ErrorContains(t, cmd.Execute(), "No signatures or cannot access nonexistent-reg-name.io/image")
 }
 
-func TestTrustViewCommandUninitializedErrors(t *testing.T) {
+func TestTrustInspectPrettyCommandUninitializedErrors(t *testing.T) {
 	cli := test.NewFakeCli(&fakeClient{})
 	cli.SetNotaryClient(getUninitializedNotaryRepository)
-	cmd := newViewCommand(cli)
+	cmd := newInspectCommand(cli)
+	cmd.Flags().Set("pretty", "true")
 	cmd.SetArgs([]string{"reg/unsigned-img"})
 	cmd.SetOutput(ioutil.Discard)
 	testutil.ErrorContains(t, cmd.Execute(), "No signatures or cannot access reg/unsigned-img")
 
 	cli = test.NewFakeCli(&fakeClient{})
 	cli.SetNotaryClient(getUninitializedNotaryRepository)
-	cmd = newViewCommand(cli)
+	cmd = newInspectCommand(cli)
+	cmd.Flags().Set("pretty", "true")
 	cmd.SetArgs([]string{"reg/unsigned-img:tag"})
 	cmd.SetOutput(ioutil.Discard)
 	testutil.ErrorContains(t, cmd.Execute(), "No signatures or cannot access reg/unsigned-img:tag")
 }
 
-func TestTrustViewCommandEmptyNotaryRepoErrors(t *testing.T) {
+func TestTrustInspectPrettyCommandEmptyNotaryRepoErrors(t *testing.T) {
 	cli := test.NewFakeCli(&fakeClient{})
 	cli.SetNotaryClient(getEmptyTargetsNotaryRepository)
-	cmd := newViewCommand(cli)
+	cmd := newInspectCommand(cli)
+	cmd.Flags().Set("pretty", "true")
 	cmd.SetArgs([]string{"reg/img:unsigned-tag"})
 	cmd.SetOutput(ioutil.Discard)
 	assert.NoError(t, cmd.Execute())
 	assert.Contains(t, cli.OutBuffer().String(), "No signatures for reg/img:unsigned-tag")
-	assert.Contains(t, cli.OutBuffer().String(), "Administrative keys for reg/img:")
+	assert.Contains(t, cli.OutBuffer().String(), "Administrative keys for reg/img")
 
 	cli = test.NewFakeCli(&fakeClient{})
 	cli.SetNotaryClient(getEmptyTargetsNotaryRepository)
-	cmd = newViewCommand(cli)
+	cmd = newInspectCommand(cli)
+	cmd.Flags().Set("pretty", "true")
 	cmd.SetArgs([]string{"reg/img"})
 	cmd.SetOutput(ioutil.Discard)
 	assert.NoError(t, cmd.Execute())
 	assert.Contains(t, cli.OutBuffer().String(), "No signatures for reg/img")
-	assert.Contains(t, cli.OutBuffer().String(), "Administrative keys for reg/img:")
+	assert.Contains(t, cli.OutBuffer().String(), "Administrative keys for reg/img")
 }
 
-func TestTrustViewCommandFullRepoWithoutSigners(t *testing.T) {
+func TestTrustInspectPrettyCommandFullRepoWithoutSigners(t *testing.T) {
 	cli := test.NewFakeCli(&fakeClient{})
 	cli.SetNotaryClient(getLoadedWithNoSignersNotaryRepository)
-	cmd := newViewCommand(cli)
+	cmd := newInspectCommand(cli)
+	cmd.Flags().Set("pretty", "true")
 	cmd.SetArgs([]string{"signed-repo"})
 	assert.NoError(t, cmd.Execute())
 
-	golden.Assert(t, cli.OutBuffer().String(), "trust-view-full-repo-no-signers.golden")
+	golden.Assert(t, cli.OutBuffer().String(), "trust-inspect-pretty-full-repo-no-signers.golden")
 }
 
-func TestTrustViewCommandOneTagWithoutSigners(t *testing.T) {
+func TestTrustInspectPrettyCommandOneTagWithoutSigners(t *testing.T) {
 	cli := test.NewFakeCli(&fakeClient{})
 	cli.SetNotaryClient(getLoadedWithNoSignersNotaryRepository)
-	cmd := newViewCommand(cli)
+	cmd := newInspectCommand(cli)
+	cmd.Flags().Set("pretty", "true")
 	cmd.SetArgs([]string{"signed-repo:green"})
 	assert.NoError(t, cmd.Execute())
 
-	golden.Assert(t, cli.OutBuffer().String(), "trust-view-one-tag-no-signers.golden")
+	golden.Assert(t, cli.OutBuffer().String(), "trust-inspect-pretty-one-tag-no-signers.golden")
 }
 
-func TestTrustViewCommandFullRepoWithSigners(t *testing.T) {
+func TestTrustInspectPrettyCommandFullRepoWithSigners(t *testing.T) {
 	cli := test.NewFakeCli(&fakeClient{})
 	cli.SetNotaryClient(getLoadedNotaryRepository)
-	cmd := newViewCommand(cli)
+	cmd := newInspectCommand(cli)
+	cmd.Flags().Set("pretty", "true")
 	cmd.SetArgs([]string{"signed-repo"})
 	assert.NoError(t, cmd.Execute())
 
-	golden.Assert(t, cli.OutBuffer().String(), "trust-view-full-repo-with-signers.golden")
+	golden.Assert(t, cli.OutBuffer().String(), "trust-inspect-pretty-full-repo-with-signers.golden")
 }
 
-func TestTrustViewCommandUnsignedTagInSignedRepo(t *testing.T) {
+func TestTrustInspectPrettyCommandUnsignedTagInSignedRepo(t *testing.T) {
 	cli := test.NewFakeCli(&fakeClient{})
 	cli.SetNotaryClient(getLoadedNotaryRepository)
-	cmd := newViewCommand(cli)
+	cmd := newInspectCommand(cli)
+	cmd.Flags().Set("pretty", "true")
 	cmd.SetArgs([]string{"signed-repo:unsigned"})
 	assert.NoError(t, cmd.Execute())
 
-	golden.Assert(t, cli.OutBuffer().String(), "trust-view-unsigned-tag-with-signers.golden")
+	golden.Assert(t, cli.OutBuffer().String(), "trust-inspect-pretty-unsigned-tag-with-signers.golden")
 }
 
 func TestNotaryRoleToSigner(t *testing.T) {

components/cli/cli/command/trust/inspect_test.go

@@ -18,12 +18,7 @@ func TestTrustInspectCommandErrors(t *testing.T) {
 	}{
 		{
 			name:          "not-enough-args",
-			expectedError: "requires exactly 1 argument",
-		},
-		{
-			name:          "too-many-args",
-			args:          []string{"remote1", "remote2"},
-			expectedError: "requires exactly 1 argument",
+			expectedError: "requires at least 1 argument",
 		},
 		{
 			name:          "sha-reference",
@@ -37,8 +32,9 @@ func TestTrustInspectCommandErrors(t *testing.T) {
 		},
 	}
 	for _, tc := range testCases {
-		cmd := newViewCommand(
+		cmd := newInspectCommand(
 			test.NewFakeCli(&fakeClient{}))
+		cmd.Flags().Set("pretty", "true")
 		cmd.SetArgs(tc.args)
 		cmd.SetOutput(ioutil.Discard)
 		testutil.ErrorContains(t, cmd.Execute(), tc.expectedError)

components/cli/cli/command/trust/key.go

@@ -10,7 +10,7 @@ import (
 func newTrustKeyCommand(dockerCli command.Streams) *cobra.Command {
 	cmd := &cobra.Command{
 		Use:   "key",
-		Short: "Manage keys for signing Docker images (experimental)",
+		Short: "Manage keys for signing Docker images",
 		Args:  cli.NoArgs,
 		RunE:  command.ShowHelp(dockerCli.Err()),
 	}

components/cli/cli/command/trust/signer.go

@@ -10,7 +10,7 @@ import (
 func newTrustSignerCommand(dockerCli command.Cli) *cobra.Command {
 	cmd := &cobra.Command{
 		Use:   "signer",
-		Short: "Manage entities who can sign Docker images (experimental)",
+		Short: "Manage entities who can sign Docker images",
 		Args:  cli.NoArgs,
 		RunE:  command.ShowHelp(dockerCli.Err()),
 	}

components/cli/cli/command/trust/testdata/trust-inspect-pretty-full-repo-no-signers.golden

@@ -1,6 +1,10 @@
+
+Signatures for signed-repo
+
 SIGNED TAG          DIGEST                     SIGNERS
 green               677265656e2d646967657374   (Repo Admin)
 
-Administrative keys for signed-repo:
-Repository Key:	targetsID
-Root Key:	rootID
+Administrative keys for signed-repo
+
+  Repository Key:	targetsID
+  Root Key:	rootID

components/cli/cli/command/trust/testdata/trust-inspect-pretty-full-repo-with-signers.golden

@@ -1,14 +1,18 @@
+
+Signatures for signed-repo
+
 SIGNED TAG          DIGEST                     SIGNERS
 blue                626c75652d646967657374     alice
 green               677265656e2d646967657374   (Repo Admin)
 red                 7265642d646967657374       alice, bob
 
-List of signers and their keys for signed-repo:
+List of signers and their keys for signed-repo
 
 SIGNER              KEYS
 alice               A
 bob                 B
 
-Administrative keys for signed-repo:
-Repository Key:	targetsID
-Root Key:	rootID
+Administrative keys for signed-repo
+
+  Repository Key:	targetsID
+  Root Key:	rootID

components/cli/cli/command/trust/testdata/trust-inspect-pretty-one-tag-no-signers.golden

@@ -0,0 +1,10 @@
+
+Signatures for signed-repo:green
+
+SIGNED TAG          DIGEST                     SIGNERS
+green               677265656e2d646967657374   (Repo Admin)
+
+Administrative keys for signed-repo:green
+
+  Repository Key:	targetsID
+  Root Key:	rootID

components/cli/cli/command/trust/testdata/trust-inspect-pretty-unsigned-tag-with-signers.golden

@@ -0,0 +1,14 @@
+
+No signatures for signed-repo:unsigned
+
+
+List of signers and their keys for signed-repo:unsigned
+
+SIGNER              KEYS
+alice               A
+bob                 B
+
+Administrative keys for signed-repo:unsigned
+
+  Repository Key:	targetsID
+  Root Key:	rootID

components/cli/cli/command/trust/testdata/trust-view-one-tag-no-signers.golden

@@ -1,6 +0,0 @@
-SIGNED TAG          DIGEST                     SIGNERS
-green               677265656e2d646967657374   (Repo Admin)
-
-Administrative keys for signed-repo:
-Repository Key:	targetsID
-Root Key:	rootID

components/cli/cli/command/trust/testdata/trust-view-unsigned-tag-with-signers.golden

@@ -1,13 +0,0 @@
-
-No signatures for signed-repo:unsigned
-
-
-List of signers and their keys for signed-repo:
-
-SIGNER              KEYS
-alice               A
-bob                 B
-
-Administrative keys for signed-repo:
-Repository Key:	targetsID
-Root Key:	rootID

components/cli/cli/command/volume/prune.go

@@ -22,7 +22,7 @@ func NewPruneCommand(dockerCli command.Cli) *cobra.Command {
 
 	cmd := &cobra.Command{
 		Use:   "prune [OPTIONS]",
-		Short: "Remove all unused volumes",
+		Short: "Remove all unused local volumes",
 		Args:  cli.NoArgs,
 		RunE: func(cmd *cobra.Command, args []string) error {
 			spaceReclaimed, output, err := runPrune(dockerCli, options)
@@ -45,7 +45,7 @@ func NewPruneCommand(dockerCli command.Cli) *cobra.Command {
 	return cmd
 }
 
-const warning = `WARNING! This will remove all volumes not used by at least one container.
+const warning = `WARNING! This will remove all local volumes not used by at least one container.
 Are you sure you want to continue?`
 
 func runPrune(dockerCli command.Cli, options pruneOptions) (spaceReclaimed uint64, output string, err error) {

components/cli/cli/command/volume/testdata/volume-prune-no.golden

@@ -1,2 +1,2 @@
-WARNING! This will remove all volumes not used by at least one container.
+WARNING! This will remove all local volumes not used by at least one container.
 Are you sure you want to continue? [y/N] Total reclaimed space: 0B

components/cli/cli/command/volume/testdata/volume-prune-yes.golden

@@ -1,4 +1,4 @@
-WARNING! This will remove all volumes not used by at least one container.
+WARNING! This will remove all local volumes not used by at least one container.
 Are you sure you want to continue? [y/N] Deleted Volumes:
 foo
 bar

components/cli/cli/compose/loader/full-example.yml

@@ -193,7 +193,7 @@ services:
 
     ports:
       - 3000
-      - "3000-3005"
+      - "3001-3005"
       - "8000:8000"
       - "9090-9091:8080-8081"
       - "49100:22"

components/cli/cli/compose/loader/full-struct_test.go

@@ -0,0 +1,391 @@
+package loader
+
+import (
+	"time"
+
+	"github.com/docker/cli/cli/compose/types"
+)
+
+func fullExampleConfig(workingDir, homeDir string) *types.Config {
+	return &types.Config{
+		Version:  "3.6",
+		Services: services(workingDir, homeDir),
+		Networks: networks(),
+		Volumes:  volumes(),
+	}
+}
+
+func services(workingDir, homeDir string) []types.ServiceConfig {
+	return []types.ServiceConfig{
+		{
+			Name: "foo",
+
+			Build: types.BuildConfig{
+				Context:    "./dir",
+				Dockerfile: "Dockerfile",
+				Args:       map[string]*string{"foo": strPtr("bar")},
+				Target:     "foo",
+				Network:    "foo",
+				CacheFrom:  []string{"foo", "bar"},
+				Labels:     map[string]string{"FOO": "BAR"},
+			},
+			CapAdd:        []string{"ALL"},
+			CapDrop:       []string{"NET_ADMIN", "SYS_ADMIN"},
+			CgroupParent:  "m-executor-abcd",
+			Command:       []string{"bundle", "exec", "thin", "-p", "3000"},
+			ContainerName: "my-web-container",
+			DependsOn:     []string{"db", "redis"},
+			Deploy: types.DeployConfig{
+				Mode:     "replicated",
+				Replicas: uint64Ptr(6),
+				Labels:   map[string]string{"FOO": "BAR"},
+				UpdateConfig: &types.UpdateConfig{
+					Parallelism:     uint64Ptr(3),
+					Delay:           time.Duration(10 * time.Second),
+					FailureAction:   "continue",
+					Monitor:         time.Duration(60 * time.Second),
+					MaxFailureRatio: 0.3,
+					Order:           "start-first",
+				},
+				Resources: types.Resources{
+					Limits: &types.Resource{
+						NanoCPUs:    "0.001",
+						MemoryBytes: 50 * 1024 * 1024,
+					},
+					Reservations: &types.Resource{
+						NanoCPUs:    "0.0001",
+						MemoryBytes: 20 * 1024 * 1024,
+						GenericResources: []types.GenericResource{
+							{
+								DiscreteResourceSpec: &types.DiscreteGenericResource{
+									Kind:  "gpu",
+									Value: 2,
+								},
+							},
+							{
+								DiscreteResourceSpec: &types.DiscreteGenericResource{
+									Kind:  "ssd",
+									Value: 1,
+								},
+							},
+						},
+					},
+				},
+				RestartPolicy: &types.RestartPolicy{
+					Condition:   "on-failure",
+					Delay:       durationPtr(5 * time.Second),
+					MaxAttempts: uint64Ptr(3),
+					Window:      durationPtr(2 * time.Minute),
+				},
+				Placement: types.Placement{
+					Constraints: []string{"node=foo"},
+					Preferences: []types.PlacementPreferences{
+						{
+							Spread: "node.labels.az",
+						},
+					},
+				},
+				EndpointMode: "dnsrr",
+			},
+			Devices:    []string{"/dev/ttyUSB0:/dev/ttyUSB0"},
+			DNS:        []string{"8.8.8.8", "9.9.9.9"},
+			DNSSearch:  []string{"dc1.example.com", "dc2.example.com"},
+			DomainName: "foo.com",
+			Entrypoint: []string{"/code/entrypoint.sh", "-p", "3000"},
+			Environment: map[string]*string{
+				"FOO": strPtr("foo_from_env_file"),
+				"BAR": strPtr("bar_from_env_file_2"),
+				"BAZ": strPtr("baz_from_service_def"),
+				"QUX": strPtr("qux_from_environment"),
+			},
+			EnvFile: []string{
+				"./example1.env",
+				"./example2.env",
+			},
+			Expose: []string{"3000", "8000"},
+			ExternalLinks: []string{
+				"redis_1",
+				"project_db_1:mysql",
+				"project_db_1:postgresql",
+			},
+			ExtraHosts: []string{
+				"somehost:162.242.195.82",
+				"otherhost:50.31.209.229",
+			},
+			HealthCheck: &types.HealthCheckConfig{
+				Test:        types.HealthCheckTest([]string{"CMD-SHELL", "echo \"hello world\""}),
+				Interval:    durationPtr(10 * time.Second),
+				Timeout:     durationPtr(1 * time.Second),
+				Retries:     uint64Ptr(5),
+				StartPeriod: durationPtr(15 * time.Second),
+			},
+			Hostname: "foo",
+			Image:    "redis",
+			Ipc:      "host",
+			Labels: map[string]string{
+				"com.example.description": "Accounting webapp",
+				"com.example.number":      "42",
+				"com.example.empty-label": "",
+			},
+			Links: []string{
+				"db",
+				"db:database",
+				"redis",
+			},
+			Logging: &types.LoggingConfig{
+				Driver: "syslog",
+				Options: map[string]string{
+					"syslog-address": "tcp://192.168.0.42:123",
+				},
+			},
+			MacAddress:  "02:42:ac:11:65:43",
+			NetworkMode: "container:0cfeab0f748b9a743dc3da582046357c6ef497631c1a016d28d2bf9b4f899f7b",
+			Networks: map[string]*types.ServiceNetworkConfig{
+				"some-network": {
+					Aliases:     []string{"alias1", "alias3"},
+					Ipv4Address: "",
+					Ipv6Address: "",
+				},
+				"other-network": {
+					Ipv4Address: "172.16.238.10",
+					Ipv6Address: "2001:3984:3989::10",
+				},
+				"other-other-network": nil,
+			},
+			Pid: "host",
+			Ports: []types.ServicePortConfig{
+				//"3000",
+				{
+					Mode:     "ingress",
+					Target:   3000,
+					Protocol: "tcp",
+				},
+				{
+					Mode:     "ingress",
+					Target:   3001,
+					Protocol: "tcp",
+				},
+				{
+					Mode:     "ingress",
+					Target:   3002,
+					Protocol: "tcp",
+				},
+				{
+					Mode:     "ingress",
+					Target:   3003,
+					Protocol: "tcp",
+				},
+				{
+					Mode:     "ingress",
+					Target:   3004,
+					Protocol: "tcp",
+				},
+				{
+					Mode:     "ingress",
+					Target:   3005,
+					Protocol: "tcp",
+				},
+				//"8000:8000",
+				{
+					Mode:      "ingress",
+					Target:    8000,
+					Published: 8000,
+					Protocol:  "tcp",
+				},
+				//"9090-9091:8080-8081",
+				{
+					Mode:      "ingress",
+					Target:    8080,
+					Published: 9090,
+					Protocol:  "tcp",
+				},
+				{
+					Mode:      "ingress",
+					Target:    8081,
+					Published: 9091,
+					Protocol:  "tcp",
+				},
+				//"49100:22",
+				{
+					Mode:      "ingress",
+					Target:    22,
+					Published: 49100,
+					Protocol:  "tcp",
+				},
+				//"127.0.0.1:8001:8001",
+				{
+					Mode:      "ingress",
+					Target:    8001,
+					Published: 8001,
+					Protocol:  "tcp",
+				},
+				//"127.0.0.1:5000-5010:5000-5010",
+				{
+					Mode:      "ingress",
+					Target:    5000,
+					Published: 5000,
+					Protocol:  "tcp",
+				},
+				{
+					Mode:      "ingress",
+					Target:    5001,
+					Published: 5001,
+					Protocol:  "tcp",
+				},
+				{
+					Mode:      "ingress",
+					Target:    5002,
+					Published: 5002,
+					Protocol:  "tcp",
+				},
+				{
+					Mode:      "ingress",
+					Target:    5003,
+					Published: 5003,
+					Protocol:  "tcp",
+				},
+				{
+					Mode:      "ingress",
+					Target:    5004,
+					Published: 5004,
+					Protocol:  "tcp",
+				},
+				{
+					Mode:      "ingress",
+					Target:    5005,
+					Published: 5005,
+					Protocol:  "tcp",
+				},
+				{
+					Mode:      "ingress",
+					Target:    5006,
+					Published: 5006,
+					Protocol:  "tcp",
+				},
+				{
+					Mode:      "ingress",
+					Target:    5007,
+					Published: 5007,
+					Protocol:  "tcp",
+				},
+				{
+					Mode:      "ingress",
+					Target:    5008,
+					Published: 5008,
+					Protocol:  "tcp",
+				},
+				{
+					Mode:      "ingress",
+					Target:    5009,
+					Published: 5009,
+					Protocol:  "tcp",
+				},
+				{
+					Mode:      "ingress",
+					Target:    5010,
+					Published: 5010,
+					Protocol:  "tcp",
+				},
+			},
+			Privileged: true,
+			ReadOnly:   true,
+			Restart:    "always",
+			SecurityOpt: []string{
+				"label=level:s0:c100,c200",
+				"label=type:svirt_apache_t",
+			},
+			StdinOpen:       true,
+			StopSignal:      "SIGUSR1",
+			StopGracePeriod: durationPtr(time.Duration(20 * time.Second)),
+			Tmpfs:           []string{"/run", "/tmp"},
+			Tty:             true,
+			Ulimits: map[string]*types.UlimitsConfig{
+				"nproc": {
+					Single: 65535,
+				},
+				"nofile": {
+					Soft: 20000,
+					Hard: 40000,
+				},
+			},
+			User: "someone",
+			Volumes: []types.ServiceVolumeConfig{
+				{Target: "/var/lib/mysql", Type: "volume"},
+				{Source: "/opt/data", Target: "/var/lib/mysql", Type: "bind"},
+				{Source: workingDir, Target: "/code", Type: "bind"},
+				{Source: workingDir + "/static", Target: "/var/www/html", Type: "bind"},
+				{Source: homeDir + "/configs", Target: "/etc/configs/", Type: "bind", ReadOnly: true},
+				{Source: "datavolume", Target: "/var/lib/mysql", Type: "volume"},
+				{Source: workingDir + "/opt", Target: "/opt", Consistency: "cached", Type: "bind"},
+				{Target: "/opt", Type: "tmpfs", Tmpfs: &types.ServiceVolumeTmpfs{
+					Size: int64(10000),
+				}},
+			},
+			WorkingDir: "/code",
+		},
+	}
+}
+
+func networks() map[string]types.NetworkConfig {
+	return map[string]types.NetworkConfig{
+		"some-network": {},
+
+		"other-network": {
+			Driver: "overlay",
+			DriverOpts: map[string]string{
+				"foo": "bar",
+				"baz": "1",
+			},
+			Ipam: types.IPAMConfig{
+				Driver: "overlay",
+				Config: []*types.IPAMPool{
+					{Subnet: "172.16.238.0/24"},
+					{Subnet: "2001:3984:3989::/64"},
+				},
+			},
+		},
+
+		"external-network": {
+			Name:     "external-network",
+			External: types.External{External: true},
+		},
+
+		"other-external-network": {
+			Name:     "my-cool-network",
+			External: types.External{External: true},
+		},
+	}
+}
+
+func volumes() map[string]types.VolumeConfig {
+	return map[string]types.VolumeConfig{
+		"some-volume": {},
+		"other-volume": {
+			Driver: "flocker",
+			DriverOpts: map[string]string{
+				"foo": "bar",
+				"baz": "1",
+			},
+		},
+		"another-volume": {
+			Name:   "user_specified_name",
+			Driver: "vsphere",
+			DriverOpts: map[string]string{
+				"foo": "bar",
+				"baz": "1",
+			},
+		},
+		"external-volume": {
+			Name:     "external-volume",
+			External: types.External{External: true},
+		},
+		"other-external-volume": {
+			Name:     "my-cool-volume",
+			External: types.External{External: true},
+		},
+		"external-volume3": {
+			Name:     "this-is-volume3",
+			External: types.External{External: true},
+		},
+	}
+}

components/cli/cli/compose/loader/loader.go

@@ -45,27 +45,43 @@ func Load(configDetails types.ConfigDetails) (*types.Config, error) {
 	if len(configDetails.ConfigFiles) < 1 {
 		return nil, errors.Errorf("No files specified")
 	}
-	if len(configDetails.ConfigFiles) > 1 {
-		return nil, errors.Errorf("Multiple files are not yet supported")
+
+	configs := []*types.Config{}
+
+	for _, file := range configDetails.ConfigFiles {
+		configDict := file.Config
+		version := schema.Version(configDict)
+		if configDetails.Version == "" {
+			configDetails.Version = version
+		}
+		if configDetails.Version != version {
+			return nil, errors.Errorf("version mismatched between two composefiles : %v and %v", configDetails.Version, version)
+		}
+
+		if err := validateForbidden(configDict); err != nil {
+			return nil, err
+		}
+
+		var err error
+		configDict, err = interpolateConfig(configDict, configDetails.LookupEnv)
+		if err != nil {
+			return nil, err
+		}
+
+		if err := schema.Validate(configDict, configDetails.Version); err != nil {
+			return nil, err
+		}
+
+		cfg, err := loadSections(configDict, configDetails)
+		if err != nil {
+			return nil, err
+		}
+		cfg.Filename = file.Filename
+
+		configs = append(configs, cfg)
 	}
 
-	configDict := getConfigDict(configDetails)
-	configDetails.Version = schema.Version(configDict)
-
-	if err := validateForbidden(configDict); err != nil {
-		return nil, err
-	}
-
-	var err error
-	configDict, err = interpolateConfig(configDict, configDetails.LookupEnv)
-	if err != nil {
-		return nil, err
-	}
-
-	if err := schema.Validate(configDict, configDetails.Version); err != nil {
-		return nil, err
-	}
-	return loadSections(configDict, configDetails)
+	return merge(configs)
 }
 
 func validateForbidden(configDict map[string]interface{}) error {
@@ -82,7 +98,9 @@ func validateForbidden(configDict map[string]interface{}) error {
 
 func loadSections(config map[string]interface{}, configDetails types.ConfigDetails) (*types.Config, error) {
 	var err error
-	cfg := types.Config{}
+	cfg := types.Config{
+		Version: schema.Version(config),
+	}
 
 	var loaders = []struct {
 		key string
@@ -142,14 +160,16 @@ func getSection(config map[string]interface{}, key string) map[string]interface{
 
 // GetUnsupportedProperties returns the list of any unsupported properties that are
 // used in the Compose files.
-func GetUnsupportedProperties(configDetails types.ConfigDetails) []string {
+func GetUnsupportedProperties(configDicts ...map[string]interface{}) []string {
 	unsupported := map[string]bool{}
 
-	for _, service := range getServices(getConfigDict(configDetails)) {
-		serviceDict := service.(map[string]interface{})
-		for _, property := range types.UnsupportedProperties {
-			if _, isSet := serviceDict[property]; isSet {
-				unsupported[property] = true
+	for _, configDict := range configDicts {
+		for _, service := range getServices(configDict) {
+			serviceDict := service.(map[string]interface{})
+			for _, property := range types.UnsupportedProperties {
+				if _, isSet := serviceDict[property]; isSet {
+					unsupported[property] = true
+				}
 			}
 		}
 	}
@@ -168,8 +188,17 @@ func sortedKeys(set map[string]bool) []string {
 
 // GetDeprecatedProperties returns the list of any deprecated properties that
 // are used in the compose files.
-func GetDeprecatedProperties(configDetails types.ConfigDetails) map[string]string {
-	return getProperties(getServices(getConfigDict(configDetails)), types.DeprecatedProperties)
+func GetDeprecatedProperties(configDicts ...map[string]interface{}) map[string]string {
+	deprecated := map[string]string{}
+
+	for _, configDict := range configDicts {
+		deprecatedProperties := getProperties(getServices(configDict), types.DeprecatedProperties)
+		for key, value := range deprecatedProperties {
+			deprecated[key] = value
+		}
+	}
+
+	return deprecated
 }
 
 func getProperties(services map[string]interface{}, propertyMap map[string]string) map[string]string {
@@ -198,11 +227,6 @@ func (e *ForbiddenPropertiesError) Error() string {
 	return "Configuration contains forbidden properties"
 }
 
-// TODO: resolve multiple files into a single config
-func getConfigDict(configDetails types.ConfigDetails) map[string]interface{} {
-	return configDetails.ConfigFiles[0].Config
-}
-
 func getServices(configDict map[string]interface{}) map[string]interface{} {
 	if services, ok := configDict["services"]; ok {
 		if servicesDict, ok := services.(map[string]interface{}); ok {
@@ -337,7 +361,9 @@ func LoadService(name string, serviceDict map[string]interface{}, workingDir str
 		return nil, err
 	}
 
-	resolveVolumePaths(serviceConfig.Volumes, workingDir, lookupEnv)
+	if err := resolveVolumePaths(serviceConfig.Volumes, workingDir, lookupEnv); err != nil {
+		return nil, err
+	}
 	return serviceConfig, nil
 }
 
@@ -376,12 +402,16 @@ func resolveEnvironment(serviceConfig *types.ServiceConfig, workingDir string, l
 	return nil
 }
 
-func resolveVolumePaths(volumes []types.ServiceVolumeConfig, workingDir string, lookupEnv template.Mapping) {
+func resolveVolumePaths(volumes []types.ServiceVolumeConfig, workingDir string, lookupEnv template.Mapping) error {
 	for i, volume := range volumes {
 		if volume.Type != "bind" {
 			continue
 		}
 
+		if volume.Source == "" {
+			return errors.New(`invalid mount config for type "bind": field Source must not be empty`)
+		}
+
 		filePath := expandUser(volume.Source, lookupEnv)
 		// Check for a Unix absolute path first, to handle a Windows client
 		// with a Unix daemon. This handles a Windows client connecting to a
@@ -394,6 +424,7 @@ func resolveVolumePaths(volumes []types.ServiceVolumeConfig, workingDir string,
 		volume.Source = filePath
 		volumes[i] = volume
 	}
+	return nil
 }
 
 // TODO: make this more robust

components/cli/cli/compose/loader/loader_test.go

@@ -119,6 +119,7 @@ func strPtr(val string) *string {
 }
 
 var sampleConfig = types.Config{
+	Version: "3.0",
 	Services: []types.ServiceConfig{
 		{
 			Name:        "foo",
@@ -174,6 +175,7 @@ func TestParseYAML(t *testing.T) {
 func TestLoad(t *testing.T) {
 	actual, err := Load(buildConfigDetails(sampleDict, nil))
 	require.NoError(t, err)
+	assert.Equal(t, sampleConfig.Version, actual.Version)
 	assert.Equal(t, serviceSort(sampleConfig.Services), serviceSort(actual.Services))
 	assert.Equal(t, sampleConfig.Networks, actual.Networks)
 	assert.Equal(t, sampleConfig.Volumes, actual.Volumes)
@@ -572,6 +574,8 @@ networks:
 	config, err := Load(buildConfigDetails(dict, env))
 	require.NoError(t, err)
 	expected := &types.Config{
+		Filename: "filename.yml",
+		Version:  "3.4",
 		Services: []types.ServiceConfig{
 			{
 				Name: "web",
@@ -670,7 +674,7 @@ services:
 	_, err = Load(configDetails)
 	require.NoError(t, err)
 
-	unsupported := GetUnsupportedProperties(configDetails)
+	unsupported := GetUnsupportedProperties(dict)
 	assert.Equal(t, []string{"build", "links", "pid"}, unsupported)
 }
 
@@ -713,7 +717,7 @@ services:
 	_, err = Load(configDetails)
 	require.NoError(t, err)
 
-	deprecated := GetDeprecatedProperties(configDetails)
+	deprecated := GetDeprecatedProperties(dict)
 	assert.Len(t, deprecated, 2)
 	assert.Contains(t, deprecated, "container_name")
 	assert.Contains(t, deprecated, "expose")
@@ -841,386 +845,11 @@ func TestFullExample(t *testing.T) {
 	workingDir, err := os.Getwd()
 	require.NoError(t, err)
 
-	stopGracePeriod := time.Duration(20 * time.Second)
+	expectedConfig := fullExampleConfig(workingDir, homeDir)
 
-	expectedServiceConfig := types.ServiceConfig{
-		Name: "foo",
-
-		Build: types.BuildConfig{
-			Context:    "./dir",
-			Dockerfile: "Dockerfile",
-			Args:       map[string]*string{"foo": strPtr("bar")},
-			Target:     "foo",
-			Network:    "foo",
-			CacheFrom:  []string{"foo", "bar"},
-			Labels:     map[string]string{"FOO": "BAR"},
-		},
-		CapAdd:        []string{"ALL"},
-		CapDrop:       []string{"NET_ADMIN", "SYS_ADMIN"},
-		CgroupParent:  "m-executor-abcd",
-		Command:       []string{"bundle", "exec", "thin", "-p", "3000"},
-		ContainerName: "my-web-container",
-		DependsOn:     []string{"db", "redis"},
-		Deploy: types.DeployConfig{
-			Mode:     "replicated",
-			Replicas: uint64Ptr(6),
-			Labels:   map[string]string{"FOO": "BAR"},
-			UpdateConfig: &types.UpdateConfig{
-				Parallelism:     uint64Ptr(3),
-				Delay:           time.Duration(10 * time.Second),
-				FailureAction:   "continue",
-				Monitor:         time.Duration(60 * time.Second),
-				MaxFailureRatio: 0.3,
-				Order:           "start-first",
-			},
-			Resources: types.Resources{
-				Limits: &types.Resource{
-					NanoCPUs:    "0.001",
-					MemoryBytes: 50 * 1024 * 1024,
-				},
-				Reservations: &types.Resource{
-					NanoCPUs:    "0.0001",
-					MemoryBytes: 20 * 1024 * 1024,
-					GenericResources: []types.GenericResource{
-						{
-							DiscreteResourceSpec: &types.DiscreteGenericResource{
-								Kind:  "gpu",
-								Value: 2,
-							},
-						},
-						{
-							DiscreteResourceSpec: &types.DiscreteGenericResource{
-								Kind:  "ssd",
-								Value: 1,
-							},
-						},
-					},
-				},
-			},
-			RestartPolicy: &types.RestartPolicy{
-				Condition:   "on-failure",
-				Delay:       durationPtr(5 * time.Second),
-				MaxAttempts: uint64Ptr(3),
-				Window:      durationPtr(2 * time.Minute),
-			},
-			Placement: types.Placement{
-				Constraints: []string{"node=foo"},
-				Preferences: []types.PlacementPreferences{
-					{
-						Spread: "node.labels.az",
-					},
-				},
-			},
-			EndpointMode: "dnsrr",
-		},
-		Devices:    []string{"/dev/ttyUSB0:/dev/ttyUSB0"},
-		DNS:        []string{"8.8.8.8", "9.9.9.9"},
-		DNSSearch:  []string{"dc1.example.com", "dc2.example.com"},
-		DomainName: "foo.com",
-		Entrypoint: []string{"/code/entrypoint.sh", "-p", "3000"},
-		Environment: map[string]*string{
-			"FOO": strPtr("foo_from_env_file"),
-			"BAR": strPtr("bar_from_env_file_2"),
-			"BAZ": strPtr("baz_from_service_def"),
-			"QUX": strPtr("qux_from_environment"),
-		},
-		EnvFile: []string{
-			"./example1.env",
-			"./example2.env",
-		},
-		Expose: []string{"3000", "8000"},
-		ExternalLinks: []string{
-			"redis_1",
-			"project_db_1:mysql",
-			"project_db_1:postgresql",
-		},
-		ExtraHosts: []string{
-			"somehost:162.242.195.82",
-			"otherhost:50.31.209.229",
-		},
-		HealthCheck: &types.HealthCheckConfig{
-			Test:        types.HealthCheckTest([]string{"CMD-SHELL", "echo \"hello world\""}),
-			Interval:    durationPtr(10 * time.Second),
-			Timeout:     durationPtr(1 * time.Second),
-			Retries:     uint64Ptr(5),
-			StartPeriod: durationPtr(15 * time.Second),
-		},
-		Hostname: "foo",
-		Image:    "redis",
-		Ipc:      "host",
-		Labels: map[string]string{
-			"com.example.description": "Accounting webapp",
-			"com.example.number":      "42",
-			"com.example.empty-label": "",
-		},
-		Links: []string{
-			"db",
-			"db:database",
-			"redis",
-		},
-		Logging: &types.LoggingConfig{
-			Driver: "syslog",
-			Options: map[string]string{
-				"syslog-address": "tcp://192.168.0.42:123",
-			},
-		},
-		MacAddress:  "02:42:ac:11:65:43",
-		NetworkMode: "container:0cfeab0f748b9a743dc3da582046357c6ef497631c1a016d28d2bf9b4f899f7b",
-		Networks: map[string]*types.ServiceNetworkConfig{
-			"some-network": {
-				Aliases:     []string{"alias1", "alias3"},
-				Ipv4Address: "",
-				Ipv6Address: "",
-			},
-			"other-network": {
-				Ipv4Address: "172.16.238.10",
-				Ipv6Address: "2001:3984:3989::10",
-			},
-			"other-other-network": nil,
-		},
-		Pid: "host",
-		Ports: []types.ServicePortConfig{
-			//"3000",
-			{
-				Mode:     "ingress",
-				Target:   3000,
-				Protocol: "tcp",
-			},
-			//"3000-3005",
-			{
-				Mode:     "ingress",
-				Target:   3000,
-				Protocol: "tcp",
-			},
-			{
-				Mode:     "ingress",
-				Target:   3001,
-				Protocol: "tcp",
-			},
-			{
-				Mode:     "ingress",
-				Target:   3002,
-				Protocol: "tcp",
-			},
-			{
-				Mode:     "ingress",
-				Target:   3003,
-				Protocol: "tcp",
-			},
-			{
-				Mode:     "ingress",
-				Target:   3004,
-				Protocol: "tcp",
-			},
-			{
-				Mode:     "ingress",
-				Target:   3005,
-				Protocol: "tcp",
-			},
-			//"8000:8000",
-			{
-				Mode:      "ingress",
-				Target:    8000,
-				Published: 8000,
-				Protocol:  "tcp",
-			},
-			//"9090-9091:8080-8081",
-			{
-				Mode:      "ingress",
-				Target:    8080,
-				Published: 9090,
-				Protocol:  "tcp",
-			},
-			{
-				Mode:      "ingress",
-				Target:    8081,
-				Published: 9091,
-				Protocol:  "tcp",
-			},
-			//"49100:22",
-			{
-				Mode:      "ingress",
-				Target:    22,
-				Published: 49100,
-				Protocol:  "tcp",
-			},
-			//"127.0.0.1:8001:8001",
-			{
-				Mode:      "ingress",
-				Target:    8001,
-				Published: 8001,
-				Protocol:  "tcp",
-			},
-			//"127.0.0.1:5000-5010:5000-5010",
-			{
-				Mode:      "ingress",
-				Target:    5000,
-				Published: 5000,
-				Protocol:  "tcp",
-			},
-			{
-				Mode:      "ingress",
-				Target:    5001,
-				Published: 5001,
-				Protocol:  "tcp",
-			},
-			{
-				Mode:      "ingress",
-				Target:    5002,
-				Published: 5002,
-				Protocol:  "tcp",
-			},
-			{
-				Mode:      "ingress",
-				Target:    5003,
-				Published: 5003,
-				Protocol:  "tcp",
-			},
-			{
-				Mode:      "ingress",
-				Target:    5004,
-				Published: 5004,
-				Protocol:  "tcp",
-			},
-			{
-				Mode:      "ingress",
-				Target:    5005,
-				Published: 5005,
-				Protocol:  "tcp",
-			},
-			{
-				Mode:      "ingress",
-				Target:    5006,
-				Published: 5006,
-				Protocol:  "tcp",
-			},
-			{
-				Mode:      "ingress",
-				Target:    5007,
-				Published: 5007,
-				Protocol:  "tcp",
-			},
-			{
-				Mode:      "ingress",
-				Target:    5008,
-				Published: 5008,
-				Protocol:  "tcp",
-			},
-			{
-				Mode:      "ingress",
-				Target:    5009,
-				Published: 5009,
-				Protocol:  "tcp",
-			},
-			{
-				Mode:      "ingress",
-				Target:    5010,
-				Published: 5010,
-				Protocol:  "tcp",
-			},
-		},
-		Privileged: true,
-		ReadOnly:   true,
-		Restart:    "always",
-		SecurityOpt: []string{
-			"label=level:s0:c100,c200",
-			"label=type:svirt_apache_t",
-		},
-		StdinOpen:       true,
-		StopSignal:      "SIGUSR1",
-		StopGracePeriod: &stopGracePeriod,
-		Tmpfs:           []string{"/run", "/tmp"},
-		Tty:             true,
-		Ulimits: map[string]*types.UlimitsConfig{
-			"nproc": {
-				Single: 65535,
-			},
-			"nofile": {
-				Soft: 20000,
-				Hard: 40000,
-			},
-		},
-		User: "someone",
-		Volumes: []types.ServiceVolumeConfig{
-			{Target: "/var/lib/mysql", Type: "volume"},
-			{Source: "/opt/data", Target: "/var/lib/mysql", Type: "bind"},
-			{Source: workingDir, Target: "/code", Type: "bind"},
-			{Source: workingDir + "/static", Target: "/var/www/html", Type: "bind"},
-			{Source: homeDir + "/configs", Target: "/etc/configs/", Type: "bind", ReadOnly: true},
-			{Source: "datavolume", Target: "/var/lib/mysql", Type: "volume"},
-			{Source: workingDir + "/opt", Target: "/opt", Consistency: "cached", Type: "bind"},
-			{Target: "/opt", Type: "tmpfs", Tmpfs: &types.ServiceVolumeTmpfs{
-				Size: int64(10000),
-			}},
-		},
-		WorkingDir: "/code",
-	}
-
-	assert.Equal(t, []types.ServiceConfig{expectedServiceConfig}, config.Services)
-
-	expectedNetworkConfig := map[string]types.NetworkConfig{
-		"some-network": {},
-
-		"other-network": {
-			Driver: "overlay",
-			DriverOpts: map[string]string{
-				"foo": "bar",
-				"baz": "1",
-			},
-			Ipam: types.IPAMConfig{
-				Driver: "overlay",
-				Config: []*types.IPAMPool{
-					{Subnet: "172.16.238.0/24"},
-					{Subnet: "2001:3984:3989::/64"},
-				},
-			},
-		},
-
-		"external-network": {
-			Name:     "external-network",
-			External: types.External{External: true},
-		},
-
-		"other-external-network": {
-			Name:     "my-cool-network",
-			External: types.External{External: true},
-		},
-	}
-
-	assert.Equal(t, expectedNetworkConfig, config.Networks)
-
-	expectedVolumeConfig := map[string]types.VolumeConfig{
-		"some-volume": {},
-		"other-volume": {
-			Driver: "flocker",
-			DriverOpts: map[string]string{
-				"foo": "bar",
-				"baz": "1",
-			},
-		},
-		"another-volume": {
-			Name:   "user_specified_name",
-			Driver: "vsphere",
-			DriverOpts: map[string]string{
-				"foo": "bar",
-				"baz": "1",
-			},
-		},
-		"external-volume": {
-			Name:     "external-volume",
-			External: types.External{External: true},
-		},
-		"other-external-volume": {
-			Name:     "my-cool-volume",
-			External: types.External{External: true},
-		},
-		"external-volume3": {
-			Name:     "this-is-volume3",
-			External: types.External{External: true},
-		},
-	}
-
-	assert.Equal(t, expectedVolumeConfig, config.Volumes)
+	assert.Equal(t, expectedConfig.Services, config.Services)
+	assert.Equal(t, expectedConfig.Networks, config.Networks)
+	assert.Equal(t, expectedConfig.Volumes, config.Volumes)
 }
 
 func TestLoadTmpfsVolume(t *testing.T) {
@@ -1266,6 +895,46 @@ services:
 	assert.Contains(t, err.Error(), "services.tmpfs.volumes.0 Additional property tmpfs is not allowed")
 }
 
+func TestLoadBindMountSourceMustNotBeEmpty(t *testing.T) {
+	_, err := loadYAML(`
+version: "3.5"
+services:
+  tmpfs:
+    image: nginx:latest
+    volumes:
+      - type: bind
+        target: /app
+`)
+	require.EqualError(t, err, `invalid mount config for type "bind": field Source must not be empty`)
+}
+
+func TestLoadBindMountWithSource(t *testing.T) {
+	config, err := loadYAML(`
+version: "3.5"
+services:
+  bind:
+    image: nginx:latest
+    volumes:
+      - type: bind
+        target: /app
+        source: "."
+`)
+	require.NoError(t, err)
+
+	workingDir, err := os.Getwd()
+	require.NoError(t, err)
+
+	expected := types.ServiceVolumeConfig{
+		Type:   "bind",
+		Source: workingDir,
+		Target: "/app",
+	}
+
+	require.Len(t, config.Services, 1)
+	assert.Len(t, config.Services[0].Volumes, 1)
+	assert.Equal(t, expected, config.Services[0].Volumes[0])
+}
+
 func TestLoadTmpfsVolumeSizeCanBeZero(t *testing.T) {
 	config, err := loadYAML(`
 version: "3.6"

components/cli/cli/compose/loader/merge.go

@@ -0,0 +1,233 @@
+package loader
+
+import (
+	"reflect"
+	"sort"
+
+	"github.com/docker/cli/cli/compose/types"
+	"github.com/imdario/mergo"
+	"github.com/pkg/errors"
+)
+
+type specials struct {
+	m map[reflect.Type]func(dst, src reflect.Value) error
+}
+
+func (s *specials) Transformer(t reflect.Type) func(dst, src reflect.Value) error {
+	if fn, ok := s.m[t]; ok {
+		return fn
+	}
+	return nil
+}
+
+func merge(configs []*types.Config) (*types.Config, error) {
+	base := configs[0]
+	for _, override := range configs[1:] {
+		var err error
+		base.Services, err = mergeServices(base.Services, override.Services)
+		if err != nil {
+			return base, errors.Wrapf(err, "cannot merge services from %s", override.Filename)
+		}
+		base.Volumes, err = mergeVolumes(base.Volumes, override.Volumes)
+		if err != nil {
+			return base, errors.Wrapf(err, "cannot merge volumes from %s", override.Filename)
+		}
+		base.Networks, err = mergeNetworks(base.Networks, override.Networks)
+		if err != nil {
+			return base, errors.Wrapf(err, "cannot merge networks from %s", override.Filename)
+		}
+		base.Secrets, err = mergeSecrets(base.Secrets, override.Secrets)
+		if err != nil {
+			return base, errors.Wrapf(err, "cannot merge secrets from %s", override.Filename)
+		}
+		base.Configs, err = mergeConfigs(base.Configs, override.Configs)
+		if err != nil {
+			return base, errors.Wrapf(err, "cannot merge configs from %s", override.Filename)
+		}
+	}
+	return base, nil
+}
+
+func mergeServices(base, override []types.ServiceConfig) ([]types.ServiceConfig, error) {
+	baseServices := mapByName(base)
+	overrideServices := mapByName(override)
+	specials := &specials{
+		m: map[reflect.Type]func(dst, src reflect.Value) error{
+			reflect.TypeOf(&types.LoggingConfig{}):           safelyMerge(mergeLoggingConfig),
+			reflect.TypeOf([]types.ServicePortConfig{}):      mergeSlice(toServicePortConfigsMap, toServicePortConfigsSlice),
+			reflect.TypeOf([]types.ServiceSecretConfig{}):    mergeSlice(toServiceSecretConfigsMap, toServiceSecretConfigsSlice),
+			reflect.TypeOf([]types.ServiceConfigObjConfig{}): mergeSlice(toServiceConfigObjConfigsMap, toSServiceConfigObjConfigsSlice),
+		},
+	}
+	for name, overrideService := range overrideServices {
+		if baseService, ok := baseServices[name]; ok {
+			if err := mergo.Merge(&baseService, &overrideService, mergo.WithOverride, mergo.WithTransformers(specials)); err != nil {
+				return base, errors.Wrapf(err, "cannot merge service %s", name)
+			}
+			baseServices[name] = baseService
+			continue
+		}
+		baseServices[name] = overrideService
+	}
+	services := []types.ServiceConfig{}
+	for _, baseService := range baseServices {
+		services = append(services, baseService)
+	}
+	sort.Slice(services, func(i, j int) bool { return services[i].Name < services[j].Name })
+	return services, nil
+}
+
+func toServiceSecretConfigsMap(s interface{}) (map[interface{}]interface{}, error) {
+	secrets, ok := s.([]types.ServiceSecretConfig)
+	if !ok {
+		return nil, errors.Errorf("not a serviceSecretConfig: %v", s)
+	}
+	m := map[interface{}]interface{}{}
+	for _, secret := range secrets {
+		m[secret.Source] = secret
+	}
+	return m, nil
+}
+
+func toServiceConfigObjConfigsMap(s interface{}) (map[interface{}]interface{}, error) {
+	secrets, ok := s.([]types.ServiceConfigObjConfig)
+	if !ok {
+		return nil, errors.Errorf("not a serviceSecretConfig: %v", s)
+	}
+	m := map[interface{}]interface{}{}
+	for _, secret := range secrets {
+		m[secret.Source] = secret
+	}
+	return m, nil
+}
+
+func toServicePortConfigsMap(s interface{}) (map[interface{}]interface{}, error) {
+	ports, ok := s.([]types.ServicePortConfig)
+	if !ok {
+		return nil, errors.Errorf("not a servicePortConfig slice: %v", s)
+	}
+	m := map[interface{}]interface{}{}
+	for _, p := range ports {
+		m[p.Published] = p
+	}
+	return m, nil
+}
+
+func toServiceSecretConfigsSlice(dst reflect.Value, m map[interface{}]interface{}) error {
+	s := []types.ServiceSecretConfig{}
+	for _, v := range m {
+		s = append(s, v.(types.ServiceSecretConfig))
+	}
+	sort.Slice(s, func(i, j int) bool { return s[i].Source < s[j].Source })
+	dst.Set(reflect.ValueOf(s))
+	return nil
+}
+
+func toSServiceConfigObjConfigsSlice(dst reflect.Value, m map[interface{}]interface{}) error {
+	s := []types.ServiceConfigObjConfig{}
+	for _, v := range m {
+		s = append(s, v.(types.ServiceConfigObjConfig))
+	}
+	sort.Slice(s, func(i, j int) bool { return s[i].Source < s[j].Source })
+	dst.Set(reflect.ValueOf(s))
+	return nil
+}
+
+func toServicePortConfigsSlice(dst reflect.Value, m map[interface{}]interface{}) error {
+	s := []types.ServicePortConfig{}
+	for _, v := range m {
+		s = append(s, v.(types.ServicePortConfig))
+	}
+	sort.Slice(s, func(i, j int) bool { return s[i].Published < s[j].Published })
+	dst.Set(reflect.ValueOf(s))
+	return nil
+}
+
+type tomapFn func(s interface{}) (map[interface{}]interface{}, error)
+type writeValueFromMapFn func(reflect.Value, map[interface{}]interface{}) error
+
+func safelyMerge(mergeFn func(dst, src reflect.Value) error) func(dst, src reflect.Value) error {
+	return func(dst, src reflect.Value) error {
+		if src.IsNil() {
+			return nil
+		}
+		if dst.IsNil() {
+			dst.Set(src)
+			return nil
+		}
+		return mergeFn(dst, src)
+	}
+}
+
+func mergeSlice(tomap tomapFn, writeValue writeValueFromMapFn) func(dst, src reflect.Value) error {
+	return func(dst, src reflect.Value) error {
+		dstMap, err := sliceToMap(tomap, dst)
+		if err != nil {
+			return err
+		}
+		srcMap, err := sliceToMap(tomap, src)
+		if err != nil {
+			return err
+		}
+		if err := mergo.Map(&dstMap, srcMap, mergo.WithOverride); err != nil {
+			return err
+		}
+		return writeValue(dst, dstMap)
+	}
+}
+
+func sliceToMap(tomap tomapFn, v reflect.Value) (map[interface{}]interface{}, error) {
+	// check if valid
+	if !v.IsValid() {
+		return nil, errors.Errorf("invalid value : %+v", v)
+	}
+	return tomap(v.Interface())
+}
+
+func mergeLoggingConfig(dst, src reflect.Value) error {
+	// Same driver, merging options
+	if getLoggingDriver(dst.Elem()) == getLoggingDriver(src.Elem()) ||
+		getLoggingDriver(dst.Elem()) == "" || getLoggingDriver(src.Elem()) == "" {
+		if getLoggingDriver(dst.Elem()) == "" {
+			dst.Elem().FieldByName("Driver").SetString(getLoggingDriver(src.Elem()))
+		}
+		dstOptions := dst.Elem().FieldByName("Options").Interface().(map[string]string)
+		srcOptions := src.Elem().FieldByName("Options").Interface().(map[string]string)
+		return mergo.Merge(&dstOptions, srcOptions, mergo.WithOverride)
+	}
+	// Different driver, override with src
+	dst.Set(src)
+	return nil
+}
+
+func getLoggingDriver(v reflect.Value) string {
+	return v.FieldByName("Driver").String()
+}
+
+func mapByName(services []types.ServiceConfig) map[string]types.ServiceConfig {
+	m := map[string]types.ServiceConfig{}
+	for _, service := range services {
+		m[service.Name] = service
+	}
+	return m
+}
+
+func mergeVolumes(base, override map[string]types.VolumeConfig) (map[string]types.VolumeConfig, error) {
+	err := mergo.Map(&base, &override)
+	return base, err
+}
+
+func mergeNetworks(base, override map[string]types.NetworkConfig) (map[string]types.NetworkConfig, error) {
+	err := mergo.Map(&base, &override)
+	return base, err
+}
+
+func mergeSecrets(base, override map[string]types.SecretConfig) (map[string]types.SecretConfig, error) {
+	err := mergo.Map(&base, &override)
+	return base, err
+}
+
+func mergeConfigs(base, override map[string]types.ConfigObjConfig) (map[string]types.ConfigObjConfig, error) {
+	err := mergo.Map(&base, &override)
+	return base, err
+}

components/cli/cli/compose/loader/merge_test.go

@@ -0,0 +1,945 @@
+package loader
+
+import (
+	"testing"
+
+	"github.com/docker/cli/cli/compose/types"
+	"github.com/stretchr/testify/require"
+)
+
+func TestLoadTwoDifferentVersion(t *testing.T) {
+	configDetails := types.ConfigDetails{
+		ConfigFiles: []types.ConfigFile{
+			{Filename: "base.yml", Config: map[string]interface{}{
+				"version": "3.1",
+			}},
+			{Filename: "override.yml", Config: map[string]interface{}{
+				"version": "3.4",
+			}},
+		},
+	}
+	_, err := Load(configDetails)
+	require.EqualError(t, err, "version mismatched between two composefiles : 3.1 and 3.4")
+}
+
+func TestLoadLogging(t *testing.T) {
+	loggingCases := []struct {
+		name            string
+		loggingBase     map[string]interface{}
+		loggingOverride map[string]interface{}
+		expected        *types.LoggingConfig
+	}{
+		{
+			name: "no_override_driver",
+			loggingBase: map[string]interface{}{
+				"logging": map[string]interface{}{
+					"driver": "json-file",
+					"options": map[string]interface{}{
+						"frequency": "2000",
+						"timeout":   "23",
+					},
+				},
+			},
+			loggingOverride: map[string]interface{}{
+				"logging": map[string]interface{}{
+					"options": map[string]interface{}{
+						"timeout":      "360",
+						"pretty-print": "on",
+					},
+				},
+			},
+			expected: &types.LoggingConfig{
+				Driver: "json-file",
+				Options: map[string]string{
+					"frequency":    "2000",
+					"timeout":      "360",
+					"pretty-print": "on",
+				},
+			},
+		},
+		{
+			name: "override_driver",
+			loggingBase: map[string]interface{}{
+				"logging": map[string]interface{}{
+					"driver": "json-file",
+					"options": map[string]interface{}{
+						"frequency": "2000",
+						"timeout":   "23",
+					},
+				},
+			},
+			loggingOverride: map[string]interface{}{
+				"logging": map[string]interface{}{
+					"driver": "syslog",
+					"options": map[string]interface{}{
+						"timeout":      "360",
+						"pretty-print": "on",
+					},
+				},
+			},
+			expected: &types.LoggingConfig{
+				Driver: "syslog",
+				Options: map[string]string{
+					"timeout":      "360",
+					"pretty-print": "on",
+				},
+			},
+		},
+		{
+			name: "no_base_driver",
+			loggingBase: map[string]interface{}{
+				"logging": map[string]interface{}{
+					"options": map[string]interface{}{
+						"frequency": "2000",
+						"timeout":   "23",
+					},
+				},
+			},
+			loggingOverride: map[string]interface{}{
+				"logging": map[string]interface{}{
+					"driver": "json-file",
+					"options": map[string]interface{}{
+						"timeout":      "360",
+						"pretty-print": "on",
+					},
+				},
+			},
+			expected: &types.LoggingConfig{
+				Driver: "json-file",
+				Options: map[string]string{
+					"frequency":    "2000",
+					"timeout":      "360",
+					"pretty-print": "on",
+				},
+			},
+		},
+		{
+			name: "no_driver",
+			loggingBase: map[string]interface{}{
+				"logging": map[string]interface{}{
+					"options": map[string]interface{}{
+						"frequency": "2000",
+						"timeout":   "23",
+					},
+				},
+			},
+			loggingOverride: map[string]interface{}{
+				"logging": map[string]interface{}{
+					"options": map[string]interface{}{
+						"timeout":      "360",
+						"pretty-print": "on",
+					},
+				},
+			},
+			expected: &types.LoggingConfig{
+				Options: map[string]string{
+					"frequency":    "2000",
+					"timeout":      "360",
+					"pretty-print": "on",
+				},
+			},
+		},
+		{
+			name: "no_override_options",
+			loggingBase: map[string]interface{}{
+				"logging": map[string]interface{}{
+					"driver": "json-file",
+					"options": map[string]interface{}{
+						"frequency": "2000",
+						"timeout":   "23",
+					},
+				},
+			},
+			loggingOverride: map[string]interface{}{
+				"logging": map[string]interface{}{
+					"driver": "syslog",
+				},
+			},
+			expected: &types.LoggingConfig{
+				Driver: "syslog",
+			},
+		},
+		{
+			name:        "no_base",
+			loggingBase: map[string]interface{}{},
+			loggingOverride: map[string]interface{}{
+				"logging": map[string]interface{}{
+					"driver": "json-file",
+					"options": map[string]interface{}{
+						"frequency": "2000",
+					},
+				},
+			},
+			expected: &types.LoggingConfig{
+				Driver: "json-file",
+				Options: map[string]string{
+					"frequency": "2000",
+				},
+			},
+		},
+	}
+
+	for _, tc := range loggingCases {
+		t.Run(tc.name, func(t *testing.T) {
+			configDetails := types.ConfigDetails{
+				ConfigFiles: []types.ConfigFile{
+					{
+						Filename: "base.yml",
+						Config: map[string]interface{}{
+							"version": "3.4",
+							"services": map[string]interface{}{
+								"foo": tc.loggingBase,
+							},
+						},
+					},
+					{
+						Filename: "override.yml",
+						Config: map[string]interface{}{
+							"version": "3.4",
+							"services": map[string]interface{}{
+								"foo": tc.loggingOverride,
+							},
+						},
+					},
+				},
+			}
+			config, err := Load(configDetails)
+			require.NoError(t, err)
+			require.Equal(t, &types.Config{
+				Filename: "base.yml",
+				Version:  "3.4",
+				Services: []types.ServiceConfig{
+					{
+						Name:        "foo",
+						Logging:     tc.expected,
+						Environment: types.MappingWithEquals{},
+					},
+				},
+				Networks: map[string]types.NetworkConfig{},
+				Volumes:  map[string]types.VolumeConfig{},
+				Secrets:  map[string]types.SecretConfig{},
+				Configs:  map[string]types.ConfigObjConfig{},
+			}, config)
+		})
+	}
+}
+
+func TestLoadMultipleServicePorts(t *testing.T) {
+	portsCases := []struct {
+		name         string
+		portBase     map[string]interface{}
+		portOverride map[string]interface{}
+		expected     []types.ServicePortConfig
+	}{
+		{
+			name: "no_override",
+			portBase: map[string]interface{}{
+				"ports": []interface{}{
+					"8080:80",
+				},
+			},
+			portOverride: map[string]interface{}{},
+			expected: []types.ServicePortConfig{
+				{
+					Mode:      "ingress",
+					Published: 8080,
+					Target:    80,
+					Protocol:  "tcp",
+				},
+			},
+		},
+		{
+			name: "override_different_published",
+			portBase: map[string]interface{}{
+				"ports": []interface{}{
+					"8080:80",
+				},
+			},
+			portOverride: map[string]interface{}{
+				"ports": []interface{}{
+					"8081:80",
+				},
+			},
+			expected: []types.ServicePortConfig{
+				{
+					Mode:      "ingress",
+					Published: 8080,
+					Target:    80,
+					Protocol:  "tcp",
+				},
+				{
+					Mode:      "ingress",
+					Published: 8081,
+					Target:    80,
+					Protocol:  "tcp",
+				},
+			},
+		},
+		{
+			name: "override_same_published",
+			portBase: map[string]interface{}{
+				"ports": []interface{}{
+					"8080:80",
+				},
+			},
+			portOverride: map[string]interface{}{
+				"ports": []interface{}{
+					"8080:81",
+				},
+			},
+			expected: []types.ServicePortConfig{
+				{
+					Mode:      "ingress",
+					Published: 8080,
+					Target:    81,
+					Protocol:  "tcp",
+				},
+			},
+		},
+	}
+
+	for _, tc := range portsCases {
+		t.Run(tc.name, func(t *testing.T) {
+			configDetails := types.ConfigDetails{
+				ConfigFiles: []types.ConfigFile{
+					{
+						Filename: "base.yml",
+						Config: map[string]interface{}{
+							"version": "3.4",
+							"services": map[string]interface{}{
+								"foo": tc.portBase,
+							},
+						},
+					},
+					{
+						Filename: "override.yml",
+						Config: map[string]interface{}{
+							"version": "3.4",
+							"services": map[string]interface{}{
+								"foo": tc.portOverride,
+							},
+						},
+					},
+				},
+			}
+			config, err := Load(configDetails)
+			require.NoError(t, err)
+			require.Equal(t, &types.Config{
+				Filename: "base.yml",
+				Version:  "3.4",
+				Services: []types.ServiceConfig{
+					{
+						Name:        "foo",
+						Ports:       tc.expected,
+						Environment: types.MappingWithEquals{},
+					},
+				},
+				Networks: map[string]types.NetworkConfig{},
+				Volumes:  map[string]types.VolumeConfig{},
+				Secrets:  map[string]types.SecretConfig{},
+				Configs:  map[string]types.ConfigObjConfig{},
+			}, config)
+		})
+	}
+}
+
+func TestLoadMultipleSecretsConfig(t *testing.T) {
+	portsCases := []struct {
+		name           string
+		secretBase     map[string]interface{}
+		secretOverride map[string]interface{}
+		expected       []types.ServiceSecretConfig
+	}{
+		{
+			name: "no_override",
+			secretBase: map[string]interface{}{
+				"secrets": []interface{}{
+					"my_secret",
+				},
+			},
+			secretOverride: map[string]interface{}{},
+			expected: []types.ServiceSecretConfig{
+				{
+					Source: "my_secret",
+				},
+			},
+		},
+		{
+			name: "override_simple",
+			secretBase: map[string]interface{}{
+				"secrets": []interface{}{
+					"foo_secret",
+				},
+			},
+			secretOverride: map[string]interface{}{
+				"secrets": []interface{}{
+					"bar_secret",
+				},
+			},
+			expected: []types.ServiceSecretConfig{
+				{
+					Source: "bar_secret",
+				},
+				{
+					Source: "foo_secret",
+				},
+			},
+		},
+		{
+			name: "override_same_source",
+			secretBase: map[string]interface{}{
+				"secrets": []interface{}{
+					"foo_secret",
+					map[string]interface{}{
+						"source": "bar_secret",
+						"target": "waw_secret",
+					},
+				},
+			},
+			secretOverride: map[string]interface{}{
+				"secrets": []interface{}{
+					map[string]interface{}{
+						"source": "bar_secret",
+						"target": "bof_secret",
+					},
+					map[string]interface{}{
+						"source": "baz_secret",
+						"target": "waw_secret",
+					},
+				},
+			},
+			expected: []types.ServiceSecretConfig{
+				{
+					Source: "bar_secret",
+					Target: "bof_secret",
+				},
+				{
+					Source: "baz_secret",
+					Target: "waw_secret",
+				},
+				{
+					Source: "foo_secret",
+				},
+			},
+		},
+	}
+
+	for _, tc := range portsCases {
+		t.Run(tc.name, func(t *testing.T) {
+			configDetails := types.ConfigDetails{
+				ConfigFiles: []types.ConfigFile{
+					{
+						Filename: "base.yml",
+						Config: map[string]interface{}{
+							"version": "3.4",
+							"services": map[string]interface{}{
+								"foo": tc.secretBase,
+							},
+						},
+					},
+					{
+						Filename: "override.yml",
+						Config: map[string]interface{}{
+							"version": "3.4",
+							"services": map[string]interface{}{
+								"foo": tc.secretOverride,
+							},
+						},
+					},
+				},
+			}
+			config, err := Load(configDetails)
+			require.NoError(t, err)
+			require.Equal(t, &types.Config{
+				Filename: "base.yml",
+				Version:  "3.4",
+				Services: []types.ServiceConfig{
+					{
+						Name:        "foo",
+						Secrets:     tc.expected,
+						Environment: types.MappingWithEquals{},
+					},
+				},
+				Networks: map[string]types.NetworkConfig{},
+				Volumes:  map[string]types.VolumeConfig{},
+				Secrets:  map[string]types.SecretConfig{},
+				Configs:  map[string]types.ConfigObjConfig{},
+			}, config)
+		})
+	}
+}
+
+func TestLoadMultipleConfigobjsConfig(t *testing.T) {
+	portsCases := []struct {
+		name           string
+		configBase     map[string]interface{}
+		configOverride map[string]interface{}
+		expected       []types.ServiceConfigObjConfig
+	}{
+		{
+			name: "no_override",
+			configBase: map[string]interface{}{
+				"configs": []interface{}{
+					"my_config",
+				},
+			},
+			configOverride: map[string]interface{}{},
+			expected: []types.ServiceConfigObjConfig{
+				{
+					Source: "my_config",
+				},
+			},
+		},
+		{
+			name: "override_simple",
+			configBase: map[string]interface{}{
+				"configs": []interface{}{
+					"foo_config",
+				},
+			},
+			configOverride: map[string]interface{}{
+				"configs": []interface{}{
+					"bar_config",
+				},
+			},
+			expected: []types.ServiceConfigObjConfig{
+				{
+					Source: "bar_config",
+				},
+				{
+					Source: "foo_config",
+				},
+			},
+		},
+		{
+			name: "override_same_source",
+			configBase: map[string]interface{}{
+				"configs": []interface{}{
+					"foo_config",
+					map[string]interface{}{
+						"source": "bar_config",
+						"target": "waw_config",
+					},
+				},
+			},
+			configOverride: map[string]interface{}{
+				"configs": []interface{}{
+					map[string]interface{}{
+						"source": "bar_config",
+						"target": "bof_config",
+					},
+					map[string]interface{}{
+						"source": "baz_config",
+						"target": "waw_config",
+					},
+				},
+			},
+			expected: []types.ServiceConfigObjConfig{
+				{
+					Source: "bar_config",
+					Target: "bof_config",
+				},
+				{
+					Source: "baz_config",
+					Target: "waw_config",
+				},
+				{
+					Source: "foo_config",
+				},
+			},
+		},
+	}
+
+	for _, tc := range portsCases {
+		t.Run(tc.name, func(t *testing.T) {
+			configDetails := types.ConfigDetails{
+				ConfigFiles: []types.ConfigFile{
+					{
+						Filename: "base.yml",
+						Config: map[string]interface{}{
+							"version": "3.4",
+							"services": map[string]interface{}{
+								"foo": tc.configBase,
+							},
+						},
+					},
+					{
+						Filename: "override.yml",
+						Config: map[string]interface{}{
+							"version": "3.4",
+							"services": map[string]interface{}{
+								"foo": tc.configOverride,
+							},
+						},
+					},
+				},
+			}
+			config, err := Load(configDetails)
+			require.NoError(t, err)
+			require.Equal(t, &types.Config{
+				Filename: "base.yml",
+				Version:  "3.4",
+				Services: []types.ServiceConfig{
+					{
+						Name:        "foo",
+						Configs:     tc.expected,
+						Environment: types.MappingWithEquals{},
+					},
+				},
+				Networks: map[string]types.NetworkConfig{},
+				Volumes:  map[string]types.VolumeConfig{},
+				Secrets:  map[string]types.SecretConfig{},
+				Configs:  map[string]types.ConfigObjConfig{},
+			}, config)
+		})
+	}
+}
+
+func TestLoadMultipleUlimits(t *testing.T) {
+	ulimitCases := []struct {
+		name           string
+		ulimitBase     map[string]interface{}
+		ulimitOverride map[string]interface{}
+		expected       map[string]*types.UlimitsConfig
+	}{
+		{
+			name: "no_override",
+			ulimitBase: map[string]interface{}{
+				"ulimits": map[string]interface{}{
+					"noproc": 65535,
+				},
+			},
+			ulimitOverride: map[string]interface{}{},
+			expected: map[string]*types.UlimitsConfig{
+				"noproc": {
+					Single: 65535,
+				},
+			},
+		},
+		{
+			name: "override_simple",
+			ulimitBase: map[string]interface{}{
+				"ulimits": map[string]interface{}{
+					"noproc": 65535,
+				},
+			},
+			ulimitOverride: map[string]interface{}{
+				"ulimits": map[string]interface{}{
+					"noproc": 44444,
+				},
+			},
+			expected: map[string]*types.UlimitsConfig{
+				"noproc": {
+					Single: 44444,
+				},
+			},
+		},
+		{
+			name: "override_different_notation",
+			ulimitBase: map[string]interface{}{
+				"ulimits": map[string]interface{}{
+					"nofile": map[string]interface{}{
+						"soft": 11111,
+						"hard": 99999,
+					},
+					"noproc": 44444,
+				},
+			},
+			ulimitOverride: map[string]interface{}{
+				"ulimits": map[string]interface{}{
+					"nofile": 55555,
+					"noproc": map[string]interface{}{
+						"soft": 22222,
+						"hard": 33333,
+					},
+				},
+			},
+			expected: map[string]*types.UlimitsConfig{
+				"noproc": {
+					Soft: 22222,
+					Hard: 33333,
+				},
+				"nofile": {
+					Single: 55555,
+				},
+			},
+		},
+	}
+
+	for _, tc := range ulimitCases {
+		t.Run(tc.name, func(t *testing.T) {
+			configDetails := types.ConfigDetails{
+				ConfigFiles: []types.ConfigFile{
+					{
+						Filename: "base.yml",
+						Config: map[string]interface{}{
+							"version": "3.4",
+							"services": map[string]interface{}{
+								"foo": tc.ulimitBase,
+							},
+						},
+					},
+					{
+						Filename: "override.yml",
+						Config: map[string]interface{}{
+							"version": "3.4",
+							"services": map[string]interface{}{
+								"foo": tc.ulimitOverride,
+							},
+						},
+					},
+				},
+			}
+			config, err := Load(configDetails)
+			require.NoError(t, err)
+			require.Equal(t, &types.Config{
+				Filename: "base.yml",
+				Version:  "3.4",
+				Services: []types.ServiceConfig{
+					{
+						Name:        "foo",
+						Ulimits:     tc.expected,
+						Environment: types.MappingWithEquals{},
+					},
+				},
+				Networks: map[string]types.NetworkConfig{},
+				Volumes:  map[string]types.VolumeConfig{},
+				Secrets:  map[string]types.SecretConfig{},
+				Configs:  map[string]types.ConfigObjConfig{},
+			}, config)
+		})
+	}
+}
+
+func TestLoadMultipleNetworks(t *testing.T) {
+	networkCases := []struct {
+		name            string
+		networkBase     map[string]interface{}
+		networkOverride map[string]interface{}
+		expected        map[string]*types.ServiceNetworkConfig
+	}{
+		{
+			name: "no_override",
+			networkBase: map[string]interface{}{
+				"networks": []interface{}{
+					"net1",
+					"net2",
+				},
+			},
+			networkOverride: map[string]interface{}{},
+			expected: map[string]*types.ServiceNetworkConfig{
+				"net1": nil,
+				"net2": nil,
+			},
+		},
+		{
+			name: "override_simple",
+			networkBase: map[string]interface{}{
+				"networks": []interface{}{
+					"net1",
+					"net2",
+				},
+			},
+			networkOverride: map[string]interface{}{
+				"networks": []interface{}{
+					"net1",
+					"net3",
+				},
+			},
+			expected: map[string]*types.ServiceNetworkConfig{
+				"net1": nil,
+				"net2": nil,
+				"net3": nil,
+			},
+		},
+		{
+			name: "override_with_aliases",
+			networkBase: map[string]interface{}{
+				"networks": map[string]interface{}{
+					"net1": map[string]interface{}{
+						"aliases": []interface{}{
+							"alias1",
+						},
+					},
+					"net2": nil,
+				},
+			},
+			networkOverride: map[string]interface{}{
+				"networks": map[string]interface{}{
+					"net1": map[string]interface{}{
+						"aliases": []interface{}{
+							"alias2",
+							"alias3",
+						},
+					},
+					"net3": map[string]interface{}{},
+				},
+			},
+			expected: map[string]*types.ServiceNetworkConfig{
+				"net1": {
+					Aliases: []string{"alias2", "alias3"},
+				},
+				"net2": nil,
+				"net3": {},
+			},
+		},
+	}
+
+	for _, tc := range networkCases {
+		t.Run(tc.name, func(t *testing.T) {
+			configDetails := types.ConfigDetails{
+				ConfigFiles: []types.ConfigFile{
+					{
+						Filename: "base.yml",
+						Config: map[string]interface{}{
+							"version": "3.4",
+							"services": map[string]interface{}{
+								"foo": tc.networkBase,
+							},
+						},
+					},
+					{
+						Filename: "override.yml",
+						Config: map[string]interface{}{
+							"version": "3.4",
+							"services": map[string]interface{}{
+								"foo": tc.networkOverride,
+							},
+						},
+					},
+				},
+			}
+			config, err := Load(configDetails)
+			require.NoError(t, err)
+			require.Equal(t, &types.Config{
+				Filename: "base.yml",
+				Version:  "3.4",
+				Services: []types.ServiceConfig{
+					{
+						Name:        "foo",
+						Networks:    tc.expected,
+						Environment: types.MappingWithEquals{},
+					},
+				},
+				Networks: map[string]types.NetworkConfig{},
+				Volumes:  map[string]types.VolumeConfig{},
+				Secrets:  map[string]types.SecretConfig{},
+				Configs:  map[string]types.ConfigObjConfig{},
+			}, config)
+		})
+	}
+}
+
+func TestLoadMultipleConfigs(t *testing.T) {
+	base := map[string]interface{}{
+		"version": "3.4",
+		"services": map[string]interface{}{
+			"foo": map[string]interface{}{
+				"image": "foo",
+				"build": map[string]interface{}{
+					"context":    ".",
+					"dockerfile": "bar.Dockerfile",
+				},
+				"ports": []interface{}{
+					"8080:80",
+					"9090:90",
+				},
+				"labels": []interface{}{
+					"foo=bar",
+				},
+				"cap_add": []interface{}{
+					"NET_ADMIN",
+				},
+			},
+		},
+		"volumes":  map[string]interface{}{},
+		"networks": map[string]interface{}{},
+		"secrets":  map[string]interface{}{},
+		"configs":  map[string]interface{}{},
+	}
+	override := map[string]interface{}{
+		"version": "3.4",
+		"services": map[string]interface{}{
+			"foo": map[string]interface{}{
+				"image": "baz",
+				"build": map[string]interface{}{
+					"dockerfile": "foo.Dockerfile",
+					"args": []interface{}{
+						"buildno=1",
+						"password=secret",
+					},
+				},
+				"ports": []interface{}{
+					map[string]interface{}{
+						"target":    81,
+						"published": 8080,
+					},
+				},
+				"labels": map[string]interface{}{
+					"foo": "baz",
+				},
+				"cap_add": []interface{}{
+					"SYS_ADMIN",
+				},
+			},
+			"bar": map[string]interface{}{
+				"image": "bar",
+			},
+		},
+		"volumes":  map[string]interface{}{},
+		"networks": map[string]interface{}{},
+		"secrets":  map[string]interface{}{},
+		"configs":  map[string]interface{}{},
+	}
+	configDetails := types.ConfigDetails{
+		ConfigFiles: []types.ConfigFile{
+			{Filename: "base.yml", Config: base},
+			{Filename: "override.yml", Config: override},
+		},
+	}
+	config, err := Load(configDetails)
+	require.NoError(t, err)
+	require.Equal(t, &types.Config{
+		Filename: "base.yml",
+		Version:  "3.4",
+		Services: []types.ServiceConfig{
+			{
+				Name:        "bar",
+				Image:       "bar",
+				Environment: types.MappingWithEquals{},
+			},
+			{
+				Name:  "foo",
+				Image: "baz",
+				Build: types.BuildConfig{
+					Context:    ".",
+					Dockerfile: "foo.Dockerfile",
+					Args: types.MappingWithEquals{
+						"buildno":  strPtr("1"),
+						"password": strPtr("secret"),
+					},
+				},
+				Ports: []types.ServicePortConfig{
+					{
+						Target:    81,
+						Published: 8080,
+					},
+					{
+						Mode:      "ingress",
+						Target:    90,
+						Published: 9090,
+						Protocol:  "tcp",
+					},
+				},
+				Labels: types.Labels{
+					"foo": "baz",
+				},
+				CapAdd:      []string{"NET_ADMIN", "SYS_ADMIN"},
+				Environment: types.MappingWithEquals{},
+			}},
+		Networks: map[string]types.NetworkConfig{},
+		Volumes:  map[string]types.VolumeConfig{},
+		Secrets:  map[string]types.SecretConfig{},
+		Configs:  map[string]types.ConfigObjConfig{},
+	}, config)
+}

components/cli/cli/compose/loader/types_test.go

@@ -0,0 +1,329 @@
+package loader
+
+import (
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+	yaml "gopkg.in/yaml.v2"
+)
+
+func TestMarshallConfig(t *testing.T) {
+	cfg := fullExampleConfig("/foo", "/bar")
+	expected := `version: "3.6"
+services:
+  foo:
+    build:
+      context: ./dir
+      dockerfile: Dockerfile
+      args:
+        foo: bar
+      labels:
+        FOO: BAR
+      cache_from:
+      - foo
+      - bar
+      network: foo
+      target: foo
+    cap_add:
+    - ALL
+    cap_drop:
+    - NET_ADMIN
+    - SYS_ADMIN
+    cgroup_parent: m-executor-abcd
+    command:
+    - bundle
+    - exec
+    - thin
+    - -p
+    - "3000"
+    container_name: my-web-container
+    depends_on:
+    - db
+    - redis
+    deploy:
+      mode: replicated
+      replicas: 6
+      labels:
+        FOO: BAR
+      update_config:
+        parallelism: 3
+        delay: 10s
+        failure_action: continue
+        monitor: 1m0s
+        max_failure_ratio: 0.3
+        order: start-first
+      resources:
+        limits:
+          cpus: "0.001"
+          memory: "52428800"
+        reservations:
+          cpus: "0.0001"
+          memory: "20971520"
+          generic_resources:
+          - discrete_resource_spec:
+              kind: gpu
+              value: 2
+          - discrete_resource_spec:
+              kind: ssd
+              value: 1
+      restart_policy:
+        condition: on-failure
+        delay: 5s
+        max_attempts: 3
+        window: 2m0s
+      placement:
+        constraints:
+        - node=foo
+        preferences:
+        - spread: node.labels.az
+      endpoint_mode: dnsrr
+    devices:
+    - /dev/ttyUSB0:/dev/ttyUSB0
+    dns:
+    - 8.8.8.8
+    - 9.9.9.9
+    dns_search:
+    - dc1.example.com
+    - dc2.example.com
+    domainname: foo.com
+    entrypoint:
+    - /code/entrypoint.sh
+    - -p
+    - "3000"
+    environment:
+      BAR: bar_from_env_file_2
+      BAZ: baz_from_service_def
+      FOO: foo_from_env_file
+      QUX: qux_from_environment
+    env_file:
+    - ./example1.env
+    - ./example2.env
+    expose:
+    - "3000"
+    - "8000"
+    external_links:
+    - redis_1
+    - project_db_1:mysql
+    - project_db_1:postgresql
+    extra_hosts:
+    - somehost:162.242.195.82
+    - otherhost:50.31.209.229
+    hostname: foo
+    healthcheck:
+      test:
+      - CMD-SHELL
+      - echo "hello world"
+      timeout: 1s
+      interval: 10s
+      retries: 5
+      start_period: 15s
+    image: redis
+    ipc: host
+    labels:
+      com.example.description: Accounting webapp
+      com.example.empty-label: ""
+      com.example.number: "42"
+    links:
+    - db
+    - db:database
+    - redis
+    logging:
+      driver: syslog
+      options:
+        syslog-address: tcp://192.168.0.42:123
+    mac_address: 02:42:ac:11:65:43
+    network_mode: container:0cfeab0f748b9a743dc3da582046357c6ef497631c1a016d28d2bf9b4f899f7b
+    networks:
+      other-network:
+        ipv4_address: 172.16.238.10
+        ipv6_address: 2001:3984:3989::10
+      other-other-network: null
+      some-network:
+        aliases:
+        - alias1
+        - alias3
+    pid: host
+    ports:
+    - mode: ingress
+      target: 3000
+      protocol: tcp
+    - mode: ingress
+      target: 3001
+      protocol: tcp
+    - mode: ingress
+      target: 3002
+      protocol: tcp
+    - mode: ingress
+      target: 3003
+      protocol: tcp
+    - mode: ingress
+      target: 3004
+      protocol: tcp
+    - mode: ingress
+      target: 3005
+      protocol: tcp
+    - mode: ingress
+      target: 8000
+      published: 8000
+      protocol: tcp
+    - mode: ingress
+      target: 8080
+      published: 9090
+      protocol: tcp
+    - mode: ingress
+      target: 8081
+      published: 9091
+      protocol: tcp
+    - mode: ingress
+      target: 22
+      published: 49100
+      protocol: tcp
+    - mode: ingress
+      target: 8001
+      published: 8001
+      protocol: tcp
+    - mode: ingress
+      target: 5000
+      published: 5000
+      protocol: tcp
+    - mode: ingress
+      target: 5001
+      published: 5001
+      protocol: tcp
+    - mode: ingress
+      target: 5002
+      published: 5002
+      protocol: tcp
+    - mode: ingress
+      target: 5003
+      published: 5003
+      protocol: tcp
+    - mode: ingress
+      target: 5004
+      published: 5004
+      protocol: tcp
+    - mode: ingress
+      target: 5005
+      published: 5005
+      protocol: tcp
+    - mode: ingress
+      target: 5006
+      published: 5006
+      protocol: tcp
+    - mode: ingress
+      target: 5007
+      published: 5007
+      protocol: tcp
+    - mode: ingress
+      target: 5008
+      published: 5008
+      protocol: tcp
+    - mode: ingress
+      target: 5009
+      published: 5009
+      protocol: tcp
+    - mode: ingress
+      target: 5010
+      published: 5010
+      protocol: tcp
+    privileged: true
+    read_only: true
+    restart: always
+    security_opt:
+    - label=level:s0:c100,c200
+    - label=type:svirt_apache_t
+    stdin_open: true
+    stop_grace_period: 20s
+    stop_signal: SIGUSR1
+    tmpfs:
+    - /run
+    - /tmp
+    tty: true
+    ulimits:
+      nofile:
+        soft: 20000
+        hard: 40000
+      nproc: 65535
+    user: someone
+    volumes:
+    - type: volume
+      target: /var/lib/mysql
+    - type: bind
+      source: /opt/data
+      target: /var/lib/mysql
+    - type: bind
+      source: /foo
+      target: /code
+    - type: bind
+      source: /foo/static
+      target: /var/www/html
+    - type: bind
+      source: /bar/configs
+      target: /etc/configs/
+      read_only: true
+    - type: volume
+      source: datavolume
+      target: /var/lib/mysql
+    - type: bind
+      source: /foo/opt
+      target: /opt
+      consistency: cached
+    - type: tmpfs
+      target: /opt
+      tmpfs:
+        size: 10000
+    working_dir: /code
+networks:
+  external-network:
+    name: external-network
+    external: true
+  other-external-network:
+    name: my-cool-network
+    external: true
+  other-network:
+    driver: overlay
+    driver_opts:
+      baz: "1"
+      foo: bar
+    ipam:
+      driver: overlay
+      config:
+      - subnet: 172.16.238.0/24
+      - subnet: 2001:3984:3989::/64
+  some-network: {}
+volumes:
+  another-volume:
+    name: user_specified_name
+    driver: vsphere
+    driver_opts:
+      baz: "1"
+      foo: bar
+  external-volume:
+    name: external-volume
+    external: true
+  external-volume3:
+    name: this-is-volume3
+    external: true
+  other-external-volume:
+    name: my-cool-volume
+    external: true
+  other-volume:
+    driver: flocker
+    driver_opts:
+      baz: "1"
+      foo: bar
+  some-volume: {}
+secrets: {}
+configs: {}
+`
+
+	actual, err := yaml.Marshal(cfg)
+	assert.NoError(t, err)
+	assert.Equal(t, expected, string(actual))
+
+	// Make sure the expected still
+	dict, err := ParseYAML([]byte("version: '3.6'\n" + expected))
+	assert.NoError(t, err)
+	_, err = Load(buildConfigDetails(dict, map[string]string{}))
+	assert.NoError(t, err)
+}

components/cli/cli/compose/schema/data/config_schema_v3.6.json

@@ -1,6 +1,6 @@
 {
   "$schema": "http://json-schema.org/draft-04/schema#",
-  "id": "config_schema_v3.5.json",
+  "id": "config_schema_v3.6.json",
   "type": "object",
   "required": ["version"],
 

components/cli/cli/compose/schema/schema.go

@@ -1,6 +1,6 @@
 package schema
 
-//go:generate go-bindata -pkg schema -nometadata data
+//go:generate esc -o bindata.go -pkg schema -private -modtime=1518458244 data
 
 import (
 	"fmt"
@@ -56,7 +56,7 @@ func normalizeVersion(version string) string {
 
 // Validate uses the jsonschema to validate the configuration
 func Validate(config map[string]interface{}, version string) error {
-	schemaData, err := Asset(fmt.Sprintf("data/config_schema_v%s.json", version))
+	schemaData, err := _escFSByte(false, fmt.Sprintf("/data/config_schema_v%s.json", version))
 	if err != nil {
 		return errors.Errorf("unsupported Compose file version: %s", version)
 	}

components/cli/cli/compose/types/types.go

@@ -1,6 +1,7 @@
 package types
 
 import (
+	"fmt"
 	"time"
 )
 
@@ -69,76 +70,90 @@ func (cd ConfigDetails) LookupEnv(key string) (string, bool) {
 
 // Config is a full compose file configuration
 type Config struct {
-	Services []ServiceConfig
+	Filename string `yaml:"-"`
+	Version  string
+	Services Services
 	Networks map[string]NetworkConfig
 	Volumes  map[string]VolumeConfig
 	Secrets  map[string]SecretConfig
 	Configs  map[string]ConfigObjConfig
 }
 
+// Services is a list of ServiceConfig
+type Services []ServiceConfig
+
+// MarshalYAML makes Services implement yaml.Marshaller
+func (s Services) MarshalYAML() (interface{}, error) {
+	services := map[string]ServiceConfig{}
+	for _, service := range s {
+		services[service.Name] = service
+	}
+	return services, nil
+}
+
 // ServiceConfig is the configuration of one service
 type ServiceConfig struct {
-	Name string
+	Name string `yaml:"-"`
 
-	Build           BuildConfig
-	CapAdd          []string `mapstructure:"cap_add"`
-	CapDrop         []string `mapstructure:"cap_drop"`
-	CgroupParent    string   `mapstructure:"cgroup_parent"`
-	Command         ShellCommand
-	Configs         []ServiceConfigObjConfig
-	ContainerName   string               `mapstructure:"container_name"`
-	CredentialSpec  CredentialSpecConfig `mapstructure:"credential_spec"`
-	DependsOn       []string             `mapstructure:"depends_on"`
-	Deploy          DeployConfig
-	Devices         []string
-	DNS             StringList
-	DNSSearch       StringList `mapstructure:"dns_search"`
-	DomainName      string     `mapstructure:"domainname"`
-	Entrypoint      ShellCommand
-	Environment     MappingWithEquals
-	EnvFile         StringList `mapstructure:"env_file"`
-	Expose          StringOrNumberList
-	ExternalLinks   []string  `mapstructure:"external_links"`
-	ExtraHosts      HostsList `mapstructure:"extra_hosts"`
-	Hostname        string
-	HealthCheck     *HealthCheckConfig
-	Image           string
-	Ipc             string
-	Labels          Labels
-	Links           []string
-	Logging         *LoggingConfig
-	MacAddress      string `mapstructure:"mac_address"`
-	NetworkMode     string `mapstructure:"network_mode"`
-	Networks        map[string]*ServiceNetworkConfig
-	Pid             string
-	Ports           []ServicePortConfig
-	Privileged      bool
-	ReadOnly        bool `mapstructure:"read_only"`
-	Restart         string
-	Secrets         []ServiceSecretConfig
-	SecurityOpt     []string       `mapstructure:"security_opt"`
-	StdinOpen       bool           `mapstructure:"stdin_open"`
-	StopGracePeriod *time.Duration `mapstructure:"stop_grace_period"`
-	StopSignal      string         `mapstructure:"stop_signal"`
-	Tmpfs           StringList
-	Tty             bool `mapstructure:"tty"`
-	Ulimits         map[string]*UlimitsConfig
-	User            string
-	Volumes         []ServiceVolumeConfig
-	WorkingDir      string `mapstructure:"working_dir"`
-	Isolation       string `mapstructure:"isolation"`
+	Build           BuildConfig                      `yaml:",omitempty"`
+	CapAdd          []string                         `mapstructure:"cap_add" yaml:"cap_add,omitempty"`
+	CapDrop         []string                         `mapstructure:"cap_drop" yaml:"cap_drop,omitempty"`
+	CgroupParent    string                           `mapstructure:"cgroup_parent" yaml:"cgroup_parent,omitempty"`
+	Command         ShellCommand                     `yaml:",omitempty"`
+	Configs         []ServiceConfigObjConfig         `yaml:",omitempty"`
+	ContainerName   string                           `mapstructure:"container_name" yaml:"container_name,omitempty"`
+	CredentialSpec  CredentialSpecConfig             `mapstructure:"credential_spec" yaml:"credential_spec,omitempty"`
+	DependsOn       []string                         `mapstructure:"depends_on" yaml:"depends_on,omitempty"`
+	Deploy          DeployConfig                     `yaml:",omitempty"`
+	Devices         []string                         `yaml:",omitempty"`
+	DNS             StringList                       `yaml:",omitempty"`
+	DNSSearch       StringList                       `mapstructure:"dns_search" yaml:"dns_search,omitempty"`
+	DomainName      string                           `mapstructure:"domainname" yaml:"domainname,omitempty"`
+	Entrypoint      ShellCommand                     `yaml:",omitempty"`
+	Environment     MappingWithEquals                `yaml:",omitempty"`
+	EnvFile         StringList                       `mapstructure:"env_file" yaml:"env_file,omitempty"`
+	Expose          StringOrNumberList               `yaml:",omitempty"`
+	ExternalLinks   []string                         `mapstructure:"external_links" yaml:"external_links,omitempty"`
+	ExtraHosts      HostsList                        `mapstructure:"extra_hosts" yaml:"extra_hosts,omitempty"`
+	Hostname        string                           `yaml:",omitempty"`
+	HealthCheck     *HealthCheckConfig               `yaml:",omitempty"`
+	Image           string                           `yaml:",omitempty"`
+	Ipc             string                           `yaml:",omitempty"`
+	Labels          Labels                           `yaml:",omitempty"`
+	Links           []string                         `yaml:",omitempty"`
+	Logging         *LoggingConfig                   `yaml:",omitempty"`
+	MacAddress      string                           `mapstructure:"mac_address" yaml:"mac_address,omitempty"`
+	NetworkMode     string                           `mapstructure:"network_mode" yaml:"network_mode,omitempty"`
+	Networks        map[string]*ServiceNetworkConfig `yaml:",omitempty"`
+	Pid             string                           `yaml:",omitempty"`
+	Ports           []ServicePortConfig              `yaml:",omitempty"`
+	Privileged      bool                             `yaml:",omitempty"`
+	ReadOnly        bool                             `mapstructure:"read_only" yaml:"read_only,omitempty"`
+	Restart         string                           `yaml:",omitempty"`
+	Secrets         []ServiceSecretConfig            `yaml:",omitempty"`
+	SecurityOpt     []string                         `mapstructure:"security_opt" yaml:"security_opt,omitempty"`
+	StdinOpen       bool                             `mapstructure:"stdin_open" yaml:"stdin_open,omitempty"`
+	StopGracePeriod *time.Duration                   `mapstructure:"stop_grace_period" yaml:"stop_grace_period,omitempty"`
+	StopSignal      string                           `mapstructure:"stop_signal" yaml:"stop_signal,omitempty"`
+	Tmpfs           StringList                       `yaml:",omitempty"`
+	Tty             bool                             `mapstructure:"tty" yaml:"tty,omitempty"`
+	Ulimits         map[string]*UlimitsConfig        `yaml:",omitempty"`
+	User            string                           `yaml:",omitempty"`
+	Volumes         []ServiceVolumeConfig            `yaml:",omitempty"`
+	WorkingDir      string                           `mapstructure:"working_dir" yaml:"working_dir,omitempty"`
+	Isolation       string                           `mapstructure:"isolation" yaml:"isolation,omitempty"`
 }
 
 // BuildConfig is a type for build
 // using the same format at libcompose: https://github.com/docker/libcompose/blob/master/yaml/build.go#L12
 type BuildConfig struct {
-	Context    string
-	Dockerfile string
-	Args       MappingWithEquals
-	Labels     Labels
-	CacheFrom  StringList `mapstructure:"cache_from"`
-	Network    string
-	Target     string
+	Context    string            `yaml:",omitempty"`
+	Dockerfile string            `yaml:",omitempty"`
+	Args       MappingWithEquals `yaml:",omitempty"`
+	Labels     Labels            `yaml:",omitempty"`
+	CacheFrom  StringList        `mapstructure:"cache_from" yaml:"cache_from,omitempty"`
+	Network    string            `yaml:",omitempty"`
+	Target     string            `yaml:",omitempty"`
 }
 
 // ShellCommand is a string or list of string args
@@ -169,30 +184,30 @@ type HostsList []string
 
 // LoggingConfig the logging configuration for a service
 type LoggingConfig struct {
-	Driver  string
-	Options map[string]string
+	Driver  string            `yaml:",omitempty"`
+	Options map[string]string `yaml:",omitempty"`
 }
 
 // DeployConfig the deployment configuration for a service
 type DeployConfig struct {
-	Mode          string
-	Replicas      *uint64
-	Labels        Labels
-	UpdateConfig  *UpdateConfig `mapstructure:"update_config"`
-	Resources     Resources
-	RestartPolicy *RestartPolicy `mapstructure:"restart_policy"`
-	Placement     Placement
-	EndpointMode  string `mapstructure:"endpoint_mode"`
+	Mode          string         `yaml:",omitempty"`
+	Replicas      *uint64        `yaml:",omitempty"`
+	Labels        Labels         `yaml:",omitempty"`
+	UpdateConfig  *UpdateConfig  `mapstructure:"update_config" yaml:"update_config,omitempty"`
+	Resources     Resources      `yaml:",omitempty"`
+	RestartPolicy *RestartPolicy `mapstructure:"restart_policy" yaml:"restart_policy,omitempty"`
+	Placement     Placement      `yaml:",omitempty"`
+	EndpointMode  string         `mapstructure:"endpoint_mode" yaml:"endpoint_mode,omitempty"`
 }
 
 // HealthCheckConfig the healthcheck configuration for a service
 type HealthCheckConfig struct {
-	Test        HealthCheckTest
-	Timeout     *time.Duration
-	Interval    *time.Duration
-	Retries     *uint64
-	StartPeriod *time.Duration `mapstructure:"start_period"`
-	Disable     bool
+	Test        HealthCheckTest `yaml:",omitempty"`
+	Timeout     *time.Duration  `yaml:",omitempty"`
+	Interval    *time.Duration  `yaml:",omitempty"`
+	Retries     *uint64         `yaml:",omitempty"`
+	StartPeriod *time.Duration  `mapstructure:"start_period" yaml:"start_period,omitempty"`
+	Disable     bool            `yaml:",omitempty"`
 }
 
 // HealthCheckTest is the command run to test the health of a service
@@ -200,32 +215,32 @@ type HealthCheckTest []string
 
 // UpdateConfig the service update configuration
 type UpdateConfig struct {
-	Parallelism     *uint64
-	Delay           time.Duration
-	FailureAction   string `mapstructure:"failure_action"`
-	Monitor         time.Duration
-	MaxFailureRatio float32 `mapstructure:"max_failure_ratio"`
-	Order           string
+	Parallelism     *uint64       `yaml:",omitempty"`
+	Delay           time.Duration `yaml:",omitempty"`
+	FailureAction   string        `mapstructure:"failure_action" yaml:"failure_action,omitempty"`
+	Monitor         time.Duration `yaml:",omitempty"`
+	MaxFailureRatio float32       `mapstructure:"max_failure_ratio" yaml:"max_failure_ratio,omitempty"`
+	Order           string        `yaml:",omitempty"`
 }
 
 // Resources the resource limits and reservations
 type Resources struct {
-	Limits       *Resource
-	Reservations *Resource
+	Limits       *Resource `yaml:",omitempty"`
+	Reservations *Resource `yaml:",omitempty"`
 }
 
 // Resource is a resource to be limited or reserved
 type Resource struct {
 	// TODO: types to convert from units and ratios
-	NanoCPUs         string            `mapstructure:"cpus"`
-	MemoryBytes      UnitBytes         `mapstructure:"memory"`
-	GenericResources []GenericResource `mapstructure:"generic_resources"`
+	NanoCPUs         string            `mapstructure:"cpus" yaml:"cpus,omitempty"`
+	MemoryBytes      UnitBytes         `mapstructure:"memory" yaml:"memory,omitempty"`
+	GenericResources []GenericResource `mapstructure:"generic_resources" yaml:"generic_resources,omitempty"`
 }
 
 // GenericResource represents a "user defined" resource which can
 // only be an integer (e.g: SSD=3) for a service
 type GenericResource struct {
-	DiscreteResourceSpec *DiscreteGenericResource `mapstructure:"discrete_resource_spec"`
+	DiscreteResourceSpec *DiscreteGenericResource `mapstructure:"discrete_resource_spec" yaml:"discrete_resource_spec,omitempty"`
 }
 
 // DiscreteGenericResource represents a "user defined" resource which is defined
@@ -240,74 +255,79 @@ type DiscreteGenericResource struct {
 // UnitBytes is the bytes type
 type UnitBytes int64
 
+// MarshalYAML makes UnitBytes implement yaml.Marshaller
+func (u UnitBytes) MarshalYAML() (interface{}, error) {
+	return fmt.Sprintf("%d", u), nil
+}
+
 // RestartPolicy the service restart policy
 type RestartPolicy struct {
-	Condition   string
-	Delay       *time.Duration
-	MaxAttempts *uint64 `mapstructure:"max_attempts"`
-	Window      *time.Duration
+	Condition   string         `yaml:",omitempty"`
+	Delay       *time.Duration `yaml:",omitempty"`
+	MaxAttempts *uint64        `mapstructure:"max_attempts" yaml:"max_attempts,omitempty"`
+	Window      *time.Duration `yaml:",omitempty"`
 }
 
 // Placement constraints for the service
 type Placement struct {
-	Constraints []string
-	Preferences []PlacementPreferences
+	Constraints []string               `yaml:",omitempty"`
+	Preferences []PlacementPreferences `yaml:",omitempty"`
 }
 
 // PlacementPreferences is the preferences for a service placement
 type PlacementPreferences struct {
-	Spread string
+	Spread string `yaml:",omitempty"`
 }
 
 // ServiceNetworkConfig is the network configuration for a service
 type ServiceNetworkConfig struct {
-	Aliases     []string
-	Ipv4Address string `mapstructure:"ipv4_address"`
-	Ipv6Address string `mapstructure:"ipv6_address"`
+	Aliases     []string `yaml:",omitempty"`
+	Ipv4Address string   `mapstructure:"ipv4_address" yaml:"ipv4_address,omitempty"`
+	Ipv6Address string   `mapstructure:"ipv6_address" yaml:"ipv6_address,omitempty"`
 }
 
 // ServicePortConfig is the port configuration for a service
 type ServicePortConfig struct {
-	Mode      string
-	Target    uint32
-	Published uint32
-	Protocol  string
+	Mode      string `yaml:",omitempty"`
+	Target    uint32 `yaml:",omitempty"`
+	Published uint32 `yaml:",omitempty"`
+	Protocol  string `yaml:",omitempty"`
 }
 
 // ServiceVolumeConfig are references to a volume used by a service
 type ServiceVolumeConfig struct {
-	Type        string
-	Source      string
-	Target      string
-	ReadOnly    bool `mapstructure:"read_only"`
-	Consistency string
-	Bind        *ServiceVolumeBind
-	Volume      *ServiceVolumeVolume
-	Tmpfs       *ServiceVolumeTmpfs
+	Type        string               `yaml:",omitempty"`
+	Source      string               `yaml:",omitempty"`
+	Target      string               `yaml:",omitempty"`
+	ReadOnly    bool                 `mapstructure:"read_only" yaml:"read_only,omitempty"`
+	Consistency string               `yaml:",omitempty"`
+	Bind        *ServiceVolumeBind   `yaml:",omitempty"`
+	Volume      *ServiceVolumeVolume `yaml:",omitempty"`
+	Tmpfs       *ServiceVolumeTmpfs  `yaml:",omitempty"`
 }
 
 // ServiceVolumeBind are options for a service volume of type bind
 type ServiceVolumeBind struct {
-	Propagation string
+	Propagation string `yaml:",omitempty"`
 }
 
 // ServiceVolumeVolume are options for a service volume of type volume
 type ServiceVolumeVolume struct {
-	NoCopy bool `mapstructure:"nocopy"`
+	NoCopy bool `mapstructure:"nocopy" yaml:"nocopy,omitempty"`
 }
 
 // ServiceVolumeTmpfs are options for a service volume of type tmpfs
 type ServiceVolumeTmpfs struct {
-	Size int64
+	Size int64 `yaml:",omitempty"`
 }
 
 // FileReferenceConfig for a reference to a swarm file object
 type FileReferenceConfig struct {
-	Source string
-	Target string
-	UID    string
-	GID    string
-	Mode   *uint32
+	Source string  `yaml:",omitempty"`
+	Target string  `yaml:",omitempty"`
+	UID    string  `yaml:",omitempty"`
+	GID    string  `yaml:",omitempty"`
+	Mode   *uint32 `yaml:",omitempty"`
 }
 
 // ServiceConfigObjConfig is the config obj configuration for a service
@@ -318,63 +338,79 @@ type ServiceSecretConfig FileReferenceConfig
 
 // UlimitsConfig the ulimit configuration
 type UlimitsConfig struct {
-	Single int
-	Soft   int
-	Hard   int
+	Single int `yaml:",omitempty"`
+	Soft   int `yaml:",omitempty"`
+	Hard   int `yaml:",omitempty"`
+}
+
+// MarshalYAML makes UlimitsConfig implement yaml.Marshaller
+func (u *UlimitsConfig) MarshalYAML() (interface{}, error) {
+	if u.Single != 0 {
+		return u.Single, nil
+	}
+	return u, nil
 }
 
 // NetworkConfig for a network
 type NetworkConfig struct {
-	Name       string
-	Driver     string
-	DriverOpts map[string]string `mapstructure:"driver_opts"`
-	Ipam       IPAMConfig
-	External   External
-	Internal   bool
-	Attachable bool
-	Labels     Labels
+	Name       string            `yaml:",omitempty"`
+	Driver     string            `yaml:",omitempty"`
+	DriverOpts map[string]string `mapstructure:"driver_opts" yaml:"driver_opts,omitempty"`
+	Ipam       IPAMConfig        `yaml:",omitempty"`
+	External   External          `yaml:",omitempty"`
+	Internal   bool              `yaml:",omitempty"`
+	Attachable bool              `yaml:",omitempty"`
+	Labels     Labels            `yaml:",omitempty"`
 }
 
 // IPAMConfig for a network
 type IPAMConfig struct {
-	Driver string
-	Config []*IPAMPool
+	Driver string      `yaml:",omitempty"`
+	Config []*IPAMPool `yaml:",omitempty"`
 }
 
 // IPAMPool for a network
 type IPAMPool struct {
-	Subnet string
+	Subnet string `yaml:",omitempty"`
 }
 
 // VolumeConfig for a volume
 type VolumeConfig struct {
-	Name       string
-	Driver     string
-	DriverOpts map[string]string `mapstructure:"driver_opts"`
-	External   External
-	Labels     Labels
+	Name       string            `yaml:",omitempty"`
+	Driver     string            `yaml:",omitempty"`
+	DriverOpts map[string]string `mapstructure:"driver_opts" yaml:"driver_opts,omitempty"`
+	External   External          `yaml:",omitempty"`
+	Labels     Labels            `yaml:",omitempty"`
 }
 
 // External identifies a Volume or Network as a reference to a resource that is
 // not managed, and should already exist.
 // External.name is deprecated and replaced by Volume.name
 type External struct {
-	Name     string
-	External bool
+	Name     string `yaml:",omitempty"`
+	External bool   `yaml:",omitempty"`
+}
+
+// MarshalYAML makes External implement yaml.Marshaller
+func (e External) MarshalYAML() (interface{}, error) {
+	if e.Name == "" {
+		return e.External, nil
+	}
+	return External{Name: e.Name}, nil
 }
 
 // CredentialSpecConfig for credential spec on Windows
 type CredentialSpecConfig struct {
-	File     string
-	Registry string
+	File     string `yaml:",omitempty"`
+	Registry string `yaml:",omitempty"`
 }
 
 // FileObjectConfig is a config type for a file used by a service
 type FileObjectConfig struct {
-	Name     string
-	File     string
-	External External
-	Labels   Labels
+	Name     string   `yaml:",omitempty"`
+	File     string   `yaml:",omitempty"`
+	External External `yaml:",omitempty"`
+	Labels   Labels   `yaml:",omitempty"`
 }
 
 // SecretConfig for a secret

components/cli/cli/config/configfile/file.go

@@ -19,7 +19,7 @@ const (
 	// This constant is only used for really old config files when the
 	// URL wasn't saved as part of the config file and it was just
 	// assumed to be this value.
-	defaultIndexserver = "https://index.docker.io/v1/"
+	defaultIndexServer = "https://index.docker.io/v1/"
 )
 
 // ConfigFile ~/.docker/config.json file info
@@ -87,8 +87,8 @@ func (configFile *ConfigFile) LegacyLoadFromReader(configData io.Reader) error {
 		if err != nil {
 			return err
 		}
-		authConfig.ServerAddress = defaultIndexserver
-		configFile.AuthConfigs[defaultIndexserver] = authConfig
+		authConfig.ServerAddress = defaultIndexServer
+		configFile.AuthConfigs[defaultIndexServer] = authConfig
 	} else {
 		for k, authConfig := range configFile.AuthConfigs {
 			authConfig.Username, authConfig.Password, err = decodeAuth(authConfig.Auth)
@@ -251,24 +251,29 @@ func decodeAuth(authStr string) (string, string, error) {
 
 // GetCredentialsStore returns a new credentials store from the settings in the
 // configuration file
-func (configFile *ConfigFile) GetCredentialsStore(serverAddress string) credentials.Store {
-	if helper := getConfiguredCredentialStore(configFile, serverAddress); helper != "" {
-		return credentials.NewNativeStore(configFile, helper)
+func (configFile *ConfigFile) GetCredentialsStore(registryHostname string) credentials.Store {
+	if helper := getConfiguredCredentialStore(configFile, registryHostname); helper != "" {
+		return newNativeStore(configFile, helper)
 	}
 	return credentials.NewFileStore(configFile)
 }
 
+// var for unit testing.
+var newNativeStore = func(configFile *ConfigFile, helperSuffix string) credentials.Store {
+	return credentials.NewNativeStore(configFile, helperSuffix)
+}
+
 // GetAuthConfig for a repository from the credential store
-func (configFile *ConfigFile) GetAuthConfig(serverAddress string) (types.AuthConfig, error) {
-	return configFile.GetCredentialsStore(serverAddress).Get(serverAddress)
+func (configFile *ConfigFile) GetAuthConfig(registryHostname string) (types.AuthConfig, error) {
+	return configFile.GetCredentialsStore(registryHostname).Get(registryHostname)
 }
 
 // getConfiguredCredentialStore returns the credential helper configured for the
 // given registry, the default credsStore, or the empty string if neither are
 // configured.
-func getConfiguredCredentialStore(c *ConfigFile, serverAddress string) string {
-	if c.CredentialHelpers != nil && serverAddress != "" {
-		if helper, exists := c.CredentialHelpers[serverAddress]; exists {
+func getConfiguredCredentialStore(c *ConfigFile, registryHostname string) string {
+	if c.CredentialHelpers != nil && registryHostname != "" {
+		if helper, exists := c.CredentialHelpers[registryHostname]; exists {
 			return helper
 		}
 	}
@@ -285,19 +290,20 @@ func (configFile *ConfigFile) GetAllCredentials() (map[string]types.AuthConfig,
 		}
 	}
 
-	for registry := range configFile.CredentialHelpers {
-		helper := configFile.GetCredentialsStore(registry)
-		newAuths, err := helper.GetAll()
-		if err != nil {
-			return nil, err
-		}
-		addAll(newAuths)
-	}
 	defaultStore := configFile.GetCredentialsStore("")
 	newAuths, err := defaultStore.GetAll()
 	if err != nil {
 		return nil, err
 	}
 	addAll(newAuths)
+
+	// Auth configs from a registry-specific helper should override those from the default store.
+	for registryHostname := range configFile.CredentialHelpers {
+		newAuth, err := configFile.GetAuthConfig(registryHostname)
+		if err != nil {
+			return nil, err
+		}
+		auths[registryHostname] = newAuth
+	}
 	return auths, nil
 }

components/cli/cli/config/configfile/file_test.go

@@ -4,6 +4,7 @@ import (
 	"fmt"
 	"testing"
 
+	"github.com/docker/cli/cli/config/credentials"
 	"github.com/docker/docker/api/types"
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
@@ -142,7 +143,37 @@ func TestConfigFile(t *testing.T) {
 	assert.Equal(t, configFilename, configFile.Filename)
 }
 
-func TestGetAllCredentials(t *testing.T) {
+type mockNativeStore struct {
+	GetAllCallCount int
+	authConfigs     map[string]types.AuthConfig
+}
+
+func (c *mockNativeStore) Erase(registryHostname string) error {
+	delete(c.authConfigs, registryHostname)
+	return nil
+}
+
+func (c *mockNativeStore) Get(registryHostname string) (types.AuthConfig, error) {
+	return c.authConfigs[registryHostname], nil
+}
+
+func (c *mockNativeStore) GetAll() (map[string]types.AuthConfig, error) {
+	c.GetAllCallCount = c.GetAllCallCount + 1
+	return c.authConfigs, nil
+}
+
+func (c *mockNativeStore) Store(authConfig types.AuthConfig) error {
+	return nil
+}
+
+// make sure it satisfies the interface
+var _ credentials.Store = (*mockNativeStore)(nil)
+
+func NewMockNativeStore(authConfigs map[string]types.AuthConfig) credentials.Store {
+	return &mockNativeStore{authConfigs: authConfigs}
+}
+
+func TestGetAllCredentialsFileStoreOnly(t *testing.T) {
 	configFile := New("filename")
 	exampleAuth := types.AuthConfig{
 		Username: "user",
@@ -157,3 +188,186 @@ func TestGetAllCredentials(t *testing.T) {
 	expected["example.com/foo"] = exampleAuth
 	assert.Equal(t, expected, authConfigs)
 }
+
+func TestGetAllCredentialsCredsStore(t *testing.T) {
+	configFile := New("filename")
+	configFile.CredentialsStore = "test_creds_store"
+	testRegistryHostname := "example.com"
+	expectedAuth := types.AuthConfig{
+		Username: "user",
+		Password: "pass",
+	}
+
+	testCredsStore := NewMockNativeStore(map[string]types.AuthConfig{testRegistryHostname: expectedAuth})
+
+	tmpNewNativeStore := newNativeStore
+	defer func() { newNativeStore = tmpNewNativeStore }()
+	newNativeStore = func(configFile *ConfigFile, helperSuffix string) credentials.Store {
+		return testCredsStore
+	}
+
+	authConfigs, err := configFile.GetAllCredentials()
+	require.NoError(t, err)
+
+	expected := make(map[string]types.AuthConfig)
+	expected[testRegistryHostname] = expectedAuth
+	assert.Equal(t, expected, authConfigs)
+	assert.Equal(t, 1, testCredsStore.(*mockNativeStore).GetAllCallCount)
+}
+
+func TestGetAllCredentialsCredHelper(t *testing.T) {
+	testCredHelperSuffix := "test_cred_helper"
+	testCredHelperRegistryHostname := "credhelper.com"
+	testExtraCredHelperRegistryHostname := "somethingweird.com"
+
+	unexpectedCredHelperAuth := types.AuthConfig{
+		Username: "file_store_user",
+		Password: "file_store_pass",
+	}
+	expectedCredHelperAuth := types.AuthConfig{
+		Username: "cred_helper_user",
+		Password: "cred_helper_pass",
+	}
+
+	configFile := New("filename")
+	configFile.CredentialHelpers = map[string]string{testCredHelperRegistryHostname: testCredHelperSuffix}
+
+	testCredHelper := NewMockNativeStore(map[string]types.AuthConfig{
+		testCredHelperRegistryHostname: expectedCredHelperAuth,
+		// Add in an extra auth entry which doesn't appear in CredentialHelpers section of the configFile.
+		// This verifies that only explicitly configured registries are being requested from the cred helpers.
+		testExtraCredHelperRegistryHostname: unexpectedCredHelperAuth,
+	})
+
+	tmpNewNativeStore := newNativeStore
+	defer func() { newNativeStore = tmpNewNativeStore }()
+	newNativeStore = func(configFile *ConfigFile, helperSuffix string) credentials.Store {
+		return testCredHelper
+	}
+
+	authConfigs, err := configFile.GetAllCredentials()
+	require.NoError(t, err)
+
+	expected := make(map[string]types.AuthConfig)
+	expected[testCredHelperRegistryHostname] = expectedCredHelperAuth
+	assert.Equal(t, expected, authConfigs)
+	assert.Equal(t, 0, testCredHelper.(*mockNativeStore).GetAllCallCount)
+}
+
+func TestGetAllCredentialsFileStoreAndCredHelper(t *testing.T) {
+	testFileStoreRegistryHostname := "example.com"
+	testCredHelperSuffix := "test_cred_helper"
+	testCredHelperRegistryHostname := "credhelper.com"
+
+	expectedFileStoreAuth := types.AuthConfig{
+		Username: "file_store_user",
+		Password: "file_store_pass",
+	}
+	expectedCredHelperAuth := types.AuthConfig{
+		Username: "cred_helper_user",
+		Password: "cred_helper_pass",
+	}
+
+	configFile := New("filename")
+	configFile.CredentialHelpers = map[string]string{testCredHelperRegistryHostname: testCredHelperSuffix}
+	configFile.AuthConfigs[testFileStoreRegistryHostname] = expectedFileStoreAuth
+
+	testCredHelper := NewMockNativeStore(map[string]types.AuthConfig{testCredHelperRegistryHostname: expectedCredHelperAuth})
+
+	newNativeStore = func(configFile *ConfigFile, helperSuffix string) credentials.Store {
+		return testCredHelper
+	}
+
+	tmpNewNativeStore := newNativeStore
+	defer func() { newNativeStore = tmpNewNativeStore }()
+	authConfigs, err := configFile.GetAllCredentials()
+	require.NoError(t, err)
+
+	expected := make(map[string]types.AuthConfig)
+	expected[testFileStoreRegistryHostname] = expectedFileStoreAuth
+	expected[testCredHelperRegistryHostname] = expectedCredHelperAuth
+	assert.Equal(t, expected, authConfigs)
+	assert.Equal(t, 0, testCredHelper.(*mockNativeStore).GetAllCallCount)
+}
+
+func TestGetAllCredentialsCredStoreAndCredHelper(t *testing.T) {
+	testCredStoreSuffix := "test_creds_store"
+	testCredStoreRegistryHostname := "credstore.com"
+	testCredHelperSuffix := "test_cred_helper"
+	testCredHelperRegistryHostname := "credhelper.com"
+
+	configFile := New("filename")
+	configFile.CredentialsStore = testCredStoreSuffix
+	configFile.CredentialHelpers = map[string]string{testCredHelperRegistryHostname: testCredHelperSuffix}
+
+	expectedCredStoreAuth := types.AuthConfig{
+		Username: "cred_store_user",
+		Password: "cred_store_pass",
+	}
+	expectedCredHelperAuth := types.AuthConfig{
+		Username: "cred_helper_user",
+		Password: "cred_helper_pass",
+	}
+
+	testCredHelper := NewMockNativeStore(map[string]types.AuthConfig{testCredHelperRegistryHostname: expectedCredHelperAuth})
+	testCredsStore := NewMockNativeStore(map[string]types.AuthConfig{testCredStoreRegistryHostname: expectedCredStoreAuth})
+
+	tmpNewNativeStore := newNativeStore
+	defer func() { newNativeStore = tmpNewNativeStore }()
+	newNativeStore = func(configFile *ConfigFile, helperSuffix string) credentials.Store {
+		if helperSuffix == testCredHelperSuffix {
+			return testCredHelper
+		}
+		return testCredsStore
+	}
+
+	authConfigs, err := configFile.GetAllCredentials()
+	require.NoError(t, err)
+
+	expected := make(map[string]types.AuthConfig)
+	expected[testCredStoreRegistryHostname] = expectedCredStoreAuth
+	expected[testCredHelperRegistryHostname] = expectedCredHelperAuth
+	assert.Equal(t, expected, authConfigs)
+	assert.Equal(t, 1, testCredsStore.(*mockNativeStore).GetAllCallCount)
+	assert.Equal(t, 0, testCredHelper.(*mockNativeStore).GetAllCallCount)
+}
+
+func TestGetAllCredentialsCredHelperOverridesDefaultStore(t *testing.T) {
+	testCredStoreSuffix := "test_creds_store"
+	testCredHelperSuffix := "test_cred_helper"
+	testRegistryHostname := "example.com"
+
+	configFile := New("filename")
+	configFile.CredentialsStore = testCredStoreSuffix
+	configFile.CredentialHelpers = map[string]string{testRegistryHostname: testCredHelperSuffix}
+
+	unexpectedCredStoreAuth := types.AuthConfig{
+		Username: "cred_store_user",
+		Password: "cred_store_pass",
+	}
+	expectedCredHelperAuth := types.AuthConfig{
+		Username: "cred_helper_user",
+		Password: "cred_helper_pass",
+	}
+
+	testCredHelper := NewMockNativeStore(map[string]types.AuthConfig{testRegistryHostname: expectedCredHelperAuth})
+	testCredsStore := NewMockNativeStore(map[string]types.AuthConfig{testRegistryHostname: unexpectedCredStoreAuth})
+
+	tmpNewNativeStore := newNativeStore
+	defer func() { newNativeStore = tmpNewNativeStore }()
+	newNativeStore = func(configFile *ConfigFile, helperSuffix string) credentials.Store {
+		if helperSuffix == testCredHelperSuffix {
+			return testCredHelper
+		}
+		return testCredsStore
+	}
+
+	authConfigs, err := configFile.GetAllCredentials()
+	require.NoError(t, err)
+
+	expected := make(map[string]types.AuthConfig)
+	expected[testRegistryHostname] = expectedCredHelperAuth
+	assert.Equal(t, expected, authConfigs)
+	assert.Equal(t, 1, testCredsStore.(*mockNativeStore).GetAllCallCount)
+	assert.Equal(t, 0, testCredHelper.(*mockNativeStore).GetAllCallCount)
+}

components/cli/cli/config/credentials/default_store_unsupported.go

@@ -2,4 +2,6 @@
 
 package credentials
 
-const defaultCredentialsStore = ""
+func defaultCredentialsStore() string {
+	return ""
+}

components/cli/cli/flags/common.go

@@ -26,6 +26,7 @@ const (
 var (
 	dockerCertPath  = os.Getenv("DOCKER_CERT_PATH")
 	dockerTLSVerify = os.Getenv("DOCKER_TLS_VERIFY") != ""
+	dockerTLS       = os.Getenv("DOCKER_TLS") != ""
 )
 
 // CommonOptions are options common to both the client and the daemon.
@@ -52,7 +53,7 @@ func (commonOpts *CommonOptions) InstallFlags(flags *pflag.FlagSet) {
 
 	flags.BoolVarP(&commonOpts.Debug, "debug", "D", false, "Enable debug mode")
 	flags.StringVarP(&commonOpts.LogLevel, "log-level", "l", "info", `Set the logging level ("debug"|"info"|"warn"|"error"|"fatal")`)
-	flags.BoolVar(&commonOpts.TLS, "tls", false, "Use TLS; implied by --tlsverify")
+	flags.BoolVar(&commonOpts.TLS, "tls", dockerTLS, "Use TLS; implied by --tlsverify")
 	flags.BoolVar(&commonOpts.TLSVerify, FlagTLSVerify, dockerTLSVerify, "Use TLS and verify the remote")
 	flags.StringVar(&commonOpts.Orchestrator, "orchestrator", "", "Which orchestrator to use with the docker cli (swarm|kubernetes) (default swarm) (experimental)")
 	flags.SetAnnotation("orchestrator", "experimentalCLI", nil)

components/cli/contrib/completion/bash/docker

@@ -178,52 +178,78 @@ __docker_complete_container_ids() {
 	COMPREPLY=( $(compgen -W "${containers[*]}" -- "$cur") )
 }
 
+# __docker_images returns a list of images. For each image, up to three representations
+# can be generated: the repository (e.g. busybox), repository:tag (e.g. busybox:latest)
+# and the ID (e.g. sha256:ee22cbbd4ea3dff63c86ba60c7691287c321e93adfc1009604eb1dde7ec88645).
+#
+# The optional arguments `--repo`, `--tag` and `--id` select the representations that
+# may be returned. Whether or not a particular representation is actually returned
+# depends on the user's customization through several environment variables:
+# - image IDs are only shown if DOCKER_COMPLETION_SHOW_IMAGE_IDS=all|non-intermediate.
+# - tags can be excluded by setting DOCKER_COMPLETION_SHOW_TAGS=no.
+# - repositories are always shown.
+#
+# In cases where an exact image specification is needed, `--force-tag` can be used.
+# It ignores DOCKER_COMPLETION_SHOW_TAGS and only lists valid repository:tag combinations,
+# avoiding repository names that would default to a potentially missing default tag.
+#
+# Additional arguments to `docker image ls` may be specified in order to filter the list,
+# e.g. `__docker_images --filter dangling=true`.
+#
 __docker_images() {
-	local images_args=""
+	local repo_format='{{.Repository}}'
+	local tag_format='{{.Repository}}:{{.Tag}}'
+	local id_format='{{.ID}}'
+	local all
+	local format
 
-	case "$DOCKER_COMPLETION_SHOW_IMAGE_IDS" in
-		all)
-			images_args="--no-trunc -a"
-			;;
-		non-intermediate)
-			images_args="--no-trunc"
-			;;
-	esac
-
-	local repo_print_command
-	if [ "${DOCKER_COMPLETION_SHOW_TAGS:-yes}" = "yes" ]; then
-		repo_print_command='print $1; print $1":"$2'
-	else
-		repo_print_command='print $1'
+	if [ "$DOCKER_COMPLETION_SHOW_IMAGE_IDS" = "all" ] ; then
+		all='--all'
 	fi
 
-	local awk_script
-	case "$DOCKER_COMPLETION_SHOW_IMAGE_IDS" in
-		all|non-intermediate)
-			awk_script='NR>1 { print $3; if ($1 != "<none>") { '"$repo_print_command"' } }'
-			;;
-		none|*)
-			awk_script='NR>1 && $1 != "<none>" { '"$repo_print_command"' }'
-			;;
-	esac
+	while true ; do
+		case "$1" in
+			--repo)
+				format+="$repo_format\n"
+				shift
+				;;
+			--tag)
+				if [ "${DOCKER_COMPLETION_SHOW_TAGS:-yes}" = "yes" ]; then
+					format+="$tag_format\n"
+				fi
+				shift
+				;;
+			--id)
+				if [[ $DOCKER_COMPLETION_SHOW_IMAGE_IDS =~ ^(all|non-intermediate)$ ]] ; then
+					format+="$id_format\n"
+				fi
+				shift
+				;;
+			--force-tag)
+				# like `--tag` but ignores environment setting
+				format+="$tag_format\n"
+				shift
+				;;
+			*)
+				break
+				;;
+		esac
+	done
 
-	__docker_q images $images_args | awk "$awk_script" | grep -v '<none>$'
+	__docker_q image ls --no-trunc --format "${format%\\n}" $all "$@" | grep -v '<none>$'
 }
 
+# __docker_complete_images applies completion of images based on the current value of `$cur` or
+# the value of the optional first option `--cur`, if given.
+# See __docker_images for customization of the returned items.
 __docker_complete_images() {
-	COMPREPLY=( $(compgen -W "$(__docker_images)" -- "$cur") )
-	__ltrim_colon_completions "$cur"
-}
-
-__docker_complete_image_repos() {
-	local repos="$(__docker_q images | awk 'NR>1 && $1 != "<none>" { print $1 }')"
-	COMPREPLY=( $(compgen -W "$repos" -- "$cur") )
-}
-
-__docker_complete_image_repos_and_tags() {
-	local reposAndTags="$(__docker_q images | awk 'NR>1 && $1 != "<none>" { print $1; print $1":"$2 }')"
-	COMPREPLY=( $(compgen -W "$reposAndTags" -- "$cur") )
-	__ltrim_colon_completions "$cur"
+	local current="$cur"
+	if [ "$1" = "--cur" ] ; then
+		current="$2"
+		shift 2
+	fi
+	COMPREPLY=( $(compgen -W "$(__docker_images "$@")" -- "$current") )
+	__ltrim_colon_completions "$current"
 }
 
 # __docker_networks returns a list of all networks. Additional options to
@@ -1354,11 +1380,8 @@ _docker_container_commit() {
 			if [ "$cword" -eq "$counter" ]; then
 				__docker_complete_containers_all
 				return
-			fi
-			(( counter++ ))
-
-			if [ "$cword" -eq "$counter" ]; then
-				__docker_complete_image_repos_and_tags
+			elif [ "$cword" -eq "$((counter + 1))" ]; then
+				__docker_complete_images --repo --tag
 				return
 			fi
 			;;
@@ -1529,8 +1552,7 @@ _docker_container_ls() {
 	local key=$(__docker_map_key_of_current_option '--filter|-f')
 	case "$key" in
 		ancestor)
-			cur="${cur##*=}"
-			__docker_complete_images
+			__docker_complete_images --cur "${cur##*=}" --repo --tag --id
 			return
 			;;
 		before)
@@ -1998,7 +2020,7 @@ _docker_container_run_and_create() {
 		*)
 			local counter=$( __docker_pos_first_nonflag "$( __docker_to_alternatives "$options_with_args" )" )
 			if [ "$cword" -eq "$counter" ]; then
-				__docker_complete_images
+				__docker_complete_images --repo --tag --id
 			fi
 			;;
 	esac
@@ -2513,7 +2535,7 @@ _docker_image_build() {
 			return
 			;;
 		--cache-from)
-			__docker_complete_image_repos_and_tags
+			__docker_complete_images --repo --tag --id
 			return
 			;;
 		--file|-f|--iidfile)
@@ -2541,7 +2563,7 @@ _docker_image_build() {
 			return
 			;;
 		--tag|-t)
-			__docker_complete_image_repos_and_tags
+			__docker_complete_images --repo --tag
 			return
 			;;
 		--target)
@@ -2587,9 +2609,9 @@ _docker_image_history() {
 			COMPREPLY=( $( compgen -W "--format --help --human=false -H=false --no-trunc --quiet -q" -- "$cur" ) )
 			;;
 		*)
-			local counter=$(__docker_pos_first_nonflag)
+			local counter=$(__docker_pos_first_nonflag '--format')
 			if [ "$cword" -eq "$counter" ]; then
-				__docker_complete_images
+				__docker_complete_images --force-tag --id
 			fi
 			;;
 	esac
@@ -2613,12 +2635,10 @@ _docker_image_import() {
 		*)
 			local counter=$(__docker_pos_first_nonflag '--change|-c|--message|-m')
 			if [ "$cword" -eq "$counter" ]; then
+				_filedir
 				return
-			fi
-			(( counter++ ))
-
-			if [ "$cword" -eq "$counter" ]; then
-				__docker_complete_image_repos_and_tags
+			elif [ "$cword" -eq "$((counter + 1))" ]; then
+				__docker_complete_images --repo --tag
 				return
 			fi
 			;;
@@ -2651,9 +2671,8 @@ _docker_image_list() {
 _docker_image_ls() {
 	local key=$(__docker_map_key_of_current_option '--filter|-f')
 	case "$key" in
-		before|since|reference)
-			cur="${cur##*=}"
-			__docker_complete_images
+		before|since)
+			__docker_complete_images --cur "${cur##*=}" --force-tag --id
 			return
 			;;
 		dangling)
@@ -2663,6 +2682,10 @@ _docker_image_ls() {
 		label)
 			return
 			;;
+		reference)
+			__docker_complete_images --cur "${cur##*=}" --repo --tag
+			return
+			;;
 	esac
 
 	case "$prev" in
@@ -2684,7 +2707,7 @@ _docker_image_ls() {
 			return
 			;;
 		*)
-			__docker_complete_image_repos
+			__docker_complete_images --repo --tag
 			;;
 	esac
 }
@@ -2725,12 +2748,12 @@ _docker_image_pull() {
 				for arg in "${COMP_WORDS[@]}"; do
 					case "$arg" in
 						--all-tags|-a)
-							__docker_complete_image_repos
+							__docker_complete_images --repo
 							return
 							;;
 					esac
 				done
-				__docker_complete_image_repos_and_tags
+				__docker_complete_images --repo --tag
 			fi
 			;;
 	esac
@@ -2744,7 +2767,7 @@ _docker_image_push() {
 		*)
 			local counter=$(__docker_pos_first_nonflag)
 			if [ "$cword" -eq "$counter" ]; then
-				__docker_complete_image_repos_and_tags
+				__docker_complete_images --repo --tag
 			fi
 			;;
 	esac
@@ -2760,7 +2783,7 @@ _docker_image_rm() {
 			COMPREPLY=( $( compgen -W "--force -f --help --no-prune" -- "$cur" ) )
 			;;
 		*)
-			__docker_complete_images
+			__docker_complete_images --force-tag --id
 			;;
 	esac
 }
@@ -2782,7 +2805,7 @@ _docker_image_save() {
 			COMPREPLY=( $( compgen -W "--help --output -o" -- "$cur" ) )
 			;;
 		*)
-			__docker_complete_images
+			__docker_complete_images --repo --tag --id
 			;;
 	esac
 }
@@ -2796,13 +2819,10 @@ _docker_image_tag() {
 			local counter=$(__docker_pos_first_nonflag)
 
 			if [ "$cword" -eq "$counter" ]; then
-				__docker_complete_image_repos_and_tags
+				__docker_complete_images --force-tag --id
 				return
-			fi
-			(( counter++ ))
-
-			if [ "$cword" -eq "$counter" ]; then
-				__docker_complete_image_repos_and_tags
+			elif [ "$cword" -eq "$((counter + 1))" ]; then
+				__docker_complete_images --repo --tag
 				return
 			fi
 			;;
@@ -2858,7 +2878,7 @@ _docker_inspect() {
 				'')
 					COMPREPLY=( $( compgen -W "
 						$(__docker_containers --all)
-						$(__docker_images)
+						$(__docker_images --force-tag --id)
 						$(__docker_networks)
 						$(__docker_nodes)
 						$(__docker_plugins_installed)
@@ -2872,7 +2892,7 @@ _docker_inspect() {
 					__docker_complete_containers_all
 					;;
 				image)
-					__docker_complete_images
+					__docker_complete_images --force-tag --id
 					;;
 				network)
 					__docker_complete_networks
@@ -3334,7 +3354,6 @@ _docker_service_update_and_create() {
 	local options_with_args="
 		--endpoint-mode
 		--entrypoint
-		--env -e
 		--force
 		--health-cmd
 		--health-interval
@@ -3343,12 +3362,10 @@ _docker_service_update_and_create() {
 		--health-timeout
 		--hostname
 		--isolation
-		--label -l
 		--limit-cpu
 		--limit-memory
 		--log-driver
 		--log-opt
-		--mount
 		--replicas
 		--reserve-cpu
 		--reserve-memory
@@ -3396,11 +3413,14 @@ _docker_service_update_and_create() {
 			--dns
 			--dns-option
 			--dns-search
+			--env -e
 			--env-file
 			--generic-resource
 			--group
 			--host
+			--label -l
 			--mode
+			--mount
 			--name
 			--network
 			--placement-pref
@@ -3409,39 +3429,14 @@ _docker_service_update_and_create() {
 		"
 
 		case "$prev" in
-			--config)
-				__docker_complete_configs
-				return
-				;;
 			--env-file)
 				_filedir
 				return
 				;;
-			--group)
-				COMPREPLY=( $(compgen -g -- "$cur") )
-				return
-				;;
-			--host)
-				case "$cur" in
-					*:)
-						__docker_complete_resolved_hostname
-						return
-						;;
-				esac
-				;;
 			--mode)
 				COMPREPLY=( $( compgen -W "global replicated" -- "$cur" ) )
 				return
 				;;
-			--placement-pref)
-				COMPREPLY=( $( compgen -W "spread" -S = -- "$cur" ) )
-				__docker_nospace
-				return
-				;;
-			--secret)
-				__docker_complete_secrets
-				return
-				;;
 		esac
 	fi
 	if [ "$subcommand" = "update" ] ; then
@@ -3459,6 +3454,8 @@ _docker_service_update_and_create() {
 			--dns-rm
 			--dns-search-add
 			--dns-search-rm
+			--env-add
+			--env-rm
 			--generic-resource-add
 			--generic-resource-rm
 			--group-add
@@ -3466,6 +3463,10 @@ _docker_service_update_and_create() {
 			--host-add
 			--host-rm
 			--image
+			--label-add
+			--label-rm
+			--mount-add
+			--mount-rm
 			--network-add
 			--network-rm
 			--placement-pref-add
@@ -3478,37 +3479,12 @@ _docker_service_update_and_create() {
 		"
 
 		case "$prev" in
-			--config-add|--config-rm)
-				__docker_complete_configs
+			--env-rm)
+				COMPREPLY=( $( compgen -e -- "$cur" ) )
 				return
 				;;
-			--group-add|--group-rm)
-				COMPREPLY=( $(compgen -g -- "$cur") )
-				return
-				;;
-			--host-add|--host-rm)
-				case "$cur" in
-					*:)
-						__docker_complete_resolved_hostname
-						return
-						;;
-				esac
-				;;
 			--image)
-				__docker_complete_image_repos_and_tags
-				return
-				;;
-			--network-add|--network-rm)
-				__docker_complete_networks
-				return
-				;;
-			--placement-pref-add|--placement-pref-rm)
-				COMPREPLY=( $( compgen -W "spread" -S = -- "$cur" ) )
-				__docker_nospace
-				return
-				;;
-			--secret-add|--secret-rm)
-				__docker_complete_secrets
+				__docker_complete_images --repo --tag --id
 				return
 				;;
 		esac
@@ -3524,16 +3500,32 @@ _docker_service_update_and_create() {
 	esac
 
 	case "$prev" in
+		--config|--config-add|--config-rm)
+			__docker_complete_configs
+			return
+			;;
 		--endpoint-mode)
 			COMPREPLY=( $( compgen -W "dnsrr vip" -- "$cur" ) )
 			return
 			;;
-		--env|-e)
+		--env|-e|--env-add)
 			# we do not append a "=" here because "-e VARNAME" is legal systax, too
 			COMPREPLY=( $( compgen -e -- "$cur" ) )
 			__docker_nospace
 			return
 			;;
+		--group|--group-add|--group-rm)
+			COMPREPLY=( $(compgen -g -- "$cur") )
+			return
+			;;
+		--host|--host-add|--host-rm)
+			case "$cur" in
+				*:)
+					__docker_complete_resolved_hostname
+					return
+					;;
+			esac
+			;;
 		--isolation)
 			__docker_complete_isolation
 			return
@@ -3546,10 +3538,15 @@ _docker_service_update_and_create() {
 			__docker_complete_log_options
 			return
 			;;
-		--network)
+		--network|--network-add|--network-rm)
 			__docker_complete_networks
 			return
 			;;
+		--placement-pref|--placement-pref-add|--placement-pref-rm)
+			COMPREPLY=( $( compgen -W "spread" -S = -- "$cur" ) )
+			__docker_nospace
+			return
+			;;
 		--restart-condition)
 			COMPREPLY=( $( compgen -W "any none on-failure" -- "$cur" ) )
 			return
@@ -3558,6 +3555,10 @@ _docker_service_update_and_create() {
 			COMPREPLY=( $( compgen -W "continue pause" -- "$cur" ) )
 			return
 			;;
+		--secret|--secret-add|--secret-rm)
+			__docker_complete_secrets
+			return
+			;;
 		--stop-signal)
 			__docker_complete_signals
 			return
@@ -3591,7 +3592,7 @@ _docker_service_update_and_create() {
 				fi
 			else
 				if [ "$cword" -eq "$counter" ]; then
-					__docker_complete_images
+					__docker_complete_images --repo --tag --id
 				fi
 			fi
 			;;
@@ -4639,8 +4640,7 @@ _docker_system_events() {
 			return
 			;;
 		image)
-			cur="${cur##*=}"
-			__docker_complete_images
+			__docker_complete_images --cur "${cur##*=}" --repo --tag
 			return
 			;;
 		network)
@@ -4648,7 +4648,7 @@ _docker_system_events() {
 			return
 			;;
 		type)
-			COMPREPLY=( $( compgen -W "container daemon image network plugin volume" -- "${cur##*=}" ) )
+			COMPREPLY=( $( compgen -W "config container daemon image network plugin secret service volume" -- "${cur##*=}" ) )
 			return
 			;;
 		volume)
@@ -4713,9 +4713,9 @@ _docker_tag() {
 
 _docker_trust() {
 	local subcommands="
+		inspect
 		revoke
 		sign
-		view
 	"
 	__docker_subcommands "$subcommands" && return
 
@@ -4729,6 +4729,20 @@ _docker_trust() {
 	esac
 }
 
+_docker_trust_inspect() {
+	case "$cur" in
+		-*)
+			COMPREPLY=( $( compgen -W "--help --pretty" -- "$cur" ) )
+			;;
+		*)
+			local counter=$(__docker_pos_first_nonflag)
+			if [ "$cword" -eq "$counter" ]; then
+				__docker_complete_images --repo --tag
+			fi
+			;;
+	esac
+}
+
 _docker_trust_revoke() {
 	case "$cur" in
 		-*)
@@ -4737,7 +4751,7 @@ _docker_trust_revoke() {
 		*)
 			local counter=$(__docker_pos_first_nonflag)
 			if [ "$cword" -eq "$counter" ]; then
-				__docker_complete_images
+				__docker_complete_images --repo --tag
 			fi
 			;;
 	esac
@@ -4751,21 +4765,7 @@ _docker_trust_sign() {
 		*)
 			local counter=$(__docker_pos_first_nonflag)
 			if [ "$cword" -eq "$counter" ]; then
-				__docker_complete_images
-			fi
-			;;
-	esac
-}
-
-_docker_trust_view() {
-	case "$cur" in
-		-*)
-			COMPREPLY=( $( compgen -W "--help" -- "$cur" ) )
-			;;
-		*)
-			local counter=$(__docker_pos_first_nonflag)
-			if [ "$cword" -eq "$counter" ]; then
-				__docker_complete_images
+				__docker_complete_images --force-tag --id
 			fi
 			;;
 	esac
@@ -4947,6 +4947,7 @@ _docker() {
 		stack
 		swarm
 		system
+		trust
 		volume
 	)
 
@@ -4999,7 +5000,6 @@ _docker() {
 	local experimental_commands=(
 		checkpoint
 		deploy
-		trust
 	)
 
 	local commands=(${management_commands[*]} ${top_level_commands[*]})

components/cli/contrib/completion/zsh/_docker

@@ -450,9 +450,9 @@ __docker_complete_events_filter() {
                 ;;
             (event)
                 local -a event_opts
-                event_opts=('attach' 'commit' 'connect' 'copy' 'create' 'delete' 'destroy' 'detach' 'die' 'disconnect' 'exec_create' 'exec_detach'
-                'exec_start' 'export' 'health_status' 'import' 'kill' 'load'  'mount' 'oom' 'pause' 'pull' 'push' 'reload' 'rename' 'resize' 'restart' 'save' 'start'
-                'stop' 'tag' 'top' 'unmount' 'unpause' 'untag' 'update')
+                event_opts=('attach' 'commit' 'connect' 'copy' 'create' 'delete' 'destroy' 'detach' 'die' 'disable' 'disconnect' 'enable' 'exec_create' 'exec_detach'
+                'exec_start' 'export' 'health_status' 'import' 'install' 'kill' 'load'  'mount' 'oom' 'pause' 'pull' 'push' 'reload' 'remove' 'rename' 'resize'
+                'restart' 'save' 'start' 'stop' 'tag' 'top' 'unmount' 'unpause' 'untag' 'update')
                 _describe -t event-filter-opts "event filter options" event_opts && ret=0
                 ;;
             (image)

components/cli/dockerfiles/Dockerfile.binary-native

@@ -1,4 +1,4 @@
-FROM    golang:1.9.2-alpine3.6
+FROM    golang:1.9.4-alpine3.6
 
 RUN     apk add -U git bash coreutils gcc musl-dev
 

components/cli/dockerfiles/Dockerfile.cross

@@ -1,3 +1,3 @@
-FROM    dockercore/golang-cross:1.9.3@sha256:2ac6046dd738cf83a7557a9fc2be52accb97c103c5e9d2c2a50daa797c8eb79f
+FROM    dockercore/golang-cross:1.9.4@sha256:b8d43ef11ccaa15bec63a1f1fd0c28a0e729074aa62fcfa51f0a5888f3571315
 ENV     DISABLE_WARN_OUTSIDE_CONTAINER=1
 WORKDIR /go/src/github.com/docker/cli

components/cli/dockerfiles/Dockerfile.dev

@@ -1,5 +1,5 @@
 
-FROM    golang:1.9.3-alpine3.6
+FROM    golang:1.9.4-alpine3.6
 
 RUN     apk add -U git make bash coreutils ca-certificates
 
@@ -10,11 +10,11 @@ RUN     go get -d github.com/LK4D4/vndr && \
         go build -v -o /usr/bin/vndr . && \
         rm -rf /go/src/* /go/pkg/* /go/bin/*
 
-ARG     BINDATA_SHA=a0ff2567cfb70903282db057e799fd826784d41d
-RUN     go get -d github.com/jteeuwen/go-bindata/go-bindata && \
-        cd /go/src/github.com/jteeuwen/go-bindata/go-bindata && \
-        git checkout -q "$BINDATA_SHA" && \
-        go build -v -o /usr/bin/go-bindata . && \
+ARG     ESC_SHA=58d9cde84f237ecdd89bd7f61c2de2853f4c5c6e
+RUN     go get -d github.com/mjibson/esc && \
+        cd /go/src/github.com/mjibson/esc && \
+        git checkout -q "$ESC_SHA" && \
+        go build -v -o /usr/bin/esc . && \
         rm -rf /go/src/* /go/pkg/* /go/bin/*
 
 ARG     FILEWATCHER_SHA=2e12ea42f6c8c089b19e992145bb94e8adaecedb

components/cli/dockerfiles/Dockerfile.lint

@@ -1,4 +1,4 @@
-FROM    golang:1.9.3-alpine3.6
+FROM    golang:1.9.4-alpine3.6
 
 RUN     apk add -U git
 

components/cli/docs/extend/plugins_network.md

@@ -26,18 +26,13 @@ LibNetwork, which shares plugin infrastructure with Engine. Effectively, network
 driver plugins are activated in the same way as other plugins, and use the same
 kind of protocol.
 
-## Network driver plugins and swarm mode
+## Network plugins and swarm mode
 
-Docker 1.12 adds support for cluster management and orchestration called
-[swarm mode](https://docs.docker.com/engine/swarm/). Docker Engine running in swarm mode currently
-only supports the built-in overlay driver for networking. Therefore existing
-networking plugins will not work in swarm mode.
+[Legacy plugins](legacy_plugins.md) do not work in swarm mode. However,
+plugins written using the [v2 plugin system](index.md) do work in swarm mode, as
+long as they are installed on each swarm worker node.
 
-When you run Docker Engine outside of swarm mode, all networking plugins that
-worked in Docker 1.11 will continue to function normally. They do not require
-any modification.
-
-## Using network driver plugins
+## Use network driver plugins
 
 The means of installing and running a network driver plugin depend on the
 particular plugin. So, be sure to install your plugin according to the
@@ -57,6 +52,13 @@ referring to that network will be sent to the plugin,
     $ docker run --network=mynet busybox top
 
 
+## Find network plugins
+
+Network plugins are written by third parties, and are published by those
+third parties, either on
+[Docker Store](https://store.docker.com/search?category=network&q=&type=plugin)
+or on the third party's site.
+
 ## Write a network plugin
 
 Network plugins implement the [Docker plugin
@@ -68,7 +70,7 @@ The network driver protocol, in addition to the plugin activation call, is
 documented as part of libnetwork:
 [https://github.com/docker/libnetwork/blob/master/docs/remote.md](https://github.com/docker/libnetwork/blob/master/docs/remote.md).
 
-# Related Information
+## Related Information
 
 To interact with the Docker maintainers and other interested users, see the IRC channel `#docker-network`.
 

components/cli/docs/reference/builder.md

@@ -1003,7 +1003,7 @@ whitespace)
 > and will not work on Windows containers. Since user and group ownership concepts do
 > not translate between Linux and Windows, the use of `/etc/passwd` and `/etc/group` for
 > translating user and group names to IDs restricts this feature to only be viable for
-> for Linux OS-based containers.
+> Linux OS-based containers.
 
 The `COPY` instruction copies new files or directories from `<src>`
 and adds them to the filesystem of the container at the path `<dest>`.

components/cli/docs/reference/commandline/cli.md

@@ -66,6 +66,7 @@ by the `docker` command line:
 * `DOCKER_NOWARN_KERNEL_VERSION` Prevent warnings that your Linux kernel is
   unsuitable for Docker.
 * `DOCKER_RAMDISK` If set this will disable 'pivot_root'.
+* `DOCKER_TLS` When set Docker uses TLS.
 * `DOCKER_TLS_VERIFY` When set Docker uses TLS and verifies the remote.
 * `DOCKER_CONTENT_TRUST` When set Docker uses notary to sign and verify images.
   Equates to `--disable-content-trust=false` for build, create, pull, push, run.

components/cli/docs/reference/commandline/events.md

@@ -179,7 +179,7 @@ The currently supported filters are:
 * container (`container=<name or id>`)
 * daemon (`daemon=<name or id>`)
 * event (`event=<event action>`)
-* image (`image=<tag or id>`)
+* image (`image=<repository or tag>`)
 * label (`label=<key>` or `label=<key>=<value>`)
 * network (`network=<name or id>`)
 * node (`node=<id>`)

components/cli/docs/reference/commandline/image_prune.md

@@ -95,6 +95,19 @@ which removes images with the specified labels. The other
 format is the `label!=...` (`label!=<key>` or `label!=<key>=<value>`), which removes
 images without the specified labels.
 
+> **Predicting what will be removed**
+>
+> If you are using positive filtering (testing for the existence of a label or
+> that a label has a specific value), you can use `docker image ls` with the
+> same filtering syntax to see which images match your filter.
+>
+> However, if you are using negative filtering (testing for the absence of a
+> label or that a label does *not* have a specific value), this type of filter
+> does not work with `docker image ls` so you cannot easily predict which images
+> will be removed. In addition, the confirmation prompt for `docker image prune`
+> always warns that *all* dangling images will be removed, even if you are using
+> `--filter`.
+
 The following removes images created before `2017-01-04T00:00:00`:
 
 ```bash
@@ -162,6 +175,35 @@ alpine              latest              88e169ea8f46        8 days ago
 busybox             latest              e02e811dd08f        2 months ago         1.09 MB
 ```
 
+The following example removes images with the label `deprecated`:
+
+```bash
+$ docker image prune --filter="label=deprecated"
+```
+
+The following example removes images with the label `maintainer` set to `john`:
+
+```bash
+$ docker image prune --filter="label=maintainer=john"
+```
+
+This example removes images which have no `maintainer` label:
+
+```bash
+$ docker image prune --filter="label!=maintainer"
+```
+
+This example removes images which have a maintainer label not set to `john`:
+
+```bash
+$ docker image prune --filter="label!=maintainer=john"
+```
+
+> **Note**: You are prompted for confirmation before the `prune` removes
+> anything, but you are not shown a list of what will potentially be removed.
+> In addition, `docker image ls` does not support negative filtering, so it
+> difficult to predict what images will actually be removed.
+
 ## Related commands
 
 * [system df](system_df.md)

components/cli/docs/reference/commandline/index.md

@@ -110,7 +110,7 @@ read the [`dockerd`](dockerd.md) reference page.
 | [volume create](volume_create.md) | Creates a new volume where containers can consume and store data |
 | [volume inspect](volume_inspect.md) | Display information about a volume     |
 | [volume ls](volume_ls.md) | Lists all the volumes Docker knows about         |
-| [volume prune](volume_prune.md) | Remove all unused volumes                  |
+| [volume prune](volume_prune.md) | Remove all unused local volumes            |
 | [volume rm](volume_rm.md) | Remove one or more volumes                       |
 
 ### Swarm node commands

components/cli/docs/reference/commandline/kill.md

@@ -27,9 +27,45 @@ Options:
 
 ## Description
 
-The main process inside the container will be sent `SIGKILL`, or any
-signal specified with option `--signal`.
+The `docker kill` subcommand kills one or more containers. The main process
+inside the container is sent `SIGKILL` signal (default), or the signal that is
+specified with the `--signal` option. You can kill a container using the
+container's ID, ID-prefix, or name.
 
 > **Note**: `ENTRYPOINT` and `CMD` in the *shell* form run as a subcommand of
 > `/bin/sh -c`, which does not pass signals. This means that the executable is
 > not the container’s PID 1 and does not receive Unix signals.
+
+## Examples
+
+
+### Send a KILL signal  to a container
+
+The following example sends the default `KILL` signal to the container named
+`my_container`:
+
+```bash
+$ docker kill my_container
+```
+
+### Send a custom signal  to a container
+
+The following example sends a `SIGHUP` signal to the container named
+`my_container`:
+
+```bash
+$ docker kill --signal=SIGHUP  my_container
+```
+
+
+You can specify a custom signal either by _name_, or _number_. The `SIG` prefix
+is optional, so the following examples are equivalent:
+
+```bash
+$ docker kill --signal=SIGHUP my_container
+$ docker kill --signal=HUP my_container
+$ docker kill --signal=1 my_container
+```
+
+Refer to the [`signal(7)`](http://man7.org/linux/man-pages/man7/signal.7.html)
+man-page for a list of standard Linux signals.

components/cli/docs/reference/commandline/network_create.md

@@ -92,6 +92,18 @@ Network names must be unique. The Docker daemon attempts to identify naming
 conflicts but this is not guaranteed. It is the user's responsibility to avoid
 name conflicts.
 
+### Overlay network limitations
+
+You should create overlay networks with `/24` blocks (the default), which limits
+you to 256 IP addresses, when you create networks using the default VIP-based
+endpoint-mode. This recommendation addresses
+[limitations with swarm mode](https://github.com/moby/moby/issues/30820). If you
+need more than 256 IP addresses, do not increase the IP block size. You can
+either use `dnsrr` endpoint mode with an external load balancer, or use multiple
+smaller overlay networks. See
+[Configure service discovery](https://docs.docker.com/engine/swarm/networking/#configure-service-discovery)
+for more information about different endpoint modes.
+
 ## Examples
 
 ### Connect containers
@@ -141,15 +153,16 @@ $ docker network create \
 
 If you omit the `--gateway` flag the Engine selects one for you from inside a
 preferred pool. For `overlay` networks and for network driver plugins that
-support it you can create multiple subnetworks.
+support it you can create multiple subnetworks. This example uses two `/25`
+subnet mask to adhere to the current guidance of not having more than 256 IPs in
+a single overlay network. Each of the subnetworks has 126 usable addresses.
 
 ```bash
 $ docker network create -d overlay \
-  --subnet=192.168.0.0/16 \
-  --subnet=192.170.0.0/16 \
-  --gateway=192.168.0.100 \
-  --gateway=192.170.0.100 \
-  --ip-range=192.168.1.0/24 \
+  --subnet=192.168.1.0/25 \
+  --subnet=192.170.2.0/25 \
+  --gateway=192.168.1.100 \
+  --gateway=192.170.2.100 \
   --aux-address="my-router=192.168.1.5" --aux-address="my-switch=192.168.1.6" \
   --aux-address="my-printer=192.170.1.5" --aux-address="my-nas=192.170.1.6" \
   my-multihost-network

components/cli/docs/reference/commandline/node_ls.md

@@ -146,6 +146,7 @@ Placeholder      | Description
 `.Availability`  | Node availability ("active", "pause", or "drain")
 `.ManagerStatus` | Manager status of the node
 `.TLSStatus`     | TLS status of the node ("Ready", or "Needs Rotation" has TLS certificate signed by an old CA)
+`.EngineVersion` | Engine version
 
 When using the `--format` option, the `node ls` command will either
 output the data exactly as the template declares or, when using the

components/cli/docs/reference/commandline/run.md

@@ -343,12 +343,12 @@ $ docker run -t -i --mount type=bind,src=/data,dst=/data busybox sh
 ### Publish or expose port (-p, --expose)
 
 ```bash
-$ docker run -p 127.0.0.1:80:8080 ubuntu bash
+$ docker run -p 127.0.0.1:80:8080/tcp ubuntu bash
 ```
 
-This binds port `8080` of the container to port `80` on `127.0.0.1` of the host
-machine. The [Docker User
-Guide](https://docs.docker.com/engine/userguide/networking/default_network/dockerlinks/)
+This binds port `8080` of the container to TCP port `80` on `127.0.0.1` of the host
+machine. You can also specify `udp` and `sctp` ports.
+The [Docker User Guide](https://docs.docker.com/engine/userguide/networking/default_network/dockerlinks/)
 explains in detail how to manipulate ports in Docker.
 
 ```bash
@@ -592,6 +592,7 @@ Docker supports the following restart policies:
 |:---------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | `no`                       | Do not automatically restart the container when it exits. This is the default.                                                                                                                                                                                   |
 | `on-failure[:max-retries]` | Restart only if the container exits with a non-zero exit status. Optionally, limit the number of restart retries the Docker daemon attempts.                                                                                                                     |
+| `unless-stopped`           | Restart the container unless it is explicitly stopped or Docker itself is stopped or restarted. |
 | `always`                   | Always restart the container regardless of the exit status. When you specify always, the Docker daemon will try to restart the container indefinitely. The container will also always start on daemon startup, regardless of the current state of the container. |
 
 ```bash

components/cli/docs/reference/commandline/secret_create.md

@@ -16,13 +16,13 @@ keywords: ["secret, create"]
 # secret create
 
 ```Markdown
-Usage:	docker secret create [OPTIONS] SECRET file|-
+Usage:	docker secret create [OPTIONS] SECRET [file|-]
 
 Create a secret from a file or STDIN as content
 
 Options:
-      --help          Print usage
-  -l, --label list    Secret labels (default [])
+  -l, --label list               Secret labels
+      --template-driver string   Template driver
 ```
 
 ## Description