Merge pull request #347 from seemethere/cherry_pick_packaging_70

[17.12] Pass VERSION to engine static builds

CHANGELOG.md

@@ -1 +1,88 @@
 # Changelog
+
+Items starting with `DEPRECATE` are important deprecation notices. For more
+information on the list of deprecated flags and APIs please have a look at
+https://docs.docker.com/engine/deprecated/ where target removal dates can also
+be found.
+
+## 17.12.0-ce (2017-12-DD)
+
+
+### Builder
+
+- Fix build cache hash for broken symlink [moby/moby#34271](https://github.com/moby/moby/pull/34271)
+- Fix long stream sync [moby/moby#35404](https://github.com/moby/moby/pull/35404)
+- Fix dockerfile parser failing silently on long tokens [moby/moby#35429](https://github.com/moby/moby/pull/35429)
+
+### Client
+
+* Remove secret/config duplication in cli/compose [docker/cli#671](https://github.com/docker/cli/pull/671)
+* Add `--local` flag to `docker trust sign` [docker/cli#575](https://github.com/docker/cli/pull/575)
+* Add `docker trust inspect` [docker/cli#694](https://github.com/docker/cli/pull/694)
++ Add `name` field to secrets and configs to allow interpolation in Compose files [docker/cli#668](https://github.com/docker/cli/pull/668)
++ Add `--isolation` for setting swarm service isolation mode [docker/cli#426](https://github.com/docker/cli/pull/426)
+* Remove deprecated "daemon" subcommand [docker/cli#689](https://github.com/docker/cli/pull/689)
+- Fix behaviour of `rmi -f` with unexpected errors [docker/cli#654](https://github.com/docker/cli/pull/654)
+* Integrated Generic resource in service create [docker/cli#429](https://github.com/docker/cli/pull/429)
+
+### Documentation
+
+* Update API version history for 1.35 [moby/moby#35724](https://github.com/moby/moby/pull/35724)
+
+### Logging
+
+* Logentries driver line-only=true []byte output fix [moby/moby#35612](https://github.com/moby/moby/pull/35612)
+* Logentries line-only logopt fix to maintain backwards compatibility [moby/moby#35628](https://github.com/moby/moby/pull/35628)
++ Add `--until` flag for docker logs [moby/moby#32914](https://github.com/moby/moby/pull/32914)
++ Add gelf log driver plugin to Windows build [moby/moby#35073](https://github.com/moby/moby/pull/35073)
+* Set timeout on splunk batch send [moby/moby#35496](https://github.com/moby/moby/pull/35496)
+
+### Networking
+
+* Move load balancer sandbox creation/deletion into libnetwork [moby/moby#35422](https://github.com/moby/moby/pull/35422)
+* Only chown network files within container metadata [moby/moby#34224](https://github.com/moby/moby/pull/34224)
+* Restore error type in FindNetwork [moby/moby#35634](https://github.com/moby/moby/pull/35634)
+- Fix consumes MIME type for NetworkConnect [moby/moby#35542](https://github.com/moby/moby/pull/35542)
++ Added support for persisting Windows network driver specific options [moby/moby#35563](https://github.com/moby/moby/pull/35563)
+- Fix timeout on netlink sockets and watchmiss leak [moby/moby#35677](https://github.com/moby/moby/pull/35677)
++ New daemon config for networking diagnosis [moby/moby#35677](https://github.com/moby/moby/pull/35677)
+
+### Runtime
+
+* Update to containerd v1.0.0 [moby/moby#35707](https://github.com/moby/moby/pull/35707)
+* Have VFS graphdriver use accelerated in-kernel copy [moby/moby#35537](https://github.com/moby/moby/pull/35537)
+* Introduce `workingdir` option for docker exec [moby/moby#35661](https://github.com/moby/moby/pull/35661)
+* Bump Go to 1.9.2 [moby/moby#33892](https://github.com/moby/moby/pull/33892) [docker/cli#716](https://github.com/docker/cli/pull/716)
+* `/dev` should not be readonly with `--readonly` flag [moby/moby#35344](https://github.com/moby/moby/pull/35344)
++ Add custom build-time Graphdrivers priority list [moby/moby#35522](https://github.com/moby/moby/pull/35522)
+* LCOW: CLI changes to add platform flag - pull, run, create and build [docker/cli#474](https://github.com/docker/cli/pull/474)
+* Fix width/height on Windoes for `docker exec` [moby/moby#35631](https://github.com/moby/moby/pull/35631)
+* Detect overlay2 support on pre-4.0 kernels [moby/moby#35527](https://github.com/moby/moby/pull/35527)
+* Devicemapper: remove container rootfs mountPath after umount [moby/moby#34573](https://github.com/moby/moby/pull/34573)
+* Disallow overlay/overlay2 on top of NFS [moby/moby#35483](https://github.com/moby/moby/pull/35483)
+- Fix potential panic during plugin set. [moby/moby#35632](https://github.com/moby/moby/pull/35632)
+- Fix some issues with locking on the container [moby/moby#35501](https://github.com/moby/moby/pull/35501)
+- Fixup some issues with plugin refcounting [moby/moby#35265](https://github.com/moby/moby/pull/35265)
++ Add missing lock in ProcessEvent [moby/moby#35516](https://github.com/moby/moby/pull/35516)
++ Add vfs quota support [moby/moby#35231](https://github.com/moby/moby/pull/35231)
+* Skip empty directories on prior graphdriver detection [moby/moby#35528](https://github.com/moby/moby/pull/35528)
+* Skip xfs quota tests when running in user namespace [moby/moby#35526](https://github.com/moby/moby/pull/35526)
++ Added SubSecondPrecision to config option. [moby/moby#35529](https://github.com/moby/moby/pull/35529)
+* Update fsnotify to fix deadlock in removing watch [moby/moby#35453](https://github.com/moby/moby/pull/35453)
+- Fix "duplicate mount point" when `--tmpfs /dev/shm` is used [moby/moby#35467](https://github.com/moby/moby/pull/35467)
+- Fix honoring tmpfs-size for user `/dev/shm` mount [moby/moby#35316](https://github.com/moby/moby/pull/35316)
+- Fix EBUSY errors under overlayfs and v4.13+ kernels [moby/moby#34948](https://github.com/moby/moby/pull/34948)
+* Container: protect health monitor channel [moby/moby#35482](https://github.com/moby/moby/pull/35482)
+* Container: protect the health status with mutex [moby/moby#35517](https://github.com/moby/moby/pull/35517)
+* Container: update real-time resources [moby/moby#33731](https://github.com/moby/moby/pull/33731)
+* Create labels when volume exists only remotely [moby/moby#34896](https://github.com/moby/moby/pull/34896)
+* Fix leaking container/exec state [moby/moby#35484](https://github.com/moby/moby/pull/35484)
+
+### Swarm Mode
+
++ Added support for swarm service isolation mode [moby/moby#34424](https://github.com/moby/moby/pull/34424)
+
+### Packaging
+
++ Added Packaging for Fedora 27 [docker/docker-ce-packaging#59](https://github.com/docker/docker-ce-packaging/pull/59)
+* Changed default versioning scheme to 0.0.0-dev unless specified for packaging [docker/docker-ce-packaging#67](https://github.com/docker/docker-ce-packaging/pull/67)

VERSION

@@ -1 +1 @@
-17.09.0-dev
+17.12.0-ce-rc2

components/cli/VERSION

@@ -1 +1 @@
-17.12.0-dev
+17.12.0-ce-rc2

components/engine/docs/api/version-history.md

@@ -23,6 +23,18 @@ keywords: "API, Docker, rcli, REST, documentation"
   configuration is only used for Windows containers.
 * `GET /containers/(name)/logs` now supports an additional query parameter: `until`,
   which returns log lines that occurred before the specified timestamp.
+* `POST /containers/{id}/exec` now accepts a `WorkingDir` property to set the
+  work-dir for the exec process, independent of the container's work-dir.
+* `Get /version` now returns a `Platform.Name` field, which can be used by products
+  using Moby as a foundation to return information about the platform.
+* `Get /version` now returns a `Components` field, which can be used to return
+  information about the components used. Information about the engine itself is
+  now included as a "Component" version, and contains all information from the
+  top-level `Version`, `GitCommit`, `APIVersion`, `MinAPIVersion`, `GoVersion`,
+  `Os`, `Arch`, `BuildTime`, `KernelVersion`, and `Experimental` fields. Going
+  forward, the information from the `Components` section is preferred over their
+  top-level counterparts.
+
 
 ## v1.34 API changes
 

components/engine/integration-cli/docker_cli_build_test.go

@@ -4211,6 +4211,7 @@ func (s *DockerTrustSuite) TestBuildContextDirIsSymlink(c *check.C) {
 }
 
 func (s *DockerTrustSuite) TestTrustedBuildTagFromReleasesRole(c *check.C) {
+	c.Skip("Blacklisting for Docker CE")
 	testRequires(c, NotaryHosting)
 
 	latestTag := s.setupTrustedImage(c, "trusted-build-releases-role")
@@ -4242,6 +4243,7 @@ func (s *DockerTrustSuite) TestTrustedBuildTagFromReleasesRole(c *check.C) {
 }
 
 func (s *DockerTrustSuite) TestTrustedBuildTagIgnoresOtherDelegationRoles(c *check.C) {
+	c.Skip("Blacklisting for Docker CE")
 	testRequires(c, NotaryHosting)
 
 	latestTag := s.setupTrustedImage(c, "trusted-build-releases-role")

components/engine/integration-cli/docker_cli_pull_trusted_test.go

@@ -133,6 +133,7 @@ func (s *DockerTrustSuite) TestTrustedPullDelete(c *check.C) {
 }
 
 func (s *DockerTrustSuite) TestTrustedPullReadsFromReleasesRole(c *check.C) {
+	c.Skip("Blacklisting for Docker CE")
 	testRequires(c, NotaryHosting)
 	repoName := fmt.Sprintf("%v/dockerclireleasesdelegationpulling/trusted", privateRegistryURL)
 	targetName := fmt.Sprintf("%s:latest", repoName)
@@ -188,6 +189,7 @@ func (s *DockerTrustSuite) TestTrustedPullReadsFromReleasesRole(c *check.C) {
 }
 
 func (s *DockerTrustSuite) TestTrustedPullIgnoresOtherDelegationRoles(c *check.C) {
+	c.Skip("Blacklisting for Docker CE")
 	testRequires(c, NotaryHosting)
 	repoName := fmt.Sprintf("%v/dockerclipullotherdelegation/trusted", privateRegistryURL)
 	targetName := fmt.Sprintf("%s:latest", repoName)

components/engine/integration-cli/docker_cli_push_test.go

@@ -282,6 +282,7 @@ func (s *DockerSchema1RegistrySuite) TestCrossRepositoryLayerPushNotSupported(c
 }
 
 func (s *DockerTrustSuite) TestTrustedPush(c *check.C) {
+	c.Skip("Blacklisting for Docker CE")
 	repoName := fmt.Sprintf("%v/dockerclitrusted/pushtest:latest", privateRegistryURL)
 	// tag the image and upload it to the private registry
 	cli.DockerCmd(c, "tag", "busybox", repoName)
@@ -366,6 +367,7 @@ func (s *DockerTrustSuite) TestTrustedPushWithExistingSignedTag(c *check.C) {
 }
 
 func (s *DockerTrustSuite) TestTrustedPushWithIncorrectPassphraseForNonRoot(c *check.C) {
+	c.Skip("Blacklisting for Docker CE")
 	repoName := fmt.Sprintf("%v/dockercliincorretpwd/trusted:latest", privateRegistryURL)
 	// tag the image and upload it to the private registry
 	cli.DockerCmd(c, "tag", "busybox", repoName)

components/engine/integration-cli/trust_server_test.go

@@ -41,7 +41,7 @@ const notaryHost = "localhost:4443"
 const notaryURL = "https://" + notaryHost
 
 var SuccessTagging = icmd.Expected{
-	Out: "Tagging",
+	Err: "Tagging",
 }
 
 var SuccessSigningAndPushing = icmd.Expected{

components/engine/vendor.conf

@@ -148,7 +148,7 @@ github.com/opencontainers/selinux b29023b86e4a69d1b46b7e7b4e2b6fda03f0b9cd
 # archive/tar
 # mkdir -p ./vendor/archive
 # git clone git://github.com/tonistiigi/go-1.git ./go
-# git --git-dir ./go/.git --work-tree ./go checkout revert-prefix-ignore
+# git --git-dir ./go/.git --work-tree ./go checkout revert-prefix-ignore-1.9
 # cp -a go/src/archive/tar ./vendor/archive/tar
 # rm -rf ./go
 # vndr

components/engine/vendor/archive/tar/common.go

@@ -158,11 +158,15 @@ func (fi headerFileInfo) Mode() (mode os.FileMode) {
 // sysStat, if non-nil, populates h from system-dependent fields of fi.
 var sysStat func(fi os.FileInfo, h *Header) error
 
-// Mode constants from the tar spec.
 const (
-	c_ISUID  = 04000   // Set uid
-	c_ISGID  = 02000   // Set gid
-	c_ISVTX  = 01000   // Save text (sticky bit)
+	// Mode constants from the USTAR spec:
+	// See http://pubs.opengroup.org/onlinepubs/9699919799/utilities/pax.html#tag_20_92_13_06
+	c_ISUID = 04000 // Set uid
+	c_ISGID = 02000 // Set gid
+	c_ISVTX = 01000 // Save text (sticky bit)
+
+	// Common Unix mode constants; these are not defined in any common tar standard.
+	// Header.FileInfo understands these, but FileInfoHeader will never produce these.
 	c_ISDIR  = 040000  // Directory
 	c_ISFIFO = 010000  // FIFO
 	c_ISREG  = 0100000 // Regular file
@@ -208,30 +212,24 @@ func FileInfoHeader(fi os.FileInfo, link string) (*Header, error) {
 	}
 	switch {
 	case fm.IsRegular():
-		h.Mode |= c_ISREG
 		h.Typeflag = TypeReg
 		h.Size = fi.Size()
 	case fi.IsDir():
 		h.Typeflag = TypeDir
-		h.Mode |= c_ISDIR
 		h.Name += "/"
 	case fm&os.ModeSymlink != 0:
 		h.Typeflag = TypeSymlink
-		h.Mode |= c_ISLNK
 		h.Linkname = link
 	case fm&os.ModeDevice != 0:
 		if fm&os.ModeCharDevice != 0 {
-			h.Mode |= c_ISCHR
 			h.Typeflag = TypeChar
 		} else {
-			h.Mode |= c_ISBLK
 			h.Typeflag = TypeBlock
 		}
 	case fm&os.ModeNamedPipe != 0:
 		h.Typeflag = TypeFifo
-		h.Mode |= c_ISFIFO
 	case fm&os.ModeSocket != 0:
-		h.Mode |= c_ISSOCK
+		return nil, fmt.Errorf("archive/tar: sockets not supported")
 	default:
 		return nil, fmt.Errorf("archive/tar: unknown file mode %v", fm)
 	}

components/engine/vendor/archive/tar/writer.go

@@ -121,9 +121,15 @@ func (tw *Writer) writeHeader(hdr *Header, allowPax bool) error {
 		needsPaxHeader := paxKeyword != paxNone && len(s) > len(b) || !isASCII(s)
 		if needsPaxHeader {
 			paxHeaders[paxKeyword] = s
-			return
 		}
-		f.formatString(b, s)
+
+		// Write string in a best-effort manner to satisfy readers that expect
+		// the field to be non-empty.
+		s = toASCII(s)
+		if len(s) > len(b) {
+			s = s[:len(b)]
+		}
+		f.formatString(b, s) // Should never error
 	}
 	var formatNumeric = func(b []byte, x int64, paxKeyword string) {
 		// Try octal first.

components/packaging/Makefile

@@ -1,7 +1,7 @@
 SHELL:=/bin/bash
 ENGINE_DIR:=$(CURDIR)/../engine
 CLI_DIR:=$(CURDIR)/../cli
-VERSION=unknown
+VERSION?=0.0.0-dev
 DOCKER_GITCOMMIT:=abcdefg
 
 .PHONY: help clean rpm deb static

components/packaging/deb/Makefile

@@ -3,7 +3,7 @@ ARCH:=$(shell uname -m)
 ENGINE_DIR:=$(CURDIR)/../../engine
 CLI_DIR:=$(CURDIR)/../../cli
 GITCOMMIT?=$(shell cd $(ENGINE_DIR) && git rev-parse --short HEAD)
-VERSION?=$(shell cat $(ENGINE_DIR)/VERSION)
+VERSION?=0.0.0-dev
 DOCKER_EXPERIMENTAL:=0
 CHOWN:=docker run --rm -v $(CURDIR):/v -w /v alpine chown
 

components/packaging/rpm/Makefile

@@ -2,7 +2,7 @@ ARCH=$(shell uname -m)
 ENGINE_DIR:=$(CURDIR)/../../engine
 CLI_DIR:=$(CURDIR)/../../cli
 GITCOMMIT=$(shell cd $(ENGINE_DIR) && git rev-parse --short HEAD)
-VERSION=$(shell cat $(ENGINE_DIR)/VERSION)
+VERSION?=0.0.0-dev
 DOCKER_EXPERIMENTAL=0
 GEN_RPM_VER=$(shell ./gen-rpm-ver $(ENGINE_DIR) $(VERSION))
 CHOWN=docker run --rm -i -v $(CURDIR):/v -w /v alpine chown

components/packaging/rpm/fedora-27/docker-ce.spec

@@ -15,6 +15,7 @@ Packager: Docker <support@docker.com>
 %global _dwz_low_mem_die_limit 0
 %global is_systemd 1
 %global with_selinux 1
+%global _missing_build_ids_terminate_build 0
 
 BuildRequires: pkgconfig(systemd)
 

components/packaging/static/Makefile

@@ -1,7 +1,7 @@
 SHELL:=/bin/bash
 ENGINE_DIR:=$(CURDIR)/../../engine
 CLI_DIR:=$(CURDIR)/../../cli
-VERSION=$(shell cat $(ENGINE_DIR)/VERSION)
+VERSION?=0.0.0-dev
 CHOWN=docker run --rm -v $(CURDIR):/v -w /v alpine chown
 HASH_CMD=docker run -v $(CURDIR):/sum -w /sum debian:jessie bash hash_files
 DIR_TO_HASH:=build/linux
@@ -50,10 +50,10 @@ static-cli:
 	$(MAKE) -C $(CLI_DIR) -f docker.Makefile VERSION=$(VERSION) build
 
 static-engine:
-	$(MAKE) -C $(ENGINE_DIR) binary
+	$(MAKE) -C $(ENGINE_DIR) VERSION=$(VERSION) binary
 
 cross-all-cli:
 	$(MAKE) -C $(CLI_DIR) -f docker.Makefile VERSION=$(VERSION) cross
 
 cross-win-engine:
-	$(MAKE) -C $(ENGINE_DIR) DOCKER_CROSSPLATFORMS=windows/amd64 cross
+	$(MAKE) -C $(ENGINE_DIR) VERSION=$(VERSION) DOCKER_CROSSPLATFORMS=windows/amd64 cross