Fix some build failures
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
This commit is contained in:
Vincent Demeester
committed by
Silvin Lubecki
Silvin Lubecki
parent
8417e49792
commit
1ff277ad8f
-60
@@ -1,60 +0,0 @@
|
||||
## About
|
||||
|
||||
This directory contains a collection of scripts used to build and manage this
|
||||
repository. If there are any issues regarding the intention of a particular
|
||||
script (or even part of a certain script), please reach out to us.
|
||||
It may help us either refine our current scripts, or add on new ones
|
||||
that are appropriate for a given use case.
|
||||
|
||||
## DinD (dind.sh)
|
||||
|
||||
DinD is a wrapper script which allows Docker to be run inside a Docker
|
||||
container. DinD requires the container to
|
||||
be run with privileged mode enabled.
|
||||
|
||||
## Generate Authors (generate-authors.sh)
|
||||
|
||||
Generates AUTHORS; a file with all the names and corresponding emails of
|
||||
individual contributors. AUTHORS can be found in the home directory of
|
||||
this repository.
|
||||
|
||||
## Make
|
||||
|
||||
There are two make files, each with different extensions. Neither are supposed
|
||||
to be called directly; only invoke `make`. Both scripts run inside a Docker
|
||||
container.
|
||||
|
||||
### make.ps1
|
||||
|
||||
- The Windows native build script that uses PowerShell semantics; it is limited
|
||||
unlike `hack\make.sh` since it does not provide support for the full set of
|
||||
operations provided by the Linux counterpart, `make.sh`. However, `make.ps1`
|
||||
does provide support for local Windows development and Windows to Windows CI.
|
||||
More information is found within `make.ps1` by the author, @jhowardmsft
|
||||
|
||||
### make.sh
|
||||
|
||||
- Referenced via `make test` when running tests on a local machine,
|
||||
or directly referenced when running tests inside a Docker development container.
|
||||
- When running on a local machine, `make test` to run all tests found in
|
||||
`test`, `test-unit`, `test-integration`, and `test-docker-py` on
|
||||
your local machine. The default timeout is set in `make.sh` to 60 minutes
|
||||
(`${TIMEOUT:=60m}`), since it currently takes up to an hour to run
|
||||
all of the tests.
|
||||
- When running inside a Docker development container, `hack/make.sh` does
|
||||
not have a single target that runs all the tests. You need to provide a
|
||||
single command line with multiple targets that performs the same thing.
|
||||
An example referenced from [Run targets inside a development container](https://docs.docker.com/opensource/project/test-and-docs/#run-targets-inside-a-development-container): `root@5f8630b873fe:/go/src/github.com/moby/moby# hack/make.sh dynbinary binary cross test-unit test-integration test-docker-py`
|
||||
- For more information related to testing outside the scope of this README,
|
||||
refer to
|
||||
[Run tests and test documentation](https://docs.docker.com/opensource/project/test-and-docs/)
|
||||
|
||||
## Release (release.sh)
|
||||
|
||||
Releases any bundles built by `make` on a public AWS S3 bucket.
|
||||
For information regarding configuration, please view `release.sh`.
|
||||
|
||||
## Vendor (vendor.sh)
|
||||
|
||||
A shell script that is a wrapper around Vndr. For information on how to use
|
||||
this, please refer to [vndr's README](https://github.com/LK4D4/vndr/blob/master/README.md)
|
||||
-69
@@ -1,69 +0,0 @@
|
||||
# Integration Testing on Swarm
|
||||
|
||||
IT on Swarm allows you to execute integration test in parallel across a Docker Swarm cluster
|
||||
|
||||
## Architecture
|
||||
|
||||
### Master service
|
||||
|
||||
- Works as a funker caller
|
||||
- Calls a worker funker (`-worker-service`) with a chunk of `-check.f` filter strings (passed as a file via `-input` flag, typically `/mnt/input`)
|
||||
|
||||
### Worker service
|
||||
|
||||
- Works as a funker callee
|
||||
- Executes an equivalent of `TESTFLAGS=-check.f TestFoo|TestBar|TestBaz ... make test-integration-cli` using the bind-mounted API socket (`docker.sock`)
|
||||
|
||||
### Client
|
||||
|
||||
- Controls master and workers via `docker stack`
|
||||
- No need to have a local daemon
|
||||
|
||||
Typically, the master and workers are supposed to be running on a cloud environment,
|
||||
while the client is supposed to be running on a laptop, e.g. Docker for Mac/Windows.
|
||||
|
||||
## Requirement
|
||||
|
||||
- Docker daemon 1.13 or later
|
||||
- Private registry for distributed execution with multiple nodes
|
||||
|
||||
## Usage
|
||||
|
||||
### Step 1: Prepare images
|
||||
|
||||
$ make build-integration-cli-on-swarm
|
||||
|
||||
Following environment variables are known to work in this step:
|
||||
|
||||
- `BUILDFLAGS`
|
||||
- `DOCKER_INCREMENTAL_BINARY`
|
||||
|
||||
Note: during the transition into Moby Project, you might need to create a symbolic link `$GOPATH/src/github.com/docker/docker` to `$GOPATH/src/github.com/moby/moby`.
|
||||
|
||||
### Step 2: Execute tests
|
||||
|
||||
$ ./hack/integration-cli-on-swarm/integration-cli-on-swarm -replicas 40 -push-worker-image YOUR_REGISTRY.EXAMPLE.COM/integration-cli-worker:latest
|
||||
|
||||
Following environment variables are known to work in this step:
|
||||
|
||||
- `DOCKER_GRAPHDRIVER`
|
||||
- `DOCKER_EXPERIMENTAL`
|
||||
|
||||
#### Flags
|
||||
|
||||
Basic flags:
|
||||
|
||||
- `-replicas N`: the number of worker service replicas. i.e. degree of parallelism.
|
||||
- `-chunks N`: the number of chunks. By default, `chunks` == `replicas`.
|
||||
- `-push-worker-image REGISTRY/IMAGE:TAG`: push the worker image to the registry. Note that if you have only single node and hence you do not need a private registry, you do not need to specify `-push-worker-image`.
|
||||
|
||||
Experimental flags for mitigating makespan nonuniformity:
|
||||
|
||||
- `-shuffle`: Shuffle the test filter strings
|
||||
|
||||
Flags for debugging IT on Swarm itself:
|
||||
|
||||
- `-rand-seed N`: the random seed. This flag is useful for deterministic replaying. By default(0), the timestamp is used.
|
||||
- `-filters-file FILE`: the file contains `-check.f` strings. By default, the file is automatically generated.
|
||||
- `-dry-run`: skip the actual workload
|
||||
- `keep-executor`: do not auto-remove executor containers, which is used for running privileged programs on Swarm
|
||||
-2
@@ -1,2 +0,0 @@
|
||||
# dependencies specific to worker (i.e. github.com/docker/docker/...) are not vendored here
|
||||
github.com/bfirsh/funker-go eaa0a2e06f30e72c9a0b7f858951e581e26ef773
|
||||
-97
@@ -1,97 +0,0 @@
|
||||
// +build ignore
|
||||
|
||||
// Simple tool to create an archive stream from an old and new directory
|
||||
//
|
||||
// By default it will stream the comparison of two temporary directories with junk files
|
||||
package main
|
||||
|
||||
import (
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"io/ioutil"
|
||||
"os"
|
||||
"path"
|
||||
|
||||
"github.com/docker/docker/pkg/archive"
|
||||
"github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
var (
|
||||
flDebug = flag.Bool("D", false, "debugging output")
|
||||
flNewDir = flag.String("newdir", "", "")
|
||||
flOldDir = flag.String("olddir", "", "")
|
||||
log = logrus.New()
|
||||
)
|
||||
|
||||
func main() {
|
||||
flag.Usage = func() {
|
||||
fmt.Println("Produce a tar from comparing two directory paths. By default a demo tar is created of around 200 files (including hardlinks)")
|
||||
fmt.Printf("%s [OPTIONS]\n", os.Args[0])
|
||||
flag.PrintDefaults()
|
||||
}
|
||||
flag.Parse()
|
||||
log.Out = os.Stderr
|
||||
if (len(os.Getenv("DEBUG")) > 0) || *flDebug {
|
||||
logrus.SetLevel(logrus.DebugLevel)
|
||||
}
|
||||
var newDir, oldDir string
|
||||
|
||||
if len(*flNewDir) == 0 {
|
||||
var err error
|
||||
newDir, err = ioutil.TempDir("", "docker-test-newDir")
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
defer os.RemoveAll(newDir)
|
||||
if _, err := prepareUntarSourceDirectory(100, newDir, true); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
} else {
|
||||
newDir = *flNewDir
|
||||
}
|
||||
|
||||
if len(*flOldDir) == 0 {
|
||||
oldDir, err := ioutil.TempDir("", "docker-test-oldDir")
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
defer os.RemoveAll(oldDir)
|
||||
} else {
|
||||
oldDir = *flOldDir
|
||||
}
|
||||
|
||||
changes, err := archive.ChangesDirs(newDir, oldDir)
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
|
||||
a, err := archive.ExportChanges(newDir, changes)
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
defer a.Close()
|
||||
|
||||
i, err := io.Copy(os.Stdout, a)
|
||||
if err != nil && err != io.EOF {
|
||||
log.Fatal(err)
|
||||
}
|
||||
fmt.Fprintf(os.Stderr, "wrote archive of %d bytes", i)
|
||||
}
|
||||
|
||||
func prepareUntarSourceDirectory(numberOfFiles int, targetPath string, makeLinks bool) (int, error) {
|
||||
fileData := []byte("fooo")
|
||||
for n := 0; n < numberOfFiles; n++ {
|
||||
fileName := fmt.Sprintf("file-%d", n)
|
||||
if err := ioutil.WriteFile(path.Join(targetPath, fileName), fileData, 0700); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if makeLinks {
|
||||
if err := os.Link(path.Join(targetPath, fileName), path.Join(targetPath, fileName+"-link")); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
}
|
||||
}
|
||||
totalSize := numberOfFiles * len(fileData)
|
||||
return totalSize, nil
|
||||
}
|
||||
-76
@@ -1,76 +0,0 @@
|
||||
// Copyright 2014 Google Inc.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
// +build ignore
|
||||
|
||||
// This binary compares memory usage between btree and gollrb.
|
||||
package main
|
||||
|
||||
import (
|
||||
"flag"
|
||||
"fmt"
|
||||
"math/rand"
|
||||
"runtime"
|
||||
"time"
|
||||
|
||||
"github.com/google/btree"
|
||||
"github.com/petar/GoLLRB/llrb"
|
||||
)
|
||||
|
||||
var (
|
||||
size = flag.Int("size", 1000000, "size of the tree to build")
|
||||
degree = flag.Int("degree", 8, "degree of btree")
|
||||
gollrb = flag.Bool("llrb", false, "use llrb instead of btree")
|
||||
)
|
||||
|
||||
func main() {
|
||||
flag.Parse()
|
||||
vals := rand.Perm(*size)
|
||||
var t, v interface{}
|
||||
v = vals
|
||||
var stats runtime.MemStats
|
||||
for i := 0; i < 10; i++ {
|
||||
runtime.GC()
|
||||
}
|
||||
fmt.Println("-------- BEFORE ----------")
|
||||
runtime.ReadMemStats(&stats)
|
||||
fmt.Printf("%+v\n", stats)
|
||||
start := time.Now()
|
||||
if *gollrb {
|
||||
tr := llrb.New()
|
||||
for _, v := range vals {
|
||||
tr.ReplaceOrInsert(llrb.Int(v))
|
||||
}
|
||||
t = tr // keep it around
|
||||
} else {
|
||||
tr := btree.New(*degree)
|
||||
for _, v := range vals {
|
||||
tr.ReplaceOrInsert(btree.Int(v))
|
||||
}
|
||||
t = tr // keep it around
|
||||
}
|
||||
fmt.Printf("%v inserts in %v\n", *size, time.Since(start))
|
||||
fmt.Println("-------- AFTER ----------")
|
||||
runtime.ReadMemStats(&stats)
|
||||
fmt.Printf("%+v\n", stats)
|
||||
for i := 0; i < 10; i++ {
|
||||
runtime.GC()
|
||||
}
|
||||
fmt.Println("-------- AFTER GC ----------")
|
||||
runtime.ReadMemStats(&stats)
|
||||
fmt.Printf("%+v\n", stats)
|
||||
if t == v {
|
||||
fmt.Println("to make sure vals and tree aren't GC'd")
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user