Fixed typo in abra ac bash output

.drone.yml

@@ -3,17 +3,27 @@ kind: pipeline
 name: coopcloud.tech/abra
 steps:
   - name: make check
-    image: golang:1.18
+    image: golang:1.17
     commands:
       - make check
 
+  - name: make static
+    image: golang:1.17
+    ignore: true # until we decide we all want this check
+    environment:
+      STATIC_CHECK_URL: honnef.co/go/tools/cmd/staticcheck
+      STATIC_CHECK_VERSION: v0.2.0
+    commands:
+      - go install $STATIC_CHECK_URL@$STATIC_CHECK_VERSION
+      - make static
+
   - name: make build
-    image: golang:1.18
+    image: golang:1.17
     commands:
       - make build
 
   - name: make test
-    image: golang:1.18
+    image: golang:1.17
     commands:
       - make test
 
@@ -45,7 +55,7 @@ steps:
       event: tag
 
   - name: release
-    image: golang:1.18
+    image: golang:1.17
     environment:
       GITEA_TOKEN:
         from_secret: goreleaser_gitea_token

.e2e.env.sample

@@ -1,4 +0,0 @@
-GANDI_TOKEN=...
-HCLOUD_TOKEN=...
-REGISTRY_PASSWORD=...
-REGISTRY_USERNAME=...

.gitignore

@@ -1,8 +1,6 @@
-*fmtcoverage.html
-.e2e.env
-.envrc
-.vscode/
 abra
-dist/
-tests/integration/.abra/catalogue
+.vscode/
 vendor/
+.envrc
+dist/
+*fmtcoverage.html

AUTHORS.md

@@ -1,10 +0,0 @@
-# authors
-
-> If you're looking at this and you hack on Abra and you're not listed here,
-> please do add yourself! This is a community project, let's show
-
-- 3wordchant
-- decentral1se
-- kawaiipunk
-- knoflook
-- roxxers

Makefile

@@ -5,7 +5,7 @@ LDFLAGS := "-X 'main.Commit=$(COMMIT)'"
 DIST_LDFLAGS := $(LDFLAGS)" -s -w"
 export GOPRIVATE=coopcloud.tech
 
-all: format check build test
+all: run test install build clean format check static
 
 run:
 	@go run -ldflags=$(LDFLAGS) $(ABRA)
@@ -28,6 +28,9 @@ format:
 check:
 	@test -z $$(gofmt -l .) || (echo "gofmt: formatting issue - run 'make format' to resolve" && exit 1)
 
+static:
+	@staticcheck $(ABRA)
+
 test:
 	@go test ./... -cover -v
 

README.md

@@ -7,6 +7,69 @@
 
 The Co-op Cloud utility belt 🎩🐇
 
-`abra` is our flagship client & command-line tool which has been developed specifically in the context of the Co-op Cloud project for the purpose of making day-to-day operations for [operators](https://docs.coopcloud.tech/operators/) and [maintainers](https://docs.coopcloud.tech/maintainers/) as convenient as possible. It is libre software, written in [Go](https://go.dev) and maintained and extended by the community ❤
+`abra` is a command-line tool for managing your own [Co-op Cloud](https://coopcloud.tech). It can provision new servers, create apps, deploy them, run backup and restore operations and a whole lot of other things. Please see [docs.coopcloud.tech](https://docs.coopcloud.tech) for more extensive documentation.
 
-Please see [docs.coopcloud.tech/abra/](https://docs.coopcloud.tech/abra/) for help on install, upgrade, hacking, troubleshooting & more!
+## Quick install
+
+```bash
+curl https://install.abra.autonomic.zone | bash
+```
+
+Or using the latest release candidate (extra experimental!):
+
+```bash
+curl https://install.abra.autonomic.zone | bash -s -- --rc
+```
+
+Source for this script is in [scripts/installer/installer](./scripts/installer/installer).
+
+## Hacking
+
+### Getting started
+
+Install [direnv](https://direnv.net), run `cp .envrc.sample .envrc`, then run `direnv allow` in this directory. This will set coopcloud repos as private due to [this bug.](https://git.coopcloud.tech/coop-cloud/coopcloud.tech/issues/20#issuecomment-8201). Or you can run `go env -w GOPRIVATE=coopcloud.tech` but I'm not sure how persistent this is.
+
+Install [Go >= 1.16](https://golang.org/doc/install) and then:
+
+- `make build` to build
+- `./abra` to run commands
+- `make test` will run tests
+- `make install` will install it to `$GOPATH/bin`
+- `go get <package>` and `go mod tidy` to add a new dependency
+
+Our [Drone CI configuration](.drone.yml) runs a number of sanity on each pushed commit. See the [Makefile](./Makefile) for more handy targets.
+
+Please use the [conventional commit format](https://www.conventionalcommits.org/en/v1.0.0/) for your commits so we can automate our change log.
+
+### Versioning
+
+We use [goreleaser](https://goreleaser.com) to help us automate releases. We use [semver](https://semver.org) for versioning all releases of the tool. While we are still in the public alpha release phase, we will maintain a `0.y.z-alpha` format. Change logs are generated from our commit logs. We are still working this out and aim to refine our release praxis as we go.
+
+For developers, while using this `-alpha` format, the `y` part is the "major" version part. So, if you make breaking changes, you increment that and _not_ the `x` part. So, if you're on `0.1.0-alpha`, then you'd go to `0.1.1-alpha` for a backwards compatible change and `0.2.0-alpha` for a backwards incompatible change.
+
+### Making a new release
+
+- Change `ABRA_VERSION` to match the new tag in [`scripts`](./scripts/installer/installer) (use [semver](https://semver.org))
+- Commit that change (e.g. `git commit -m 'chore: publish next tag x.y.z-alpha'`)
+- Make a new tag (e.g. `git tag -a x.y.z-alpha`)
+- Push the new tag (e.g. `git push && git push --tags`)
+- Wait until the build finishes on [build.coopcloud.tech](https://build.coopcloud.tech/coop-cloud/abra)
+- Deploy the new installer script (e.g. `cd ./scripts/installer && make`)
+- Check the release worked, (e.g. `abra upgrade; abra -v`)
+
+### Fork maintenance
+
+#### `godotenv`
+
+We maintain a fork of [godotenv](https://github.com/Autonomic-Cooperative/godotenv) for two features:
+
+1. multi-line env var support
+2. inline comment parsing
+
+You can upgrade the version here by running `go get github.com/Autonomic-Cooperative/godotenv@<commit>` where `<commit>` is the
+latest commit you want to pin to. We are aiming to migrate to YAML format for the environment configuration, so this should only
+be a temporary thing.
+
+#### `docker/client`
+
+A number of modules in [pkg/upstream](./pkg/upstream) are copy/pasta'd from the upstream [docker/docker/client](https://pkg.go.dev/github.com/docker/docker/client). We had to do this because upstream are not exposing their API as public.

cli/app/app.go

@@ -1,22 +1,29 @@
 package app
 
 import (
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
-var AppCommand = cli.Command{
-	Name:        "app",
-	Aliases:     []string{"a"},
-	Usage:       "Manage apps",
-	ArgsUsage:   "<domain>",
-	Description: "This command provides functionality for managing the life cycle of your apps",
-	Subcommands: []cli.Command{
+// AppCommand defines the `abra app` command and ets subcommands
+var AppCommand = &cli.Command{
+	Name:      "app",
+	Usage:     "Manage deployed apps",
+	Aliases:   []string{"a"},
+	ArgsUsage: "<app>",
+	Description: `
+This command provides all the functionality you need to manage the life cycle
+of your apps. From initial deployment, day-2 operations (e.g. backup/restore)
+to scaling apps up and spinning them down.
+`,
+	Subcommands: []*cli.Command{
 		appNewCommand,
 		appConfigCommand,
 		appRestartCommand,
 		appDeployCommand,
 		appUpgradeCommand,
 		appUndeployCommand,
+		appBackupCommand,
+		appRestoreCommand,
 		appRemoveCommand,
 		appCheckCommand,
 		appListCommand,
@@ -29,6 +36,5 @@ var AppCommand = cli.Command{
 		appVolumeCommand,
 		appVersionCommand,
 		appErrorsCommand,
-		appCmdCommand,
 	},
 }

cli/app/backup.go

@@ -0,0 +1,77 @@
+package app
+
+import (
+	"errors"
+	"fmt"
+	"io/ioutil"
+	"os"
+	"os/exec"
+	"path"
+	"strings"
+
+	"coopcloud.tech/abra/cli/internal"
+	"coopcloud.tech/abra/pkg/autocomplete"
+	"coopcloud.tech/abra/pkg/config"
+	"github.com/sirupsen/logrus"
+	"github.com/urfave/cli/v2"
+)
+
+var backupAllServices bool
+var backupAllServicesFlag = &cli.BoolFlag{
+	Name:        "all",
+	Value:       false,
+	Destination: &backupAllServices,
+	Aliases:     []string{"a"},
+	Usage:       "Backup all services",
+}
+
+var appBackupCommand = &cli.Command{
+	Name:      "backup",
+	Usage:     "Backup an app",
+	Aliases:   []string{"b"},
+	Flags:     []cli.Flag{backupAllServicesFlag},
+	ArgsUsage: "<service>",
+	Action: func(c *cli.Context) error {
+		app := internal.ValidateApp(c)
+
+		if c.Args().Get(1) != "" && backupAllServices {
+			internal.ShowSubcommandHelpAndError(c, errors.New("cannot use '<service>' and '--all' together"))
+		}
+
+		abraSh := path.Join(config.ABRA_DIR, "apps", app.Type, "abra.sh")
+		if _, err := os.Stat(abraSh); err != nil {
+			if os.IsNotExist(err) {
+				logrus.Fatalf("'%s' does not exist?", abraSh)
+			}
+			logrus.Fatal(err)
+		}
+
+		sourceCmd := fmt.Sprintf("source %s", abraSh)
+
+		execCmd := "abra_backup"
+		if !backupAllServices {
+			serviceName := c.Args().Get(1)
+			if serviceName == "" {
+				internal.ShowSubcommandHelpAndError(c, errors.New("no service(s) target provided"))
+			}
+			execCmd = fmt.Sprintf("abra_backup_%s", serviceName)
+		}
+
+		bytes, err := ioutil.ReadFile(abraSh)
+		if err != nil {
+			logrus.Fatal(err)
+		}
+		if !strings.Contains(string(bytes), execCmd) {
+			logrus.Fatalf("%s doesn't have a '%s' function", app.Type, execCmd)
+		}
+
+		sourceAndExec := fmt.Sprintf("%s; %s", sourceCmd, execCmd)
+		cmd := exec.Command("bash", "-c", sourceAndExec)
+		if err := internal.RunCmd(cmd); err != nil {
+			logrus.Fatal(err)
+		}
+
+		return nil
+	},
+	BashComplete: autocomplete.AppNameComplete,
+}

cli/app/check.go

@@ -9,25 +9,21 @@ import (
 	"coopcloud.tech/abra/pkg/autocomplete"
 	"coopcloud.tech/abra/pkg/config"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
-var appCheckCommand = cli.Command{
+var appCheckCommand = &cli.Command{
 	Name:      "check",
-	Aliases:   []string{"chk"},
 	Usage:     "Check if app is configured correctly",
-	ArgsUsage: "<domain>",
-	Flags: []cli.Flag{
-		internal.DebugFlag,
-	},
-	Before: internal.SubCommandBefore,
+	Aliases:   []string{"c"},
+	ArgsUsage: "<service>",
 	Action: func(c *cli.Context) error {
 		app := internal.ValidateApp(c)
 
-		envSamplePath := path.Join(config.RECIPES_DIR, app.Recipe, ".env.sample")
+		envSamplePath := path.Join(config.ABRA_DIR, "apps", app.Type, ".env.sample")
 		if _, err := os.Stat(envSamplePath); err != nil {
 			if os.IsNotExist(err) {
-				logrus.Fatalf("%s does not exist?", envSamplePath)
+				logrus.Fatalf("'%s' does not exist?", envSamplePath)
 			}
 			logrus.Fatal(err)
 		}
@@ -49,7 +45,7 @@ var appCheckCommand = cli.Command{
 			logrus.Fatalf("%s is missing %s", app.Path, missingEnvVars)
 		}
 
-		logrus.Infof("all necessary environment variables defined for %s", app.Name)
+		logrus.Infof("all necessary environment variables defined for '%s'", app.Name)
 
 		return nil
 	},

cli/app/cmd.go

@@ -1,214 +0,0 @@
-package app
-
-import (
-	"context"
-	"errors"
-	"fmt"
-	"io/ioutil"
-	"os"
-	"os/exec"
-	"path"
-	"strings"
-
-	"coopcloud.tech/abra/cli/internal"
-	"coopcloud.tech/abra/pkg/autocomplete"
-	"coopcloud.tech/abra/pkg/client"
-	"coopcloud.tech/abra/pkg/config"
-	containerPkg "coopcloud.tech/abra/pkg/container"
-	"coopcloud.tech/abra/pkg/formatter"
-	"coopcloud.tech/abra/pkg/upstream/container"
-	"github.com/docker/cli/cli/command"
-	"github.com/docker/docker/api/types"
-	"github.com/docker/docker/api/types/filters"
-	"github.com/docker/docker/pkg/archive"
-	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
-)
-
-var localCmd bool
-var localCmdFlag = &cli.BoolFlag{
-	Name:        "local, l",
-	Usage:       "Run command locally",
-	Destination: &localCmd,
-}
-
-var appCmdCommand = cli.Command{
-	Name:    "command",
-	Aliases: []string{"cmd"},
-	Usage:   "Run app commands",
-	Description: `
-This command runs app specific commands.
-
-These commands are bash functions, defined in the abra.sh of the recipe itself.
-They can be run within the context of a service (e.g. app) or locally on your
-work station by passing "--local". Arguments can be passed into these functions
-using the "-- <args>" syntax.
-
-Example:
-
-  abra app cmd example.com app create_user -- me@example.com
-`,
-	ArgsUsage: "<domain> [<service>] <command>",
-	Flags: []cli.Flag{
-		internal.DebugFlag,
-		localCmdFlag,
-	},
-	BashComplete: autocomplete.AppNameComplete,
-	Before:       internal.SubCommandBefore,
-	Action: func(c *cli.Context) error {
-		app := internal.ValidateApp(c)
-
-		if len(c.Args()) <= 2 && !localCmd {
-			internal.ShowSubcommandHelpAndError(c, errors.New("missing <service>/<command>? did you mean to pass --local?"))
-		}
-
-		if len(c.Args()) > 2 && localCmd {
-			internal.ShowSubcommandHelpAndError(c, errors.New("cannot specify <service> and --local together"))
-		}
-
-		abraSh := path.Join(config.RECIPES_DIR, app.Recipe, "abra.sh")
-		if _, err := os.Stat(abraSh); err != nil {
-			if os.IsNotExist(err) {
-				logrus.Fatalf("%s does not exist for %s?", abraSh, app.Name)
-			}
-			logrus.Fatal(err)
-		}
-
-		if localCmd {
-			cmdName := c.Args().Get(1)
-			if err := ensureCommand(abraSh, app.Recipe, cmdName); err != nil {
-				logrus.Fatal(err)
-			}
-
-			logrus.Debugf("--local detected, running %s on local work station", cmdName)
-
-			sourceAndExec := fmt.Sprintf("TARGET=local; APP_NAME=%s; . %s; %s", app.StackName(), abraSh, cmdName)
-			cmd := exec.Command("/bin/sh", "-c", sourceAndExec)
-			if err := internal.RunCmd(cmd); err != nil {
-				logrus.Fatal(err)
-			}
-		} else {
-			targetServiceName := c.Args().Get(1)
-
-			cmdName := c.Args().Get(2)
-			if err := ensureCommand(abraSh, app.Recipe, cmdName); err != nil {
-				logrus.Fatal(err)
-			}
-
-			serviceNames, err := config.GetAppServiceNames(app.Name)
-			if err != nil {
-				logrus.Fatal(err)
-			}
-
-			matchingServiceName := false
-			for _, serviceName := range serviceNames {
-				if serviceName == targetServiceName {
-					matchingServiceName = true
-				}
-			}
-
-			if !matchingServiceName {
-				logrus.Fatalf("no service %s for %s?", targetServiceName, app.Name)
-			}
-
-			logrus.Debugf("running command %s within the context of %s_%s", cmdName, app.StackName(), targetServiceName)
-
-			var parsedCmdArgs string
-			var cmdArgsIdx int
-			var hasCmdArgs bool
-			for idx, arg := range c.Args() {
-				if arg == "--" {
-					cmdArgsIdx = idx
-					hasCmdArgs = true
-				}
-
-				if hasCmdArgs && idx > cmdArgsIdx {
-					parsedCmdArgs += fmt.Sprintf("%s ", c.Args().Get(idx))
-				}
-			}
-
-			if hasCmdArgs {
-				logrus.Debugf("parsed following command arguments: %s", parsedCmdArgs)
-			} else {
-				logrus.Debug("did not detect any command arguments")
-			}
-
-			if err := runCmdRemote(app, abraSh, targetServiceName, cmdName, parsedCmdArgs); err != nil {
-				logrus.Fatal(err)
-			}
-		}
-
-		return nil
-	},
-}
-
-func ensureCommand(abraSh, recipeName, execCmd string) error {
-	bytes, err := ioutil.ReadFile(abraSh)
-	if err != nil {
-		return err
-	}
-
-	if !strings.Contains(string(bytes), execCmd) {
-		return fmt.Errorf("%s doesn't have a %s function", recipeName, execCmd)
-	}
-
-	return nil
-}
-
-func runCmdRemote(app config.App, abraSh, serviceName, cmdName, cmdArgs string) error {
-	cl, err := client.New(app.Server)
-	if err != nil {
-		return err
-	}
-
-	filters := filters.NewArgs()
-	filters.Add("name", fmt.Sprintf("^%s_%s", app.StackName(), serviceName))
-
-	targetContainer, err := containerPkg.GetContainer(context.Background(), cl, filters, true)
-	if err != nil {
-		return err
-	}
-
-	logrus.Debugf("retrieved %s as target container on %s", formatter.ShortenID(targetContainer.ID), app.Server)
-
-	toTarOpts := &archive.TarOptions{NoOverwriteDirNonDir: true, Compression: archive.Gzip}
-	content, err := archive.TarWithOptions(abraSh, toTarOpts)
-	if err != nil {
-		return err
-	}
-
-	copyOpts := types.CopyToContainerOptions{AllowOverwriteDirWithFile: false, CopyUIDGID: false}
-	if err := cl.CopyToContainer(context.Background(), targetContainer.ID, "/tmp", content, copyOpts); err != nil {
-		return err
-	}
-
-	var cmd []string
-	if cmdArgs != "" {
-		cmd = []string{"/bin/sh", "-c", fmt.Sprintf("TARGET=%s; APP_NAME=%s; . /tmp/abra.sh; %s %s", serviceName, app.StackName(), cmdName, cmdArgs)}
-	} else {
-		cmd = []string{"/bin/sh", "-c", fmt.Sprintf("TARGET=%s; APP_NAME=%s; . /tmp/abra.sh; %s", serviceName, app.StackName(), cmdName)}
-	}
-
-	logrus.Debugf("running command: %s", strings.Join(cmd, " "))
-
-	execCreateOpts := types.ExecConfig{
-		AttachStderr: true,
-		AttachStdin:  true,
-		AttachStdout: true,
-		Cmd:          cmd,
-		Detach:       false,
-		Tty:          true,
-	}
-
-	// FIXME: avoid instantiating a new CLI
-	dcli, err := command.NewDockerCli()
-	if err != nil {
-		return err
-	}
-
-	if err := container.RunExec(dcli, cl, targetContainer.ID, &execCreateOpts); err != nil {
-		return err
-	}
-
-	return nil
-}

cli/app/config.go

@@ -10,18 +10,13 @@ import (
 	"coopcloud.tech/abra/pkg/config"
 	"github.com/AlecAivazis/survey/v2"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
-var appConfigCommand = cli.Command{
-	Name:      "config",
-	Aliases:   []string{"cfg"},
-	Usage:     "Edit app config",
-	ArgsUsage: "<domain>",
-	Flags: []cli.Flag{
-		internal.DebugFlag,
-	},
-	Before: internal.SubCommandBefore,
+var appConfigCommand = &cli.Command{
+	Name:    "config",
+	Aliases: []string{"c"},
+	Usage:   "Edit app config",
 	Action: func(c *cli.Context) error {
 		appName := c.Args().First()
 
@@ -36,7 +31,7 @@ var appConfigCommand = cli.Command{
 
 		appFile, exists := files[appName]
 		if !exists {
-			logrus.Fatalf("cannot find app with name %s", appName)
+			logrus.Fatalf("cannot find app with name '%s'", appName)
 		}
 
 		ed, ok := os.LookupEnv("EDITOR")

cli/app/cp.go

@@ -1,44 +1,31 @@
 package app
 
 import (
-	"context"
-	"fmt"
 	"os"
 	"strings"
 
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/autocomplete"
-	"coopcloud.tech/abra/pkg/client"
-	"coopcloud.tech/abra/pkg/config"
-	"coopcloud.tech/abra/pkg/container"
-	"coopcloud.tech/abra/pkg/formatter"
-	"github.com/docker/docker/api/types"
-	"github.com/docker/docker/api/types/filters"
-	"github.com/docker/docker/pkg/archive"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
-var appCpCommand = cli.Command{
+var appCpCommand = &cli.Command{
 	Name:      "cp",
 	Aliases:   []string{"c"},
-	ArgsUsage: "<domain> <src> <dst>",
-	Flags: []cli.Flag{
-		internal.DebugFlag,
-		internal.NoInputFlag,
-	},
-	Before: internal.SubCommandBefore,
-	Usage:  "Copy files to/from a running app service",
+	ArgsUsage: "<src> <dst>",
+	Usage:     "Copy files to/from a running app service",
 	Description: `
 This command supports copying files to and from any app service file system.
 
 If you want to copy a myfile.txt to the root of the app service:
 
-    abra app cp <domain> myfile.txt app:/
+    abra app cp <app> myfile.txt app:/
 
 And if you want to copy that file back to your current working directory locally:
 
-		abra app cp <domain> app:/myfile.txt .
+		abra app cp <app> app:/myfile.txt .
+
 `,
 	Action: func(c *cli.Context) error {
 		app := internal.ValidateApp(c)
@@ -88,7 +75,7 @@ And if you want to copy that file back to your current working directory locally
 				logrus.Fatalf("%s does not exist locally?", dstPath)
 			}
 		}
-		err := configureAndCp(c, app, srcPath, dstPath, service, isToContainer)
+		err := internal.ConfigureAndCp(c, app, srcPath, dstPath, service, isToContainer)
 		if err != nil {
 			logrus.Fatal(err)
 		}
@@ -97,55 +84,3 @@ And if you want to copy that file back to your current working directory locally
 	},
 	BashComplete: autocomplete.AppNameComplete,
 }
-
-func configureAndCp(
-	c *cli.Context,
-	app config.App,
-	srcPath string,
-	dstPath string,
-	service string,
-	isToContainer bool) error {
-	cl, err := client.New(app.Server)
-	if err != nil {
-		logrus.Fatal(err)
-	}
-
-	filters := filters.NewArgs()
-	filters.Add("name", fmt.Sprintf("^%s_%s", app.StackName(), service))
-
-	container, err := container.GetContainer(context.Background(), cl, filters, internal.NoInput)
-	if err != nil {
-		logrus.Fatal(err)
-	}
-
-	logrus.Debugf("retrieved %s as target container on %s", formatter.ShortenID(container.ID), app.Server)
-
-	if isToContainer {
-		if _, err := os.Stat(srcPath); err != nil {
-			logrus.Fatalf("%s does not exist?", srcPath)
-		}
-
-		toTarOpts := &archive.TarOptions{NoOverwriteDirNonDir: true, Compression: archive.Gzip}
-		content, err := archive.TarWithOptions(srcPath, toTarOpts)
-		if err != nil {
-			logrus.Fatal(err)
-		}
-
-		copyOpts := types.CopyToContainerOptions{AllowOverwriteDirWithFile: false, CopyUIDGID: false}
-		if err := cl.CopyToContainer(context.Background(), container.ID, dstPath, content, copyOpts); err != nil {
-			logrus.Fatal(err)
-		}
-	} else {
-		content, _, err := cl.CopyFromContainer(context.Background(), container.ID, srcPath)
-		if err != nil {
-			logrus.Fatal(err)
-		}
-		defer content.Close()
-		fromTarOpts := &archive.TarOptions{NoOverwriteDirNonDir: true, Compression: archive.Gzip}
-		if err := archive.Untar(content, dstPath, fromTarOpts); err != nil {
-			logrus.Fatal(err)
-		}
-	}
-
-	return nil
-}

cli/app/deploy.go

@@ -3,27 +3,22 @@ package app
 import (
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/autocomplete"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
-var appDeployCommand = cli.Command{
-	Name:      "deploy",
-	Aliases:   []string{"d"},
-	Usage:     "Deploy an app",
-	ArgsUsage: "<domain>",
+var appDeployCommand = &cli.Command{
+	Name:    "deploy",
+	Aliases: []string{"d"},
+	Usage:   "Deploy an app",
 	Flags: []cli.Flag{
-		internal.DebugFlag,
-		internal.NoInputFlag,
 		internal.ForceFlag,
 		internal.ChaosFlag,
 		internal.NoDomainChecksFlag,
-		internal.DontWaitConvergeFlag,
 	},
-	Before: internal.SubCommandBefore,
 	Description: `
-This command deploys an app. It does not support incrementing the version of a
-deployed app, for this you need to look at the "abra app upgrade <domain>"
-command.
+This command deploys a new instance of an app. It does not support changing the
+version of an existing deployed app, for this you need to look at the "abra app
+upgrade <app>" command.
 
 You may pass "--force" to re-deploy the same version again. This can be useful
 if the container runtime has gotten into a weird state.

cli/app/errors.go

@@ -1,143 +1,27 @@
 package app
 
 import (
-	"context"
-	"fmt"
-	"strconv"
-	"strings"
-	"time"
-
-	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/autocomplete"
-	"coopcloud.tech/abra/pkg/client"
-	"coopcloud.tech/abra/pkg/config"
-	"coopcloud.tech/abra/pkg/recipe"
-	stack "coopcloud.tech/abra/pkg/upstream/stack"
-	"github.com/docker/docker/api/types"
-	"github.com/docker/docker/api/types/filters"
-	dockerClient "github.com/docker/docker/client"
-	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
-var appErrorsCommand = cli.Command{
-	Name:      "errors",
-	Usage:     "List errors for a deployed app",
-	ArgsUsage: "<domain>",
+var appErrorsCommand = &cli.Command{
+	Name:  "errors",
+	Usage: "List errors for a deployed app",
 	Description: `
-This command lists errors for a deployed app.
-
-This is a best-effort implementation and an attempt to gather a number of tips
-& tricks for finding errors together into one convenient command. When an app
-is failing to deploy or having issues, it could be a lot of things.
-
-This command currently takes into account:
-
-    Is the service deployed?
-    Is the service killed by an OOM error?
-    Is the service reporting an error (like in "ps --no-trunc" output)
-    Is the service healthcheck failing? what are the healthcheck logs?
-
-Got any more ideas? Please let us know:
-
-    https://git.coopcloud.tech/coop-cloud/organising/issues/new/choose
-
-This command is best accompanied by "abra app logs <domain>" which may reveal
-further information which can help you debug the cause of an app failure via
-the logs.
+This command will list errors for a deployed app. This is a best-effort
+implementation and an attempt to gather a number of tips & tricks for finding
+errors together into one convenient command. When an app is failing to deploy
+or having issues, it could be a lot of things. This command is best accompanied
+by "abra app logs <app>".
 `,
-	Aliases: []string{"e"},
-	Flags: []cli.Flag{
-		internal.DebugFlag,
-		internal.WatchFlag,
-	},
-	Before:       internal.SubCommandBefore,
+	Aliases:      []string{"e"},
+	Flags:        []cli.Flag{},
 	BashComplete: autocomplete.AppNameComplete,
 	Action: func(c *cli.Context) error {
-		app := internal.ValidateApp(c)
-
-		cl, err := client.New(app.Server)
-		if err != nil {
-			logrus.Fatal(err)
-		}
-
-		isDeployed, _, err := stack.IsDeployed(context.Background(), cl, app.StackName())
-		if err != nil {
-			logrus.Fatal(err)
-		}
-
-		if !isDeployed {
-			logrus.Fatalf("%s is not deployed?", app.Name)
-		}
-
-		if !internal.Watch {
-			if err := checkErrors(c, cl, app); err != nil {
-				logrus.Fatal(err)
-			}
-			return nil
-		}
-
-		for {
-			if err := checkErrors(c, cl, app); err != nil {
-				logrus.Fatal(err)
-			}
-			time.Sleep(2 * time.Second)
-		}
-
+		// TODO: entrypoint error
+		// TODO: ps --no-trunc errors
+		// TODO: failing healthcheck
 		return nil
 	},
 }
-
-func checkErrors(c *cli.Context, cl *dockerClient.Client, app config.App) error {
-	recipe, err := recipe.Get(app.Recipe)
-	if err != nil {
-		return err
-	}
-
-	for _, service := range recipe.Config.Services {
-		filters := filters.NewArgs()
-		filters.Add("name", fmt.Sprintf("^%s_%s", app.StackName(), service.Name))
-
-		containers, err := cl.ContainerList(context.Background(), types.ContainerListOptions{Filters: filters})
-		if err != nil {
-			return err
-		}
-
-		if len(containers) == 0 {
-			logrus.Warnf("%s is not up, something seems wrong", service.Name)
-			continue
-		}
-
-		container := containers[0]
-		containerState, err := cl.ContainerInspect(context.Background(), container.ID)
-		if err != nil {
-			logrus.Fatal(err)
-		}
-
-		if containerState.State.OOMKilled {
-			logrus.Warnf("%s has been killed due to an out of memory error", service.Name)
-		}
-
-		if containerState.State.Error != "" {
-			logrus.Warnf("%s reports this error: %s", service.Name, containerState.State.Error)
-		}
-
-		if containerState.State.Health != nil {
-			if containerState.State.Health.Status != "healthy" {
-				logrus.Warnf("%s healthcheck status is %s", service.Name, containerState.State.Health.Status)
-				logrus.Warnf("%s healthcheck has failed %s times", service.Name, strconv.Itoa(containerState.State.Health.FailingStreak))
-				for _, log := range containerState.State.Health.Log {
-					logrus.Warnf("%s healthcheck logs: %s", service.Name, strings.TrimSpace(log.Output))
-				}
-			}
-		}
-	}
-
-	return nil
-}
-
-func getServiceName(names []string) string {
-	containerName := strings.Join(names, " ")
-	trimmed := strings.TrimPrefix(containerName, "/")
-	return strings.Split(trimmed, ".")[0]
-}

cli/app/list.go

@@ -5,34 +5,38 @@ import (
 	"sort"
 	"strings"
 
+	abraFormatter "coopcloud.tech/abra/cli/formatter"
 	"coopcloud.tech/abra/cli/internal"
+	"coopcloud.tech/abra/pkg/catalogue"
 	"coopcloud.tech/abra/pkg/config"
-	"coopcloud.tech/abra/pkg/formatter"
-	"coopcloud.tech/abra/pkg/recipe"
 	"coopcloud.tech/abra/pkg/ssh"
 	"coopcloud.tech/tagcmp"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
 var status bool
 var statusFlag = &cli.BoolFlag{
-	Name:        "status, S",
+	Name:        "status",
+	Aliases:     []string{"S"},
+	Value:       false,
 	Usage:       "Show app deployment status",
 	Destination: &status,
 }
 
-var appRecipe string
-var recipeFlag = &cli.StringFlag{
-	Name:        "recipe, r",
+var appType string
+var typeFlag = &cli.StringFlag{
+	Name:        "type",
+	Aliases:     []string{"t"},
 	Value:       "",
-	Usage:       "Show apps of a specific recipe",
-	Destination: &appRecipe,
+	Usage:       "Show apps of a specific type",
+	Destination: &appType,
 }
 
 var listAppServer string
 var listAppServerFlag = &cli.StringFlag{
-	Name:        "server, s",
+	Name:        "server",
+	Aliases:     []string{"s"},
 	Value:       "",
 	Usage:       "Show apps of a specific server",
 	Destination: &listAppServer,
@@ -57,10 +61,9 @@ type serverStatus struct {
 	upgradeCount     int
 }
 
-var appListCommand = cli.Command{
-	Name:    "list",
-	Aliases: []string{"ls"},
-	Usage:   "List all managed apps",
+var appListCommand = &cli.Command{
+	Name:  "list",
+	Usage: "List all managed apps",
 	Description: `
 This command looks at your local file system listing of apps and servers (e.g.
 in ~/.abra/) to generate a report of all your apps.
@@ -68,14 +71,13 @@ in ~/.abra/) to generate a report of all your apps.
 By passing the "--status/-S" flag, you can query all your servers for the
 actual live deployment status. Depending on how many servers you manage, this
 can take some time.
-`,
+	`,
+	Aliases: []string{"ls"},
 	Flags: []cli.Flag{
-		internal.DebugFlag,
 		statusFlag,
 		listAppServerFlag,
-		recipeFlag,
+		typeFlag,
 	},
-	Before: internal.SubCommandBefore,
 	Action: func(c *cli.Context) error {
 		appFiles, err := config.LoadAppFiles(listAppServer)
 		if err != nil {
@@ -87,10 +89,10 @@ can take some time.
 			logrus.Fatal(err)
 		}
 
-		sort.Sort(config.ByServerAndRecipe(apps))
+		sort.Sort(config.ByServerAndType(apps))
 
 		statuses := make(map[string]map[string]string)
-		var catl recipe.RecipeCatalogue
+		var catl catalogue.RecipeCatalogue
 		if status {
 			alreadySeen := make(map[string]bool)
 			for _, app := range apps {
@@ -108,7 +110,7 @@ can take some time.
 			}
 
 			var err error
-			catl, err = recipe.ReadRecipeCatalogue()
+			catl, err = catalogue.ReadRecipeCatalogue()
 			if err != nil {
 				logrus.Fatal(err)
 			}
@@ -122,30 +124,21 @@ can take some time.
 			var ok bool
 			if stats, ok = allStats[app.Server]; !ok {
 				stats = serverStatus{}
-				if appRecipe == "" {
-					// count server, no filtering
-					totalServersCount++
-				}
+				totalServersCount++
 			}
 
-			if app.Recipe == appRecipe || appRecipe == "" {
-				if appRecipe != "" {
-					// only count server if matches filter
-					totalServersCount++
-				}
-
+			if app.Type == appType || appType == "" {
 				appStats := appStatus{}
 				stats.appCount++
 				totalAppsCount++
 
 				if status {
+					stackName := app.StackName()
 					status := "unknown"
 					version := "unknown"
-					if statusMeta, ok := statuses[app.StackName()]; ok {
+					if statusMeta, ok := statuses[stackName]; ok {
 						if currentVersion, exists := statusMeta["version"]; exists {
-							if currentVersion != "" {
-								version = currentVersion
-							}
+							version = currentVersion
 						}
 						if statusMeta["status"] != "" {
 							status = statusMeta["status"]
@@ -160,7 +153,7 @@ can take some time.
 
 					var newUpdates []string
 					if version != "unknown" {
-						updates, err := recipe.GetRecipeCatalogueVersions(app.Recipe, catl)
+						updates, err := catalogue.GetRecipeCatalogueVersions(app.Type, catl)
 						if err != nil {
 							logrus.Fatal(err)
 						}
@@ -190,15 +183,18 @@ can take some time.
 							stats.latestCount++
 						}
 					} else {
-						newUpdates = internal.ReverseStringList(newUpdates)
+						// FIXME: jeezus golang why do you not have a list reverse function
+						for i, j := 0, len(newUpdates)-1; i < j; i, j = i+1, j-1 {
+							newUpdates[i], newUpdates[j] = newUpdates[j], newUpdates[i]
+						}
 						appStats.upgrade = strings.Join(newUpdates, "\n")
 						stats.upgradeCount++
 					}
 				}
 
 				appStats.server = app.Server
-				appStats.recipe = app.Recipe
-				appStats.appName = app.Name
+				appStats.recipe = app.Type
+				appStats.appName = app.StackName()
 				appStats.domain = app.Domain
 
 				stats.apps = append(stats.apps, appStats)
@@ -207,52 +203,41 @@ can take some time.
 			allStats[app.Server] = stats
 		}
 
-		alreadySeen := make(map[string]bool)
-		for _, app := range apps {
-			if _, ok := alreadySeen[app.Server]; ok {
-				continue
-			}
-
-			serverStat := allStats[app.Server]
-
-			tableCol := []string{"recipe", "domain"}
+		for serverName, serverStat := range allStats {
+			tableCol := []string{"recipe", "app name", "domain"}
 			if status {
 				tableCol = append(tableCol, []string{"status", "version", "upgrade"}...)
 			}
 
-			table := formatter.CreateTable(tableCol)
+			table := abraFormatter.CreateTable(tableCol)
 
 			for _, appStat := range serverStat.apps {
-				tableRow := []string{appStat.recipe, appStat.domain}
+				tableRow := []string{appStat.recipe, appStat.appName, appStat.domain}
 				if status {
 					tableRow = append(tableRow, []string{appStat.status, appStat.version, appStat.upgrade}...)
 				}
 				table.Append(tableRow)
 			}
 
-			if table.NumLines() > 0 {
-				table.Render()
+			table.Render()
 
-				if status {
-					fmt.Println(fmt.Sprintf(
-						"server: %s | total apps: %v | versioned: %v | unversioned: %v | latest: %v | upgrade: %v",
-						app.Server,
-						serverStat.appCount,
-						serverStat.versionCount,
-						serverStat.unversionedCount,
-						serverStat.latestCount,
-						serverStat.upgradeCount,
-					))
-				} else {
-					fmt.Println(fmt.Sprintf("server: %s | total apps: %v", app.Server, serverStat.appCount))
-				}
+			if status {
+				fmt.Println(fmt.Sprintf(
+					"server: %s | total apps: %v | versioned: %v | unversioned: %v | latest: %v | upgrade: %v",
+					serverName,
+					serverStat.appCount,
+					serverStat.versionCount,
+					serverStat.unversionedCount,
+					serverStat.latestCount,
+					serverStat.upgradeCount,
+				))
+			} else {
+				fmt.Println(fmt.Sprintf("server: %s | total apps: %v", serverName, serverStat.appCount))
 			}
 
-			if len(allStats) > 1 && table.NumLines() > 0 {
+			if len(allStats) > 1 {
 				fmt.Println() // newline separator for multiple servers
 			}
-
-			alreadySeen[app.Server] = true
 		}
 
 		if len(allStats) > 1 {

cli/app/logs.go

@@ -1,7 +1,6 @@
 package app
 
 import (
-	"context"
 	"fmt"
 	"io"
 	"os"
@@ -11,12 +10,11 @@ import (
 	"coopcloud.tech/abra/pkg/autocomplete"
 	"coopcloud.tech/abra/pkg/client"
 	"coopcloud.tech/abra/pkg/config"
-	"coopcloud.tech/abra/pkg/service"
 	"github.com/docker/docker/api/types"
 	"github.com/docker/docker/api/types/filters"
 	dockerClient "github.com/docker/docker/client"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
 var logOpts = types.ContainerLogsOptions{
@@ -29,14 +27,11 @@ var logOpts = types.ContainerLogsOptions{
 }
 
 // stackLogs lists logs for all stack services
-func stackLogs(c *cli.Context, app config.App, client *dockerClient.Client) {
-	filters, err := app.Filters()
-	if err != nil {
-		logrus.Fatal(err)
-	}
-
+func stackLogs(c *cli.Context, stackName string, client *dockerClient.Client) {
+	filters := filters.NewArgs()
+	filters.Add("name", stackName)
 	serviceOpts := types.ServiceListOptions{Filters: filters}
-	services, err := client.ServiceList(context.Background(), serviceOpts)
+	services, err := client.ServiceList(c.Context, serviceOpts)
 	if err != nil {
 		logrus.Fatal(err)
 	}
@@ -49,10 +44,11 @@ func stackLogs(c *cli.Context, app config.App, client *dockerClient.Client) {
 				logOpts.ShowStdout = false
 			}
 
-			logs, err := client.ServiceLogs(context.Background(), s, logOpts)
+			logs, err := client.ServiceLogs(c.Context, s, logOpts)
 			if err != nil {
 				logrus.Fatal(err)
 			}
+			// defer after err check as any err returns a nil io.ReadCloser
 			defer logs.Close()
 
 			_, err = io.Copy(os.Stdout, logs)
@@ -61,22 +57,18 @@ func stackLogs(c *cli.Context, app config.App, client *dockerClient.Client) {
 			}
 		}(service.ID)
 	}
-
 	wg.Wait()
-
 	os.Exit(0)
 }
 
-var appLogsCommand = cli.Command{
+var appLogsCommand = &cli.Command{
 	Name:      "logs",
 	Aliases:   []string{"l"},
-	ArgsUsage: "<domain> [<service>]",
+	ArgsUsage: "[<service>]",
 	Usage:     "Tail app logs",
 	Flags: []cli.Flag{
 		internal.StdErrOnlyFlag,
-		internal.DebugFlag,
 	},
-	Before:       internal.SubCommandBefore,
 	BashComplete: autocomplete.AppNameComplete,
 	Action: func(c *cli.Context) error {
 		app := internal.ValidateApp(c)
@@ -88,8 +80,8 @@ var appLogsCommand = cli.Command{
 
 		serviceName := c.Args().Get(1)
 		if serviceName == "" {
-			logrus.Debugf("tailing logs for all %s services", app.Recipe)
-			stackLogs(c, app, cl)
+			logrus.Debugf("tailing logs for all %s services", app.Type)
+			stackLogs(c, app.StackName(), cl)
 		} else {
 			logrus.Debugf("tailing logs for %s", serviceName)
 			if err := tailServiceLogs(c, cl, app, serviceName); err != nil {
@@ -102,22 +94,27 @@ var appLogsCommand = cli.Command{
 }
 
 func tailServiceLogs(c *cli.Context, cl *dockerClient.Client, app config.App, serviceName string) error {
+	service := fmt.Sprintf("%s_%s", app.StackName(), serviceName)
 	filters := filters.NewArgs()
-	filters.Add("name", fmt.Sprintf("^%s_%s", app.StackName(), serviceName))
-
-	chosenService, err := service.GetService(context.Background(), cl, filters, internal.NoInput)
+	filters.Add("name", service)
+	serviceOpts := types.ServiceListOptions{Filters: filters}
+	services, err := cl.ServiceList(c.Context, serviceOpts)
 	if err != nil {
 		logrus.Fatal(err)
 	}
+	if len(services) != 1 {
+		logrus.Fatalf("expected 1 service but got %v", len(services))
+	}
 
 	if internal.StdErrOnly {
 		logOpts.ShowStdout = false
 	}
 
-	logs, err := cl.ServiceLogs(context.Background(), chosenService.ID, logOpts)
+	logs, err := cl.ServiceLogs(c.Context, services[0].ID, logOpts)
 	if err != nil {
 		logrus.Fatal(err)
 	}
+	// defer after err check as any err returns a nil io.ReadCloser
 	defer logs.Close()
 
 	_, err = io.Copy(os.Stdout, logs)

cli/app/new.go

@@ -3,7 +3,7 @@ package app
 import (
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/autocomplete"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
 var appNewDescription = `
@@ -11,7 +11,7 @@ This command takes a recipe and uses it to create a new app. This new app
 configuration is stored in your ~/.abra directory under the appropriate server.
 
 This command does not deploy your app for you. You will need to run "abra app
-deploy <domain>" to do so.
+deploy <app>" to do so.
 
 You can see what recipes are available (i.e. values for the <recipe> argument)
 by running "abra recipe ls".
@@ -26,21 +26,19 @@ pass store (see passwordstore.org for more). The pass command must be available
 on your $PATH.
 `
 
-var appNewCommand = cli.Command{
+var appNewCommand = &cli.Command{
 	Name:        "new",
-	Aliases:     []string{"n"},
 	Usage:       "Create a new app",
+	Aliases:     []string{"n"},
 	Description: appNewDescription,
 	Flags: []cli.Flag{
-		internal.DebugFlag,
-		internal.NoInputFlag,
 		internal.NewAppServerFlag,
 		internal.DomainFlag,
+		internal.NewAppNameFlag,
 		internal.PassFlag,
 		internal.SecretsFlag,
 	},
-	Before:       internal.SubCommandBefore,
-	ArgsUsage:    "[<recipe>]",
+	ArgsUsage:    "<recipe>",
 	Action:       internal.NewAction,
 	BashComplete: autocomplete.RecipeNameComplete,
 }

cli/app/ps.go

@@ -1,83 +1,71 @@
 package app
 
 import (
-	"context"
 	"strings"
 	"time"
 
+	abraFormatter "coopcloud.tech/abra/cli/formatter"
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/autocomplete"
 	"coopcloud.tech/abra/pkg/client"
-	"coopcloud.tech/abra/pkg/config"
-	"coopcloud.tech/abra/pkg/formatter"
-	"coopcloud.tech/abra/pkg/service"
-	stack "coopcloud.tech/abra/pkg/upstream/stack"
-	"github.com/buger/goterm"
-	dockerFormatter "github.com/docker/cli/cli/command/formatter"
+	"github.com/docker/cli/cli/command/formatter"
 	"github.com/docker/docker/api/types"
-	dockerClient "github.com/docker/docker/client"
+	"github.com/docker/docker/api/types/filters"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
-var appPsCommand = cli.Command{
+var watch bool
+var watchFlag = &cli.BoolFlag{
+	Name:        "watch",
+	Aliases:     []string{"w"},
+	Value:       false,
+	Usage:       "Watch status by polling repeatedly",
+	Destination: &watch,
+}
+
+var appPsCommand = &cli.Command{
 	Name:        "ps",
-	Aliases:     []string{"p"},
 	Usage:       "Check app status",
-	ArgsUsage:   "<domain>",
 	Description: "This command shows a more detailed status output of a specific deployed app.",
+	Aliases:     []string{"p"},
 	Flags: []cli.Flag{
-		internal.WatchFlag,
-		internal.DebugFlag,
+		watchFlag,
 	},
-	Before:       internal.SubCommandBefore,
 	BashComplete: autocomplete.AppNameComplete,
 	Action: func(c *cli.Context) error {
-		app := internal.ValidateApp(c)
-
-		cl, err := client.New(app.Server)
-		if err != nil {
-			logrus.Fatal(err)
-		}
-
-		isDeployed, _, err := stack.IsDeployed(context.Background(), cl, app.StackName())
-		if err != nil {
-			logrus.Fatal(err)
-		}
-
-		if !isDeployed {
-			logrus.Fatalf("%s is not deployed?", app.Name)
-		}
-
-		if !internal.Watch {
-			showPSOutput(c, app, cl)
+		if !watch {
+			showPSOutput(c)
 			return nil
 		}
 
-		goterm.Clear()
+		// TODO: how do we make this update in-place in an x-platform way?
 		for {
-			goterm.MoveCursor(1, 1)
-			showPSOutput(c, app, cl)
-			goterm.Flush()
+			showPSOutput(c)
 			time.Sleep(2 * time.Second)
 		}
 	},
 }
 
 // showPSOutput renders ps output.
-func showPSOutput(c *cli.Context, app config.App, cl *dockerClient.Client) {
-	filters, err := app.Filters()
+func showPSOutput(c *cli.Context) {
+	app := internal.ValidateApp(c)
+
+	cl, err := client.New(app.Server)
 	if err != nil {
 		logrus.Fatal(err)
 	}
 
-	containers, err := cl.ContainerList(context.Background(), types.ContainerListOptions{Filters: filters})
+	filters := filters.NewArgs()
+	filters.Add("name", app.StackName())
+
+	containers, err := cl.ContainerList(c.Context, types.ContainerListOptions{Filters: filters})
 	if err != nil {
 		logrus.Fatal(err)
 	}
 
-	tableCol := []string{"service name", "image", "created", "status", "state", "ports"}
-	table := formatter.CreateTable(tableCol)
+	tableCol := []string{"image", "created", "status", "ports"}
+	table := abraFormatter.CreateTable(tableCol)
 
 	for _, container := range containers {
 		var containerNames []string
@@ -87,12 +75,10 @@ func showPSOutput(c *cli.Context, app config.App, cl *dockerClient.Client) {
 		}
 
 		tableRow := []string{
-			service.ContainerToServiceName(container.Names, app.StackName()),
-			formatter.RemoveSha(container.Image),
-			formatter.HumanDuration(container.Created),
+			abraFormatter.RemoveSha(container.Image),
+			abraFormatter.HumanDuration(container.Created),
 			container.Status,
-			container.State,
-			dockerFormatter.DisplayablePorts(container.Ports),
+			formatter.DisplayablePorts(container.Ports),
 		}
 		table.Append(tableRow)
 	}

cli/app/remove.go

@@ -1,7 +1,6 @@
 package app
 
 import (
-	"context"
 	"fmt"
 	"os"
 
@@ -11,8 +10,9 @@ import (
 	stack "coopcloud.tech/abra/pkg/upstream/stack"
 	"github.com/AlecAivazis/survey/v2"
 	"github.com/docker/docker/api/types"
+	"github.com/docker/docker/api/types/filters"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
 // Volumes stores the variable from VolumesFlag
@@ -20,35 +20,32 @@ var Volumes bool
 
 // VolumesFlag is used to specify if volumes should be deleted when deleting an app
 var VolumesFlag = &cli.BoolFlag{
-	Name:        "volumes, V",
+	Name:        "volumes",
+	Value:       false,
 	Destination: &Volumes,
 }
 
-var appRemoveCommand = cli.Command{
-	Name:      "remove",
-	Aliases:   []string{"rm"},
-	ArgsUsage: "<domain>",
-	Usage:     "Remove an already undeployed app",
+var appRemoveCommand = &cli.Command{
+	Name:    "remove",
+	Usage:   "Remove an already undeployed app",
+	Aliases: []string{"rm"},
 	Flags: []cli.Flag{
 		VolumesFlag,
 		internal.ForceFlag,
-		internal.DebugFlag,
-		internal.NoInputFlag,
 	},
-	Before: internal.SubCommandBefore,
 	Action: func(c *cli.Context) error {
 		app := internal.ValidateApp(c)
 
-		if !internal.Force && !internal.NoInput {
+		if !internal.Force {
 			response := false
 			prompt := &survey.Confirm{
-				Message: fmt.Sprintf("about to remove %s, are you sure?", app.Name),
+				Message: fmt.Sprintf("about to delete %s, are you sure?", app.Name),
 			}
 			if err := survey.AskOne(prompt, &response); err != nil {
 				logrus.Fatal(err)
 			}
 			if !response {
-				logrus.Fatal("aborting as requested")
+				logrus.Fatal("user aborted app removal")
 			}
 		}
 
@@ -57,20 +54,19 @@ var appRemoveCommand = cli.Command{
 			logrus.Fatal(err)
 		}
 
-		isDeployed, _, err := stack.IsDeployed(context.Background(), cl, app.StackName())
-		if err != nil {
-			logrus.Fatal(err)
-		}
-		if isDeployed {
-			logrus.Fatalf("%s is still deployed. Run \"abra app undeploy %s\"", app.Name, app.Name)
+		if !internal.Force {
+			isDeployed, _, err := stack.IsDeployed(c.Context, cl, app.StackName())
+			if err != nil {
+				logrus.Fatal(err)
+			}
+			if isDeployed {
+				logrus.Fatalf("'%s' is still deployed. Run \"abra app undeploy %s \" or pass --force", app.Name, app.Name)
+			}
 		}
 
-		fs, err := app.Filters()
-		if err != nil {
-			logrus.Fatal(err)
-		}
-
-		secretList, err := cl.SecretList(context.Background(), types.SecretListOptions{Filters: fs})
+		fs := filters.NewArgs()
+		fs.Add("name", app.Name)
+		secretList, err := cl.SecretList(c.Context, types.SecretListOptions{Filters: fs})
 		if err != nil {
 			logrus.Fatal(err)
 		}
@@ -85,8 +81,7 @@ var appRemoveCommand = cli.Command{
 
 		if len(secrets) > 0 {
 			var secretNamesToRemove []string
-
-			if !internal.Force && !internal.NoInput {
+			if !internal.Force {
 				secretsPrompt := &survey.MultiSelect{
 					Message: "which secrets do you want to remove?",
 					Help:    "'x' indicates selected, enter / return to confirm, ctrl-c to exit, vim mode is enabled",
@@ -99,12 +94,8 @@ var appRemoveCommand = cli.Command{
 				}
 			}
 
-			if internal.Force || internal.NoInput {
-				secretNamesToRemove = secretNames
-			}
-
 			for _, name := range secretNamesToRemove {
-				err := cl.SecretRemove(context.Background(), secrets[name])
+				err := cl.SecretRemove(c.Context, secrets[name])
 				if err != nil {
 					logrus.Fatal(err)
 				}
@@ -114,7 +105,7 @@ var appRemoveCommand = cli.Command{
 			logrus.Info("no secrets to remove")
 		}
 
-		volumeListOKBody, err := cl.VolumeList(context.Background(), fs)
+		volumeListOKBody, err := cl.VolumeList(c.Context, fs)
 		volumeList := volumeListOKBody.Volumes
 		if err != nil {
 			logrus.Fatal(err)
@@ -128,7 +119,7 @@ var appRemoveCommand = cli.Command{
 		if len(vols) > 0 {
 			if Volumes {
 				var removeVols []string
-				if !internal.Force && !internal.NoInput {
+				if !internal.Force {
 					volumesPrompt := &survey.MultiSelect{
 						Message: "which volumes do you want to remove?",
 						Help:    "'x' indicates selected, enter / return to confirm, ctrl-c to exit, vim mode is enabled",
@@ -140,9 +131,8 @@ var appRemoveCommand = cli.Command{
 						logrus.Fatal(err)
 					}
 				}
-
 				for _, vol := range removeVols {
-					err := cl.VolumeRemove(context.Background(), vol, internal.Force) // last argument is for force removing
+					err := cl.VolumeRemove(c.Context, vol, internal.Force) // last argument is for force removing
 					if err != nil {
 						logrus.Fatal(err)
 					}
@@ -152,9 +142,7 @@ var appRemoveCommand = cli.Command{
 				logrus.Info("no volumes were removed")
 			}
 		} else {
-			if Volumes {
-				logrus.Info("no volumes to remove")
-			}
+			logrus.Info("no volumes to remove")
 		}
 
 		err = os.Remove(app.Path)

cli/app/restart.go

@@ -1,35 +1,30 @@
 package app
 
 import (
-	"context"
 	"errors"
 	"fmt"
+	"time"
 
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/autocomplete"
 	"coopcloud.tech/abra/pkg/client"
-	upstream "coopcloud.tech/abra/pkg/upstream/service"
-	stack "coopcloud.tech/abra/pkg/upstream/stack"
+	containerPkg "coopcloud.tech/abra/pkg/container"
+	"github.com/docker/docker/api/types/filters"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
-var appRestartCommand = cli.Command{
-	Name:      "restart",
-	Aliases:   []string{"re"},
-	Usage:     "Restart an app",
-	ArgsUsage: "<domain>",
-	Flags: []cli.Flag{
-		internal.DebugFlag,
-	},
-	Before:       internal.SubCommandBefore,
-	Description:  `This command restarts a service within a deployed app.`,
-	BashComplete: autocomplete.AppNameComplete,
+var appRestartCommand = &cli.Command{
+	Name:        "restart",
+	Usage:       "Restart an app",
+	Aliases:     []string{"R"},
+	ArgsUsage:   "<service>",
+	Description: `This command restarts a service within a deployed app.`,
 	Action: func(c *cli.Context) error {
 		app := internal.ValidateApp(c)
 
-		serviceNameShort := c.Args().Get(1)
-		if serviceNameShort == "" {
+		serviceName := c.Args().Get(1)
+		if serviceName == "" {
 			err := errors.New("missing service?")
 			internal.ShowSubcommandHelpAndError(c, err)
 		}
@@ -39,32 +34,25 @@ var appRestartCommand = cli.Command{
 			logrus.Fatal(err)
 		}
 
-		serviceName := fmt.Sprintf("%s_%s", app.StackName(), serviceNameShort)
+		serviceFilter := fmt.Sprintf("%s_%s", app.StackName(), serviceName)
+		filters := filters.NewArgs()
+		filters.Add("name", serviceFilter)
 
-		logrus.Debugf("attempting to scale %s to 0 (restart logic)", serviceName)
-		if err := upstream.RunServiceScale(context.Background(), cl, serviceName, 0); err != nil {
+		targetContainer, err := containerPkg.GetContainer(c.Context, cl, filters, true)
+		if err != nil {
 			logrus.Fatal(err)
 		}
 
-		if err := stack.WaitOnService(context.Background(), cl, serviceName, app.Name); err != nil {
+		logrus.Debugf("attempting to restart %s", serviceFilter)
+
+		timeout := 30 * time.Second
+		if err := cl.ContainerRestart(c.Context, targetContainer.ID, &timeout); err != nil {
 			logrus.Fatal(err)
 		}
 
-		logrus.Debugf("%s has been scaled to 0 (restart logic)", serviceName)
-
-		logrus.Debugf("attempting to scale %s to 1 (restart logic)", serviceName)
-		if err := upstream.RunServiceScale(context.Background(), cl, serviceName, 1); err != nil {
-			logrus.Fatal(err)
-		}
-
-		if err := stack.WaitOnService(context.Background(), cl, serviceName, app.Name); err != nil {
-			logrus.Fatal(err)
-		}
-
-		logrus.Debugf("%s has been scaled to 1 (restart logic)", serviceName)
-
-		logrus.Infof("%s service successfully restarted", serviceNameShort)
+		logrus.Infof("%s service restarted", serviceFilter)
 
 		return nil
 	},
+	BashComplete: autocomplete.AppNameComplete,
 }

cli/app/restore.go

@@ -0,0 +1,79 @@
+package app
+
+import (
+	"errors"
+	"fmt"
+	"io/ioutil"
+	"os"
+	"os/exec"
+	"path"
+	"strings"
+
+	"coopcloud.tech/abra/cli/internal"
+	"coopcloud.tech/abra/pkg/config"
+	"github.com/sirupsen/logrus"
+	"github.com/urfave/cli/v2"
+)
+
+var restoreAllServices bool
+var restoreAllServicesFlag = &cli.BoolFlag{
+	Name:        "all",
+	Value:       false,
+	Destination: &restoreAllServices,
+	Aliases:     []string{"a"},
+	Usage:       "Restore all services",
+}
+
+var appRestoreCommand = &cli.Command{
+	Name:      "restore",
+	Usage:     "Restore an app from a backup",
+	Aliases:   []string{"r"},
+	Flags:     []cli.Flag{restoreAllServicesFlag},
+	ArgsUsage: "<service> [<backup file>]",
+	Action: func(c *cli.Context) error {
+		app := internal.ValidateApp(c)
+
+		if c.Args().Len() > 1 && restoreAllServices {
+			internal.ShowSubcommandHelpAndError(c, errors.New("cannot use <service>/<backup file> and '--all' together"))
+		}
+
+		abraSh := path.Join(config.ABRA_DIR, "apps", app.Type, "abra.sh")
+		if _, err := os.Stat(abraSh); err != nil {
+			if os.IsNotExist(err) {
+				logrus.Fatalf("'%s' does not exist?", abraSh)
+			}
+			logrus.Fatal(err)
+		}
+
+		sourceCmd := fmt.Sprintf("source %s", abraSh)
+		execCmd := "abra_restore"
+		if !restoreAllServices {
+			serviceName := c.Args().Get(1)
+			if serviceName == "" {
+				internal.ShowSubcommandHelpAndError(c, errors.New("no service(s) target provided"))
+			}
+			execCmd = fmt.Sprintf("abra_restore_%s", serviceName)
+		}
+
+		bytes, err := ioutil.ReadFile(abraSh)
+		if err != nil {
+			logrus.Fatal(err)
+		}
+		if !strings.Contains(string(bytes), execCmd) {
+			logrus.Fatalf("%s doesn't have a '%s' function", app.Type, execCmd)
+		}
+
+		backupFile := c.Args().Get(2)
+		if backupFile != "" {
+			execCmd = fmt.Sprintf("%s %s", execCmd, backupFile)
+		}
+
+		sourceAndExec := fmt.Sprintf("%s; %s", sourceCmd, execCmd)
+		cmd := exec.Command("bash", "-c", sourceAndExec)
+		if err := internal.RunCmd(cmd); err != nil {
+			logrus.Fatal(err)
+		}
+
+		return nil
+	},
+}

cli/app/rollback.go

@@ -1,12 +1,11 @@
 package app
 
 import (
-	"context"
 	"fmt"
 
 	"coopcloud.tech/abra/pkg/autocomplete"
+	"coopcloud.tech/abra/pkg/catalogue"
 	"coopcloud.tech/abra/pkg/config"
-	"coopcloud.tech/abra/pkg/lint"
 	"coopcloud.tech/abra/pkg/recipe"
 	stack "coopcloud.tech/abra/pkg/upstream/stack"
 	"coopcloud.tech/tagcmp"
@@ -15,23 +14,18 @@ import (
 	"coopcloud.tech/abra/pkg/client"
 	"github.com/AlecAivazis/survey/v2"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
-var appRollbackCommand = cli.Command{
+var appRollbackCommand = &cli.Command{
 	Name:      "rollback",
-	Aliases:   []string{"rl"},
 	Usage:     "Roll an app back to a previous version",
-	ArgsUsage: "<domain>",
+	Aliases:   []string{"r", "downgrade"},
+	ArgsUsage: "<app>",
 	Flags: []cli.Flag{
-		internal.DebugFlag,
-		internal.NoInputFlag,
 		internal.ForceFlag,
 		internal.ChaosFlag,
-		internal.NoDomainChecksFlag,
-		internal.DontWaitConvergeFlag,
 	},
-	Before: internal.SubCommandBefore,
 	Description: `
 This command rolls an app back to a previous version if one exists.
 
@@ -39,7 +33,7 @@ You may pass "--force/-f" to downgrade to the same version again. This can be
 useful if the container runtime has gotten into a weird state.
 
 This action could be destructive, please ensure you have a copy of your app
-data beforehand.
+data beforehand - see "abra app backup <app>" for more.
 
 Chas mode ("--chaos") will deploy your local checkout of a recipe as-is,
 including unstaged changes and can be useful for live hacking and testing new
@@ -50,55 +44,37 @@ recipes.
 		app := internal.ValidateApp(c)
 		stackName := app.StackName()
 
-		if !internal.Chaos {
-			if err := recipe.EnsureUpToDate(app.Recipe); err != nil {
-				logrus.Fatal(err)
-			}
-		}
-
-		r, err := recipe.Get(app.Recipe)
-		if err != nil {
-			logrus.Fatal(err)
-		}
-
-		if err := lint.LintForErrors(r); err != nil {
-			logrus.Fatal(err)
-		}
-
 		cl, err := client.New(app.Server)
 		if err != nil {
 			logrus.Fatal(err)
 		}
 
-		logrus.Debugf("checking whether %s is already deployed", stackName)
+		logrus.Debugf("checking whether '%s' is already deployed", stackName)
 
-		isDeployed, deployedVersion, err := stack.IsDeployed(context.Background(), cl, stackName)
+		isDeployed, deployedVersion, err := stack.IsDeployed(c.Context, cl, stackName)
 		if err != nil {
 			logrus.Fatal(err)
 		}
 
 		if !isDeployed {
-			logrus.Fatalf("%s is not deployed?", app.Name)
+			logrus.Fatalf("'%s' is not deployed?", app.Name)
 		}
 
-		catl, err := recipe.ReadRecipeCatalogue()
+		catl, err := catalogue.ReadRecipeCatalogue()
 		if err != nil {
 			logrus.Fatal(err)
 		}
 
-		versions, err := recipe.GetRecipeCatalogueVersions(app.Recipe, catl)
+		versions, err := catalogue.GetRecipeCatalogueVersions(app.Type, catl)
 		if err != nil {
 			logrus.Fatal(err)
 		}
 
-		if len(versions) == 0 && !internal.Chaos {
-			logrus.Fatalf("no published releases for %s in the recipe catalogue?", app.Recipe)
-		}
-
 		var availableDowngrades []string
-		if deployedVersion == "unknown" {
+		if deployedVersion == "" {
+			deployedVersion = "unknown"
 			availableDowngrades = versions
-			logrus.Warnf("failed to determine version of deployed %s", app.Name)
+			logrus.Warnf("failed to determine version of deployed '%s'", app.Name)
 		}
 
 		if deployedVersion != "unknown" && !internal.Chaos {
@@ -122,16 +98,19 @@ recipes.
 			}
 		}
 
-		availableDowngrades = internal.ReverseStringList(availableDowngrades)
+		// FIXME: jeezus golang why do you not have a list reverse function
+		for i, j := 0, len(availableDowngrades)-1; i < j; i, j = i+1, j-1 {
+			availableDowngrades[i], availableDowngrades[j] = availableDowngrades[j], availableDowngrades[i]
+		}
 
 		var chosenDowngrade string
 		if !internal.Chaos {
-			if internal.Force || internal.NoInput {
+			if internal.Force {
 				chosenDowngrade = availableDowngrades[0]
-				logrus.Debugf("choosing %s as version to downgrade to (--force)", chosenDowngrade)
+				logrus.Debugf("choosing '%s' as version to downgrade to (--force)", chosenDowngrade)
 			} else {
 				prompt := &survey.Select{
-					Message: fmt.Sprintf("Please select a downgrade (current version: %s):", deployedVersion),
+					Message: fmt.Sprintf("Please select a downgrade (current version: '%s'):", deployedVersion),
 					Options: availableDowngrades,
 				}
 				if err := survey.AskOne(prompt, &chosenDowngrade); err != nil {
@@ -141,7 +120,7 @@ recipes.
 		}
 
 		if !internal.Chaos {
-			if err := recipe.EnsureVersion(app.Recipe, chosenDowngrade); err != nil {
+			if err := recipe.EnsureVersion(app.Type, chosenDowngrade); err != nil {
 				logrus.Fatal(err)
 			}
 		}
@@ -149,13 +128,13 @@ recipes.
 		if internal.Chaos {
 			logrus.Warn("chaos mode engaged")
 			var err error
-			chosenDowngrade, err = recipe.ChaosVersion(app.Recipe)
+			chosenDowngrade, err = recipe.ChaosVersion(app.Type)
 			if err != nil {
 				logrus.Fatal(err)
 			}
 		}
 
-		abraShPath := fmt.Sprintf("%s/%s/%s", config.RECIPES_DIR, app.Recipe, "abra.sh")
+		abraShPath := fmt.Sprintf("%s/%s/%s", config.APPS_DIR, app.Type, "abra.sh")
 		abraShEnv, err := config.ReadAbraShEnvVars(abraShPath)
 		if err != nil {
 			logrus.Fatal(err)
@@ -164,7 +143,7 @@ recipes.
 			app.Env[k] = v
 		}
 
-		composeFiles, err := config.GetAppComposeFiles(app.Recipe, app.Env)
+		composeFiles, err := config.GetAppComposeFiles(app.Type, app.Env)
 		if err != nil {
 			logrus.Fatal(err)
 		}
@@ -180,12 +159,12 @@ recipes.
 		}
 
 		if !internal.Force {
-			if err := internal.NewVersionOverview(app, deployedVersion, chosenDowngrade, ""); err != nil {
+			if err := internal.NewVersionOverview(app, deployedVersion, chosenDowngrade); err != nil {
 				logrus.Fatal(err)
 			}
 		}
 
-		if err := stack.RunDeploy(cl, deployOpts, compose, app.StackName(), internal.DontWaitConverge); err != nil {
+		if err := stack.RunDeploy(cl, deployOpts, compose, app.Type); err != nil {
 			logrus.Fatal(err)
 		}
 

cli/app/run.go

@@ -1,55 +1,52 @@
 package app
 
 import (
-	"context"
 	"errors"
 	"fmt"
 
 	"coopcloud.tech/abra/cli/internal"
-	"coopcloud.tech/abra/pkg/autocomplete"
 	"coopcloud.tech/abra/pkg/client"
+	"coopcloud.tech/abra/pkg/config"
 	containerPkg "coopcloud.tech/abra/pkg/container"
 	"coopcloud.tech/abra/pkg/upstream/container"
 	"github.com/docker/cli/cli/command"
 	"github.com/docker/docker/api/types"
 	"github.com/docker/docker/api/types/filters"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
 var user string
 var userFlag = &cli.StringFlag{
-	Name:        "user, u",
+	Name:        "user",
 	Value:       "",
 	Destination: &user,
 }
 
 var noTTY bool
 var noTTYFlag = &cli.BoolFlag{
-	Name:        "no-tty, t",
+	Name:        "no-tty",
+	Value:       false,
 	Destination: &noTTY,
 }
 
-var appRunCommand = cli.Command{
-	Name:    "run",
-	Aliases: []string{"r"},
+var appRunCommand = &cli.Command{
+	Name: "run",
 	Flags: []cli.Flag{
-		internal.DebugFlag,
 		noTTYFlag,
 		userFlag,
 	},
-	Before:       internal.SubCommandBefore,
-	ArgsUsage:    "<domain> <service> <args>...",
-	Usage:        "Run a command in a service container",
-	BashComplete: autocomplete.AppNameComplete,
+	Aliases:   []string{"r"},
+	ArgsUsage: "<service> <args>...",
+	Usage:     "Run a command in a service container",
 	Action: func(c *cli.Context) error {
 		app := internal.ValidateApp(c)
 
-		if len(c.Args()) < 2 {
+		if c.Args().Len() < 2 {
 			internal.ShowSubcommandHelpAndError(c, errors.New("no <service> provided?"))
 		}
 
-		if len(c.Args()) < 3 {
+		if c.Args().Len() < 3 {
 			internal.ShowSubcommandHelpAndError(c, errors.New("no <args> provided?"))
 		}
 
@@ -59,16 +56,16 @@ var appRunCommand = cli.Command{
 		}
 
 		serviceName := c.Args().Get(1)
-		stackAndServiceName := fmt.Sprintf("^%s_%s", app.StackName(), serviceName)
+		stackAndServiceName := fmt.Sprintf("%s_%s", app.StackName(), serviceName)
 		filters := filters.NewArgs()
 		filters.Add("name", stackAndServiceName)
 
-		targetContainer, err := containerPkg.GetContainer(context.Background(), cl, filters, false)
+		targetContainer, err := containerPkg.GetContainer(c.Context, cl, filters, true)
 		if err != nil {
 			logrus.Fatal(err)
 		}
 
-		cmd := c.Args()[2:]
+		cmd := c.Args().Slice()[2:]
 		execCreateOpts := types.ExecConfig{
 			AttachStderr: true,
 			AttachStdin:  true,
@@ -85,7 +82,11 @@ var appRunCommand = cli.Command{
 			execCreateOpts.Tty = false
 		}
 
-		// FIXME: avoid instantiating a new CLI
+		// FIXME: an absolutely monumental hack to instantiate another command-line
+		// client withing our command-line client so that we pass something down
+		// the tubes that satisfies the necessary interface requirements. We should
+		// refactor our vendored container code to not require all this cruft.  For
+		// now, It Works.
 		dcli, err := command.NewDockerCli()
 		if err != nil {
 			logrus.Fatal(err)
@@ -97,4 +98,25 @@ var appRunCommand = cli.Command{
 
 		return nil
 	},
+	BashComplete: func(c *cli.Context) {
+		switch c.NArg() {
+		case 0:
+			appNames, err := config.GetAppNames()
+			if err != nil {
+				logrus.Warn(err)
+			}
+			for _, a := range appNames {
+				fmt.Println(a)
+			}
+		case 1:
+			appName := c.Args().First()
+			serviceNames, err := config.GetAppServiceNames(appName)
+			if err != nil {
+				logrus.Warn(err)
+			}
+			for _, s := range serviceNames {
+				fmt.Println(s)
+			}
+		}
+	},
 }

cli/app/secret.go

@@ -1,54 +1,41 @@
 package app
 
 import (
-	"context"
 	"errors"
 	"fmt"
 	"os"
 	"strconv"
 
+	abraFormatter "coopcloud.tech/abra/cli/formatter"
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/autocomplete"
 	"coopcloud.tech/abra/pkg/client"
-	"coopcloud.tech/abra/pkg/config"
-	"coopcloud.tech/abra/pkg/formatter"
 	"coopcloud.tech/abra/pkg/secret"
 	"github.com/docker/docker/api/types"
-	dockerClient "github.com/docker/docker/client"
+	"github.com/docker/docker/api/types/filters"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
 var allSecrets bool
 var allSecretsFlag = &cli.BoolFlag{
-	Name:        "all, a",
+	Name:        "all",
+	Aliases:     []string{"A"},
+	Value:       false,
 	Destination: &allSecrets,
 	Usage:       "Generate all secrets",
 }
 
-var rmAllSecrets bool
-var rmAllSecretsFlag = &cli.BoolFlag{
-	Name:        "all, a",
-	Destination: &rmAllSecrets,
-	Usage:       "Remove all secrets",
-}
-
-var appSecretGenerateCommand = cli.Command{
+var appSecretGenerateCommand = &cli.Command{
 	Name:      "generate",
 	Aliases:   []string{"g"},
 	Usage:     "Generate secrets",
-	ArgsUsage: "<domain> <secret> <version>",
-	Flags: []cli.Flag{
-		internal.DebugFlag,
-		allSecretsFlag,
-		internal.PassFlag,
-	},
-	Before:       internal.SubCommandBefore,
-	BashComplete: autocomplete.AppNameComplete,
+	ArgsUsage: "<secret> <version>",
+	Flags:     []cli.Flag{allSecretsFlag, internal.PassFlag},
 	Action: func(c *cli.Context) error {
 		app := internal.ValidateApp(c)
 
-		if len(c.Args()) == 1 && !allSecrets {
+		if c.Args().Len() == 1 && !allSecrets {
 			err := errors.New("missing arguments <secret>/<version> or '--all'")
 			internal.ShowSubcommandHelpAndError(c, err)
 		}
@@ -70,12 +57,10 @@ var appSecretGenerateCommand = cli.Command{
 				parsed := secret.ParseSecretEnvVarName(sec)
 				if secretName == parsed {
 					secretsToCreate[sec] = secretVersion
-					matches = true
 				}
 			}
-
 			if !matches {
-				logrus.Fatalf("%s doesn't exist in the env config?", secretName)
+				logrus.Fatalf("'%s' doesn't exist in the env config?", secretName)
 			}
 		}
 
@@ -86,7 +71,7 @@ var appSecretGenerateCommand = cli.Command{
 
 		if internal.Pass {
 			for name, data := range secretVals {
-				if err := secret.PassInsertSecret(data, name, app.Name, app.Server); err != nil {
+				if err := secret.PassInsertSecret(data, name, app.StackName(), app.Server); err != nil {
 					logrus.Fatal(err)
 				}
 			}
@@ -98,7 +83,7 @@ var appSecretGenerateCommand = cli.Command{
 		}
 
 		tableCol := []string{"name", "value"}
-		table := formatter.CreateTable(tableCol)
+		table := abraFormatter.CreateTable(tableCol)
 		for name, val := range secretVals {
 			table.Append([]string{name, val})
 		}
@@ -109,17 +94,12 @@ var appSecretGenerateCommand = cli.Command{
 	},
 }
 
-var appSecretInsertCommand = cli.Command{
-	Name:    "insert",
-	Aliases: []string{"i"},
-	Usage:   "Insert secret",
-	Flags: []cli.Flag{
-		internal.DebugFlag,
-		internal.PassFlag,
-	},
-	Before:       internal.SubCommandBefore,
-	ArgsUsage:    "<domain> <secret-name> <version> <data>",
-	BashComplete: autocomplete.AppNameComplete,
+var appSecretInsertCommand = &cli.Command{
+	Name:      "insert",
+	Aliases:   []string{"i"},
+	Usage:     "Insert secret",
+	Flags:     []cli.Flag{internal.PassFlag},
+	ArgsUsage: "<app> <secret-name> <version> <data>",
 	Description: `
 This command inserts a secret into an app environment.
 
@@ -135,7 +115,7 @@ Example:
 	Action: func(c *cli.Context) error {
 		app := internal.ValidateApp(c)
 
-		if len(c.Args()) != 4 {
+		if c.Args().Len() != 4 {
 			internal.ShowSubcommandHelpAndError(c, errors.New("missing arguments?"))
 		}
 
@@ -148,10 +128,8 @@ Example:
 			logrus.Fatal(err)
 		}
 
-		logrus.Infof("%s successfully stored on server", secretName)
-
 		if internal.Pass {
-			if err := secret.PassInsertSecret(data, name, app.Name, app.Server); err != nil {
+			if err := secret.PassInsertSecret(data, name, app.StackName(), app.Server); err != nil {
 				logrus.Fatal(err)
 			}
 		}
@@ -160,54 +138,28 @@ Example:
 	},
 }
 
-// secretRm removes a secret.
-func secretRm(cl *dockerClient.Client, app config.App, secretName, parsed string) error {
-	if err := cl.SecretRemove(context.Background(), secretName); err != nil {
-		return err
-	}
-
-	logrus.Infof("deleted %s successfully from server", secretName)
-
-	if internal.PassRemove {
-		if err := secret.PassRmSecret(parsed, app.StackName(), app.Server); err != nil {
-			return err
-		}
-
-		logrus.Infof("deleted %s successfully from local pass store", secretName)
-	}
-
-	return nil
-}
-
-var appSecretRmCommand = cli.Command{
-	Name:    "remove",
-	Aliases: []string{"rm"},
-	Usage:   "Remove a secret",
-	Flags: []cli.Flag{
-		internal.DebugFlag,
-		internal.NoInputFlag,
-		rmAllSecretsFlag,
-		internal.PassRemoveFlag,
-	},
-	Before:       internal.SubCommandBefore,
-	ArgsUsage:    "<domain> [<secret-name>]",
-	BashComplete: autocomplete.AppNameComplete,
+var appSecretRmCommand = &cli.Command{
+	Name:      "remove",
+	Usage:     "Remove a secret",
+	Aliases:   []string{"rm"},
+	Flags:     []cli.Flag{allSecretsFlag, internal.PassFlag},
+	ArgsUsage: "<app> <secret-name>",
 	Description: `
-This command removes app secrets.
+This command removes a secret from an app environment.
 
 Example:
 
     abra app secret remove myapp db_pass
+
 `,
 	Action: func(c *cli.Context) error {
 		app := internal.ValidateApp(c)
-		secrets := secret.ReadSecretEnvVars(app.Env)
 
-		if c.Args().Get(1) != "" && rmAllSecrets {
+		if c.Args().Get(1) != "" && allSecrets {
 			internal.ShowSubcommandHelpAndError(c, errors.New("cannot use '<secret-name>' and '--all' together"))
 		}
 
-		if c.Args().Get(1) == "" && !rmAllSecrets {
+		if c.Args().Get(1) == "" && !allSecrets {
 			internal.ShowSubcommandHelpAndError(c, errors.New("no secret(s) specified?"))
 		}
 
@@ -216,89 +168,63 @@ Example:
 			logrus.Fatal(err)
 		}
 
-		filters, err := app.Filters()
+		filters := filters.NewArgs()
+		filters.Add("name", app.StackName())
+		secretList, err := cl.SecretList(c.Context, types.SecretListOptions{Filters: filters})
 		if err != nil {
 			logrus.Fatal(err)
 		}
 
-		secretList, err := cl.SecretList(context.Background(), types.SecretListOptions{Filters: filters})
-		if err != nil {
-			logrus.Fatal(err)
-		}
-
-		remoteSecretNames := make(map[string]bool)
-		for _, cont := range secretList {
-			remoteSecretNames[cont.Spec.Annotations.Name] = true
-		}
-
-		match := false
 		secretToRm := c.Args().Get(1)
-		for sec := range secrets {
-			secretName := secret.ParseSecretEnvVarName(sec)
-
-			secVal, err := secret.ParseSecretEnvVarValue(secrets[sec])
-			if err != nil {
-				logrus.Fatal(err)
-			}
-
-			secretRemoteName := fmt.Sprintf("%s_%s_%s", app.StackName(), secretName, secVal.Version)
-			if _, ok := remoteSecretNames[secretRemoteName]; ok {
-				if secretToRm != "" {
-					if secretName == secretToRm {
-						if err := secretRm(cl, app, secretRemoteName, secretName); err != nil {
-							logrus.Fatal(err)
-						}
-
-						return nil
-					}
-				} else {
-					match = true
-
-					if err := secretRm(cl, app, secretRemoteName, secretName); err != nil {
+		for _, cont := range secretList {
+			secretName := cont.Spec.Annotations.Name
+			parsed := secret.ParseGeneratedSecretName(secretName, app)
+			if allSecrets {
+				if err := cl.SecretRemove(c.Context, secretName); err != nil {
+					logrus.Fatal(err)
+				}
+				if internal.Pass {
+					if err := secret.PassRmSecret(parsed, app.StackName(), app.Server); err != nil {
 						logrus.Fatal(err)
 					}
 				}
+			} else {
+				if parsed == secretToRm {
+					if err := cl.SecretRemove(c.Context, secretName); err != nil {
+						logrus.Fatal(err)
+					}
+					if internal.Pass {
+						if err := secret.PassRmSecret(parsed, app.StackName(), app.Server); err != nil {
+							logrus.Fatal(err)
+						}
+					}
+				}
 			}
 		}
 
-		if !match && secretToRm != "" {
-			logrus.Fatalf("%s doesn't exist on server?", secretToRm)
-		}
-
-		if !match {
-			logrus.Fatal("no secrets to remove?")
-		}
-
 		return nil
 	},
 }
 
-var appSecretLsCommand = cli.Command{
+var appSecretLsCommand = &cli.Command{
 	Name:    "list",
+	Usage:   "List all secrets",
 	Aliases: []string{"ls"},
-	Flags: []cli.Flag{
-		internal.DebugFlag,
-	},
-	Before: internal.SubCommandBefore,
-	Usage:  "List all secrets",
 	Action: func(c *cli.Context) error {
 		app := internal.ValidateApp(c)
 		secrets := secret.ReadSecretEnvVars(app.Env)
 
 		tableCol := []string{"Name", "Version", "Generated Name", "Created On Server"}
-		table := formatter.CreateTable(tableCol)
+		table := abraFormatter.CreateTable(tableCol)
 
 		cl, err := client.New(app.Server)
 		if err != nil {
 			logrus.Fatal(err)
 		}
 
-		filters, err := app.Filters()
-		if err != nil {
-			logrus.Fatal(err)
-		}
-
-		secretList, err := cl.SecretList(context.Background(), types.SecretListOptions{Filters: filters})
+		filters := filters.NewArgs()
+		filters.Add("name", app.StackName())
+		secretList, err := cl.SecretList(c.Context, types.SecretListOptions{Filters: filters})
 		if err != nil {
 			logrus.Fatal(err)
 		}
@@ -323,23 +249,18 @@ var appSecretLsCommand = cli.Command{
 			table.Append(tableRow)
 		}
 
-		if table.NumLines() > 0 {
-			table.Render()
-		} else {
-			logrus.Warnf("no secrets stored for %s", app.Name)
-		}
-
+		table.Render()
 		return nil
 	},
 	BashComplete: autocomplete.AppNameComplete,
 }
 
-var appSecretCommand = cli.Command{
+var appSecretCommand = &cli.Command{
 	Name:      "secret",
 	Aliases:   []string{"s"},
 	Usage:     "Manage app secrets",
-	ArgsUsage: "<domain>",
-	Subcommands: []cli.Command{
+	ArgsUsage: "<command>",
+	Subcommands: []*cli.Command{
 		appSecretGenerateCommand,
 		appSecretInsertCommand,
 		appSecretRmCommand,

cli/app/undeploy.go

@@ -1,26 +1,18 @@
 package app
 
 import (
-	"context"
-
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/autocomplete"
 	"coopcloud.tech/abra/pkg/client"
 	stack "coopcloud.tech/abra/pkg/upstream/stack"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
-var appUndeployCommand = cli.Command{
-	Name:      "undeploy",
-	Aliases:   []string{"un"},
-	ArgsUsage: "<domain>",
-	Flags: []cli.Flag{
-		internal.DebugFlag,
-		internal.NoInputFlag,
-	},
-	Before: internal.SubCommandBefore,
-	Usage:  "Undeploy an app",
+var appUndeployCommand = &cli.Command{
+	Name:    "undeploy",
+	Aliases: []string{"u"},
+	Usage:   "Undeploy an app",
 	Description: `
 This does not destroy any of the application data. However, you should remain
 vigilant, as your swarm installation will consider any previously attached
@@ -35,15 +27,15 @@ volumes as eligiblef or pruning once undeployed.
 			logrus.Fatal(err)
 		}
 
-		logrus.Debugf("checking whether %s is already deployed", stackName)
+		logrus.Debugf("checking whether '%s' is already deployed", stackName)
 
-		isDeployed, deployedVersion, err := stack.IsDeployed(context.Background(), cl, stackName)
+		isDeployed, deployedVersion, err := stack.IsDeployed(c.Context, cl, stackName)
 		if err != nil {
 			logrus.Fatal(err)
 		}
 
 		if !isDeployed {
-			logrus.Fatalf("%s is not deployed?", app.Name)
+			logrus.Fatalf("'%s' is not deployed?", stackName)
 		}
 
 		if err := internal.DeployOverview(app, deployedVersion, "continue with undeploy?"); err != nil {
@@ -51,7 +43,7 @@ volumes as eligiblef or pruning once undeployed.
 		}
 
 		rmOpts := stack.Remove{Namespaces: []string{app.StackName()}}
-		if err := stack.RunRemove(context.Background(), cl, rmOpts); err != nil {
+		if err := stack.RunRemove(c.Context, cl, rmOpts); err != nil {
 			logrus.Fatal(err)
 		}
 

cli/app/upgrade.go

@@ -1,49 +1,44 @@
 package app
 
 import (
-	"context"
 	"fmt"
 
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/autocomplete"
+	"coopcloud.tech/abra/pkg/catalogue"
 	"coopcloud.tech/abra/pkg/client"
 	"coopcloud.tech/abra/pkg/config"
-	"coopcloud.tech/abra/pkg/lint"
 	"coopcloud.tech/abra/pkg/recipe"
 	stack "coopcloud.tech/abra/pkg/upstream/stack"
 	"coopcloud.tech/tagcmp"
 	"github.com/AlecAivazis/survey/v2"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
-var appUpgradeCommand = cli.Command{
+var appUpgradeCommand = &cli.Command{
 	Name:      "upgrade",
-	Aliases:   []string{"up"},
+	Aliases:   []string{"u"},
 	Usage:     "Upgrade an app",
-	ArgsUsage: "<domain>",
+	ArgsUsage: "<app>",
 	Flags: []cli.Flag{
-		internal.DebugFlag,
-		internal.NoInputFlag,
 		internal.ForceFlag,
 		internal.ChaosFlag,
 		internal.NoDomainChecksFlag,
-		internal.DontWaitConvergeFlag,
 	},
-	Before: internal.SubCommandBefore,
 	Description: `
 This command supports upgrading an app. You can use it to choose and roll out a
 new upgrade to an existing app.
 
-This command specifically supports incrementing the version of running apps, as
-opposed to "abra app deploy <domain>" which will not change the version of a
+This command specifically supports changing the version of running apps, as
+opposed to "abra app deploy <app>" which will not change the version of a
 deployed app.
 
 You may pass "--force/-f" to upgrade to the same version again. This can be
 useful if the container runtime has gotten into a weird state.
 
 This action could be destructive, please ensure you have a copy of your app
-data beforehand.
+data beforehand - see "abra app backup <app>" for more.
 
 Chas mode ("--chaos") will deploy your local checkout of a recipe as-is,
 including unstaged changes and can be useful for live hacking and testing new
@@ -53,21 +48,6 @@ recipes.
 		app := internal.ValidateApp(c)
 		stackName := app.StackName()
 
-		if !internal.Chaos {
-			if err := recipe.EnsureUpToDate(app.Recipe); err != nil {
-				logrus.Fatal(err)
-			}
-		}
-
-		r, err := recipe.Get(app.Recipe)
-		if err != nil {
-			logrus.Fatal(err)
-		}
-
-		if err := lint.LintForErrors(r); err != nil {
-			logrus.Fatal(err)
-		}
-
 		cl, err := client.New(app.Server)
 		if err != nil {
 			logrus.Fatal(err)
@@ -75,7 +55,7 @@ recipes.
 
 		logrus.Debugf("checking whether %s is already deployed", stackName)
 
-		isDeployed, deployedVersion, err := stack.IsDeployed(context.Background(), cl, stackName)
+		isDeployed, deployedVersion, err := stack.IsDeployed(c.Context, cl, stackName)
 		if err != nil {
 			logrus.Fatal(err)
 		}
@@ -84,22 +64,23 @@ recipes.
 			logrus.Fatalf("%s is not deployed?", app.Name)
 		}
 
-		catl, err := recipe.ReadRecipeCatalogue()
+		catl, err := catalogue.ReadRecipeCatalogue()
 		if err != nil {
 			logrus.Fatal(err)
 		}
 
-		versions, err := recipe.GetRecipeCatalogueVersions(app.Recipe, catl)
+		versions, err := catalogue.GetRecipeCatalogueVersions(app.Type, catl)
 		if err != nil {
 			logrus.Fatal(err)
 		}
 
 		if len(versions) == 0 && !internal.Chaos {
-			logrus.Fatalf("no published releases for %s in the recipe catalogue?", app.Recipe)
+			logrus.Fatalf("no published releases for %s in the recipe catalogue?", app.Type)
 		}
 
 		var availableUpgrades []string
-		if deployedVersion == "unknown" {
+		if deployedVersion == "" {
+			deployedVersion = "unknown"
 			availableUpgrades = versions
 			logrus.Warnf("failed to determine version of deployed %s", app.Name)
 		}
@@ -120,16 +101,14 @@ recipes.
 			}
 
 			if len(availableUpgrades) == 0 && !internal.Force {
-				logrus.Infof("no available upgrades, you're on latest (%s) ✌️", deployedVersion)
+				logrus.Info("no available upgrades, you're on latest ✌️")
 				return nil
 			}
 		}
 
-		availableUpgrades = internal.ReverseStringList(availableUpgrades)
-
 		var chosenUpgrade string
 		if len(availableUpgrades) > 0 && !internal.Chaos {
-			if internal.Force || internal.NoInput {
+			if internal.Force {
 				chosenUpgrade = availableUpgrades[len(availableUpgrades)-1]
 				logrus.Debugf("choosing %s as version to upgrade to", chosenUpgrade)
 			} else {
@@ -143,16 +122,8 @@ recipes.
 			}
 		}
 
-		// if release notes written after git tag published, read them before we
-		// check out the tag and then they'll appear to be missing. this covers
-		// when we obviously will forget to write release notes before publishing
-		releaseNotes, err := internal.GetReleaseNotes(app.Recipe, chosenUpgrade)
-		if err != nil {
-			return err
-		}
-
 		if !internal.Chaos {
-			if err := recipe.EnsureVersion(app.Recipe, chosenUpgrade); err != nil {
+			if err := recipe.EnsureVersion(app.Type, chosenUpgrade); err != nil {
 				logrus.Fatal(err)
 			}
 		}
@@ -160,13 +131,13 @@ recipes.
 		if internal.Chaos {
 			logrus.Warn("chaos mode engaged")
 			var err error
-			chosenUpgrade, err = recipe.ChaosVersion(app.Recipe)
+			chosenUpgrade, err = recipe.ChaosVersion(app.Type)
 			if err != nil {
 				logrus.Fatal(err)
 			}
 		}
 
-		abraShPath := fmt.Sprintf("%s/%s/%s", config.RECIPES_DIR, app.Recipe, "abra.sh")
+		abraShPath := fmt.Sprintf("%s/%s/%s", config.APPS_DIR, app.Type, "abra.sh")
 		abraShEnv, err := config.ReadAbraShEnvVars(abraShPath)
 		if err != nil {
 			logrus.Fatal(err)
@@ -175,7 +146,7 @@ recipes.
 			app.Env[k] = v
 		}
 
-		composeFiles, err := config.GetAppComposeFiles(app.Recipe, app.Env)
+		composeFiles, err := config.GetAppComposeFiles(app.Type, app.Env)
 		if err != nil {
 			logrus.Fatal(err)
 		}
@@ -190,11 +161,11 @@ recipes.
 			logrus.Fatal(err)
 		}
 
-		if err := internal.NewVersionOverview(app, deployedVersion, chosenUpgrade, releaseNotes); err != nil {
+		if err := internal.NewVersionOverview(app, deployedVersion, chosenUpgrade); err != nil {
 			logrus.Fatal(err)
 		}
 
-		if err := stack.RunDeploy(cl, deployOpts, compose, app.StackName(), internal.DontWaitConverge); err != nil {
+		if err := stack.RunDeploy(cl, deployOpts, compose, app.Type); err != nil {
 			logrus.Fatal(err)
 		}
 

cli/app/version.go

@@ -1,17 +1,17 @@
 package app
 
 import (
-	"context"
+	"strings"
 
+	abraFormatter "coopcloud.tech/abra/cli/formatter"
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/autocomplete"
+	"coopcloud.tech/abra/pkg/catalogue"
 	"coopcloud.tech/abra/pkg/client"
-	"coopcloud.tech/abra/pkg/formatter"
-	"coopcloud.tech/abra/pkg/recipe"
 	"coopcloud.tech/abra/pkg/upstream/stack"
 	"github.com/docker/distribution/reference"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
 // getImagePath returns the image name
@@ -22,24 +22,19 @@ func getImagePath(image string) (string, error) {
 	}
 
 	path := reference.Path(img)
-
-	path = formatter.StripTagMeta(path)
+	if strings.Contains(path, "library") {
+		path = strings.Split(path, "/")[1]
+	}
 
 	logrus.Debugf("parsed %s from %s", path, image)
 
 	return path, nil
 }
 
-var appVersionCommand = cli.Command{
-	Name:      "version",
-	Aliases:   []string{"v"},
-	ArgsUsage: "<domain>",
-	Flags: []cli.Flag{
-		internal.DebugFlag,
-		internal.NoInputFlag,
-	},
-	Before: internal.SubCommandBefore,
-	Usage:  "Show app versions",
+var appVersionCommand = &cli.Command{
+	Name:    "version",
+	Aliases: []string{"v"},
+	Usage:   "Show app versions",
 	Description: `
 This command shows all information about versioning related to a deployed app.
 This includes the individual image names, tags and digests. But also the Co-op
@@ -56,12 +51,12 @@ Cloud recipe version.
 
 		logrus.Debugf("checking whether %s is already deployed", stackName)
 
-		isDeployed, deployedVersion, err := stack.IsDeployed(context.Background(), cl, stackName)
+		isDeployed, deployedVersion, err := stack.IsDeployed(c.Context, cl, stackName)
 		if err != nil {
 			logrus.Fatal(err)
 		}
 
-		if deployedVersion == "unknown" {
+		if deployedVersion == "" {
 			logrus.Fatalf("failed to determine version of deployed %s", app.Name)
 		}
 
@@ -69,12 +64,12 @@ Cloud recipe version.
 			logrus.Fatalf("%s is not deployed?", app.Name)
 		}
 
-		recipeMeta, err := recipe.GetRecipeMeta(app.Recipe)
+		recipeMeta, err := catalogue.GetRecipeMeta(app.Type)
 		if err != nil {
 			logrus.Fatal(err)
 		}
 
-		versionsMeta := make(map[string]recipe.ServiceMeta)
+		versionsMeta := make(map[string]catalogue.ServiceMeta)
 		for _, recipeVersion := range recipeMeta.Versions {
 			if currentVersion, exists := recipeVersion[deployedVersion]; exists {
 				versionsMeta = currentVersion
@@ -86,7 +81,7 @@ Cloud recipe version.
 		}
 
 		tableCol := []string{"version", "service", "image", "digest"}
-		table := formatter.CreateTable(tableCol)
+		table := abraFormatter.CreateTable(tableCol)
 		table.SetAutoMergeCellsByColumnIndex([]int{0})
 
 		for serviceName, versionMeta := range versionsMeta {

cli/app/volume.go

@@ -1,98 +1,74 @@
 package app
 
 import (
-	"context"
-
+	abraFormatter "coopcloud.tech/abra/cli/formatter"
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/autocomplete"
 	"coopcloud.tech/abra/pkg/client"
-	"coopcloud.tech/abra/pkg/formatter"
 	"github.com/AlecAivazis/survey/v2"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
-var appVolumeListCommand = cli.Command{
-	Name:      "list",
-	Aliases:   []string{"ls"},
-	ArgsUsage: "<domain>",
-	Flags: []cli.Flag{
-		internal.DebugFlag,
-		internal.NoInputFlag,
-	},
-	Before:       internal.SubCommandBefore,
+var appVolumeListCommand = &cli.Command{
+	Name:         "list",
 	Usage:        "List volumes associated with an app",
+	Aliases:      []string{"ls"},
 	BashComplete: autocomplete.AppNameComplete,
 	Action: func(c *cli.Context) error {
 		app := internal.ValidateApp(c)
 
-		filters, err := app.Filters()
+		volumeList, err := client.GetVolumes(c.Context, app.Server, app.Name)
 		if err != nil {
 			logrus.Fatal(err)
 		}
 
-		volumeList, err := client.GetVolumes(context.Background(), app.Server, filters)
-		if err != nil {
-			logrus.Fatal(err)
-		}
-
-		table := formatter.CreateTable([]string{"name", "created", "mounted"})
+		table := abraFormatter.CreateTable([]string{"driver", "volume name"})
 		var volTable [][]string
 		for _, volume := range volumeList {
-			volRow := []string{volume.Name, volume.CreatedAt, volume.Mountpoint}
+			volRow := []string{
+				volume.Driver,
+				volume.Name,
+			}
 			volTable = append(volTable, volRow)
 		}
 
 		table.AppendBulk(volTable)
-
-		if table.NumLines() > 0 {
-			table.Render()
-		} else {
-			logrus.Warnf("no volumes created for %s", app.Name)
-		}
+		table.Render()
 
 		return nil
 	},
 }
 
-var appVolumeRemoveCommand = cli.Command{
+var appVolumeRemoveCommand = &cli.Command{
 	Name:  "remove",
 	Usage: "Remove volume(s) associated with an app",
 	Description: `
 This command supports removing volumes associated with an app. The app in
 question must be undeployed before you try to remove volumes. See "abra app
-undeploy <domain>" for more.
+undeploy <app>" for more.
 
 The command is interactive and will show a multiple select input which allows
 you to make a seclection. Use the "?" key to see more help on navigating this
 interface.
 
-Passing "--force/-f" will select all volumes for removal. Be careful.
+Passing "--force" will select all volumes for removal. Be careful.
 `,
-	ArgsUsage: "<domain>",
-	Aliases:   []string{"rm"},
+	Aliases: []string{"rm"},
 	Flags: []cli.Flag{
-		internal.DebugFlag,
-		internal.NoInputFlag,
 		internal.ForceFlag,
 	},
-	Before: internal.SubCommandBefore,
 	Action: func(c *cli.Context) error {
 		app := internal.ValidateApp(c)
 
-		filters, err := app.Filters()
-		if err != nil {
-			logrus.Fatal(err)
-		}
-
-		volumeList, err := client.GetVolumes(context.Background(), app.Server, filters)
+		volumeList, err := client.GetVolumes(c.Context, app.Server, app.Name)
 		if err != nil {
 			logrus.Fatal(err)
 		}
 		volumeNames := client.GetVolumeNames(volumeList)
 
 		var volumesToRemove []string
-		if !internal.Force && !internal.NoInput {
+		if !internal.Force {
 			volumesPrompt := &survey.MultiSelect{
 				Message: "which volumes do you want to remove?",
 				Help:    "'x' indicates selected, enter / return to confirm, ctrl-c to exit, vim mode is enabled",
@@ -103,13 +79,11 @@ Passing "--force/-f" will select all volumes for removal. Be careful.
 			if err := survey.AskOne(volumesPrompt, &volumesToRemove); err != nil {
 				logrus.Fatal(err)
 			}
-		}
-
-		if internal.Force || internal.NoInput {
+		} else {
 			volumesToRemove = volumeNames
 		}
 
-		err = client.RemoveVolumes(context.Background(), app.Server, volumesToRemove, internal.Force)
+		err = client.RemoveVolumes(c.Context, app.Server, volumesToRemove, internal.Force)
 		if err != nil {
 			logrus.Fatal(err)
 		}
@@ -121,12 +95,12 @@ Passing "--force/-f" will select all volumes for removal. Be careful.
 	BashComplete: autocomplete.AppNameComplete,
 }
 
-var appVolumeCommand = cli.Command{
+var appVolumeCommand = &cli.Command{
 	Name:      "volume",
 	Aliases:   []string{"vl"},
 	Usage:     "Manage app volumes",
-	ArgsUsage: "<domain>",
-	Subcommands: []cli.Command{
+	ArgsUsage: "<command>",
+	Subcommands: []*cli.Command{
 		appVolumeListCommand,
 		appVolumeRemoveCommand,
 	},

cli/autocomplete.go

@@ -0,0 +1,97 @@
+package cli
+
+import (
+	"errors"
+	"fmt"
+	"os"
+	"path"
+
+	"coopcloud.tech/abra/cli/internal"
+	"coopcloud.tech/abra/pkg/config"
+	"coopcloud.tech/abra/pkg/web"
+	"github.com/sirupsen/logrus"
+	"github.com/urfave/cli/v2"
+)
+
+// AutoCompleteCommand helps people set up auto-complete in their shells
+var AutoCompleteCommand = &cli.Command{
+	Name:    "autocomplete",
+	Usage:   "Help set up shell autocompletion",
+	Aliases: []string{"ac"},
+	Description: `
+This command helps set up autocompletion in your shell by downloading the
+relevant autocompletion files and laying out what additional information must
+be loaded.
+
+Example:
+
+    abra autocomplete bash
+
+Supported shells are as follows:
+
+		fizsh
+		zsh
+    bash
+
+`,
+	ArgsUsage: "<shell>",
+	Action: func(c *cli.Context) error {
+		shellType := c.Args().First()
+
+		if shellType == "" {
+			internal.ShowSubcommandHelpAndError(c, errors.New("no shell provided"))
+		}
+
+		supportedShells := map[string]bool{
+			"bash":  true,
+			"zsh":   true,
+			"fizsh": true,
+		}
+
+		if _, ok := supportedShells[shellType]; !ok {
+			logrus.Fatalf("%s is not a supported shell right now, sorry", shellType)
+		}
+
+		if shellType == "fizsh" {
+			shellType = "zsh" // handled the same on the autocompletion side
+		}
+
+		autocompletionDir := path.Join(config.ABRA_DIR, "autocompletion")
+		if err := os.Mkdir(autocompletionDir, 0764); err != nil {
+			if !os.IsExist(err) {
+				logrus.Fatal(err)
+			}
+			logrus.Debugf("%s already created", autocompletionDir)
+		}
+
+		autocompletionFile := path.Join(config.ABRA_DIR, "autocompletion", shellType)
+		if _, err := os.Stat(autocompletionFile); err != nil && os.IsNotExist(err) {
+			url := fmt.Sprintf("https://git.coopcloud.tech/coop-cloud/abra/raw/branch/main/scripts/autocomplete/%s", shellType)
+			logrus.Infof("fetching %s", url)
+			if err := web.GetFile(autocompletionFile, url); err != nil {
+				logrus.Fatal(err)
+			}
+		}
+
+		switch shellType {
+		case "bash":
+			fmt.Println(fmt.Sprintf(`
+# Run the following commands to install autocompletion
+sudo mkdir /etc/bash_completion.d/
+sudo cp %s /etc/bash_completion.d/abra
+echo "source /etc/bash_completion.d/abra" >> ~/.bashrc
+# And finally run "abra app ps <hit tab key>" to test things are working, you should see app names listed!
+`, autocompletionFile))
+		case "zsh":
+			fmt.Println(fmt.Sprintf(`
+# Run the following commands to install autocompletion
+sudo mkdir /etc/zsh/completion.d/
+sudo cp %s /etc/zsh/completion.d/abra
+echo "PROG=abra\n_CLI_ZSH_AUTOCOMPLETE_HACK=1\nsource /etc/zsh/completion.d/abra" >> ~/.zshrc
+# And finally run "abra app ps <hit tab key>" to test things are working, you should see app names listed!
+`, autocompletionFile))
+		}
+
+		return nil
+	},
+}

cli/catalogue/catalogue.go

@@ -4,122 +4,161 @@ import (
 	"encoding/json"
 	"fmt"
 	"io/ioutil"
+	"os"
 	"path"
 
+	"coopcloud.tech/abra/cli/formatter"
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/autocomplete"
+	"coopcloud.tech/abra/pkg/catalogue"
 	"coopcloud.tech/abra/pkg/config"
-	"coopcloud.tech/abra/pkg/formatter"
 	gitPkg "coopcloud.tech/abra/pkg/git"
 	"coopcloud.tech/abra/pkg/limit"
-	"coopcloud.tech/abra/pkg/recipe"
-	"github.com/go-git/go-git/v5"
+	"github.com/AlecAivazis/survey/v2"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
 // CatalogueSkipList is all the repos that are not recipes.
 var CatalogueSkipList = map[string]bool{
-	"abra":                   true,
-	"abra-apps":              true,
-	"abra-aur":               true,
-	"abra-bash":              true,
-	"abra-capsul":            true,
-	"abra-gandi":             true,
-	"abra-hetzner":           true,
-	"apps":                   true,
-	"aur-abra-git":           true,
-	"auto-apps-json":         true,
-	"auto-mirror":            true,
-	"backup-bot":             true,
-	"backup-bot-two":         true,
-	"beta.coopcloud.tech":    true,
-	"comrade-renovate-bot":   true,
-	"coopcloud.tech":         true,
-	"coturn":                 true,
-	"docker-cp-deploy":       true,
-	"docker-dind-bats-kcov":  true,
-	"docs.coopcloud.tech":    true,
-	"drone-abra":             true,
-	"example":                true,
-	"gardening":              true,
-	"go-abra":                true,
-	"organising":             true,
-	"outline-with-patch":     true,
-	"pyabra":                 true,
-	"radicle-seed-node":      true,
-	"recipes-catalogue-json": true,
-	"recipes-wishlist":       true,
-	"recipes.coopcloud.tech": true,
-	"stack-ssh-deploy":       true,
-	"swarm-cronjob":          true,
-	"tagcmp":                 true,
-	"traefik-cert-dumper":    true,
-	"tyop":                   true,
+	"abra":                  true,
+	"abra-apps":             true,
+	"abra-aur":              true,
+	"abra-bash":             true,
+	"abra-capsul":           true,
+	"abra-gandi":            true,
+	"abra-hetzner":          true,
+	"apps":                  true,
+	"aur-abra-git":          true,
+	"auto-apps-json":        true,
+	"auto-mirror":           true,
+	"backup-bot":            true,
+	"backup-bot-two":        true,
+	"comrade-renovate-bot":  true,
+	"coopcloud.tech":        true,
+	"coturn":                true,
+	"docker-cp-deploy":      true,
+	"docker-dind-bats-kcov": true,
+	"docs.coopcloud.tech":   true,
+	"drone-abra":            true,
+	"example":               true,
+	"gardening":             true,
+	"go-abra":               true,
+	"organising":            true,
+	"outline-with-patch":    true,
+	"pyabra":                true,
+	"radicle-seed-node":     true,
+	"recipes":               true,
+	"stack-ssh-deploy":      true,
+	"swarm-cronjob":         true,
+	"tagcmp":                true,
+	"traefik-cert-dumper":   true,
+	"tyop":                  true,
 }
 
-var catalogueGenerateCommand = cli.Command{
+var catalogueGenerateCommand = &cli.Command{
 	Name:    "generate",
 	Aliases: []string{"g"},
-	Usage:   "Generate the recipe catalogue",
+	Usage:   "Generate a new copy of the catalogue",
 	Flags: []cli.Flag{
-		internal.DebugFlag,
-		internal.NoInputFlag,
-		internal.PublishFlag,
+		internal.PushFlag,
+		internal.CommitFlag,
+		internal.CommitMessageFlag,
 		internal.DryFlag,
-		internal.SkipUpdatesFlag,
 	},
-	Before: internal.SubCommandBefore,
 	Description: `
 This command generates a new copy of the recipe catalogue which can be found on:
 
     https://recipes.coopcloud.tech
 
 It polls the entire git.coopcloud.tech/coop-cloud/... recipe repository
-listing, parses README.md and git tags of those repositories to produce recipe
-metadata and produces a recipes JSON file.
+listing, parses README and tags to produce recipe metadata and produces a
+apps.json file which is placed in your ~/.abra/catalogue/recipes.json.
 
 It is possible to generate new metadata for a single recipe by passing
 <recipe>. The existing local catalogue will be updated, not overwritten.
 
-It is quite easy to get rate limited by Docker Hub when running this command.
-If you have a Hub account you can have Abra log you in to avoid this. Pass
-"--user" and "--pass".
+A new catalogue copy can be published to the recipes repository by passing the
+"--commit" and "--push" flags. The recipes repository is available here:
+
+    https://git.coopcloud.tech/coop-cloud/recipes
 
-Push your new release git.coopcloud.tech with "-p/--publish". This requires
-that you have permission to git push to these repositories and have your SSH
-keys configured on your account.
 `,
 	ArgsUsage: "[<recipe>]",
 	Action: func(c *cli.Context) error {
 		recipeName := c.Args().First()
 		if recipeName != "" {
-			internal.ValidateRecipe(c, true)
+			internal.ValidateRecipe(c)
 		}
 
-		repos, err := recipe.ReadReposMetadata()
+		catalogueDir := path.Join(config.ABRA_DIR, "catalogue")
+		url := fmt.Sprintf("%s/%s.git", config.REPOS_BASE_URL, "recipes")
+		if err := gitPkg.Clone(catalogueDir, url); err != nil {
+			return err
+		}
+
+		repos, err := catalogue.ReadReposMetadata()
 		if err != nil {
 			logrus.Fatal(err)
 		}
 
+		logrus.Debugf("ensuring '%v' recipe(s) are locally present and up-to-date", len(repos))
+
 		var barLength int
-		var logMsg string
 		if recipeName != "" {
 			barLength = 1
-			logMsg = fmt.Sprintf("ensuring %v recipe is cloned & up-to-date", barLength)
 		} else {
 			barLength = len(repos)
-			logMsg = fmt.Sprintf("ensuring %v recipes are cloned & up-to-date, this could take some time...", barLength)
 		}
 
-		if !internal.SkipUpdates {
-			logrus.Warn(logMsg)
-			if err := updateRepositories(repos, recipeName); err != nil {
-				logrus.Fatal(err)
-			}
+		cloneLimiter := limit.New(10)
+		retrieveBar := formatter.CreateProgressbar(len(repos), "retrieving recipes from recipes.coopcloud.tech...")
+		ch := make(chan string, barLength)
+		for _, repoMeta := range repos {
+			go func(rm catalogue.RepoMeta) {
+				cloneLimiter.Begin()
+				defer cloneLimiter.End()
+
+				if recipeName != "" && recipeName != rm.Name {
+					ch <- rm.Name
+					retrieveBar.Add(1)
+					return
+				}
+				if _, exists := CatalogueSkipList[rm.Name]; exists {
+					ch <- rm.Name
+					retrieveBar.Add(1)
+					return
+				}
+
+				recipeDir := path.Join(config.ABRA_DIR, "apps", rm.Name)
+
+				if err := gitPkg.Clone(recipeDir, rm.SSHURL); err != nil {
+					logrus.Fatal(err)
+				}
+
+				isClean, err := gitPkg.IsClean(rm.Name)
+				if err != nil {
+					logrus.Fatal(err)
+				}
+
+				if !isClean {
+					logrus.Fatalf("'%s' has locally unstaged changes", rm.Name)
+				}
+
+				if err := gitPkg.EnsureUpToDate(recipeDir); err != nil {
+					logrus.Fatal(err)
+				}
+
+				ch <- rm.Name
+				retrieveBar.Add(1)
+			}(repoMeta)
 		}
 
-		catl := make(recipe.RecipeCatalogue)
+		for range repos {
+			<-ch // wait for everything
+		}
+
+		catl := make(catalogue.RecipeCatalogue)
 		catlBar := formatter.CreateProgressbar(barLength, "generating catalogue metadata...")
 		for _, recipeMeta := range repos {
 			if recipeName != "" && recipeName != recipeMeta.Name {
@@ -132,20 +171,19 @@ keys configured on your account.
 				continue
 			}
 
-			versions, err := recipe.GetRecipeVersions(recipeMeta.Name)
+			versions, err := catalogue.GetRecipeVersions(recipeMeta.Name)
+			if err != nil {
+				logrus.Fatal(err)
+			}
+
+			features, category, err := catalogue.GetRecipeFeaturesAndCategory(recipeMeta.Name)
 			if err != nil {
 				logrus.Warn(err)
 			}
 
-			features, category, err := recipe.GetRecipeFeaturesAndCategory(recipeMeta.Name)
-			if err != nil {
-				logrus.Warn(err)
-			}
-
-			catl[recipeMeta.Name] = recipe.RecipeMeta{
+			catl[recipeMeta.Name] = catalogue.RecipeMeta{
 				Name:          recipeMeta.Name,
 				Repository:    recipeMeta.CloneURL,
-				SSHURL:        recipeMeta.SSHURL,
 				Icon:          recipeMeta.AvatarURL,
 				DefaultBranch: recipeMeta.DefaultBranch,
 				Description:   recipeMeta.Description,
@@ -154,7 +192,6 @@ keys configured on your account.
 				Category:      category,
 				Features:      features,
 			}
-
 			catlBar.Add(1)
 		}
 
@@ -163,81 +200,45 @@ keys configured on your account.
 			logrus.Fatal(err)
 		}
 
-		if recipeName == "" {
-			if err := ioutil.WriteFile(config.RECIPES_JSON, recipesJSON, 0764); err != nil {
+		if _, err := os.Stat(config.APPS_JSON); err != nil && os.IsNotExist(err) {
+			if err := ioutil.WriteFile(config.APPS_JSON, recipesJSON, 0764); err != nil {
 				logrus.Fatal(err)
 			}
 		} else {
-			catlFS, err := recipe.ReadRecipeCatalogue()
-			if err != nil {
-				logrus.Fatal(err)
-			}
+			if recipeName != "" {
+				catlFS, err := catalogue.ReadRecipeCatalogue()
+				if err != nil {
+					logrus.Fatal(err)
+				}
+				catlFS[recipeName] = catl[recipeName]
 
-			catlFS[recipeName] = catl[recipeName]
-
-			updatedRecipesJSON, err := json.MarshalIndent(catlFS, "", "    ")
-			if err != nil {
-				logrus.Fatal(err)
-			}
-
-			if err := ioutil.WriteFile(config.RECIPES_JSON, updatedRecipesJSON, 0764); err != nil {
-				logrus.Fatal(err)
+				updatedRecipesJSON, err := json.MarshalIndent(catlFS, "", "    ")
+				if err != nil {
+					logrus.Fatal(err)
+				}
+				if err := ioutil.WriteFile(config.APPS_JSON, updatedRecipesJSON, 0764); err != nil {
+					logrus.Fatal(err)
+				}
 			}
 		}
 
-		logrus.Infof("generated new recipe catalogue in %s", config.RECIPES_JSON)
+		logrus.Infof("generated new recipe catalogue in %s", config.APPS_JSON)
 
-		cataloguePath := path.Join(config.ABRA_DIR, "catalogue")
-		if internal.Publish {
-
-			isClean, err := gitPkg.IsClean(cataloguePath)
-			if err != nil {
-				logrus.Fatal(err)
-			}
-
-			if isClean {
-				if !internal.Dry {
-					logrus.Fatalf("no changes discovered in %s, nothing to publish?", cataloguePath)
+		if internal.Commit {
+			if internal.CommitMessage == "" && !internal.NoInput {
+				prompt := &survey.Input{
+					Message: "commit message",
+					Default: fmt.Sprintf("chore: publish catalogue changes"),
+				}
+				if err := survey.AskOne(prompt, &internal.CommitMessage); err != nil {
+					logrus.Fatal(err)
 				}
 			}
 
-			msg := "chore: publish new catalogue release changes"
-			if err := gitPkg.Commit(cataloguePath, "**.json", msg, internal.Dry); err != nil {
+			cataloguePath := path.Join(config.ABRA_DIR, "catalogue")
+			if err := gitPkg.Commit(cataloguePath, "**.json", internal.CommitMessage, internal.Dry, internal.Push); err != nil {
 				logrus.Fatal(err)
 			}
-
-			repo, err := git.PlainOpen(cataloguePath)
-			if err != nil {
-				logrus.Fatal(err)
-			}
-
-			sshURL := fmt.Sprintf(config.SSH_URL_TEMPLATE, config.CATALOGUE_JSON_REPO_NAME)
-			if err := gitPkg.CreateRemote(repo, "origin-ssh", sshURL, internal.Dry); err != nil {
-				logrus.Fatal(err)
-			}
-
-			if err := gitPkg.Push(cataloguePath, "origin-ssh", false, internal.Dry); err != nil {
-				logrus.Fatal(err)
-			}
-		}
-
-		repo, err := git.PlainOpen(cataloguePath)
-		if err != nil {
-			logrus.Fatal(err)
-		}
-
-		head, err := repo.Head()
-		if err != nil {
-			logrus.Fatal(err)
-		}
-
-		if !internal.Dry && internal.Publish {
-			url := fmt.Sprintf("%s/%s/commit/%s", config.REPOS_BASE_URL, config.CATALOGUE_JSON_REPO_NAME, head.Hash())
-			logrus.Infof("new changes published: %s", url)
-		}
-
-		if internal.Dry {
-			logrus.Info("dry run: no changes published")
 		}
 
 		return nil
@@ -246,72 +247,13 @@ keys configured on your account.
 }
 
 // CatalogueCommand defines the `abra catalogue` command and sub-commands.
-var CatalogueCommand = cli.Command{
+var CatalogueCommand = &cli.Command{
 	Name:        "catalogue",
 	Usage:       "Manage the recipe catalogue",
 	Aliases:     []string{"c"},
 	ArgsUsage:   "<recipe>",
 	Description: "This command helps recipe packagers interact with the recipe catalogue",
-	Subcommands: []cli.Command{
+	Subcommands: []*cli.Command{
 		catalogueGenerateCommand,
 	},
 }
-
-func updateRepositories(repos recipe.RepoCatalogue, recipeName string) error {
-	var barLength int
-	if recipeName != "" {
-		barLength = 1
-	} else {
-		barLength = len(repos)
-	}
-
-	cloneLimiter := limit.New(10)
-
-	retrieveBar := formatter.CreateProgressbar(barLength, "ensuring recipes are cloned & up-to-date...")
-	ch := make(chan string, barLength)
-	for _, repoMeta := range repos {
-		go func(rm recipe.RepoMeta) {
-			cloneLimiter.Begin()
-			defer cloneLimiter.End()
-
-			if recipeName != "" && recipeName != rm.Name {
-				ch <- rm.Name
-				retrieveBar.Add(1)
-				return
-			}
-			if _, exists := CatalogueSkipList[rm.Name]; exists {
-				ch <- rm.Name
-				retrieveBar.Add(1)
-				return
-			}
-
-			recipeDir := path.Join(config.RECIPES_DIR, rm.Name)
-
-			if err := gitPkg.Clone(recipeDir, rm.CloneURL); err != nil {
-				logrus.Fatal(err)
-			}
-
-			isClean, err := gitPkg.IsClean(recipeDir)
-			if err != nil {
-				logrus.Fatal(err)
-			}
-
-			if !isClean {
-				logrus.Fatalf("%s has locally unstaged changes", rm.Name)
-			}
-
-			if err := recipe.EnsureUpToDate(rm.Name); err != nil {
-				logrus.Fatal(err)
-			}
-
-			ch <- rm.Name
-			retrieveBar.Add(1)
-		}(repoMeta)
-	}
-
-	for range repos {
-		<-ch // wait for everything
-	}
-
-	return nil
-}

cli/cli.go

@@ -2,10 +2,8 @@
 package cli
 
 import (
-	"errors"
 	"fmt"
 	"os"
-	"os/exec"
 	"path"
 
 	"coopcloud.tech/abra/cli/app"
@@ -14,132 +12,12 @@ import (
 	"coopcloud.tech/abra/cli/recipe"
 	"coopcloud.tech/abra/cli/record"
 	"coopcloud.tech/abra/cli/server"
-	"coopcloud.tech/abra/pkg/autocomplete"
 	"coopcloud.tech/abra/pkg/config"
-	"coopcloud.tech/abra/pkg/web"
+	logrusStack "github.com/Gurpartap/logrus-stack"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
-// AutoCompleteCommand helps people set up auto-complete in their shells
-var AutoCompleteCommand = cli.Command{
-	Name:    "autocomplete",
-	Aliases: []string{"ac"},
-	Usage:   "Configure shell autocompletion (recommended)",
-	Description: `
-This command helps set up autocompletion in your shell by downloading the
-relevant autocompletion files and laying out what additional information must
-be loaded.
-
-Example:
-
-    abra autocomplete bash
-
-Supported shells are as follows:
-
-		fizsh
-		zsh
-    bash
-`,
-	ArgsUsage: "<shell>",
-	Flags: []cli.Flag{
-		internal.DebugFlag,
-	},
-	Action: func(c *cli.Context) error {
-		shellType := c.Args().First()
-
-		if shellType == "" {
-			internal.ShowSubcommandHelpAndError(c, errors.New("no shell provided"))
-		}
-
-		supportedShells := map[string]bool{
-			"bash":  true,
-			"zsh":   true,
-			"fizsh": true,
-		}
-
-		if _, ok := supportedShells[shellType]; !ok {
-			logrus.Fatalf("%s is not a supported shell right now, sorry", shellType)
-		}
-
-		if shellType == "fizsh" {
-			shellType = "zsh" // handled the same on the autocompletion side
-		}
-
-		autocompletionDir := path.Join(config.ABRA_DIR, "autocompletion")
-		if err := os.Mkdir(autocompletionDir, 0764); err != nil {
-			if !os.IsExist(err) {
-				logrus.Fatal(err)
-			}
-			logrus.Debugf("%s already created", autocompletionDir)
-		}
-
-		autocompletionFile := path.Join(config.ABRA_DIR, "autocompletion", shellType)
-		if _, err := os.Stat(autocompletionFile); err != nil && os.IsNotExist(err) {
-			url := fmt.Sprintf("https://git.coopcloud.tech/coop-cloud/abra/raw/branch/main/scripts/autocomplete/%s", shellType)
-			logrus.Infof("fetching %s", url)
-			if err := web.GetFile(autocompletionFile, url); err != nil {
-				logrus.Fatal(err)
-			}
-		}
-
-		switch shellType {
-		case "bash":
-			fmt.Println(fmt.Sprintf(`
-# Run the following commands to install autocompletion
-sudo mkdir /etc/bash_completion.d/
-sudo cp %s /etc/bash_completion.d/abra
-echo "source /etc/bash_completion.d/abra" >> ~/.bashrc
-# And finally run "abra app ps <hit tab key>" to test things are working, you should see app domains listed!
-`, autocompletionFile))
-		case "zsh":
-			fmt.Println(fmt.Sprintf(`
-# Run the following commands to install autocompletion
-sudo mkdir /etc/zsh/completion.d/
-sudo cp %s /etc/zsh/completion.d/abra
-echo "PROG=abra\n_CLI_ZSH_AUTOCOMPLETE_HACK=1\nsource /etc/zsh/completion.d/abra" >> ~/.zshrc
-# And finally run "abra app ps <hit tab key>" to test things are working, you should see app domains listed!
-`, autocompletionFile))
-		}
-
-		return nil
-	},
-}
-
-// UpgradeCommand upgrades abra in-place.
-var UpgradeCommand = cli.Command{
-	Name:    "upgrade",
-	Aliases: []string{"u"},
-	Usage:   "Upgrade Abra itself",
-	Description: `
-This command allows you to upgrade Abra in-place with the latest stable or
-release candidate.
-
-If you would like to install the latest release candidate, please pass the
-"-r/--rc" option. Please bear in mind that the latest release candidate may
-have some catastrophic bugs contained in it. In any case, thank you very much
-for the testing efforts!
-`,
-	Flags: []cli.Flag{internal.RCFlag},
-	Action: func(c *cli.Context) error {
-		mainURL := "https://install.abra.coopcloud.tech"
-		cmd := exec.Command("bash", "-c", fmt.Sprintf("wget -q -O- %s | bash", mainURL))
-
-		if internal.RC {
-			releaseCandidateURL := "https://git.coopcloud.tech/coop-cloud/abra/raw/branch/main/scripts/installer/installer"
-			cmd = exec.Command("bash", "-c", fmt.Sprintf("wget -q -O- %s | bash -s -- --rc", releaseCandidateURL))
-		}
-
-		logrus.Debugf("attempting to run %s", cmd)
-
-		if err := internal.RunCmd(cmd); err != nil {
-			logrus.Fatal(err)
-		}
-
-		return nil
-	},
-}
-
 func newAbraApp(version, commit string) *cli.App {
 	app := &cli.App{
 		Name: "abra",
@@ -150,9 +28,12 @@ func newAbraApp(version, commit string) *cli.App {
   | |__| (_) |_____| (_) | |_) | | |___| | (_) | |_| | (_| |
    \____\___/       \___/| .__/   \____|_|\___/ \__,_|\__,_|
                          |_|
-`,
+
+If you haven't already done so, consider setting up autocompletion for a more
+convenient command-line experience. See "abra autocomplete -h" for more.
+		`,
 		Version: fmt.Sprintf("%s-%s", version, commit[:7]),
-		Commands: []cli.Command{
+		Commands: []*cli.Command{
 			app.AppCommand,
 			server.ServerCommand,
 			recipe.RecipeCommand,
@@ -161,17 +42,37 @@ func newAbraApp(version, commit string) *cli.App {
 			UpgradeCommand,
 			AutoCompleteCommand,
 		},
-		BashComplete: autocomplete.SubcommandComplete,
+		Flags: []cli.Flag{
+			internal.VerboseFlag,
+			internal.DebugFlag,
+			internal.NoInputFlag,
+		},
+		Authors: []*cli.Author{
+			// If you're looking at this and you hack on Abra and you're not listed
+			// here, please do add yourself! This is a community project, let's show
+			// some love
+			{Name: "3wordchant"},
+			{Name: "decentral1se"},
+			{Name: "knoflook"},
+			{Name: "roxxers"},
+		},
 	}
 
 	app.EnableBashCompletion = true
 
 	app.Before = func(c *cli.Context) error {
+		if internal.Debug {
+			logrus.SetLevel(logrus.DebugLevel)
+			logrus.SetFormatter(&logrus.TextFormatter{})
+			logrus.SetOutput(os.Stderr)
+			logrus.AddHook(logrusStack.StandardHook())
+		}
+
 		paths := []string{
 			config.ABRA_DIR,
-			path.Join(config.SERVERS_DIR),
-			path.Join(config.RECIPES_DIR),
-			path.Join(config.VENDOR_DIR),
+			path.Join(config.ABRA_DIR, "servers"),
+			path.Join(config.ABRA_DIR, "apps"),
+			path.Join(config.ABRA_DIR, "vendor"),
 		}
 
 		for _, path := range paths {
@@ -183,7 +84,7 @@ func newAbraApp(version, commit string) *cli.App {
 			}
 		}
 
-		logrus.Debugf("abra version %s, commit %s", version, commit)
+		logrus.Debugf("abra version '%s', commit '%s'", version, commit)
 
 		return nil
 	}

cli/formatter/formatter.go

@@ -1,20 +1,25 @@
 package formatter
 
 import (
+	"fmt"
 	"os"
 	"strings"
 	"time"
 
+	"github.com/docker/cli/cli/command/formatter"
 	"github.com/docker/go-units"
 	"github.com/olekukonko/tablewriter"
 	"github.com/schollz/progressbar/v3"
-	"github.com/sirupsen/logrus"
 )
 
 func ShortenID(str string) string {
 	return str[:12]
 }
 
+func Truncate(str string) string {
+	return fmt.Sprintf(`"%s"`, formatter.Ellipsis(str, 19))
+}
+
 func SmallSHA(hash string) string {
 	return hash[:8]
 }
@@ -34,7 +39,6 @@ func HumanDuration(timestamp int64) string {
 // CreateTable prepares a table layout for output.
 func CreateTable(columns []string) *tablewriter.Table {
 	table := tablewriter.NewWriter(os.Stdout)
-	table.SetAutoWrapText(false)
 	table.SetHeader(columns)
 	return table
 }
@@ -50,22 +54,3 @@ func CreateProgressbar(length int, title string) *progressbar.ProgressBar {
 		progressbar.OptionSetDescription(title),
 	)
 }
-
-// StripTagMeta strips front-matter image tag data that we don't need for parsing.
-func StripTagMeta(image string) string {
-	originalImage := image
-
-	if strings.Contains(image, "docker.io") {
-		image = strings.Split(image, "/")[1]
-	}
-
-	if strings.Contains(image, "library") {
-		image = strings.Split(image, "/")[1]
-	}
-
-	if originalImage != image {
-		logrus.Debugf("stripped %s to %s for parsing", originalImage, image)
-	}
-
-	return image
-}

cli/internal/common.go

@@ -1,11 +1,7 @@
 package internal
 
 import (
-	"os"
-
-	logrusStack "github.com/Gurpartap/logrus-stack"
-	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
 // Secrets stores the variable from SecretsFlag
@@ -13,7 +9,9 @@ var Secrets bool
 
 // SecretsFlag turns on/off automatically generating secrets
 var SecretsFlag = &cli.BoolFlag{
-	Name:        "secrets, S",
+	Name:        "secrets",
+	Aliases:     []string{"S"},
+	Value:       false,
 	Usage:       "Automatically generate secrets",
 	Destination: &Secrets,
 }
@@ -23,19 +21,22 @@ var Pass bool
 
 // PassFlag turns on/off storing generated secrets in pass
 var PassFlag = &cli.BoolFlag{
-	Name:        "pass, p",
+	Name:        "pass",
+	Aliases:     []string{"P"},
+	Value:       false,
 	Usage:       "Store the generated secrets in a local pass store",
 	Destination: &Pass,
 }
 
-// PassRemove stores the variable for PassRemoveFlag
-var PassRemove bool
+// Context is temp
+var Context string
 
-// PassRemoveFlag turns on/off removing generated secrets from pass
-var PassRemoveFlag = &cli.BoolFlag{
-	Name:        "pass, p",
-	Usage:       "Remove generated secrets from a local pass store",
-	Destination: &PassRemove,
+// ContextFlag is temp
+var ContextFlag = &cli.StringFlag{
+	Name:        "context",
+	Value:       "",
+	Aliases:     []string{"c"},
+	Destination: &Context,
 }
 
 // Force force functionality without asking.
@@ -43,8 +44,9 @@ var Force bool
 
 // ForceFlag turns on/off force functionality.
 var ForceFlag = &cli.BoolFlag{
-	Name:        "force, f",
-	Usage:       "Perform action without further prompt. Use with care!",
+	Name:        "force",
+	Value:       false,
+	Aliases:     []string{"f"},
 	Destination: &Force,
 }
 
@@ -53,7 +55,9 @@ var Chaos bool
 
 // ChaosFlag turns on/off chaos functionality.
 var ChaosFlag = &cli.BoolFlag{
-	Name:        "chaos, C",
+	Name:        "chaos",
+	Value:       false,
+	Aliases:     []string{"ch"},
 	Usage:       "Deploy uncommitted recipes changes. Use with care!",
 	Destination: &Chaos,
 }
@@ -63,15 +67,18 @@ var DNSProvider string
 
 // DNSProviderFlag selects a DNS provider.
 var DNSProviderFlag = &cli.StringFlag{
-	Name:        "provider, p",
+	Name:        "provider",
 	Value:       "",
+	Aliases:     []string{"p"},
 	Usage:       "DNS provider",
 	Destination: &DNSProvider,
 }
 
 var NoInput bool
 var NoInputFlag = &cli.BoolFlag{
-	Name:        "no-input, n",
+	Name:        "no-input",
+	Value:       false,
+	Aliases:     []string{"n"},
 	Usage:       "Toggle non-interactive mode",
 	Destination: &NoInput,
 }
@@ -79,8 +86,9 @@ var NoInputFlag = &cli.BoolFlag{
 var DNSType string
 
 var DNSTypeFlag = &cli.StringFlag{
-	Name:        "record-type, rt",
+	Name:        "type",
 	Value:       "",
+	Aliases:     []string{"t"},
 	Usage:       "Domain name record type (e.g. A)",
 	Destination: &DNSType,
 }
@@ -88,8 +96,9 @@ var DNSTypeFlag = &cli.StringFlag{
 var DNSName string
 
 var DNSNameFlag = &cli.StringFlag{
-	Name:        "record-name, rn",
+	Name:        "name",
 	Value:       "",
+	Aliases:     []string{"n"},
 	Usage:       "Domain name record name (e.g. mysubdomain)",
 	Destination: &DNSName,
 }
@@ -97,25 +106,29 @@ var DNSNameFlag = &cli.StringFlag{
 var DNSValue string
 
 var DNSValueFlag = &cli.StringFlag{
-	Name:        "record-value, rv",
+	Name:        "value",
 	Value:       "",
+	Aliases:     []string{"v"},
 	Usage:       "Domain name record value (e.g. 192.168.1.1)",
 	Destination: &DNSValue,
 }
 
-var DNSTTL string
-var DNSTTLFlag = &cli.StringFlag{
-	Name:        "record-ttl, rl",
-	Value:       "600s",
-	Usage:       "Domain name TTL value (seconds)",
+var DNSTTL int
+
+var DNSTTLFlag = &cli.IntFlag{
+	Name:        "ttl",
+	Value:       86400,
+	Aliases:     []string{"T"},
+	Usage:       "Domain name TTL value)",
 	Destination: &DNSTTL,
 }
 
 var DNSPriority int
 
 var DNSPriorityFlag = &cli.IntFlag{
-	Name:        "record-priority, rp",
+	Name:        "priority",
 	Value:       10,
+	Aliases:     []string{"P"},
 	Usage:       "Domain name priority value",
 	Destination: &DNSPriority,
 }
@@ -123,7 +136,8 @@ var DNSPriorityFlag = &cli.IntFlag{
 var ServerProvider string
 
 var ServerProviderFlag = &cli.StringFlag{
-	Name:        "provider, p",
+	Name:        "provider",
+	Aliases:     []string{"p"},
 	Usage:       "3rd party server provider",
 	Destination: &ServerProvider,
 }
@@ -131,8 +145,9 @@ var ServerProviderFlag = &cli.StringFlag{
 var CapsulInstanceURL string
 
 var CapsulInstanceURLFlag = &cli.StringFlag{
-	Name:        "capsul-url, cu",
+	Name:        "capsul-url",
 	Value:       "yolo.servers.coop",
+	Aliases:     []string{"cu"},
 	Usage:       "capsul instance URL",
 	Destination: &CapsulInstanceURL,
 }
@@ -140,8 +155,9 @@ var CapsulInstanceURLFlag = &cli.StringFlag{
 var CapsulName string
 
 var CapsulNameFlag = &cli.StringFlag{
-	Name:        "capsul-name, cn",
+	Name:        "capsul-name",
 	Value:       "",
+	Aliases:     []string{"cn"},
 	Usage:       "capsul name",
 	Destination: &CapsulName,
 }
@@ -149,8 +165,9 @@ var CapsulNameFlag = &cli.StringFlag{
 var CapsulType string
 
 var CapsulTypeFlag = &cli.StringFlag{
-	Name:        "capsul-type, ct",
+	Name:        "capsul-type",
 	Value:       "f1-xs",
+	Aliases:     []string{"ct"},
 	Usage:       "capsul type",
 	Destination: &CapsulType,
 }
@@ -158,33 +175,38 @@ var CapsulTypeFlag = &cli.StringFlag{
 var CapsulImage string
 
 var CapsulImageFlag = &cli.StringFlag{
-	Name:        "capsul-image, ci",
+	Name:        "capsul-image",
 	Value:       "debian10",
+	Aliases:     []string{"ci"},
 	Usage:       "capsul image",
 	Destination: &CapsulImage,
 }
 
 var CapsulSSHKeys cli.StringSlice
+
 var CapsulSSHKeysFlag = &cli.StringSliceFlag{
-	Name:  "capsul-ssh-keys, cs",
-	Usage: "capsul SSH key",
-	Value: &CapsulSSHKeys,
+	Name:        "capsul-ssh-keys",
+	Aliases:     []string{"cs"},
+	Usage:       "capsul SSH key",
+	Destination: &CapsulSSHKeys,
 }
 
 var CapsulAPIToken string
 
 var CapsulAPITokenFlag = &cli.StringFlag{
-	Name:        "capsul-token, ca",
+	Name:        "capsul-token",
+	Aliases:     []string{"ca"},
 	Usage:       "capsul API token",
-	EnvVar:      "CAPSUL_TOKEN",
+	EnvVars:     []string{"CAPSUL_TOKEN"},
 	Destination: &CapsulAPIToken,
 }
 
 var HetznerCloudName string
 
 var HetznerCloudNameFlag = &cli.StringFlag{
-	Name:        "hetzner-name, hn",
+	Name:        "hetzner-name",
 	Value:       "",
+	Aliases:     []string{"hn"},
 	Usage:       "hetzner cloud name",
 	Destination: &HetznerCloudName,
 }
@@ -192,7 +214,8 @@ var HetznerCloudNameFlag = &cli.StringFlag{
 var HetznerCloudType string
 
 var HetznerCloudTypeFlag = &cli.StringFlag{
-	Name:        "hetzner-type, ht",
+	Name:        "hetzner-type",
+	Aliases:     []string{"ht"},
 	Usage:       "hetzner cloud type",
 	Destination: &HetznerCloudType,
 	Value:       "cx11",
@@ -201,7 +224,8 @@ var HetznerCloudTypeFlag = &cli.StringFlag{
 var HetznerCloudImage string
 
 var HetznerCloudImageFlag = &cli.StringFlag{
-	Name:        "hetzner-image, hi",
+	Name:        "hetzner-image",
+	Aliases:     []string{"hi"},
 	Usage:       "hetzner cloud image",
 	Value:       "debian-10",
 	Destination: &HetznerCloudImage,
@@ -210,15 +234,17 @@ var HetznerCloudImageFlag = &cli.StringFlag{
 var HetznerCloudSSHKeys cli.StringSlice
 
 var HetznerCloudSSHKeysFlag = &cli.StringSliceFlag{
-	Name:  "hetzner-ssh-keys, hs",
-	Usage: "hetzner cloud SSH keys (e.g. me@foo.com)",
-	Value: &HetznerCloudSSHKeys,
+	Name:        "hetzner-ssh-keys",
+	Aliases:     []string{"hs"},
+	Usage:       "hetzner cloud SSH keys (e.g. me@foo.com)",
+	Destination: &HetznerCloudSSHKeys,
 }
 
 var HetznerCloudLocation string
 
 var HetznerCloudLocationFlag = &cli.StringFlag{
-	Name:        "hetzner-location, hl",
+	Name:        "hetzner-location",
+	Aliases:     []string{"hl"},
 	Usage:       "hetzner cloud server location",
 	Value:       "hel1",
 	Destination: &HetznerCloudLocation,
@@ -227,9 +253,10 @@ var HetznerCloudLocationFlag = &cli.StringFlag{
 var HetznerCloudAPIToken string
 
 var HetznerCloudAPITokenFlag = &cli.StringFlag{
-	Name:        "hetzner-token, ha",
+	Name:        "hetzner-token",
+	Aliases:     []string{"ha"},
 	Usage:       "hetzner cloud API token",
-	EnvVar:      "HCLOUD_TOKEN",
+	EnvVars:     []string{"HCLOUD_TOKEN"},
 	Destination: &HetznerCloudAPIToken,
 }
 
@@ -238,59 +265,110 @@ var Debug bool
 
 // DebugFlag turns on/off verbose logging down to the DEBUG level.
 var DebugFlag = &cli.BoolFlag{
-	Name:        "debug, d",
+	Name:        "debug",
+	Aliases:     []string{"d"},
+	Value:       false,
 	Destination: &Debug,
 	Usage:       "Show DEBUG messages",
 }
 
+// Verbose stores the variable from VerboseFlag.
+var Verbose bool
+
+// VerboseFlag turns on/off verbose logging down to the INFO level.
+var VerboseFlag = &cli.BoolFlag{
+	Name:        "verbose",
+	Aliases:     []string{"V"},
+	Value:       false,
+	Destination: &Verbose,
+	Usage:       "Show INFO messages",
+}
+
 // RC signifies the latest release candidate
 var RC bool
 
 // RCFlag chooses the latest release candidate for install
 var RCFlag = &cli.BoolFlag{
-	Name:        "rc, r",
+	Name:        "rc",
+	Value:       false,
 	Destination: &RC,
 	Usage:       "Insatll the latest release candidate",
 }
 
 var Major bool
 var MajorFlag = &cli.BoolFlag{
-	Name:        "major, x",
+	Name:        "major",
 	Usage:       "Increase the major part of the version",
+	Value:       false,
+	Aliases:     []string{"ma", "x"},
 	Destination: &Major,
 }
 
 var Minor bool
 var MinorFlag = &cli.BoolFlag{
-	Name:        "minor, y",
+	Name:        "minor",
 	Usage:       "Increase the minor part of the version",
+	Value:       false,
+	Aliases:     []string{"mi", "y"},
 	Destination: &Minor,
 }
 
 var Patch bool
 var PatchFlag = &cli.BoolFlag{
-	Name:        "patch, z",
+	Name:        "patch",
 	Usage:       "Increase the patch part of the version",
+	Value:       false,
+	Aliases:     []string{"p", "z"},
 	Destination: &Patch,
 }
 
 var Dry bool
 var DryFlag = &cli.BoolFlag{
-	Name:        "dry-run, r",
+	Name:        "dry-run",
 	Usage:       "Only reports changes that would be made",
+	Value:       false,
+	Aliases:     []string{"d"},
 	Destination: &Dry,
 }
 
-var Publish bool
-var PublishFlag = &cli.BoolFlag{
-	Name:        "publish, p",
-	Usage:       "Publish changes to git.coopcloud.tech",
-	Destination: &Publish,
+var Push bool
+var PushFlag = &cli.BoolFlag{
+	Name:        "push",
+	Usage:       "Git push changes",
+	Value:       false,
+	Aliases:     []string{"P"},
+	Destination: &Push,
+}
+
+var CommitMessage string
+var CommitMessageFlag = &cli.StringFlag{
+	Name:        "commit-message",
+	Usage:       "Commit message (implies --commit)",
+	Aliases:     []string{"cm"},
+	Destination: &CommitMessage,
+}
+
+var Commit bool
+var CommitFlag = &cli.BoolFlag{
+	Name:        "commit",
+	Usage:       "Commit new changes",
+	Value:       false,
+	Aliases:     []string{"c"},
+	Destination: &Commit,
+}
+
+var TagMessage string
+var TagMessageFlag = &cli.StringFlag{
+	Name:        "tag-comment",
+	Usage:       "Description for release tag",
+	Aliases:     []string{"t", "tm"},
+	Destination: &TagMessage,
 }
 
 var Domain string
 var DomainFlag = &cli.StringFlag{
-	Name:        "domain, D",
+	Name:        "domain",
+	Aliases:     []string{"d"},
 	Value:       "",
 	Usage:       "Choose a domain name",
 	Destination: &Domain,
@@ -298,59 +376,47 @@ var DomainFlag = &cli.StringFlag{
 
 var NewAppServer string
 var NewAppServerFlag = &cli.StringFlag{
-	Name:        "server, s",
+	Name:        "server",
+	Aliases:     []string{"s"},
 	Value:       "",
 	Usage:       "Show apps of a specific server",
 	Destination: &NewAppServer,
 }
 
+var NewAppName string
+var NewAppNameFlag = &cli.StringFlag{
+	Name:        "app-name",
+	Aliases:     []string{"a"},
+	Value:       "",
+	Usage:       "Choose an app name",
+	Destination: &NewAppName,
+}
+
 var NoDomainChecks bool
 var NoDomainChecksFlag = &cli.BoolFlag{
-	Name:        "no-domain-checks, D",
+	Name:        "no-domain-checks",
+	Aliases:     []string{"nd"},
+	Value:       false,
 	Usage:       "Disable app domain sanity checks",
 	Destination: &NoDomainChecks,
 }
 
 var StdErrOnly bool
 var StdErrOnlyFlag = &cli.BoolFlag{
-	Name:        "stderr, s",
+	Name:        "stderr",
+	Aliases:     []string{"s"},
+	Value:       false,
 	Usage:       "Only tail stderr",
 	Destination: &StdErrOnly,
 }
 
-var DontWaitConverge bool
-var DontWaitConvergeFlag = &cli.BoolFlag{
-	Name:        "no-converge-checks, c",
-	Usage:       "Don't wait for converge logic checks",
-	Destination: &DontWaitConverge,
-}
-
-var Watch bool
-var WatchFlag = &cli.BoolFlag{
-	Name:        "watch, w",
-	Usage:       "Watch status by polling repeatedly",
-	Destination: &Watch,
-}
-
-var OnlyErrors bool
-var OnlyErrorFlag = &cli.BoolFlag{
-	Name:        "errors, e",
-	Usage:       "Only show errors",
-	Destination: &OnlyErrors,
-}
-
-var SkipUpdates bool
-var SkipUpdatesFlag = &cli.BoolFlag{
-	Name:        "skip-updates, s",
-	Usage:       "Skip updating recipe repositories",
-	Destination: &SkipUpdates,
-}
-
-var AllTags bool
-var AllTagsFlag = &cli.BoolFlag{
-	Name:        "all-tags, a",
-	Usage:       "List all tags, not just upgrades",
-	Destination: &AllTags,
+var AutoDNSRecord bool
+var AutoDNSRecordFlag = &cli.BoolFlag{
+	Name:        "auto",
+	Aliases:     []string{"a"},
+	Value:       false,
+	Usage:       "Automatically configure DNS records",
+	Destination: &StdErrOnly,
 }
 
 // SSHFailMsg is a hopefully helpful SSH failure message
@@ -401,33 +467,3 @@ Host foo.coopcloud.tech
 Good luck!
 
 `
-
-var ServerAddFailMsg = `
-Failed to add server %s.
-
-This could be caused by two things.
-
-Abra isn't picking up your SSH configuration or you need to specify it on the
-command-line (e.g you use a non-standard port or username to connect). Run
-"server add" with "-d/--debug" to learn more about what Abra is doing under the
-hood.
-
-Docker is not installed on your server. You can pass "-p/--provision" to
-install Docker and initialise Docker Swarm mode. See help output for "server
-add"
-
-See "abra server add -h" for more.
-
-`
-
-// SubCommandBefore wires up pre-action machinery (e.g. --debug handling).
-func SubCommandBefore(c *cli.Context) error {
-	if Debug {
-		logrus.SetLevel(logrus.DebugLevel)
-		logrus.SetFormatter(&logrus.TextFormatter{})
-		logrus.SetOutput(os.Stderr)
-		logrus.AddHook(logrusStack.StandardHook())
-	}
-
-	return nil
-}

cli/internal/copy.go

@@ -0,0 +1,71 @@
+package internal
+
+import (
+	"fmt"
+	"os"
+
+	"coopcloud.tech/abra/cli/formatter"
+	"coopcloud.tech/abra/pkg/client"
+	"coopcloud.tech/abra/pkg/config"
+	"coopcloud.tech/abra/pkg/container"
+	"github.com/docker/docker/api/types"
+	"github.com/docker/docker/api/types/filters"
+	"github.com/docker/docker/pkg/archive"
+	"github.com/sirupsen/logrus"
+	"github.com/urfave/cli/v2"
+)
+
+func ConfigureAndCp(c *cli.Context, app config.App, srcPath string, dstPath string, service string, isToContainer bool) error {
+	appFiles, err := config.LoadAppFiles("")
+	if err != nil {
+		logrus.Fatal(err)
+	}
+
+	appEnv, err := config.GetApp(appFiles, app.Name)
+	if err != nil {
+		logrus.Fatal(err)
+	}
+
+	cl, err := client.New(app.Server)
+	if err != nil {
+		logrus.Fatal(err)
+	}
+
+	filters := filters.NewArgs()
+	filters.Add("name", fmt.Sprintf("%s_%s", appEnv.StackName(), service))
+
+	container, err := container.GetContainer(c.Context, cl, filters, true)
+	if err != nil {
+		logrus.Fatal(err)
+	}
+
+	logrus.Debugf("retrieved %s as target container on %s", formatter.ShortenID(container.ID), app.Server)
+
+	if isToContainer {
+		if _, err := os.Stat(srcPath); err != nil {
+			logrus.Fatalf("%s does not exist?", srcPath)
+		}
+
+		toTarOpts := &archive.TarOptions{NoOverwriteDirNonDir: true, Compression: archive.Gzip}
+		content, err := archive.TarWithOptions(srcPath, toTarOpts)
+		if err != nil {
+			logrus.Fatal(err)
+		}
+
+		copyOpts := types.CopyToContainerOptions{AllowOverwriteDirWithFile: false, CopyUIDGID: false}
+		if err := cl.CopyToContainer(c.Context, container.ID, dstPath, content, copyOpts); err != nil {
+			logrus.Fatal(err)
+		}
+	} else {
+		content, _, err := cl.CopyFromContainer(c.Context, container.ID, srcPath)
+		if err != nil {
+			logrus.Fatal(err)
+		}
+		defer content.Close()
+		fromTarOpts := &archive.TarOptions{NoOverwriteDirNonDir: true, Compression: archive.Gzip}
+		if err := archive.Untar(content, dstPath, fromTarOpts); err != nil {
+			logrus.Fatal(err)
+		}
+	}
+	return nil
+}

cli/internal/deploy.go

@@ -1,103 +1,80 @@
 package internal
 
 import (
-	"context"
 	"fmt"
-	"io/ioutil"
-	"os"
-	"path"
 	"strings"
 
+	abraFormatter "coopcloud.tech/abra/cli/formatter"
+	"coopcloud.tech/abra/pkg/catalogue"
 	"coopcloud.tech/abra/pkg/client"
 	"coopcloud.tech/abra/pkg/config"
 	"coopcloud.tech/abra/pkg/dns"
-	"coopcloud.tech/abra/pkg/formatter"
-	"coopcloud.tech/abra/pkg/git"
-	"coopcloud.tech/abra/pkg/lint"
 	"coopcloud.tech/abra/pkg/recipe"
 	"coopcloud.tech/abra/pkg/upstream/stack"
 	"github.com/AlecAivazis/survey/v2"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
 // DeployAction is the main command-line action for this package
 func DeployAction(c *cli.Context) error {
 	app := ValidateApp(c)
-
-	if !Chaos {
-		if err := recipe.EnsureUpToDate(app.Recipe); err != nil {
-			logrus.Fatal(err)
-		}
-	}
-
-	r, err := recipe.Get(app.Recipe)
-	if err != nil {
-		logrus.Fatal(err)
-	}
-
-	if err := lint.LintForErrors(r); err != nil {
-		logrus.Fatal(err)
-	}
+	stackName := app.StackName()
 
 	cl, err := client.New(app.Server)
 	if err != nil {
 		logrus.Fatal(err)
 	}
 
-	logrus.Debugf("checking whether %s is already deployed", app.StackName())
+	logrus.Debugf("checking whether %s is already deployed", stackName)
 
-	isDeployed, deployedVersion, err := stack.IsDeployed(context.Background(), cl, app.StackName())
+	isDeployed, deployedVersion, err := stack.IsDeployed(c.Context, cl, stackName)
 	if err != nil {
 		logrus.Fatal(err)
 	}
 
 	if isDeployed {
 		if Force || Chaos {
-			logrus.Warnf("%s is already deployed but continuing (--force/--chaos)", app.Name)
+			logrus.Warnf("%s is already deployed but continuing (--force/--chaos)", stackName)
 		} else {
-			logrus.Fatalf("%s is already deployed", app.Name)
+			logrus.Fatalf("%s is already deployed", stackName)
 		}
 	}
 
 	version := deployedVersion
-	if version == "unknown" && !Chaos {
-		catl, err := recipe.ReadRecipeCatalogue()
+	if version == "" && !Chaos {
+		catl, err := catalogue.ReadRecipeCatalogue()
 		if err != nil {
 			logrus.Fatal(err)
 		}
-		versions, err := recipe.GetRecipeCatalogueVersions(app.Recipe, catl)
+		versions, err := catalogue.GetRecipeCatalogueVersions(app.Type, catl)
 		if err != nil {
 			logrus.Fatal(err)
 		}
 		if len(versions) > 0 {
 			version = versions[len(versions)-1]
 			logrus.Debugf("choosing %s as version to deploy", version)
-			if err := recipe.EnsureVersion(app.Recipe, version); err != nil {
+			if err := recipe.EnsureVersion(app.Type, version); err != nil {
 				logrus.Fatal(err)
 			}
 		} else {
-			head, err := git.GetRecipeHead(app.Recipe)
-			if err != nil {
-				logrus.Fatal(err)
-			}
-			version = formatter.SmallSHA(head.String())
+			version = "latest commit"
 			logrus.Warn("no versions detected, using latest commit")
-			if err := recipe.EnsureLatest(app.Recipe); err != nil {
+			if err := recipe.EnsureLatest(app.Type); err != nil {
 				logrus.Fatal(err)
 			}
 		}
 	}
 
-	if version == "unknown" && !Chaos {
+	if version == "" && !Chaos {
 		logrus.Debugf("choosing %s as version to deploy", version)
-		if err := recipe.EnsureVersion(app.Recipe, version); err != nil {
+		if err := recipe.EnsureVersion(app.Type, version); err != nil {
 			logrus.Fatal(err)
 		}
 	}
 
-	if version != "unknown" && !Chaos {
-		if err := recipe.EnsureVersion(app.Recipe, version); err != nil {
+	if version != "" && !Chaos {
+		if err := recipe.EnsureVersion(app.Type, version); err != nil {
 			logrus.Fatal(err)
 		}
 	}
@@ -105,13 +82,13 @@ func DeployAction(c *cli.Context) error {
 	if Chaos {
 		logrus.Warnf("chaos mode engaged")
 		var err error
-		version, err = recipe.ChaosVersion(app.Recipe)
+		version, err = recipe.ChaosVersion(app.Type)
 		if err != nil {
 			logrus.Fatal(err)
 		}
 	}
 
-	abraShPath := fmt.Sprintf("%s/%s/%s", config.RECIPES_DIR, app.Recipe, "abra.sh")
+	abraShPath := fmt.Sprintf("%s/%s/%s", config.APPS_DIR, app.Type, "abra.sh")
 	abraShEnv, err := config.ReadAbraShEnvVars(abraShPath)
 	if err != nil {
 		logrus.Fatal(err)
@@ -120,13 +97,13 @@ func DeployAction(c *cli.Context) error {
 		app.Env[k] = v
 	}
 
-	composeFiles, err := config.GetAppComposeFiles(app.Recipe, app.Env)
+	composeFiles, err := config.GetAppComposeFiles(app.Type, app.Env)
 	if err != nil {
 		logrus.Fatal(err)
 	}
 	deployOpts := stack.Deploy{
 		Composefiles: composeFiles,
-		Namespace:    app.StackName(),
+		Namespace:    stackName,
 		Prune:        false,
 		ResolveImage: stack.ResolveImageAlways,
 	}
@@ -141,6 +118,11 @@ func DeployAction(c *cli.Context) error {
 
 	if !NoDomainChecks {
 		domainName := app.Env["DOMAIN"]
+		ipv4, err := dns.EnsureIPv4(domainName)
+		if err != nil || ipv4 == "" {
+			logrus.Fatalf("could not find an IP address assigned to %s?", domainName)
+		}
+
 		if _, err = dns.EnsureDomainsResolveSameIPv4(domainName, app.Server); err != nil {
 			logrus.Fatal(err)
 		}
@@ -148,7 +130,7 @@ func DeployAction(c *cli.Context) error {
 		logrus.Warn("skipping domain checks as requested")
 	}
 
-	if err := stack.RunDeploy(cl, deployOpts, compose, app.Name, DontWaitConverge); err != nil {
+	if err := stack.RunDeploy(cl, deployOpts, compose, app.Env["TYPE"]); err != nil {
 		logrus.Fatal(err)
 	}
 
@@ -157,8 +139,8 @@ func DeployAction(c *cli.Context) error {
 
 // DeployOverview shows a deployment overview
 func DeployOverview(app config.App, version, message string) error {
-	tableCol := []string{"server", "recipe", "config", "domain", "version"}
-	table := formatter.CreateTable(tableCol)
+	tableCol := []string{"server", "compose", "domain", "stack", "version"}
+	table := abraFormatter.CreateTable(tableCol)
 
 	deployConfig := "compose.yml"
 	if composeFiles, ok := app.Env["COMPOSE_FILE"]; ok {
@@ -170,7 +152,7 @@ func DeployOverview(app config.App, version, message string) error {
 		server = "local"
 	}
 
-	table.Append([]string{server, app.Recipe, deployConfig, app.Domain, version})
+	table.Append([]string{server, deployConfig, app.Domain, app.StackName(), version})
 	table.Render()
 
 	if NoInput {
@@ -194,9 +176,9 @@ func DeployOverview(app config.App, version, message string) error {
 }
 
 // NewVersionOverview shows an upgrade or downgrade overview
-func NewVersionOverview(app config.App, currentVersion, newVersion, releaseNotes string) error {
-	tableCol := []string{"server", "recipe", "config", "domain", "current version", "to be deployed"}
-	table := formatter.CreateTable(tableCol)
+func NewVersionOverview(app config.App, currentVersion, newVersion string) error {
+	tableCol := []string{"server", "compose", "domain", "stack", "current version", "to be deployed"}
+	table := abraFormatter.CreateTable(tableCol)
 
 	deployConfig := "compose.yml"
 	if composeFiles, ok := app.Env["COMPOSE_FILE"]; ok {
@@ -208,24 +190,9 @@ func NewVersionOverview(app config.App, currentVersion, newVersion, releaseNotes
 		server = "local"
 	}
 
-	table.Append([]string{server, app.Recipe, deployConfig, app.Domain, currentVersion, newVersion})
+	table.Append([]string{server, deployConfig, app.Domain, app.StackName(), currentVersion, newVersion})
 	table.Render()
 
-	if releaseNotes == "" {
-		var err error
-		releaseNotes, err = GetReleaseNotes(app.Recipe, newVersion)
-		if err != nil {
-			return err
-		}
-	}
-
-	if releaseNotes != "" && newVersion != "" {
-		fmt.Println()
-		fmt.Println(fmt.Sprintf("%s release notes:\n\n%s", newVersion, releaseNotes))
-	} else {
-		logrus.Warnf("no release notes available for %s", newVersion)
-	}
-
 	if NoInput {
 		return nil
 	}
@@ -245,22 +212,3 @@ func NewVersionOverview(app config.App, currentVersion, newVersion, releaseNotes
 
 	return nil
 }
-
-// GetReleaseNotes prints release notes for a recipe version
-func GetReleaseNotes(recipeName, version string) (string, error) {
-	if version == "" {
-		return "", nil
-	}
-
-	fpath := path.Join(config.RECIPES_DIR, recipeName, "release", version)
-
-	if _, err := os.Stat(fpath); !os.IsNotExist(err) {
-		releaseNotes, err := ioutil.ReadFile(fpath)
-		if err != nil {
-			return "", err
-		}
-		return string(releaseNotes), nil
-	}
-
-	return "", nil
-}

cli/internal/errors.go

@@ -4,7 +4,7 @@ import (
 	"os"
 
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
 // ShowSubcommandHelpAndError exits the program on error, logs the error to the

cli/internal/list.go

@@ -1,10 +0,0 @@
-package internal
-
-// ReverseStringList reverses a list of a strings. Roll on Go generics.
-func ReverseStringList(strings []string) []string {
-	for i, j := 0, len(strings)-1; i < j; i, j = i+1, j-1 {
-		strings[i], strings[j] = strings[j], strings[i]
-	}
-
-	return strings
-}

cli/internal/new.go

@@ -4,17 +4,14 @@ import (
 	"fmt"
 	"path"
 
-	"coopcloud.tech/abra/pkg/app"
+	abraFormatter "coopcloud.tech/abra/cli/formatter"
 	"coopcloud.tech/abra/pkg/config"
-	"coopcloud.tech/abra/pkg/formatter"
 	"coopcloud.tech/abra/pkg/recipe"
-	recipePkg "coopcloud.tech/abra/pkg/recipe"
 	"coopcloud.tech/abra/pkg/secret"
 	"coopcloud.tech/abra/pkg/ssh"
 	"github.com/AlecAivazis/survey/v2"
-	"github.com/olekukonko/tablewriter"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
 // AppSecrets represents all app secrest
@@ -25,7 +22,7 @@ var RecipeName string
 
 // createSecrets creates all secrets for a new app.
 func createSecrets(sanitisedAppName string) (AppSecrets, error) {
-	appEnvPath := path.Join(config.ABRA_DIR, "servers", NewAppServer, fmt.Sprintf("%s.env", Domain))
+	appEnvPath := path.Join(config.ABRA_DIR, "servers", NewAppServer, fmt.Sprintf("%s.env", sanitisedAppName))
 	appEnv, err := config.ReadEnv(appEnvPath)
 	if err != nil {
 		return nil, err
@@ -40,7 +37,7 @@ func createSecrets(sanitisedAppName string) (AppSecrets, error) {
 	if Pass {
 		for secretName := range secrets {
 			secretValue := secrets[secretName]
-			if err := secret.PassInsertSecret(secretValue, secretName, Domain, NewAppServer); err != nil {
+			if err := secret.PassInsertSecret(secretValue, secretName, sanitisedAppName, NewAppServer); err != nil {
 				return nil, err
 			}
 		}
@@ -67,31 +64,6 @@ func ensureDomainFlag(recipe recipe.Recipe, server string) error {
 	return nil
 }
 
-// promptForSecrets asks if we should generate secrets for a new app.
-func promptForSecrets(appName string) error {
-	app, err := app.Get(appName)
-	if err != nil {
-		return err
-	}
-
-	secretEnvVars := secret.ReadSecretEnvVars(app.Env)
-	if len(secretEnvVars) == 0 {
-		logrus.Debugf("%s has no secrets to generate, skipping...", app.Recipe)
-		return nil
-	}
-
-	if !Secrets && !NoInput {
-		prompt := &survey.Confirm{
-			Message: "Generate app secrets?",
-		}
-		if err := survey.AskOne(prompt, &Secrets); err != nil {
-			return err
-		}
-	}
-
-	return nil
-}
-
 // ensureServerFlag checks if the server flag was used. if not, asks the user for it.
 func ensureServerFlag() error {
 	servers, err := config.GetServers()
@@ -116,11 +88,30 @@ func ensureServerFlag() error {
 	return nil
 }
 
+// ensureServerFlag checks if the AppName flag was used. if not, asks the user for it.
+func ensureAppNameFlag() error {
+	if NewAppName == "" && !NoInput {
+		prompt := &survey.Input{
+			Message: "Specify app name:",
+			Default: config.SanitiseAppName(Domain),
+		}
+		if err := survey.AskOne(prompt, &NewAppName); err != nil {
+			return err
+		}
+	}
+
+	if NewAppName == "" {
+		return fmt.Errorf("no app name provided")
+	}
+
+	return nil
+}
+
 // NewAction is the new app creation logic
 func NewAction(c *cli.Context) error {
-	recipe := ValidateRecipeWithPrompt(c, false)
+	recipe := ValidateRecipeWithPrompt(c)
 
-	if err := recipePkg.EnsureUpToDate(recipe.Name); err != nil {
+	if err := config.EnsureAbraDirExists(); err != nil {
 		logrus.Fatal(err)
 	}
 
@@ -132,45 +123,48 @@ func NewAction(c *cli.Context) error {
 		logrus.Fatal(err)
 	}
 
-	sanitisedAppName := config.SanitiseAppName(Domain)
-	logrus.Debugf("%s sanitised as %s for new app", Domain, sanitisedAppName)
-
-	if err := config.TemplateAppEnvSample(recipe.Name, Domain, NewAppServer, Domain); err != nil {
+	if err := ensureAppNameFlag(); err != nil {
 		logrus.Fatal(err)
 	}
 
-	if err := promptForSecrets(Domain); err != nil {
+	sanitisedAppName := config.SanitiseAppName(NewAppName)
+	if len(sanitisedAppName) > 45 {
+		logrus.Fatalf("%s cannot be longer than 45 characters", sanitisedAppName)
+	}
+	logrus.Debugf("%s sanitised as %s for new app", NewAppName, sanitisedAppName)
+
+	if err := config.TemplateAppEnvSample(recipe.Name, NewAppName, NewAppServer, Domain); err != nil {
 		logrus.Fatal(err)
 	}
 
-	var secrets AppSecrets
-	var secretTable *tablewriter.Table
 	if Secrets {
 		if err := ssh.EnsureHostKey(NewAppServer); err != nil {
 			logrus.Fatal(err)
 		}
 
-		var err error
-		secrets, err = createSecrets(sanitisedAppName)
+		secrets, err := createSecrets(sanitisedAppName)
 		if err != nil {
 			logrus.Fatal(err)
 		}
 
 		secretCols := []string{"Name", "Value"}
-		secretTable = formatter.CreateTable(secretCols)
+		secretTable := abraFormatter.CreateTable(secretCols)
 		for secret := range secrets {
 			secretTable.Append([]string{secret, secrets[secret]})
 		}
 
+		if len(secrets) > 0 {
+			defer secretTable.Render()
+		}
 	}
 
 	if NewAppServer == "default" {
 		NewAppServer = "local"
 	}
 
-	tableCol := []string{"server", "recipe", "domain"}
-	table := formatter.CreateTable(tableCol)
-	table.Append([]string{NewAppServer, recipe.Name, Domain})
+	tableCol := []string{"Name", "Domain", "Type", "Server"}
+	table := abraFormatter.CreateTable(tableCol)
+	table.Append([]string{sanitisedAppName, Domain, recipe.Name, NewAppServer})
 
 	fmt.Println("")
 	fmt.Println(fmt.Sprintf("A new %s app has been created! Here is an overview:", recipe.Name))
@@ -178,19 +172,11 @@ func NewAction(c *cli.Context) error {
 	table.Render()
 	fmt.Println("")
 	fmt.Println("You can configure this app by running the following:")
-	fmt.Println(fmt.Sprintf("\n    abra app config %s", Domain))
+	fmt.Println(fmt.Sprintf("\n    abra app config %s", sanitisedAppName))
 	fmt.Println("")
 	fmt.Println("You can deploy this app by running the following:")
-	fmt.Println(fmt.Sprintf("\n    abra app deploy %s", Domain))
+	fmt.Println(fmt.Sprintf("\n    abra app deploy %s", sanitisedAppName))
 	fmt.Println("")
 
-	if len(secrets) > 0 {
-		fmt.Println("Here are your generated secrets:")
-		fmt.Println("")
-		secretTable.Render()
-		fmt.Println("")
-		logrus.Warn("generated secrets are not shown again, please take note of them *now*")
-	}
-
 	return nil
 }

cli/internal/recipe.go

@@ -2,41 +2,21 @@ package internal
 
 import (
 	"fmt"
+	"strings"
 
-	"coopcloud.tech/abra/pkg/formatter"
 	"coopcloud.tech/abra/pkg/recipe"
 	"github.com/AlecAivazis/survey/v2"
-	"github.com/docker/distribution/reference"
 	"github.com/sirupsen/logrus"
 )
 
 // PromptBumpType prompts for version bump type
-func PromptBumpType(tagString, latestRelease string) error {
+func PromptBumpType(tagString string) error {
 	if (!Major && !Minor && !Patch) && tagString == "" {
-		fmt.Printf(`
-You need to make a decision about what kind of an update this new recipe
-version is. If someone else performs this upgrade, do they have to do some
-migration work or take care of some breaking changes? This can be signaled in
-the version you specify on the recipe deploy label and is called a semantic
-version.
-
-The latest published version is %s.
-
-Here is a semver cheat sheet (more on https://semver.org):
-
-    major: new features/bug fixes, backwards incompatible (e.g 1.0.0 -> 2.0.0).
-           the upgrade won't work without some preparation work and others need
-           to take care when performing it. "it could go wrong".
-
-    minor: new features/bug fixes, backwards compatible (e.g. 0.1.0 -> 0.2.0).
-           the upgrade should Just Work and there are no breaking changes in
-           the app and the recipe config. "it should go fine".
-
-    patch: bug fixes, backwards compatible (e.g. 0.0.1 -> 0.0.2). this upgrade
-           should also Just Work and is mostly to do with minor bug fixes
-           and/or security patches. "nothing to worry about".
-
-`, latestRelease)
+		fmt.Printf(`semver cheat sheet (more via semver.org):
+  major: new features/bug fixes, backwards incompatible
+  minor: new features/bug fixes, backwards compatible
+  patch: bug fixes, backwards compatible
+`)
 
 		var chosenBumpType string
 		prompt := &survey.Select{
@@ -84,27 +64,20 @@ func SetBumpType(bumpType string) {
 	}
 }
 
-// GetMainAppImage retrieves the main 'app' image name
-func GetMainAppImage(recipe recipe.Recipe) (string, error) {
-	var path string
+// GetMainApp retrieves the main 'app' image name
+func GetMainApp(recipe recipe.Recipe) string {
+	var app string
 
 	for _, service := range recipe.Config.Services {
-		if service.Name == "app" {
-			img, err := reference.ParseNormalizedNamed(service.Image)
-			if err != nil {
-				return "", err
-			}
-
-			path = reference.Path(img)
-			path = formatter.StripTagMeta(path)
-
-			return path, nil
+		name := service.Name
+		if name == "app" {
+			app = strings.Split(service.Image, ":")[0]
 		}
 	}
 
-	if path == "" {
-		return path, fmt.Errorf("%s has no main 'app' service?", recipe.Name)
+	if app == "" {
+		logrus.Fatalf("%s has no main 'app' service?", recipe.Name)
 	}
 
-	return path, nil
+	return app
 }

cli/internal/record.go

@@ -0,0 +1,106 @@
+package internal
+
+import (
+	"errors"
+	"fmt"
+
+	"github.com/AlecAivazis/survey/v2"
+	"github.com/urfave/cli/v2"
+)
+
+// EnsureDNSProvider ensures a DNS provider is chosen.
+func EnsureDNSProvider() error {
+	if DNSProvider == "" && !NoInput {
+		prompt := &survey.Select{
+			Message: "Select DNS provider",
+			Options: []string{"gandi"},
+		}
+
+		if err := survey.AskOne(prompt, &DNSProvider); err != nil {
+			return err
+		}
+	}
+
+	if DNSProvider == "" {
+		return fmt.Errorf("missing DNS provider?")
+	}
+
+	return nil
+}
+
+// EnsureDNSTypeFlag ensures a DNS type flag is present.
+func EnsureDNSTypeFlag(c *cli.Context) error {
+	if DNSType == "" && !NoInput {
+		prompt := &survey.Input{
+			Message: "Specify DNS record type",
+			Default: "A",
+		}
+		if err := survey.AskOne(prompt, &DNSType); err != nil {
+			return err
+		}
+	}
+
+	if DNSType == "" {
+		ShowSubcommandHelpAndError(c, errors.New("no record type provided"))
+	}
+
+	return nil
+}
+
+// EnsureDNSNameFlag ensures a DNS name flag is present.
+func EnsureDNSNameFlag(c *cli.Context) error {
+	if DNSName == "" && !NoInput {
+		prompt := &survey.Input{
+			Message: "Specify DNS record name",
+			Default: "mysubdomain",
+		}
+		if err := survey.AskOne(prompt, &DNSName); err != nil {
+			return err
+		}
+	}
+
+	if DNSName == "" {
+		ShowSubcommandHelpAndError(c, errors.New("no record name provided"))
+	}
+
+	return nil
+}
+
+// EnsureDNSValueFlag ensures a DNS value flag is present.
+func EnsureDNSValueFlag(c *cli.Context) error {
+	if DNSValue == "" && !NoInput {
+		prompt := &survey.Input{
+			Message: "Specify DNS record value",
+			Default: "192.168.1.2",
+		}
+		if err := survey.AskOne(prompt, &DNSValue); err != nil {
+			return err
+		}
+	}
+
+	if DNSName == "" {
+		ShowSubcommandHelpAndError(c, errors.New("no record value provided"))
+	}
+
+	return nil
+}
+
+// EnsureZoneArgument ensures a zone argument is present.
+func EnsureZoneArgument(c *cli.Context) (string, error) {
+	var zone string
+	if c.Args().First() == "" && !NoInput {
+		prompt := &survey.Input{
+			Message: "Specify a domain name zone",
+			Default: "example.com",
+		}
+		if err := survey.AskOne(prompt, &zone); err != nil {
+			return zone, err
+		}
+	}
+
+	if zone == "" {
+		ShowSubcommandHelpAndError(c, errors.New("no zone value provided"))
+	}
+
+	return zone, nil
+}

cli/internal/server.go

@@ -0,0 +1,208 @@
+package internal
+
+import (
+	"fmt"
+	"os"
+	"strings"
+
+	"github.com/AlecAivazis/survey/v2"
+	"github.com/urfave/cli/v2"
+)
+
+// EnsureServerProvider ensures a 3rd party server provider is chosen.
+func EnsureServerProvider() error {
+	if ServerProvider == "" && !NoInput {
+		prompt := &survey.Select{
+			Message: "Select server provider",
+			Options: []string{"capsul", "hetzner-cloud"},
+		}
+
+		if err := survey.AskOne(prompt, &ServerProvider); err != nil {
+			return err
+		}
+	}
+
+	if ServerProvider == "" {
+		return fmt.Errorf("missing server provider?")
+	}
+
+	return nil
+}
+
+// EnsureNewCapsulVPSFlags ensure all flags are present.
+func EnsureNewCapsulVPSFlags(c *cli.Context) error {
+	if CapsulName == "" && !NoInput {
+		prompt := &survey.Input{
+			Message: "specify capsul name",
+		}
+		if err := survey.AskOne(prompt, &CapsulName); err != nil {
+			return err
+		}
+	}
+
+	if !NoInput {
+		prompt := &survey.Input{
+			Message: "specify capsul instance URL",
+			Default: CapsulInstanceURL,
+		}
+		if err := survey.AskOne(prompt, &CapsulInstanceURL); err != nil {
+			return err
+		}
+	}
+
+	if !NoInput {
+		prompt := &survey.Input{
+			Message: "specify capsul type",
+			Default: CapsulType,
+		}
+		if err := survey.AskOne(prompt, &CapsulType); err != nil {
+			return err
+		}
+	}
+
+	if !NoInput {
+		prompt := &survey.Input{
+			Message: "specify capsul image",
+			Default: CapsulImage,
+		}
+		if err := survey.AskOne(prompt, &CapsulImage); err != nil {
+			return err
+		}
+	}
+
+	if len(CapsulSSHKeys.Value()) == 0 && !NoInput {
+		var sshKeys string
+		prompt := &survey.Input{
+			Message: "specify capsul SSH keys (e.g. me@foo.com)",
+			Default: "",
+		}
+		if err := survey.AskOne(prompt, &CapsulSSHKeys); err != nil {
+			return err
+		}
+		CapsulSSHKeys = *cli.NewStringSlice(strings.Split(sshKeys, ",")...)
+	}
+
+	if CapsulAPIToken == "" && !NoInput {
+		token, ok := os.LookupEnv("CAPSUL_TOKEN")
+		if !ok {
+			prompt := &survey.Input{
+				Message: "specify capsul API token",
+			}
+			if err := survey.AskOne(prompt, &CapsulAPIToken); err != nil {
+				return err
+			}
+		} else {
+			CapsulAPIToken = token
+		}
+	}
+
+	if CapsulName == "" {
+		ShowSubcommandHelpAndError(c, fmt.Errorf("missing capsul name?"))
+	}
+	if CapsulInstanceURL == "" {
+		ShowSubcommandHelpAndError(c, fmt.Errorf("missing capsul instance url?"))
+	}
+	if CapsulType == "" {
+		ShowSubcommandHelpAndError(c, fmt.Errorf("missing capsul type?"))
+	}
+	if CapsulImage == "" {
+		ShowSubcommandHelpAndError(c, fmt.Errorf("missing capsul image?"))
+	}
+	if len(CapsulSSHKeys.Value()) == 0 {
+		ShowSubcommandHelpAndError(c, fmt.Errorf("missing capsul ssh keys?"))
+	}
+	if CapsulAPIToken == "" {
+		ShowSubcommandHelpAndError(c, fmt.Errorf("missing capsul API token?"))
+	}
+
+	return nil
+}
+
+// EnsureNewHetznerCloudVPSFlags ensure all flags are present.
+func EnsureNewHetznerCloudVPSFlags(c *cli.Context) error {
+	if HetznerCloudName == "" && !NoInput {
+		prompt := &survey.Input{
+			Message: "specify hetzner cloud VPS name",
+		}
+		if err := survey.AskOne(prompt, &HetznerCloudName); err != nil {
+			return err
+		}
+	}
+
+	if !NoInput {
+		prompt := &survey.Input{
+			Message: "specify hetzner cloud VPS type",
+			Default: HetznerCloudType,
+		}
+		if err := survey.AskOne(prompt, &HetznerCloudType); err != nil {
+			return err
+		}
+	}
+
+	if !NoInput {
+		prompt := &survey.Input{
+			Message: "specify hetzner cloud VPS image",
+			Default: HetznerCloudImage,
+		}
+		if err := survey.AskOne(prompt, &HetznerCloudImage); err != nil {
+			return err
+		}
+	}
+
+	if len(HetznerCloudSSHKeys.Value()) == 0 && !NoInput {
+		var sshKeys string
+		prompt := &survey.Input{
+			Message: "specify hetzner cloud SSH keys (e.g. me@foo.com)",
+			Default: "",
+		}
+		if err := survey.AskOne(prompt, &sshKeys); err != nil {
+			return err
+		}
+		HetznerCloudSSHKeys = *cli.NewStringSlice(strings.Split(sshKeys, ",")...)
+	}
+
+	if !NoInput {
+		prompt := &survey.Input{
+			Message: "specify hetzner cloud VPS location",
+			Default: HetznerCloudLocation,
+		}
+		if err := survey.AskOne(prompt, &HetznerCloudLocation); err != nil {
+			return err
+		}
+	}
+
+	if HetznerCloudAPIToken == "" && !NoInput {
+		token, ok := os.LookupEnv("HCLOUD_TOKEN")
+		if !ok {
+			prompt := &survey.Input{
+				Message: "specify hetzner cloud API token",
+			}
+			if err := survey.AskOne(prompt, &HetznerCloudAPIToken); err != nil {
+				return err
+			}
+		} else {
+			HetznerCloudAPIToken = token
+		}
+	}
+
+	if HetznerCloudName == "" {
+		ShowSubcommandHelpAndError(c, fmt.Errorf("missing hetzner cloud VPS name?"))
+	}
+	if HetznerCloudType == "" {
+		ShowSubcommandHelpAndError(c, fmt.Errorf("missing hetzner cloud VPS type?"))
+	}
+	if HetznerCloudImage == "" {
+		ShowSubcommandHelpAndError(c, fmt.Errorf("missing hetzner cloud image?"))
+	}
+	if len(HetznerCloudSSHKeys.Value()) == 0 {
+		ShowSubcommandHelpAndError(c, fmt.Errorf("missing hetzner cloud ssh keys?"))
+	}
+	if HetznerCloudLocation == "" {
+		ShowSubcommandHelpAndError(c, fmt.Errorf("missing hetzner cloud VPS location?"))
+	}
+	if HetznerCloudAPIToken == "" {
+		ShowSubcommandHelpAndError(c, fmt.Errorf("missing hetzner cloud API token?"))
+	}
+
+	return nil
+}

cli/internal/validate.go

@@ -2,31 +2,30 @@ package internal
 
 import (
 	"errors"
-	"fmt"
-	"os"
 	"strings"
 
 	"coopcloud.tech/abra/pkg/app"
+	"coopcloud.tech/abra/pkg/catalogue"
 	"coopcloud.tech/abra/pkg/config"
 	"coopcloud.tech/abra/pkg/recipe"
 	"coopcloud.tech/abra/pkg/ssh"
 	"github.com/AlecAivazis/survey/v2"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
 // AppName is used for configuring app name programmatically
 var AppName string
 
 // ValidateRecipe ensures the recipe arg is valid.
-func ValidateRecipe(c *cli.Context, ensureLatest bool) recipe.Recipe {
+func ValidateRecipe(c *cli.Context) recipe.Recipe {
 	recipeName := c.Args().First()
 
 	if recipeName == "" {
-		ShowSubcommandHelpAndError(c, errors.New("no recipe name provided"))
+		ShowSubcommandHelpAndError(c, errors.New("no recipe provided"))
 	}
 
-	chosenRecipe, err := recipe.Get(recipeName)
+	recipe, err := recipe.Get(recipeName)
 	if err != nil {
 		if c.Command.Name == "generate" {
 			if strings.Contains(err.Error(), "missing a compose") {
@@ -38,26 +37,20 @@ func ValidateRecipe(c *cli.Context, ensureLatest bool) recipe.Recipe {
 		}
 	}
 
-	if ensureLatest {
-		if err := recipe.EnsureLatest(recipeName); err != nil {
-			logrus.Fatal(err)
-		}
-	}
+	logrus.Debugf("validated '%s' as recipe argument", recipeName)
 
-	logrus.Debugf("validated %s as recipe argument", recipeName)
-
-	return chosenRecipe
+	return recipe
 }
 
 // ValidateRecipeWithPrompt ensures a recipe argument is present before
 // validating, asking for input if required.
-func ValidateRecipeWithPrompt(c *cli.Context, ensureLatest bool) recipe.Recipe {
+func ValidateRecipeWithPrompt(c *cli.Context) recipe.Recipe {
 	recipeName := c.Args().First()
 
 	if recipeName == "" && !NoInput {
 		var recipes []string
 
-		catl, err := recipe.ReadRecipeCatalogue()
+		catl, err := catalogue.ReadRecipeCatalogue()
 		if err != nil {
 			logrus.Fatal(err)
 		}
@@ -97,23 +90,17 @@ func ValidateRecipeWithPrompt(c *cli.Context, ensureLatest bool) recipe.Recipe {
 	}
 
 	if recipeName == "" {
-		ShowSubcommandHelpAndError(c, errors.New("no recipe name provided"))
+		ShowSubcommandHelpAndError(c, errors.New("no recipe provided"))
 	}
 
-	chosenRecipe, err := recipe.Get(recipeName)
+	recipe, err := recipe.Get(recipeName)
 	if err != nil {
 		logrus.Fatal(err)
 	}
 
-	if ensureLatest {
-		if err := recipe.EnsureLatest(recipeName); err != nil {
-			logrus.Fatal(err)
-		}
-	}
-
 	logrus.Debugf("validated %s as recipe argument", recipeName)
 
-	return chosenRecipe
+	return recipe
 }
 
 // ValidateApp ensures the app name arg is valid.
@@ -134,7 +121,7 @@ func ValidateApp(c *cli.Context) config.App {
 		logrus.Fatal(err)
 	}
 
-	if err := recipe.EnsureExists(app.Recipe); err != nil {
+	if err := recipe.EnsureExists(app.Type); err != nil {
 		logrus.Fatal(err)
 	}
 
@@ -148,7 +135,7 @@ func ValidateApp(c *cli.Context) config.App {
 }
 
 // ValidateDomain ensures the domain name arg is valid.
-func ValidateDomain(c *cli.Context) string {
+func ValidateDomain(c *cli.Context) (string, error) {
 	domainName := c.Args().First()
 
 	if domainName == "" && !NoInput {
@@ -157,7 +144,7 @@ func ValidateDomain(c *cli.Context) string {
 			Default: "example.com",
 		}
 		if err := survey.AskOne(prompt, &domainName); err != nil {
-			logrus.Fatal(err)
+			return domainName, err
 		}
 	}
 
@@ -167,14 +154,14 @@ func ValidateDomain(c *cli.Context) string {
 
 	logrus.Debugf("validated %s as domain argument", domainName)
 
-	return domainName
+	return domainName, nil
 }
 
 // ValidateSubCmdFlags ensures flag order conforms to correct order
 func ValidateSubCmdFlags(c *cli.Context) bool {
-	for argIdx, arg := range c.Args() {
+	for argIdx, arg := range c.Args().Slice() {
 		if !strings.HasPrefix(arg, "--") {
-			for _, flag := range c.Args()[argIdx:] {
+			for _, flag := range c.Args().Slice()[argIdx:] {
 				if strings.HasPrefix(flag, "--") {
 					return false
 				}
@@ -185,12 +172,12 @@ func ValidateSubCmdFlags(c *cli.Context) bool {
 }
 
 // ValidateServer ensures the server name arg is valid.
-func ValidateServer(c *cli.Context) string {
+func ValidateServer(c *cli.Context) (string, error) {
 	serverName := c.Args().First()
 
 	serverNames, err := config.ReadServerNames()
 	if err != nil {
-		logrus.Fatal(err)
+		return serverName, err
 	}
 
 	if serverName == "" && !NoInput {
@@ -199,320 +186,15 @@ func ValidateServer(c *cli.Context) string {
 			Options: serverNames,
 		}
 		if err := survey.AskOne(prompt, &serverName); err != nil {
-			logrus.Fatal(err)
+			return serverName, err
 		}
 	}
 
-	matched := false
-	for _, name := range serverNames {
-		if name == serverName {
-			matched = true
-		}
-	}
-
-	if !matched {
-		ShowSubcommandHelpAndError(c, errors.New("server doesn't exist?"))
-	}
-
 	if serverName == "" {
 		ShowSubcommandHelpAndError(c, errors.New("no server provided"))
 	}
 
 	logrus.Debugf("validated %s as server argument", serverName)
 
-	return serverName
-}
-
-// EnsureDNSProvider ensures a DNS provider is chosen.
-func EnsureDNSProvider() error {
-	if DNSProvider == "" && !NoInput {
-		prompt := &survey.Select{
-			Message: "Select DNS provider",
-			Options: []string{"gandi"},
-		}
-
-		if err := survey.AskOne(prompt, &DNSProvider); err != nil {
-			return err
-		}
-	}
-
-	if DNSProvider == "" {
-		return fmt.Errorf("missing DNS provider?")
-	}
-
-	return nil
-}
-
-// EnsureDNSTypeFlag ensures a DNS type flag is present.
-func EnsureDNSTypeFlag(c *cli.Context) error {
-	if DNSType == "" && !NoInput {
-		prompt := &survey.Input{
-			Message: "Specify DNS record type",
-			Default: "A",
-		}
-		if err := survey.AskOne(prompt, &DNSType); err != nil {
-			return err
-		}
-	}
-
-	if DNSType == "" {
-		ShowSubcommandHelpAndError(c, errors.New("no record type provided"))
-	}
-
-	return nil
-}
-
-// EnsureDNSNameFlag ensures a DNS name flag is present.
-func EnsureDNSNameFlag(c *cli.Context) error {
-	if DNSName == "" && !NoInput {
-		prompt := &survey.Input{
-			Message: "Specify DNS record name",
-			Default: "mysubdomain",
-		}
-		if err := survey.AskOne(prompt, &DNSName); err != nil {
-			return err
-		}
-	}
-
-	if DNSName == "" {
-		ShowSubcommandHelpAndError(c, errors.New("no record name provided"))
-	}
-
-	return nil
-}
-
-// EnsureDNSValueFlag ensures a DNS value flag is present.
-func EnsureDNSValueFlag(c *cli.Context) error {
-	if DNSValue == "" && !NoInput {
-		prompt := &survey.Input{
-			Message: "Specify DNS record value",
-			Default: "192.168.1.2",
-		}
-		if err := survey.AskOne(prompt, &DNSValue); err != nil {
-			return err
-		}
-	}
-
-	if DNSValue == "" {
-		ShowSubcommandHelpAndError(c, errors.New("no record value provided"))
-	}
-
-	return nil
-}
-
-// EnsureZoneArgument ensures a zone argument is present.
-func EnsureZoneArgument(c *cli.Context) (string, error) {
-	zone := c.Args().First()
-
-	if zone == "" && !NoInput {
-		prompt := &survey.Input{
-			Message: "Specify a domain name zone",
-			Default: "example.com",
-		}
-
-		if err := survey.AskOne(prompt, &zone); err != nil {
-			return zone, err
-		}
-	}
-
-	if zone == "" {
-		ShowSubcommandHelpAndError(c, errors.New("no zone value provided"))
-	}
-
-	return zone, nil
-}
-
-// EnsureServerProvider ensures a 3rd party server provider is chosen.
-func EnsureServerProvider() error {
-	if ServerProvider == "" && !NoInput {
-		prompt := &survey.Select{
-			Message: "Select server provider",
-			Options: []string{"capsul", "hetzner-cloud"},
-		}
-
-		if err := survey.AskOne(prompt, &ServerProvider); err != nil {
-			return err
-		}
-	}
-
-	if ServerProvider == "" {
-		return fmt.Errorf("missing server provider?")
-	}
-
-	return nil
-}
-
-// EnsureNewCapsulVPSFlags ensure all flags are present.
-func EnsureNewCapsulVPSFlags(c *cli.Context) error {
-	if CapsulName == "" && !NoInput {
-		prompt := &survey.Input{
-			Message: "specify capsul name",
-		}
-		if err := survey.AskOne(prompt, &CapsulName); err != nil {
-			return err
-		}
-	}
-
-	if !NoInput {
-		prompt := &survey.Input{
-			Message: "specify capsul instance URL",
-			Default: CapsulInstanceURL,
-		}
-		if err := survey.AskOne(prompt, &CapsulInstanceURL); err != nil {
-			return err
-		}
-	}
-
-	if !NoInput {
-		prompt := &survey.Input{
-			Message: "specify capsul type",
-			Default: CapsulType,
-		}
-		if err := survey.AskOne(prompt, &CapsulType); err != nil {
-			return err
-		}
-	}
-
-	if !NoInput {
-		prompt := &survey.Input{
-			Message: "specify capsul image",
-			Default: CapsulImage,
-		}
-		if err := survey.AskOne(prompt, &CapsulImage); err != nil {
-			return err
-		}
-	}
-
-	if len(CapsulSSHKeys.Value()) == 0 && !NoInput {
-		var sshKeys string
-		prompt := &survey.Input{
-			Message: "specify capsul SSH keys (e.g. me@foo.com)",
-			Default: "",
-		}
-		if err := survey.AskOne(prompt, &CapsulSSHKeys); err != nil {
-			return err
-		}
-		CapsulSSHKeys = cli.StringSlice(strings.Split(sshKeys, ","))
-	}
-
-	if CapsulAPIToken == "" && !NoInput {
-		token, ok := os.LookupEnv("CAPSUL_TOKEN")
-		if !ok {
-			prompt := &survey.Input{
-				Message: "specify capsul API token",
-			}
-			if err := survey.AskOne(prompt, &CapsulAPIToken); err != nil {
-				return err
-			}
-		} else {
-			CapsulAPIToken = token
-		}
-	}
-
-	if CapsulName == "" {
-		ShowSubcommandHelpAndError(c, fmt.Errorf("missing capsul name?"))
-	}
-	if CapsulInstanceURL == "" {
-		ShowSubcommandHelpAndError(c, fmt.Errorf("missing capsul instance url?"))
-	}
-	if CapsulType == "" {
-		ShowSubcommandHelpAndError(c, fmt.Errorf("missing capsul type?"))
-	}
-	if CapsulImage == "" {
-		ShowSubcommandHelpAndError(c, fmt.Errorf("missing capsul image?"))
-	}
-	if len(CapsulSSHKeys.Value()) == 0 {
-		ShowSubcommandHelpAndError(c, fmt.Errorf("missing capsul ssh keys?"))
-	}
-	if CapsulAPIToken == "" {
-		ShowSubcommandHelpAndError(c, fmt.Errorf("missing capsul API token?"))
-	}
-
-	return nil
-}
-
-// EnsureNewHetznerCloudVPSFlags ensure all flags are present.
-func EnsureNewHetznerCloudVPSFlags(c *cli.Context) error {
-	if HetznerCloudName == "" && !NoInput {
-		prompt := &survey.Input{
-			Message: "specify hetzner cloud VPS name",
-		}
-		if err := survey.AskOne(prompt, &HetznerCloudName); err != nil {
-			return err
-		}
-	}
-
-	if !NoInput {
-		prompt := &survey.Input{
-			Message: "specify hetzner cloud VPS type",
-			Default: HetznerCloudType,
-		}
-		if err := survey.AskOne(prompt, &HetznerCloudType); err != nil {
-			return err
-		}
-	}
-
-	if !NoInput {
-		prompt := &survey.Input{
-			Message: "specify hetzner cloud VPS image",
-			Default: HetznerCloudImage,
-		}
-		if err := survey.AskOne(prompt, &HetznerCloudImage); err != nil {
-			return err
-		}
-	}
-
-	if len(HetznerCloudSSHKeys.Value()) == 0 && !NoInput {
-		var sshKeys string
-		prompt := &survey.Input{
-			Message: "specify hetzner cloud SSH keys (e.g. me@foo.com)",
-			Default: "",
-		}
-		if err := survey.AskOne(prompt, &sshKeys); err != nil {
-			return err
-		}
-		HetznerCloudSSHKeys = cli.StringSlice(strings.Split(sshKeys, ","))
-	}
-
-	if !NoInput {
-		prompt := &survey.Input{
-			Message: "specify hetzner cloud VPS location",
-			Default: HetznerCloudLocation,
-		}
-		if err := survey.AskOne(prompt, &HetznerCloudLocation); err != nil {
-			return err
-		}
-	}
-
-	if HetznerCloudAPIToken == "" && !NoInput {
-		token, ok := os.LookupEnv("HCLOUD_TOKEN")
-		if !ok {
-			prompt := &survey.Input{
-				Message: "specify hetzner cloud API token",
-			}
-			if err := survey.AskOne(prompt, &HetznerCloudAPIToken); err != nil {
-				return err
-			}
-		} else {
-			HetznerCloudAPIToken = token
-		}
-	}
-
-	if HetznerCloudName == "" {
-		ShowSubcommandHelpAndError(c, fmt.Errorf("missing hetzner cloud VPS name?"))
-	}
-	if HetznerCloudType == "" {
-		ShowSubcommandHelpAndError(c, fmt.Errorf("missing hetzner cloud VPS type?"))
-	}
-	if HetznerCloudImage == "" {
-		ShowSubcommandHelpAndError(c, fmt.Errorf("missing hetzner cloud image?"))
-	}
-	if HetznerCloudLocation == "" {
-		ShowSubcommandHelpAndError(c, fmt.Errorf("missing hetzner cloud VPS location?"))
-	}
-	if HetznerCloudAPIToken == "" {
-		ShowSubcommandHelpAndError(c, fmt.Errorf("missing hetzner cloud API token?"))
-	}
-
-	return nil
+	return serverName, nil
 }

cli/recipe/lint.go

@@ -2,78 +2,101 @@ package recipe
 
 import (
 	"fmt"
+	"os"
+	"strconv"
 
+	"coopcloud.tech/abra/cli/formatter"
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/autocomplete"
-	"coopcloud.tech/abra/pkg/formatter"
-	"coopcloud.tech/abra/pkg/lint"
-	recipePkg "coopcloud.tech/abra/pkg/recipe"
+	"coopcloud.tech/abra/pkg/config"
+	"coopcloud.tech/tagcmp"
+	"github.com/docker/distribution/reference"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
-var recipeLintCommand = cli.Command{
+var recipeLintCommand = &cli.Command{
 	Name:      "lint",
 	Usage:     "Lint a recipe",
 	Aliases:   []string{"l"},
 	ArgsUsage: "<recipe>",
-	Flags: []cli.Flag{
-		internal.DebugFlag,
-		internal.OnlyErrorFlag,
-	},
-	Before:       internal.SubCommandBefore,
-	BashComplete: autocomplete.RecipeNameComplete,
 	Action: func(c *cli.Context) error {
-		recipe := internal.ValidateRecipe(c, true)
+		recipe := internal.ValidateRecipe(c)
 
-		if err := recipePkg.EnsureUpToDate(recipe.Name); err != nil {
+		expectedVersion := false
+		if recipe.Config.Version == "3.8" {
+			expectedVersion = true
+		}
+
+		envSampleProvided := false
+		envSample := fmt.Sprintf("%s/%s/.env.sample", config.APPS_DIR, recipe.Name)
+		if _, err := os.Stat(envSample); !os.IsNotExist(err) {
+			envSampleProvided = true
+		} else if err != nil {
 			logrus.Fatal(err)
 		}
 
-		tableCol := []string{"ref", "rule", "satisfied", "severity", "resolve"}
-		table := formatter.CreateTable(tableCol)
+		serviceNamedApp := false
+		traefikEnabled := false
+		healthChecksForAllServices := true
+		allImagesTagged := true
+		noUnstableTags := true
+		semverLikeTags := true
+		for _, service := range recipe.Config.Services {
+			if service.Name == "app" {
+				serviceNamedApp = true
+			}
 
-		hasError := false
-		bar := formatter.CreateProgressbar(-1, "running recipe lint rules...")
-		for level := range lint.LintRules {
-			for _, rule := range lint.LintRules[level] {
-				ok, err := rule.Function(recipe)
-				if err != nil {
-					logrus.Warn(err)
-				}
-
-				if !ok && rule.Level == "error" {
-					hasError = true
-				}
-
-				var result string
-				if ok {
-					result = "yes"
-				} else {
-					result = "NO"
-				}
-
-				if internal.OnlyErrors {
-					if !ok && rule.Level == "error" {
-						table.Append([]string{rule.Ref, rule.Description, result, rule.Level, rule.HowToResolve})
-						bar.Add(1)
+			for label := range service.Deploy.Labels {
+				if label == "traefik.enable" {
+					if service.Deploy.Labels[label] == "true" {
+						traefikEnabled = true
 					}
-				} else {
-					table.Append([]string{rule.Ref, rule.Description, result, rule.Level, rule.HowToResolve})
-					bar.Add(1)
 				}
 			}
+
+			img, err := reference.ParseNormalizedNamed(service.Image)
+			if err != nil {
+				logrus.Fatal(err)
+			}
+			if reference.IsNameOnly(img) {
+				allImagesTagged = false
+			}
+
+			var tag string
+			switch img.(type) {
+			case reference.NamedTagged:
+				tag = img.(reference.NamedTagged).Tag()
+			case reference.Named:
+				noUnstableTags = false
+			}
+
+			if tag == "latest" {
+				noUnstableTags = false
+			}
+
+			if !tagcmp.IsParsable(tag) {
+				semverLikeTags = false
+			}
+
+			if service.HealthCheck == nil {
+				healthChecksForAllServices = false
+			}
 		}
 
-		if table.NumLines() > 0 {
-			fmt.Println()
-			table.Render()
-		}
-
-		if hasError {
-			logrus.Warn("watch out, some critical errors are present in your recipe config")
-		}
+		tableCol := []string{"rule", "satisfied"}
+		table := formatter.CreateTable(tableCol)
+		table.Append([]string{"compose files have the expected version", strconv.FormatBool(expectedVersion)})
+		table.Append([]string{"environment configuration is provided", strconv.FormatBool(envSampleProvided)})
+		table.Append([]string{"recipe contains a service named 'app'", strconv.FormatBool(serviceNamedApp)})
+		table.Append([]string{"traefik routing enabled on at least one service", strconv.FormatBool(traefikEnabled)})
+		table.Append([]string{"all services have a healthcheck enabled", strconv.FormatBool(healthChecksForAllServices)})
+		table.Append([]string{"all images are using a tag", strconv.FormatBool(allImagesTagged)})
+		table.Append([]string{"no usage of unstable 'latest' tags", strconv.FormatBool(noUnstableTags)})
+		table.Append([]string{"all tags are using a semver-like format", strconv.FormatBool(semverLikeTags)})
+		table.Render()
 
 		return nil
 	},
+	BashComplete: autocomplete.RecipeNameComplete,
 }

cli/recipe/list.go

@@ -3,74 +3,36 @@ package recipe
 import (
 	"fmt"
 	"sort"
-	"strconv"
-	"strings"
 
-	"coopcloud.tech/abra/cli/internal"
-	"coopcloud.tech/abra/pkg/formatter"
-	"coopcloud.tech/abra/pkg/recipe"
+	"coopcloud.tech/abra/cli/formatter"
+	"coopcloud.tech/abra/pkg/catalogue"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
-var pattern string
-var patternFlag = &cli.StringFlag{
-	Name:        "pattern, p",
-	Value:       "",
-	Usage:       "Simple string to filter recipes",
-	Destination: &pattern,
-}
-
-var recipeListCommand = cli.Command{
+var recipeListCommand = &cli.Command{
 	Name:    "list",
 	Usage:   "List available recipes",
 	Aliases: []string{"ls"},
-	Flags: []cli.Flag{
-		internal.DebugFlag,
-		patternFlag,
-	},
-	Before: internal.SubCommandBefore,
 	Action: func(c *cli.Context) error {
-		catl, err := recipe.ReadRecipeCatalogue()
+		catl, err := catalogue.ReadRecipeCatalogue()
 		if err != nil {
 			logrus.Fatal(err.Error())
 		}
 
 		recipes := catl.Flatten()
-		sort.Sort(recipe.ByRecipeName(recipes))
+		sort.Sort(catalogue.ByRecipeName(recipes))
 
-		tableCol := []string{"name", "category", "status", "healthcheck", "backups", "email", "tests", "SSO"}
+		tableCol := []string{"name", "category", "status"}
 		table := formatter.CreateTable(tableCol)
 
-		len := 0
 		for _, recipe := range recipes {
-			tableRow := []string{
-				recipe.Name,
-				recipe.Category,
-				strconv.Itoa(recipe.Features.Status),
-				recipe.Features.Healthcheck,
-				recipe.Features.Backups,
-				recipe.Features.Email,
-				recipe.Features.Tests,
-				recipe.Features.SSO,
-			}
-
-			if pattern != "" {
-				if strings.Contains(recipe.Name, pattern) {
-					table.Append(tableRow)
-					len++
-				}
-			} else {
-				table.Append(tableRow)
-				len++
-			}
+			status := fmt.Sprintf("%v", recipe.Features.Status)
+			tableRow := []string{recipe.Name, recipe.Category, status}
+			table.Append(tableRow)
 		}
 
-		table.SetCaption(true, fmt.Sprintf("total recipes: %v", len))
-
-		if table.NumLines() > 0 {
-			table.Render()
-		}
+		table.Render()
 
 		return nil
 	},

cli/recipe/new.go

@@ -1,10 +1,8 @@
 package recipe
 
 import (
-	"bytes"
 	"errors"
 	"fmt"
-	"io/ioutil"
 	"os"
 	"path"
 	"text/template"
@@ -13,32 +11,13 @@ import (
 	"coopcloud.tech/abra/pkg/config"
 	"coopcloud.tech/abra/pkg/git"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
-// recipeMetadata is the recipe metadata for the README.md
-type recipeMetadata struct {
-	Name        string
-	Description string
-	Category    string
-	Status      string
-	Image       string
-	Healthcheck string
-	Backups     string
-	Email       string
-	Tests       string
-	SSO         string
-}
-
-var recipeNewCommand = cli.Command{
-	Name:    "new",
-	Aliases: []string{"n"},
-	Flags: []cli.Flag{
-		internal.DebugFlag,
-		internal.NoInputFlag,
-	},
-	Before:    internal.SubCommandBefore,
+var recipeNewCommand = &cli.Command{
+	Name:      "new",
 	Usage:     "Create a new recipe",
+	Aliases:   []string{"n"},
 	ArgsUsage: "<recipe>",
 	Description: `
 This command creates a new recipe.
@@ -50,15 +29,17 @@ Abra uses our built-in example repository which is available here:
 Files within the example repository make use of the Golang templating system
 which Abra uses to inject values into the generated recipe folder (e.g. name of
 recipe and domain in the sample environment config).
+
+The new example repository is cloned to ~/.abra/apps/<recipe>.
 `,
 	Action: func(c *cli.Context) error {
 		recipeName := c.Args().First()
 
 		if recipeName == "" {
-			internal.ShowSubcommandHelpAndError(c, errors.New("no recipe name provided"))
+			internal.ShowSubcommandHelpAndError(c, errors.New("no recipe provided"))
 		}
 
-		directory := path.Join(config.RECIPES_DIR, recipeName)
+		directory := path.Join(config.APPS_DIR, recipeName)
 		if _, err := os.Stat(directory); !os.IsNotExist(err) {
 			logrus.Fatalf("%s recipe directory already exists?", directory)
 		}
@@ -68,73 +49,49 @@ recipe and domain in the sample environment config).
 			logrus.Fatal(err)
 		}
 
-		gitRepo := path.Join(config.RECIPES_DIR, recipeName, ".git")
+		gitRepo := path.Join(config.APPS_DIR, recipeName, ".git")
 		if err := os.RemoveAll(gitRepo); err != nil {
 			logrus.Fatal(err)
 		}
-		logrus.Debugf("removed example git repo in %s", gitRepo)
-
-		meta := newRecipeMeta(recipeName)
+		logrus.Debugf("removed git repo in %s", gitRepo)
 
 		toParse := []string{
-			path.Join(config.RECIPES_DIR, recipeName, "README.md"),
-			path.Join(config.RECIPES_DIR, recipeName, ".env.sample"),
+			path.Join(config.APPS_DIR, recipeName, "README.md"),
+			path.Join(config.APPS_DIR, recipeName, ".env.sample"),
+			path.Join(config.APPS_DIR, recipeName, ".drone.yml"),
 		}
 		for _, path := range toParse {
+			file, err := os.OpenFile(path, os.O_RDWR, 0664)
+			if err != nil {
+				logrus.Fatal(err)
+			}
+
 			tpl, err := template.ParseFiles(path)
 			if err != nil {
 				logrus.Fatal(err)
 			}
 
-			var templated bytes.Buffer
-			if err := tpl.Execute(&templated, meta); err != nil {
+			// TODO: ask for description and probably other things so that the
+			// template repository is more "ready" to go than the current best-guess
+			// mode of templating
+			if err := tpl.Execute(file, struct {
+				Name        string
+				Description string
+			}{recipeName, "TODO"}); err != nil {
 				logrus.Fatal(err)
 			}
-
-			if err := ioutil.WriteFile(path, templated.Bytes(), 0644); err != nil {
-				logrus.Fatal(err)
-			}
-
 		}
 
-		newGitRepo := path.Join(config.RECIPES_DIR, recipeName)
+		newGitRepo := path.Join(config.APPS_DIR, recipeName)
 		if err := git.Init(newGitRepo, true); err != nil {
 			logrus.Fatal(err)
 		}
 
-		fmt.Print(fmt.Sprintf(`
-Your new %s recipe has been created in %s.
-
-In order to share your recipe, you can upload it the git repository to:
-
-    https://git.coopcloud.tech/coop-cloud/%s
-
-If you're not sure how to do that, come chat with us:
-
-    https://docs.coopcloud.tech/contact
-
-See "abra recipe -h" for additional recipe maintainer commands.
-
-Happy Hacking!
-
-`, recipeName, path.Join(config.RECIPES_DIR, recipeName), recipeName))
+		logrus.Infof(
+			"new recipe %s created in %s, happy hacking!\n",
+			recipeName, path.Join(config.APPS_DIR, recipeName),
+		)
 
 		return nil
 	},
 }
-
-// newRecipeMeta creates a new recipeMetadata instance with defaults
-func newRecipeMeta(recipeName string) recipeMetadata {
-	return recipeMetadata{
-		Name:        recipeName,
-		Description: "> One line description of the recipe",
-		Category:    "Apps",
-		Status:      "0",
-		Image:       fmt.Sprintf("[`%s`](https://hub.docker.com/r/%s), 4, upstream", recipeName, recipeName),
-		Healthcheck: "No",
-		Backups:     "No",
-		Email:       "No",
-		Tests:       "No",
-		SSO:         "No",
-	}
-}

cli/recipe/recipe.go

@@ -1,15 +1,15 @@
 package recipe
 
 import (
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
 // RecipeCommand defines all recipe related sub-commands.
-var RecipeCommand = cli.Command{
+var RecipeCommand = &cli.Command{
 	Name:      "recipe",
-	Aliases:   []string{"r"},
 	Usage:     "Manage recipes",
 	ArgsUsage: "<recipe>",
+	Aliases:   []string{"r"},
 	Description: `
 A recipe is a blueprint for an app. It is a bunch of config files which
 describe how to deploy and maintain an app. Recipes are maintained by the Co-op
@@ -20,7 +20,7 @@ sure the recipe is in good working order and the config upgraded in a timely
 manner. Abra supports convenient automation for recipe maintainenace, see the
 "abra recipe upgrade", "abra recipe sync" and "abra recipe release" commands.
 `,
-	Subcommands: []cli.Command{
+	Subcommands: []*cli.Command{
 		recipeListCommand,
 		recipeVersionCommand,
 		recipeReleaseCommand,

cli/recipe/release.go

@@ -6,10 +6,10 @@ import (
 	"strconv"
 	"strings"
 
+	abraFormatter "coopcloud.tech/abra/cli/formatter"
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/autocomplete"
 	"coopcloud.tech/abra/pkg/config"
-	"coopcloud.tech/abra/pkg/formatter"
 	gitPkg "coopcloud.tech/abra/pkg/git"
 	"coopcloud.tech/abra/pkg/recipe"
 	recipePkg "coopcloud.tech/abra/pkg/recipe"
@@ -17,19 +17,22 @@ import (
 	"github.com/AlecAivazis/survey/v2"
 	"github.com/docker/distribution/reference"
 	"github.com/go-git/go-git/v5"
+	configPkg "github.com/go-git/go-git/v5/config"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
-var recipeReleaseCommand = cli.Command{
+var recipeReleaseCommand = &cli.Command{
 	Name:      "release",
-	Aliases:   []string{"rl"},
 	Usage:     "Release a new recipe version",
+	Aliases:   []string{"rl"},
 	ArgsUsage: "<recipe> [<version>]",
 	Description: `
-This command is used to specify a new version of a recipe. These versions are
-then published on the Co-op Cloud recipe catalogue. These versions take the
-following form:
+This command is used to specify a new tag for a recipe. These tags are used to
+identify different versions of the recipe and are published on the Co-op Cloud
+recipe catalogue.
+
+These tags take the following form:
 
     a.b.c+x.y.z
 
@@ -38,39 +41,38 @@ the "x.y.z" part is the image tag of the recipe "app" service (the main
 container which contains the software to be used).
 
 We maintain a semantic versioning scheme ("a.b.c") alongside the libre app
-versioning scheme ("x.y.z") in order to maximise the chances that the nature of
-recipe updates are properly communicated. I.e. developers of an app might
-publish a minor version but that might lead to changes in the recipe which are
-major and therefore require intervention while doing the upgrade work.
+versioning scheme in order to maximise the chances that the nature of recipe
+updates are properly communicated.
+
+Abra does its best to read the "a.b.c" version scheme and communicate what
+action needs to be taken when performing different operations such as an update
+or a rollback of an app.
+
+You may invoke this command in "wizard" mode and be prompted for input:
+
+    abra recipe release gitea
 
-Publish your new release to git.coopcloud.tech with "-p/--publish". This
-requires that you have permission to git push to these repositories and have
-your SSH keys configured on your account.
 `,
 	Flags: []cli.Flag{
-		internal.DebugFlag,
-		internal.NoInputFlag,
 		internal.DryFlag,
 		internal.MajorFlag,
 		internal.MinorFlag,
 		internal.PatchFlag,
-		internal.PublishFlag,
+		internal.PushFlag,
+		internal.CommitFlag,
+		internal.CommitMessageFlag,
+		internal.TagMessageFlag,
 	},
-	Before:       internal.SubCommandBefore,
 	BashComplete: autocomplete.RecipeNameComplete,
 	Action: func(c *cli.Context) error {
-		recipe := internal.ValidateRecipeWithPrompt(c, false)
+		recipe := internal.ValidateRecipeWithPrompt(c)
 
 		imagesTmp, err := getImageVersions(recipe)
 		if err != nil {
 			logrus.Fatal(err)
 		}
 
-		mainApp, err := internal.GetMainAppImage(recipe)
-		if err != nil {
-			logrus.Fatal(err)
-		}
-
+		mainApp := internal.GetMainApp(recipe)
 		mainAppVersion := imagesTmp[mainApp]
 		if mainAppVersion == "" {
 			logrus.Fatalf("main app service version for %s is empty?", recipe.Name)
@@ -95,15 +97,7 @@ your SSH keys configured on your account.
 
 		tags, err := recipe.Tags()
 		if err != nil {
-			logrus.Fatal(err)
-		}
-
-		if tagString == "" && (!internal.Major && !internal.Minor && !internal.Patch) {
-			var err error
-			tagString, err = getLabelVersion(recipe, false)
-			if err != nil {
-				logrus.Fatal(err)
-			}
+			return err
 		}
 
 		if len(tags) > 0 {
@@ -114,8 +108,28 @@ your SSH keys configured on your account.
 		} else {
 			logrus.Warnf("no tag specified and no previous tag available for %s, assuming this is the initial release", recipe.Name)
 
-			if err := createReleaseFromTag(recipe, tagString, mainAppVersion); err != nil {
-				if cleanUpErr := cleanUpTag(tagString, recipe.Name); err != nil {
+			initTag, err := recipePkg.GetVersionLabelLocal(recipe)
+			if err != nil {
+				logrus.Fatal(err)
+			}
+
+			logrus.Warnf("discovered %s as currently synced recipe label", initTag)
+
+			prompt := &survey.Confirm{
+				Message: fmt.Sprintf("use %s as the initial release?", initTag),
+			}
+
+			var response bool
+			if err := survey.AskOne(prompt, &response); err != nil {
+				return err
+			}
+
+			if !response {
+				logrus.Fatalf("please fix your synced label for %s and re-run this command", recipe.Name)
+			}
+
+			if err := createReleaseFromTag(recipe, initTag, mainAppVersion); err != nil {
+				if cleanUpErr := cleanUpTag(initTag, recipe.Name); err != nil {
 					logrus.Fatal(cleanUpErr)
 				}
 				logrus.Fatal(err)
@@ -130,7 +144,6 @@ your SSH keys configured on your account.
 func getImageVersions(recipe recipe.Recipe) (map[string]string, error) {
 	var services = make(map[string]string)
 
-	missingTag := false
 	for _, service := range recipe.Config.Services {
 		if service.Image == "" {
 			continue
@@ -142,27 +155,21 @@ func getImageVersions(recipe recipe.Recipe) (map[string]string, error) {
 		}
 
 		path := reference.Path(img)
-
-		path = formatter.StripTagMeta(path)
+		if strings.Contains(path, "library") {
+			path = strings.Split(path, "/")[1]
+		}
 
 		var tag string
 		switch img.(type) {
 		case reference.NamedTagged:
 			tag = img.(reference.NamedTagged).Tag()
 		case reference.Named:
-			if service.Name == "app" {
-				missingTag = true
-			}
-			continue
+			return services, fmt.Errorf("%s service is missing image tag?", path)
 		}
 
 		services[path] = tag
 	}
 
-	if missingTag {
-		return services, fmt.Errorf("app service is missing image tag?")
-	}
-
 	return services, nil
 }
 
@@ -170,7 +177,7 @@ func getImageVersions(recipe recipe.Recipe) (map[string]string, error) {
 func createReleaseFromTag(recipe recipe.Recipe, tagString, mainAppVersion string) error {
 	var err error
 
-	directory := path.Join(config.RECIPES_DIR, recipe.Name)
+	directory := path.Join(config.APPS_DIR, recipe.Name)
 	repo, err := git.PlainOpen(directory)
 	if err != nil {
 		return err
@@ -191,19 +198,19 @@ func createReleaseFromTag(recipe recipe.Recipe, tagString, mainAppVersion string
 		tag.MissingPatch = false
 	}
 
-	if tagString == "" {
-		tagString = fmt.Sprintf("%s+%s", tag.String(), mainAppVersion)
+	if err := commitRelease(recipe); err != nil {
+		logrus.Fatal(err)
 	}
 
-	if err := commitRelease(recipe, tagString); err != nil {
-		logrus.Fatal(err)
+	if tagString == "" {
+		tagString = fmt.Sprintf("%s+%s", tag.String(), mainAppVersion)
 	}
 
 	if err := tagRelease(tagString, repo); err != nil {
 		logrus.Fatal(err)
 	}
 
-	if err := pushRelease(recipe, tagString); err != nil {
+	if err := pushRelease(tagString, repo); err != nil {
 		logrus.Fatal(err)
 	}
 
@@ -220,32 +227,52 @@ func btoi(b bool) int {
 }
 
 // getTagCreateOptions constructs git tag create options
-func getTagCreateOptions(tag string) (git.CreateTagOptions, error) {
-	msg := fmt.Sprintf("chore: publish %s release", tag)
-	return git.CreateTagOptions{Message: msg}, nil
-}
+func getTagCreateOptions() (git.CreateTagOptions, error) {
+	if internal.TagMessage == "" && !internal.NoInput {
+		prompt := &survey.Input{
+			Message: "git tag message",
+			Default: "chore: publish new release",
+		}
 
-func commitRelease(recipe recipe.Recipe, tag string) error {
-	if internal.Dry {
-		logrus.Debugf("dry run: no changes committed")
-		return nil
-	}
-
-	isClean, err := gitPkg.IsClean(recipe.Dir())
-	if err != nil {
-		return err
-	}
-
-	if isClean {
-		if !internal.Dry {
-			return fmt.Errorf("no changes discovered in %s, nothing to publish?", recipe.Dir())
+		if err := survey.AskOne(prompt, &internal.TagMessage); err != nil {
+			return git.CreateTagOptions{}, err
 		}
 	}
 
-	msg := fmt.Sprintf("chore: publish %s release", tag)
-	repoPath := path.Join(config.RECIPES_DIR, recipe.Name)
-	if err := gitPkg.Commit(repoPath, ".", msg, internal.Dry); err != nil {
-		return err
+	return git.CreateTagOptions{Message: internal.TagMessage}, nil
+}
+
+func commitRelease(recipe recipe.Recipe) error {
+	if internal.Dry {
+		logrus.Info("dry run: no changed committed")
+		return nil
+	}
+
+	if !internal.Commit && !internal.NoInput {
+		prompt := &survey.Confirm{
+			Message: "git commit changes?",
+		}
+
+		if err := survey.AskOne(prompt, &internal.Commit); err != nil {
+			return err
+		}
+	}
+
+	if internal.CommitMessage == "" && !internal.NoInput && internal.Commit {
+		prompt := &survey.Input{
+			Message: "commit message",
+			Default: "chore: publish new version",
+		}
+		if err := survey.AskOne(prompt, &internal.CommitMessage); err != nil {
+			return err
+		}
+	}
+
+	if internal.Commit {
+		repoPath := path.Join(config.APPS_DIR, recipe.Name)
+		if err := gitPkg.Commit(repoPath, "compose.**yml", internal.CommitMessage, internal.Dry, false); err != nil {
+			return err
+		}
 	}
 
 	return nil
@@ -253,7 +280,7 @@ func commitRelease(recipe recipe.Recipe, tag string) error {
 
 func tagRelease(tagString string, repo *git.Repository) error {
 	if internal.Dry {
-		logrus.Debugf("dry run: no git tag created (%s)", tagString)
+		logrus.Infof("dry run: no git tag created (%s)", tagString)
 		return nil
 	}
 
@@ -262,7 +289,7 @@ func tagRelease(tagString string, repo *git.Repository) error {
 		return err
 	}
 
-	createTagOptions, err := getTagCreateOptions(tagString)
+	createTagOptions, err := getTagCreateOptions()
 	if err != nil {
 		return err
 	}
@@ -272,43 +299,46 @@ func tagRelease(tagString string, repo *git.Repository) error {
 		return err
 	}
 
-	hash := formatter.SmallSHA(head.Hash().String())
-	logrus.Debugf(fmt.Sprintf("created tag %s at %s", tagString, hash))
+	hash := abraFormatter.SmallSHA(head.Hash().String())
+	logrus.Info(fmt.Sprintf("created tag %s at %s", tagString, hash))
 
 	return nil
 }
 
-func pushRelease(recipe recipe.Recipe, tagString string) error {
+func pushRelease(tagString string, repo *git.Repository) error {
 	if internal.Dry {
-		logrus.Info("dry run: no changes published")
+		logrus.Info("dry run: no changes pushed")
 		return nil
 	}
 
-	if !internal.Publish && !internal.NoInput {
+	if !internal.Push && !internal.NoInput {
 		prompt := &survey.Confirm{
-			Message: "publish new release?",
+			Message: "git push changes?",
 		}
 
-		if err := survey.AskOne(prompt, &internal.Publish); err != nil {
+		if err := survey.AskOne(prompt, &internal.Push); err != nil {
 			return err
 		}
 	}
 
-	if internal.Publish {
-		if err := recipe.Push(internal.Dry); err != nil {
+	if internal.Push {
+		tagRef := fmt.Sprintf("+refs/tags/%s:refs/tags/%s", tagString, tagString)
+		pushOpts := &git.PushOptions{
+			RefSpecs: []configPkg.RefSpec{
+				configPkg.RefSpec(tagRef),
+			},
+		}
+		if err := repo.Push(pushOpts); err != nil {
 			return err
 		}
-		url := fmt.Sprintf("%s/%s/src/tag/%s", config.REPOS_BASE_URL, recipe.Name, tagString)
-		logrus.Infof("new release published: %s", url)
-	} else {
-		logrus.Info("no -p/--publish passed, not publishing")
+		logrus.Info(fmt.Sprintf("pushed tag %s to remote", tagString))
 	}
 
 	return nil
 }
 
 func createReleaseFromPreviousTag(tagString, mainAppVersion string, recipe recipe.Recipe, tags []string) error {
-	directory := path.Join(config.RECIPES_DIR, recipe.Name)
+	directory := path.Join(config.APPS_DIR, recipe.Name)
 	repo, err := git.PlainOpen(directory)
 	if err != nil {
 		return err
@@ -321,6 +351,10 @@ func createReleaseFromPreviousTag(tagString, mainAppVersion string, recipe recip
 		}
 	}
 
+	if err := internal.PromptBumpType(tagString); err != nil {
+		return err
+	}
+
 	var lastGitTag tagcmp.Tag
 	for _, tag := range tags {
 		parsed, err := tagcmp.Parse(tag)
@@ -362,45 +396,18 @@ func createReleaseFromPreviousTag(tagString, mainAppVersion string, recipe recip
 		newTag.Major = strconv.Itoa(now + 1)
 	}
 
-	if tagString == "" {
-		if err := internal.PromptBumpType(tagString, lastGitTag.String()); err != nil {
-			return err
-		}
-	}
-
-	if internal.Major || internal.Minor || internal.Patch {
-		newTag.Metadata = mainAppVersion
-		tagString = newTag.String()
-	}
-
-	if lastGitTag.String() == tagString {
-		logrus.Fatalf("latest git tag (%s) and synced lable (%s) are the same?", lastGitTag, tagString)
-	}
-
-	if !internal.NoInput {
-		prompt := &survey.Confirm{
-			Message: fmt.Sprintf("current: %s, new: %s, correct?", lastGitTag, tagString),
-		}
-
-		var ok bool
-		if err := survey.AskOne(prompt, &ok); err != nil {
-			logrus.Fatal(err)
-		}
-
-		if !ok {
-			logrus.Fatal("exiting as requested")
-		}
-	}
-
-	if err := commitRelease(recipe, tagString); err != nil {
+	if err := commitRelease(recipe); err != nil {
 		logrus.Fatal(err)
 	}
 
-	if err := tagRelease(tagString, repo); err != nil {
+	newTag.Metadata = mainAppVersion
+	newTagString := newTag.String()
+
+	if err := tagRelease(newTagString, repo); err != nil {
 		logrus.Fatal(err)
 	}
 
-	if err := pushRelease(recipe, tagString); err != nil {
+	if err := pushRelease(newTagString, repo); err != nil {
 		logrus.Fatal(err)
 	}
 
@@ -409,46 +416,17 @@ func createReleaseFromPreviousTag(tagString, mainAppVersion string, recipe recip
 
 // cleanUpTag removes a freshly created tag
 func cleanUpTag(tag, recipeName string) error {
-	directory := path.Join(config.RECIPES_DIR, recipeName)
+	directory := path.Join(config.APPS_DIR, recipeName)
 	repo, err := git.PlainOpen(directory)
 	if err != nil {
 		return err
 	}
 
 	if err := repo.DeleteTag(tag); err != nil {
-		if !strings.Contains(err.Error(), "not found") {
-			return err
-		}
+		return err
 	}
 
-	logrus.Debugf("removed freshly created tag %s", tag)
+	logrus.Warn("removed freshly created tag %s")
 
 	return nil
 }
-
-func getLabelVersion(recipe recipe.Recipe, prompt bool) (string, error) {
-	initTag, err := recipePkg.GetVersionLabelLocal(recipe)
-	if err != nil {
-		return "", err
-	}
-
-	if initTag == "" {
-		logrus.Fatalf("unable to read version for %s from synced label. Did you try running \"abra recipe sync %s\" already?", recipe.Name, recipe.Name)
-	}
-
-	logrus.Warnf("discovered %s as currently synced recipe label", initTag)
-
-	if prompt && !internal.NoInput {
-		var response bool
-		prompt := &survey.Confirm{Message: fmt.Sprintf("use %s as the new version?", initTag)}
-		if err := survey.AskOne(prompt, &response); err != nil {
-			return "", err
-		}
-
-		if !response {
-			return "", fmt.Errorf("please fix your synced label for %s and re-run this command", recipe.Name)
-		}
-	}
-
-	return initTag, nil
-}

cli/recipe/sync.go

@@ -13,40 +13,39 @@ import (
 	"github.com/go-git/go-git/v5"
 	"github.com/go-git/go-git/v5/plumbing"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
-var recipeSyncCommand = cli.Command{
+var recipeSyncCommand = &cli.Command{
 	Name:      "sync",
+	Usage:     "Ensure recipe version labels are up-to-date",
 	Aliases:   []string{"s"},
-	Usage:     "Sync recipe version label",
 	ArgsUsage: "<recipe> [<version>]",
 	Flags: []cli.Flag{
-		internal.DebugFlag,
-		internal.NoInputFlag,
 		internal.DryFlag,
 		internal.MajorFlag,
 		internal.MinorFlag,
 		internal.PatchFlag,
 	},
-	Before: internal.SubCommandBefore,
 	Description: `
 This command will generate labels for the main recipe service (i.e. by
-convention, the service named 'app') which corresponds to the following format:
+convention, the service named "app") which corresponds to the following format:
 
     coop-cloud.${STACK_NAME}.version=<version>
 
-Where <version> can be specifed on the command-line or Abra can attempt to
-auto-generate it for you. The <recipe> configuration will be updated on the
-local file system.
+The <version> is determined by the recipe maintainer and is specified on the
+command-line. The <recipe> configuration will be updated on the local file
+system.
+
+You may invoke this command in "wizard" mode and be prompted for input:
+
+    abra recipe sync
+
 `,
 	Action: func(c *cli.Context) error {
-		recipe := internal.ValidateRecipeWithPrompt(c, false)
+		recipe := internal.ValidateRecipeWithPrompt(c)
 
-		mainApp, err := internal.GetMainAppImage(recipe)
-		if err != nil {
-			logrus.Fatal(err)
-		}
+		mainApp := internal.GetMainApp(recipe)
 
 		imagesTmp, err := getImageVersions(recipe)
 		if err != nil {
@@ -64,21 +63,16 @@ local file system.
 		if len(tags) == 0 && nextTag == "" {
 			logrus.Warnf("no git tags found for %s", recipe.Name)
 			fmt.Println(fmt.Sprintf(`
-The following options are two types of initial semantic version that you can
-pick for %s that will be published in the recipe catalogue. This follows the
-semver convention (more on https://semver.org), here is a short cheatsheet
+The following options are two types of initial version that you can pick for
+the first published version of %s that will be in the recipe catalogue. This
+follows the semver convention (more on semver.org), here is a short cheatsheet
 
-    0.1.0: development release, still hacking. when you make a major upgrade
-           you increment the "y" part (i.e. 0.1.0 -> 0.2.0) and only move to
-           using the "x" part when things are stable.
+    0.1.0 -> development release, still hacking
+    1.0.0 -> public release, assumed to be working
 
-    1.0.0: public release, assumed to be working. you already have a stable
-           and reliable deployment of this app and feel relatively confident
-           about it.
-
-If you want people to be able alpha test your current config for %s but don't
-think it is quite reliable, go with 0.1.0 and people will know that things are
-likely to change.
+In other words, if you want people to be able alpha test your current config
+for %s but don't think it is quite ready and reliable, go with 0.1.0 and people
+will know that things are likely to change.
 
 `, recipe.Name, recipe.Name))
 			var chosenVersion string
@@ -95,14 +89,13 @@ likely to change.
 		}
 
 		if nextTag == "" && (!internal.Major && !internal.Minor && !internal.Patch) {
-			latestRelease := tags[len(tags)-1]
-			if err := internal.PromptBumpType("", latestRelease); err != nil {
+			if err := internal.PromptBumpType(""); err != nil {
 				logrus.Fatal(err)
 			}
 		}
 
 		if nextTag == "" {
-			recipeDir := path.Join(config.RECIPES_DIR, recipe.Name)
+			recipeDir := path.Join(config.APPS_DIR, recipe.Name)
 			repo, err := git.PlainOpen(recipeDir)
 			if err != nil {
 				logrus.Fatal(err)

cli/recipe/upgrade.go

@@ -9,16 +9,14 @@ import (
 	"strings"
 
 	"coopcloud.tech/abra/cli/internal"
-	"coopcloud.tech/abra/pkg/autocomplete"
+	"coopcloud.tech/abra/pkg/catalogue"
 	"coopcloud.tech/abra/pkg/client"
 	"coopcloud.tech/abra/pkg/config"
-	"coopcloud.tech/abra/pkg/formatter"
-	recipePkg "coopcloud.tech/abra/pkg/recipe"
 	"coopcloud.tech/tagcmp"
 	"github.com/AlecAivazis/survey/v2"
 	"github.com/docker/distribution/reference"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
 type imgPin struct {
@@ -26,10 +24,10 @@ type imgPin struct {
 	version tagcmp.Tag
 }
 
-var recipeUpgradeCommand = cli.Command{
+var recipeUpgradeCommand = &cli.Command{
 	Name:    "upgrade",
-	Aliases: []string{"u"},
 	Usage:   "Upgrade recipe image tags",
+	Aliases: []string{"u"},
 	Description: `
 This command reads and attempts to parse all image tags within the given
 <recipe> configuration and prompt with more recent tags to upgrade to. It will
@@ -39,27 +37,19 @@ Some image tags cannot be parsed because they do not follow some sort of
 semver-like convention. In this case, all possible tags will be listed and it
 is up to the end-user to decide.
 
-The command is interactive and will show a select input which allows you to
-make a seclection. Use the "?" key to see more help on navigating this
-interface.
-
 You may invoke this command in "wizard" mode and be prompted for input:
 
     abra recipe upgrade
+
 `,
-	BashComplete: autocomplete.RecipeNameComplete,
-	ArgsUsage:    "<recipe>",
+	ArgsUsage: "<recipe>",
 	Flags: []cli.Flag{
-		internal.DebugFlag,
-		internal.NoInputFlag,
 		internal.PatchFlag,
 		internal.MinorFlag,
 		internal.MajorFlag,
-		internal.AllTagsFlag,
 	},
-	Before: internal.SubCommandBefore,
 	Action: func(c *cli.Context) error {
-		recipe := internal.ValidateRecipeWithPrompt(c, true)
+		recipe := internal.ValidateRecipeWithPrompt(c)
 
 		bumpType := btoi(internal.Major)*4 + btoi(internal.Minor)*2 + btoi(internal.Patch)
 		if bumpType != 0 {
@@ -71,7 +61,7 @@ You may invoke this command in "wizard" mode and be prompted for input:
 
 		// check for versions file and load pinned versions
 		versionsPresent := false
-		recipeDir := path.Join(config.RECIPES_DIR, recipe.Name)
+		recipeDir := path.Join(config.ABRA_DIR, "apps", recipe.Name)
 		versionsPath := path.Join(recipeDir, "versions")
 		var servicePins = make(map[string]imgPin)
 		if _, err := os.Stat(versionsPath); err == nil {
@@ -112,36 +102,33 @@ You may invoke this command in "wizard" mode and be prompted for input:
 				logrus.Fatal(err)
 			}
 
-			regVersions, err := client.GetRegistryTags(img)
+			image := reference.Path(img)
+			regVersions, err := client.GetRegistryTags(image)
 			if err != nil {
 				logrus.Fatal(err)
 			}
-
-			image := reference.Path(img)
 			logrus.Debugf("retrieved %s from remote registry for %s", regVersions, image)
-			image = formatter.StripTagMeta(image)
 
-			switch img.(type) {
-			case reference.NamedTagged:
-				if !tagcmp.IsParsable(img.(reference.NamedTagged).Tag()) {
-					logrus.Debugf("%s not considered semver-like", img.(reference.NamedTagged).Tag())
-				}
-			default:
-				logrus.Warnf("unable to read tag for image %s, is it missing? skipping upgrade for %s", image, service.Name)
-				continue
+			if strings.Contains(image, "library") {
+				// ParseNormalizedNamed prepends 'library' to images like nginx:<tag>,
+				// postgres:<tag>, i.e. images which do not have a username in the
+				// first position of the string
+				image = strings.Split(image, "/")[1]
+			}
+			semverLikeTag := true
+			if !tagcmp.IsParsable(img.(reference.NamedTagged).Tag()) {
+				logrus.Debugf("%s not considered semver-like", img.(reference.NamedTagged).Tag())
+				semverLikeTag = false
 			}
 
 			tag, err := tagcmp.Parse(img.(reference.NamedTagged).Tag())
-			if err != nil {
-				logrus.Warnf("unable to parse %s, error was: %s, skipping upgrade for %s", image, err.Error(), service.Name)
-				continue
+			if err != nil && semverLikeTag {
+				logrus.Fatal(err)
 			}
-
 			logrus.Debugf("parsed %s for %s", tag, service.Name)
-
 			var compatible []tagcmp.Tag
 			for _, regVersion := range regVersions {
-				other, err := tagcmp.Parse(regVersion)
+				other, err := tagcmp.Parse(regVersion.Name)
 				if err != nil {
 					continue // skip tags that cannot be parsed
 				}
@@ -155,17 +142,17 @@ You may invoke this command in "wizard" mode and be prompted for input:
 
 			sort.Sort(tagcmp.ByTagDesc(compatible))
 
-			if len(compatible) == 0 && !internal.AllTags {
-				logrus.Info(fmt.Sprintf("no new versions available for %s, assuming %s is the latest (use -a/--all-tags to see all anyway)", image, tag))
+			if len(compatible) == 0 && semverLikeTag {
+				logrus.Info(fmt.Sprintf("no new versions available for %s, %s is the latest", image, tag))
 				continue // skip on to the next tag and don't update any compose files
 			}
 
-			catlVersions, err := recipePkg.VersionsOfService(recipe.Name, service.Name)
+			catlVersions, err := catalogue.VersionsOfService(recipe.Name, service.Name)
 			if err != nil {
 				logrus.Fatal(err)
 			}
 
-			compatibleStrings := []string{"skip"}
+			var compatibleStrings []string
 			for _, compat := range compatible {
 				skip := false
 				for _, catlVersion := range catlVersions {
@@ -195,13 +182,13 @@ You may invoke this command in "wizard" mode and be prompted for input:
 						}
 					}
 					if contains {
-						logrus.Infof("upgrading service %s from %s to %s (pinned tag: %s)", service.Name, tag.String(), upgradeTag, pinnedTagString)
+						logrus.Infof("Upgrading service %s from %s to %s (pinned tag: %s)", service.Name, tag.String(), upgradeTag, pinnedTagString)
 					} else {
 						logrus.Infof("service %s, image %s pinned to %s, no compatible upgrade found", service.Name, servicePins[service.Name].image, pinnedTagString)
 						continue
 					}
 				} else {
-					logrus.Fatalf("service %s is at version %s, but pinned to %s, please correct your compose.yml file manually!", service.Name, tag.String(), pinnedTag.String())
+					logrus.Fatalf("Service %s is at version %s, but pinned to %s, please correct your compose.yml file manually!", service.Name, tag.String(), pinnedTag.String())
 					continue
 				}
 			} else {
@@ -218,27 +205,23 @@ You may invoke this command in "wizard" mode and be prompted for input:
 						}
 					}
 					if upgradeTag == "" {
-						logrus.Warnf("not upgrading from %s to %s for %s, because the upgrade type is more serious than what user wants", tag.String(), compatible[0].String(), image)
+						logrus.Warnf("not upgrading from %s to %s for %s, because the upgrade type is more serious than what user wants.", tag.String(), compatible[0].String(), image)
 						continue
 					}
 				} else {
-					msg := fmt.Sprintf("upgrade to which tag? (service: %s, image: %s, tag: %s)", service.Name, image, tag)
-					if !tagcmp.IsParsable(img.(reference.NamedTagged).Tag()) || internal.AllTags {
+					msg := fmt.Sprintf("upgrade to which tag? (service: %s, tag: %s)", service.Name, tag)
+					if !tagcmp.IsParsable(img.(reference.NamedTagged).Tag()) {
 						tag := img.(reference.NamedTagged).Tag()
-						if !internal.AllTags {
-							logrus.Warning(fmt.Sprintf("unable to determine versioning semantics of %s, listing all tags", tag))
-						}
+						logrus.Warning(fmt.Sprintf("unable to determine versioning semantics of %s, listing all tags", tag))
 						msg = fmt.Sprintf("upgrade to which tag? (service: %s, tag: %s)", service.Name, tag)
-						compatibleStrings = []string{"skip"}
+						compatibleStrings = []string{}
 						for _, regVersion := range regVersions {
-							compatibleStrings = append(compatibleStrings, regVersion)
+							compatibleStrings = append(compatibleStrings, regVersion.Name)
 						}
 					}
 
 					prompt := &survey.Select{
 						Message: msg,
-						Help:    "enter / return to confirm, choose 'skip' to not upgrade this tag, vim mode is enabled",
-						VimMode: true,
 						Options: compatibleStrings,
 					}
 					if err := survey.AskOne(prompt, &upgradeTag); err != nil {
@@ -246,17 +229,10 @@ You may invoke this command in "wizard" mode and be prompted for input:
 					}
 				}
 			}
-			if upgradeTag != "skip" {
-				ok, err := recipe.UpdateTag(image, upgradeTag)
-				if err != nil {
-					logrus.Fatal(err)
-				}
-				if ok {
-					logrus.Infof("tag upgraded from %s to %s for %s", tag.String(), upgradeTag, image)
-				}
-			} else {
-				logrus.Warnf("not upgrading %s, skipping as requested", image)
+			if err := recipe.UpdateTag(image, upgradeTag); err != nil {
+				logrus.Fatal(err)
 			}
+			logrus.Infof("tag upgraded from %s to %s for %s", tag.String(), upgradeTag, image)
 		}
 
 		return nil

cli/recipe/version.go

@@ -1,42 +1,36 @@
 package recipe
 
 import (
+	"coopcloud.tech/abra/cli/formatter"
 	"coopcloud.tech/abra/cli/internal"
-	"coopcloud.tech/abra/pkg/autocomplete"
-	"coopcloud.tech/abra/pkg/formatter"
-	recipePkg "coopcloud.tech/abra/pkg/recipe"
+	"coopcloud.tech/abra/pkg/catalogue"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
-var recipeVersionCommand = cli.Command{
+var recipeVersionCommand = &cli.Command{
 	Name:      "versions",
-	Aliases:   []string{"v"},
 	Usage:     "List recipe versions",
+	Aliases:   []string{"v"},
 	ArgsUsage: "<recipe>",
-	Flags: []cli.Flag{
-		internal.DebugFlag,
-	},
-	Before:       internal.SubCommandBefore,
-	BashComplete: autocomplete.RecipeNameComplete,
 	Action: func(c *cli.Context) error {
-		recipe := internal.ValidateRecipe(c, false)
+		recipe := internal.ValidateRecipe(c)
 
-		catalogue, err := recipePkg.ReadRecipeCatalogue()
+		catalogue, err := catalogue.ReadRecipeCatalogue()
 		if err != nil {
 			logrus.Fatal(err)
 		}
 
 		recipeMeta, ok := catalogue[recipe.Name]
 		if !ok {
-			logrus.Fatalf("%s recipe doesn't exist?", recipe.Name)
+			logrus.Fatalf("'%s' recipe doesn't exist?", recipe.Name)
 		}
 
 		tableCol := []string{"Version", "Service", "Image", "Tag", "Digest"}
 		table := formatter.CreateTable(tableCol)
 
-		for i := len(recipeMeta.Versions) - 1; i >= 0; i-- {
-			for tag, meta := range recipeMeta.Versions[i] {
+		for _, serviceVersion := range recipeMeta.Versions {
+			for tag, meta := range serviceVersion {
 				for service, serviceMeta := range meta {
 					table.Append([]string{tag, service, serviceMeta.Image, serviceMeta.Tag, serviceMeta.Digest})
 				}
@@ -44,12 +38,7 @@ var recipeVersionCommand = cli.Command{
 		}
 
 		table.SetAutoMergeCells(true)
-
-		if table.NumLines() > 0 {
-			table.Render()
-		} else {
-			logrus.Fatalf("%s has no published versions?", recipe.Name)
-		}
+		table.Render()
 
 		return nil
 	},

cli/record/list.go

@@ -1,29 +1,26 @@
 package record
 
 import (
-	"context"
 	"fmt"
 	"strconv"
 
+	abraFormatter "coopcloud.tech/abra/cli/formatter"
 	"coopcloud.tech/abra/cli/internal"
 	gandiPkg "coopcloud.tech/abra/pkg/dns/gandi"
-	"coopcloud.tech/abra/pkg/formatter"
 	"github.com/libdns/gandi"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
 // RecordListCommand lists domains.
-var RecordListCommand = cli.Command{
+var RecordListCommand = &cli.Command{
 	Name:      "list",
 	Usage:     "List domain name records",
 	Aliases:   []string{"ls"},
 	ArgsUsage: "<zone>",
 	Flags: []cli.Flag{
-		internal.DebugFlag,
 		internal.DNSProviderFlag,
 	},
-	Before: internal.SubCommandBefore,
 	Description: `
 This command lists all domain name records managed by a 3rd party provider for
 a specific zone.
@@ -49,16 +46,16 @@ are listed. This zone must already be created on your provider account.
 				logrus.Fatal(err)
 			}
 		default:
-			logrus.Fatalf("%s is not a supported DNS provider", internal.DNSProvider)
+			logrus.Fatalf("'%s' is not a supported DNS provider", internal.DNSProvider)
 		}
 
-		records, err := provider.GetRecords(context.Background(), zone)
+		records, err := provider.GetRecords(c.Context, zone)
 		if err != nil {
 			logrus.Fatal(err)
 		}
 
 		tableCol := []string{"type", "name", "value", "TTL", "priority"}
-		table := formatter.CreateTable(tableCol)
+		table := abraFormatter.CreateTable(tableCol)
 
 		for _, record := range records {
 			value := record.Value

cli/record/new.go

@@ -1,37 +1,35 @@
 package record
 
 import (
-	"context"
 	"fmt"
 	"strconv"
+	"time"
 
+	abraFormatter "coopcloud.tech/abra/cli/formatter"
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/dns"
 	gandiPkg "coopcloud.tech/abra/pkg/dns/gandi"
-	"coopcloud.tech/abra/pkg/formatter"
 	"github.com/libdns/gandi"
 	"github.com/libdns/libdns"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
 // RecordNewCommand creates a new domain name record.
-var RecordNewCommand = cli.Command{
+var RecordNewCommand = &cli.Command{
 	Name:      "new",
 	Usage:     "Create a new domain record",
 	Aliases:   []string{"n"},
 	ArgsUsage: "<zone>",
 	Flags: []cli.Flag{
-		internal.DebugFlag,
-		internal.NoInputFlag,
 		internal.DNSProviderFlag,
 		internal.DNSTypeFlag,
 		internal.DNSNameFlag,
 		internal.DNSValueFlag,
 		internal.DNSTTLFlag,
 		internal.DNSPriorityFlag,
+		internal.AutoDNSRecordFlag,
 	},
-	Before: internal.SubCommandBefore,
 	Description: `
 This command creates a new domain name record for a specific zone.
 
@@ -42,7 +40,13 @@ Example:
 
     abra record new foo.com -p gandi -t A -n myapp -v 192.168.178.44
 
-You may also invoke this command in "wizard" mode and be prompted for input:
+Typically, you need two records, an A record which points at the zone (@.) and
+a wildcard record for your apps (*.). Pass "--auto" to have Abra automatically
+set this up.
+
+    abra record new --auto
+
+You may also invoke this command in "wizard" mode and be prompted for input
 
     abra record new
 `,
@@ -64,7 +68,15 @@ You may also invoke this command in "wizard" mode and be prompted for input:
 				logrus.Fatal(err)
 			}
 		default:
-			logrus.Fatalf("%s is not a supported DNS provider", internal.DNSProvider)
+			logrus.Fatalf("'%s' is not a supported DNS provider", internal.DNSProvider)
+		}
+
+		if internal.AutoDNSRecord {
+			logrus.Infof("automatically configuring @./*. A records for %s (--auto)", zone)
+			if err := autoConfigure(c, &provider, zone); err != nil {
+				logrus.Fatal(err)
+			}
+			return nil
 		}
 
 		if err := internal.EnsureDNSTypeFlag(c); err != nil {
@@ -79,23 +91,18 @@ You may also invoke this command in "wizard" mode and be prompted for input:
 			logrus.Fatal(err)
 		}
 
-		ttl, err := dns.GetTTL(internal.DNSTTL)
-		if err != nil {
-			return err
-		}
-
 		record := libdns.Record{
 			Type:  internal.DNSType,
 			Name:  internal.DNSName,
 			Value: internal.DNSValue,
-			TTL:   ttl,
+			TTL:   time.Duration(internal.DNSTTL),
 		}
 
 		if internal.DNSType == "MX" || internal.DNSType == "SRV" || internal.DNSType == "URI" {
 			record.Priority = internal.DNSPriority
 		}
 
-		records, err := provider.GetRecords(context.Background(), zone)
+		records, err := provider.GetRecords(c.Context, zone)
 		if err != nil {
 			logrus.Fatal(err)
 		}
@@ -109,7 +116,7 @@ You may also invoke this command in "wizard" mode and be prompted for input:
 		}
 
 		createdRecords, err := provider.SetRecords(
-			context.Background(),
+			c.Context,
 			zone,
 			[]libdns.Record{record},
 		)
@@ -124,7 +131,7 @@ You may also invoke this command in "wizard" mode and be prompted for input:
 		createdRecord := createdRecords[0]
 
 		tableCol := []string{"type", "name", "value", "TTL", "priority"}
-		table := formatter.CreateTable(tableCol)
+		table := abraFormatter.CreateTable(tableCol)
 
 		value := createdRecord.Value
 		if len(createdRecord.Value) > 30 {
@@ -146,3 +153,79 @@ You may also invoke this command in "wizard" mode and be prompted for input:
 		return nil
 	},
 }
+
+func autoConfigure(c *cli.Context, provider *gandi.Provider, zone string) error {
+	ipv4, err := dns.EnsureIPv4(zone)
+	if err != nil {
+		return err
+	}
+
+	atRecord := libdns.Record{
+		Type:  "A",
+		Name:  "@",
+		Value: ipv4,
+		TTL:   time.Duration(internal.DNSTTL),
+	}
+
+	wildcardRecord := libdns.Record{
+		Type:  "A",
+		Name:  "*",
+		Value: ipv4,
+		TTL:   time.Duration(internal.DNSTTL),
+	}
+
+	records := []libdns.Record{atRecord, wildcardRecord}
+
+	tableCol := []string{"type", "name", "value", "TTL", "priority"}
+	table := abraFormatter.CreateTable(tableCol)
+
+	for _, record := range records {
+		existingRecords, err := provider.GetRecords(c.Context, zone)
+		if err != nil {
+			return err
+		}
+
+		for _, existingRecord := range existingRecords {
+			if existingRecord.Type == record.Type &&
+				existingRecord.Name == record.Name &&
+				existingRecord.Value == record.Value {
+				logrus.Warnf("%s record for %s already exists?", record.Type, zone)
+				continue
+			}
+		}
+
+		createdRecords, err := provider.SetRecords(
+			c.Context,
+			zone,
+			[]libdns.Record{record},
+		)
+		if err != nil {
+			return err
+		}
+
+		if len(createdRecords) == 0 {
+			return fmt.Errorf("provider library reports that no record was created?")
+		}
+
+		createdRecord := createdRecords[0]
+
+		value := createdRecord.Value
+		if len(createdRecord.Value) > 30 {
+			value = fmt.Sprintf("%s...", createdRecord.Value[:30])
+		}
+
+		table.Append([]string{
+			createdRecord.Type,
+			createdRecord.Name,
+			value,
+			createdRecord.TTL.String(),
+			strconv.Itoa(createdRecord.Priority),
+		})
+	}
+
+	if table.NumLines() > 0 {
+		table.Render()
+	}
+
+	return nil
+}

cli/record/record.go

@@ -1,13 +1,13 @@
 package record
 
 import (
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
 // RecordCommand supports managing DNS entries.
-var RecordCommand = cli.Command{
+var RecordCommand = &cli.Command{
 	Name:      "record",
-	Usage:     "Manage domain name records",
+	Usage:     "Manage domain name records via 3rd party providers",
 	Aliases:   []string{"rc"},
 	ArgsUsage: "<record>",
 	Description: `
@@ -28,8 +28,9 @@ library documentation for more. It supports many existing providers and allows
 to implement new provider support easily.
 
     https://pkg.go.dev/github.com/libdns/libdns
+
 `,
-	Subcommands: []cli.Command{
+	Subcommands: []*cli.Command{
 		RecordListCommand,
 		RecordNewCommand,
 		RecordRemoveCommand,

cli/record/remove.go

@@ -1,34 +1,30 @@
 package record
 
 import (
-	"context"
 	"fmt"
 	"strconv"
 
+	abraFormatter "coopcloud.tech/abra/cli/formatter"
 	"coopcloud.tech/abra/cli/internal"
 	gandiPkg "coopcloud.tech/abra/pkg/dns/gandi"
-	"coopcloud.tech/abra/pkg/formatter"
 	"github.com/AlecAivazis/survey/v2"
 	"github.com/libdns/gandi"
 	"github.com/libdns/libdns"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
 // RecordRemoveCommand lists domains.
-var RecordRemoveCommand = cli.Command{
+var RecordRemoveCommand = &cli.Command{
 	Name:      "remove",
 	Usage:     "Remove a domain name record",
 	Aliases:   []string{"rm"},
 	ArgsUsage: "<zone>",
 	Flags: []cli.Flag{
-		internal.DebugFlag,
-		internal.NoInputFlag,
 		internal.DNSProviderFlag,
 		internal.DNSTypeFlag,
 		internal.DNSNameFlag,
 	},
-	Before: internal.SubCommandBefore,
 	Description: `
 This command removes a domain name record for a specific zone.
 
@@ -41,7 +37,7 @@ Example:
 
     abra record remove foo.com -p gandi -t A -n myapp
 
-You may also invoke this command in "wizard" mode and be prompted for input:
+You may also invoke this command in "wizard" mode and be prompted for input
 
     abra record rm
 `,
@@ -63,7 +59,7 @@ You may also invoke this command in "wizard" mode and be prompted for input:
 				logrus.Fatal(err)
 			}
 		default:
-			logrus.Fatalf("%s is not a supported DNS provider", internal.DNSProvider)
+			logrus.Fatalf("'%s' is not a supported DNS provider", internal.DNSProvider)
 		}
 
 		if err := internal.EnsureDNSTypeFlag(c); err != nil {
@@ -74,7 +70,7 @@ You may also invoke this command in "wizard" mode and be prompted for input:
 			logrus.Fatal(err)
 		}
 
-		records, err := provider.GetRecords(context.Background(), zone)
+		records, err := provider.GetRecords(c.Context, zone)
 		if err != nil {
 			logrus.Fatal(err)
 		}
@@ -92,7 +88,7 @@ You may also invoke this command in "wizard" mode and be prompted for input:
 		}
 
 		tableCol := []string{"type", "name", "value", "TTL", "priority"}
-		table := formatter.CreateTable(tableCol)
+		table := abraFormatter.CreateTable(tableCol)
 
 		value := toDelete.Value
 		if len(toDelete.Value) > 30 {
@@ -109,22 +105,20 @@ You may also invoke this command in "wizard" mode and be prompted for input:
 
 		table.Render()
 
-		if !internal.NoInput {
-			response := false
-			prompt := &survey.Confirm{
-				Message: "continue with record deletion?",
-			}
-
-			if err := survey.AskOne(prompt, &response); err != nil {
-				return err
-			}
-
-			if !response {
-				logrus.Fatal("exiting as requested")
-			}
+		response := false
+		prompt := &survey.Confirm{
+			Message: "continue with record deletion?",
 		}
 
-		_, err = provider.DeleteRecords(context.Background(), zone, []libdns.Record{toDelete})
+		if err := survey.AskOne(prompt, &response); err != nil {
+			return err
+		}
+
+		if !response {
+			logrus.Fatal("exiting as requested")
+		}
+
+		_, err = provider.DeleteRecords(c.Context, zone, []libdns.Record{toDelete})
 		if err != nil {
 			logrus.Fatal(err)
 		}

cli/server/add.go

@@ -1,12 +1,12 @@
 package server
 
 import (
-	"context"
 	"errors"
 	"fmt"
 	"os"
 	"os/exec"
 	"os/user"
+	"path"
 	"path/filepath"
 	"strings"
 
@@ -22,7 +22,7 @@ import (
 	"github.com/docker/docker/api/types/swarm"
 	dockerClient "github.com/docker/docker/client"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
 var (
@@ -41,26 +41,32 @@ such purposes. Docker stable is now installed by default by this script. The
 source for this script can be seen here:
 
     https://github.com/docker/docker-install
+
 `
 )
 
 var local bool
 var localFlag = &cli.BoolFlag{
-	Name:        "local, l",
+	Name:        "local",
+	Aliases:     []string{"l"},
+	Value:       false,
 	Usage:       "Use local server",
 	Destination: &local,
 }
 
 var provision bool
 var provisionFlag = &cli.BoolFlag{
-	Name:        "provision, p",
+	Name:        "provision",
+	Aliases:     []string{"p"},
+	Value:       false,
 	Usage:       "Provision server so it can deploy apps",
 	Destination: &provision,
 }
 
 var sshAuth string
 var sshAuthFlag = &cli.StringFlag{
-	Name:        "ssh-auth, s",
+	Name:        "ssh-auth",
+	Aliases:     []string{"sh"},
 	Value:       "identity-file",
 	Usage:       "Select SSH authentication method (identity-file, password)",
 	Destination: &sshAuth,
@@ -68,11 +74,22 @@ var sshAuthFlag = &cli.StringFlag{
 
 var askSudoPass bool
 var askSudoPassFlag = &cli.BoolFlag{
-	Name:        "ask-sudo-pass, a",
+	Name:        "ask-sudo-pass",
+	Aliases:     []string{"as"},
+	Value:       false,
 	Usage:       "Ask for sudo password",
 	Destination: &askSudoPass,
 }
 
+var traefik bool
+var traefikFlag = &cli.BoolFlag{
+	Name:        "traefik",
+	Aliases:     []string{"t"},
+	Value:       false,
+	Usage:       "Deploy traefik",
+	Destination: &traefik,
+}
+
 func cleanUp(domainName string) {
 	logrus.Warnf("cleaning up context for %s", domainName)
 	if err := client.DeleteContext(domainName); err != nil {
@@ -80,7 +97,7 @@ func cleanUp(domainName string) {
 	}
 
 	logrus.Warnf("cleaning up server directory for %s", domainName)
-	if err := os.RemoveAll(filepath.Join(config.SERVERS_DIR, domainName)); err != nil {
+	if err := os.RemoveAll(filepath.Join(config.ABRA_SERVER_FOLDER, domainName)); err != nil {
 		logrus.Fatal(err)
 	}
 }
@@ -146,6 +163,12 @@ func newLocalServer(c *cli.Context, domainName string) error {
 		}
 	}
 
+	if traefik {
+		if err := deployTraefik(c, cl, domainName); err != nil {
+			return err
+		}
+	}
+
 	logrus.Info("local server has been added")
 
 	return nil
@@ -297,7 +320,7 @@ If nothing works, you try running the Docker install script manually on your ser
 
 func initSwarmLocal(c *cli.Context, cl *dockerClient.Client, domainName string) error {
 	initReq := swarm.InitRequest{ListenAddr: "0.0.0.0:2377"}
-	if _, err := cl.SwarmInit(context.Background(), initReq); err != nil {
+	if _, err := cl.SwarmInit(c.Context, initReq); err != nil {
 		if strings.Contains(err.Error(), "is already part of a swarm") ||
 			strings.Contains(err.Error(), "must specify a listening address") {
 			logrus.Infof("swarm mode already initialised on %s", domainName)
@@ -309,7 +332,7 @@ func initSwarmLocal(c *cli.Context, cl *dockerClient.Client, domainName string)
 	}
 
 	netOpts := types.NetworkCreate{Driver: "overlay", Scope: "swarm"}
-	if _, err := cl.NetworkCreate(context.Background(), "proxy", netOpts); err != nil {
+	if _, err := cl.NetworkCreate(c.Context, "proxy", netOpts); err != nil {
 		if !strings.Contains(err.Error(), "proxy already exists") {
 			return err
 		}
@@ -331,7 +354,7 @@ func initSwarm(c *cli.Context, cl *dockerClient.Client, domainName string) error
 		ListenAddr:    "0.0.0.0:2377",
 		AdvertiseAddr: ipv4,
 	}
-	if _, err := cl.SwarmInit(context.Background(), initReq); err != nil {
+	if _, err := cl.SwarmInit(c.Context, initReq); err != nil {
 		if strings.Contains(err.Error(), "is already part of a swarm") ||
 			strings.Contains(err.Error(), "must specify a listening address") {
 			logrus.Infof("swarm mode already initialised on %s", domainName)
@@ -343,7 +366,7 @@ func initSwarm(c *cli.Context, cl *dockerClient.Client, domainName string) error
 	}
 
 	netOpts := types.NetworkCreate{Driver: "overlay", Scope: "swarm"}
-	if _, err := cl.NetworkCreate(context.Background(), "proxy", netOpts); err != nil {
+	if _, err := cl.NetworkCreate(c.Context, "proxy", netOpts); err != nil {
 		if !strings.Contains(err.Error(), "proxy already exists") {
 			return err
 		}
@@ -365,33 +388,70 @@ func createServerDir(domainName string) error {
 	return nil
 }
 
-var serverAddCommand = cli.Command{
-	Name:    "add",
-	Aliases: []string{"a"},
-	Usage:   "Add a server to your configuration",
+func deployTraefik(c *cli.Context, cl *dockerClient.Client, domainName string) error {
+	internal.NoInput = true
+
+	internal.RecipeName = "traefik"
+	internal.NewAppServer = domainName
+	internal.Domain = fmt.Sprintf("%s.%s", "traefik", domainName)
+	internal.NewAppName = fmt.Sprintf("%s_%s", "traefik", config.SanitiseAppName(domainName))
+
+	appEnvPath := path.Join(config.ABRA_DIR, "servers", internal.Domain, fmt.Sprintf("%s.env", internal.NewAppName))
+	if _, err := os.Stat(appEnvPath); os.IsNotExist(err) {
+		logrus.Info(fmt.Sprintf("-t/--traefik specified, automatically deploying traefik to %s", internal.NewAppServer))
+		if err := internal.NewAction(c); err != nil {
+			logrus.Fatal(err)
+		}
+	} else {
+		logrus.Infof("%s already exists, not creating again", appEnvPath)
+	}
+
+	internal.AppName = internal.NewAppName
+	if err := internal.DeployAction(c); err != nil {
+		logrus.Fatal(err)
+	}
+
+	return nil
+}
+
+var serverAddCommand = &cli.Command{
+	Name:  "add",
+	Usage: "Add a server to your configuration",
 	Description: `
 This command adds a new server to your configuration so that it can be managed
-by Abra. This command can also provision your server ("--provision/-p") with a
-Docker installation so that it is capable of hosting Co-op Cloud apps.
+by Abra. This can be useful when you already have a server provisioned and want
+to start running Abra commands against it. 
 
-Abra will default to expecting that you have a running ssh-agent and are using
-SSH keys to connect to your new server. Abra will also read your SSH config
-(matching "Host" as <domain>). SSH connection details precedence follows as
-such: command-line > SSH config > guessed defaults.
+This command can also provision your server ("--provision/-p") so that it is
+capable of hosting Co-op Cloud apps. Abra will default to expecting that you
+have a running ssh-agent and are using SSH keys to connect to your new server.
+Abra will also read your SSH config (matching "Host" as <domain>). SSH
+connection details precedence follows as such: command-line > SSH config >
+guessed defaults.
 
 If you have no SSH key configured for this host and are instead using password
 authentication, you may pass "--ssh-auth password" to have Abra ask you for the
 password. "--ask-sudo-pass" may be passed if you run your provisioning commands
 via sudo privilege escalation.
 
-The <domain> argument must be a publicy accessible domain name which points to
-your server. You should working SSH access to this server already, Abra will
-assume port 22 and will use your current system username to make an initial
-connection. You can use the <user> and <port> arguments to adjust this.
+If "--local" is passed, then Abra assumes that the current local server is
+intended as the target server. This is useful when you want to have your entire
+Co-op Cloud config located on the server itself, and not on your local
+developer machine.
 
 Example:
 
-    abra server add varia.zone glodemodem 12345 -p
+    abra server add --local
+
+Otherwise, you may specify a remote server. The <domain> argument must be a
+publicy accessible domain name which points to your server. You should have SSH
+access to this server, Abra will assume port 22 and will use your current
+system username to make an initial connection. You can use the <user> and
+<port> arguments to adjust this.
+
+Example:
+
+    abra server add --provision --traefik varia.zone glodemodem 12345
 
 Abra will construct the following SSH connection and Docker context:
 
@@ -399,23 +459,25 @@ Abra will construct the following SSH connection and Docker context:
 
 All communication between Abra and the server will use this SSH connection.
 
-If "--local" is passed, then Abra assumes that the current local server is
-intended as the target server. This is useful when you want to have your entire
-Co-op Cloud config located on the server itself, and not on your local
-developer machine.
+In this example, Abra will run the following operations:
+
+    1. Install Docker
+		2. Initialise Swarm mode
+		3. Deploy Traefik (core web proxy)
+
+You may omit flags to avoid performing this provisioning logic.
 `,
+	Aliases: []string{"a"},
 	Flags: []cli.Flag{
-		internal.DebugFlag,
-		internal.NoInputFlag,
 		localFlag,
 		provisionFlag,
 		sshAuthFlag,
 		askSudoPassFlag,
+		traefikFlag,
 	},
-	Before:    internal.SubCommandBefore,
 	ArgsUsage: "<domain> [<user>] [<port>]",
 	Action: func(c *cli.Context) error {
-		if len(c.Args()) > 0 && local || !internal.ValidateSubCmdFlags(c) {
+		if c.Args().Len() > 0 && local || !internal.ValidateSubCmdFlags(c) {
 			err := errors.New("cannot use <domain> and --local together")
 			internal.ShowSubcommandHelpAndError(c, err)
 		}
@@ -425,8 +487,6 @@ developer machine.
 			internal.ShowSubcommandHelpAndError(c, err)
 		}
 
-		domainName := internal.ValidateDomain(c)
-
 		if local {
 			if err := newLocalServer(c, "default"); err != nil {
 				logrus.Fatal(err)
@@ -434,6 +494,11 @@ developer machine.
 			return nil
 		}
 
+		domainName, err := internal.ValidateDomain(c)
+		if err != nil {
+			logrus.Fatal(err)
+		}
+
 		username := c.Args().Get(1)
 		if username == "" {
 			systemUser, err := user.Current()
@@ -458,17 +523,14 @@ developer machine.
 
 		cl, err := newClient(c, domainName)
 		if err != nil {
-			cleanUp(domainName)
-			logrus.Debugf("failed to construct client for %s, saw %s", domainName, err.Error())
-			logrus.Fatalf(fmt.Sprintf(internal.ServerAddFailMsg, domainName))
+			logrus.Fatal(err)
 		}
 
 		if provision {
 			logrus.Debugf("attempting to construct SSH client for %s", domainName)
 			sshCl, err := ssh.New(domainName, sshAuth, username, port)
 			if err != nil {
-				cleanUp(domainName)
-				logrus.Fatalf(fmt.Sprintf(internal.ServerAddFailMsg, domainName))
+				logrus.Fatal(err)
 			}
 			defer sshCl.Close()
 			logrus.Debugf("successfully created SSH client for %s", domainName)
@@ -481,9 +543,15 @@ developer machine.
 			}
 		}
 
-		if _, err := cl.Info(context.Background()); err != nil {
+		if _, err := cl.Info(c.Context); err != nil {
 			cleanUp(domainName)
-			logrus.Fatalf(fmt.Sprintf(internal.ServerAddFailMsg, domainName))
+			logrus.Fatalf("couldn't make a remote docker connection to %s? use --provision/-p to attempt to install", domainName)
+		}
+
+		if traefik {
+			if err := deployTraefik(c, cl, domainName); err != nil {
+				logrus.Fatal(err)
+			}
 		}
 
 		return nil

cli/server/list.go

@@ -3,23 +3,20 @@ package server
 import (
 	"strings"
 
-	"coopcloud.tech/abra/cli/internal"
+	"coopcloud.tech/abra/cli/formatter"
 	"coopcloud.tech/abra/pkg/config"
 	"coopcloud.tech/abra/pkg/context"
-	"coopcloud.tech/abra/pkg/formatter"
 	"github.com/docker/cli/cli/connhelper/ssh"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
-var serverListCommand = cli.Command{
-	Name:    "list",
-	Aliases: []string{"ls"},
-	Usage:   "List managed servers",
-	Flags: []cli.Flag{
-		internal.DebugFlag,
-	},
-	Before: internal.SubCommandBefore,
+var serverListCommand = &cli.Command{
+	Name:      "list",
+	Aliases:   []string{"ls"},
+	Usage:     "List managed servers",
+	ArgsUsage: " ",
+	HideHelp:  true,
 	Action: func(c *cli.Context) error {
 		dockerContextStore := context.NewDefaultDockerContextStore()
 		contexts, err := dockerContextStore.Store.List()

cli/server/new.go

@@ -1,17 +1,16 @@
 package server
 
 import (
-	"context"
 	"fmt"
 	"strings"
 
+	"coopcloud.tech/abra/cli/formatter"
 	"coopcloud.tech/abra/cli/internal"
-	"coopcloud.tech/abra/pkg/formatter"
 	"coopcloud.tech/libcapsul"
 	"github.com/AlecAivazis/survey/v2"
 	"github.com/hetznercloud/hcloud-go/hcloud"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
 func newHetznerCloudVPS(c *cli.Context) error {
@@ -28,7 +27,7 @@ func newHetznerCloudVPS(c *cli.Context) error {
 			continue
 		}
 
-		sshKey, _, err := client.SSHKey.GetByName(context.Background(), sshKey)
+		sshKey, _, err := client.SSHKey.GetByName(c.Context, sshKey)
 		if err != nil {
 			return err
 		}
@@ -44,18 +43,13 @@ func newHetznerCloudVPS(c *cli.Context) error {
 		Location:   &hcloud.Location{Name: internal.HetznerCloudLocation},
 	}
 
-	sshKeyIDs := strings.Join(sshKeysRaw, "\n")
-	if sshKeyIDs == "" {
-		sshKeyIDs = "N/A (password auth)"
-	}
-
 	tableColumns := []string{"name", "type", "image", "ssh-keys", "location"}
 	table := formatter.CreateTable(tableColumns)
 	table.Append([]string{
 		internal.HetznerCloudName,
 		internal.HetznerCloudType,
 		internal.HetznerCloudImage,
-		sshKeyIDs,
+		strings.Join(sshKeysRaw, "\n"),
 		internal.HetznerCloudLocation,
 	})
 	table.Render()
@@ -73,7 +67,7 @@ func newHetznerCloudVPS(c *cli.Context) error {
 		logrus.Fatal("exiting as requested")
 	}
 
-	res, _, err := client.Server.Create(context.Background(), serverOpts)
+	res, _, err := client.Server.Create(c.Context, serverOpts)
 	if err != nil {
 		return err
 	}
@@ -111,6 +105,9 @@ bar.example.com).
     @  1800 IN A %s
     *  1800 IN A %s
 
+"abra record new --auto" can help you do this quickly if you use a supported
+DNS provider.
+
 	`,
 		internal.HetznerCloudName, ip, rootPassword,
 		ip, ip, ip,
@@ -198,7 +195,7 @@ bar.example.com).
 	return nil
 }
 
-var serverNewCommand = cli.Command{
+var serverNewCommand = &cli.Command{
 	Name:    "new",
 	Aliases: []string{"n"},
 	Usage:   "Create a new server using a 3rd party provider",
@@ -220,9 +217,8 @@ API tokens are read from the environment if specified, e.g.
 
 Where "$provider_TOKEN" is the expected env var format.
 `,
+	ArgsUsage: "<provider>",
 	Flags: []cli.Flag{
-		internal.DebugFlag,
-		internal.NoInputFlag,
 		internal.ServerProviderFlag,
 
 		// Capsul
@@ -240,7 +236,6 @@ Where "$provider_TOKEN" is the expected env var format.
 		internal.HetznerCloudLocationFlag,
 		internal.HetznerCloudAPITokenFlag,
 	},
-	Before: internal.SubCommandBefore,
 	Action: func(c *cli.Context) error {
 		if err := internal.EnsureServerProvider(); err != nil {
 			logrus.Fatal(err)

cli/server/remove.go

@@ -1,24 +1,25 @@
 package server
 
 import (
-	"context"
 	"fmt"
 	"os"
 	"path/filepath"
 
+	"coopcloud.tech/abra/cli/formatter"
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/client"
 	"coopcloud.tech/abra/pkg/config"
-	"coopcloud.tech/abra/pkg/formatter"
 	"github.com/AlecAivazis/survey/v2"
 	"github.com/hetznercloud/hcloud-go/hcloud"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
 var rmServer bool
 var rmServerFlag = &cli.BoolFlag{
-	Name:        "server, s",
+	Name:        "server",
+	Aliases:     []string{"s"},
+	Value:       false,
 	Usage:       "remove the actual server also",
 	Destination: &rmServer,
 }
@@ -49,7 +50,7 @@ func rmHetznerCloudVPS(c *cli.Context) error {
 
 	client := hcloud.NewClient(hcloud.WithToken(internal.HetznerCloudAPIToken))
 
-	server, _, err := client.Server.Get(context.Background(), internal.HetznerCloudName)
+	server, _, err := client.Server.Get(c.Context, internal.HetznerCloudName)
 	if err != nil {
 		return err
 	}
@@ -88,7 +89,7 @@ destroyed.
 		logrus.Fatal("exiting as requested")
 	}
 
-	_, err = client.Server.Delete(context.Background(), server)
+	_, err = client.Server.Delete(c.Context, server)
 	if err != nil {
 		return err
 	}
@@ -98,10 +99,10 @@ destroyed.
 	return nil
 }
 
-var serverRemoveCommand = cli.Command{
+var serverRemoveCommand = &cli.Command{
 	Name:      "remove",
 	Aliases:   []string{"rm"},
-	ArgsUsage: "[<server>]",
+	ArgsUsage: "<server>",
 	Usage:     "Remove a managed server",
 	Description: `
 This command removes a server from Abra management.
@@ -115,35 +116,24 @@ underlying client connection context. This server will then be lost in time,
 like tears in rain.
 `,
 	Flags: []cli.Flag{
-		internal.DebugFlag,
-		internal.NoInputFlag,
 		rmServerFlag,
-		internal.ServerProviderFlag,
 
 		// Hetzner
 		internal.HetznerCloudNameFlag,
 		internal.HetznerCloudAPITokenFlag,
 	},
-	Before: internal.SubCommandBefore,
 	Action: func(c *cli.Context) error {
-		serverName := internal.ValidateServer(c)
-
-		warnMsg := `Did not pass -s/--server for actual server deletion, prompting!
-
-Abra doesn't currently know if it helped you create this server with one of the
-3rd party integrations (e.g. Capsul). You have a choice here to actually,
-really and finally destroy this server using those integrations. If you want to
-do this, choose Yes.
-
-If you just want to remove the server config files & context, choose No.
-`
+		serverName, err := internal.ValidateServer(c)
+		if err != nil {
+			logrus.Fatal(err)
+		}
 
 		if !rmServer {
-			logrus.Warn(fmt.Sprintf(warnMsg))
+			logrus.Warn("did not pass -s/--server for actual server deletion, prompting")
 
 			response := false
 			prompt := &survey.Confirm{
-				Message: "delete actual live server?",
+				Message: "prompt to actual server deletion?",
 			}
 			if err := survey.AskOne(prompt, &response); err != nil {
 				logrus.Fatal(err)
@@ -167,17 +157,18 @@ If you just want to remove the server config files & context, choose No.
 					logrus.Fatal(err)
 				}
 			}
+
 		}
 
 		if err := client.DeleteContext(serverName); err != nil {
 			logrus.Fatal(err)
 		}
 
-		if err := os.RemoveAll(filepath.Join(config.SERVERS_DIR, serverName)); err != nil {
+		if err := os.RemoveAll(filepath.Join(config.ABRA_SERVER_FOLDER, serverName)); err != nil {
 			logrus.Fatal(err)
 		}
 
-		logrus.Infof("server at %s has been lost in time, like tears in rain", serverName)
+		logrus.Infof("server at '%s' has been lost in time, like tears in rain", serverName)
 
 		return nil
 	},

cli/server/server.go

@@ -1,14 +1,14 @@
 package server
 
 import (
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
 // ServerCommand defines the `abra server` command and its subcommands
-var ServerCommand = cli.Command{
+var ServerCommand = &cli.Command{
 	Name:    "server",
 	Aliases: []string{"s"},
-	Usage:   "Manage servers",
+	Usage:   "Manage servers via 3rd party providers",
 	Description: `
 These commands support creating, managing and removing servers using 3rd party
 integrations. 
@@ -18,7 +18,7 @@ already have a server, you can add it to your configuration using "abra server
 add". Abra can provision servers so that they are ready to deploy Co-op Cloud
 apps, see available flags on "server add" for more.
 `,
-	Subcommands: []cli.Command{
+	Subcommands: []*cli.Command{
 		serverNewCommand,
 		serverAddCommand,
 		serverListCommand,

cli/upgrade.go

@@ -0,0 +1,44 @@
+package cli
+
+import (
+	"fmt"
+	"os/exec"
+
+	"coopcloud.tech/abra/cli/internal"
+	"github.com/sirupsen/logrus"
+	"github.com/urfave/cli/v2"
+)
+
+var mainURL = "https://install.abra.coopcloud.tech"
+
+var releaseCandidateURL = "https://git.coopcloud.tech/coop-cloud/abra/raw/branch/main/scripts/installer/installer"
+
+// UpgradeCommand upgrades abra in-place.
+var UpgradeCommand = &cli.Command{
+	Name:  "upgrade",
+	Usage: "Upgrade Abra",
+	Description: `
+This command allows you to upgrade Abra in-place with the latest stable or
+release candidate.
+
+If you would like to install the latest release candidate, please pass the
+"--rc" option. Please bear in mind that the latest release candidate may have
+some catastrophic bugs contained in it. In any case, thank you very much for
+testing efforts!
+`,
+	Flags: []cli.Flag{internal.RCFlag},
+	Action: func(c *cli.Context) error {
+		cmd := exec.Command("bash", "-c", fmt.Sprintf("curl -s %s | bash", mainURL))
+		if internal.RC {
+			cmd = exec.Command("bash", "-c", fmt.Sprintf("curl -s %s | bash -s -- --rc", releaseCandidateURL))
+		}
+
+		logrus.Debugf("attempting to run '%s'", cmd)
+
+		if err := internal.RunCmd(cmd); err != nil {
+			logrus.Fatal(err)
+		}
+
+		return nil
+	},
+}

cmd/abra/main.go

@@ -5,10 +5,10 @@ import (
 	"coopcloud.tech/abra/cli"
 )
 
-// Version is the current version of Abra
+// Version is the current version of abra.
 var Version string
 
-// Commit is the current git commit of Abra
+// Commit is the current commit of abra.
 var Commit string
 
 func main() {

go.mod

@@ -4,32 +4,30 @@ go 1.16
 
 require (
 	coopcloud.tech/tagcmp v0.0.0-20211103052201-885b22f77d52
-	github.com/AlecAivazis/survey/v2 v2.3.4
-	github.com/Autonomic-Cooperative/godotenv v1.3.1-0.20210731094149-b031ea1211e7
+	github.com/AlecAivazis/survey/v2 v2.3.2
+	github.com/Autonomic-Cooperative/godotenv v1.3.1-0.20210731170023-c37c0920d1a4
 	github.com/Gurpartap/logrus-stack v0.0.0-20170710170904-89c00d8a28f4
-	github.com/docker/cli v20.10.14+incompatible
-	github.com/docker/distribution v2.8.1+incompatible
-	github.com/docker/docker v20.10.14+incompatible
+	github.com/docker/cli v20.10.12+incompatible
+	github.com/docker/distribution v2.7.1+incompatible
+	github.com/docker/docker v20.10.12+incompatible
 	github.com/docker/go-units v0.4.0
 	github.com/go-git/go-git/v5 v5.4.2
 	github.com/hetznercloud/hcloud-go v1.33.1
-	github.com/moby/sys/signal v0.7.0
+	github.com/moby/sys/signal v0.6.0
 	github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6
 	github.com/olekukonko/tablewriter v0.0.5
 	github.com/pkg/errors v0.9.1
-	github.com/schollz/progressbar/v3 v3.8.6
+	github.com/schollz/progressbar/v3 v3.8.5
 	github.com/schultz-is/passgen v1.0.1
 	github.com/sirupsen/logrus v1.8.1
-	gotest.tools/v3 v3.1.0
+	github.com/urfave/cli/v2 v2.3.0
+	gotest.tools/v3 v3.0.3
 )
 
 require (
 	coopcloud.tech/libcapsul v0.0.0-20211022074848-c35e78fe3f3e
-	github.com/ProtonMail/go-crypto v0.0.0-20211112122917-428f8eabeeb3 // indirect
-	github.com/buger/goterm v1.0.4
-	github.com/containerd/containerd v1.5.9 // indirect
-	github.com/containers/image v3.0.2+incompatible
-	github.com/containers/storage v1.38.2 // indirect
+	github.com/Microsoft/hcsshim v0.8.21 // indirect
+	github.com/containerd/containerd v1.5.5 // indirect
 	github.com/docker/docker-credential-helpers v0.6.4 // indirect
 	github.com/facebookgo/stack v0.0.0-20160209184415-751773369052 // indirect
 	github.com/fvbommel/sortorder v1.0.2 // indirect
@@ -41,13 +39,10 @@ require (
 	github.com/libdns/gandi v1.0.2
 	github.com/libdns/libdns v0.2.1
 	github.com/moby/sys/mount v0.2.0 // indirect
-	github.com/opencontainers/image-spec v1.0.3-0.20211202193544-a5463b7f9c84 // indirect
-	github.com/sergi/go-diff v1.2.0 // indirect
-	github.com/spf13/cobra v1.3.0 // indirect
+	github.com/morikuni/aec v1.0.0 // indirect
+	github.com/opencontainers/runc v1.0.2 // indirect
 	github.com/theupdateframework/notary v0.7.0 // indirect
-	github.com/urfave/cli v1.22.5
-	github.com/xanzy/ssh-agent v0.3.1 // indirect
-	github.com/xeipuuv/gojsonpointer v0.0.0-20190809123943-df4f5c81cb3b // indirect
-	golang.org/x/crypto v0.0.0-20220131195533-30dcbda58838
-	golang.org/x/sys v0.0.0-20220128215802-99c3d69c2c27
+	github.com/xeipuuv/gojsonschema v1.2.0 // indirect
+	golang.org/x/crypto v0.0.0-20211215153901-e495a2d5b3d3
+	golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e
 )

pkg/app/app.go

@@ -23,7 +23,7 @@ func Get(appName string) (config.App, error) {
 		return config.App{}, err
 	}
 
-	logrus.Debugf("retrieved %s for %s", app, appName)
+	logrus.Debugf("retrieved '%s' for '%s'", app, appName)
 
 	return app, nil
 }

pkg/autocomplete/autocomplete.go

@@ -3,10 +3,10 @@ package autocomplete
 import (
 	"fmt"
 
+	"coopcloud.tech/abra/pkg/catalogue"
 	"coopcloud.tech/abra/pkg/config"
-	"coopcloud.tech/abra/pkg/recipe"
 	"github.com/sirupsen/logrus"
-	"github.com/urfave/cli"
+	"github.com/urfave/cli/v2"
 )
 
 // AppNameComplete copletes app names
@@ -27,7 +27,7 @@ func AppNameComplete(c *cli.Context) {
 
 // RecipeNameComplete completes recipe names
 func RecipeNameComplete(c *cli.Context) {
-	catl, err := recipe.ReadRecipeCatalogue()
+	catl, err := catalogue.ReadRecipeCatalogue()
 	if err != nil {
 		logrus.Warn(err)
 	}
@@ -40,24 +40,3 @@ func RecipeNameComplete(c *cli.Context) {
 		fmt.Println(name)
 	}
 }
-
-// SubcommandComplete completes subcommands.
-func SubcommandComplete(c *cli.Context) {
-	if c.NArg() > 0 {
-		return
-	}
-
-	subcmds := []string{
-		"app",
-		"autocomplete",
-		"catalogue",
-		"recipe",
-		"record",
-		"server",
-		"upgrade",
-	}
-
-	for _, cmd := range subcmds {
-		fmt.Println(cmd)
-	}
-}

pkg/catalogue/catalogue.go

@@ -0,0 +1,642 @@
+// Package catalogue provides ways of interacting with recipe catalogues which
+// are JSON data structures which contain meta information about recipes (e.g.
+// what versions of the Nextcloud recipe are available?).
+package catalogue
+
+import (
+	"encoding/json"
+	"fmt"
+	"io/ioutil"
+	"os"
+	"path"
+	"strings"
+	"time"
+
+	"coopcloud.tech/abra/cli/formatter"
+	"coopcloud.tech/abra/pkg/client"
+	"coopcloud.tech/abra/pkg/config"
+	"coopcloud.tech/abra/pkg/recipe"
+	"coopcloud.tech/abra/pkg/web"
+	"github.com/docker/distribution/reference"
+	"github.com/go-git/go-git/v5"
+	"github.com/go-git/go-git/v5/plumbing"
+	"github.com/sirupsen/logrus"
+)
+
+// RecipeCatalogueURL is the only current recipe catalogue available.
+const RecipeCatalogueURL = "https://apps.coopcloud.tech"
+
+// ReposMetadataURL is the recipe repository metadata
+const ReposMetadataURL = "https://git.coopcloud.tech/api/v1/orgs/coop-cloud/repos"
+
+// image represents a recipe container image.
+type image struct {
+	Image  string `json:"image"`
+	Rating string `json:"rating"`
+	Source string `json:"source"`
+	URL    string `json:"url"`
+}
+
+// features represent what top-level features a recipe supports (e.g. does this
+// recipe support backups?).
+type features struct {
+	Backups     string `json:"backups"`
+	Email       string `json:"email"`
+	Healthcheck string `json:"healthcheck"`
+	Image       image  `json:"image"`
+	Status      int    `json:"status"`
+	Tests       string `json:"tests"`
+	SSO         string `json:"sso"`
+}
+
+// tag represents a git tag.
+type tag = string
+
+// service represents a service within a recipe.
+type service = string
+
+// ServiceMeta represents meta info associated with a service.
+type ServiceMeta struct {
+	Digest string `json:"digest"`
+	Image  string `json:"image"`
+	Tag    string `json:"tag"`
+}
+
+// RecipeVersions are the versions associated with a recipe.
+type RecipeVersions []map[tag]map[service]ServiceMeta
+
+// RecipeMeta represents metadata for a recipe in the abra catalogue.
+type RecipeMeta struct {
+	Category      string         `json:"category"`
+	DefaultBranch string         `json:"default_branch"`
+	Description   string         `json:"description"`
+	Features      features       `json:"features"`
+	Icon          string         `json:"icon"`
+	Name          string         `json:"name"`
+	Repository    string         `json:"repository"`
+	Versions      RecipeVersions `json:"versions"`
+	Website       string         `json:"website"`
+}
+
+// LatestVersion returns the latest version of a recipe.
+func (r RecipeMeta) LatestVersion() string {
+	var version string
+
+	// apps.json versions are sorted so the last key is latest
+	latest := r.Versions[len(r.Versions)-1]
+
+	for tag := range latest {
+		version = tag
+	}
+
+	logrus.Debugf("choosing %s as latest version of %s", version, r.Name)
+
+	return version
+}
+
+// Name represents a recipe name.
+type Name = string
+
+// RecipeCatalogue represents the entire recipe catalogue.
+type RecipeCatalogue map[Name]RecipeMeta
+
+// Flatten converts AppCatalogue to slice
+func (r RecipeCatalogue) Flatten() []RecipeMeta {
+	recipes := make([]RecipeMeta, 0, len(r))
+
+	for name := range r {
+		recipes = append(recipes, r[name])
+	}
+
+	return recipes
+}
+
+// ByRecipeName sorts recipes by name.
+type ByRecipeName []RecipeMeta
+
+func (r ByRecipeName) Len() int      { return len(r) }
+func (r ByRecipeName) Swap(i, j int) { r[i], r[j] = r[j], r[i] }
+func (r ByRecipeName) Less(i, j int) bool {
+	return strings.ToLower(r[i].Name) < strings.ToLower(r[j].Name)
+}
+
+// recipeCatalogueFSIsLatest checks whether the recipe catalogue stored locally
+// is up to date.
+func recipeCatalogueFSIsLatest() (bool, error) {
+	httpClient := web.NewHTTPRetryClient()
+	res, err := httpClient.Head(RecipeCatalogueURL)
+	if err != nil {
+		return false, err
+	}
+
+	lastModified := res.Header["Last-Modified"][0]
+	parsed, err := time.Parse(time.RFC1123, lastModified)
+	if err != nil {
+		return false, err
+	}
+
+	info, err := os.Stat(config.APPS_JSON)
+	if err != nil {
+		if os.IsNotExist(err) {
+			logrus.Debugf("no recipe catalogue found in file system cache")
+			return false, nil
+		}
+		return false, err
+	}
+
+	localModifiedTime := info.ModTime().Unix()
+	remoteModifiedTime := parsed.Unix()
+
+	if localModifiedTime < remoteModifiedTime {
+		logrus.Debug("file system cached recipe catalogue is out-of-date")
+		return false, nil
+	}
+
+	logrus.Debug("file system cached recipe catalogue is now up-to-date")
+
+	return true, nil
+}
+
+// ReadRecipeCatalogue reads the recipe catalogue.
+func ReadRecipeCatalogue() (RecipeCatalogue, error) {
+	recipes := make(RecipeCatalogue)
+
+	recipeFSIsLatest, err := recipeCatalogueFSIsLatest()
+	if err != nil {
+		return nil, err
+	}
+
+	if !recipeFSIsLatest {
+		logrus.Debugf("reading recipe catalogue from web to get latest")
+		if err := readRecipeCatalogueWeb(&recipes); err != nil {
+			return nil, err
+		}
+		return recipes, nil
+	}
+
+	logrus.Debugf("reading recipe catalogue from file system cache to get latest")
+	if err := readRecipeCatalogueFS(&recipes); err != nil {
+		return nil, err
+	}
+
+	return recipes, nil
+}
+
+// readRecipeCatalogueFS reads the catalogue from the file system.
+func readRecipeCatalogueFS(target interface{}) error {
+	recipesJSONFS, err := ioutil.ReadFile(config.APPS_JSON)
+	if err != nil {
+		return err
+	}
+
+	if err := json.Unmarshal(recipesJSONFS, &target); err != nil {
+		return err
+	}
+
+	logrus.Debugf("read recipe catalogue from file system cache in %s", config.APPS_JSON)
+
+	return nil
+}
+
+// readRecipeCatalogueWeb reads the catalogue from the web.
+func readRecipeCatalogueWeb(target interface{}) error {
+	if err := web.ReadJSON(RecipeCatalogueURL, &target); err != nil {
+		return err
+	}
+
+	recipesJSON, err := json.MarshalIndent(target, "", "    ")
+	if err != nil {
+		return err
+	}
+
+	if err := ioutil.WriteFile(config.APPS_JSON, recipesJSON, 0764); err != nil {
+		return err
+	}
+
+	logrus.Debugf("read recipe catalogue from web at %s", RecipeCatalogueURL)
+
+	return nil
+}
+
+// VersionsOfService lists the version of a service.
+func VersionsOfService(recipe, serviceName string) ([]string, error) {
+	var versions []string
+
+	catalogue, err := ReadRecipeCatalogue()
+	if err != nil {
+		return nil, err
+	}
+
+	rec, ok := catalogue[recipe]
+	if !ok {
+		return versions, nil
+	}
+
+	alreadySeen := make(map[string]bool)
+	for _, serviceVersion := range rec.Versions {
+		for tag := range serviceVersion {
+			if _, ok := alreadySeen[tag]; !ok {
+				alreadySeen[tag] = true
+				versions = append(versions, tag)
+			}
+		}
+	}
+
+	logrus.Debugf("detected versions %s for %s", strings.Join(versions, ", "), recipe)
+
+	return versions, nil
+}
+
+// GetRecipeMeta retrieves the recipe metadata from the recipe catalogue.
+func GetRecipeMeta(recipeName string) (RecipeMeta, error) {
+	catl, err := ReadRecipeCatalogue()
+	if err != nil {
+		return RecipeMeta{}, err
+	}
+
+	recipeMeta, ok := catl[recipeName]
+	if !ok {
+		err := fmt.Errorf("recipe %s does not exist?", recipeName)
+		return RecipeMeta{}, err
+	}
+
+	if err := recipe.EnsureExists(recipeName); err != nil {
+		return RecipeMeta{}, err
+	}
+
+	logrus.Debugf("recipe metadata retrieved for %s", recipeName)
+
+	return recipeMeta, nil
+}
+
+// RepoMeta is a single recipe repo metadata.
+type RepoMeta struct {
+	ID                        int `json:"id"`
+	Owner                     Owner
+	Name                      string      `json:"name"`
+	FullName                  string      `json:"full_name"`
+	Description               string      `json:"description"`
+	Empty                     bool        `json:"empty"`
+	Private                   bool        `json:"private"`
+	Fork                      bool        `json:"fork"`
+	Template                  bool        `json:"template"`
+	Parent                    interface{} `json:"parent"`
+	Mirror                    bool        `json:"mirror"`
+	Size                      int         `json:"size"`
+	HTMLURL                   string      `json:"html_url"`
+	SSHURL                    string      `json:"ssh_url"`
+	CloneURL                  string      `json:"clone_url"`
+	OriginalURL               string      `json:"original_url"`
+	Website                   string      `json:"website"`
+	StarsCount                int         `json:"stars_count"`
+	ForksCount                int         `json:"forks_count"`
+	WatchersCount             int         `json:"watchers_count"`
+	OpenIssuesCount           int         `json:"open_issues_count"`
+	OpenPRCount               int         `json:"open_pr_counter"`
+	ReleaseCounter            int         `json:"release_counter"`
+	DefaultBranch             string      `json:"default_branch"`
+	Archived                  bool        `json:"archived"`
+	CreatedAt                 string      `json:"created_at"`
+	UpdatedAt                 string      `json:"updated_at"`
+	Permissions               Permissions
+	HasIssues                 bool `json:"has_issues"`
+	InternalTracker           InternalTracker
+	HasWiki                   bool   `json:"has_wiki"`
+	HasPullRequests           bool   `json:"has_pull_requests"`
+	HasProjects               bool   `json:"has_projects"`
+	IgnoreWhitespaceConflicts bool   `json:"ignore_whitespace_conflicts"`
+	AllowMergeCommits         bool   `json:"allow_merge_commits"`
+	AllowRebase               bool   `json:"allow_rebase"`
+	AllowRebaseExplicit       bool   `json:"allow_rebase_explicit"`
+	AllowSquashMerge          bool   `json:"allow_squash_merge"`
+	AvatarURL                 string `json:"avatar_url"`
+	Internal                  bool   `json:"internal"`
+	MirrorInterval            string `json:"mirror_interval"`
+}
+
+// Owner is the repo organisation owner metadata.
+type Owner struct {
+	ID         int    `json:"id"`
+	Login      string `json:"login"`
+	FullName   string `json:"full_name"`
+	Email      string `json:"email"`
+	AvatarURL  string `json:"avatar_url"`
+	Language   string `json:"language"`
+	IsAdmin    bool   `json:"is_admin"`
+	LastLogin  string `json:"last_login"`
+	Created    string `json:"created"`
+	Restricted bool   `json:"restricted"`
+	Username   string `json:"username"`
+}
+
+// Permissions is perms metadata for a repo.
+type Permissions struct {
+	Admin bool `json:"admin"`
+	Push  bool `json:"push"`
+	Pull  bool `json:"pull"`
+}
+
+// InternalTracker is issue tracker metadata for a repo.
+type InternalTracker struct {
+	EnableTimeTracker                bool `json:"enable_time_tracker"`
+	AllowOnlyContributorsToTrackTime bool `json:"allow_only_contributors_to_track_time"`
+	EnableIssuesDependencies         bool `json:"enable_issue_dependencies"`
+}
+
+// RepoCatalogue represents all the recipe repo metadata.
+type RepoCatalogue map[string]RepoMeta
+
+// ReadReposMetadata retrieves coop-cloud/... repo metadata from Gitea.
+func ReadReposMetadata() (RepoCatalogue, error) {
+	reposMeta := make(RepoCatalogue)
+
+	pageIdx := 1
+	bar := formatter.CreateProgressbar(-1, "retrieving recipe repos list from git.coopcloud.tech...")
+	for {
+		var reposList []RepoMeta
+
+		pagedURL := fmt.Sprintf("%s?page=%v", ReposMetadataURL, pageIdx)
+
+		logrus.Debugf("fetching repo metadata from %s", pagedURL)
+
+		if err := web.ReadJSON(pagedURL, &reposList); err != nil {
+			return reposMeta, err
+		}
+
+		if len(reposList) == 0 {
+			bar.Add(1)
+			break
+		}
+
+		for idx, repo := range reposList {
+			reposMeta[repo.Name] = reposList[idx]
+		}
+
+		pageIdx++
+		bar.Add(1)
+	}
+
+	return reposMeta, nil
+}
+
+func GetStringInBetween(str, start, end string) (result string, err error) {
+	// GetStringInBetween returns empty string if no start or end string found
+	s := strings.Index(str, start)
+	if s == -1 {
+		return "", fmt.Errorf("marker string '%s' not found", start)
+	}
+	s += len(start)
+	e := strings.Index(str[s:], end)
+	if e == -1 {
+		return "", fmt.Errorf("end marker '%s' not found", end)
+	}
+	return str[s : s+e], nil
+}
+
+func GetImageMetadata(imageRowString, recipeName string) (image, error) {
+	img := image{}
+
+	imgFields := strings.Split(imageRowString, ",")
+
+	for i, elem := range imgFields {
+		imgFields[i] = strings.TrimSpace(elem)
+	}
+
+	if len(imgFields) < 3 {
+		if imageRowString != "" {
+			logrus.Warnf("%s image meta has incorrect format: %s", recipeName, imageRowString)
+		} else {
+			logrus.Warnf("%s image meta is empty?", recipeName)
+		}
+		return img, nil
+	}
+
+	img.Rating = imgFields[1]
+	img.Source = imgFields[2]
+
+	imgString := imgFields[0]
+
+	imageName, err := GetStringInBetween(imgString, "[", "]")
+	if err != nil {
+		logrus.Fatal(err)
+	}
+	img.Image = strings.ReplaceAll(imageName, "`", "")
+
+	imageURL, err := GetStringInBetween(imgString, "(", ")")
+	if err != nil {
+		logrus.Fatal(err)
+	}
+	img.URL = imageURL
+
+	return img, nil
+}
+
+func GetRecipeFeaturesAndCategory(recipeName string) (features, string, error) {
+	feat := features{}
+
+	var category string
+
+	readmePath := path.Join(config.ABRA_DIR, "apps", recipeName, "README.md")
+
+	logrus.Debugf("attempting to open %s for recipe metadata parsing", readmePath)
+
+	readmeFS, err := ioutil.ReadFile(readmePath)
+	if err != nil {
+		return feat, category, err
+	}
+
+	readmeMetadata, err := GetStringInBetween( // Find text between delimiters
+		string(readmeFS),
+		"<!-- metadata -->", "<!-- endmetadata -->",
+	)
+	if err != nil {
+		return feat, category, err
+	}
+
+	readmeLines := strings.Split( // Array item from lines
+		strings.ReplaceAll( // Remove \t tabs
+			readmeMetadata, "\t", "",
+		),
+		"\n")
+
+	for _, val := range readmeLines {
+		if strings.Contains(val, "**Category**") {
+			category = strings.TrimSpace(
+				strings.TrimPrefix(val, "* **Category**:"),
+			)
+		}
+		if strings.Contains(val, "**Backups**") {
+			feat.Backups = strings.TrimSpace(
+				strings.TrimPrefix(val, "* **Backups**:"),
+			)
+		}
+		if strings.Contains(val, "**Email**") {
+			feat.Email = strings.TrimSpace(
+				strings.TrimPrefix(val, "* **Email**:"),
+			)
+		}
+		if strings.Contains(val, "**SSO**") {
+			feat.SSO = strings.TrimSpace(
+				strings.TrimPrefix(val, "* **SSO**:"),
+			)
+		}
+		if strings.Contains(val, "**Healthcheck**") {
+			feat.Healthcheck = strings.TrimSpace(
+				strings.TrimPrefix(val, "* **Healthcheck**:"),
+			)
+		}
+		if strings.Contains(val, "**Tests**") {
+			feat.Tests = strings.TrimSpace(
+				strings.TrimPrefix(val, "* **Tests**:"),
+			)
+		}
+		if strings.Contains(val, "**Image**") {
+			imageMetadata, err := GetImageMetadata(strings.TrimSpace(
+				strings.TrimPrefix(val, "* **Image**:"),
+			), recipeName)
+			if err != nil {
+				continue
+			}
+			feat.Image = imageMetadata
+		}
+	}
+
+	return feat, category, nil
+}
+
+// GetRecipeVersions retrieves all recipe versions.
+func GetRecipeVersions(recipeName string) (RecipeVersions, error) {
+	versions := RecipeVersions{}
+
+	recipeDir := path.Join(config.ABRA_DIR, "apps", recipeName)
+
+	logrus.Debugf("attempting to open git repository in %s", recipeDir)
+
+	repo, err := git.PlainOpen(recipeDir)
+	if err != nil {
+		return versions, err
+	}
+
+	worktree, err := repo.Worktree()
+	if err != nil {
+		logrus.Fatal(err)
+	}
+
+	gitTags, err := repo.Tags()
+	if err != nil {
+		return versions, err
+	}
+
+	if err := gitTags.ForEach(func(ref *plumbing.Reference) (err error) {
+		tag := strings.TrimPrefix(string(ref.Name()), "refs/tags/")
+
+		logrus.Debugf("processing %s for %s", tag, recipeName)
+
+		checkOutOpts := &git.CheckoutOptions{
+			Create: false,
+			Force:  true,
+			Branch: plumbing.ReferenceName(ref.Name()),
+		}
+		if err := worktree.Checkout(checkOutOpts); err != nil {
+			logrus.Debugf("failed to check out %s in %s", tag, recipeDir)
+			return err
+		}
+
+		logrus.Debugf("successfully checked out %s in %s", ref.Name(), recipeDir)
+
+		recipe, err := recipe.Get(recipeName)
+		if err != nil {
+			return err
+		}
+
+		cl, err := client.New("default") // only required for docker.io registry calls
+		if err != nil {
+			logrus.Fatal(err)
+		}
+
+		versionMeta := make(map[string]ServiceMeta)
+		for _, service := range recipe.Config.Services {
+
+			img, err := reference.ParseNormalizedNamed(service.Image)
+			if err != nil {
+				return err
+			}
+
+			path := reference.Path(img)
+			if strings.Contains(path, "library") {
+				path = strings.Split(path, "/")[1]
+			}
+
+			var tag string
+			switch img.(type) {
+			case reference.NamedTagged:
+				tag = img.(reference.NamedTagged).Tag()
+			case reference.Named:
+				logrus.Warnf("%s service is missing image tag?", path)
+				continue
+			}
+
+			logrus.Debugf("looking up image: %s from %s", img, path)
+
+			digest, err := client.GetTagDigest(cl, img)
+			if err != nil {
+				logrus.Warn(err)
+				continue
+			}
+
+			versionMeta[service.Name] = ServiceMeta{
+				Digest: digest,
+				Image:  path,
+				Tag:    img.(reference.NamedTagged).Tag(),
+			}
+
+			logrus.Debugf("collecting digest: %s, image: %s, tag: %s", digest, path, tag)
+		}
+
+		versions = append(versions, map[string]map[string]ServiceMeta{tag: versionMeta})
+
+		return nil
+	}); err != nil {
+		return versions, err
+	}
+
+	branch := "master"
+	if _, err := repo.Branch("master"); err != nil {
+		if _, err := repo.Branch("main"); err != nil {
+			logrus.Debugf("failed to select branch in %s", recipeDir)
+			logrus.Fatal(err)
+		}
+		branch = "main"
+	}
+
+	refName := fmt.Sprintf("refs/heads/%s", branch)
+	checkOutOpts := &git.CheckoutOptions{
+		Create: false,
+		Force:  true,
+		Branch: plumbing.ReferenceName(refName),
+	}
+	if err := worktree.Checkout(checkOutOpts); err != nil {
+		logrus.Debugf("failed to check out %s in %s", branch, recipeDir)
+		logrus.Fatal(err)
+	}
+
+	logrus.Debugf("switched back to %s in %s", branch, recipeDir)
+	logrus.Debugf("collected %s for %s", versions, recipeName)
+
+	return versions, nil
+}
+
+// GetRecipeCatalogueVersions list the recipe versions listed in the recipe catalogue.
+func GetRecipeCatalogueVersions(recipeName string, catl RecipeCatalogue) ([]string, error) {
+	var versions []string
+
+	if recipeMeta, exists := catl[recipeName]; exists {
+		for _, versionMeta := range recipeMeta.Versions {
+			for tag := range versionMeta {
+				versions = append(versions, tag)
+			}
+		}
+	}
+
+	return versions, nil
+}

pkg/client/client.go

@@ -27,11 +27,7 @@ func New(contextName string) (*client.Client, error) {
 			return nil, err
 		}
 
-		helper, err := commandconnPkg.NewConnectionHelper(ctxEndpoint)
-		if err != nil {
-			return nil, err
-		}
-
+		helper := commandconnPkg.NewConnectionHelper(ctxEndpoint)
 		httpClient := &http.Client{
 			// No tls, no proxy
 			Transport: &http.Transport{
@@ -59,7 +55,7 @@ func New(contextName string) (*client.Client, error) {
 		return nil, err
 	}
 
-	logrus.Debugf("created client for %s", contextName)
+	logrus.Debugf("created client for '%s'", contextName)
 
 	return cl, nil
 }

pkg/client/context.go

@@ -26,7 +26,7 @@ func CreateContext(contextName string, user string, port string) error {
 	if err := createContext(contextName, host); err != nil {
 		return err
 	}
-	logrus.Debugf("created the %s context", contextName)
+	logrus.Debugf("created the '%s' context", contextName)
 	return nil
 }
 
@@ -72,6 +72,8 @@ func DeleteContext(name string) error {
 		return err
 	}
 
+	// remove any context that might be loaded
+	// TODO: Check if the context we are removing is the active one rather than doing it all the time
 	cfg := dConfig.LoadDefaultConfigFile(nil)
 	cfg.CurrentContext = ""
 	if err := cfg.Save(); err != nil {

pkg/client/registry.go

@@ -1,57 +1,189 @@
 package client
 
 import (
-	"context"
+	"encoding/json"
 	"fmt"
+	"io/ioutil"
+	"net/http"
+	"os"
 	"strings"
 
-	"github.com/containers/image/docker"
-	"github.com/containers/image/types"
+	"coopcloud.tech/abra/pkg/web"
 	"github.com/docker/distribution/reference"
 	"github.com/docker/docker/client"
+	"github.com/hashicorp/go-retryablehttp"
 	"github.com/sirupsen/logrus"
 )
 
-// GetRegistryTags retrieves all tags of an image from a container registry.
-func GetRegistryTags(img reference.Named) ([]string, error) {
-	var tags []string
+type RawTag struct {
+	Layer string
+	Name  string
+}
 
-	ref, err := docker.ParseReference(fmt.Sprintf("//%s", img))
-	if err != nil {
-		return tags, fmt.Errorf("failed to parse image %s, saw: %s", img, err.Error())
-	}
+type RawTags []RawTag
 
-	ctx := context.Background()
-	tags, err = docker.GetRepositoryTags(ctx, &types.SystemContext{}, ref)
-	if err != nil {
+var registryURL = "https://registry.hub.docker.com/v1/repositories/%s/tags"
+
+func GetRegistryTags(image string) (RawTags, error) {
+	var tags RawTags
+
+	tagsUrl := fmt.Sprintf(registryURL, image)
+	if err := web.ReadJSON(tagsUrl, &tags); err != nil {
 		return tags, err
 	}
 
 	return tags, nil
 }
 
-// GetTagDigest retrieves an image digest from a container registry.
+// getRegv2Token retrieves a registry v2 authentication token.
+func getRegv2Token(cl *client.Client, image reference.Named) (string, error) {
+	img := reference.Path(image)
+	tokenURL := "https://auth.docker.io/token"
+	values := fmt.Sprintf("service=registry.docker.io&scope=repository:%s:pull", img)
+
+	username, userOk := os.LookupEnv("DOCKER_USERNAME")
+	password, passOk := os.LookupEnv("DOCKER_PASSWORD")
+	if userOk && passOk {
+		logrus.Debugf("using docker log in credentials for registry token request")
+		values = fmt.Sprintf("%s&grant_type=password&client_id=coopcloud.tech&username=%s&password=%s", values, username, password)
+	}
+
+	fullURL := fmt.Sprintf("%s?%s", tokenURL, values)
+	req, err := retryablehttp.NewRequest("GET", fullURL, nil)
+	if err != nil {
+		return "", err
+	}
+
+	client := web.NewHTTPRetryClient()
+	res, err := client.Do(req)
+	if err != nil {
+		return "", err
+	}
+	defer res.Body.Close()
+
+	if res.StatusCode != http.StatusOK {
+		_, err := ioutil.ReadAll(res.Body)
+		if err != nil {
+			return "", err
+		}
+	}
+
+	body, err := ioutil.ReadAll(res.Body)
+	if err != nil {
+		return "", nil
+	}
+
+	tokenRes := struct {
+		AccessToken string `json:"access_token"`
+		Expiry      int    `json:"expires_in"`
+		Issued      string `json:"issued_at"`
+		Token       string `json:"token"`
+	}{}
+
+	if err := json.Unmarshal(body, &tokenRes); err != nil {
+		return "", err
+	}
+
+	return tokenRes.Token, nil
+}
+
+// GetTagDigest retrieves an image digest from a v2 registry
 func GetTagDigest(cl *client.Client, image reference.Named) (string, error) {
-	target := fmt.Sprintf("//%s", reference.Path(image))
+	img := reference.Path(image)
+	tag := image.(reference.NamedTagged).Tag()
+	manifestURL := fmt.Sprintf("https://index.docker.io/v2/%s/manifests/%s", img, tag)
 
-	ref, err := docker.ParseReference(target)
+	req, err := retryablehttp.NewRequest("GET", manifestURL, nil)
 	if err != nil {
-		return "", fmt.Errorf("failed to parse image %s, saw: %s", image, err.Error())
+		return "", err
 	}
 
-	ctx := context.Background()
-	img, err := ref.NewImage(ctx, nil)
+	token, err := getRegv2Token(cl, image)
 	if err != nil {
-		logrus.Debugf("failed to query remote registry for %s, saw: %s", image, err.Error())
-		return "", fmt.Errorf("unable to read digest for %s", image)
+		return "", err
 	}
-	defer img.Close()
 
-	digest := img.ConfigInfo().Digest.String()
+	if token == "" {
+		return "", fmt.Errorf("unable to retrieve registry token?")
+	}
+
+	req.Header = http.Header{
+		"Accept": []string{
+			"application/vnd.docker.distribution.manifest.v2+json",
+			"application/vnd.docker.distribution.manifest.list.v2+json",
+		},
+		"Authorization": []string{fmt.Sprintf("Bearer %s", token)},
+	}
+
+	client := web.NewHTTPRetryClient()
+	res, err := client.Do(req)
+	if err != nil {
+		return "", err
+	}
+	defer res.Body.Close()
+
+	if res.StatusCode != http.StatusOK {
+		_, err := ioutil.ReadAll(res.Body)
+		if err != nil {
+			return "", err
+		}
+	}
+
+	body, err := ioutil.ReadAll(res.Body)
+	if err != nil {
+		return "", err
+	}
+
+	registryResT1 := struct {
+		SchemaVersion int
+		MediaType     string
+		Manifests     []struct {
+			MediaType string
+			Size      int
+			Digest    string
+			Platform  struct {
+				Architecture string
+				Os           string
+			}
+		}
+	}{}
+
+	registryResT2 := struct {
+		SchemaVersion int
+		MediaType     string
+		Config        struct {
+			MediaType string
+			Size      int
+			Digest    string
+		}
+		Layers []struct {
+			MediaType string
+			Size      int
+			Digest    string
+		}
+	}{}
+
+	if err := json.Unmarshal(body, &registryResT1); err != nil {
+		return "", err
+	}
+
+	var digest string
+	for _, manifest := range registryResT1.Manifests {
+		if string(manifest.Platform.Architecture) == "amd64" {
+			digest = strings.Split(manifest.Digest, ":")[1][:7]
+		}
+	}
 
 	if digest == "" {
-		return digest, fmt.Errorf("unable to read digest for %s", image)
+		if err := json.Unmarshal(body, &registryResT2); err != nil {
+			return "", err
+		}
+		digest = strings.Split(registryResT2.Config.Digest, ":")[1][:7]
 	}
 
-	return strings.Split(digest, ":")[1][:7], nil
+	if digest == "" {
+		return "", fmt.Errorf("Unable to retrieve amd64 digest for '%s'", image)
+	}
+
+	return digest, nil
 }

pkg/client/volumes.go

@@ -5,18 +5,23 @@ import (
 
 	"github.com/docker/docker/api/types"
 	"github.com/docker/docker/api/types/filters"
+	"github.com/sirupsen/logrus"
 )
 
-func GetVolumes(ctx context.Context, server string, fs filters.Args) ([]*types.Volume, error) {
+func GetVolumes(ctx context.Context, server string, appName string) ([]*types.Volume, error) {
+
 	cl, err := New(server)
 	if err != nil {
 		return nil, err
 	}
 
+	fs := filters.NewArgs()
+	fs.Add("name", appName)
+
 	volumeListOKBody, err := cl.VolumeList(ctx, fs)
 	volumeList := volumeListOKBody.Volumes
 	if err != nil {
-		return volumeList, err
+		logrus.Fatal(err)
 	}
 
 	return volumeList, nil
@@ -24,11 +29,9 @@ func GetVolumes(ctx context.Context, server string, fs filters.Args) ([]*types.V
 
 func GetVolumeNames(volumes []*types.Volume) []string {
 	var volumeNames []string
-
 	for _, vol := range volumes {
 		volumeNames = append(volumeNames, vol.Name)
 	}
-
 	return volumeNames
 }
 
@@ -37,13 +40,12 @@ func RemoveVolumes(ctx context.Context, server string, volumeNames []string, for
 	if err != nil {
 		return err
 	}
-
 	for _, volName := range volumeNames {
 		err := cl.VolumeRemove(ctx, volName, force)
 		if err != nil {
 			return err
 		}
 	}
-
 	return nil
+
 }

pkg/compose/compose.go

@@ -8,7 +8,6 @@ import (
 	"strings"
 
 	"coopcloud.tech/abra/pkg/config"
-	"coopcloud.tech/abra/pkg/formatter"
 	"coopcloud.tech/abra/pkg/upstream/stack"
 	loader "coopcloud.tech/abra/pkg/upstream/stack"
 	composetypes "github.com/docker/cli/cli/compose/types"
@@ -17,26 +16,26 @@ import (
 )
 
 // UpdateTag updates an image tag in-place on file system local compose files.
-func UpdateTag(pattern, image, tag, recipeName string) (bool, error) {
+func UpdateTag(pattern, image, tag, recipeName string) error {
 	composeFiles, err := filepath.Glob(pattern)
 	if err != nil {
-		return false, err
+		return err
 	}
 
-	logrus.Debugf("considering %s config(s) for tag update", strings.Join(composeFiles, ", "))
+	logrus.Debugf("considering '%s' config(s) for tag update", strings.Join(composeFiles, ", "))
 
 	for _, composeFile := range composeFiles {
 		opts := stack.Deploy{Composefiles: []string{composeFile}}
 
-		envSamplePath := path.Join(config.RECIPES_DIR, recipeName, ".env.sample")
+		envSamplePath := path.Join(config.ABRA_DIR, "apps", recipeName, ".env.sample")
 		sampleEnv, err := config.ReadEnv(envSamplePath)
 		if err != nil {
-			return false, err
+			return err
 		}
 
 		compose, err := loader.LoadComposefile(opts, sampleEnv)
 		if err != nil {
-			return false, err
+			return err
 		}
 
 		for _, service := range compose.Services {
@@ -46,42 +45,40 @@ func UpdateTag(pattern, image, tag, recipeName string) (bool, error) {
 
 			img, _ := reference.ParseNormalizedNamed(service.Image)
 			if err != nil {
-				return false, err
+				return err
 			}
 
-			var composeTag string
-			switch img.(type) {
-			case reference.NamedTagged:
-				composeTag = img.(reference.NamedTagged).Tag()
-			default:
-				logrus.Debugf("unable to parse %s, skipping", img)
-				continue
+			composeImage := reference.Path(img)
+			if strings.Contains(composeImage, "library") {
+				// ParseNormalizedNamed prepends 'library' to images like nginx:<tag>,
+				// postgres:<tag>, i.e. images which do not have a username in the
+				// first position of the string
+				composeImage = strings.Split(composeImage, "/")[1]
 			}
+			composeTag := img.(reference.NamedTagged).Tag()
 
-			composeImage := formatter.StripTagMeta(reference.Path(img))
-
-			logrus.Debugf("parsed %s from %s", composeTag, service.Image)
+			logrus.Debugf("parsed '%s' from '%s'", composeTag, service.Image)
 
 			if image == composeImage {
 				bytes, err := ioutil.ReadFile(composeFile)
 				if err != nil {
-					return false, err
+					return err
 				}
 
 				old := fmt.Sprintf("%s:%s", composeImage, composeTag)
 				new := fmt.Sprintf("%s:%s", composeImage, tag)
 				replacedBytes := strings.Replace(string(bytes), old, new, -1)
 
-				logrus.Debugf("updating %s to %s in %s", old, new, compose.Filename)
+				logrus.Debugf("updating '%s' to '%s' in '%s'", old, new, compose.Filename)
 
 				if err := ioutil.WriteFile(compose.Filename, []byte(replacedBytes), 0764); err != nil {
-					return false, err
+					return err
 				}
 			}
 		}
 	}
 
-	return false, nil
+	return nil
 }
 
 // UpdateLabel updates a label in-place on file system local compose files.
@@ -96,7 +93,7 @@ func UpdateLabel(pattern, serviceName, label, recipeName string) error {
 	for _, composeFile := range composeFiles {
 		opts := stack.Deploy{Composefiles: []string{composeFile}}
 
-		envSamplePath := path.Join(config.RECIPES_DIR, recipeName, ".env.sample")
+		envSamplePath := path.Join(config.ABRA_DIR, "apps", recipeName, ".env.sample")
 		sampleEnv, err := config.ReadEnv(envSamplePath)
 		if err != nil {
 			return err
@@ -149,8 +146,8 @@ func UpdateLabel(pattern, serviceName, label, recipeName string) error {
 		}
 
 		if !discovered {
-			logrus.Warn("no existing label found, automagic insertion not supported yet")
-			logrus.Fatalf("add '- \"%s\"' manually to the 'app' service in %s", label, composeFile)
+			logrus.Warn("no existing label found, cannot continue...")
+			logrus.Fatalf("add \"%s\" manually, automagic insertion not supported yet", label)
 		}
 	}
 

pkg/config/app.go

@@ -2,18 +2,16 @@ package config
 
 import (
 	"fmt"
-	"html/template"
 	"io/ioutil"
 	"os"
 	"path"
 	"strings"
 
-	"coopcloud.tech/abra/pkg/formatter"
+	"coopcloud.tech/abra/cli/formatter"
 	"coopcloud.tech/abra/pkg/upstream/convert"
 	loader "coopcloud.tech/abra/pkg/upstream/stack"
 	stack "coopcloud.tech/abra/pkg/upstream/stack"
 	composetypes "github.com/docker/cli/cli/compose/types"
-	"github.com/docker/docker/api/types/filters"
 	"github.com/sirupsen/logrus"
 )
 
@@ -37,55 +35,24 @@ type AppFiles map[AppName]AppFile
 // App reprents an app with its env file read into memory
 type App struct {
 	Name   AppName
-	Recipe string
+	Type   string
 	Domain string
 	Env    AppEnv
 	Server string
 	Path   string
 }
 
-// StackName gets what the docker safe stack name is for the app. This should
-// not not shown to the user, use a.Name for that. Give the output of this
-// command to Docker only.
+// StackName gets what the docker safe stack name is for the app
 func (a App) StackName() string {
 	if _, exists := a.Env["STACK_NAME"]; exists {
 		return a.Env["STACK_NAME"]
 	}
-
 	stackName := SanitiseAppName(a.Name)
-
-	if len(stackName) > 45 {
-		logrus.Debugf("trimming %s to %s to avoid runtime limits", stackName, stackName[:45])
-		stackName = stackName[:45]
-	}
-
 	a.Env["STACK_NAME"] = stackName
-
 	return stackName
 }
 
-// Filters retrieves exact app filters for querying the container runtime.
-func (a App) Filters() (filters.Args, error) {
-	filters := filters.NewArgs()
-
-	composeFiles, err := GetAppComposeFiles(a.Recipe, a.Env)
-	if err != nil {
-		return filters, err
-	}
-
-	opts := stack.Deploy{Composefiles: composeFiles}
-	compose, err := GetAppComposeConfig(a.Recipe, opts, a.Env)
-	if err != nil {
-		return filters, err
-	}
-
-	for _, service := range compose.Services {
-		filter := fmt.Sprintf("^%s_%s", a.StackName(), service.Name)
-		filters.Add("name", filter)
-	}
-
-	return filters, nil
-}
+// SORTING TYPES
 
 // ByServer sort a slice of Apps
 type ByServer []App
@@ -96,25 +63,25 @@ func (a ByServer) Less(i, j int) bool {
 	return strings.ToLower(a[i].Server) < strings.ToLower(a[j].Server)
 }
 
-// ByServerAndRecipe sort a slice of Apps
-type ByServerAndRecipe []App
+// ByServerAndType sort a slice of Apps
+type ByServerAndType []App
 
-func (a ByServerAndRecipe) Len() int      { return len(a) }
-func (a ByServerAndRecipe) Swap(i, j int) { a[i], a[j] = a[j], a[i] }
-func (a ByServerAndRecipe) Less(i, j int) bool {
+func (a ByServerAndType) Len() int      { return len(a) }
+func (a ByServerAndType) Swap(i, j int) { a[i], a[j] = a[j], a[i] }
+func (a ByServerAndType) Less(i, j int) bool {
 	if a[i].Server == a[j].Server {
-		return strings.ToLower(a[i].Recipe) < strings.ToLower(a[j].Recipe)
+		return strings.ToLower(a[i].Type) < strings.ToLower(a[j].Type)
 	}
 	return strings.ToLower(a[i].Server) < strings.ToLower(a[j].Server)
 }
 
-// ByRecipe sort a slice of Apps
-type ByRecipe []App
+// ByType sort a slice of Apps
+type ByType []App
 
-func (a ByRecipe) Len() int      { return len(a) }
-func (a ByRecipe) Swap(i, j int) { a[i], a[j] = a[j], a[i] }
-func (a ByRecipe) Less(i, j int) bool {
-	return strings.ToLower(a[i].Recipe) < strings.ToLower(a[j].Recipe)
+func (a ByType) Len() int      { return len(a) }
+func (a ByType) Swap(i, j int) { a[i], a[j] = a[j], a[i] }
+func (a ByType) Less(i, j int) bool {
+	return strings.ToLower(a[i].Type) < strings.ToLower(a[j].Type)
 }
 
 // ByName sort a slice of Apps
@@ -129,14 +96,14 @@ func (a ByName) Less(i, j int) bool {
 func readAppEnvFile(appFile AppFile, name AppName) (App, error) {
 	env, err := ReadEnv(appFile.Path)
 	if err != nil {
-		return App{}, fmt.Errorf("env file for %s couldn't be read: %s", name, err.Error())
+		return App{}, fmt.Errorf("env file for '%s' couldn't be read: %s", name, err.Error())
 	}
 
-	logrus.Debugf("read env %s from %s", env, appFile.Path)
+	logrus.Debugf("read env '%s' from '%s'", env, appFile.Path)
 
 	app, err := newApp(env, name, appFile)
 	if err != nil {
-		return App{}, fmt.Errorf("env file for %s has issues: %s", name, err.Error())
+		return App{}, fmt.Errorf("env file for '%s' has issues: %s", name, err.Error())
 	}
 
 	return app, nil
@@ -146,18 +113,15 @@ func readAppEnvFile(appFile AppFile, name AppName) (App, error) {
 func newApp(env AppEnv, name string, appFile AppFile) (App, error) {
 	domain := env["DOMAIN"]
 
-	recipe, exists := env["RECIPE"]
+	appType, exists := env["TYPE"]
 	if !exists {
-		recipe, exists = env["TYPE"]
-		if !exists {
-			return App{}, fmt.Errorf("%s is missing the RECIPE env var", name)
-		}
+		return App{}, fmt.Errorf("%s is missing the TYPE env var", name)
 	}
 
 	return App{
 		Name:   name,
 		Domain: domain,
-		Recipe: recipe,
+		Type:   appType,
 		Env:    env,
 		Server: appFile.Server,
 		Path:   appFile.Path,
@@ -171,7 +135,7 @@ func LoadAppFiles(servers ...string) (AppFiles, error) {
 		if servers[0] == "" {
 			// Empty servers flag, one string will always be passed
 			var err error
-			servers, err = GetAllFoldersInDirectory(SERVERS_DIR)
+			servers, err = GetAllFoldersInDirectory(ABRA_SERVER_FOLDER)
 			if err != nil {
 				return nil, err
 			}
@@ -181,14 +145,14 @@ func LoadAppFiles(servers ...string) (AppFiles, error) {
 	logrus.Debugf("collecting metadata from %v servers: %s", len(servers), strings.Join(servers, ", "))
 
 	for _, server := range servers {
-		serverDir := path.Join(SERVERS_DIR, server)
+		serverDir := path.Join(ABRA_SERVER_FOLDER, server)
 		files, err := getAllFilesInDirectory(serverDir)
 		if err != nil {
-			return nil, fmt.Errorf("server %s doesn't exist? Run \"abra server ls\" to check", server)
+			return nil, err
 		}
 		for _, file := range files {
 			appName := strings.TrimSuffix(file.Name(), ".env")
-			appFilePath := path.Join(SERVERS_DIR, server, file.Name())
+			appFilePath := path.Join(ABRA_SERVER_FOLDER, server, file.Name())
 			appFiles[appName] = AppFile{
 				Path:   appFilePath,
 				Server: server,
@@ -244,13 +208,13 @@ func GetAppServiceNames(appName string) ([]string, error) {
 		return serviceNames, err
 	}
 
-	composeFiles, err := GetAppComposeFiles(app.Recipe, app.Env)
+	composeFiles, err := GetAppComposeFiles(app.Type, app.Env)
 	if err != nil {
 		return serviceNames, err
 	}
 
 	opts := stack.Deploy{Composefiles: composeFiles}
-	compose, err := GetAppComposeConfig(app.Recipe, opts, app.Env)
+	compose, err := GetAppComposeConfig(app.Type, opts, app.Env)
 	if err != nil {
 		return serviceNames, err
 	}
@@ -284,8 +248,8 @@ func GetAppNames() ([]string, error) {
 }
 
 // TemplateAppEnvSample copies the example env file for the app into the users env files
-func TemplateAppEnvSample(recipeName, appName, server, domain string) error {
-	envSamplePath := path.Join(RECIPES_DIR, recipeName, ".env.sample")
+func TemplateAppEnvSample(recipe, appName, server, domain string) error {
+	envSamplePath := path.Join(ABRA_DIR, "apps", recipe, ".env.sample")
 	envSample, err := ioutil.ReadFile(envSamplePath)
 	if err != nil {
 		return err
@@ -296,33 +260,15 @@ func TemplateAppEnvSample(recipeName, appName, server, domain string) error {
 		return fmt.Errorf("%s already exists?", appEnvPath)
 	}
 
+	envSample = []byte(strings.Replace(string(envSample), fmt.Sprintf("%s.example.com", recipe), domain, -1))
+	envSample = []byte(strings.Replace(string(envSample), "example.com", domain, -1))
+
 	err = ioutil.WriteFile(appEnvPath, envSample, 0664)
 	if err != nil {
 		return err
 	}
 
-	file, err := os.OpenFile(appEnvPath, os.O_RDWR, 0664)
-	if err != nil {
-		return err
-	}
-	defer file.Close()
-
-	tpl, err := template.ParseFiles(appEnvPath)
-	if err != nil {
-		return err
-	}
-
-	type templateVars struct {
-		Name   string
-		Domain string
-	}
-
-	tvars := templateVars{Name: recipeName, Domain: domain}
-	if err := tpl.Execute(file, tvars); err != nil {
-		return err
-	}
-
-	logrus.Debugf("copied & templated %s to %s", envSamplePath, appEnvPath)
+	logrus.Debugf("copied %s to %s", envSamplePath, appEnvPath)
 
 	return nil
 }
@@ -368,6 +314,9 @@ func GetAppStatuses(appFiles AppFiles) (map[string]map[string]string, error) {
 			if version, ok := service.Spec.Labels[labelKey]; ok {
 				result["version"] = version
 			} else {
+				//FIXME: we only need to check containers with the version label not
+				//       every single container and then skip when we see no label perf gains
+				//       to be had here
 				continue
 			}
 
@@ -387,7 +336,7 @@ func GetAppComposeFiles(recipe string, appEnv AppEnv) ([]string, error) {
 
 	if _, ok := appEnv["COMPOSE_FILE"]; !ok {
 		logrus.Debug("no COMPOSE_FILE detected, loading compose.yml")
-		path := fmt.Sprintf("%s/%s/compose.yml", RECIPES_DIR, recipe)
+		path := fmt.Sprintf("%s/%s/compose.yml", APPS_DIR, recipe)
 		composeFiles = append(composeFiles, path)
 		return composeFiles, nil
 	}
@@ -396,7 +345,7 @@ func GetAppComposeFiles(recipe string, appEnv AppEnv) ([]string, error) {
 	envVars := strings.Split(composeFileEnvVar, ":")
 	logrus.Debugf("COMPOSE_FILE detected (%s), loading %s", composeFileEnvVar, strings.Join(envVars, ", "))
 	for _, file := range strings.Split(composeFileEnvVar, ":") {
-		path := fmt.Sprintf("%s/%s/%s", RECIPES_DIR, recipe, file)
+		path := fmt.Sprintf("%s/%s/%s", APPS_DIR, recipe, file)
 		composeFiles = append(composeFiles, path)
 	}
 

pkg/config/app_test.go

@@ -26,6 +26,7 @@ func TestReadAppEnvFile(t *testing.T) {
 }
 
 func TestGetApp(t *testing.T) {
+	// TODO: Test failures as well as successes
 	app, err := GetApp(expectedAppFiles, appName)
 	if err != nil {
 		t.Fatal(err)

pkg/config/env.go

@@ -15,19 +15,16 @@ import (
 )
 
 var ABRA_DIR = os.ExpandEnv("$HOME/.abra")
-var SERVERS_DIR = path.Join(ABRA_DIR, "servers")
-var RECIPES_DIR = path.Join(ABRA_DIR, "recipes")
-var VENDOR_DIR = path.Join(ABRA_DIR, "vendor")
-var RECIPES_JSON = path.Join(ABRA_DIR, "catalogue", "recipes.json")
+var ABRA_SERVER_FOLDER = path.Join(ABRA_DIR, "servers")
+var APPS_JSON = path.Join(ABRA_DIR, "catalogue", "recipes.json")
+var APPS_DIR = path.Join(ABRA_DIR, "apps")
 var REPOS_BASE_URL = "https://git.coopcloud.tech/coop-cloud"
-var CATALOGUE_JSON_REPO_NAME = "recipes-catalogue-json"
-var SSH_URL_TEMPLATE = "ssh://git@git.coopcloud.tech:2222/coop-cloud/%s.git"
 
 // GetServers retrieves all servers.
 func GetServers() ([]string, error) {
 	var servers []string
 
-	servers, err := GetAllFoldersInDirectory(SERVERS_DIR)
+	servers, err := GetAllFoldersInDirectory(ABRA_SERVER_FOLDER)
 	if err != nil {
 		return servers, err
 	}
@@ -53,13 +50,13 @@ func ReadEnv(filePath string) (AppEnv, error) {
 
 // ReadServerNames retrieves all server names.
 func ReadServerNames() ([]string, error) {
-	serverNames, err := GetAllFoldersInDirectory(SERVERS_DIR)
+	serverNames, err := GetAllFoldersInDirectory(ABRA_SERVER_FOLDER)
 
 	if err != nil {
 		return nil, err
 	}
 
-	logrus.Debugf("read %s from %s", strings.Join(serverNames, ","), SERVERS_DIR)
+	logrus.Debugf("read %s from %s", strings.Join(serverNames, ","), ABRA_SERVER_FOLDER)
 
 	return serverNames, nil
 }
@@ -127,6 +124,17 @@ func GetAllFoldersInDirectory(directory string) ([]string, error) {
 	return folders, nil
 }
 
+// EnsureAbraDirExists checks for the abra config folder and throws error if not
+func EnsureAbraDirExists() error {
+	if _, err := os.Stat(ABRA_DIR); os.IsNotExist(err) {
+		logrus.Debugf("%s does not exist, creating it", ABRA_DIR)
+		if err := os.Mkdir(ABRA_DIR, 0764); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
 // ReadAbraShEnvVars reads env vars from an abra.sh recipe file.
 func ReadAbraShEnvVars(abraSh string) (map[string]string, error) {
 	envVars := make(map[string]string)

pkg/config/env_test.go

@@ -20,12 +20,12 @@ var serverName = "evil.corp"
 
 var expectedAppEnv = AppEnv{
 	"DOMAIN": "ecloud.evil.corp",
-	"RECIPE": "ecloud",
+	"TYPE":   "ecloud",
 }
 
 var expectedApp = App{
 	Name:   appName,
-	Recipe: expectedAppEnv["RECIPE"],
+	Type:   expectedAppEnv["TYPE"],
 	Domain: expectedAppEnv["DOMAIN"],
 	Env:    expectedAppEnv,
 	Path:   expectedAppFile.Path,
@@ -44,7 +44,7 @@ var expectedAppFiles = map[string]AppFile{
 // var expectedServerNames = []string{"evil.corp"}
 
 func TestGetAllFoldersInDirectory(t *testing.T) {
-	folders, err := GetAllFoldersInDirectory(testFolder)
+	folders, err := getAllFoldersInDirectory(testFolder)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -74,11 +74,11 @@ func TestReadEnv(t *testing.T) {
 	}
 	if !reflect.DeepEqual(env, expectedAppEnv) {
 		t.Fatalf(
-			"did not get expected application settings. Expected: DOMAIN=%s RECIPE=%s; Got: DOMAIN=%s RECIPE=%s",
+			"did not get expected application settings. Expected: DOMAIN=%s TYPE=%s; Got: DOMAIN=%s TYPE=%s",
 			expectedAppEnv["DOMAIN"],
-			expectedAppEnv["RECIPE"],
+			expectedAppEnv["TYPE"],
 			env["DOMAIN"],
-			env["RECIPE"],
+			env["TYPE"],
 		)
 	}
 }

pkg/container/container.go

@@ -5,7 +5,7 @@ import (
 	"fmt"
 	"strings"
 
-	"coopcloud.tech/abra/pkg/formatter"
+	abraFormatter "coopcloud.tech/abra/cli/formatter"
 	"github.com/AlecAivazis/survey/v2"
 	"github.com/docker/docker/api/types"
 	"github.com/docker/docker/api/types/filters"
@@ -13,10 +13,10 @@ import (
 	"github.com/sirupsen/logrus"
 )
 
-// GetContainer retrieves a container. If noInput is false and the retrievd
-// count of containers does not match 1, then a prompt is presented to let the
-// user choose. A count of 0 is handled gracefully.
-func GetContainer(c context.Context, cl *client.Client, filters filters.Args, noInput bool) (types.Container, error) {
+// GetContainer retrieves a container. If prompt is true and the retrievd count
+// of containers does not match expectedN, then a prompt is presented to let
+// the user choose.
+func GetContainer(c context.Context, cl *client.Client, filters filters.Args, prompt bool) (types.Container, error) {
 	containerOpts := types.ContainerListOptions{Filters: filters}
 	containers, err := cl.ContainerList(c, containerOpts)
 	if err != nil {
@@ -33,11 +33,11 @@ func GetContainer(c context.Context, cl *client.Client, filters filters.Args, no
 		for _, container := range containers {
 			containerName := strings.Join(container.Names, " ")
 			trimmed := strings.TrimPrefix(containerName, "/")
-			created := formatter.HumanDuration(container.Created)
+			created := abraFormatter.HumanDuration(container.Created)
 			containersRaw = append(containersRaw, fmt.Sprintf("%s (created %v)", trimmed, created))
 		}
 
-		if noInput {
+		if !prompt {
 			err := fmt.Errorf("expected 1 container but found %v: %s", len(containers), strings.Join(containersRaw, " "))
 			return types.Container{}, err
 		}

pkg/dns/common.go

@@ -47,6 +47,8 @@ func EnsureIPv4(domainName string) (string, error) {
 		},
 	}
 
+	logrus.Debugf("created DNS resolver via '%s'", freifunkDNS)
+
 	ctx := context.Background()
 	ips, err := resolver.LookupIPAddr(ctx, domainName)
 	if err != nil {
@@ -58,7 +60,7 @@ func EnsureIPv4(domainName string) (string, error) {
 	}
 
 	ipv4 = ips[0].IP.To4().String()
-	logrus.Debugf("%s points to %s (resolver: %s)", domainName, ipv4, freifunkDNS)
+	logrus.Debugf("discovered the following ipv4 addr: %s", ipv4)
 
 	return ipv4, nil
 }
@@ -92,12 +94,3 @@ func EnsureDomainsResolveSameIPv4(domainName, server string) (string, error) {
 
 	return ipv4, nil
 }
-
-// GetTTL parses a ttl string into a duration
-func GetTTL(ttl string) (time.Duration, error) {
-	val, err := time.ParseDuration(ttl)
-	if err != nil {
-		return val, err
-	}
-	return val, nil
-}

pkg/git/branch.go

@@ -1,35 +0,0 @@
-package git
-
-import (
-	"github.com/go-git/go-git/v5"
-	"github.com/go-git/go-git/v5/plumbing"
-)
-
-// GetCurrentBranch retrieves the current branch of a repository
-func GetCurrentBranch(repository *git.Repository) (string, error) {
-	branchRefs, err := repository.Branches()
-	if err != nil {
-		return "", err
-	}
-
-	headRef, err := repository.Head()
-	if err != nil {
-		return "", err
-	}
-
-	var currentBranchName string
-	err = branchRefs.ForEach(func(branchRef *plumbing.Reference) error {
-		if branchRef.Hash() == headRef.Hash() {
-			currentBranchName = branchRef.Name().String()
-
-			return nil
-		}
-
-		return nil
-	})
-	if err != nil {
-		return "", err
-	}
-
-	return currentBranchName, nil
-}

pkg/git/clone.go

@@ -7,6 +7,7 @@ import (
 	"strings"
 
 	"github.com/go-git/go-git/v5"
+	"github.com/go-git/go-git/v5/config"
 	"github.com/go-git/go-git/v5/plumbing"
 	"github.com/sirupsen/logrus"
 )
@@ -14,10 +15,10 @@ import (
 // Clone runs a git clone which accounts for different default branches.
 func Clone(dir, url string) error {
 	if _, err := os.Stat(dir); os.IsNotExist(err) {
-		logrus.Debugf("%s does not exist, attempting to git clone from %s", dir, url)
+		logrus.Debugf("'%s' does not exist, attempting to git clone from '%s'", dir, url)
 		_, err := git.PlainClone(dir, false, &git.CloneOptions{URL: url, Tags: git.AllTags})
 		if err != nil {
-			logrus.Debugf("cloning %s default branch failed, attempting from main branch", url)
+			logrus.Debugf("cloning '%s' default branch failed, attempting from main branch", url)
 			_, err := git.PlainClone(dir, false, &git.CloneOptions{
 				URL:           url,
 				Tags:          git.AllTags,
@@ -31,10 +32,77 @@ func Clone(dir, url string) error {
 				return err
 			}
 		}
-		logrus.Debugf("%s has been git cloned successfully", dir)
+		logrus.Debugf("'%s' has been git cloned successfully", dir)
 	} else {
-		logrus.Debugf("%s already exists", dir)
+		logrus.Debugf("'%s' already exists, doing nothing", dir)
 	}
 
 	return nil
 }
+
+// EnsureUpToDate ensures that a git repo on disk has the latest changes (git-fetch).
+func EnsureUpToDate(dir string) error {
+	repo, err := git.PlainOpen(dir)
+	if err != nil {
+		return err
+	}
+
+	recipeName := filepath.Base(dir)
+	isClean, err := IsClean(recipeName)
+	if err != nil {
+		return err
+	}
+
+	if !isClean {
+		return fmt.Errorf("'%s' has locally unstaged changes", recipeName)
+	}
+
+	branch := "master"
+	if _, err := repo.Branch("master"); err != nil {
+		if _, err := repo.Branch("main"); err != nil {
+			logrus.Debugf("failed to select branch in '%s'", dir)
+			return err
+		}
+		branch = "main"
+	}
+
+	logrus.Debugf("choosing '%s' as main git branch in '%s'", branch, dir)
+
+	worktree, err := repo.Worktree()
+	if err != nil {
+		return err
+	}
+
+	refName := fmt.Sprintf("refs/heads/%s", branch)
+	checkOutOpts := &git.CheckoutOptions{
+		Create: false,
+		Force:  true,
+		Branch: plumbing.ReferenceName(refName),
+	}
+	if err := worktree.Checkout(checkOutOpts); err != nil {
+		logrus.Debugf("failed to check out '%s' in '%s'", refName, dir)
+		return err
+	}
+
+	logrus.Debugf("successfully checked out '%s' in '%s'", branch, dir)
+
+	remote, err := repo.Remote("origin")
+	if err != nil {
+		return err
+	}
+
+	fetchOpts := &git.FetchOptions{
+		RemoteName: "origin",
+		RefSpecs:   []config.RefSpec{"refs/heads/*:refs/remotes/origin/*"},
+		Force:      true,
+	}
+	if err := remote.Fetch(fetchOpts); err != nil {
+		if !strings.Contains(err.Error(), "already up-to-date") {
+			return err
+		}
+	}
+
+	logrus.Debugf("successfully fetched all changes in '%s'", dir)
+
+	return nil
+}

pkg/git/commit.go

@@ -8,7 +8,7 @@ import (
 )
 
 // Commit runs a git commit
-func Commit(repoPath, glob, commitMessage string, dryRun bool) error {
+func Commit(repoPath, glob, commitMessage string, dryRun, push bool) error {
 	if commitMessage == "" {
 		return fmt.Errorf("no commit message specified?")
 	}
@@ -47,9 +47,18 @@ func Commit(repoPath, glob, commitMessage string, dryRun bool) error {
 		if err != nil {
 			return err
 		}
-		logrus.Debug("git changes commited")
+		logrus.Info("changes commited")
 	} else {
-		logrus.Debug("dry run: no changes commited")
+		logrus.Info("dry run: no changes commited")
+	}
+
+	if !dryRun && push {
+		if err := commitRepo.Push(&git.PushOptions{}); err != nil {
+			return err
+		}
+		logrus.Info("changes pushed")
+	} else {
+		logrus.Info("dry run: no changes pushed")
 	}
 
 	return nil

pkg/git/push.go

@@ -1,43 +0,0 @@
-package git
-
-import (
-	"github.com/go-git/go-git/v5"
-	"github.com/go-git/go-git/v5/config"
-	"github.com/sirupsen/logrus"
-)
-
-// Push pushes the latest changes & optionally tags to the default remote
-func Push(repoDir string, remote string, tags bool, dryRun bool) error {
-	if dryRun {
-		logrus.Debugf("dry run: no git changes pushed in %s", repoDir)
-		return nil
-	}
-
-	commitRepo, err := git.PlainOpen(repoDir)
-	if err != nil {
-		return err
-	}
-
-	opts := &git.PushOptions{}
-	if remote != "" {
-		opts.RemoteName = remote
-	}
-
-	if err := commitRepo.Push(opts); err != nil {
-		return err
-	}
-
-	logrus.Debugf("git changes pushed")
-
-	if tags {
-		opts.RefSpecs = append(opts.RefSpecs, config.RefSpec("+refs/tags/*:refs/tags/*"))
-
-		if err := commitRepo.Push(opts); err != nil {
-			return err
-		}
-
-		logrus.Debugf("git tags pushed")
-	}
-
-	return nil
-}

pkg/git/read.go

@@ -18,7 +18,7 @@ import (
 
 // GetRecipeHead retrieves latest HEAD metadata.
 func GetRecipeHead(recipeName string) (*plumbing.Reference, error) {
-	recipeDir := path.Join(config.RECIPES_DIR, recipeName)
+	recipeDir := path.Join(config.ABRA_DIR, "apps", recipeName)
 
 	repo, err := git.PlainOpen(recipeDir)
 	if err != nil {
@@ -34,8 +34,10 @@ func GetRecipeHead(recipeName string) (*plumbing.Reference, error) {
 }
 
 // IsClean checks if a repo has unstaged changes
-func IsClean(repoPath string) (bool, error) {
-	repo, err := git.PlainOpen(repoPath)
+func IsClean(recipeName string) (bool, error) {
+	recipeDir := path.Join(config.ABRA_DIR, "apps", recipeName)
+
+	repo, err := git.PlainOpen(recipeDir)
 	if err != nil {
 		return false, err
 	}
@@ -60,9 +62,9 @@ func IsClean(repoPath string) (bool, error) {
 	}
 
 	if status.String() != "" {
-		logrus.Debugf("discovered git status in %s: %s", repoPath, status.String())
+		logrus.Debugf("discovered git status for %s repository: %s", recipeName, status.String())
 	} else {
-		logrus.Debugf("discovered clean git status in %s", repoPath)
+		logrus.Debugf("discovered clean git status for %s repository", recipeName)
 	}
 
 	return status.IsClean(), nil

pkg/git/remote.go

@@ -1,28 +0,0 @@
-package git
-
-import (
-	"strings"
-
-	"github.com/go-git/go-git/v5"
-	"github.com/go-git/go-git/v5/config"
-	"github.com/sirupsen/logrus"
-)
-
-// CreateRemote creates a new git remote in a repository
-func CreateRemote(repo *git.Repository, name, url string, dryRun bool) error {
-	if dryRun {
-		logrus.Debugf("dry run: remote %s (%s) not created", name, url)
-		return nil
-	}
-
-	if _, err := repo.CreateRemote(&config.RemoteConfig{
-		Name: name,
-		URLs: []string{url},
-	}); err != nil {
-		if !strings.Contains(err.Error(), "remote already exists") {
-			return err
-		}
-	}
-
-	return nil
-}

pkg/integration/integration.go

@@ -1,12 +0,0 @@
-package integration
-
-import (
-	"os"
-	"testing"
-)
-
-func skipIfNotIntegration(t *testing.T) {
-	if os.Getenv("ABRA_INTEGRATION") == "" {
-		t.Skip("missing 'ABRA_INTEGRATION', not running integration tests")
-	}
-}

pkg/lint/recipe.go

@@ -1,338 +0,0 @@
-package lint
-
-import (
-	"fmt"
-	"net/http"
-	"os"
-	"path"
-
-	"coopcloud.tech/abra/pkg/config"
-	"coopcloud.tech/abra/pkg/recipe"
-	recipePkg "coopcloud.tech/abra/pkg/recipe"
-	"coopcloud.tech/tagcmp"
-	"github.com/docker/distribution/reference"
-	"github.com/sirupsen/logrus"
-)
-
-var Warn = "warn"
-var Critical = "critical"
-
-type LintFunction func(recipe.Recipe) (bool, error)
-
-type LintRule struct {
-	Ref          string
-	Level        string
-	Description  string
-	HowToResolve string
-	Function     LintFunction
-}
-
-var LintRules = map[string][]LintRule{
-	"warn": {
-		{
-			Ref:          "R001",
-			Level:        "warn",
-			Description:  "compose config has expected version",
-			HowToResolve: "ensure 'version: \"3.8\"' in compose configs",
-			Function:     LintComposeVersion,
-		},
-		{
-			Ref:          "R002",
-			Level:        "warn",
-			Description:  "healthcheck enabled for all services",
-			HowToResolve: "wire up healthchecks",
-			Function:     LintHealthchecks,
-		},
-		{
-			Ref:          "R003",
-			Level:        "warn",
-			Description:  "all images use a tag",
-			HowToResolve: "use a tag for all images",
-			Function:     LintAllImagesTagged,
-		},
-		{
-			Ref:          "R004",
-			Level:        "warn",
-			Description:  "no unstable tags",
-			HowToResolve: "tag all images with stable tags",
-			Function:     LintNoUnstableTags,
-		},
-		{
-			Ref:          "R005",
-			Level:        "warn",
-			Description:  "tags use semver-like format",
-			HowToResolve: "use semver-like tags",
-			Function:     LintSemverLikeTags,
-		},
-		{
-			Ref:          "R006",
-			Level:        "warn",
-			Description:  "has published catalogue version",
-			HowToResolve: "publish a recipe version to the catalogue",
-			Function:     LintHasPublishedVersion,
-		},
-		{
-			Ref:          "R007",
-			Level:        "warn",
-			Description:  "README.md metadata filled in",
-			HowToResolve: "fill out all the metadata",
-			Function:     LintMetadataFilledIn,
-		},
-		{
-			Ref:          "R013",
-			Level:        "warn",
-			Description:  "git.coopcloud.tech repo exists",
-			HowToResolve: "upload your recipe to git.coopcloud.tech/coop-cloud/...",
-			Function:     LintHasRecipeRepo,
-		},
-	},
-	"error": {
-		{
-			Ref:          "R008",
-			Level:        "error",
-			Description:  ".env.sample provided",
-			HowToResolve: "create an example .env.sample",
-			Function:     LintEnvConfigPresent,
-		},
-		{
-			Ref:          "R009",
-			Level:        "error",
-			Description:  "one service named 'app'",
-			HowToResolve: "name a servce 'app'",
-			Function:     LintAppService,
-		},
-		{
-			Ref:          "R010",
-			Level:        "error",
-			Description:  "traefik routing enabled",
-			HowToResolve: "include \"traefik.enable=true\" deploy label",
-			Function:     LintTraefikEnabled,
-		},
-		{
-			Ref:          "R011",
-			Level:        "error",
-			Description:  "all services have images",
-			HowToResolve: "ensure \"image: ...\" set on all services",
-			Function:     LintImagePresent,
-		},
-		{
-			Ref:          "R012",
-			Level:        "error",
-			Description:  "config version are vendored",
-			HowToResolve: "vendor config versions in an abra.sh",
-			Function:     LintAbraShVendors,
-		},
-	},
-}
-
-func LintForErrors(recipe recipe.Recipe) error {
-	logrus.Debugf("linting for critical errors in %s configs", recipe.Name)
-
-	for level := range LintRules {
-		if level != "error" {
-			continue
-		}
-		for _, rule := range LintRules[level] {
-			ok, err := rule.Function(recipe)
-			if err != nil {
-				return err
-			}
-			if !ok {
-				return fmt.Errorf("lint error in %s configs: \"%s\" failed lint checks (%s)", recipe.Name, rule.Description, rule.Ref)
-			}
-		}
-	}
-
-	logrus.Debugf("linting successful, %s is well configured", recipe.Name)
-
-	return nil
-}
-
-func LintComposeVersion(recipe recipe.Recipe) (bool, error) {
-	if recipe.Config.Version == "3.8" {
-		return true, nil
-	}
-
-	return true, nil
-}
-
-func LintEnvConfigPresent(recipe recipe.Recipe) (bool, error) {
-	envSample := fmt.Sprintf("%s/%s/.env.sample", config.RECIPES_DIR, recipe.Name)
-	if _, err := os.Stat(envSample); !os.IsNotExist(err) {
-		return true, nil
-	}
-
-	return false, nil
-}
-
-func LintAppService(recipe recipe.Recipe) (bool, error) {
-	for _, service := range recipe.Config.Services {
-		if service.Name == "app" {
-			return true, nil
-		}
-	}
-
-	return false, nil
-}
-
-func LintTraefikEnabled(recipe recipe.Recipe) (bool, error) {
-	for _, service := range recipe.Config.Services {
-		for label := range service.Deploy.Labels {
-			if label == "traefik.enable" {
-				if service.Deploy.Labels[label] == "true" {
-					return true, nil
-				}
-			}
-		}
-	}
-
-	return false, nil
-}
-
-func LintHealthchecks(recipe recipe.Recipe) (bool, error) {
-	for _, service := range recipe.Config.Services {
-		if service.HealthCheck == nil {
-			return false, nil
-		}
-	}
-
-	return true, nil
-}
-
-func LintAllImagesTagged(recipe recipe.Recipe) (bool, error) {
-	for _, service := range recipe.Config.Services {
-		img, err := reference.ParseNormalizedNamed(service.Image)
-		if err != nil {
-			return false, err
-		}
-		if reference.IsNameOnly(img) {
-			return false, nil
-		}
-	}
-
-	return true, nil
-}
-
-func LintNoUnstableTags(recipe recipe.Recipe) (bool, error) {
-	for _, service := range recipe.Config.Services {
-		img, err := reference.ParseNormalizedNamed(service.Image)
-		if err != nil {
-			return false, err
-		}
-
-		var tag string
-		switch img.(type) {
-		case reference.NamedTagged:
-			tag = img.(reference.NamedTagged).Tag()
-		case reference.Named:
-			return false, nil
-		}
-
-		if tag == "latest" {
-			return false, nil
-		}
-	}
-
-	return true, nil
-}
-
-func LintSemverLikeTags(recipe recipe.Recipe) (bool, error) {
-	for _, service := range recipe.Config.Services {
-		img, err := reference.ParseNormalizedNamed(service.Image)
-		if err != nil {
-			return false, err
-		}
-
-		var tag string
-		switch img.(type) {
-		case reference.NamedTagged:
-			tag = img.(reference.NamedTagged).Tag()
-		case reference.Named:
-			return false, nil
-		}
-
-		if !tagcmp.IsParsable(tag) {
-			return false, nil
-		}
-	}
-
-	return true, nil
-}
-
-func LintImagePresent(recipe recipe.Recipe) (bool, error) {
-	for _, service := range recipe.Config.Services {
-		if service.Image == "" {
-			return false, nil
-		}
-	}
-	return true, nil
-}
-
-func LintHasPublishedVersion(recipe recipe.Recipe) (bool, error) {
-	catl, err := recipePkg.ReadRecipeCatalogue()
-	if err != nil {
-		logrus.Fatal(err)
-	}
-
-	versions, err := recipePkg.GetRecipeCatalogueVersions(recipe.Name, catl)
-	if err != nil {
-		logrus.Fatal(err)
-	}
-
-	if len(versions) == 0 {
-		return false, nil
-	}
-
-	return true, nil
-}
-
-func LintMetadataFilledIn(r recipe.Recipe) (bool, error) {
-	features, category, err := recipe.GetRecipeFeaturesAndCategory(r.Name)
-	if err != nil {
-		return false, err
-	}
-
-	if category == "" {
-		return false, nil
-	}
-
-	if features.Backups == "" ||
-		features.Email == "" ||
-		features.Healthcheck == "" ||
-		features.Image.Image == "" ||
-		features.SSO == "" {
-		return false, nil
-	}
-
-	return true, nil
-}
-
-func LintAbraShVendors(recipe recipe.Recipe) (bool, error) {
-	for _, service := range recipe.Config.Services {
-		if len(service.Configs) > 0 {
-			abraSh := path.Join(config.RECIPES_DIR, recipe.Name, "abra.sh")
-			if _, err := os.Stat(abraSh); err != nil {
-				if os.IsNotExist(err) {
-					return false, err
-				}
-				return false, err
-			}
-		}
-	}
-	return true, nil
-}
-
-func LintHasRecipeRepo(recipe recipe.Recipe) (bool, error) {
-	url := fmt.Sprintf("%s/%s.git", config.REPOS_BASE_URL, recipe.Name)
-
-	res, err := http.Get(url)
-	if err != nil {
-		return false, err
-	}
-
-	if res.StatusCode != 200 {
-		return false, err
-	}
-
-	return true, nil
-}

pkg/recipe/recipe.go

@@ -1,161 +1,32 @@
 package recipe
 
 import (
-	"encoding/json"
 	"fmt"
-	"io/ioutil"
 	"os"
 	"path"
 	"path/filepath"
 	"strings"
-	"time"
 
-	"coopcloud.tech/abra/pkg/client"
 	"coopcloud.tech/abra/pkg/compose"
 	"coopcloud.tech/abra/pkg/config"
-	"coopcloud.tech/abra/pkg/formatter"
 	gitPkg "coopcloud.tech/abra/pkg/git"
 	"coopcloud.tech/abra/pkg/upstream/stack"
 	loader "coopcloud.tech/abra/pkg/upstream/stack"
-	"coopcloud.tech/abra/pkg/web"
 	composetypes "github.com/docker/cli/cli/compose/types"
-	"github.com/docker/distribution/reference"
 	"github.com/go-git/go-git/v5"
 	"github.com/go-git/go-git/v5/plumbing"
 	"github.com/sirupsen/logrus"
 )
 
-// RecipeCatalogueURL is the only current recipe catalogue available.
-const RecipeCatalogueURL = "https://recipes.coopcloud.tech"
-
-// ReposMetadataURL is the recipe repository metadata
-const ReposMetadataURL = "https://git.coopcloud.tech/api/v1/orgs/coop-cloud/repos"
-
-// tag represents a git tag.
-type tag = string
-
-// service represents a service within a recipe.
-type service = string
-
-// ServiceMeta represents meta info associated with a service.
-type ServiceMeta struct {
-	Digest string `json:"digest"`
-	Image  string `json:"image"`
-	Tag    string `json:"tag"`
-}
-
-// RecipeVersions are the versions associated with a recipe.
-type RecipeVersions []map[tag]map[service]ServiceMeta
-
-// RecipeMeta represents metadata for a recipe in the abra catalogue.
-type RecipeMeta struct {
-	Category      string         `json:"category"`
-	DefaultBranch string         `json:"default_branch"`
-	Description   string         `json:"description"`
-	Features      Features       `json:"features"`
-	Icon          string         `json:"icon"`
-	Name          string         `json:"name"`
-	Repository    string         `json:"repository"`
-	SSHURL        string         `json:"ssh_url"`
-	Versions      RecipeVersions `json:"versions"`
-	Website       string         `json:"website"`
-}
-
-// LatestVersion returns the latest version of a recipe.
-func (r RecipeMeta) LatestVersion() string {
-	var version string
-
-	// apps.json versions are sorted so the last key is latest
-	latest := r.Versions[len(r.Versions)-1]
-
-	for tag := range latest {
-		version = tag
-	}
-
-	logrus.Debugf("choosing %s as latest version of %s", version, r.Name)
-
-	return version
-}
-
-// Name represents a recipe name.
-type Name = string
-
-// RecipeCatalogue represents the entire recipe catalogue.
-type RecipeCatalogue map[Name]RecipeMeta
-
-// Flatten converts AppCatalogue to slice
-func (r RecipeCatalogue) Flatten() []RecipeMeta {
-	recipes := make([]RecipeMeta, 0, len(r))
-
-	for name := range r {
-		recipes = append(recipes, r[name])
-	}
-
-	return recipes
-}
-
-// ByRecipeName sorts recipes by name.
-type ByRecipeName []RecipeMeta
-
-func (r ByRecipeName) Len() int      { return len(r) }
-func (r ByRecipeName) Swap(i, j int) { r[i], r[j] = r[j], r[i] }
-func (r ByRecipeName) Less(i, j int) bool {
-	return strings.ToLower(r[i].Name) < strings.ToLower(r[j].Name)
-}
-
-// Image represents a recipe container image.
-type Image struct {
-	Image  string `json:"image"`
-	Rating string `json:"rating"`
-	Source string `json:"source"`
-	URL    string `json:"url"`
-}
-
-// Features represent what top-level features a recipe supports (e.g. does this recipe support backups?).
-type Features struct {
-	Backups     string `json:"backups"`
-	Email       string `json:"email"`
-	Healthcheck string `json:"healthcheck"`
-	Image       Image  `json:"image"`
-	Status      int    `json:"status"`
-	Tests       string `json:"tests"`
-	SSO         string `json:"sso"`
-}
-
 // Recipe represents a recipe.
 type Recipe struct {
 	Name   string
 	Config *composetypes.Config
-	Meta   RecipeMeta
-}
-
-// Push pushes the latest changes to a SSH URL remote. You need to have your
-// local SSH configuration for git.coopcloud.tech working for this to work
-func (r Recipe) Push(dryRun bool) error {
-	repo, err := git.PlainOpen(r.Dir())
-	if err != nil {
-		return err
-	}
-
-	if err := gitPkg.CreateRemote(repo, "origin-ssh", r.Meta.SSHURL, dryRun); err != nil {
-		return err
-	}
-
-	if err := gitPkg.Push(r.Dir(), "origin-ssh", true, dryRun); err != nil {
-		return err
-	}
-
-	return nil
-}
-
-// Dir retrieves the recipe repository path
-func (r Recipe) Dir() string {
-	return path.Join(config.RECIPES_DIR, r.Name)
 }
 
 // UpdateLabel updates a recipe label
 func (r Recipe) UpdateLabel(pattern, serviceName, label string) error {
-	fullPattern := fmt.Sprintf("%s/%s/%s", config.RECIPES_DIR, r.Name, pattern)
+	fullPattern := fmt.Sprintf("%s/%s/%s", config.APPS_DIR, r.Name, pattern)
 	if err := compose.UpdateLabel(fullPattern, serviceName, label, r.Name); err != nil {
 		return err
 	}
@@ -163,24 +34,20 @@ func (r Recipe) UpdateLabel(pattern, serviceName, label string) error {
 }
 
 // UpdateTag updates a recipe tag
-func (r Recipe) UpdateTag(image, tag string) (bool, error) {
-	pattern := fmt.Sprintf("%s/%s/compose**yml", config.RECIPES_DIR, r.Name)
-
-	image = formatter.StripTagMeta(image)
-
-	ok, err := compose.UpdateTag(pattern, image, tag, r.Name)
-	if err != nil {
-		return false, err
+func (r Recipe) UpdateTag(image, tag string) error {
+	pattern := fmt.Sprintf("%s/%s/compose**yml", config.APPS_DIR, r.Name)
+	if err := compose.UpdateTag(pattern, image, tag, r.Name); err != nil {
+		return err
 	}
-
-	return ok, nil
+	return nil
 }
 
 // Tags list the recipe tags
 func (r Recipe) Tags() ([]string, error) {
 	var tags []string
 
-	repo, err := git.PlainOpen(r.Dir())
+	recipeDir := path.Join(config.ABRA_DIR, "apps", r.Name)
+	repo, err := git.PlainOpen(recipeDir)
 	if err != nil {
 		return tags, err
 	}
@@ -197,7 +64,7 @@ func (r Recipe) Tags() ([]string, error) {
 		return tags, err
 	}
 
-	logrus.Debugf("detected %s as tags for recipe %s", strings.Join(tags, ", "), r.Name)
+	logrus.Debugf("detected '%s' as tags for recipe '%s'", strings.Join(tags, ", "), r.Name)
 
 	return tags, nil
 }
@@ -208,7 +75,7 @@ func Get(recipeName string) (Recipe, error) {
 		return Recipe{}, err
 	}
 
-	pattern := fmt.Sprintf("%s/%s/compose**yml", config.RECIPES_DIR, recipeName)
+	pattern := fmt.Sprintf("%s/%s/compose**yml", config.APPS_DIR, recipeName)
 	composeFiles, err := filepath.Glob(pattern)
 	if err != nil {
 		return Recipe{}, err
@@ -218,7 +85,7 @@ func Get(recipeName string) (Recipe, error) {
 		return Recipe{}, fmt.Errorf("%s is missing a compose.yml or compose.*.yml file?", recipeName)
 	}
 
-	envSamplePath := path.Join(config.RECIPES_DIR, recipeName, ".env.sample")
+	envSamplePath := path.Join(config.ABRA_DIR, "apps", recipeName, ".env.sample")
 	sampleEnv, err := config.ReadEnv(envSamplePath)
 	if err != nil {
 		return Recipe{}, err
@@ -230,29 +97,16 @@ func Get(recipeName string) (Recipe, error) {
 		return Recipe{}, err
 	}
 
-	meta, err := GetRecipeMeta(recipeName)
-	if err != nil {
-		if strings.Contains(err.Error(), "does not exist") {
-			meta = RecipeMeta{}
-		} else {
-			return Recipe{}, err
-		}
-	}
-
-	return Recipe{
-		Name:   recipeName,
-		Config: config,
-		Meta:   meta,
-	}, nil
+	return Recipe{Name: recipeName, Config: config}, nil
 }
 
 // EnsureExists ensures that a recipe is locally cloned
-func EnsureExists(recipeName string) error {
-	recipeDir := path.Join(config.RECIPES_DIR, recipeName)
+func EnsureExists(recipe string) error {
+	recipeDir := path.Join(config.ABRA_DIR, "apps", strings.ToLower(recipe))
 
 	if _, err := os.Stat(recipeDir); os.IsNotExist(err) {
 		logrus.Debugf("%s does not exist, attemmpting to clone", recipeDir)
-		url := fmt.Sprintf("%s/%s.git", config.REPOS_BASE_URL, recipeName)
+		url := fmt.Sprintf("%s/%s.git", config.REPOS_BASE_URL, recipe)
 		if err := gitPkg.Clone(recipeDir, url); err != nil {
 			return err
 		}
@@ -267,15 +121,15 @@ func EnsureExists(recipeName string) error {
 
 // EnsureVersion checks whether a specific version exists for a recipe.
 func EnsureVersion(recipeName, version string) error {
-	recipeDir := path.Join(config.RECIPES_DIR, recipeName)
+	recipeDir := path.Join(config.ABRA_DIR, "apps", recipeName)
 
-	isClean, err := gitPkg.IsClean(recipeDir)
+	isClean, err := gitPkg.IsClean(recipeName)
 	if err != nil {
 		return err
 	}
 
 	if !isClean {
-		return fmt.Errorf("%s has locally unstaged changes", recipeName)
+		return fmt.Errorf("'%s' has locally unstaged changes", recipeName)
 	}
 
 	if err := gitPkg.EnsureGitRepo(recipeDir); err != nil {
@@ -307,7 +161,7 @@ func EnsureVersion(recipeName, version string) error {
 	logrus.Debugf("read %s as tags for recipe %s", strings.Join(parsedTags, ", "), recipeName)
 
 	if tagRef.String() == "" {
-		logrus.Warnf("no published release discovered for %s", recipeName)
+		logrus.Warnf("%s recipe has no local tag: %s? this recipe version is not released?", recipeName, version)
 		return nil
 	}
 
@@ -330,11 +184,11 @@ func EnsureVersion(recipeName, version string) error {
 	return nil
 }
 
-// EnsureLatest makes sure the latest commit is checked out for a local recipe repository
+// EnsureLatest makes sure the latest commit is checkout on for a local recipe repository.
 func EnsureLatest(recipeName string) error {
-	recipeDir := path.Join(config.RECIPES_DIR, recipeName)
+	recipeDir := path.Join(config.ABRA_DIR, "apps", recipeName)
 
-	isClean, err := gitPkg.IsClean(recipeDir)
+	isClean, err := gitPkg.IsClean(recipeName)
 	if err != nil {
 		return err
 	}
@@ -359,15 +213,20 @@ func EnsureLatest(recipeName string) error {
 		return err
 	}
 
-	branch, err := GetDefaultBranch(repo, recipeName)
-	if err != nil {
-		return err
+	branch := "master"
+	if _, err := repo.Branch("master"); err != nil {
+		if _, err := repo.Branch("main"); err != nil {
+			logrus.Debugf("failed to select branch in %s", path.Join(config.APPS_DIR, recipeName))
+			return err
+		}
+		branch = "main"
 	}
 
+	refName := fmt.Sprintf("refs/heads/%s", branch)
 	checkOutOpts := &git.CheckoutOptions{
 		Create: false,
 		Force:  true,
-		Branch: plumbing.ReferenceName(branch),
+		Branch: plumbing.ReferenceName(refName),
 	}
 
 	if err := worktree.Checkout(checkOutOpts); err != nil {
@@ -387,10 +246,9 @@ func ChaosVersion(recipeName string) (string, error) {
 		return version, err
 	}
 
-	version = formatter.SmallSHA(head.String())
+	version = head.String()[:8]
 
-	recipeDir := path.Join(config.RECIPES_DIR, recipeName)
-	isClean, err := gitPkg.IsClean(recipeDir)
+	isClean, err := gitPkg.IsClean(recipeName)
 	if err != nil {
 		return version, err
 	}
@@ -406,7 +264,7 @@ func ChaosVersion(recipeName string) (string, error) {
 func GetRecipesLocal() ([]string, error) {
 	var recipes []string
 
-	recipes, err := config.GetAllFoldersInDirectory(config.RECIPES_DIR)
+	recipes, err := config.GetAllFoldersInDirectory(config.APPS_DIR)
 	if err != nil {
 		return recipes, err
 	}
@@ -427,646 +285,8 @@ func GetVersionLabelLocal(recipe Recipe) (string, error) {
 	}
 
 	if label == "" {
-		return label, fmt.Errorf("%s has no version label? try running \"abra recipe sync %s\" first?", recipe.Name, recipe.Name)
+		return label, fmt.Errorf("unable to retrieve synced version label for %s", recipe.Name)
 	}
 
 	return label, nil
 }
-
-func GetRecipeFeaturesAndCategory(recipeName string) (Features, string, error) {
-	feat := Features{}
-
-	var category string
-
-	readmePath := path.Join(config.RECIPES_DIR, recipeName, "README.md")
-
-	logrus.Debugf("attempting to open %s for recipe metadata parsing", readmePath)
-
-	readmeFS, err := ioutil.ReadFile(readmePath)
-	if err != nil {
-		return feat, category, err
-	}
-
-	readmeMetadata, err := GetStringInBetween( // Find text between delimiters
-		recipeName,
-		string(readmeFS),
-		"<!-- metadata -->", "<!-- endmetadata -->",
-	)
-	if err != nil {
-		return feat, category, err
-	}
-
-	readmeLines := strings.Split( // Array item from lines
-		strings.ReplaceAll( // Remove \t tabs
-			readmeMetadata, "\t", "",
-		),
-		"\n")
-
-	for _, val := range readmeLines {
-		if strings.Contains(val, "**Category**") {
-			category = strings.TrimSpace(
-				strings.TrimPrefix(val, "* **Category**:"),
-			)
-		}
-		if strings.Contains(val, "**Backups**") {
-			feat.Backups = strings.TrimSpace(
-				strings.TrimPrefix(val, "* **Backups**:"),
-			)
-		}
-		if strings.Contains(val, "**Email**") {
-			feat.Email = strings.TrimSpace(
-				strings.TrimPrefix(val, "* **Email**:"),
-			)
-		}
-		if strings.Contains(val, "**SSO**") {
-			feat.SSO = strings.TrimSpace(
-				strings.TrimPrefix(val, "* **SSO**:"),
-			)
-		}
-		if strings.Contains(val, "**Healthcheck**") {
-			feat.Healthcheck = strings.TrimSpace(
-				strings.TrimPrefix(val, "* **Healthcheck**:"),
-			)
-		}
-		if strings.Contains(val, "**Tests**") {
-			feat.Tests = strings.TrimSpace(
-				strings.TrimPrefix(val, "* **Tests**:"),
-			)
-		}
-		if strings.Contains(val, "**Image**") {
-			imageMetadata, err := GetImageMetadata(strings.TrimSpace(
-				strings.TrimPrefix(val, "* **Image**:"),
-			), recipeName)
-			if err != nil {
-				continue
-			}
-			feat.Image = imageMetadata
-		}
-	}
-
-	return feat, category, nil
-}
-
-func GetImageMetadata(imageRowString, recipeName string) (Image, error) {
-	img := Image{}
-
-	imgFields := strings.Split(imageRowString, ",")
-
-	for i, elem := range imgFields {
-		imgFields[i] = strings.TrimSpace(elem)
-	}
-
-	if len(imgFields) < 3 {
-		if imageRowString != "" {
-			logrus.Warnf("%s image meta has incorrect format: %s", recipeName, imageRowString)
-		} else {
-			logrus.Warnf("%s image meta is empty?", recipeName)
-		}
-		return img, nil
-	}
-
-	img.Rating = imgFields[1]
-	img.Source = imgFields[2]
-
-	imgString := imgFields[0]
-
-	imageName, err := GetStringInBetween(recipeName, imgString, "[", "]")
-	if err != nil {
-		logrus.Fatal(err)
-	}
-	img.Image = strings.ReplaceAll(imageName, "`", "")
-
-	imageURL, err := GetStringInBetween(recipeName, imgString, "(", ")")
-	if err != nil {
-		logrus.Fatal(err)
-	}
-	img.URL = imageURL
-
-	return img, nil
-}
-
-// GetStringInBetween returns empty string if no start or end string found
-func GetStringInBetween(recipeName, str, start, end string) (result string, err error) {
-	s := strings.Index(str, start)
-	if s == -1 {
-		return "", fmt.Errorf("%s: marker string %s not found", recipeName, start)
-	}
-
-	s += len(start)
-	e := strings.Index(str[s:], end)
-
-	if e == -1 {
-		return "", fmt.Errorf("%s: end marker %s not found", recipeName, end)
-	}
-
-	return str[s : s+e], nil
-}
-
-// EnsureUpToDate ensures that the local repo is synced to the remote
-func EnsureUpToDate(recipeName string) error {
-	recipeDir := path.Join(config.RECIPES_DIR, recipeName)
-
-	isClean, err := gitPkg.IsClean(recipeDir)
-	if err != nil {
-		return err
-	}
-
-	if !isClean {
-		return fmt.Errorf("%s has locally unstaged changes", recipeName)
-	}
-
-	repo, err := git.PlainOpen(recipeDir)
-	if err != nil {
-		return err
-	}
-
-	remotes, err := repo.Remotes()
-	if err != nil {
-		return err
-	}
-
-	if len(remotes) == 0 {
-		logrus.Debugf("cannot ensure %s is up-to-date, no git remotes configured", recipeName)
-		return nil
-	}
-
-	worktree, err := repo.Worktree()
-	if err != nil {
-		return err
-	}
-
-	branch, err := CheckoutDefaultBranch(repo, recipeName)
-	if err != nil {
-		return err
-	}
-
-	opts := &git.PullOptions{
-		Force:         true,
-		ReferenceName: branch,
-	}
-
-	if err := worktree.Pull(opts); err != nil {
-		if !strings.Contains(err.Error(), "already up-to-date") {
-			return err
-		}
-	}
-
-	logrus.Debugf("fetched latest git changes for %s", recipeName)
-
-	return nil
-}
-
-func GetDefaultBranch(repo *git.Repository, recipeName string) (plumbing.ReferenceName, error) {
-	recipeDir := path.Join(config.RECIPES_DIR, recipeName)
-
-	meta, _ := GetRecipeMeta(recipeName)
-	if meta.DefaultBranch != "" {
-		return plumbing.ReferenceName(fmt.Sprintf("refs/heads/%s", meta.DefaultBranch)), nil
-	}
-
-	branch := "master"
-	if _, err := repo.Branch("master"); err != nil {
-		if _, err := repo.Branch("main"); err != nil {
-			return "", fmt.Errorf("failed to select default branch in %s", recipeDir)
-		}
-		branch = "main"
-	}
-
-	return plumbing.ReferenceName(fmt.Sprintf("refs/heads/%s", branch)), nil
-}
-
-func CheckoutDefaultBranch(repo *git.Repository, recipeName string) (plumbing.ReferenceName, error) {
-	recipeDir := path.Join(config.RECIPES_DIR, recipeName)
-
-	branch, err := GetDefaultBranch(repo, recipeName)
-	if err != nil {
-		return plumbing.ReferenceName(""), err
-	}
-
-	worktree, err := repo.Worktree()
-	if err != nil {
-		return plumbing.ReferenceName(""), err
-	}
-
-	checkOutOpts := &git.CheckoutOptions{
-		Create: false,
-		Force:  true,
-		Branch: branch,
-	}
-
-	if err := worktree.Checkout(checkOutOpts); err != nil {
-		recipeDir := path.Join(config.RECIPES_DIR, recipeName)
-		logrus.Debugf("failed to check out %s in %s", branch, recipeDir)
-		return branch, err
-	}
-
-	logrus.Debugf("successfully checked out %v in %s", branch, recipeDir)
-
-	return branch, nil
-}
-
-// recipeCatalogueFSIsLatest checks whether the recipe catalogue stored locally
-// is up to date.
-func recipeCatalogueFSIsLatest() (bool, error) {
-	httpClient := web.NewHTTPRetryClient()
-	res, err := httpClient.Head(RecipeCatalogueURL)
-	if err != nil {
-		return false, err
-	}
-
-	lastModified := res.Header["Last-Modified"][0]
-	parsed, err := time.Parse(time.RFC1123, lastModified)
-	if err != nil {
-		return false, err
-	}
-
-	info, err := os.Stat(config.RECIPES_JSON)
-	if err != nil {
-		if os.IsNotExist(err) {
-			logrus.Debugf("no recipe catalogue found in file system cache")
-			return false, nil
-		}
-		return false, err
-	}
-
-	localModifiedTime := info.ModTime().Unix()
-	remoteModifiedTime := parsed.Unix()
-
-	if localModifiedTime < remoteModifiedTime {
-		logrus.Debug("file system cached recipe catalogue is out-of-date")
-		return false, nil
-	}
-
-	logrus.Debug("file system cached recipe catalogue is up-to-date")
-
-	return true, nil
-}
-
-// ReadRecipeCatalogue reads the recipe catalogue.
-func ReadRecipeCatalogue() (RecipeCatalogue, error) {
-	recipes := make(RecipeCatalogue)
-
-	if err := EnsureCatalogue(); err != nil {
-		return nil, err
-	}
-
-	recipeFSIsLatest, err := recipeCatalogueFSIsLatest()
-	if err != nil {
-		return nil, err
-	}
-
-	if !recipeFSIsLatest {
-		if err := readRecipeCatalogueWeb(&recipes); err != nil {
-			return nil, err
-		}
-		return recipes, nil
-	}
-
-	if err := readRecipeCatalogueFS(&recipes); err != nil {
-		return nil, err
-	}
-
-	return recipes, nil
-}
-
-// readRecipeCatalogueFS reads the catalogue from the file system.
-func readRecipeCatalogueFS(target interface{}) error {
-	recipesJSONFS, err := ioutil.ReadFile(config.RECIPES_JSON)
-	if err != nil {
-		return err
-	}
-
-	if err := json.Unmarshal(recipesJSONFS, &target); err != nil {
-		return err
-	}
-
-	logrus.Debugf("read recipe catalogue from file system cache in %s", config.RECIPES_JSON)
-
-	return nil
-}
-
-// readRecipeCatalogueWeb reads the catalogue from the web.
-func readRecipeCatalogueWeb(target interface{}) error {
-	if err := web.ReadJSON(RecipeCatalogueURL, &target); err != nil {
-		return err
-	}
-
-	recipesJSON, err := json.MarshalIndent(target, "", "    ")
-	if err != nil {
-		return err
-	}
-
-	if err := ioutil.WriteFile(config.RECIPES_JSON, recipesJSON, 0764); err != nil {
-		return err
-	}
-
-	logrus.Debugf("read recipe catalogue from web at %s", RecipeCatalogueURL)
-
-	return nil
-}
-
-// VersionsOfService lists the version of a service.
-func VersionsOfService(recipe, serviceName string) ([]string, error) {
-	var versions []string
-
-	catalogue, err := ReadRecipeCatalogue()
-	if err != nil {
-		return nil, err
-	}
-
-	rec, ok := catalogue[recipe]
-	if !ok {
-		return versions, nil
-	}
-
-	alreadySeen := make(map[string]bool)
-	for _, serviceVersion := range rec.Versions {
-		for tag := range serviceVersion {
-			if _, ok := alreadySeen[tag]; !ok {
-				alreadySeen[tag] = true
-				versions = append(versions, tag)
-			}
-		}
-	}
-
-	logrus.Debugf("detected versions %s for %s", strings.Join(versions, ", "), recipe)
-
-	return versions, nil
-}
-
-// GetRecipeMeta retrieves the recipe metadata from the recipe catalogue.
-func GetRecipeMeta(recipeName string) (RecipeMeta, error) {
-	catl, err := ReadRecipeCatalogue()
-	if err != nil {
-		return RecipeMeta{}, err
-	}
-
-	recipeMeta, ok := catl[recipeName]
-	if !ok {
-		return RecipeMeta{}, fmt.Errorf("recipe %s does not exist?", recipeName)
-	}
-
-	if err := EnsureExists(recipeName); err != nil {
-		return RecipeMeta{}, err
-	}
-
-	logrus.Debugf("recipe metadata retrieved for %s", recipeName)
-
-	return recipeMeta, nil
-}
-
-// RepoMeta is a single recipe repo metadata.
-type RepoMeta struct {
-	ID                        int `json:"id"`
-	Owner                     Owner
-	Name                      string      `json:"name"`
-	FullName                  string      `json:"full_name"`
-	Description               string      `json:"description"`
-	Empty                     bool        `json:"empty"`
-	Private                   bool        `json:"private"`
-	Fork                      bool        `json:"fork"`
-	Template                  bool        `json:"template"`
-	Parent                    interface{} `json:"parent"`
-	Mirror                    bool        `json:"mirror"`
-	Size                      int         `json:"size"`
-	HTMLURL                   string      `json:"html_url"`
-	SSHURL                    string      `json:"ssh_url"`
-	CloneURL                  string      `json:"clone_url"`
-	OriginalURL               string      `json:"original_url"`
-	Website                   string      `json:"website"`
-	StarsCount                int         `json:"stars_count"`
-	ForksCount                int         `json:"forks_count"`
-	WatchersCount             int         `json:"watchers_count"`
-	OpenIssuesCount           int         `json:"open_issues_count"`
-	OpenPRCount               int         `json:"open_pr_counter"`
-	ReleaseCounter            int         `json:"release_counter"`
-	DefaultBranch             string      `json:"default_branch"`
-	Archived                  bool        `json:"archived"`
-	CreatedAt                 string      `json:"created_at"`
-	UpdatedAt                 string      `json:"updated_at"`
-	Permissions               Permissions
-	HasIssues                 bool `json:"has_issues"`
-	InternalTracker           InternalTracker
-	HasWiki                   bool   `json:"has_wiki"`
-	HasPullRequests           bool   `json:"has_pull_requests"`
-	HasProjects               bool   `json:"has_projects"`
-	IgnoreWhitespaceConflicts bool   `json:"ignore_whitespace_conflicts"`
-	AllowMergeCommits         bool   `json:"allow_merge_commits"`
-	AllowRebase               bool   `json:"allow_rebase"`
-	AllowRebaseExplicit       bool   `json:"allow_rebase_explicit"`
-	AllowSquashMerge          bool   `json:"allow_squash_merge"`
-	AvatarURL                 string `json:"avatar_url"`
-	Internal                  bool   `json:"internal"`
-	MirrorInterval            string `json:"mirror_interval"`
-}
-
-// Owner is the repo organisation owner metadata.
-type Owner struct {
-	ID         int    `json:"id"`
-	Login      string `json:"login"`
-	FullName   string `json:"full_name"`
-	Email      string `json:"email"`
-	AvatarURL  string `json:"avatar_url"`
-	Language   string `json:"language"`
-	IsAdmin    bool   `json:"is_admin"`
-	LastLogin  string `json:"last_login"`
-	Created    string `json:"created"`
-	Restricted bool   `json:"restricted"`
-	Username   string `json:"username"`
-}
-
-// Permissions is perms metadata for a repo.
-type Permissions struct {
-	Admin bool `json:"admin"`
-	Push  bool `json:"push"`
-	Pull  bool `json:"pull"`
-}
-
-// InternalTracker is issue tracker metadata for a repo.
-type InternalTracker struct {
-	EnableTimeTracker                bool `json:"enable_time_tracker"`
-	AllowOnlyContributorsToTrackTime bool `json:"allow_only_contributors_to_track_time"`
-	EnableIssuesDependencies         bool `json:"enable_issue_dependencies"`
-}
-
-// RepoCatalogue represents all the recipe repo metadata.
-type RepoCatalogue map[string]RepoMeta
-
-// ReadReposMetadata retrieves coop-cloud/... repo metadata from Gitea.
-func ReadReposMetadata() (RepoCatalogue, error) {
-	reposMeta := make(RepoCatalogue)
-
-	pageIdx := 1
-	bar := formatter.CreateProgressbar(-1, "retrieving recipe repos list from git.coopcloud.tech...")
-	for {
-		var reposList []RepoMeta
-
-		pagedURL := fmt.Sprintf("%s?page=%v", ReposMetadataURL, pageIdx)
-
-		logrus.Debugf("fetching repo metadata from %s", pagedURL)
-
-		if err := web.ReadJSON(pagedURL, &reposList); err != nil {
-			return reposMeta, err
-		}
-
-		if len(reposList) == 0 {
-			bar.Add(1)
-			break
-		}
-
-		for idx, repo := range reposList {
-			reposMeta[repo.Name] = reposList[idx]
-		}
-
-		pageIdx++
-		bar.Add(1)
-	}
-
-	fmt.Println() // newline for spinner
-
-	return reposMeta, nil
-}
-
-// GetRecipeVersions retrieves all recipe versions.
-func GetRecipeVersions(recipeName string) (RecipeVersions, error) {
-	versions := RecipeVersions{}
-
-	recipeDir := path.Join(config.RECIPES_DIR, recipeName)
-
-	logrus.Debugf("attempting to open git repository in %s", recipeDir)
-
-	repo, err := git.PlainOpen(recipeDir)
-	if err != nil {
-		return versions, err
-	}
-
-	worktree, err := repo.Worktree()
-	if err != nil {
-		return versions, err
-	}
-
-	gitTags, err := repo.Tags()
-	if err != nil {
-		return versions, err
-	}
-
-	if err := gitTags.ForEach(func(ref *plumbing.Reference) (err error) {
-		tag := strings.TrimPrefix(string(ref.Name()), "refs/tags/")
-
-		logrus.Debugf("processing %s for %s", tag, recipeName)
-
-		checkOutOpts := &git.CheckoutOptions{
-			Create: false,
-			Force:  true,
-			Branch: plumbing.ReferenceName(ref.Name()),
-		}
-		if err := worktree.Checkout(checkOutOpts); err != nil {
-			logrus.Debugf("failed to check out %s in %s", tag, recipeDir)
-			return err
-		}
-
-		logrus.Debugf("successfully checked out %s in %s", ref.Name(), recipeDir)
-
-		recipe, err := Get(recipeName)
-		if err != nil {
-			return err
-		}
-
-		cl, err := client.New("default") // only required for container registry calls
-		if err != nil {
-			return err
-		}
-
-		queryCache := make(map[reference.Named]string)
-		versionMeta := make(map[string]ServiceMeta)
-		for _, service := range recipe.Config.Services {
-
-			img, err := reference.ParseNormalizedNamed(service.Image)
-			if err != nil {
-				return err
-			}
-
-			path := reference.Path(img)
-
-			path = formatter.StripTagMeta(path)
-
-			var tag string
-			switch img.(type) {
-			case reference.NamedTagged:
-				tag = img.(reference.NamedTagged).Tag()
-			case reference.Named:
-				logrus.Warnf("%s service is missing image tag?", path)
-				continue
-			}
-
-			var exists bool
-			var digest string
-			if digest, exists = queryCache[img]; !exists {
-				logrus.Debugf("cache miss: querying for image: %s, tag: %s", path, tag)
-
-				var err error
-				digest, err = client.GetTagDigest(cl, img)
-				if err != nil {
-					logrus.Warn(err)
-					digest = "unknown"
-				}
-
-				queryCache[img] = digest
-				logrus.Debugf("cached insert: %s, tag: %s, digest: %s", path, tag, digest)
-			} else {
-				logrus.Debugf("cache hit: image: %s, tag: %s, digest: %s", path, tag, digest)
-			}
-
-			versionMeta[service.Name] = ServiceMeta{
-				Digest: digest,
-				Image:  path,
-				Tag:    img.(reference.NamedTagged).Tag(),
-			}
-		}
-
-		versions = append(versions, map[string]map[string]ServiceMeta{tag: versionMeta})
-
-		return nil
-	}); err != nil {
-		return versions, err
-	}
-
-	_, err = CheckoutDefaultBranch(repo, recipeName)
-	if err != nil {
-		return versions, err
-	}
-
-	logrus.Debugf("collected %s for %s", versions, recipeName)
-
-	return versions, nil
-}
-
-// GetRecipeCatalogueVersions list the recipe versions listed in the recipe catalogue.
-func GetRecipeCatalogueVersions(recipeName string, catl RecipeCatalogue) ([]string, error) {
-	var versions []string
-
-	if recipeMeta, exists := catl[recipeName]; exists {
-		for _, versionMeta := range recipeMeta.Versions {
-			for tag := range versionMeta {
-				versions = append(versions, tag)
-			}
-		}
-	}
-
-	return versions, nil
-}
-
-// EnsureCatalogue ensures that the catalogue is cloned locally & present.
-func EnsureCatalogue() error {
-	catalogueDir := path.Join(config.ABRA_DIR, "catalogue")
-	if _, err := os.Stat(catalogueDir); err != nil && os.IsNotExist(err) {
-		url := fmt.Sprintf("%s/%s.git", config.REPOS_BASE_URL, config.CATALOGUE_JSON_REPO_NAME)
-		if err := gitPkg.Clone(catalogueDir, url); err != nil {
-			return err
-		}
-
-		logrus.Debugf("cloned catalogue repository to %s", catalogueDir)
-	}
-
-	return nil
-}

pkg/secret/pass.go

@@ -19,13 +19,13 @@ func PassInsertSecret(secretValue, secretName, appName, server string) error {
 		secretValue, server, appName, secretName,
 	)
 
-	logrus.Debugf("attempting to run %s", cmd)
+	logrus.Debugf("attempting to run '%s'", cmd)
 
 	if err := exec.Command("bash", "-c", cmd).Run(); err != nil {
 		return err
 	}
 
-	logrus.Infof("%s inserted into pass store", secretName)
+	logrus.Infof("'%s' inserted into pass store", secretName)
 
 	return nil
 }
@@ -41,13 +41,13 @@ func PassRmSecret(secretName, appName, server string) error {
 		server, appName, secretName,
 	)
 
-	logrus.Debugf("attempting to run %s", cmd)
+	logrus.Debugf("attempting to run '%s'", cmd)
 
 	if err := exec.Command("bash", "-c", cmd).Run(); err != nil {
 		return err
 	}
 
-	logrus.Infof("%s removed from pass store", secretName)
+	logrus.Infof("'%s' removed from pass store", secretName)
 
 	return nil
 }

pkg/secret/secret.go

@@ -8,7 +8,6 @@ import (
 	"regexp"
 	"strconv"
 	"strings"
-	"sync"
 
 	"coopcloud.tech/abra/pkg/client"
 	"coopcloud.tech/abra/pkg/config"
@@ -35,7 +34,7 @@ func GeneratePasswords(count, length uint) ([]string, error) {
 		return nil, err
 	}
 
-	logrus.Debugf("generated %s", strings.Join(passwords, ", "))
+	logrus.Debugf("generated '%s'", strings.Join(passwords, ", "))
 
 	return passwords, nil
 }
@@ -54,7 +53,7 @@ func GeneratePassphrases(count uint) ([]string, error) {
 		return nil, err
 	}
 
-	logrus.Debugf("generated %s", strings.Join(passphrases, ", "))
+	logrus.Debugf("generated '%s'", strings.Join(passphrases, ", "))
 
 	return passphrases, nil
 }
@@ -70,32 +69,35 @@ func ReadSecretEnvVars(appEnv config.AppEnv) map[string]string {
 		}
 	}
 
-	logrus.Debugf("read %s as secrets from %s", secretEnvVars, appEnv)
+	logrus.Debugf("read '%s' as secrets from '%s'", secretEnvVars, appEnv)
 
 	return secretEnvVars
 }
 
+// TODO: should probably go in the config/app package?
 func ParseSecretEnvVarName(secretEnvVar string) string {
 	withoutPrefix := strings.TrimPrefix(secretEnvVar, "SECRET_")
 	withoutSuffix := strings.TrimSuffix(withoutPrefix, "_VERSION")
 	name := strings.ToLower(withoutSuffix)
-	logrus.Debugf("parsed %s as name from %s", name, secretEnvVar)
+	logrus.Debugf("parsed '%s' as name from '%s'", name, secretEnvVar)
 	return name
 }
 
+// TODO: should probably go in the config/app package?
 func ParseGeneratedSecretName(secret string, appEnv config.App) string {
 	name := fmt.Sprintf("%s_", appEnv.StackName())
 	withoutAppName := strings.TrimPrefix(secret, name)
 	idx := strings.LastIndex(withoutAppName, "_")
 	parsed := withoutAppName[:idx]
-	logrus.Debugf("parsed %s as name from %s", parsed, secret)
+	logrus.Debugf("parsed '%s' as name from '%s'", parsed, secret)
 	return parsed
 }
 
+// TODO: should probably go in the config/app package?
 func ParseSecretEnvVarValue(secret string) (secretValue, error) {
 	values := strings.Split(secret, "#")
 	if len(values) == 0 {
-		return secretValue{}, fmt.Errorf("unable to parse %s", secret)
+		return secretValue{}, fmt.Errorf("unable to parse '%s'", secret)
 	}
 
 	if len(values) == 1 {
@@ -111,7 +113,7 @@ func ParseSecretEnvVarValue(secret string) (secretValue, error) {
 	}
 	version := strings.ReplaceAll(values[0], " ", "")
 
-	logrus.Debugf("parsed version %s and length '%v' from %s", version, length, secret)
+	logrus.Debugf("parsed version '%s' and length '%v' from '%s'", version, length, secret)
 
 	return secretValue{Version: version, Length: length}, nil
 }
@@ -120,32 +122,23 @@ func ParseSecretEnvVarValue(secret string) (secretValue, error) {
 func GenerateSecrets(secretEnvVars map[string]string, appName, server string) (map[string]string, error) {
 	secrets := make(map[string]string)
 
-	var mutex sync.Mutex
-	var wg sync.WaitGroup
 	ch := make(chan error, len(secretEnvVars))
 	for secretEnvVar := range secretEnvVars {
-		wg.Add(1)
-
 		go func(s string) {
-			defer wg.Done()
-
 			secretName := ParseSecretEnvVarName(s)
 			secretValue, err := ParseSecretEnvVarValue(secretEnvVars[s])
 			if err != nil {
 				ch <- err
 				return
 			}
-
 			secretRemoteName := fmt.Sprintf("%s_%s_%s", appName, secretName, secretValue.Version)
-			logrus.Debugf("attempting to generate and store %s on %s", secretRemoteName, server)
-
+			logrus.Debugf("attempting to generate and store '%s' on '%s'", secretRemoteName, server)
 			if secretValue.Length > 0 {
 				passwords, err := GeneratePasswords(1, uint(secretValue.Length))
 				if err != nil {
 					ch <- err
 					return
 				}
-
 				if err := client.StoreSecret(secretRemoteName, passwords[0], server); err != nil {
 					if strings.Contains(err.Error(), "AlreadyExists") {
 						logrus.Warnf("%s already exists, moving on...", secretRemoteName)
@@ -155,9 +148,6 @@ func GenerateSecrets(secretEnvVars map[string]string, appName, server string) (m
 					}
 					return
 				}
-
-				mutex.Lock()
-				defer mutex.Unlock()
 				secrets[secretName] = passwords[0]
 			} else {
 				passphrases, err := GeneratePassphrases(1)
@@ -165,7 +155,6 @@ func GenerateSecrets(secretEnvVars map[string]string, appName, server string) (m
 					ch <- err
 					return
 				}
-
 				if err := client.StoreSecret(secretRemoteName, passphrases[0], server); err != nil {
 					if strings.Contains(err.Error(), "AlreadyExists") {
 						logrus.Warnf("%s already exists, moving on...", secretRemoteName)
@@ -175,17 +164,12 @@ func GenerateSecrets(secretEnvVars map[string]string, appName, server string) (m
 					}
 					return
 				}
-
-				mutex.Lock()
-				defer mutex.Unlock()
 				secrets[secretName] = passphrases[0]
 			}
 			ch <- nil
 		}(secretEnvVar)
 	}
 
-	wg.Wait()
-
 	for range secretEnvVars {
 		err := <-ch
 		if err != nil {
@@ -193,7 +177,7 @@ func GenerateSecrets(secretEnvVars map[string]string, appName, server string) (m
 		}
 	}
 
-	logrus.Debugf("generated and stored %s on %s", secrets, server)
+	logrus.Debugf("generated and stored '%s' on '%s'", secrets, server)
 
 	return secrets, nil
 }

pkg/service/service.go

@@ -1,78 +0,0 @@
-package service
-
-import (
-	"context"
-	"fmt"
-	"strings"
-
-	"coopcloud.tech/abra/pkg/formatter"
-	"github.com/AlecAivazis/survey/v2"
-	"github.com/docker/docker/api/types"
-	"github.com/docker/docker/api/types/filters"
-	"github.com/docker/docker/api/types/swarm"
-	"github.com/docker/docker/client"
-	"github.com/sirupsen/logrus"
-)
-
-// GetService retrieves a service container. If prompt is true and the retrievd
-// count of service containers does not match 1, then a prompt is presented to
-// let the user choose. A count of 0 is handled gracefully.
-func GetService(c context.Context, cl *client.Client, filters filters.Args, prompt bool) (swarm.Service, error) {
-	serviceOpts := types.ServiceListOptions{Filters: filters}
-	services, err := cl.ServiceList(c, serviceOpts)
-	if err != nil {
-		return swarm.Service{}, err
-	}
-
-	if len(services) == 0 {
-		filter := filters.Get("name")[0]
-		return swarm.Service{}, fmt.Errorf("no services matching the %v filter found?", filter)
-	}
-
-	if len(services) != 1 {
-		var servicesRaw []string
-		for _, service := range services {
-			serviceName := service.Spec.Name
-			created := formatter.HumanDuration(service.CreatedAt.Unix())
-			servicesRaw = append(servicesRaw, fmt.Sprintf("%s (created %v)", serviceName, created))
-		}
-
-		if !prompt {
-			err := fmt.Errorf("expected 1 service but found %v: %s", len(services), strings.Join(servicesRaw, " "))
-			return swarm.Service{}, err
-		}
-
-		logrus.Warnf("ambiguous service list received, prompting for input")
-
-		var response string
-		prompt := &survey.Select{
-			Message: "which service are you looking for?",
-			Options: servicesRaw,
-		}
-
-		if err := survey.AskOne(prompt, &response); err != nil {
-			return swarm.Service{}, err
-		}
-
-		chosenService := strings.TrimSpace(strings.Split(response, " ")[0])
-		for _, service := range services {
-			serviceName := strings.ToLower(service.Spec.Name)
-			if serviceName == chosenService {
-				return service, nil
-			}
-		}
-
-		logrus.Panic("failed to match chosen service")
-	}
-
-	return services[0], nil
-}
-
-// ContainerToServiceName converts a container name to a service name.
-func ContainerToServiceName(containerNames []string, stackName string) string {
-	containerName := strings.Join(containerNames, "")
-	trimmed := strings.TrimPrefix(containerName, "/")
-	stackNameServiceName := strings.Split(trimmed, ".")[0]
-	splitter := fmt.Sprintf("%s_", stackName)
-	return strings.Split(stackNameServiceName, splitter)[1]
-}

pkg/ssh/ssh.go

@@ -325,7 +325,7 @@ func connect(username, host, port string, authMethod ssh.AuthMethod, timeout tim
 	conn, err = net.DialTimeout("tcp", hostnameAndPort, timeout)
 	if err != nil {
 		logrus.Debugf("tcp dialing %s failed, trying via ~/.ssh/config", hostnameAndPort)
-		hostConfig, err := GetHostConfig(host, username, port, true)
+		hostConfig, err := GetHostConfig(host, username, port)
 		if err != nil {
 			return nil, err
 		}
@@ -452,7 +452,7 @@ func GetContextConnDetails(serverName string) (*dockerSSHPkg.Spec, error) {
 		}
 	}
 
-	hostConfig, err := GetHostConfig(serverName, "", "", false)
+	hostConfig, err := GetHostConfig(serverName, "", "")
 	if err != nil {
 		return &dockerSSHPkg.Spec{}, err
 	}
@@ -472,51 +472,45 @@ func GetContextConnDetails(serverName string) (*dockerSSHPkg.Spec, error) {
 }
 
 // GetHostConfig retrieves a ~/.ssh/config config for a host.
-func GetHostConfig(hostname, username, port string, override bool) (HostConfig, error) {
+func GetHostConfig(hostname, username, port string) (HostConfig, error) {
 	var hostConfig HostConfig
 
-	if hostname == "" || override {
-		if sshHost := ssh_config.Get(hostname, "Hostname"); sshHost != "" {
-			hostname = sshHost
-		}
+	var host, idf string
+
+	if host = ssh_config.Get(hostname, "Hostname"); host == "" {
+		logrus.Debugf("no hostname found in SSH config, assuming %s", hostname)
+		host = hostname
 	}
 
-	if username == "" || override {
-		if sshUser := ssh_config.Get(hostname, "User"); sshUser != "" {
-			username = sshUser
-		} else {
+	if username == "" {
+		if username = ssh_config.Get(hostname, "User"); username == "" {
 			systemUser, err := user.Current()
 			if err != nil {
 				return hostConfig, err
 			}
+			logrus.Debugf("no username found in SSH config or passed on command-line, assuming %s", username)
 			username = systemUser.Username
 		}
 	}
 
-	if port == "" || override {
-		if sshPort := ssh_config.Get(hostname, "Port"); sshPort != "" {
-			// skip override probably correct port with dummy default value from
-			// ssh_config which is 22. only when the original port number is empty
-			// should we try this default. this might not cover all cases
-			// unfortunately.
-			if port != "" && sshPort != "22" {
-				port = sshPort
-			}
+	if port == "" {
+		if port = ssh_config.Get(hostname, "Port"); port == "" {
+			logrus.Debugf("no port found in SSH config or passed on command-line, assuming 22")
+			port = "22"
 		}
 	}
 
-	if idf := ssh_config.Get(hostname, "IdentityFile"); idf != "" && idf != "~/.ssh/identity" {
+	idf = ssh_config.Get(hostname, "IdentityFile")
+	if idf != "" {
 		var err error
 		idf, err = identityFileAbsPath(idf)
 		if err != nil {
 			return hostConfig, err
 		}
 		hostConfig.IdentityFile = idf
-	} else {
-		hostConfig.IdentityFile = ""
 	}
 
-	hostConfig.Host = hostname
+	hostConfig.Host = host
 	hostConfig.Port = port
 	hostConfig.User = username
 

pkg/upstream/commandconn/commandconn.go

@@ -188,14 +188,14 @@ func ignorableCloseError(err error) bool {
 func (c *commandConn) CloseRead() error {
 	// NOTE: maybe already closed here
 	if err := c.stdout.Close(); err != nil && !ignorableCloseError(err) {
-		// muted because https://github.com/docker/compose/issues/8544
+		// TODO: muted because https://github.com/docker/compose/issues/8544
 		// logrus.Warnf("commandConn.CloseRead: %v", err)
 	}
 	c.stdioClosedMu.Lock()
 	c.stdoutClosed = true
 	c.stdioClosedMu.Unlock()
 	if err := c.killIfStdioClosed(); err != nil {
-		// muted because https://github.com/docker/compose/issues/8544
+		// TODO: muted because https://github.com/docker/compose/issues/8544
 		// logrus.Warnf("commandConn.CloseRead: %v", err)
 	}
 	return nil
@@ -212,14 +212,14 @@ func (c *commandConn) Read(p []byte) (int, error) {
 func (c *commandConn) CloseWrite() error {
 	// NOTE: maybe already closed here
 	if err := c.stdin.Close(); err != nil && !ignorableCloseError(err) {
-		// muted because https://github.com/docker/compose/issues/8544
+		// TODO: muted because https://github.com/docker/compose/issues/8544
 		// logrus.Warnf("commandConn.CloseWrite: %v", err)
 	}
 	c.stdioClosedMu.Lock()
 	c.stdinClosed = true
 	c.stdioClosedMu.Unlock()
 	if err := c.killIfStdioClosed(); err != nil {
-		// muted because https://github.com/docker/compose/issues/8544
+		// TODO: muted because https://github.com/docker/compose/issues/8544
 		// logrus.Warnf("commandConn.CloseWrite: %v", err)
 	}
 	return nil
@@ -239,7 +239,7 @@ func (c *commandConn) Close() error {
 		logrus.Warnf("commandConn.Close: CloseRead: %v", err)
 	}
 	if err = c.CloseWrite(); err != nil {
-		// muted because https://github.com/docker/compose/issues/8544
+		// TODO: muted because https://github.com/docker/compose/issues/8544
 		// logrus.Warnf("commandConn.Close: CloseWrite: %v", err)
 	}
 	return err

pkg/upstream/commandconn/connection.go

@@ -44,7 +44,6 @@ func getConnectionHelper(daemonURL string, sshFlags []string) (*connhelper.Conne
 			ctxConnDetails.Host,
 			ctxConnDetails.User,
 			ctxConnDetails.Port,
-			false,
 		)
 		if err != nil {
 			return nil, err
@@ -67,13 +66,13 @@ func getConnectionHelper(daemonURL string, sshFlags []string) (*connhelper.Conne
 	return nil, err
 }
 
-func NewConnectionHelper(daemonURL string) (*connhelper.ConnectionHelper, error) {
+func NewConnectionHelper(daemonURL string) *connhelper.ConnectionHelper {
 	helper, err := GetConnectionHelper(daemonURL)
 	if err != nil {
-		return nil, err
+		logrus.Fatal(err)
 	}
 
-	return helper, nil
+	return helper
 }
 
 func getDockerEndpoint(host string) (docker.Endpoint, error) {

pkg/upstream/container/hijack.go

@@ -16,6 +16,7 @@ import (
 )
 
 // The default escape key sequence: ctrl-p, ctrl-q
+// TODO: This could be moved to `pkg/term`.
 var defaultEscapeKeys = []byte{16, 17}
 
 // A hijackedIOStreamer handles copying input to and output from streams to the

pkg/upstream/convert/service.go

@@ -399,6 +399,7 @@ func convertServiceNetworks(
 	return nets, nil
 }
 
+// TODO: fix secrets API so that SecretAPIClient is not required here
 func convertServiceSecrets(
 	client client.SecretAPIClient,
 	namespace Namespace,
@@ -420,12 +421,6 @@ func convertServiceSecrets(
 			return nil, err
 		}
 
-		// NOTE(d1): strip # length=... modifiers
-		if strings.Contains(obj.Name, "#") {
-			vals := strings.Split(obj.Name, "#")
-			obj.Name = strings.TrimSpace(vals[0])
-		}
-
 		file := swarm.SecretReferenceFileTarget(obj.File)
 		refs = append(refs, &swarm.SecretReference{
 			File:       &file,
@@ -447,6 +442,8 @@ func convertServiceSecrets(
 // required by the serivce. Unlike convertServiceSecrets, this takes the whole
 // ServiceConfig, because some Configs may be needed as a result of other
 // fields (like CredentialSpecs).
+//
+// TODO: fix configs API so that ConfigsAPIClient is not required here
 func convertServiceConfigObjs(
 	client client.ConfigAPIClient,
 	namespace Namespace,
@@ -629,6 +626,7 @@ func convertHealthcheck(healthcheck *composetypes.HealthCheckConfig) (*container
 }
 
 func convertRestartPolicy(restart string, source *composetypes.RestartPolicy) (*swarm.RestartPolicy, error) {
+	// TODO: log if restart is being ignored
 	if source == nil {
 		policy, err := opts.ParseRestartPolicy(restart)
 		if err != nil {

pkg/upstream/service/scale.go

@@ -1,44 +0,0 @@
-package upstream
-
-import (
-	"context"
-	"fmt"
-
-	"github.com/docker/docker/api/types"
-	"github.com/docker/docker/client"
-	"github.com/sirupsen/logrus"
-)
-
-// RunServiceScale scales a service (useful for restart action)
-func RunServiceScale(ctx context.Context, cl *client.Client, serviceID string, scale uint64) error {
-	service, _, err := cl.ServiceInspectWithRaw(ctx, serviceID, types.ServiceInspectOptions{})
-	if err != nil {
-		return err
-	}
-
-	serviceMode := &service.Spec.Mode
-	if serviceMode.Replicated != nil {
-		serviceMode.Replicated.Replicas = &scale
-	} else if serviceMode.ReplicatedJob != nil {
-		serviceMode.ReplicatedJob.TotalCompletions = &scale
-	} else {
-		return fmt.Errorf("scale can only be used with replicated or replicated-job mode")
-	}
-
-	response, err := cl.ServiceUpdate(
-		ctx,
-		service.ID,
-		service.Version,
-		service.Spec,
-		types.ServiceUpdateOptions{},
-	)
-	if err != nil {
-		return err
-	}
-
-	for _, warning := range response.Warnings {
-		logrus.Warn(warning)
-	}
-
-	return nil
-}

pkg/upstream/stack/loader.go

@@ -35,21 +35,16 @@ func LoadComposefile(opts Deploy, appEnv map[string]string) (*composetypes.Confi
 		return nil, err
 	}
 
-	recipeName, exists := appEnv["RECIPE"]
-	if !exists {
-		recipeName, _ = appEnv["TYPE"]
-	}
-
 	unsupportedProperties := loader.GetUnsupportedProperties(dicts...)
 	if len(unsupportedProperties) > 0 {
 		logrus.Warnf("%s: ignoring unsupported options: %s",
-			recipeName, strings.Join(unsupportedProperties, ", "))
+			appEnv["TYPE"], strings.Join(unsupportedProperties, ", "))
 	}
 
 	deprecatedProperties := loader.GetDeprecatedProperties(dicts...)
 	if len(deprecatedProperties) > 0 {
 		logrus.Warnf("%s: ignoring deprecated options: %s",
-			recipeName, propertyWarnings(deprecatedProperties))
+			appEnv["TYPE"], propertyWarnings(deprecatedProperties))
 	}
 	return config, nil
 }

pkg/upstream/stack/stack.go

@@ -112,7 +112,7 @@ func GetDeployedServicesByName(ctx context.Context, cl *dockerclient.Client, sta
 
 // IsDeployed chekcks whether an appp is deployed or not.
 func IsDeployed(ctx context.Context, cl *dockerclient.Client, stackName string) (bool, string, error) {
-	version := "unknown"
+	version := ""
 	isDeployed := false
 
 	filter := filters.NewArgs()
@@ -132,12 +132,12 @@ func IsDeployed(ctx context.Context, cl *dockerclient.Client, stackName string)
 			}
 		}
 
-		logrus.Debugf("%s has been detected as deployed with version %s", stackName, version)
+		logrus.Debugf("'%s' has been detected as deployed with version '%s'", stackName, version)
 
 		return true, version, nil
 	}
 
-	logrus.Debugf("%s has been detected as not deployed", stackName)
+	logrus.Debugf("'%s' has been detected as not deployed", stackName)
 	return isDeployed, version, nil
 }
 
@@ -158,7 +158,7 @@ func pruneServices(ctx context.Context, cl *dockerclient.Client, namespace conve
 }
 
 // RunDeploy is the swarm implementation of docker stack deploy
-func RunDeploy(cl *dockerclient.Client, opts Deploy, cfg *composetypes.Config, appName string, dontWait bool) error {
+func RunDeploy(cl *dockerclient.Client, opts Deploy, cfg *composetypes.Config, recipeName string) error {
 	ctx := context.Background()
 
 	if err := validateResolveImageFlag(&opts); err != nil {
@@ -170,7 +170,7 @@ func RunDeploy(cl *dockerclient.Client, opts Deploy, cfg *composetypes.Config, a
 		opts.ResolveImage = ResolveImageNever
 	}
 
-	return deployCompose(ctx, cl, opts, cfg, appName, dontWait)
+	return deployCompose(ctx, cl, opts, cfg, recipeName)
 }
 
 // validateResolveImageFlag validates the opts.resolveImage command line option
@@ -183,7 +183,7 @@ func validateResolveImageFlag(opts *Deploy) error {
 	}
 }
 
-func deployCompose(ctx context.Context, cl *dockerclient.Client, opts Deploy, config *composetypes.Config, appName string, dontWait bool) error {
+func deployCompose(ctx context.Context, cl *dockerclient.Client, opts Deploy, config *composetypes.Config, recipeName string) error {
 	namespace := convert.NewNamespace(opts.Namespace)
 
 	if opts.Prune {
@@ -224,7 +224,7 @@ func deployCompose(ctx context.Context, cl *dockerclient.Client, opts Deploy, co
 		return err
 	}
 
-	return deployServices(ctx, cl, services, namespace, opts.SendRegistryAuth, opts.ResolveImage, appName, dontWait)
+	return deployServices(ctx, cl, services, namespace, opts.SendRegistryAuth, opts.ResolveImage, recipeName)
 }
 
 func getServicesDeclaredNetworks(serviceConfigs []composetypes.ServiceConfig) map[string]struct{} {
@@ -340,8 +340,7 @@ func deployServices(
 	namespace convert.Namespace,
 	sendAuth bool,
 	resolveImage string,
-	appName string,
-	dontWait bool) error {
+	recipeName string) error {
 	existingServices, err := GetStackServices(ctx, cl, namespace.Name())
 	if err != nil {
 		return err
@@ -360,6 +359,18 @@ func deployServices(
 			encodedAuth string
 		)
 
+		// FIXME: disable for now as not sure how to avoid having a `dockerCli`
+		// instance here and would rather not copy/pasta that entire module in
+		// right now for something that we don't even support right now. Will skip
+		// this for now.
+		if sendAuth {
+			// Retrieve encoded auth token from the image reference
+			// encodedAuth, err = command.RetrieveAuthTokenFromImage(ctx, dockerCli, image)
+			// if err != nil {
+			// 	return err
+			// }
+		}
+
 		if service, exists := existingServiceMap[name]; exists {
 			logrus.Infof("Updating service %s (id: %s)\n", name, service.ID)
 
@@ -392,6 +403,7 @@ func deployServices(
 
 			// Stack deploy does not have a `--force` option. Preserve existing
 			// ForceUpdate value so that tasks are not re-deployed if not updated.
+			// TODO move this to API client?
 			serviceSpec.TaskTemplate.ForceUpdate = service.Spec.TaskTemplate.ForceUpdate
 
 			response, err := cl.ServiceUpdate(ctx, service.ID, service.Version, serviceSpec, updateOpts)
@@ -427,19 +439,14 @@ func deployServices(
 	for _, serviceName := range serviceIDs {
 		serviceNames = append(serviceNames, serviceName)
 	}
-
-	if dontWait {
-		logrus.Warn("skipping converge logic checks")
-		return nil
-	}
-
 	logrus.Infof("waiting for services to converge: %s", strings.Join(serviceNames, ", "))
+
 	ch := make(chan error, len(serviceIDs))
 	for serviceID, serviceName := range serviceIDs {
 		logrus.Debugf("waiting on %s to converge", serviceName)
-		go func(sID, sName, aName string) {
-			ch <- WaitOnService(ctx, cl, sID, aName)
-		}(serviceID, serviceName, appName)
+		go func(sID, sName, rName string) {
+			ch <- waitOnService(ctx, cl, sID, sName, rName)
+		}(serviceID, serviceName, recipeName)
 	}
 
 	for _, serviceID := range serviceIDs {
@@ -469,7 +476,7 @@ func getStackConfigs(ctx context.Context, dockerclient client.APIClient, namespa
 
 // https://github.com/docker/cli/blob/master/cli/command/service/helpers.go
 // https://github.com/docker/cli/blob/master/cli/command/service/progress/progress.go
-func WaitOnService(ctx context.Context, cl *dockerclient.Client, serviceID, appName string) error {
+func waitOnService(ctx context.Context, cl *dockerclient.Client, serviceID, serviceName, recipeName string) error {
 	errChan := make(chan error, 1)
 	pipeReader, pipeWriter := io.Pipe()
 
@@ -479,31 +486,27 @@ func WaitOnService(ctx context.Context, cl *dockerclient.Client, serviceID, appN
 
 	go io.Copy(ioutil.Discard, pipeReader)
 
-	timeout := 50 * time.Second
+	timeout := 60 * time.Second
 
 	select {
 	case err := <-errChan:
 		return err
 	case <-time.After(timeout):
 		return fmt.Errorf(fmt.Sprintf(`
-%s has not converged (%s second timeout reached).
+%s has still not converged (%s second timeout reached)
 
 This does not necessarily mean your deployment has failed, it may just be that
 the app is taking longer to deploy based on your server resources or network
-latency.
-
-You can track latest deployment status with:
-
-    abra app ps --watch %s
-
-And inspect the logs with:
+latency. Please run the following the inspect the logs of your deployed app:
 
     abra app logs %s
 
-If a service is failing to even start, try smoke out the error with:
+If a service is failing to even start (run "abra app ps %s" to see what
+services are running) there could be a few things. The follow command will
+try to smoke those out for you:
 
-    abra app errors --watch %s
+    abra app errors %s
 
-`, appName, timeout, appName, appName, appName))
+`, recipeName, timeout, recipeName, recipeName, recipeName))
 	}
 }

scripts/installer/installer

@@ -1,8 +1,8 @@
 #!/usr/bin/env bash
 
-ABRA_VERSION="0.3.0-alpha"
+ABRA_VERSION="0.4.0-alpha"
 ABRA_RELEASE_URL="https://git.coopcloud.tech/api/v1/repos/coop-cloud/abra/releases/tags/$ABRA_VERSION"
-RC_VERSION="0.4.0-alpha-rc7"
+RC_VERSION="0.4.0-alpha"
 RC_VERSION_URL="https://git.coopcloud.tech/api/v1/repos/coop-cloud/abra/releases/tags/$RC_VERSION"
 
 for arg in "$@"; do
@@ -37,28 +37,29 @@ function print_checksum_error {
 function install_abra_release {
   mkdir -p "$HOME/.local/bin"
 
-  if ! type "wget" > /dev/null 2>&1; then
-    echo "'wget' is not installed, cannot proceed..."
+  if ! type "curl" > /dev/null 2>&1; then
+    echo "'curl' is not installed, cannot proceed..."
     echo "perhaps try installing manually via the releases URL?"
     echo "https://git.coopcloud.tech/coop-cloud/abra/releases"
     exit 1
   fi
 
 
+  # FIXME: support different architectures
   PLATFORM=$(uname -s | tr '[:upper:]' '[:lower:]')_$(uname -m)
   FILENAME="abra_"$ABRA_VERSION"_"$PLATFORM""
   sed_command_rel='s/.*"assets":\[\{[^]]*"name":"'$FILENAME'"[^}]*"browser_download_url":"([^"]*)".*\].*/\1/p'
   sed_command_checksums='s/.*"assets":\[\{[^\]*"name":"checksums.txt"[^}]*"browser_download_url":"([^"]*)".*\].*/\1/p'
 
-  json=$(wget -q -O- $ABRA_RELEASE_URL)
+  json=$(curl -s $ABRA_RELEASE_URL)
   release_url=$(echo $json | sed -En $sed_command_rel)
   checksums_url=$(echo $json | sed -En $sed_command_checksums)
 
-  checksums=$(wget -q -O- $checksums_url)
+  checksums=$(curl -s $checksums_url)
   checksum=$(echo "$checksums" | grep "$FILENAME" - | sed -En 's/([0-9a-f]{64})\s+'"$FILENAME"'.*/\1/p')
 
   echo "downloading $ABRA_VERSION $PLATFORM binary release for abra..."
-  wget -q "$release_url" -O "$HOME/.local/bin/.abra-download"
+  curl --progress-bar "$release_url" --output "$HOME/.local/bin/.abra-download"
   localsum=$(sha256sum $HOME/.local/bin/.abra-download | sed -En 's/([0-9a-f]{64})\s+.*/\1/p')
   echo "checking if checksums match..."
   if [[ "$localsum" != "$checksum" ]]; then

tests/integration/.envrc.sample

@@ -1,4 +0,0 @@
-GANDI_TOKEN=...
-HCLOUD_TOKEN=...
-REGISTRY_PASSWORD=...
-REGISTRY_USERNAME=...

tests/integration/README.md

@@ -1,28 +0,0 @@
-# integration tests
-
-> You need to be a member of Autonomic Co-op to run these tests, sorry!
-
-`testfunctions.sh` contains the functions necessary to save and manipulate
-logs. Run `test_all.sh logdir` to run tests specified in that file and save the
-logs to `logdir`.
-
-When creating new tests, make sure the test command is a one-liner (you can use
-`;` to separate commands). Include `testfunctions.sh` and then write your tests
-like this:
-
-```
-run_test '$ABRA other stuff here'
-```
-
-By default, the testing script will ask after every command if the execution
-succeeded. If you reply `n`, it will log the test in the `logdir`. If you want
-all tests to run without questions, run `export logall=yes` before executing
-the test script.
-
-To run tests, you'll need to prepare your environment:
-
-```
-cp .envrc.sample .envrc # fill out values...
-direnv allow
-./test_all.sh logs
-```