chore: publish next release

Closes coop-cloud/organising#145.

.gitea/ISSUE_TEMPLATE.md

@@ -0,0 +1,8 @@
+---
+name: "Do not use this issue tracker"
+about: "Do not use this issue tracker"
+title: "Do not use this issue tracker"
+labels: []
+---
+
+Please report your issue on [`coop-cloud/organising`](https://git.coopcloud.tech/coop-cloud/organising)

.goreleaser.yml

@@ -22,6 +22,7 @@ archives:
       linux: Linux
       386: i386
       amd64: x86_64
+    format: binary
 checksum:
   name_template: "checksums.txt"
 snapshot:

README.md

@@ -38,6 +38,30 @@ make install
 
 The abra binary will be in `$GOPATH/bin`.
 
+## Autocompletion
+
+**bash**
+
+Copy `scripts/autocomplete/bash` into `/etc/bash_completion.d/` and rename
+it to abra.
+
+```
+sudo cp scripts/autocomplete/bash /etc/bash_completion.d/abra
+source /etc/bash_completion.d/abra
+```
+
+**(fi)zsh**
+
+(fi)zsh doesn't have an autocompletion folder by default but you can create one, then copy `scripts/autocomplete/zsh` into it and add a couple lines to your `~/.zshrc` or `~/.fizsh/.fizshrc`
+
+```
+sudo mkdir /etc/zsh/completion.d/
+sudo cp scripts/autocomplete/zsh /etc/zsh/completion.d/abra
+echo "PROG=abra\n_CLI_ZSH_AUTOCOMPLETE_HACK=1\nsource /etc/zsh/completion.d/abra" >> ~/.zshrc
+```
+
+(replace .zshrc with ~/.fizsh/.fizshrc if you use fizsh)
+
 ## Hacking
 
 Install direnv, run `cp .envrc.sample .envrc`, then run `direnv allow` in this directory. This will set coopcloud repos as private due to [this bug.](https://git.coopcloud.tech/coop-cloud/coopcloud.tech/issues/20#issuecomment-8201). Or you can run `go env -w GOPRIVATE=coopcloud.tech` but I'm not sure how persistent this is.
@@ -54,11 +78,19 @@ Please use the [conventional commit format](https://www.conventionalcommits.org/
 
 ## Versioning
 
-We use [goreleaser](https://goreleaser.com) to help us automate releases. We
+We use [goreleaser](https://goreleaser.com) to help us automate releases. We use [semver](https://semver.org) for versioning all releases of the tool. While we are still in the public alpha release phase, we will maintain a `0.y.z-alpha` format. Change logs are generated from our commit logs. We are still working this out and aim to refine our release praxis as we go.
-use [semver](https://semver.org) for versioning all releases of the tool. While
+
-we are still in the public alpha release phase, we will maintain a
+For developers, while using this `-alpha` format, the `y` part is the "major" version part. So, if you make breaking changes, you increment that and _not_ the `x` part. So, if you're on `0.1.0-alpha`, then you'd go to `0.1.1-alpha` for a backwards compatible change and `0.2.0-alpha` for a backwards incompatible change.
-`0.y.z-alpha` format. Change logs are generated from our commit logs. We are
+
-still working this out and aim to refine our release praxis as we go.
+## Making a new release
+
+- Change `ABRA_VERSION` to match the new tag in [`scripts`](./scripts/installer/installer) (use [semver](https://semver.org))
+- Commit that change (e.g. `git commit -m 'chore: publish next tag 0.3.1-alpha'`)
+- Make a new tag (e.g. `git tag 0.y.z-alpha`)
+- Push the new tag (e.g. `git push && git push --tags`)
+- Wait until the build finishes on [build.coopcloud.tech](https://build.coopcloud.tech/coop-cloud/abra)
+- Deploy the new installer script (e.g. `cd ./scripts/installer && make`)
+- Check the release worked, (e.g. `abra upgrade; abra version`)
 
 ## Fork maintenance
 

TODO.md

@@ -1,67 +0,0 @@
-# TODO
-
-## Bash feature parity
-
-- [ ] Commands
-  - [x] `abra server`
-    - [x] `ls`
-    - [x] `add`
-    - [x] `new`
-      - [x] `capsul`
-      - [x] `hetzner`
-    - [x] `rm`
-    - [x] `init`
-  - [ ] `abra app`
-    - [x] `ls`
-    - [x] `new`
-    - [x] `backup`
-    - [x] `deploy`
-    - [x] `check`
-    - [x] `version`
-    - [x] `config`
-    - [x] `cp`
-    - [x] `logs`
-    - [x] `ps`
-    - [x] `restore`
-    - [x] `rm`
-    - [x] `run`
-    - [ ] `rollback`
-    - [x] `secret`
-      - [x] `generate`
-      - [x] `insert`
-      - [x] `rm`
-      - [x] `ls`
-    - [x] `undeploy`
-    - [ ] `volume`
-      - [x] `ls` (WIP: knoflook)
-      - [ ] `rm` (WIP: knoflook)
-  - [x] `abra recipe`
-    - [x] `ls`
-    - [x] `create`
-    - [x] `upgrade`
-    - [x] `sync`
-    - [x] `versions`
-    - [x] `lint`
-  - [ ] `upgrade`
-  - [x] `version`
-
-## Next phase
-
-- [ ] Polishing UI/UX and testing
-- [ ] Refactoring and code organisation
-- [ ] Automated builds for releasing
-
-## New features
-
-- [ ] Commands
-  - [ ] `abra server`
-    - [ ] `dns`
-      - [ ] `gandi`
-  - [ ] `abra recipe`
-    - [ ] "TBD apps.json generating command" (see [#40](https://git.coopcloud.tech/coop-cloud/go-abra/issues/40))
-- [ ] Package manager integration
-  - [x] AUR
-  - [ ] Debian
-  - [ ] Ubuntu
-  - [ ] Fedora
-  - [ ] Homebrew

cli/app/backup.go

@@ -66,13 +66,22 @@ var appBackupCommand = &cli.Command{
 
 		sourceAndExec := fmt.Sprintf("%s; %s", sourceCmd, execCmd)
 		cmd := exec.Command("bash", "-c", sourceAndExec)
-		output, err := cmd.Output()
+		if err := internal.RunCmd(cmd); err != nil {
-		if err != nil {
 			logrus.Fatal(err)
 		}
 
-		fmt.Print(string(output))
-
 		return nil
 	},
+	BashComplete: func(c *cli.Context) {
+		appNames, err := config.GetAppNames()
+		if err != nil {
+			logrus.Warn(err)
+		}
+		if c.NArg() > 0 {
+			return
+		}
+		for _, a := range appNames {
+			fmt.Println(a)
+		}
+	},
 }

cli/app/check.go

@@ -1,6 +1,7 @@
 package app
 
 import (
+	"fmt"
 	"os"
 	"path"
 	"strings"
@@ -44,8 +45,20 @@ var appCheckCommand = &cli.Command{
 			logrus.Fatalf("%s is missing %s", app.Path, missingEnvVars)
 		}
 
-		logrus.Info("All necessary environment variables defined")
+		logrus.Infof("all necessary environment variables defined for '%s'", app.Name)
 
 		return nil
 	},
+	BashComplete: func(c *cli.Context) {
+		appNames, err := config.GetAppNames()
+		if err != nil {
+			logrus.Warn(err)
+		}
+		if c.NArg() > 0 {
+			return
+		}
+		for _, a := range appNames {
+			fmt.Println(a)
+		}
+	},
 }

cli/app/config.go

@@ -1,10 +1,12 @@
 package app
 
 import (
+	"fmt"
 	"os"
 	"os/exec"
 
 	"coopcloud.tech/abra/cli/internal"
+	"coopcloud.tech/abra/pkg/config"
 	"github.com/AlecAivazis/survey/v2"
 	"github.com/sirupsen/logrus"
 	"github.com/urfave/cli/v2"
@@ -38,4 +40,16 @@ var appConfigCommand = &cli.Command{
 
 		return nil
 	},
+	BashComplete: func(c *cli.Context) {
+		appNames, err := config.GetAppNames()
+		if err != nil {
+			logrus.Warn(err)
+		}
+		if c.NArg() > 0 {
+			return
+		}
+		for _, a := range appNames {
+			fmt.Println(a)
+		}
+	},
 }

cli/app/cp.go

@@ -1,11 +1,11 @@
 package app
 
 import (
-	"context"
 	"fmt"
 	"os"
 	"strings"
 
+	"coopcloud.tech/abra/cli/formatter"
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/client"
 	"coopcloud.tech/abra/pkg/config"
@@ -55,11 +55,13 @@ var appCpCommand = &cli.Command{
 			service = parsedSrc[0]
 			srcPath = parsedSrc[1]
 			dstPath = dst
+			logrus.Debugf("assuming transfer is coming FROM the container")
 		} else if len(parsedDst) == 2 {
 			service = parsedDst[0]
 			dstPath = parsedDst[1]
 			srcPath = src
 			isToContainer = true // <src> <container:dst>
+			logrus.Debugf("assuming transfer is going TO the container")
 		}
 
 		appFiles, err := config.LoadAppFiles("")
@@ -72,7 +74,6 @@ var appCpCommand = &cli.Command{
 			logrus.Fatal(err)
 		}
 
-		ctx := context.Background()
 		cl, err := client.New(app.Server)
 		if err != nil {
 			logrus.Fatal(err)
@@ -80,7 +81,7 @@ var appCpCommand = &cli.Command{
 
 		filters := filters.NewArgs()
 		filters.Add("name", fmt.Sprintf("%s_%s", appEnv.StackName(), service))
-		containers, err := cl.ContainerList(ctx, types.ContainerListOptions{Filters: filters})
+		containers, err := cl.ContainerList(c.Context, types.ContainerListOptions{Filters: filters})
 		if err != nil {
 			logrus.Fatal(err)
 		}
@@ -90,6 +91,8 @@ var appCpCommand = &cli.Command{
 		}
 		container := containers[0]
 
+		logrus.Debugf("retrieved '%s' as target container on '%s'", formatter.ShortenID(container.ID), app.Server)
+
 		if isToContainer {
 			if _, err := os.Stat(srcPath); err != nil {
 				logrus.Fatalf("'%s' does not exist?", srcPath)
@@ -102,11 +105,11 @@ var appCpCommand = &cli.Command{
 			}
 
 			copyOpts := types.CopyToContainerOptions{AllowOverwriteDirWithFile: false, CopyUIDGID: false}
-			if err := cl.CopyToContainer(ctx, container.ID, dstPath, content, copyOpts); err != nil {
+			if err := cl.CopyToContainer(c.Context, container.ID, dstPath, content, copyOpts); err != nil {
 				logrus.Fatal(err)
 			}
 		} else {
-			content, _, err := cl.CopyFromContainer(ctx, container.ID, srcPath)
+			content, _, err := cl.CopyFromContainer(c.Context, container.ID, srcPath)
 			if err != nil {
 				logrus.Fatal(err)
 			}

cli/app/deploy.go

@@ -54,4 +54,16 @@ var appDeployCommand = &cli.Command{
 
 		return nil
 	},
+	BashComplete: func(c *cli.Context) {
+		appNames, err := config.GetAppNames()
+		if err != nil {
+			logrus.Warn(err)
+		}
+		if c.NArg() > 0 {
+			return
+		}
+		for _, a := range appNames {
+			fmt.Println(a)
+		}
+	},
 }

cli/app/list.go

@@ -95,6 +95,7 @@ can take some time.
 		}
 
 		table.Render()
+
 		return nil
 	},
 }

cli/app/logs.go

@@ -1,7 +1,6 @@
 package app
 
 import (
-	"context"
 	"fmt"
 	"io"
 	"os"
@@ -9,6 +8,7 @@ import (
 
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/client"
+	"coopcloud.tech/abra/pkg/config"
 	"github.com/docker/docker/api/types"
 	"github.com/docker/docker/api/types/filters"
 	dockerClient "github.com/docker/docker/client"
@@ -17,12 +17,11 @@ import (
 )
 
 // stackLogs lists logs for all stack services
-func stackLogs(stackName string, client *dockerClient.Client) {
+func stackLogs(c *cli.Context, stackName string, client *dockerClient.Client) {
-	ctx := context.Background()
 	filters := filters.NewArgs()
 	filters.Add("name", stackName)
 	serviceOpts := types.ServiceListOptions{Filters: filters}
-	services, err := client.ServiceList(ctx, serviceOpts)
+	services, err := client.ServiceList(c.Context, serviceOpts)
 	if err != nil {
 		logrus.Fatal(err)
 	}
@@ -39,7 +38,7 @@ func stackLogs(stackName string, client *dockerClient.Client) {
 				Tail:       "20",
 				Timestamps: true,
 			}
-			logs, err := client.ServiceLogs(ctx, s, logOpts)
+			logs, err := client.ServiceLogs(c.Context, s, logOpts)
 			if err != nil {
 				logrus.Fatal(err)
 			}
@@ -64,7 +63,6 @@ var appLogsCommand = &cli.Command{
 	Action: func(c *cli.Context) error {
 		app := internal.ValidateApp(c)
 
-		ctx := context.Background()
 		cl, err := client.New(app.Server)
 		if err != nil {
 			logrus.Fatal(err)
@@ -72,14 +70,16 @@ var appLogsCommand = &cli.Command{
 
 		serviceName := c.Args().Get(1)
 		if serviceName == "" {
-			stackLogs(app.StackName(), cl)
+			logrus.Debug("tailing logs for all app services")
+			stackLogs(c, app.StackName(), cl)
 		}
+		logrus.Debugf("tailing logs for '%s'", serviceName)
 
 		service := fmt.Sprintf("%s_%s", app.StackName(), serviceName)
 		filters := filters.NewArgs()
 		filters.Add("name", service)
 		serviceOpts := types.ServiceListOptions{Filters: filters}
-		services, err := cl.ServiceList(ctx, serviceOpts)
+		services, err := cl.ServiceList(c.Context, serviceOpts)
 		if err != nil {
 			logrus.Fatal(err)
 		}
@@ -95,7 +95,7 @@ var appLogsCommand = &cli.Command{
 			Tail:       "20",
 			Timestamps: true,
 		}
-		logs, err := cl.ServiceLogs(ctx, services[0].ID, logOpts)
+		logs, err := cl.ServiceLogs(c.Context, services[0].ID, logOpts)
 		if err != nil {
 			logrus.Fatal(err)
 		}
@@ -109,4 +109,16 @@ var appLogsCommand = &cli.Command{
 
 		return nil
 	},
+	BashComplete: func(c *cli.Context) {
+		appNames, err := config.GetAppNames()
+		if err != nil {
+			logrus.Warn(err)
+		}
+		if c.NArg() > 0 {
+			return
+		}
+		for _, a := range appNames {
+			fmt.Println(a)
+		}
+	},
 }

cli/app/new.go

@@ -8,7 +8,6 @@ import (
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/catalogue"
 	"coopcloud.tech/abra/pkg/config"
-	recipePkg "coopcloud.tech/abra/pkg/recipe"
 	"coopcloud.tech/abra/pkg/secret"
 	"github.com/AlecAivazis/survey/v2"
 	"github.com/sirupsen/logrus"
@@ -32,7 +31,7 @@ var newAppServerFlag = &cli.StringFlag{
 	Aliases:     []string{"s"},
 	Value:       "",
 	Usage:       "Show apps of a specific server",
-	Destination: &listAppServer,
+	Destination: &newAppServer,
 }
 
 var newAppName string
@@ -78,25 +77,18 @@ var appNewCommand = &cli.Command{
 	},
 	ArgsUsage: "<recipe>",
 	Action:    action,
-}
+	BashComplete: func(c *cli.Context) {
-
+		catl, err := catalogue.ReadRecipeCatalogue()
-// getRecipeMeta retrieves the recipe metadata from the recipe catalogue.
+		if err != nil {
-func getRecipeMeta(recipeName string) (catalogue.RecipeMeta, error) {
+			logrus.Warn(err)
-	catl, err := catalogue.ReadRecipeCatalogue()
+		}
-	if err != nil {
+		if c.NArg() > 0 {
-		return catalogue.RecipeMeta{}, err
+			return
-	}
+		}
-
+		for name := range catl {
-	recipeMeta, ok := catl[recipeName]
+			fmt.Println(name)
-	if !ok {
+		}
-		err := fmt.Errorf("recipe '%s' does not exist?", recipeName)
+	},
-		return catalogue.RecipeMeta{}, err
-	}
-	if err := recipePkg.EnsureExists(recipeMeta.Name); err != nil {
-		return catalogue.RecipeMeta{}, err
-	}
-
-	return recipeMeta, nil
 }
 
 // ensureDomainFlag checks if the domain flag was used. if not, asks the user for it/
@@ -114,11 +106,11 @@ func ensureDomainFlag() error {
 
 // ensureServerFlag checks if the server flag was used. if not, asks the user for it.
 func ensureServerFlag() error {
-	appFiles, err := config.LoadAppFiles(newAppServer)
+	servers, err := config.GetServers()
 	if err != nil {
 		return err
 	}
-	servers := appFiles.GetServers()
+
 	if newAppServer == "" {
 		prompt := &survey.Select{
 			Message: "Select app server:",
@@ -128,6 +120,7 @@ func ensureServerFlag() error {
 			return err
 		}
 	}
+
 	return nil
 }
 
@@ -178,16 +171,6 @@ func action(c *cli.Context) error {
 		logrus.Fatal(err)
 	}
 
-	recipeMeta, err := getRecipeMeta(recipe.Name)
-	if err != nil {
-		logrus.Fatal(err)
-	}
-
-	latestVersion := recipeMeta.LatestVersion()
-	if err := recipePkg.EnsureVersion(recipe.Name, latestVersion); err != nil {
-		logrus.Fatal(err)
-	}
-
 	if err := ensureServerFlag(); err != nil {
 		logrus.Fatal(err)
 	}
@@ -204,8 +187,9 @@ func action(c *cli.Context) error {
 	if len(sanitisedAppName) > 45 {
 		logrus.Fatalf("'%s' cannot be longer than 45 characters", sanitisedAppName)
 	}
+	logrus.Debugf("'%s' sanitised as '%s' for new app", newAppName, sanitisedAppName)
 
-	if err := config.CopyAppEnvSample(recipe.Name, newAppName, newAppServer); err != nil {
+	if err := config.TemplateAppEnvSample(recipe.Name, newAppName, newAppServer, domain, recipe.Name); err != nil {
 		logrus.Fatal(err)
 	}
 
@@ -220,13 +204,27 @@ func action(c *cli.Context) error {
 		for secret := range secrets {
 			secretTable.Append([]string{secret, secrets[secret]})
 		}
-		defer secretTable.Render()
+
+		if len(secrets) > 0 {
+			defer secretTable.Render()
+		}
 	}
 
 	tableCol := []string{"Name", "Domain", "Type", "Server"}
 	table := abraFormatter.CreateTable(tableCol)
 	table.Append([]string{sanitisedAppName, domain, recipe.Name, newAppServer})
-	defer table.Render()
+
+	fmt.Println("")
+	fmt.Println(fmt.Sprintf("New '%s' created! Here is your new app overview:", recipe.Name))
+	fmt.Println("")
+	table.Render()
+	fmt.Println("")
+	fmt.Println("You can configure this app by running the following:")
+	fmt.Println(fmt.Sprintf("\n    abra app config %s", sanitisedAppName))
+	fmt.Println("")
+	fmt.Println("You can deploy this app by running the following:")
+	fmt.Println(fmt.Sprintf("\n    abra app deploy %s", sanitisedAppName))
+	fmt.Println("")
 
 	return nil
 }

cli/app/ps.go

@@ -1,12 +1,13 @@
 package app
 
 import (
-	"context"
+	"fmt"
 	"strings"
 
 	abraFormatter "coopcloud.tech/abra/cli/formatter"
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/client"
+	"coopcloud.tech/abra/pkg/config"
 	"github.com/docker/cli/cli/command/formatter"
 	"github.com/docker/docker/api/types"
 	"github.com/docker/docker/api/types/filters"
@@ -21,7 +22,6 @@ var appPsCommand = &cli.Command{
 	Action: func(c *cli.Context) error {
 		app := internal.ValidateApp(c)
 
-		ctx := context.Background()
 		cl, err := client.New(app.Server)
 		if err != nil {
 			logrus.Fatal(err)
@@ -30,12 +30,12 @@ var appPsCommand = &cli.Command{
 		filters := filters.NewArgs()
 		filters.Add("name", app.StackName())
 
-		containers, err := cl.ContainerList(ctx, types.ContainerListOptions{Filters: filters})
+		containers, err := cl.ContainerList(c.Context, types.ContainerListOptions{Filters: filters})
 		if err != nil {
 			logrus.Fatal(err)
 		}
 
-		tableCol := []string{"ID", "Image", "Command", "Created", "Status", "Ports", "Names"}
+		tableCol := []string{"id", "image", "command", "created", "status", "ports", "names"}
 		table := abraFormatter.CreateTable(tableCol)
 
 		for _, container := range containers {
@@ -46,7 +46,7 @@ var appPsCommand = &cli.Command{
 				abraFormatter.HumanDuration(container.Created),
 				container.Status,
 				formatter.DisplayablePorts(container.Ports),
-				strings.Join(container.Names, ","),
+				strings.Join(container.Names, ", "),
 			}
 			table.Append(tableRow)
 		}
@@ -54,4 +54,16 @@ var appPsCommand = &cli.Command{
 		table.Render()
 		return nil
 	},
+	BashComplete: func(c *cli.Context) {
+		appNames, err := config.GetAppNames()
+		if err != nil {
+			logrus.Warn(err)
+		}
+		if c.NArg() > 0 {
+			return
+		}
+		for _, a := range appNames {
+			fmt.Println(a)
+		}
+	},
 }

cli/app/remove.go

@@ -1,7 +1,6 @@
 package app
 
 import (
-	"context"
 	"fmt"
 	"os"
 
@@ -39,13 +38,13 @@ var appRemoveCommand = &cli.Command{
 		if !internal.Force {
 			response := false
 			prompt := &survey.Confirm{
-				Message: fmt.Sprintf("About to delete %s, are you sure", app.Name),
+				Message: fmt.Sprintf("about to delete %s, are you sure?", app.Name),
 			}
 			if err := survey.AskOne(prompt, &response); err != nil {
 				logrus.Fatal(err)
 			}
 			if !response {
-				logrus.Fatal("User aborted app removal")
+				logrus.Fatal("user aborted app removal")
 			}
 		}
 
@@ -54,7 +53,6 @@ var appRemoveCommand = &cli.Command{
 			logrus.Fatal(err)
 		}
 
-		ctx := context.Background()
 		cl, err := client.New(app.Server)
 		if err != nil {
 			logrus.Fatal(err)
@@ -72,7 +70,7 @@ var appRemoveCommand = &cli.Command{
 
 		fs := filters.NewArgs()
 		fs.Add("name", app.Name)
-		secretList, err := cl.SecretList(ctx, types.SecretListOptions{Filters: fs})
+		secretList, err := cl.SecretList(c.Context, types.SecretListOptions{Filters: fs})
 		if err != nil {
 			logrus.Fatal(err)
 		}
@@ -89,7 +87,7 @@ var appRemoveCommand = &cli.Command{
 			var secretNamesToRemove []string
 			if !internal.Force {
 				secretsPrompt := &survey.MultiSelect{
-					Message: "Which secrets do you want to remove?",
+					Message: "which secrets do you want to remove?",
 					Options: secretNames,
 					Default: secretNames,
 				}
@@ -99,17 +97,17 @@ var appRemoveCommand = &cli.Command{
 			}
 
 			for _, name := range secretNamesToRemove {
-				err := cl.SecretRemove(ctx, secrets[name])
+				err := cl.SecretRemove(c.Context, secrets[name])
 				if err != nil {
 					logrus.Fatal(err)
 				}
-				logrus.Info(fmt.Sprintf("Secret: %s removed", name))
+				logrus.Info(fmt.Sprintf("secret: %s removed", name))
 			}
 		} else {
-			logrus.Info("No secrets to remove")
+			logrus.Info("no secrets to remove")
 		}
 
-		volumeListOKBody, err := cl.VolumeList(ctx, fs)
+		volumeListOKBody, err := cl.VolumeList(c.Context, fs)
 		volumeList := volumeListOKBody.Volumes
 		if err != nil {
 			logrus.Fatal(err)
@@ -125,7 +123,7 @@ var appRemoveCommand = &cli.Command{
 				var removeVols []string
 				if !internal.Force {
 					volumesPrompt := &survey.MultiSelect{
-						Message: "Which volumes do you want to remove?",
+						Message: "which volumes do you want to remove?",
 						Options: vols,
 						Default: vols,
 					}
@@ -134,25 +132,37 @@ var appRemoveCommand = &cli.Command{
 					}
 				}
 				for _, vol := range removeVols {
-					err := cl.VolumeRemove(ctx, vol, internal.Force) // last argument is for force removing
+					err := cl.VolumeRemove(c.Context, vol, internal.Force) // last argument is for force removing
 					if err != nil {
 						logrus.Fatal(err)
 					}
-					logrus.Info(fmt.Sprintf("Volume %s removed", vol))
+					logrus.Info(fmt.Sprintf("volume %s removed", vol))
 				}
 			} else {
-				logrus.Info("No volumes were removed")
+				logrus.Info("no volumes were removed")
 			}
 		} else {
-			logrus.Info("No volumes to remove")
+			logrus.Info("no volumes to remove")
 		}
 
 		err = os.Remove(app.Path)
 		if err != nil {
 			logrus.Fatal(err)
 		}
-		logrus.Info(fmt.Sprintf("File: %s removed", app.Path))
+		logrus.Info(fmt.Sprintf("file: %s removed", app.Path))
 
 		return nil
 	},
+	BashComplete: func(c *cli.Context) {
+		appNames, err := config.GetAppNames()
+		if err != nil {
+			logrus.Warn(err)
+		}
+		if c.NArg() > 0 {
+			return
+		}
+		for _, a := range appNames {
+			fmt.Println(a)
+		}
+	},
 }

cli/app/restore.go

@@ -70,13 +70,10 @@ var appRestoreCommand = &cli.Command{
 
 		sourceAndExec := fmt.Sprintf("%s; %s", sourceCmd, execCmd)
 		cmd := exec.Command("bash", "-c", sourceAndExec)
-		output, err := cmd.Output()
+		if err := internal.RunCmd(cmd); err != nil {
-		if err != nil {
 			logrus.Fatal(err)
 		}
 
-		fmt.Print(string(output))
-
 		return nil
 	},
 }

cli/app/rollback.go

@@ -1,10 +1,82 @@
 package app
 
-import "github.com/urfave/cli/v2"
+import (
+	"fmt"
+
+	"context"
+
+	"coopcloud.tech/abra/pkg/config"
+
+	"coopcloud.tech/abra/cli/internal"
+	appPkg "coopcloud.tech/abra/pkg/app"
+	"coopcloud.tech/abra/pkg/catalogue"
+	"coopcloud.tech/abra/pkg/client"
+	"github.com/sirupsen/logrus"
+	"github.com/urfave/cli/v2"
+)
 
 var appRollbackCommand = &cli.Command{
 	Name:      "rollback",
 	Usage:     "Roll an app back to a previous version",
-	Aliases:   []string{"b"},
+	Aliases:   []string{"r"},
 	ArgsUsage: "[<version>]",
+	BashComplete: func(c *cli.Context) {
+		appNames, err := config.GetAppNames()
+		if err != nil {
+			logrus.Warn(err)
+		}
+		if c.NArg() > 0 {
+			return
+		}
+		for _, a := range appNames {
+			fmt.Println(a)
+		}
+	},
+	Action: func(c *cli.Context) error {
+		app := internal.ValidateApp(c)
+
+		ctx := context.Background()
+		cl, err := client.New(app.Server)
+		if err != nil {
+			logrus.Fatal(err)
+		}
+
+		recipeMeta, err := catalogue.GetRecipeMeta(app.Type)
+		if err != nil {
+			logrus.Fatal(err)
+		}
+		if len(recipeMeta.Versions) == 0 {
+			logrus.Fatalf("no catalogue versions available for '%s'", app.Type)
+		}
+
+		deployedVersions, isDeployed, err := appPkg.DeployedVersions(ctx, cl, app)
+		if err != nil {
+			logrus.Fatal(err)
+		}
+		if !isDeployed {
+			logrus.Fatalf("'%s' is not deployed?", app.Name)
+		}
+		if _, exists := deployedVersions["app"]; !exists {
+			logrus.Fatalf("no versioned 'app' service for '%s', cannot determine version", app.Name)
+		}
+
+		version := c.Args().Get(1)
+		if version == "" {
+			// TODO:
+			// using deployedVersions["app"], get index+1 version from catalogue
+			// otherwise bail out saying there is nothing to rollback to
+		} else {
+			// TODO
+			// ensure this version is listed in the catalogue
+			// ensure this version is "older" (lower down in the list)
+		}
+
+		// TODO
+		// display table of existing state and expected state and prompt
+		// run the deployment with this target version!
+
+		logrus.Fatal("command not implemented yet, coming soon TM")
+
+		return nil
+	},
 }

cli/app/run.go

@@ -1,13 +1,13 @@
 package app
 
 import (
-	"context"
 	"errors"
 	"fmt"
 
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/client"
 	"coopcloud.tech/abra/pkg/client/container"
+	"coopcloud.tech/abra/pkg/config"
 	"github.com/docker/cli/cli/command"
 	"github.com/docker/docker/api/types"
 	"github.com/docker/docker/api/types/filters"
@@ -42,10 +42,13 @@ var appRunCommand = &cli.Command{
 		app := internal.ValidateApp(c)
 
 		if c.Args().Len() < 2 {
-			internal.ShowSubcommandHelpAndError(c, errors.New("no <service> provided"))
+			internal.ShowSubcommandHelpAndError(c, errors.New("no <service> provided?"))
+		}
+
+		if c.Args().Len() < 3 {
+			internal.ShowSubcommandHelpAndError(c, errors.New("no <args> provided?"))
 		}
 
-		ctx := context.Background()
 		cl, err := client.New(app.Server)
 		if err != nil {
 			logrus.Fatal(err)
@@ -55,7 +58,7 @@ var appRunCommand = &cli.Command{
 		filters := filters.NewArgs()
 		filters.Add("name", fmt.Sprintf("%s_%s", app.StackName(), serviceName))
 
-		containers, err := cl.ContainerList(ctx, types.ContainerListOptions{Filters: filters})
+		containers, err := cl.ContainerList(c.Context, types.ContainerListOptions{Filters: filters})
 		if err != nil {
 			logrus.Fatal(err)
 		}
@@ -96,4 +99,25 @@ var appRunCommand = &cli.Command{
 
 		return nil
 	},
+	BashComplete: func(c *cli.Context) {
+		switch c.NArg() {
+		case 0:
+			appNames, err := config.GetAppNames()
+			if err != nil {
+				logrus.Warn(err)
+			}
+			for _, a := range appNames {
+				fmt.Println(a)
+			}
+		case 1:
+			appName := c.Args().First()
+			serviceNames, err := config.GetAppServiceNames(appName)
+			if err != nil {
+				logrus.Warn(err)
+			}
+			for _, s := range serviceNames {
+				fmt.Println(s)
+			}
+		}
+	},
 }

cli/app/secret.go

@@ -1,7 +1,6 @@
 package app
 
 import (
-	"context"
 	"errors"
 	"fmt"
 	"os"
@@ -10,6 +9,7 @@ import (
 	abraFormatter "coopcloud.tech/abra/cli/formatter"
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/client"
+	"coopcloud.tech/abra/pkg/config"
 	"coopcloud.tech/abra/pkg/secret"
 	"github.com/docker/docker/api/types"
 	"github.com/docker/docker/api/types/filters"
@@ -82,13 +82,13 @@ var appSecretGenerateCommand = &cli.Command{
 			os.Exit(1)
 		}
 
-		tableCol := []string{"Name", "Value"}
+		tableCol := []string{"name", "value"}
 		table := abraFormatter.CreateTable(tableCol)
 		for name, val := range secretVals {
 			table.Append([]string{name, val})
 		}
 		table.Render()
-		logrus.Warn("these secrets are not shown again, please take note of them *now*")
+		logrus.Warn("generated secrets are not shown again, please take note of them *now*")
 
 		return nil
 	},
@@ -143,7 +143,6 @@ var appSecretRmCommand = &cli.Command{
 			internal.ShowSubcommandHelpAndError(c, errors.New("no secret(s) specified?"))
 		}
 
-		ctx := context.Background()
 		cl, err := client.New(app.Server)
 		if err != nil {
 			logrus.Fatal(err)
@@ -151,7 +150,7 @@ var appSecretRmCommand = &cli.Command{
 
 		filters := filters.NewArgs()
 		filters.Add("name", app.StackName())
-		secretList, err := cl.SecretList(ctx, types.SecretListOptions{Filters: filters})
+		secretList, err := cl.SecretList(c.Context, types.SecretListOptions{Filters: filters})
 		if err != nil {
 			logrus.Fatal(err)
 		}
@@ -161,7 +160,7 @@ var appSecretRmCommand = &cli.Command{
 			secretName := cont.Spec.Annotations.Name
 			parsed := secret.ParseGeneratedSecretName(secretName, app)
 			if allSecrets {
-				if err := cl.SecretRemove(ctx, secretName); err != nil {
+				if err := cl.SecretRemove(c.Context, secretName); err != nil {
 					logrus.Fatal(err)
 				}
 				if internal.Pass {
@@ -171,7 +170,7 @@ var appSecretRmCommand = &cli.Command{
 				}
 			} else {
 				if parsed == secretToRm {
-					if err := cl.SecretRemove(ctx, secretName); err != nil {
+					if err := cl.SecretRemove(c.Context, secretName); err != nil {
 						logrus.Fatal(err)
 					}
 					if internal.Pass {
@@ -198,7 +197,6 @@ var appSecretLsCommand = &cli.Command{
 		tableCol := []string{"Name", "Version", "Generated Name", "Created On Server"}
 		table := abraFormatter.CreateTable(tableCol)
 
-		ctx := context.Background()
 		cl, err := client.New(app.Server)
 		if err != nil {
 			logrus.Fatal(err)
@@ -206,7 +204,7 @@ var appSecretLsCommand = &cli.Command{
 
 		filters := filters.NewArgs()
 		filters.Add("name", app.StackName())
-		secretList, err := cl.SecretList(ctx, types.SecretListOptions{Filters: filters})
+		secretList, err := cl.SecretList(c.Context, types.SecretListOptions{Filters: filters})
 		if err != nil {
 			logrus.Fatal(err)
 		}
@@ -234,6 +232,18 @@ var appSecretLsCommand = &cli.Command{
 		table.Render()
 		return nil
 	},
+	BashComplete: func(c *cli.Context) {
+		appNames, err := config.GetAppNames()
+		if err != nil {
+			logrus.Warn(err)
+		}
+		if c.NArg() > 0 {
+			return
+		}
+		for _, a := range appNames {
+			fmt.Println(a)
+		}
+	},
 }
 
 var appSecretCommand = &cli.Command{

cli/app/undeploy.go

@@ -1,11 +1,12 @@
 package app
 
 import (
-	"context"
+	"fmt"
 
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/client"
 	stack "coopcloud.tech/abra/pkg/client/stack"
+	"coopcloud.tech/abra/pkg/config"
 	"github.com/sirupsen/logrus"
 	"github.com/urfave/cli/v2"
 )
@@ -22,17 +23,28 @@ volumes as eligiblef or pruning once undeployed.
 	Action: func(c *cli.Context) error {
 		app := internal.ValidateApp(c)
 
-		ctx := context.Background()
 		cl, err := client.New(app.Server)
 		if err != nil {
 			logrus.Fatal(err)
 		}
 
 		rmOpts := stack.Remove{Namespaces: []string{app.StackName()}}
-		if err := stack.RunRemove(ctx, cl, rmOpts); err != nil {
+		if err := stack.RunRemove(c.Context, cl, rmOpts); err != nil {
 			logrus.Fatal(err)
 		}
 
 		return nil
 	},
+	BashComplete: func(c *cli.Context) {
+		appNames, err := config.GetAppNames()
+		if err != nil {
+			logrus.Warn(err)
+		}
+		if c.NArg() > 0 {
+			return
+		}
+		for _, a := range appNames {
+			fmt.Println(a)
+		}
+	},
 }

cli/app/version.go

@@ -7,6 +7,7 @@ import (
 
 	abraFormatter "coopcloud.tech/abra/cli/formatter"
 	"coopcloud.tech/abra/cli/internal"
+	appPkg "coopcloud.tech/abra/pkg/app"
 	"coopcloud.tech/abra/pkg/client/stack"
 	"coopcloud.tech/abra/pkg/config"
 	"github.com/docker/distribution/reference"
@@ -24,22 +25,10 @@ func getImagePath(image string) (string, error) {
 	if strings.Contains(path, "library") {
 		path = strings.Split(path, "/")[1]
 	}
+	logrus.Debugf("parsed '%s' from '%s'", path, image)
 	return path, nil
 }
 
-// parseVersionLabel parses a $STACK_NAME_$SERVICE_NAME service label
-func parseServiceName(label string) string {
-	idx := strings.LastIndex(label, "_")
-	return label[idx+1:]
-}
-
-// parseVersionLabel parses a $VERSION-$DIGEST service label
-func parseVersionLabel(label string) (string, string) {
-	// versions may look like v4.2-abcd or v4.2-alpine-abcd
-	idx := strings.LastIndex(label, "-")
-	return label[:idx], label[idx+1:]
-}
-
 var appVersionCommand = &cli.Command{
 	Name:    "version",
 	Aliases: []string{"v"},
@@ -65,14 +54,14 @@ var appVersionCommand = &cli.Command{
 			}(app.Server, label)
 		}
 
-		tableCol := []string{"Name", "Image", "Version", "Digest"}
+		tableCol := []string{"name", "image", "version", "digest"}
 		table := abraFormatter.CreateTable(tableCol)
 
 		statuses := make(map[string]stack.StackStatus)
 		for range compose.Services {
 			status := <-ch
 			if len(status.Services) > 0 {
-				serviceName := parseServiceName(status.Services[0].Spec.Name)
+				serviceName := appPkg.ParseServiceName(status.Services[0].Spec.Name)
 				statuses[serviceName] = status
 			}
 		}
@@ -85,7 +74,7 @@ var appVersionCommand = &cli.Command{
 			if status, ok := statuses[service.Name]; ok {
 				statusService := status.Services[0]
 				label := fmt.Sprintf("coop-cloud.%s.%s.version", app.StackName(), service.Name)
-				version, digest := parseVersionLabel(statusService.Spec.Labels[label])
+				version, digest := appPkg.ParseVersionLabel(statusService.Spec.Labels[label])
 				image, err := getImagePath(statusService.Spec.Labels["com.docker.stack.image"])
 				if err != nil {
 					logrus.Fatal(err)
@@ -104,4 +93,16 @@ var appVersionCommand = &cli.Command{
 		table.Render()
 		return nil
 	},
+	BashComplete: func(c *cli.Context) {
+		appNames, err := config.GetAppNames()
+		if err != nil {
+			logrus.Warn(err)
+		}
+		if c.NArg() > 0 {
+			return
+		}
+		for _, a := range appNames {
+			fmt.Println(a)
+		}
+	},
 }

cli/app/volume.go

@@ -1,11 +1,12 @@
 package app
 
 import (
-	"context"
+	"fmt"
 
 	abraFormatter "coopcloud.tech/abra/cli/formatter"
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/client"
+	"coopcloud.tech/abra/pkg/config"
 	"github.com/AlecAivazis/survey/v2"
 	"github.com/sirupsen/logrus"
 	"github.com/urfave/cli/v2"
@@ -13,18 +14,17 @@ import (
 
 var appVolumeListCommand = &cli.Command{
 	Name:    "list",
-	Usage:   "list volumes associated with an app",
+	Usage:   "List volumes associated with an app",
 	Aliases: []string{"ls"},
 	Action: func(c *cli.Context) error {
 		app := internal.ValidateApp(c)
 
-		ctx := context.Background()
+		volumeList, err := client.GetVolumes(c.Context, app.Server, app.Name)
-		volumeList, err := client.GetVolumes(ctx, app.Server, app.Name)
 		if err != nil {
 			logrus.Fatal(err)
 		}
 
-		table := abraFormatter.CreateTable([]string{"DRIVER", "VOLUME NAME"})
+		table := abraFormatter.CreateTable([]string{"driver", "volume name"})
 		var volTable [][]string
 		for _, volume := range volumeList {
 			volRow := []string{
@@ -43,7 +43,7 @@ var appVolumeListCommand = &cli.Command{
 
 var appVolumeRemoveCommand = &cli.Command{
 	Name:    "remove",
-	Usage:   "remove volume(s) associated with an app",
+	Usage:   "Remove volume(s) associated with an app",
 	Aliases: []string{"rm"},
 	Flags: []cli.Flag{
 		internal.ForceFlag,
@@ -51,8 +51,7 @@ var appVolumeRemoveCommand = &cli.Command{
 	Action: func(c *cli.Context) error {
 		app := internal.ValidateApp(c)
 
-		ctx := context.Background()
+		volumeList, err := client.GetVolumes(c.Context, app.Server, app.Name)
-		volumeList, err := client.GetVolumes(ctx, app.Server, app.Name)
 		if err != nil {
 			logrus.Fatal(err)
 		}
@@ -61,7 +60,7 @@ var appVolumeRemoveCommand = &cli.Command{
 		var volumesToRemove []string
 		if !internal.Force {
 			volumesPrompt := &survey.MultiSelect{
-				Message: "Which volumes do you want to remove?",
+				Message: "which volumes do you want to remove?",
 				Options: volumeNames,
 				Default: volumeNames,
 			}
@@ -72,15 +71,27 @@ var appVolumeRemoveCommand = &cli.Command{
 			volumesToRemove = volumeNames
 		}
 
-		err = client.RemoveVolumes(ctx, app.Server, volumesToRemove, internal.Force)
+		err = client.RemoveVolumes(c.Context, app.Server, volumesToRemove, internal.Force)
 		if err != nil {
 			logrus.Fatal(err)
 		}
 
-		logrus.Info("Volumes removed successfully.")
+		logrus.Info("volumes removed successfully")
 
 		return nil
 	},
+	BashComplete: func(c *cli.Context) {
+		appNames, err := config.GetAppNames()
+		if err != nil {
+			logrus.Warn(err)
+		}
+		if c.NArg() > 0 {
+			return
+		}
+		for _, a := range appNames {
+			fmt.Println(a)
+		}
+	},
 }
 
 var appVolumeCommand = &cli.Command{

cli/catalogue/catalogue.go

@@ -0,0 +1,17 @@
+package catalogue
+
+import (
+	"github.com/urfave/cli/v2"
+)
+
+// CatalogueCommand defines the `abra catalogue` command and sub-commands.
+var CatalogueCommand = &cli.Command{
+	Name:        "catalogue",
+	Usage:       "Manage the recipe catalogue",
+	Aliases:     []string{"c"},
+	ArgsUsage:   "<recipe>",
+	Description: "This command helps recipe packagers interact with the recipe catalogue",
+	Subcommands: []*cli.Command{
+		catalogueGenerateCommand,
+	},
+}

cli/catalogue/generate.go

@@ -0,0 +1,132 @@
+package catalogue
+
+import (
+	"encoding/json"
+	"io/ioutil"
+	"path"
+
+	"coopcloud.tech/abra/cli/formatter"
+	"coopcloud.tech/abra/pkg/catalogue"
+	"coopcloud.tech/abra/pkg/config"
+	"coopcloud.tech/abra/pkg/git"
+	"github.com/sirupsen/logrus"
+	"github.com/urfave/cli/v2"
+)
+
+// CatalogueSkipList is all the repos that are not recipes.
+var CatalogueSkipList = map[string]bool{
+	"abra":                  true,
+	"abra-bash":             true,
+	"abra-apps":             true,
+	"abra-aur":              true,
+	"abra-capsul":           true,
+	"abra-gandi":            true,
+	"abra-hetzner":          true,
+	"apps":                  true,
+	"aur-abra-git":          true,
+	"auto-apps-json":        true,
+	"auto-mirror":           true,
+	"backup-bot":            true,
+	"coopcloud.tech":        true,
+	"coturn":                true,
+	"docker-cp-deploy":      true,
+	"docker-dind-bats-kcov": true,
+	"docs.coopcloud.tech":   true,
+	"example":               true,
+	"gardening":             true,
+	"go-abra":               true,
+	"organising":            true,
+	"pyabra":                true,
+	"radicle-seed-node":     true,
+	"stack-ssh-deploy":      true,
+	"swarm-cronjob":         true,
+	"tagcmp":                true,
+	"tyop":                  true,
+}
+
+var catalogueGenerateCommand = &cli.Command{
+	Name:         "generate",
+	Aliases:      []string{"g"},
+	Usage:        "Generate a new copy of the catalogue",
+	ArgsUsage:    "[<recipe>]",
+	BashComplete: func(c *cli.Context) {},
+	Action: func(c *cli.Context) error {
+		recipeName := c.Args().First()
+
+		repos, err := catalogue.ReadReposMetadata()
+		if err != nil {
+			logrus.Fatal(err)
+		}
+
+		logrus.Debugf("ensuring '%v' recipe(s) are locally present and up-to-date", len(repos))
+
+		bar := formatter.CreateProgressbar(len(repos), "retrieving recipes...")
+		ch := make(chan string, len(repos))
+		for _, repoMeta := range repos {
+			go func(rm catalogue.RepoMeta) {
+				if recipeName != "" && recipeName != rm.Name {
+					ch <- rm.Name
+					bar.Add(1)
+					return
+				}
+				if _, exists := CatalogueSkipList[rm.Name]; exists {
+					ch <- rm.Name
+					bar.Add(1)
+					return
+				}
+
+				recipeDir := path.Join(config.ABRA_DIR, "apps", rm.Name)
+
+				if err := git.Clone(recipeDir, rm.SSHURL); err != nil {
+					logrus.Fatal(err)
+				}
+
+				if err := git.EnsureUpToDate(recipeDir); err != nil {
+					logrus.Fatal(err)
+				}
+
+				ch <- rm.Name
+				bar.Add(1)
+			}(repoMeta)
+		}
+
+		for range repos {
+			<-ch // wait for everything
+		}
+
+		catl := make(catalogue.RecipeCatalogue)
+		for _, recipeMeta := range repos {
+			if recipeName != "" && recipeName != recipeMeta.Name {
+				continue
+			}
+			if _, exists := CatalogueSkipList[recipeMeta.Name]; exists {
+				continue
+			}
+
+			catl[recipeMeta.Name] = catalogue.RecipeMeta{
+				Name:          recipeMeta.Name,
+				Repository:    recipeMeta.CloneURL,
+				Icon:          recipeMeta.AvatarURL,
+				DefaultBranch: recipeMeta.DefaultBranch,
+				Description:   recipeMeta.Description,
+				Website:       recipeMeta.Website,
+				// Versions:      ..., // FIXME: once the new versions work goes down
+				// Category:      ..., // FIXME: once we sort out the machine-readable catalogue interface
+				// Features:      ..., // FIXME: once we figure out the machine-readable catalogue interface
+			}
+		}
+
+		recipesJSON, err := json.MarshalIndent(catl, "", "    ")
+		if err != nil {
+			logrus.Fatal(err)
+		}
+
+		if err := ioutil.WriteFile(config.APPS_JSON, recipesJSON, 0644); err != nil {
+			logrus.Fatal(err)
+		}
+
+		logrus.Debugf("generated new recipe catalogue in '%s'", config.APPS_JSON)
+
+		return nil
+	},
+}

cli/cli.go

@@ -6,6 +6,7 @@ import (
 	"os"
 
 	"coopcloud.tech/abra/cli/app"
+	"coopcloud.tech/abra/cli/catalogue"
 	"coopcloud.tech/abra/cli/recipe"
 	"coopcloud.tech/abra/cli/server"
 	"github.com/sirupsen/logrus"
@@ -54,20 +55,33 @@ func RunApp(version, commit string) {
 			app.AppCommand,
 			server.ServerCommand,
 			recipe.RecipeCommand,
+			catalogue.CatalogueCommand,
 			VersionCommand,
+			UpgradeCommand,
 		},
 		Flags: []cli.Flag{
 			VerboseFlag,
 			DebugFlag,
 		},
 		Authors: []*cli.Author{
-			&cli.Author{
+			{
 				Name:  "Autonomic Co-op",
 				Email: "helo@autonomic.zone",
 			},
 		},
 	}
 
+	app.EnableBashCompletion = true
+
+	app.Before = func(c *cli.Context) error {
+		if Debug {
+			logrus.SetLevel(logrus.DebugLevel)
+		}
+		return nil
+	}
+
+	logrus.Debugf("Flying abra version '%s', commit '%s', enjoy the ride", version, commit)
+
 	if err := app.Run(os.Args); err != nil {
 		logrus.Fatal(err)
 	}

cli/formatter/formatter.go

@@ -9,6 +9,7 @@ import (
 	"github.com/docker/cli/cli/command/formatter"
 	"github.com/docker/go-units"
 	"github.com/olekukonko/tablewriter"
+	"github.com/schollz/progressbar/v3"
 )
 
 func ShortenID(str string) string {
@@ -31,8 +32,21 @@ func HumanDuration(timestamp int64) string {
 	return units.HumanDuration(now.Sub(date)) + " ago"
 }
 
+// CreateTable prepares a table layout for output.
 func CreateTable(columns []string) *tablewriter.Table {
 	table := tablewriter.NewWriter(os.Stdout)
 	table.SetHeader(columns)
 	return table
 }
+
+// CreateProgressbar generates a progress bar
+func CreateProgressbar(length int, title string) *progressbar.ProgressBar {
+	return progressbar.NewOptions(
+		length,
+		progressbar.OptionClearOnFinish(),
+		progressbar.OptionSetPredictTime(false),
+		progressbar.OptionShowCount(),
+		progressbar.OptionFullWidth(),
+		progressbar.OptionSetDescription(title),
+	)
+}

cli/internal/command.go

@@ -0,0 +1,39 @@
+package internal
+
+import (
+	"bufio"
+	"fmt"
+	"os/exec"
+)
+
+// RunCmd runs a shell command and streams stdout/stderr in real-time.
+func RunCmd(cmd *exec.Cmd) error {
+	r, err := cmd.StdoutPipe()
+	if err != nil {
+		return err
+	}
+
+	cmd.Stderr = cmd.Stdout
+	done := make(chan struct{})
+	scanner := bufio.NewScanner(r)
+
+	go func() {
+		for scanner.Scan() {
+			line := scanner.Text()
+			fmt.Println(line)
+		}
+		done <- struct{}{}
+	}()
+
+	if err := cmd.Start(); err != nil {
+		return err
+	}
+
+	<-done
+
+	if err := cmd.Wait(); err != nil {
+		return err
+	}
+
+	return nil
+}

cli/internal/validate.go

@@ -2,7 +2,6 @@ package internal
 
 import (
 	"errors"
-	"os"
 
 	"coopcloud.tech/abra/pkg/app"
 	"coopcloud.tech/abra/pkg/config"
@@ -22,9 +21,10 @@ func ValidateRecipe(c *cli.Context) recipe.Recipe {
 	recipe, err := recipe.Get(recipeName)
 	if err != nil {
 		logrus.Fatal(err)
-		os.Exit(1)
 	}
 
+	logrus.Debugf("validated '%s' as recipe argument", recipeName)
+
 	return recipe
 }
 
@@ -39,8 +39,22 @@ func ValidateApp(c *cli.Context) config.App {
 	app, err := app.Get(appName)
 	if err != nil {
 		logrus.Fatal(err)
-		os.Exit(1)
 	}
 
+	logrus.Debugf("validated '%s' as app argument", appName)
+
 	return app
 }
+
+// ValidateDomain ensures the domain name arg is valid.
+func ValidateDomain(c *cli.Context) string {
+	domainName := c.Args().First()
+
+	if domainName == "" {
+		ShowSubcommandHelpAndError(c, errors.New("no domain provided"))
+	}
+
+	logrus.Debugf("validated '%s' as domain argument", domainName)
+
+	return domainName
+}

cli/recipe/lint.go

@@ -7,6 +7,7 @@ import (
 
 	"coopcloud.tech/abra/cli/formatter"
 	"coopcloud.tech/abra/cli/internal"
+	"coopcloud.tech/abra/pkg/catalogue"
 	"coopcloud.tech/abra/pkg/config"
 	"coopcloud.tech/tagcmp"
 	"github.com/docker/distribution/reference"
@@ -76,18 +77,30 @@ var recipeLintCommand = &cli.Command{
 			}
 		}
 
-		tableCol := []string{"Rule", "Satisfied"}
+		tableCol := []string{"rule", "satisfied"}
 		table := formatter.CreateTable(tableCol)
-		table.Append([]string{"Compose files have the expected version", strconv.FormatBool(expectedVersion)})
+		table.Append([]string{"compose files have the expected version", strconv.FormatBool(expectedVersion)})
-		table.Append([]string{"Environment configuration is provided", strconv.FormatBool(envSampleProvided)})
+		table.Append([]string{"environment configuration is provided", strconv.FormatBool(envSampleProvided)})
-		table.Append([]string{"Recipe contains a service named 'app'", strconv.FormatBool(serviceNamedApp)})
+		table.Append([]string{"recipe contains a service named 'app'", strconv.FormatBool(serviceNamedApp)})
-		table.Append([]string{"Traefik routing enabled on at least one service", strconv.FormatBool(traefikEnabled)})
+		table.Append([]string{"traefik routing enabled on at least one service", strconv.FormatBool(traefikEnabled)})
-		table.Append([]string{"All services have a healthcheck enabled", strconv.FormatBool(healthChecksForAllServices)})
+		table.Append([]string{"all services have a healthcheck enabled", strconv.FormatBool(healthChecksForAllServices)})
-		table.Append([]string{"All images are using a tag", strconv.FormatBool(allImagesTagged)})
+		table.Append([]string{"all images are using a tag", strconv.FormatBool(allImagesTagged)})
-		table.Append([]string{"No usage of unstable 'latest' tags", strconv.FormatBool(noUnstableTags)})
+		table.Append([]string{"no usage of unstable 'latest' tags", strconv.FormatBool(noUnstableTags)})
-		table.Append([]string{"All tags are using a semver-like format", strconv.FormatBool(semverLikeTags)})
+		table.Append([]string{"all tags are using a semver-like format", strconv.FormatBool(semverLikeTags)})
 		table.Render()
 
 		return nil
 	},
+	BashComplete: func(c *cli.Context) {
+		catl, err := catalogue.ReadRecipeCatalogue()
+		if err != nil {
+			logrus.Warn(err)
+		}
+		if c.NArg() > 0 {
+			return
+		}
+		for name := range catl {
+			fmt.Println(name)
+		}
+	},
 }

cli/recipe/list.go

@@ -23,7 +23,7 @@ var recipeListCommand = &cli.Command{
 		recipes := catl.Flatten()
 		sort.Sort(catalogue.ByRecipeName(recipes))
 
-		tableCol := []string{"Name", "Category", "Status"}
+		tableCol := []string{"name", "category", "status"}
 		table := formatter.CreateTable(tableCol)
 
 		for _, recipe := range recipes {

cli/recipe/new.go

@@ -8,7 +8,7 @@ import (
 
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/config"
-	"github.com/go-git/go-git/v5"
+	"coopcloud.tech/abra/pkg/git"
 	"github.com/sirupsen/logrus"
 	"github.com/urfave/cli/v2"
 )
@@ -28,10 +28,8 @@ var recipeNewCommand = &cli.Command{
 		}
 
 		url := fmt.Sprintf("%s/example.git", config.REPOS_BASE_URL)
-		_, err := git.PlainClone(directory, false, &git.CloneOptions{URL: url, Tags: git.AllTags})
+		if err := git.Clone(directory, url); err != nil {
-		if err != nil {
+			return err
-			logrus.Fatal(err)
-			return nil
 		}
 
 		gitRepo := path.Join(config.APPS_DIR, recipe.Name, ".git")
@@ -39,6 +37,7 @@ var recipeNewCommand = &cli.Command{
 			logrus.Fatal(err)
 			return nil
 		}
+		logrus.Debugf("removed git repo in '%s'", gitRepo)
 
 		toParse := []string{
 			path.Join(config.APPS_DIR, recipe.Name, "README.md"),
@@ -71,7 +70,7 @@ var recipeNewCommand = &cli.Command{
 		}
 
 		logrus.Infof(
-			"New recipe '%s' created in %s, happy hacking!\n",
+			"new recipe '%s' created in %s, happy hacking!\n",
 			recipe.Name, path.Join(config.APPS_DIR, recipe.Name),
 		)
 

cli/recipe/sync.go

@@ -36,7 +36,7 @@ the versioning metadata of up-and-running containers are.
 		}
 
 		if !hasAppService {
-			logrus.Fatal(fmt.Sprintf("No 'app' service defined in '%s', cannot proceed", recipe.Name))
+			logrus.Fatal(fmt.Sprintf("no 'app' service defined in '%s'", recipe.Name))
 		}
 
 		for _, service := range recipe.Config.Services {
@@ -44,17 +44,20 @@ the versioning metadata of up-and-running containers are.
 			if err != nil {
 				logrus.Fatal(err)
 			}
+			logrus.Debugf("detected image '%s' for service '%s'", img, service.Name)
 
 			digest, err := client.GetTagDigest(img)
 			if err != nil {
 				logrus.Fatal(err)
 			}
+			logrus.Debugf("retrieved digest '%s' for '%s'", digest, img)
 
 			tag := img.(reference.NamedTagged).Tag()
 			label := fmt.Sprintf("coop-cloud.${STACK_NAME}.%s.version=%s-%s", service.Name, tag, digest)
 			if err := recipe.UpdateLabel(service.Name, label); err != nil {
 				logrus.Fatal(err)
 			}
+			logrus.Debugf("added label '%s' to service '%s'", label, service.Name)
 		}
 
 		return nil

cli/recipe/upgrade.go

@@ -40,6 +40,7 @@ This is step 1 of upgrading a recipe. Step 2 is running "abra recipe sync
 			if err != nil {
 				logrus.Fatal(err)
 			}
+			logrus.Debugf("read '%s' from the recipe catalogue for '%s'", catlVersions, service.Name)
 
 			img, err := reference.ParseNormalizedNamed(service.Image)
 			if err != nil {
@@ -51,6 +52,7 @@ This is step 1 of upgrading a recipe. Step 2 is running "abra recipe sync
 			if err != nil {
 				logrus.Fatal(err)
 			}
+			logrus.Debugf("retrieved '%s' from remote registry for '%s'", regVersions, image)
 
 			if strings.Contains(image, "library") {
 				// ParseNormalizedNamed prepends 'library' to images like nginx:<tag>,
@@ -61,6 +63,7 @@ This is step 1 of upgrading a recipe. Step 2 is running "abra recipe sync
 
 			semverLikeTag := true
 			if !tagcmp.IsParsable(img.(reference.NamedTagged).Tag()) {
+				logrus.Debugf("'%s' not considered semver-like", img.(reference.NamedTagged).Tag())
 				semverLikeTag = false
 			}
 
@@ -68,6 +71,7 @@ This is step 1 of upgrading a recipe. Step 2 is running "abra recipe sync
 			if err != nil && semverLikeTag {
 				logrus.Fatal(err)
 			}
+			logrus.Debugf("parsed '%s' for '%s'", tag, service.Name)
 
 			var compatible []tagcmp.Tag
 			for _, regVersion := range regVersions {
@@ -81,10 +85,12 @@ This is step 1 of upgrading a recipe. Step 2 is running "abra recipe sync
 				}
 			}
 
+			logrus.Debugf("detected potential upgradable tags '%s' for '%s'", compatible, service.Name)
+
 			sort.Sort(tagcmp.ByTagDesc(compatible))
 
 			if len(compatible) == 0 && semverLikeTag {
-				logrus.Info(fmt.Sprintf("No new versions available for '%s', '%s' is the latest", image, tag))
+				logrus.Info(fmt.Sprintf("no new versions available for '%s', '%s' is the latest", image, tag))
 				continue // skip on to the next tag and don't update any compose files
 			}
 
@@ -101,11 +107,13 @@ This is step 1 of upgrading a recipe. Step 2 is running "abra recipe sync
 				}
 			}
 
-			msg := fmt.Sprintf("Which tag would you like to upgrade to? (service: %s, tag: %s)", service.Name, tag)
+			logrus.Debugf("detected compatible upgradable tags '%s' for '%s'", compatibleStrings, service.Name)
+
+			msg := fmt.Sprintf("upgrade to which tag? (service: %s, tag: %s)", service.Name, tag)
 			if !tagcmp.IsParsable(img.(reference.NamedTagged).Tag()) {
 				tag := img.(reference.NamedTagged).Tag()
-				logrus.Warning(fmt.Sprintf("Unable to determine versioning semantics of '%s', listing all tags...", tag))
+				logrus.Warning(fmt.Sprintf("unable to determine versioning semantics of '%s', listing all tags", tag))
-				msg = fmt.Sprintf("Which tag would you like to upgrade to? (service: %s, tag: %s)", service.Name, tag)
+				msg = fmt.Sprintf("upgrade to which tag? (service: %s, tag: %s)", service.Name, tag)
 				compatibleStrings = []string{}
 				for _, regVersion := range regVersions {
 					compatibleStrings = append(compatibleStrings, regVersion.Name)
@@ -124,6 +132,7 @@ This is step 1 of upgrading a recipe. Step 2 is running "abra recipe sync
 			if err := recipe.UpdateTag(image, upgradeTag); err != nil {
 				logrus.Fatal(err)
 			}
+			logrus.Debugf("tag updated from '%s' to '%s' for '%s'", image, upgradeTag, recipe.Name)
 		}
 
 		return nil

cli/recipe/version.go

@@ -26,13 +26,13 @@ var recipeVersionCommand = &cli.Command{
 			logrus.Fatalf("'%s' recipe doesn't exist?", recipe.Name)
 		}
 
-		tableCol := []string{"Version", "Service", "Image", "Digest"}
+		tableCol := []string{"Version", "Service", "Image", "Tag", "Digest"}
 		table := formatter.CreateTable(tableCol)
 
 		for _, serviceVersion := range recipeMeta.Versions {
 			for tag, meta := range serviceVersion {
 				for service, serviceMeta := range meta {
-					table.Append([]string{tag, service, serviceMeta.Image, serviceMeta.Digest})
+					table.Append([]string{tag, service, serviceMeta.Image, serviceMeta.Tag, serviceMeta.Digest})
 				}
 			}
 		}

cli/server/add.go

@@ -1,29 +1,96 @@
 package server
 
 import (
-	"fmt"
+	"context"
+	"os/user"
+	"strings"
 
+	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/client"
 	"github.com/sirupsen/logrus"
 	"github.com/urfave/cli/v2"
 )
 
 var serverAddCommand = &cli.Command{
-	Name:        "add",
+	Name:  "add",
-	Usage:       "Add a new server, reachable on <server>.",
+	Usage: "Add a new server",
-	Aliases:     []string{"a"},
+	Description: `
-	ArgsUsage:   "<server> [<user>] [<port>]",
+This command adds a new server that abra will communicate with, to deploy apps.
-	Description: "[<user>], [<port>]    SSH connection details",
+
+The <domain> argument must be a publicy accessible domain name which points to
+your server. You should have SSH access to this server, Abra will assume port
+22 and will use your current system username to make an initial connection. You
+can use the <user> and <port> arguments to adjust this.
+
+For example:
+
+    abra server add varia.zone glodemodem 12345
+
+Abra will construct the following SSH connection string then:
+
+    ssh://globemodem@varia.zone:12345
+
+All communication between Abra and the server will use this SSH connection.
+
+`,
+	Aliases:   []string{"a"},
+	ArgsUsage: "<domain> [<user>] [<port>]",
 	Action: func(c *cli.Context) error {
-		argLen := c.Args().Len()
+		domainName := internal.ValidateDomain(c)
-		args := c.Args().Slice()
+
-		if argLen < 3 {
+		var username string
-			args = append(args, make([]string, 3-argLen)...)
+		var port string
+
+		username = c.Args().Get(1)
+		if username == "" {
+			systemUser, err := user.Current()
+			if err != nil {
+				logrus.Fatal(err)
+			}
+			username = systemUser.Username
 		}
-		if err := client.CreateContext(args[0], args[1], args[2]); err != nil {
+
+		port = c.Args().Get(2)
+		if port == "" {
+			port = "22"
+		}
+
+		store := client.NewDefaultDockerContextStore()
+		contexts, err := store.Store.List()
+		if err != nil {
 			logrus.Fatal(err)
 		}
-		fmt.Println(args[0])
+
+		for _, context := range contexts {
+			if context.Name == domainName {
+				logrus.Fatalf("server at '%s' already exists?", domainName)
+			}
+		}
+
+		logrus.Debugf("creating context with domain '%s', username '%s' and port '%s'", domainName, username, port)
+
+		if err := client.CreateContext(domainName, username, port); err != nil {
+			logrus.Fatal(err)
+		}
+
+		ctx := context.Background()
+		cl, err := client.New(domainName)
+		if err != nil {
+			logrus.Fatal(err)
+		}
+
+		if _, err := cl.Info(ctx); err != nil {
+			if strings.Contains(err.Error(), "command not found") {
+				logrus.Fatalf("docker is not installed on '%s'?", domainName)
+			} else {
+				logrus.Fatalf("unable to make a connection to '%s'?", domainName)
+			}
+			logrus.Debug(err)
+		}
+
+		logrus.Debugf("remote connection to '%s' is definitely up", domainName)
+		logrus.Infof("server at '%s' has been added", domainName)
+
 		return nil
 	},
 }

cli/server/init.go

@@ -2,7 +2,6 @@ package server
 
 import (
 	"context"
-	"errors"
 	"fmt"
 	"net"
 	"time"
@@ -20,21 +19,18 @@ var serverInitCommand = &cli.Command{
 	Usage:     "Initialise server for deploying apps",
 	Aliases:   []string{"i"},
 	HideHelp:  true,
-	ArgsUsage: "<server>",
+	ArgsUsage: "<domain>",
 	Description: `
-Initialise swarm mode on the target <server>.
+Initialise swarm mode on the target <domain>.
 
 This initialisation explicitly chooses the "single host swarm" mode which uses
 the default IPv4 address as the advertising address. This can be re-configured
 later for more advanced use cases.
 `,
 	Action: func(c *cli.Context) error {
-		server := c.Args().First()
+		domainName := internal.ValidateDomain(c)
-		if server == "" {
-			internal.ShowSubcommandHelpAndError(c, errors.New("no server provided"))
-		}
 
-		cl, err := client.New(server)
+		cl, err := client.New(domainName)
 		if err != nil {
 			return err
 		}
@@ -49,15 +45,16 @@ later for more advanced use cases.
 				return d.DialContext(ctx, "udp", "95.216.24.230:53")
 			},
 		}
+		logrus.Debugf("created DNS resolver via 95.216.24.230")
 
 		ctx := context.Background()
-		ips, err := resolver.LookupIPAddr(ctx, server)
+		ips, err := resolver.LookupIPAddr(ctx, domainName)
 		if err != nil {
 			logrus.Fatal(err)
 		}
 
 		if len(ips) == 0 {
-			return fmt.Errorf("unable to retrieve ipv4 address for %s", server)
+			return fmt.Errorf("unable to retrieve ipv4 address for %s", domainName)
 		}
 		ipv4 := ips[0].IP.To4().String()
 
@@ -68,11 +65,13 @@ later for more advanced use cases.
 		if _, err := cl.SwarmInit(ctx, initReq); err != nil {
 			return err
 		}
+		logrus.Debugf("initialised swarm on '%s'", domainName)
 
 		netOpts := types.NetworkCreate{Driver: "overlay", Scope: "swarm"}
 		if _, err := cl.NetworkCreate(ctx, "proxy", netOpts); err != nil {
 			return err
 		}
+		logrus.Debug("swarm overlay network 'proxy' created")
 
 		return nil
 	},

cli/server/list.go

@@ -13,7 +13,7 @@ import (
 var serverListCommand = &cli.Command{
 	Name:      "list",
 	Aliases:   []string{"ls"},
-	Usage:     "List locally-defined servers.",
+	Usage:     "List managed servers",
 	ArgsUsage: " ",
 	HideHelp:  true,
 	Action: func(c *cli.Context) error {
@@ -22,6 +22,7 @@ var serverListCommand = &cli.Command{
 		if err != nil {
 			logrus.Fatal(err)
 		}
+
 		tableColumns := []string{"Name", "Connection"}
 		table := formatter.CreateTable(tableColumns)
 		defer table.Render()
@@ -30,8 +31,8 @@ var serverListCommand = &cli.Command{
 		if err != nil {
 			logrus.Fatal(err)
 		}
-		for _, serverName := range serverNames {
 
+		for _, serverName := range serverNames {
 			var row []string
 			for _, ctx := range contexts {
 				endpoint, err := client.GetContextEndpoint(ctx)
@@ -47,9 +48,8 @@ var serverListCommand = &cli.Command{
 				row = []string{serverName, "UNKNOWN"}
 			}
 			table.Append(row)
-
 		}
-		return nil
 
+		return nil
 	},
 }

cli/server/new.go

@@ -79,12 +79,14 @@ environment variable or otherwise passing the "--env/-e" flag.
 		}
 
 		if hetznerCloudAPIToken == "" {
-			logrus.Fatal("Hetzner Cloud API token is missing, cannot continue")
+			logrus.Fatal("Hetzner Cloud API token is missing")
 		}
 
 		ctx := context.Background()
 		client := hcloud.NewClient(hcloud.WithToken(hetznerCloudAPIToken))
 
+		logrus.Debugf("successfully created hetzner cloud API client")
+
 		var sshKeys []*hcloud.SSHKey
 		for _, sshKey := range c.StringSlice("ssh-keys") {
 			sshKey, _, err := client.SSHKey.GetByName(ctx, sshKey)
@@ -106,13 +108,17 @@ environment variable or otherwise passing the "--env/-e" flag.
 			logrus.Fatal(err)
 		}
 
+		logrus.Debugf("new server '%s' created", name)
+
 		tableColumns := []string{"Name", "IPv4", "Root Password"}
 		table := formatter.CreateTable(tableColumns)
+
 		if len(sshKeys) > 0 {
 			table.Append([]string{name, res.Server.PublicNet.IPv4.IP.String(), "N/A (using SSH keys)"})
 		} else {
 			table.Append([]string{name, res.Server.PublicNet.IPv4.IP.String(), res.RootPassword})
 		}
+
 		table.Render()
 
 		return nil
@@ -182,13 +188,14 @@ environment variable or otherwise passing the "--env/-e" flag.
 		}
 
 		if capsulAPIToken == "" {
-			logrus.Fatal("Capsul API token is missing, cannot continue")
+			logrus.Fatal("Capsul API token is missing")
 		}
 
-		// yep, the response time is quite slow, something to fix Capsul side
+		// yep, the response time is quite slow, something to fix on the Capsul side
 		client := &http.Client{Timeout: 20 * time.Second}
 
 		capsulCreateURL := fmt.Sprintf("https://%s/api/capsul/create", capsulInstance)
+		logrus.Debugf("using '%s' as capsul create url", capsulCreateURL)
 		values := map[string]string{
 			"name":      name,
 			"size":      capsulType,
@@ -230,6 +237,7 @@ environment variable or otherwise passing the "--env/-e" flag.
 		if err := json.NewDecoder(res.Body).Decode(&resp); err != nil {
 			logrus.Fatal(err)
 		}
+		logrus.Debugf("capsul created with ID: '%s'", resp.ID)
 
 		tableColumns := []string{"Name", "ID"}
 		table := formatter.CreateTable(tableColumns)
@@ -241,10 +249,14 @@ environment variable or otherwise passing the "--env/-e" flag.
 }
 
 var serverNewCommand = &cli.Command{
-	Name:        "new",
+	Name:    "new",
-	Usage:       "Create a new server using a 3rd party provider",
+	Aliases: []string{"n"},
-	Description: "Use a provider plugin to create a new server which can then be used to house a new Co-op Cloud installation.",
+	Usage:   "Create a new server using a 3rd party provider",
-	ArgsUsage:   "<provider>",
+	Description: `
+Use a provider plugin to create a new server which can then be used to house a
+new Co-op Cloud installation.
+`,
+	ArgsUsage: "<provider>",
 	Subcommands: []*cli.Command{
 		serverNewHetznerCloudCommand,
 		serverNewCapsulCommand,

cli/server/remove.go

@@ -1,8 +1,6 @@
 package server
 
 import (
-	"errors"
-
 	"coopcloud.tech/abra/cli/internal"
 	"coopcloud.tech/abra/pkg/client"
 	"github.com/sirupsen/logrus"
@@ -19,13 +17,14 @@ internal bookkeeping so that it is not managed any more.
 `,
 	HideHelp: true,
 	Action: func(c *cli.Context) error {
-		server := c.Args().First()
+		domainName := internal.ValidateDomain(c)
-		if server == "" {
+
-			internal.ShowSubcommandHelpAndError(c, errors.New("no server provided"))
+		if err := client.DeleteContext(domainName); err != nil {
-		}
-		if err := client.DeleteContext(server); err != nil {
 			logrus.Fatal(err)
 		}
+
+		logrus.Infof("server at '%s' has been forgotten", domainName)
+
 		return nil
 	},
 }

cli/upgrade.go

@@ -0,0 +1,23 @@
+package cli
+
+import (
+	"os/exec"
+
+	"coopcloud.tech/abra/cli/internal"
+	"github.com/sirupsen/logrus"
+	"github.com/urfave/cli/v2"
+)
+
+// UpgradeCommand upgrades abra in-place.
+var UpgradeCommand = &cli.Command{
+	Name:  "upgrade",
+	Usage: "Upgrade abra",
+	Action: func(c *cli.Context) error {
+		cmd := exec.Command("bash", "-c", "curl -s https://install.abra.coopcloud.tech | bash")
+		logrus.Debugf("attempting to run '%s'", cmd)
+		if err := internal.RunCmd(cmd); err != nil {
+			logrus.Fatal(err)
+		}
+		return nil
+	},
+}

go.mod

@@ -5,7 +5,7 @@ go 1.17
 require (
 	coopcloud.tech/tagcmp v0.0.0-20210906102006-2a8edd82d75d
 	github.com/AlecAivazis/survey/v2 v2.3.1
-	github.com/Autonomic-Cooperative/godotenv v1.3.1
+	github.com/Autonomic-Cooperative/godotenv v1.3.1-0.20210731170023-c37c0920d1a4
 	github.com/docker/cli v20.10.8+incompatible
 	github.com/docker/distribution v2.7.1+incompatible
 	github.com/docker/docker v20.10.8+incompatible
@@ -16,6 +16,7 @@ require (
 	github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6
 	github.com/olekukonko/tablewriter v0.0.5
 	github.com/pkg/errors v0.9.1
+	github.com/schollz/progressbar/v3 v3.8.3
 	github.com/schultz-is/passgen v1.0.1
 	github.com/sirupsen/logrus v1.8.1
 	github.com/urfave/cli/v2 v2.3.0
@@ -53,11 +54,12 @@ require (
 	github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
 	github.com/kevinburke/ssh_config v0.0.0-20201106050909-4977a11b4351 // indirect
 	github.com/mattn/go-colorable v0.1.2 // indirect
-	github.com/mattn/go-isatty v0.0.8 // indirect
+	github.com/mattn/go-isatty v0.0.14 // indirect
-	github.com/mattn/go-runewidth v0.0.9 // indirect
+	github.com/mattn/go-runewidth v0.0.13 // indirect
 	github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 // indirect
 	github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b // indirect
 	github.com/miekg/pkcs11 v1.0.3 // indirect
+	github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db // indirect
 	github.com/mitchellh/go-homedir v1.1.0 // indirect
 	github.com/mitchellh/mapstructure v1.1.2 // indirect
 	github.com/moby/sys/mount v0.2.0 // indirect
@@ -70,6 +72,7 @@ require (
 	github.com/prometheus/client_model v0.2.0 // indirect
 	github.com/prometheus/common v0.26.0 // indirect
 	github.com/prometheus/procfs v0.6.0 // indirect
+	github.com/rivo/uniseg v0.2.0 // indirect
 	github.com/russross/blackfriday/v2 v2.0.1 // indirect
 	github.com/sergi/go-diff v1.1.0 // indirect
 	github.com/shurcooL/sanitized_anchor_name v1.0.0 // indirect
@@ -81,10 +84,10 @@ require (
 	github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
 	github.com/xeipuuv/gojsonschema v1.2.0 // indirect
 	go.opencensus.io v0.22.3 // indirect
-	golang.org/x/crypto v0.0.0-20210421170649-83a5a9bb288b // indirect
+	golang.org/x/crypto v0.0.0-20210817164053-32db794688a5 // indirect
 	golang.org/x/net v0.0.0-20210326060303-6b1517762897 // indirect
-	golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c // indirect
+	golang.org/x/sys v0.0.0-20210910150752-751e447fb3d0 // indirect
-	golang.org/x/term v0.0.0-20210503060354-a79de5458b56 // indirect
+	golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b // indirect
 	golang.org/x/text v0.3.4 // indirect
 	google.golang.org/genproto v0.0.0-20201110150050-8816d57aaa9a // indirect
 	google.golang.org/grpc v1.33.2 // indirect

go.sum

@@ -26,8 +26,8 @@ coopcloud.tech/tagcmp v0.0.0-20210906102006-2a8edd82d75d/go.mod h1:ESVm0wQKcbcFi
 dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
 github.com/AlecAivazis/survey/v2 v2.3.1 h1:lzkuHA60pER7L4eYL8qQJor4bUWlJe4V0gqAT19tdOA=
 github.com/AlecAivazis/survey/v2 v2.3.1/go.mod h1:TH2kPCDU3Kqq7pLbnCWwZXDBjnhZtmsCle5EiYDJ2fg=
-github.com/Autonomic-Cooperative/godotenv v1.3.1 h1:LxRTdqBgXyBu7sM1kY8RXuYYA8OFmeLKowLGOAT0Yw0=
+github.com/Autonomic-Cooperative/godotenv v1.3.1-0.20210731170023-c37c0920d1a4 h1:aYUdiI42a4fWfPoUr25XlaJrFEICv24+o/gWhqYS/jk=
-github.com/Autonomic-Cooperative/godotenv v1.3.1/go.mod h1:oZRCMMRS318l07ei4DTqbZoOawfJlJ4yyo8juk2v4Rk=
+github.com/Autonomic-Cooperative/godotenv v1.3.1-0.20210731170023-c37c0920d1a4/go.mod h1:oZRCMMRS318l07ei4DTqbZoOawfJlJ4yyo8juk2v4Rk=
 github.com/Azure/azure-sdk-for-go v16.2.1+incompatible/go.mod h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc=
 github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78/go.mod h1:LmzpDX56iTiv29bbRTIsUNlaFfuhWRQBWjQdVyAevI8=
 github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 h1:UQHMgLO+TxOElx5B5HZ4hJQsoJ/PvUvKRhJHDQXO8P8=
@@ -479,6 +479,7 @@ github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfV
 github.com/juju/loggo v0.0.0-20190526231331-6e530bcce5d8/go.mod h1:vgyd7OREkbtVEN/8IXZe5Ooef3LQePvuBm9UWj6ZL8U=
 github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
 github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM=
+github.com/k0kubun/go-ansi v0.0.0-20180517002512-3bf9e2903213/go.mod h1:vNUNkEQ1e29fT/6vq2aBdFsgNPmy8qMdSay1npru+Sw=
 github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs=
 github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8=
 github.com/kevinburke/ssh_config v0.0.0-20201106050909-4977a11b4351 h1:DowS9hvgyYSX4TO5NpyC606/Z4SxnNYbT+WX27or6Ck=
@@ -518,11 +519,13 @@ github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaO
 github.com/mattn/go-colorable v0.1.2 h1:/bC9yWikZXAL9uJdulbSfyVNIR3n3trXl+v8+1sx8mU=
 github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
 github.com/mattn/go-isatty v0.0.4/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
-github.com/mattn/go-isatty v0.0.8 h1:HLtExJ+uU2HOZ+wI0Tt5DtUDrx8yhUqDcp7fYERX4CE=
 github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
+github.com/mattn/go-isatty v0.0.14 h1:yVuAays6BHfxijgZPzw+3Zlu5yQgKGP2/hcQbHb7S9Y=
+github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
 github.com/mattn/go-runewidth v0.0.2/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
-github.com/mattn/go-runewidth v0.0.9 h1:Lm995f3rfxdpd6TSmuVCHVb/QhupuXlYr8sCI/QdE+0=
 github.com/mattn/go-runewidth v0.0.9/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
+github.com/mattn/go-runewidth v0.0.13 h1:lTGmDsbAYt5DmK6OnoV7EuIF1wEIFAcxld6ypU4OSgU=
+github.com/mattn/go-runewidth v0.0.13/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
 github.com/mattn/go-shellwords v1.0.3/go.mod h1:3xCvwCdWdlDJUrvuMn7Wuy9eWs4pE8vqg+NOMyg4B2o=
 github.com/mattn/go-sqlite3 v1.6.0/go.mod h1:FPy6KqzDD04eiIsT53CuJW3U88zkxoIYsOqkbpncsNc=
 github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
@@ -534,6 +537,8 @@ github.com/miekg/pkcs11 v1.0.2/go.mod h1:XsNlhZGX73bx86s2hdc/FuaLm2CPZJemRLMA+WT
 github.com/miekg/pkcs11 v1.0.3 h1:iMwmD7I5225wv84WxIG/bmxz9AXjWvTWIbM/TYHvWtw=
 github.com/miekg/pkcs11 v1.0.3/go.mod h1:XsNlhZGX73bx86s2hdc/FuaLm2CPZJemRLMA+WTFxgs=
 github.com/mistifyio/go-zfs v2.1.2-0.20190413222219-f784269be439+incompatible/go.mod h1:8AuVvqP/mXw1px98n46wfvcGfQ4ci2FwoAjKYxuo3Z4=
+github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db h1:62I3jR2EmQ4l5rM/4FEfDWcRD+abF5XlKShorW5LRoQ=
+github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db/go.mod h1:l0dey0ia/Uv7NcFFVbCLtqEBQbrT4OCwCSKTEv6enCw=
 github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y=
 github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
 github.com/mitchellh/mapstructure v0.0.0-20150613213606-2caf8efc9366/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
@@ -662,12 +667,16 @@ github.com/prometheus/procfs v0.2.0/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4O
 github.com/prometheus/procfs v0.6.0 h1:mxy4L2jP6qMonqmq+aTtOx1ifVWUgG/TAmntgbh3xv4=
 github.com/prometheus/procfs v0.6.0/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA=
 github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU=
+github.com/rivo/uniseg v0.2.0 h1:S1pD9weZBuJdFmowNwbpi7BJ8TNftyUImj/0WQi72jY=
+github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
 github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg=
 github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
 github.com/russross/blackfriday/v2 v2.0.1 h1:lPqVAte+HuHNfhJ/0LC98ESWRz8afy9tM/0RK8m9o+Q=
 github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
 github.com/safchain/ethtool v0.0.0-20190326074333-42ed695e3de8/go.mod h1:Z0q5wiBQGYcxhMZ6gUqHn6pYNLypFAvaL3UvgZLR0U4=
 github.com/satori/go.uuid v1.2.0/go.mod h1:dA0hQrYB0VpLJoorglMZABFdXlWrHn1NEOzdhQKdks0=
+github.com/schollz/progressbar/v3 v3.8.3 h1:FnLGl3ewlDUP+YdSwveXBaXs053Mem/du+wr7XSYKl8=
+github.com/schollz/progressbar/v3 v3.8.3/go.mod h1:pWnVCjSBZsT2X3nx9HfRdnCDrpbevliMeoEVhStwHko=
 github.com/schultz-is/passgen v1.0.1 h1:wUINzqW1Xmmy3yREHR6YTj+83VlFYjj2DIDMHzIi5TQ=
 github.com/schultz-is/passgen v1.0.1/go.mod h1:NnqzT2aSfvyheNQvBtlLUa0YlPFLDj60Jw2DZVwqiJk=
 github.com/seccomp/libseccomp-golang v0.9.1/go.mod h1:GbW5+tmTXfcxTToHLXlScSlAvWlF4P2Ca7zGrPiEpWo=
@@ -798,8 +807,9 @@ golang.org/x/crypto v0.0.0-20200728195943-123391ffb6de/go.mod h1:LzIPMQfyMNhhGPh
 golang.org/x/crypto v0.0.0-20201002170205-7f63de1d35b0/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
 golang.org/x/crypto v0.0.0-20201117144127-c1f2f97bffc9/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
 golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=
-golang.org/x/crypto v0.0.0-20210421170649-83a5a9bb288b h1:7mWr3k41Qtv8XlltBkDkl8LoP3mpSgBW8BUoxtEdbXg=
 golang.org/x/crypto v0.0.0-20210421170649-83a5a9bb288b/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=
+golang.org/x/crypto v0.0.0-20210817164053-32db794688a5 h1:HWj/xjIHfjYU5nVXpTM0s39J9CbLn7Cc5a7IC5rwsMQ=
+golang.org/x/crypto v0.0.0-20210817164053-32db794688a5/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
 golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
 golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
 golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
@@ -947,13 +957,16 @@ golang.org/x/sys v0.0.0-20210324051608-47abb6519492/go.mod h1:h1NjWce9XRLGQEsW7w
 golang.org/x/sys v0.0.0-20210426230700-d19ff857e887/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210502180810-71e4cd670f79/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c h1:F1jZWGFhYfh0Ci55sIpILtKKK8p3i2/krTr0H1rg74I=
 golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20210910150752-751e447fb3d0 h1:xrCZDmdtoloIiooiA9q0OQb9r8HejIHYoHGhGCe1pGg=
+golang.org/x/sys v0.0.0-20210910150752-751e447fb3d0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
 golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
-golang.org/x/term v0.0.0-20210503060354-a79de5458b56 h1:b8jxX3zqjpqb2LklXPzKSGJhzyxCOZSz8ncv8Nv+y7w=
 golang.org/x/term v0.0.0-20210503060354-a79de5458b56/go.mod h1:tfny5GFUkzUvx4ps4ajbZsCe5lw1metzhBm9T3x7oIY=
+golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b h1:9zKuko04nR4gjZ4+DNjHqRlAJqbJETHwiNKDqTfOjfE=
+golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
 golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=

pkg/app/app.go

@@ -1,7 +1,14 @@
 package app
 
 import (
+	"context"
+	"fmt"
+	"strings"
+
+	"coopcloud.tech/abra/pkg/client/stack"
 	"coopcloud.tech/abra/pkg/config"
+	apiclient "github.com/docker/docker/client"
+	"github.com/sirupsen/logrus"
 )
 
 // Get retrieves an app
@@ -16,5 +23,63 @@ func Get(appName string) (config.App, error) {
 		return config.App{}, err
 	}
 
+	logrus.Debugf("retrieved '%s' for '%s'", app, appName)
+
 	return app, nil
 }
+
+// deployedServiceSpec represents a deployed service of an app.
+type deployedServiceSpec struct {
+	Name    string
+	Version string
+}
+
+// VersionSpec represents a deployed app and associated metadata.
+type VersionSpec map[string]deployedServiceSpec
+
+// DeployedVersions lists metadata (e.g. versions) for deployed
+func DeployedVersions(ctx context.Context, cl *apiclient.Client, app config.App) (VersionSpec, bool, error) {
+	services, err := stack.GetStackServices(ctx, cl, app.StackName())
+	if err != nil {
+		return VersionSpec{}, false, err
+	}
+
+	appSpec := make(VersionSpec)
+	for _, service := range services {
+		serviceName := ParseServiceName(service.Spec.Name)
+		label := fmt.Sprintf("coop-cloud.%s.%s.version", app.StackName(), serviceName)
+		if deployLabel, ok := service.Spec.Labels[label]; ok {
+			version, _ := ParseVersionLabel(deployLabel)
+			appSpec[serviceName] = deployedServiceSpec{Name: serviceName, Version: version}
+		}
+	}
+
+	deployed := len(services) > 0
+
+	if deployed {
+		logrus.Debugf("detected '%s' as deployed versions of '%s'", appSpec, app.Name)
+	} else {
+		logrus.Debugf("detected '%s' as not deployed", app.Name)
+	}
+
+	return appSpec, len(services) > 0, nil
+}
+
+// ParseVersionLabel parses a $VERSION-$DIGEST app service label.
+func ParseVersionLabel(label string) (string, string) {
+	// versions may look like v4.2-abcd or v4.2-alpine-abcd
+	idx := strings.LastIndex(label, "-")
+	version := label[:idx]
+	digest := label[idx+1:]
+	logrus.Debugf("parsed '%s' as version from '%s'", version, label)
+	logrus.Debugf("parsed '%s' as digest from '%s'", digest, label)
+	return version, digest
+}
+
+// ParseVersionName parses a $STACK_NAME_$SERVICE_NAME service label.
+func ParseServiceName(label string) string {
+	idx := strings.LastIndex(label, "_")
+	serviceName := label[idx+1:]
+	logrus.Debugf("parsed '%s' as service name from '%s'", serviceName, label)
+	return serviceName
+}

pkg/catalogue/catalogue.go

@@ -13,12 +13,17 @@ import (
 	"time"
 
 	"coopcloud.tech/abra/pkg/config"
+	"coopcloud.tech/abra/pkg/recipe"
 	"coopcloud.tech/abra/pkg/web"
+	"github.com/sirupsen/logrus"
 )
 
 // RecipeCatalogueURL is the only current recipe catalogue available.
 const RecipeCatalogueURL = "https://apps.coopcloud.tech"
 
+// ReposMetadataURL is the recipe repository metadata
+const ReposMetadataURL = "https://git.coopcloud.tech/api/v1/orgs/coop-cloud/repos"
+
 // image represents a recipe container image.
 type image struct {
 	Image  string `json:"image"`
@@ -75,6 +80,8 @@ func (r RecipeMeta) LatestVersion() string {
 		version = tag
 	}
 
+	logrus.Debugf("choosing '%s' as latest version of '%s'", version, r.Name)
+
 	return version
 }
 
@@ -87,9 +94,11 @@ type RecipeCatalogue map[Name]RecipeMeta
 // Flatten converts AppCatalogue to slice
 func (r RecipeCatalogue) Flatten() []RecipeMeta {
 	recipes := make([]RecipeMeta, 0, len(r))
+
 	for name := range r {
 		recipes = append(recipes, r[name])
 	}
+
 	return recipes
 }
 
@@ -116,9 +125,11 @@ func recipeCatalogueFSIsLatest() (bool, error) {
 	if err != nil {
 		return false, err
 	}
+
 	info, err := os.Stat(config.APPS_JSON)
 	if err != nil {
 		if os.IsNotExist(err) {
+			logrus.Debugf("no recipe catalogue found in file system cache")
 			return false, nil
 		}
 		return false, err
@@ -128,9 +139,12 @@ func recipeCatalogueFSIsLatest() (bool, error) {
 	remoteModifiedTime := parsed.Unix()
 
 	if localModifiedTime < remoteModifiedTime {
+		logrus.Debug("file system cached recipe catalogue is out-of-date")
 		return false, nil
 	}
 
+	logrus.Debug("file system cached recipe catalogue is up-to-date")
+
 	return true, nil
 }
 
@@ -144,12 +158,14 @@ func ReadRecipeCatalogue() (RecipeCatalogue, error) {
 	}
 
 	if !recipeFSIsLatest {
+		logrus.Debugf("reading recipe catalogue from web to get latest")
 		if err := readRecipeCatalogueWeb(&recipes); err != nil {
 			return nil, err
 		}
 		return recipes, nil
 	}
 
+	logrus.Debugf("reading recipe catalogue from file system cache to get latest")
 	if err := readRecipeCatalogueFS(&recipes); err != nil {
 		return nil, err
 	}
@@ -163,9 +179,13 @@ func readRecipeCatalogueFS(target interface{}) error {
 	if err != nil {
 		return err
 	}
+
 	if err := json.Unmarshal(recipesJSONFS, &target); err != nil {
 		return err
 	}
+
+	logrus.Debugf("read recipe catalogue from file system cache in '%s'", config.APPS_JSON)
+
 	return nil
 }
 
@@ -184,6 +204,8 @@ func readRecipeCatalogueWeb(target interface{}) error {
 		return err
 	}
 
+	logrus.Debugf("read recipe catalogue from web at '%s'", RecipeCatalogueURL)
+
 	return nil
 }
 
@@ -210,5 +232,136 @@ func VersionsOfService(recipe, serviceName string) ([]string, error) {
 		}
 	}
 
+	logrus.Debugf("detected versions '%s' for '%s'", strings.Join(versions, ", "), recipe)
+
 	return versions, nil
 }
+
+// GetRecipeMeta retrieves the recipe metadata from the recipe catalogue.
+func GetRecipeMeta(recipeName string) (RecipeMeta, error) {
+	catl, err := ReadRecipeCatalogue()
+	if err != nil {
+		return RecipeMeta{}, err
+	}
+
+	recipeMeta, ok := catl[recipeName]
+	if !ok {
+		err := fmt.Errorf("recipe '%s' does not exist?", recipeName)
+		return RecipeMeta{}, err
+	}
+
+	if err := recipe.EnsureExists(recipeName); err != nil {
+		return RecipeMeta{}, err
+	}
+
+	logrus.Debugf("recipe metadata retrieved for '%s'", recipeName)
+
+	return recipeMeta, nil
+}
+
+// RepoMeta is a single recipe repo metadata.
+type RepoMeta struct {
+	ID                        int `json:"id"`
+	Owner                     Owner
+	Name                      string      `json:"name"`
+	FullName                  string      `json:"full_name"`
+	Description               string      `json:"description"`
+	Empty                     bool        `json:"empty"`
+	Private                   bool        `json:"private"`
+	Fork                      bool        `json:"fork"`
+	Template                  bool        `json:"template"`
+	Parent                    interface{} `json:"parent"`
+	Mirror                    bool        `json:"mirror"`
+	Size                      int         `json:"size"`
+	HTMLURL                   string      `json:"html_url"`
+	SSHURL                    string      `json:"ssh_url"`
+	CloneURL                  string      `json:"clone_url"`
+	OriginalURL               string      `json:"original_url"`
+	Website                   string      `json:"website"`
+	StarsCount                int         `json:"stars_count"`
+	ForksCount                int         `json:"forks_count"`
+	WatchersCount             int         `json:"watchers_count"`
+	OpenIssuesCount           int         `json:"open_issues_count"`
+	OpenPRCount               int         `json:"open_pr_counter"`
+	ReleaseCounter            int         `json:"release_counter"`
+	DefaultBranch             string      `json:"default_branch"`
+	Archived                  bool        `json:"archived"`
+	CreatedAt                 string      `json:"created_at"`
+	UpdatedAt                 string      `json:"updated_at"`
+	Permissions               Permissions
+	HasIssues                 bool `json:"has_issues"`
+	InternalTracker           InternalTracker
+	HasWiki                   bool   `json:"has_wiki"`
+	HasPullRequests           bool   `json:"has_pull_requests"`
+	HasProjects               bool   `json:"has_projects"`
+	IgnoreWhitespaceConflicts bool   `json:"ignore_whitespace_conflicts"`
+	AllowMergeCommits         bool   `json:"allow_merge_commits"`
+	AllowRebase               bool   `json:"allow_rebase"`
+	AllowRebaseExplicit       bool   `json:"allow_rebase_explicit"`
+	AllowSquashMerge          bool   `json:"allow_squash_merge"`
+	AvatarURL                 string `json:"avatar_url"`
+	Internal                  bool   `json:"internal"`
+	MirrorInterval            string `json:"mirror_interval"`
+}
+
+// Owner is the repo organisation owner metadata.
+type Owner struct {
+	ID         int    `json:"id"`
+	Login      string `json:"login"`
+	FullName   string `json:"full_name"`
+	Email      string `json:"email"`
+	AvatarURL  string `json:"avatar_url"`
+	Language   string `json:"language"`
+	IsAdmin    bool   `json:"is_admin"`
+	LastLogin  string `json:"last_login"`
+	Created    string `json:"created"`
+	Restricted bool   `json:"restricted"`
+	Username   string `json:"username"`
+}
+
+// Permissions is perms metadata for a repo.
+type Permissions struct {
+	Admin bool `json:"admin"`
+	Push  bool `json:"push"`
+	Pull  bool `json:"pull"`
+}
+
+// InternalTracker is issue tracker metadata for a repo.
+type InternalTracker struct {
+	EnableTimeTracker                bool `json:"enable_time_tracker"`
+	AllowOnlyContributorsToTrackTime bool `json:"allow_only_contributors_to_track_time"`
+	EnableIssuesDependencies         bool `json:"enable_issue_dependencies"`
+}
+
+// RepoCatalogue represents all the recipe repo metadata.
+type RepoCatalogue map[string]RepoMeta
+
+// ReadReposMetadata retrieves coop-cloud/... repo metadata from Gitea.
+func ReadReposMetadata() (RepoCatalogue, error) {
+	reposMeta := make(RepoCatalogue)
+
+	pageIdx := 1
+	for {
+		var reposList []RepoMeta
+
+		pagedURL := fmt.Sprintf("%s?page=%v", ReposMetadataURL, pageIdx)
+
+		logrus.Debugf("fetching repo metadata from '%s'", pagedURL)
+
+		if err := web.ReadJSON(pagedURL, &reposList); err != nil {
+			return reposMeta, err
+		}
+
+		if len(reposList) == 0 {
+			break
+		}
+
+		for idx, repo := range reposList {
+			reposMeta[repo.Name] = reposList[idx]
+		}
+
+		pageIdx++
+	}
+
+	return reposMeta, nil
+}

pkg/client/client.go

@@ -48,5 +48,7 @@ func New(contextName string) (*client.Client, error) {
 		logrus.Fatalf("unable to create Docker client: %s", err)
 	}
 
+	logrus.Debugf("created client for '%s'", contextName)
+
 	return cl, nil
 }

pkg/client/context.go

@@ -10,6 +10,7 @@ import (
 	"github.com/docker/cli/cli/context/docker"
 	contextStore "github.com/docker/cli/cli/context/store"
 	"github.com/moby/term"
+	"github.com/sirupsen/logrus"
 )
 
 type Context = contextStore.Metadata
@@ -26,6 +27,7 @@ func CreateContext(contextName string, user string, port string) error {
 	if err := createContext(contextName, host); err != nil {
 		return err
 	}
+	logrus.Debugf("created the '%s' context", contextName)
 	return nil
 }
 
@@ -36,13 +38,16 @@ func createContext(name string, host string) error {
 		Endpoints: make(map[string]interface{}),
 		Name:      name,
 	}
+
 	contextTLSData := contextStore.ContextTLSData{
 		Endpoints: make(map[string]contextStore.EndpointTLSData),
 	}
+
 	dockerEP, dockerTLS, err := getDockerEndpointMetadataAndTLS(host)
 	if err != nil {
 		return err
 	}
+
 	contextMetadata.Endpoints[docker.DockerEndpoint] = dockerEP
 	if dockerTLS != nil {
 		contextTLSData.Endpoints[docker.DockerEndpoint] = *dockerTLS
@@ -51,9 +56,11 @@ func createContext(name string, host string) error {
 	if err := s.CreateOrUpdate(contextMetadata); err != nil {
 		return err
 	}
+
 	if err := s.ResetTLSMaterial(name, &contextTLSData); err != nil {
 		return err
 	}
+
 	return nil
 }
 
@@ -61,6 +68,7 @@ func DeleteContext(name string) error {
 	if name == "default" {
 		return errors.New("context 'default' cannot be removed")
 	}
+
 	if _, err := GetContext(name); err != nil {
 		return err
 	}
@@ -81,6 +89,7 @@ func GetContext(contextName string) (contextStore.Metadata, error) {
 	if err != nil {
 		return contextStore.Metadata{}, err
 	}
+
 	return ctx, nil
 }
 

pkg/client/stack/remove.go

@@ -18,7 +18,7 @@ import (
 func RunRemove(ctx context.Context, client *apiclient.Client, opts Remove) error {
 	var errs []string
 	for _, namespace := range opts.Namespaces {
-		services, err := getStackServices(ctx, client, namespace)
+		services, err := GetStackServices(ctx, client, namespace)
 		if err != nil {
 			return err
 		}

pkg/client/stack/stack.go

@@ -47,7 +47,7 @@ func getAllStacksFilter() filters.Args {
 	return filter
 }
 
-func getStackServices(ctx context.Context, dockerclient client.APIClient, namespace string) ([]swarm.Service, error) {
+func GetStackServices(ctx context.Context, dockerclient client.APIClient, namespace string) ([]swarm.Service, error) {
 	return dockerclient.ServiceList(ctx, types.ServiceListOptions{Filters: getStackServiceFilter(namespace)})
 }
 
@@ -94,7 +94,7 @@ func GetAllDeployedServices(contextName string) StackStatus {
 
 // pruneServices removes services that are no longer referenced in the source
 func pruneServices(ctx context.Context, cl *dockerclient.Client, namespace convert.Namespace, services map[string]struct{}) {
-	oldServices, err := getStackServices(ctx, cl, namespace.Name())
+	oldServices, err := GetStackServices(ctx, cl, namespace.Name())
 	if err != nil {
 		logrus.Infof("Failed to list services: %s\n", err)
 	}
@@ -174,6 +174,7 @@ func deployCompose(ctx context.Context, cl *dockerclient.Client, opts Deploy, co
 	if err != nil {
 		return err
 	}
+
 	return deployServices(ctx, cl, services, namespace, opts.SendRegistryAuth, opts.ResolveImage)
 }
 
@@ -290,7 +291,7 @@ func deployServices(
 	namespace convert.Namespace,
 	sendAuth bool,
 	resolveImage string) error {
-	existingServices, err := getStackServices(ctx, cl, namespace.Name())
+	existingServices, err := GetStackServices(ctx, cl, namespace.Name())
 	if err != nil {
 		return err
 	}

pkg/compose/compose.go

@@ -20,6 +20,8 @@ func UpdateTag(pattern, image, tag string) error {
 		return err
 	}
 
+	logrus.Debugf("considering '%s' config(s) for tag update", strings.Join(composeFiles, ", "))
+
 	for _, composeFile := range composeFiles {
 		opts := stack.Deploy{Composefiles: []string{composeFile}}
 		emptyEnv := make(map[string]string)
@@ -35,7 +37,7 @@ func UpdateTag(pattern, image, tag string) error {
 
 			img, _ := reference.ParseNormalizedNamed(service.Image)
 			if err != nil {
-				logrus.Fatal(err)
+				return err
 			}
 
 			composeImage := reference.Path(img)
@@ -47,16 +49,20 @@ func UpdateTag(pattern, image, tag string) error {
 			}
 			composeTag := img.(reference.NamedTagged).Tag()
 
+			logrus.Debugf("parsed '%s' from '%s'", composeTag, service.Image)
+
 			if image == composeImage {
 				bytes, err := ioutil.ReadFile(composeFile)
 				if err != nil {
-					logrus.Fatal(err)
+					return err
 				}
 
 				old := fmt.Sprintf("%s:%s", composeImage, composeTag)
 				new := fmt.Sprintf("%s:%s", composeImage, tag)
 				replacedBytes := strings.Replace(string(bytes), old, new, -1)
 
+				logrus.Debugf("updating '%s' to '%s' in '%s'", old, new, compose.Filename)
+
 				if err := ioutil.WriteFile(compose.Filename, []byte(replacedBytes), 0644); err != nil {
 					return err
 				}
@@ -74,6 +80,8 @@ func UpdateLabel(pattern, serviceName, label string) error {
 		return err
 	}
 
+	logrus.Debugf("considering '%s' config(s) for label update", strings.Join(composeFiles, ", "))
+
 	for _, composeFile := range composeFiles {
 		opts := stack.Deploy{Composefiles: []string{composeFile}}
 		emptyEnv := make(map[string]string)
@@ -104,6 +112,9 @@ func UpdateLabel(pattern, serviceName, label string) error {
 
 				old := fmt.Sprintf("coop-cloud.${STACK_NAME}.%s.version=%s", service.Name, value)
 				replacedBytes := strings.Replace(string(bytes), old, label, -1)
+
+				logrus.Debugf("updating '%s' to '%s' in '%s'", old, label, compose.Filename)
+
 				if err := ioutil.WriteFile(compose.Filename, []byte(replacedBytes), 0644); err != nil {
 					return err
 				}

pkg/config/app.go

@@ -9,10 +9,12 @@ import (
 	"path/filepath"
 	"strings"
 
+	"coopcloud.tech/abra/cli/formatter"
 	"coopcloud.tech/abra/pkg/client/convert"
 	loader "coopcloud.tech/abra/pkg/client/stack"
 	stack "coopcloud.tech/abra/pkg/client/stack"
 	composetypes "github.com/docker/cli/cli/compose/types"
+	"github.com/sirupsen/logrus"
 )
 
 // Type aliases to make code hints easier to understand
@@ -93,10 +95,14 @@ func readAppEnvFile(appFile AppFile, name AppName) (App, error) {
 	if err != nil {
 		return App{}, fmt.Errorf("env file for '%s' couldn't be read: %s", name, err.Error())
 	}
+
+	logrus.Debugf("read env '%s' from '%s'", env, appFile.Path)
+
 	app, err := newApp(env, name, appFile)
 	if err != nil {
 		return App{}, fmt.Errorf("env file for '%s' has issues: %s", name, err.Error())
 	}
+
 	return app, nil
 }
 
@@ -108,6 +114,7 @@ func newApp(env AppEnv, name string, appFile AppFile) (App, error) {
 	if !ok {
 		return App{}, errors.New("missing TYPE variable")
 	}
+
 	return App{
 		Name:   name,
 		Domain: domain,
@@ -131,6 +138,9 @@ func LoadAppFiles(servers ...string) (AppFiles, error) {
 			}
 		}
 	}
+
+	logrus.Debugf("collecting metadata from '%v' servers: '%s'", len(servers), servers)
+
 	for _, server := range servers {
 		serverDir := path.Join(ABRA_SERVER_FOLDER, server)
 		files, err := getAllFilesInDirectory(serverDir)
@@ -157,16 +167,19 @@ func GetApp(apps AppFiles, name AppName) (App, error) {
 	if !exists {
 		return App{}, fmt.Errorf("cannot find app with name '%s'", name)
 	}
+
 	app, err := readAppEnvFile(appFile, name)
 	if err != nil {
 		return App{}, err
 	}
+
 	return app, nil
 }
 
 // GetApps returns a slice of Apps with their env files read from a given slice of AppFiles
 func GetApps(appFiles AppFiles) ([]App, error) {
 	var apps []App
+
 	for name := range appFiles {
 		app, err := GetApp(appFiles, name)
 		if err != nil {
@@ -174,11 +187,65 @@ func GetApps(appFiles AppFiles) ([]App, error) {
 		}
 		apps = append(apps, app)
 	}
+
 	return apps, nil
 }
 
-// CopyAppEnvSample copies the example env file for the app into the users env files
+// GetAppServiceNames retrieves a list of app service names.
-func CopyAppEnvSample(appType, appName, server string) error {
+func GetAppServiceNames(appName string) ([]string, error) {
+	var serviceNames []string
+
+	appFiles, err := LoadAppFiles("")
+	if err != nil {
+		return serviceNames, err
+	}
+
+	app, err := GetApp(appFiles, appName)
+	if err != nil {
+		return serviceNames, err
+	}
+
+	composeFiles, err := GetAppComposeFiles(app.Type, app.Env)
+	if err != nil {
+		return serviceNames, err
+	}
+
+	opts := stack.Deploy{Composefiles: composeFiles}
+	compose, err := GetAppComposeConfig(app.Type, opts, app.Env)
+	if err != nil {
+		return serviceNames, err
+	}
+
+	for _, service := range compose.Services {
+		serviceNames = append(serviceNames, service.Name)
+	}
+
+	return serviceNames, nil
+}
+
+// GetAppNames retrieves a list of app names.
+func GetAppNames() ([]string, error) {
+	var appNames []string
+
+	appFiles, err := LoadAppFiles("")
+	if err != nil {
+		return appNames, err
+	}
+
+	apps, err := GetApps(appFiles)
+	if err != nil {
+		return appNames, err
+	}
+
+	for _, app := range apps {
+		appNames = append(appNames, app.Name)
+	}
+
+	return appNames, nil
+}
+
+// TemplateAppEnvSample copies the example env file for the app into the users env files
+func TemplateAppEnvSample(appType, appName, server, domain, recipe string) error {
 	envSamplePath := path.Join(ABRA_DIR, "apps", appType, ".env.sample")
 	envSample, err := ioutil.ReadFile(envSamplePath)
 	if err != nil {
@@ -190,11 +257,16 @@ func CopyAppEnvSample(appType, appName, server string) error {
 		return fmt.Errorf("%s already exists?", appEnvPath)
 	}
 
+	envSample = []byte(strings.Replace(string(envSample), fmt.Sprintf("%s.example.com", recipe), domain, -1))
+	envSample = []byte(strings.Replace(string(envSample), "example.com", domain, -1))
+
 	err = ioutil.WriteFile(appEnvPath, envSample, 0755)
 	if err != nil {
 		return err
 	}
 
+	logrus.Debugf("copied '%s' to '%s'", envSamplePath, appEnvPath)
+
 	return nil
 }
 
@@ -205,13 +277,26 @@ func SanitiseAppName(name string) string {
 
 // GetAppStatuses queries servers to check the deployment status of given apps
 func GetAppStatuses(appFiles AppFiles) (map[string]string, error) {
-	servers := appFiles.GetServers()
+	statuses := map[string]string{}
-	ch := make(chan stack.StackStatus, len(servers))
+
-	for _, server := range servers {
+	var unique []string
-		go func(s string) { ch <- stack.GetAllDeployedServices(s) }(server)
+	servers := make(map[string]struct{})
+	for _, appFile := range appFiles {
+		if _, ok := servers[appFile.Server]; !ok {
+			servers[appFile.Server] = struct{}{}
+			unique = append(unique, appFile.Server)
+		}
+	}
+
+	bar := formatter.CreateProgressbar(len(servers), "querying remote servers...")
+	ch := make(chan stack.StackStatus, len(servers))
+	for server := range servers {
+		go func(s string) {
+			ch <- stack.GetAllDeployedServices(s)
+			bar.Add(1)
+		}(server)
 	}
 
-	statuses := map[string]string{}
 	for range servers {
 		status := <-ch
 		for _, service := range status.Services {
@@ -222,6 +307,8 @@ func GetAppStatuses(appFiles AppFiles) (map[string]string, error) {
 		}
 	}
 
+	logrus.Debugf("retrieved app statuses: '%s'", statuses)
+
 	return statuses, nil
 }
 
@@ -229,6 +316,7 @@ func GetAppStatuses(appFiles AppFiles) (map[string]string, error) {
 // merged into a composetypes.Config while respecting the COMPOSE_FILE env var.
 func GetAppComposeFiles(recipe string, appEnv AppEnv) ([]string, error) {
 	if _, ok := appEnv["COMPOSE_FILE"]; !ok {
+		logrus.Debug("no COMPOSE_FILE detected, loading all compose files")
 		pattern := fmt.Sprintf("%s/%s/compose**yml", APPS_DIR, recipe)
 		composeFiles, err := filepath.Glob(pattern)
 		if err != nil {
@@ -239,10 +327,15 @@ func GetAppComposeFiles(recipe string, appEnv AppEnv) ([]string, error) {
 
 	var composeFiles []string
 	composeFileEnvVar := appEnv["COMPOSE_FILE"]
+	envVars := strings.Split(composeFileEnvVar, ":")
+	logrus.Debugf("COMPOSE_FILE detected ('%s'), loading '%s'", composeFileEnvVar, envVars)
 	for _, file := range strings.Split(composeFileEnvVar, ":") {
 		path := fmt.Sprintf("%s/%s/%s", APPS_DIR, recipe, file)
 		composeFiles = append(composeFiles, path)
 	}
+
+	logrus.Debugf("retrieved '%s' configs for '%s'", strings.Join(composeFiles, ", "), recipe)
+
 	return composeFiles, nil
 }
 
@@ -254,5 +347,8 @@ func GetAppComposeConfig(recipe string, opts stack.Deploy, appEnv AppEnv) (*comp
 	if err != nil {
 		return &composetypes.Config{}, err
 	}
+
+	logrus.Debugf("retrieved '%s' for '%s'", compose.Filename, recipe)
+
 	return compose, nil
 }

pkg/config/env.go

@@ -20,42 +20,56 @@ var APPS_JSON = path.Join(ABRA_DIR, "apps.json")
 var APPS_DIR = path.Join(ABRA_DIR, "apps")
 var REPOS_BASE_URL = "https://git.coopcloud.tech/coop-cloud"
 
-func (a AppFiles) GetServers() []string {
+// GetServers retrieves all servers.
-	var unique []string
+func GetServers() ([]string, error) {
-	servers := make(map[string]struct{})
+	var servers []string
-	for _, appFile := range a {
+
-		if _, ok := servers[appFile.Server]; !ok {
+	servers, err := getAllFoldersInDirectory(ABRA_SERVER_FOLDER)
-			servers[appFile.Server] = struct{}{}
+	if err != nil {
-			unique = append(unique, appFile.Server)
+		return servers, err
-		}
 	}
-	return unique
+
+	logrus.Debugf("retrieved '%v' servers: '%s'", len(servers), servers)
+
+	return servers, nil
 }
 
+// ReadEnv loads an app envivornment into a map.
 func ReadEnv(filePath string) (AppEnv, error) {
 	var envFile AppEnv
+
 	envFile, err := godotenv.Read(filePath)
 	if err != nil {
 		return nil, err
 	}
+
+	logrus.Debugf("read '%s' from '%s'", envFile, filePath)
+
 	return envFile, nil
 }
 
+// ReadServerNames retrieves all server names.
 func ReadServerNames() ([]string, error) {
 	serverNames, err := getAllFoldersInDirectory(ABRA_SERVER_FOLDER)
+
 	if err != nil {
 		return nil, err
 	}
+
+	logrus.Debugf("read '%s' from '%s'", strings.Join(serverNames, ","), ABRA_SERVER_FOLDER)
+
 	return serverNames, nil
 }
 
 // getAllFilesInDirectory returns filenames of all files in directory
 func getAllFilesInDirectory(directory string) ([]fs.FileInfo, error) {
 	var realFiles []fs.FileInfo
+
 	files, err := ioutil.ReadDir(directory)
 	if err != nil {
 		return nil, err
 	}
+
 	for _, file := range files {
 		// Follow any symlinks
 		filePath := path.Join(directory, file.Name())
@@ -71,14 +85,15 @@ func getAllFilesInDirectory(directory string) ([]fs.FileInfo, error) {
 				realFiles = append(realFiles, file)
 			}
 		}
-
 	}
+
 	return realFiles, nil
 }
 
 // getAllFoldersInDirectory returns both folder and symlink paths
 func getAllFoldersInDirectory(directory string) ([]string, error) {
 	var folders []string
+
 	files, err := ioutil.ReadDir(directory)
 	if err != nil {
 		return nil, err
@@ -86,6 +101,7 @@ func getAllFoldersInDirectory(directory string) ([]string, error) {
 	if len(files) == 0 {
 		return nil, fmt.Errorf("directory is empty: '%s'", directory)
 	}
+
 	for _, file := range files {
 		// Check if file is directory or symlink
 		if file.IsDir() || file.Mode()&fs.ModeSymlink != 0 {
@@ -99,12 +115,14 @@ func getAllFoldersInDirectory(directory string) ([]string, error) {
 			}
 		}
 	}
+
 	return folders, nil
 }
 
 // EnsureAbraDirExists checks for the abra config folder and throws error if not
 func EnsureAbraDirExists() error {
 	if _, err := os.Stat(ABRA_DIR); os.IsNotExist(err) {
+		logrus.Debugf("'%s' does not exist, creating it", ABRA_DIR)
 		if err := os.Mkdir(ABRA_DIR, 0777); err != nil {
 			return err
 		}
@@ -112,6 +130,7 @@ func EnsureAbraDirExists() error {
 	return nil
 }
 
+// ReadAbraShEnvVars reads env vars from an abra.sh recipe file.
 func ReadAbraShEnvVars(abraSh string) (map[string]string, error) {
 	envVars := make(map[string]string)
 
@@ -137,5 +156,7 @@ func ReadAbraShEnvVars(abraSh string) (map[string]string, error) {
 		}
 	}
 
+	logrus.Debugf("read '%s' from '%s'", envVars, abraSh)
+
 	return envVars, nil
 }

pkg/git/clone.go

@@ -0,0 +1,94 @@
+package git
+
+import (
+	"fmt"
+	"os"
+	"strings"
+
+	"github.com/go-git/go-git/v5"
+	"github.com/go-git/go-git/v5/config"
+	"github.com/go-git/go-git/v5/plumbing"
+	"github.com/sirupsen/logrus"
+)
+
+// Clone runs a git clone which accounts for different default branches.
+func Clone(dir, url string) error {
+	if _, err := os.Stat(dir); os.IsNotExist(err) {
+		logrus.Debugf("'%s' does not exist, attempting to git clone from '%s'", dir, url)
+		_, err := git.PlainClone(dir, false, &git.CloneOptions{URL: url, Tags: git.AllTags})
+		if err != nil {
+			logrus.Debugf("cloning '%s' default branch failed, attempting from main branch", url)
+			_, err := git.PlainClone(dir, false, &git.CloneOptions{
+				URL:           url,
+				Tags:          git.AllTags,
+				ReferenceName: plumbing.ReferenceName("refs/heads/main"),
+			})
+			if err != nil {
+				return err
+			}
+		}
+		logrus.Debugf("'%s' has been git cloned successfully", dir)
+	} else {
+		logrus.Debugf("'%s' already exists, doing nothing", dir)
+	}
+
+	return nil
+}
+
+// EnsureUpToDate ensures that a git repo on disk has the latest changes (git-fetch).
+func EnsureUpToDate(dir string) error {
+	repo, err := git.PlainOpen(dir)
+	if err != nil {
+		return err
+	}
+
+	branch := "master"
+	if _, err := repo.Branch("master"); err != nil {
+		if _, err := repo.Branch("main"); err != nil {
+			logrus.Debugf("failed to select branch in '%s'", dir)
+			return err
+		}
+		branch = "main"
+	}
+
+	logrus.Debugf("choosing '%s' as main git branch in '%s'", branch, dir)
+
+	worktree, err := repo.Worktree()
+	if err != nil {
+		return err
+	}
+
+	refName := fmt.Sprintf("refs/heads/%s", branch)
+	checkOutOpts := &git.CheckoutOptions{
+		Create: false,
+		Force:  true,
+		Keep:   false,
+		Branch: plumbing.ReferenceName(refName),
+	}
+	if err := worktree.Checkout(checkOutOpts); err != nil {
+		logrus.Debugf("failed to check out '%s' in '%s'", refName, dir)
+		return err
+	}
+
+	logrus.Debugf("successfully checked out '%s' in '%s'", branch, dir)
+
+	remote, err := repo.Remote("origin")
+	if err != nil {
+		return err
+	}
+
+	fetchOpts := &git.FetchOptions{
+		RemoteName: "origin",
+		RefSpecs:   []config.RefSpec{"refs/heads/*:refs/remotes/origin/*"},
+		Force:      true,
+	}
+	if err := remote.Fetch(fetchOpts); err != nil {
+		if !strings.Contains(err.Error(), "already up-to-date") {
+			return err
+		}
+	}
+
+	logrus.Debugf("successfully fetched all changes in '%s'", dir)
+
+	return nil
+}

pkg/recipe/recipe.go

@@ -2,7 +2,6 @@ package recipe
 
 import (
 	"fmt"
-	"os"
 	"path"
 	"path/filepath"
 	"strings"
@@ -11,9 +10,11 @@ import (
 	loader "coopcloud.tech/abra/pkg/client/stack"
 	"coopcloud.tech/abra/pkg/compose"
 	"coopcloud.tech/abra/pkg/config"
+	gitPkg "coopcloud.tech/abra/pkg/git"
 	composetypes "github.com/docker/cli/cli/compose/types"
 	"github.com/go-git/go-git/v5"
 	"github.com/go-git/go-git/v5/plumbing"
+	"github.com/sirupsen/logrus"
 )
 
 // Recipe represents a recipe.
@@ -52,9 +53,14 @@ func Get(recipeName string) (Recipe, error) {
 		return Recipe{}, err
 	}
 
+	envSamplePath := path.Join(config.ABRA_DIR, "apps", recipeName, ".env.sample")
+	sampleEnv, err := config.ReadEnv(envSamplePath)
+	if err != nil {
+		logrus.Fatal(err)
+	}
+
 	opts := stack.Deploy{Composefiles: composeFiles}
-	emptyEnv := make(map[string]string)
+	config, err := loader.LoadComposefile(opts, sampleEnv)
-	config, err := loader.LoadComposefile(opts, emptyEnv)
 	if err != nil {
 		return Recipe{}, err
 	}
@@ -65,23 +71,10 @@ func Get(recipeName string) (Recipe, error) {
 // EnsureExists checks whether a recipe has been cloned locally or not.
 func EnsureExists(recipe string) error {
 	recipeDir := path.Join(config.ABRA_DIR, "apps", strings.ToLower(recipe))
-
+	url := fmt.Sprintf("%s/%s.git", config.REPOS_BASE_URL, recipe)
-	if _, err := os.Stat(recipeDir); os.IsNotExist(err) {
+	if err := gitPkg.Clone(recipeDir, url); err != nil {
-		url := fmt.Sprintf("%s/%s.git", config.REPOS_BASE_URL, recipe)
+		return err
-		_, err := git.PlainClone(recipeDir, false, &git.CloneOptions{URL: url, Tags: git.AllTags})
-		if err != nil {
-			// try with main branch because Git is being a Git
-			_, err := git.PlainClone(recipeDir, false, &git.CloneOptions{
-				URL:           url,
-				Tags:          git.AllTags,
-				ReferenceName: plumbing.ReferenceName("refs/heads/main"),
-			})
-			if err != nil {
-				return err
-			}
-		}
 	}
-
 	return nil
 }
 
@@ -99,6 +92,8 @@ func EnsureVersion(recipeName, version string) error {
 		return nil
 	}
 
+	logrus.Debugf("read '%s' as tags for recipe '%s'", tags, recipeName)
+
 	var tagRef plumbing.ReferenceName
 	if err := tags.ForEach(func(ref *plumbing.Reference) (err error) {
 		if ref.Name().Short() == version {
@@ -123,5 +118,7 @@ func EnsureVersion(recipeName, version string) error {
 		return err
 	}
 
+	logrus.Debugf("successfully checked '%s' out to '%s' in '%s'", recipeName, tagRef, recipeDir)
+
 	return nil
 }

pkg/secret/pass.go

@@ -19,6 +19,8 @@ func PassInsertSecret(secretValue, secretName, appName, server string) error {
 		secretValue, server, appName, secretName,
 	)
 
+	logrus.Debugf("attempting to run '%s'", cmd)
+
 	if err := exec.Command("bash", "-c", cmd).Run(); err != nil {
 		return err
 	}
@@ -39,6 +41,8 @@ func PassRmSecret(secretName, appName, server string) error {
 		server, appName, secretName,
 	)
 
+	logrus.Debugf("attempting to run '%s'", cmd)
+
 	if err := exec.Command("bash", "-c", cmd).Run(); err != nil {
 		return err
 	}

pkg/secret/secret.go

@@ -12,6 +12,7 @@ import (
 	"coopcloud.tech/abra/pkg/client"
 	"coopcloud.tech/abra/pkg/config"
 	"github.com/schultz-is/passgen"
+	"github.com/sirupsen/logrus"
 )
 
 // secretValue represents a parsed `SECRET_FOO=v1 # length=bar` env var config
@@ -33,6 +34,8 @@ func GeneratePasswords(count, length uint) ([]string, error) {
 		return nil, err
 	}
 
+	logrus.Debugf("generated '%s'", strings.Join(passwords, ", "))
+
 	return passwords, nil
 }
 
@@ -50,17 +53,24 @@ func GeneratePassphrases(count uint) ([]string, error) {
 		return nil, err
 	}
 
+	logrus.Debugf("generated '%s'", strings.Join(passphrases, ", "))
+
 	return passphrases, nil
 }
 
+// ReadSecretEnvVars reads secret env vars from an app env var config.
 func ReadSecretEnvVars(appEnv config.AppEnv) map[string]string {
 	secretEnvVars := make(map[string]string)
+
 	for envVar := range appEnv {
 		regex := regexp.MustCompile(`^SECRET.*VERSION.*`)
 		if string(regex.Find([]byte(envVar))) != "" {
 			secretEnvVars[envVar] = appEnv[envVar]
 		}
 	}
+
+	logrus.Debugf("read '%s' as secrets from '%s'", secretEnvVars, appEnv)
+
 	return secretEnvVars
 }
 
@@ -68,7 +78,9 @@ func ReadSecretEnvVars(appEnv config.AppEnv) map[string]string {
 func ParseSecretEnvVarName(secretEnvVar string) string {
 	withoutPrefix := strings.TrimPrefix(secretEnvVar, "SECRET_")
 	withoutSuffix := strings.TrimSuffix(withoutPrefix, "_VERSION")
-	return strings.ToLower(withoutSuffix)
+	name := strings.ToLower(withoutSuffix)
+	logrus.Debugf("parsed '%s' as name from '%s'", name, secretEnvVar)
+	return name
 }
 
 // TODO: should probably go in the config/app package?
@@ -76,7 +88,9 @@ func ParseGeneratedSecretName(secret string, appEnv config.App) string {
 	name := fmt.Sprintf("%s_", appEnv.StackName())
 	withoutAppName := strings.TrimPrefix(secret, name)
 	idx := strings.LastIndex(withoutAppName, "_")
-	return withoutAppName[:idx]
+	parsed := withoutAppName[:idx]
+	logrus.Debugf("parsed '%s' as name from '%s'", parsed, secret)
+	return parsed
 }
 
 // TODO: should probably go in the config/app package?
@@ -85,19 +99,23 @@ func ParseSecretEnvVarValue(secret string) (secretValue, error) {
 	if len(values) == 0 {
 		return secretValue{}, fmt.Errorf("unable to parse '%s'", secret)
 	}
+
 	if len(values) == 1 {
 		return secretValue{Version: values[0], Length: 0}, nil
-	} else {
-		split := strings.Split(values[1], "=")
-		parsed := split[len(split)-1]
-		stripped := strings.ReplaceAll(parsed, " ", "")
-		length, err := strconv.Atoi(stripped)
-		if err != nil {
-			return secretValue{}, err
-		}
-		version := strings.ReplaceAll(values[0], " ", "")
-		return secretValue{Version: version, Length: length}, nil
 	}
+
+	split := strings.Split(values[1], "=")
+	parsed := split[len(split)-1]
+	stripped := strings.ReplaceAll(parsed, " ", "")
+	length, err := strconv.Atoi(stripped)
+	if err != nil {
+		return secretValue{}, err
+	}
+	version := strings.ReplaceAll(values[0], " ", "")
+
+	logrus.Debugf("parsed version '%s' and length '%v' from '%s'", version, length, secret)
+
+	return secretValue{Version: version, Length: length}, nil
 }
 
 // GenerateSecrets generates secrets locally and sends them to a remote server for storage.
@@ -114,6 +132,7 @@ func GenerateSecrets(secretEnvVars map[string]string, appName, server string) (m
 				return
 			}
 			secretRemoteName := fmt.Sprintf("%s_%s_%s", appName, secretName, secretValue.Version)
+			logrus.Debugf("attempting to generate and store '%s' on '%s'", secretRemoteName, server)
 			if secretValue.Length > 0 {
 				passwords, err := GeneratePasswords(1, uint(secretValue.Length))
 				if err != nil {
@@ -147,5 +166,7 @@ func GenerateSecrets(secretEnvVars map[string]string, appName, server string) (m
 		}
 	}
 
+	logrus.Debugf("generated and stored '%s' on '%s'", secrets, server)
+
 	return secrets, nil
 }

scripts/autocomplete/bash

@@ -0,0 +1,21 @@
+#! /bin/bash
+
+: ${PROG:=$(basename ${BASH_SOURCE})}
+
+_cli_bash_autocomplete() {
+  if [[ "${COMP_WORDS[0]}" != "source" ]]; then
+    local cur opts base
+    COMPREPLY=()
+    cur="${COMP_WORDS[COMP_CWORD]}"
+    if [[ "$cur" == "-"* ]]; then
+      opts=$( ${COMP_WORDS[@]:0:$COMP_CWORD} ${cur} --generate-bash-completion )
+    else
+      opts=$( ${COMP_WORDS[@]:0:$COMP_CWORD} --generate-bash-completion )
+    fi
+    COMPREPLY=( $(compgen -W "${opts}" -- ${cur}) )
+    return 0
+  fi
+}
+
+complete -o bashdefault -o default -o nospace -F _cli_bash_autocomplete $PROG
+unset PROG

scripts/autocomplete/powershell

@@ -0,0 +1,9 @@
+$fn = $($MyInvocation.MyCommand.Name)
+$name = $fn -replace "(.*)\.ps1$", '$1'
+Register-ArgumentCompleter -Native -CommandName $name -ScriptBlock {
+     param($commandName, $wordToComplete, $cursorPosition)
+     $other = "$wordToComplete --generate-bash-completion"
+         Invoke-Expression $other | ForEach-Object {
+            [System.Management.Automation.CompletionResult]::new($_, $_, 'ParameterValue', $_)
+         }
+ }

scripts/autocomplete/zsh

@@ -0,0 +1,23 @@
+#compdef $PROG
+
+_cli_zsh_autocomplete() {
+
+  local -a opts
+  local cur
+  cur=${words[-1]}
+  if [[ "$cur" == "-"* ]]; then
+    opts=("${(@f)$(_CLI_ZSH_AUTOCOMPLETE_HACK=1 ${words[@]:0:#words[@]-1} ${cur} --generate-bash-completion)}")
+  else
+    opts=("${(@f)$(_CLI_ZSH_AUTOCOMPLETE_HACK=1 ${words[@]:0:#words[@]-1} --generate-bash-completion)}")
+  fi
+
+  if [[ "${opts[1]}" != "" ]]; then
+    _describe 'values' opts
+  else
+    _files
+  fi
+
+  return
+}
+
+compdef _cli_zsh_autocomplete $PROG

scripts/installer/README.md

@@ -0,0 +1,5 @@
+# install.abra.coopcloud.tech
+
+To deploy, run `make`.
+
+You have to be an [Autonomic](https://autonomic.zone) member to do this.

scripts/installer/compose.yml

@@ -0,0 +1,38 @@
+---
+version: "3.8"
+
+services:
+  app:
+    image: "nginx:stable"
+    configs:
+      - source: abra_conf
+        target: /etc/nginx/conf.d/abra.conf
+      - source: abra_installer
+        target: /var/www/abra-installer/installer
+    volumes:
+      - "public:/var/www/abra-installer"
+    networks:
+      - proxy
+    deploy:
+      update_config:
+        failure_action: rollback
+        order: start-first
+      labels:
+        - "traefik.enable=true"
+        - "traefik.http.services.abra-installer.loadbalancer.server.port=80"
+        - "traefik.http.routers.abra-installer.rule=Host(`install.abra.autonomic.zone`,`install.abra.coopcloud.tech`)"
+        - "traefik.http.routers.abra-installer.entrypoints=web-secure"
+        - "traefik.http.routers.abra-installer.tls.certresolver=production"
+
+configs:
+  abra_installer:
+    file: installer
+  abra_conf:
+    file: nginx.conf
+
+networks:
+  proxy:
+    external: true
+
+volumes:
+  public:

scripts/installer/installer

@@ -0,0 +1,57 @@
+#!/usr/bin/env bash
+
+ABRA_VERSION="0.1.4-alpha"
+ABRA_RELEASE_URL="https://git.coopcloud.tech/api/v1/repos/coop-cloud/abra/releases/tags/$ABRA_VERSION"
+
+function show_banner {
+  echo ""
+  echo "   ____                           ____ _                 _ "
+  echo "  / ___|___         ___  _ __    / ___| | ___  _   _  __| |"
+  echo " | |   / _ \ _____ / _ \| '_ \  | |   | |/ _ \| | | |/ _' |"
+  echo " | |__| (_) |_____| (_) | |_) | | |___| | (_) | |_| | (_| |"
+  echo "  \____\___/       \___/| .__/   \____|_|\___/ \__,_|\__,_|"
+  echo "                        |_|"
+  echo ""
+  echo ""
+  echo "          === Public interest infrastructure ===           "
+  echo ""
+  echo ""
+}
+
+function install_abra_release {
+  mkdir -p "$HOME/.local/bin"
+
+  if ! type "curl" > /dev/null 2>&1; then
+    echo "'curl' is not installed, cannot proceed..."
+    echo "perhaps try installing manually via the releases URL?"
+    echo "https://git.coopcloud.tech/coop-cloud/abra/releases"
+    exit 1
+  fi
+
+  if ! type "curl" > /dev/null 2>&1; then
+    error "'python3' is not installed, cannot proceed..."
+    echo "perhaps try installing manually via the releases URL?"
+    echo "https://git.coopcloud.tech/coop-cloud/abra/releases"
+  fi
+
+  # FIXME: support different architectures
+  release_url=$(curl -s "$ABRA_RELEASE_URL" |
+    python3 -c "import sys, json; \
+                payload = json.load(sys.stdin); \
+                url = [a['browser_download_url'] for a in payload['assets'] if 'x86_64' in a['name']][0]; \
+                print(url)")
+
+  echo "downloading $ABRA_VERSION x86_64 binary release for abra..."
+  curl --progress-bar "$release_url" --output "$HOME/.local/bin/abra"
+  chmod +x "$HOME/.local/bin/abra"
+
+  echo "abra installed to $HOME/.local/bin/abra"
+}
+
+function run_installation {
+  show_banner
+  install_abra_release
+}
+
+run_installation "$@"
+exit 0

scripts/installer/makefile

@@ -0,0 +1,7 @@
+STACK := abra_installer_script
+
+default: deploy
+
+deploy:
+	@docker stack rm $(STACK) && \
+		docker stack deploy -c compose.yml $(STACK)

scripts/installer/nginx.conf

@@ -0,0 +1,10 @@
+server {
+  listen         80 default_server;
+  server_name    install.abra.autonomic.zone install.abra.coopcloud.tech;
+
+  location / {
+    root /var/www/abra-installer;
+    add_header Content-Type text/plain;
+    index installer;
+  }
+}