Further tidy-up (+ add warning())

Merge branch 'main' into monorepo
Update README and tweak default STACK_DIR
2020-09-23 11:53:34 +02:00 · 2020-09-22 16:15:33 +02:00 · 2020-09-22 16:13:18 +02:00 · 2020-09-22 16:04:28 +02:00
21 changed files with 401 additions and 4600 deletions
--- a/.drone.yml
+++ b/.drone.yml
@ -3,47 +3,11 @@ kind: pipeline
 name: linters
 steps:
  - name: run shellcheck
-    image: koalaman/shellcheck-alpine:v0.7.1
+    image: debian:buster
    commands:
-      - shellcheck abra
-      - shellcheck bin/*.sh
-
-  - name: run flake8
-    image: alpine/flake8:3.9.0
-    commands:
-      - flake8 --max-line-length 100 bin/app-json.py
-
-  - name: run unit tests
-    image: decentral1se/docker-dind-bats-kcov
-    commands:
-      - bats tests
-
-  - name: collect code coverage
-    failure: ignore # until we fix this
-    image: decentral1se/docker-dind-bats-kcov
-    commands:
-      - kcov . bats tests || true
-
-  - name: send code coverage report to codecov
-    failure: ignore # until we fix this
-    image: plugins/codecov
-    settings:
-      token:
-        from_secret: codecov_token
-        required: true
-
-  - name: notify rocket chat
-    image: plugins/slack
-    settings:
-      webhook:
-        from_secret: rc_builds_url
-      username: comradebritney
-      channel: "internal.builds"
-      template: "{{repo.owner}}/{{repo.name}} build failed: {{build.link}}"
-    when:
-      status:
-        - failure
-
+      - apt update
+      - apt install -y shellcheck
+      - shellcheck abra installer
 trigger:
  branch:
    - main
--- a/.gitignore
+++ b/.gitignore
@ -1,2 +0,0 @@
-/.venv
-coverage/
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -1,124 +1,3 @@
-> 🔥 🔥 🔥 Please note, while we are still in
-> [public alpha](https://docs.cloud.autonomic.zone/roadmap/), the `abra` release
-> versioning scheme is not following [semver](https://semver.org/) conventions
-> because we are still in the exploratory phases of building this tool. Please
-> read the changes before upgrading your `abra` installation as there are
-> **most likely** breaking changes coming each release. Sorry for any
-> inconvenience caused, we're working hard to make this tool stable. Semver
-> will be respected when we reach public beta. 🔥 🔥 🔥
-
-# abra x.x.x (UNRELEASED)
-
-# abra 0.7.3 (2021-04-28)
-
- Only check for pw(q)gen if we're actually trying to use them ([#147](https://git.autonomic.zone/coop-cloud/abra/issues/147))
- Use apps.coopcloud.tech for app data hosting & download ([75bd599](https://git.autonomic.zone/coop-cloud/abra/commit/75bd599))
- Choose latest commit messages for new tags ([#144](https://git.autonomic.zone/coop-cloud/abra/issues/144))
- Handle recipes without an `app` service in `recipe .. release` ([#151](https://git.autonomic.zone/coop-cloud/abra/issues/151))
-
-# abra 0.7.2 (2021-04-07)
-
- Fix installation script development installs (again! Thanks Bash!) ([4747d9b7](https://git.autonomic.zone/coop-cloud/abra/commit/4747d9b7fb5fba914f210b6570bfe2db0b53da23))
-
-# abra 0.7.1 (2021-04-07)
-
- Fix installation script development installs ([8f2fadb3c](https://git.autonomic.zone/coop-cloud/abra/commit/8f2fadb3c43c5915520f5ea531ea3815c2ba8531))
-
-# abra 0.7.0 (2021-04-07)
-
- Add `--force` to the `deploy` command to allow overriding deployment logic ([#105](https://git.autonomic.zone/coop-cloud/abra/issues/105))
- Handle undeployed apps in version summaries when deploying ([#104](https://git.autonomic.zone/coop-cloud/abra/issues/104))
- Add `--force` to `undeploy` command ([e5e98d5](https://git.autonomic.zone/coop-cloud/abra/commit/e5e98d5))
- Rename "app type" back to "stack" in the deployment overview ([54b6acc](https://git.autonomic.zone/coop-cloud/abra/commit/54b6acc))
- Show context connection details on `abra server ls` ([#110](https://git.autonomic.zone/coop-cloud/abra/issues/110))
- Allow to debug the SSH connection details on swarm init ([#109](https://git.autonomic.zone/coop-cloud/abra/issues/109))
- Show correct status for apps deployed on servers with missing context ([#99](https://git.autonomic.zone/coop-cloud/abra/issues/99))
- Search for subcommands in descending order of how many components there are ([#108](https://git.autonomic.zone/coop-cloud/abra/issues/108))
- Add specific app version checking command (`abra app <app> version`) ([#108](https://git.autonomic.zone/coop-cloud/abra/issues/108))
- Add docker version check (guestimating < v19 is a bad idea) ([#15](https://git.autonomic.zone/coop-cloud/abra/issues/15))
- Fix git branch handling when not passing `-b <branch>` ([#122](https://git.autonomic.zone/coop-cloud/abra/issues/122))
- Add work-around to correctly git clone non-master default branch app repositories ([#122](https://git.autonomic.zone/coop-cloud/abra/issues/122))
- Replace `--force` (except for the `deploy` command) with a global `--no-prompt` for avoiding interactive questions ([#118](https://git.autonomic.zone/coop-cloud/abra/issues/118))
- Use [docker-stack-wait-deploy](https://github.com/vitalets/docker-stack-wait-deploy) inspired logic to deploy apps ([#116](https://git.autonomic.zone/coop-cloud/abra/issues/116))
- Add a domain polling check when deploying apps ([#113](https://git.autonomic.zone/coop-cloud/abra/issues/113))
- Recognise when apps are already undeployed with `abra app <app> undeploy` ([#123](https://git.autonomic.zone/coop-cloud/abra/issues/123))
- Add `abra doctor` command to help diagnose setup issues ([#119](https://git.autonomic.zone/coop-cloud/abra/issues/119))
- Add apps version and feature catalogue generation script ([#121](https://git.autonomic.zone/coop-cloud/abra/issues/121))
- New `--skip-version-check` option to `deploy` ([df4e504](https://git.autonomic.zone/coop-cloud/abra/commit/df4e504))
- Look up local available version from compose files instead of `abra.sh` ([#131](https://git.autonomic.zone/coop-cloud/abra/issues/131))
- Improve domain polling logging and allow to skip the check altogether with `--no-domain-poll` ([#140](https://git.autonomic.zone/coop-cloud/abra/issues/140), [#141](https://git.autonomic.zone/coop-cloud/abra/issues/141))
- Support `ABRA_DIR` in the installer script ([4e94a424e94a42](https://git.autonomic.zone/coop-cloud/abra/commit/4e94a424e94a42))
- Support [abra-hetzner](https://git.autonomic.zone/coop-cloud/abra-hetzner) plugin ([#88](https://git.autonomic.zone/coop-cloud/abra/issues/88))
-
-# abra 0.6.0 (2021-03-17)
-
- Show version and digest of app if labelled ([98e674b8e8](https://git.autonomic.zone/coop-cloud/abra/commit/98e674b8e83458a83dcbf331e8e34c7188559c4a))
- Implement basic version checking on deployment ([#82](https://git.autonomic.zone/coop-cloud/abra/issues/82))
- New `app-catalogue.sh` script to auto-generate app list for documentation ([f163d4b](https://git.autonomic.zone/coop-cloud/abra/commit/f163d4b0fa920232e9d995a22d20fe78b174b3a9))
- Support app service rollbacks with `abra <app> rollback <service>` ([#76](https://git.autonomic.zone/coop-cloud/abra/issues/76))
- Detect when latest version is deployed and perform a no-op ([#87](https://git.autonomic.zone/coop-cloud/abra/issues/87))
- Allow cloning of app repos with different main branches using `-b, --branch=<branch>` ([#80](https://git.autonomic.zone/coop-cloud/abra/issues/80))
- Protect against lengthy app names which gives Docker trouble later on ([#83](https://git.autonomic.zone/coop-cloud/abra/issues/83))
- Support removal of secrets and volumes when `rm`'ing apps ([#44](https://git.autonomic.zone/coop-cloud/abra/issues/44))
- Always choose the default IPv4 address with `abra server <host> init` ([#91](https://git.autonomic.zone/coop-cloud/abra/issues/91))
- Add `--type=<type>` filtering option to `abra <app> ls` ([0828189](https://git.autonomic.zone/coop-cloud/abra/commit/0828189))
- Check for bash 4+ ([#96](https://git.autonomic.zone/coop-cloud/abra/commit/0828189))
- Add `--dev` option to installer using `git clone` ([88d2a75](https://git.autonomic.zone/coop-cloud/abra/commit/88d2a75))
- Support `--dev` on the `abra upgrade` command also ([bcc15ec](https://git.autonomic.zone/coop-cloud/abra/commit/bcc15ec))
- Vendor [yq](https://github.com/mikefarah/yq/releases) automatically ([3b59adf](https://git.autonomic.zone/coop-cloud/abra/commit/3b59adf))
- Extend version handling logic to support all underlying services ([#90](https://git.autonomic.zone/coop-cloud/abra/issues/90))
- Fix development installation script symlink issue ([#98](https://git.autonomic.zone/coop-cloud/abra/issues/98))
- Add `app-version.sh` script to help packagers version apps ([28618bd](https://git.autonomic.zone/coop-cloud/abra/commit/28618bd))
- Add git digest to `abra version` output ([8b41416](https://git.autonomic.zone/coop-cloud/abra/commit/8b41416))
-
-# abra 0.5.0 (2021-03-01)
-
- `secret auto` merged into `secret generate` and `app new --auto` is now `app new --secrets` ([#64](https://git.autonomic.zone/coop-cloud/abra/pulls/64))
- Avoid outputting length during secret generation when not in use ([#67](https://git.autonomic.zone/coop-cloud/abra/issues/67))
- Support graceful failure when missing secret generation commands ([44d3ac3](https://git.autonomic.zone/coop-cloud/abra/commit/44d3ac3a1cb86edc9b9e91eea1a00e70eae14965))
- Fix secret detection when using new `.env` file format in apps ([5532452](https://git.autonomic.zone/coop-cloud/abra/commit/55324524ca77141666ffe6cc41b62cc71cf89ace))
- Support choosing an `$EDITOR` when editing configs ([29cc392](https://git.autonomic.zone/coop-cloud/abra/commit/29cc392dff3e93e48e0e2edd3ce11b405c66a95a))
- "server" shell completion fixed ([8839bd4](https://git.autonomic.zone/coop-cloud/abra/commit/8839bd45951d00dccf4ef81ece445bcc49e13ee6))
- Drop `multilogs` command ([#56](https://git.autonomic.zone/coop-cloud/abra/pulls/56))
- Remove `server use` command ([#51](https://git.autonomic.zone/coop-cloud/abra/issues/51))
- `new <app>` becomes `new <type>` ([#48](https://git.autonomic.zone/coop-cloud/abra/issues/48))
- `check` is run on `deploy` now and configurable ([77ba565](https://git.autonomic.zone/coop-cloud/abra/commit/77ba5652b2fe15820f5edfa0f642636f7b8eae7e))
- App configurations are always updated now ([#42](https://git.autonomic.zone/coop-cloud/abra/issues/42))
- We use docker format `.env` files (no "export" syntax) from now now ([#55](https://git.autonomic.zone/coop-cloud/abra/pulls/55))
- Rename `<domain>` option to `<app>` and `APP` variable to `TYPE`, see ([#47](https://git.autonomic.zone/coop-cloud/abra/issues/47))
- Use Docker-in-Docker (dind), and `dind-bats-kcov` Docker image, for `make test` ([1600b62](https://git.autonomic.zone/coop-cloud/abra/commit/1600b6277fbbffc4c6de1e4ba799c7bbe72ec6a0))
- Add built-in documentation using `abra help <subcommand>...`, see ([#50](https://git.autonomic.zone/coop-cloud/abra/issues/50))
- `version` subcommand ([e6b24fe](https://git.autonomic.zone/coop-cloud/abra/commit/e6b24fe))
- Use `# length=x` comments to generate passwords with `pwgen` and drop `KEY`/`PASSWORD` logic ([#68](https://git.autonomic.zone/coop-cloud/abra/issues/68))
- Global `--skip-update|-U` / `--skip-check|-C` options to make things quicker ([37e8b00](https://git.autonomic.zone/coop-cloud/abra/commit/37e8b00))
- `app backup` and `app restore` commands; requires per-app definition ([#70](https://git.autonomic.zone/coop-cloud/abra/issues/70))
- Rename per-type `abra-commands.sh` to `abra.sh`, and include config versions as type-level instead of app-level config ([#43](https://git.autonomic.zone/coop-cloud/abra/issues/43))
- Show per-subcommand help by adding `-h/--help` to a command line ([#38](https://git.autonomic.zone/coop-cloud/abra/issues/78))
-
-# abra 0.4.1 (2020-12-24)
-
- Bug-fixes on `app ls --status` & custom commands
- Add `app ls --server=...` and alias
-
-# abra 0.4.0 (2020-12-24)
-
- New command-line interface based on docopt
- `~/.abra` directory instead of expecting local `.env` files
- Integration tests & code coverage
-
-# abra 0.3.1 (2020-09-27)
-
- Fix installer version
-
-# abra 0.3.0 (2020-09-27)
-
- Add multilogs stack logs implementation ([#8](https://git.autonomic.zone/compose-stacks/abra/issues/8))
- Add beginnings of "monorepo" functionality
-
-# abra 0.2.0 (2020-09-24)
-
- Prepare for swarm install script using script.d ([#12](https://git.autonomic.zone/compose-stacks/planning/issues/12))
-
 # abra 0.1.2 (2020-09-22)

 - Add upgrade command ([#10](https://git.autonomic.zone/autonomic-cooperative/abra/issues/10))
--- a/14
+++ b/14
@ -0,0 +1,14 @@
+default: install
+
+dev_install:
+	ln -sf $(PWD)/abra ~/.local/bin
+
+install:
+	install abra /usr/bin/abra
+
+get_yq:
+	wget https://github.com/mikefarah/yq/releases/download/3.3.2/yq_linux_amd64 && \
+		chmod +x yq_linux_amd64 && \
+		mv yq_linux_amd64 yq
+
+.PHONY: dev_install install get_yq
--- a/README.md
+++ b/README.md
@ -1,93 +1,56 @@
 # abra

-[![Build Status](https://drone.autonomic.zone/api/badges/coop-cloud/abra/status.svg)](https://drone.autonomic.zone/coop-cloud/abra)
-[![codecov](https://codecov.io/gh/Autonomic-Cooperative/abra/branch/main/graph/badge.svg?token=aX3I5NMRsj)](undefined)
+[![Build Status](https://drone.autonomic.zone/api/badges/autonomic-cooperative/abra/status.svg)](https://drone.autonomic.zone/autonomic-cooperative/abra)

-> https://coopcloud.tech
-
-The cooperative cloud utility belt 🎩🐇
-
-`abra` is a command-line tool for managing your own [Co-op Cloud](https://coopcloud.tech). It can provision new servers, create applications, deploy them, run backup and restore operations and a whole lot of other things. It is the go-to tool for day-to-day operations when managing a Co-op Cloud instance.
-
-## Change log
-
-> 🔥 🔥 🔥 Please note, while we are still in [public
-> alpha](https://docs.coopcloud.tech/roadmap/), the `abra` release
-> versioning scheme is not following [semver](https://semver.org/) conventions
-> because we are still in the exploratory phases of building this tool. Please
-> read the changes before upgrading your `abra` installation as there are
-> **most likely** breaking changes coming each release. Sorry for any
-> inconvenience caused, we're working hard to make this tool stable. Semver
-> will be respected when we reach public beta. 🔥 🔥 🔥
-
-See [CHANGELOG.md](./CHANGELOG.md).
-
-## Documentation
-
-> [docs.coopcloud.tech](https://docs.coopcloud.tech)
+Docker stack magic 🎩🐇

 ## Install

-Install the latest stable release:
+```sh
+curl -fsSL https://install.abra.autonomic.zone | bash
+```
+
+Specific releases are available via the project [release page](https://git.autonomic.zone/autonomic-cooperative/abra/releases).
+
+## Changes
+
+See [CHANGELOG.md](./CHANGELOG.md).
+
+## Hacking

 ```sh
-curl https://install.abra.autonomic.zone | bash
+git clone ssh://git@git.autonomic.zone:2222/autonomic-cooperative/abra.git
+cd abra
+make dev_install
 ```

-or the bleeding-edge development version:
+See [autonomic-cooperative/installer-scripts](https://git.autonomic.zone/autonomic-cooperative/installer-scripts) for the installer script deployment. To make a release, just add an entry to [CHANGELOG.md](./CHANGELOG.md) and the [installer](./installer) (following [semver](https://semver.org/) please) and then `git tag x.x.x && git push origin main --tags`. If you want the [installer-scripts](https://git.autonomic.zone/autonomic-cooperative/installer-scripts) deployment to pick that up, you'll need to change the version number in the [Makefile](https://git.autonomic.zone/autonomic-cooperative/installer-scripts/src/branch/main/Makefile) and run `make` in that repository and push the changes.

-```sh
-curl https://install.abra.autonomic.zone | bash -s -- --dev
-```
+## Specify what to deploy where

-The source for this script is [here](./deploy/install.abra.autonomic.zone/installer).
+You can use `abra` in one of 2 ways:

-## Update
+1. Clone a `compose-stack`, create an `.envrc` in it, and run `abra` in that
+   directory. Be sure to set `ABRA_STACK_DIR=.`
+2. "Monorepo mode": keep all your `compose-stack`s in one directory and all your `env` files in
+   another, e.g.:
+   ```
+	 $ tree
+   .
+   ├── apps
+   │   ├── mediawiki.demo.autonomic.zone.env
+   │   ├── wordpress.demo.autonomic.zone.env
+   └── stacks
+       ├── gitea
+       ├── matrix-synapse
+       ├── mediawiki
+       ├── nextcloud
+       ├── swarmpit
+       ├── traefik
+       └── wordpress
+		$ abra -e apps/mediawiki.demo.autonomic.zone.env deploy
+    ```

-Run `abra upgrade` to automatically download and install the latest release
-version.
+## Examples

-To update the development version, run `abra upgrade --dev`.
-
-## Hack
-
-It's written in Bash version 4 or greater!
-
-Install it via `curl https://install.abra.autonomic.zone | bash -s -- --dev`, then you can hack on the source in `~/.abra/src`.
-
-The command-line interface is generated via [docopt](http://docopt.org/). If you add arguments then you need to run `make docopt` ro regenerate the parser.
-
-Please remember to update the [CHANGELOG](./CHANGELOG.md) when you make a change.
-
-## Generating a new apps.json
-
-You'll need to install the following requirements:
-
- [requests](https://docs.python-requests.org/en/master/) (`apt install python3-requests` / `pip install requests`)
- [skopeo](https://github.com/containers/skopeo) (check [the install docs](https://github.com/containers/skopeo/blob/master/install.md))
- [jq](https://stedolan.github.io/jq/tutorial/) (`sudo apt-get install jq` or see [the install docs](https://stedolan.github.io/jq/download/))
- [yq](https://mikefarah.gitbook.io/yq/) (see [the install docs](https://mikefarah.gitbook.io/yq/#install))
-
-Then run `./bin/app-json.py` ([source](./bin/app-json.py)) and it will spit out the JSON file into [deploy/apps.coopcloud.tech/apps.json](./deploy/apps.coopcloud.tech/apps.json).
-
-## Releasing
-
-### `abra`
-
-> [install.abra.autonomic.zone](https://install.abra.autonomic.zone)
-
- Change the `x.x.x` header in [CHANGELOG.md](./CHANGELOG.md) to reflect new version and mark date
- Update the version in [abra](./abra)
- Update the version in [deploy/install.abra.autonomic.zone/installer](./deploy/install.abra.autonomic.zone/installer)
- `git commit` the above changes and then tag it with `git tag <your-new-version>`
- `git push` and `git push --tags`
- Deploy a new installer script `make release-installer`
- Tell the world (CoTech forum, Matrix public channel, Autonomic mastodon, etc.)
-
-### apps.coopcloud.tech
-
-> [apps.coopcloud.tech](https://apps.coopcloud.tech)
-
-```bash
-$ make release-apps
-```
+- `abra run mariadb mysqldump gitea -p'GdIbMeS09SURRktBnm3jcTufsL5z0MPd' | gzip > ../git.autonomic.zone_mariadb_`date +%F`.sql.gz`
--- a/2741
+++ b/2741
--- a/bin/app-catalogue.sh
+++ b/bin/app-catalogue.sh
@ -1,103 +0,0 @@
-#!/bin/bash
-
-# shellcheck disable=SC2119
-
-# Usage: ./app-catalogue.sh
-#
-# Gather metadata from Co-op Cloud apps in $ABRA_DIR/apps (default
-# ~/.abra/apps), and format it as a Markdown table for this page:
-# https://docs.cloud.autonomic.zone/apps/
-
-stack_dir="${ABRA_DIR:-$HOME/.abra}/apps/"
-
-cd "$stack_dir" || exit
-
-# load all README files into ENV_FILES array
-mapfile -t readmes < <(find -L . -name "README.md")
-# FIXME 3wc: requires bash 4, use for loop instead
-
-base_url="https://git.autonomic.zone/coop-cloud"
-
-cat_apps=()
-cat_development=()
-cat_utilities=()
-cat_graveyard=()
-
-get_var() {
-	echo "$1" | grep "$2" | sed 's/^[^:]*: //'
-}
-
-# shellcheck disable=SC2120
-trim() {
-	# accept input as argument or from STDIN, see here:
-	# https://zwbetz.com/passing-input-to-a-bash-function-via-arguments-or-stdin/
-	# shellcheck disable=SC2155
-	local input="$([[ -p /dev/stdin ]] && cat - || echo "$@")"
-	[[ -z "$input" ]] && return 1
-	echo "$input" | tr -d ' '
-}
-
-# shellcheck disable=SC2120
-prettify() {
-	# as above
-	# shellcheck disable=SC2155
-	local input="$([[ -p /dev/stdin ]] && cat - || echo "$@")"
-	[[ -z "$input" ]] && return 1
-
-	echo "$input" | sed -e 's/Yes/✅/' -e 's/No/❌/' -e 's/N\/A/⛔/'
-}
-
-for readme in "${readmes[@]}"; do
-	type="$(basename "${readme%README.md}")"
-	if [ "$type" = "example" ]; then
-		continue
-	fi
-	title="$(grep '^# ' "$type/README.md" | sed 's/^# //' )"
-	# find section between 'metadata' and 'endmetadata' comments
-	metadata="$(awk '/-- metadata --/,/-- endmetadata --/' "$type/README.md")"
-	status="$(get_var "$metadata" "Status")"
-	category="$(get_var "$metadata" "Category" | cut -d',' -f2 | trim)"
-
-	if [ -z "$category" ]; then
-		echo "ERROR: missing category for $type"
-		continue
-	fi
-
-	image="$(get_var "$metadata" "Image" | cut -d',' -f2 | trim)"
-	healthcheck="$(get_var "$metadata" "Healthcheck" | prettify)"
-	backups="$(get_var "$metadata" "Backups" | prettify)"
-	email="$(get_var "$metadata" "Email" | prettify)"
-	tests="$(get_var "$metadata" "Tests" | prettify)"
-	sso="$(get_var "$metadata" "SSO" | prettify)"
-
-	row="| [$title]($base_url/$type) | $status | $image | $healthcheck | $backups | $email | $tests | $sso |"
-
-	category_lower="$(echo "$category" | tr '[:upper:]' '[:lower:]')"
-	eval "cat_$category_lower+=( '$row' )"
-done
-
-headers="
-| **Name** | **Status** | **Image** | **Healtcheck** | **Backups** | **Email** | **CI** | **Single-Sign-On** |
-| --- | --- | --- | --- | --- | --- | --- | --- |"
-
-echo "## Applications"
-echo "$headers"
-printf '%s\n' "${cat_apps[@]}" | sort
-
-echo
-
-echo "## Developer tools"
-echo "$headers"
-printf '%s\n' "${cat_development[@]}" | sort
-
-echo
-
-echo "## Utilities"
-echo "$headers"
-printf '%s\n' "${cat_utilities[@]}" | sort
-
-echo
-
-echo "## Graveyard"
-echo "$headers"
-printf '%s\n' "${cat_graveyard[@]}" | sort
--- a/bin/app-json.py
+++ b/bin/app-json.py
@ -1,284 +0,0 @@
-#!/usr/bin/env python3
-
-# Usage: ./app-json.py
-#
-# Gather metadata from Co-op Cloud apps in $ABRA_DIR/apps (default
-# ~/.abra/apps), and format it as JSON so that it can be hosted here:
-# https://apps.coopcloud.tech
-
-from json import dump
-from logging import DEBUG, basicConfig, getLogger
-from os import chdir, listdir, mkdir
-from os.path import basename, exists, expanduser
-from pathlib import Path
-from re import findall, search
-from shlex import split
-from subprocess import DEVNULL, check_output
-from sys import exit
-
-from requests import get
-
-HOME_PATH = expanduser("~/")
-CLONES_PATH = Path(f"{HOME_PATH}/.abra/apps").absolute()
-YQ_PATH = Path(f"{HOME_PATH}/.abra/vendor/yq")
-SCRIPT_PATH = Path(__file__).absolute().parent
-REPOS_TO_SKIP = (
-    "abra",
-    "abra-apps",
-    "backup-bot",
-    "cloud.autonomic.zone",
-    "docs.cloud.autonomic.zone",
-    "example",
-    "organising",
-    "pyabra",
-    "stack-ssh-deploy",
-    "radicle-seed-node",
-    "coturn",
-)
-
-log = getLogger(__name__)
-basicConfig()
-log.setLevel(DEBUG)
-
-
-def _run_cmd(cmd, shell=False, **kwargs):
-    """Run a shell command."""
-    args = [split(cmd)]
-
-    if shell:
-        args = [cmd]
-        kwargs = {"shell": shell}
-
-    try:
-        return check_output(*args, **kwargs).decode("utf-8").strip()
-    except Exception as exception:
-        log.error(f"Failed to run {cmd}, saw {str(exception)}")
-        exit(1)
-
-
-def get_repos_json():
-    """ Retrieve repo list from Gitea """
-
-    url = "https://git.autonomic.zone/api/v1/orgs/coop-cloud/repos"
-
-    log.info(f"Retrieving {url}")
-
-    repos = []
-    response = True
-    page = 1
-
-    try:
-        while response:
-            log.info(f"Trying to fetch page {page}")
-            response = get(url + f"?page={page}", timeout=10).json()
-            repos.extend(response)
-            page += 1
-
-        return repos
-    except Exception as exception:
-        log.error(f"Failed to retrieve {url}, saw {str(exception)}")
-        exit(1)
-
-
-def get_published_apps_json():
-    """Retrieve already published apps json."""
-    url = "https://apps.coopcloud.tech"
-
-    log.info(f"Retrieving {url}")
-
-    try:
-        return get(url, timeout=5).json()
-    except Exception as exception:
-        log.error(f"Failed to retrieve {url}, saw {str(exception)}")
-        return {}
-
-
-def clone_all_apps(repos_json):
-    """Clone all Co-op Cloud apps to ~/.abra/apps."""
-    if not exists(CLONES_PATH):
-        mkdir(CLONES_PATH)
-
-    repos = [[p["name"], p["ssh_url"]] for p in repos_json]
-
-    for name, url in repos:
-        continue
-        if name in REPOS_TO_SKIP:
-            continue
-
-        if not exists(f"{CLONES_PATH}/{name}"):
-            log.info(f"Retrieving {url}")
-            _run_cmd(f"git clone {url} {CLONES_PATH}/{name}")
-
-            chdir(f"{CLONES_PATH}/{name}")
-            if not int(_run_cmd("git branch --list | wc -l", shell=True)):
-                log.info(f"Guessing main branch is HEAD for {name}")
-                _run_cmd("git checkout main")
-        else:
-            log.info(f"Updating {name}")
-            chdir(f"{CLONES_PATH}/{name}")
-            _run_cmd("git fetch -a")
-
-
-def generate_apps_json(repos_json):
-    """Generate the abra-apps.json application versions file."""
-    apps_json = {}
-    cached_apps_json = get_published_apps_json()
-
-    for app in listdir(CLONES_PATH):
-        if app in REPOS_TO_SKIP:
-            log.info(f"Skipping {app}")
-            continue
-
-        repo_details = next(filter(lambda x: x["name"] == app, repos_json), {})
-
-        app_path = f"{CLONES_PATH}/{app}"
-        chdir(app_path)
-
-        metadata = get_app_metadata(app_path)
-        name = metadata.pop("name", "")
-
-        log.info(f"Processing {app}")
-        apps_json[app] = {
-            "name": name,
-            "category": metadata.get("category", ""),
-            "repository": repo_details.get("clone_url", ""),
-            "default_branch": repo_details.get("default_branch", ""),
-            "description": repo_details.get("description", ""),
-            "website": repo_details.get("website", ""),
-            # Note(decentral1se): please note that the app metadata do not
-            # correspond to version tags. We simply parse the latest metadata
-            # list from HEAD. This may lead to unexpected situations where
-            # users believe X feature is available under Y version but it is
-            # not.
-            "features": metadata,
-            "versions": get_app_versions(app_path, cached_apps_json),
-            "icon": repo_details.get("avatar_url", ""),
-        }
-
-    return apps_json
-
-
-def get_app_metadata(app_path):
-    """Parse metadata from app repo README files."""
-    metadata = {}
-
-    chdir(app_path)
-
-    with open(f"{app_path}/README.md", "r") as handle:
-        log.info(f"{app_path}/README.md")
-        contents = handle.read()
-
-    try:
-        for match in findall(r"\*\*.*\s\*", contents):
-            title = search(r"(?<=\*\*).*(?=\*\*)", match).group().lower()
-
-            if title == "image":
-                value = {
-                    "image": search(r"(?<=`).*(?=`)", match).group(),
-                    "url": search(r"(?<=\().*(?=\))", match).group(),
-                    "rating": match.split(",")[1].strip(),
-                    "source": match.split(",")[-1].replace("*", "").strip(),
-                }
-            else:
-                value = match.split(":")[-1].replace("*", "").strip()
-
-            metadata[title] = value
-        metadata["name"] = findall(r"^# (.*)", contents)[0]
-    except (IndexError, AttributeError):
-        log.info(f"Can't parse {app_path}/README.md")
-        return {}
-    finally:
-        _run_cmd("git checkout HEAD")
-
-    log.info(f"Parsed {metadata}")
-
-    return metadata
-
-
-def get_app_versions(app_path, cached_apps_json):
-    versions = {}
-
-    chdir(app_path)
-
-    tags = _run_cmd("git tag --list").split()
-
-    if not tags:
-        log.info("No tags discovered, moving on")
-        return {}
-
-    initial_branch = _run_cmd("git rev-parse --abbrev-ref HEAD")
-
-    app_name = basename(app_path)
-
-    try:
-        existing_tags = cached_apps_json[app_name]["versions"].keys()
-    except KeyError:
-        existing_tags = []
-
-    for tag in tags:
-        _run_cmd(f"git checkout {tag}", stderr=DEVNULL)
-
-        services_cmd = f"{YQ_PATH} e '.services | keys | .[]' compose*.yml"
-        services = _run_cmd(services_cmd, shell=True).split()
-
-        parsed_services = []
-        service_versions = {}
-        for service in services:
-            if service in ("null", "---"):
-                continue
-
-            if (
-                tag in existing_tags
-                and service in cached_apps_json[app_name]["versions"][tag]
-            ):
-                log.info(f"Skipping {tag} because we've already processed it")
-                existing_versions = cached_apps_json[app_name]["versions"][tag][service]
-                service_versions[service] = existing_versions
-                _run_cmd(f"git checkout {initial_branch}")
-                continue
-
-            if service in parsed_services:
-                log.info(f"Skipped {service}, we've already parsed it locally")
-                continue
-
-            services_cmd = f"{YQ_PATH} e '.services.{service}.image' compose*.yml"
-            images = _run_cmd(services_cmd, shell=True).split()
-
-            for image in images:
-                if image in ("null", "---"):
-                    continue
-
-                images_cmd = f"skopeo inspect docker://{image} | jq '.Digest'"
-                output = _run_cmd(images_cmd, shell=True)
-
-                service_version_info = {
-                    "image": image.split(":")[0],
-                    "tag": image.split(":")[-1],
-                    "digest": output.split(":")[-1][:8],
-                }
-
-                log.info(f"Parsed {service_version_info}")
-                service_versions[service] = service_version_info
-
-            parsed_services.append(service)
-
-        versions[tag] = service_versions
-
-    _run_cmd(f"git checkout {initial_branch}")
-
-    return versions
-
-
-def main():
-    """Run the script."""
-    repos_json = get_repos_json()
-    clone_all_apps(repos_json)
-
-    target = f"{SCRIPT_PATH}/../deploy/apps.coopcloud.tech/apps.json"
-    with open(target, "w", encoding="utf-8") as handle:
-        dump(generate_apps_json(repos_json), handle, ensure_ascii=False, indent=4)
-
-    log.info(f"Successfully generated {target}")
-
-
-main()
--- a/completion/_abra
+++ b/completion/_abra
@ -1,52 +0,0 @@
-#compdef abra
-
-_abra () {
-	local context state line curcontext="$curcontext" ret=1
-	_arguments -n : \
-		{-h,--help}'[Help message]' \
-		'1:commands:(app server)' \
-		'*::arguments:->arguments' \
-		&& ret=0
-
-	case $state in
-		(arguments)
-			curcontext="${curcontext%:*:*}:abra-arguments-$words[1]:"
-			case $words[1] in
-				(app)
-					_arguments \
-						'1: :_abra_apps' \
-					&& ret=0
-				;;
-				(server)
-					_arguments \
-						'1:servers:_abra_servers' \
-					&& ret=0
-				;;
-			esac
-		;;
-	esac
-
-	return ret
-}
-
-_abra_servers() {
-	_path_files -/W $HOME/.abra/servers
-}
-
-_abra_apps()
-{
-	local newapps apps=($HOME/.abra/servers/*/*.env)
-	typeset -a apps
-	newapps=()
-	for app in $apps; do
-		newapps+=($(_abra_basename "${app}"))
-	done
-	_describe -t apps 'app' newapps
-}
-
-_abra_basename()
-{
-    printf -- "${1##*/}"
-}
-
-_abra "$@"
--- a/completion/abra.bash
+++ b/completion/abra.bash
@ -1,117 +0,0 @@
-#!/usr/bin/env bash
-
-_abra_basename()
-{
-    echo "${1##*/}"
-}
-
-_abra_servers()
-{
-    # FIXME 3wc: copied from abra/get_servers()
-    shopt -s nullglob dotglob
-    local SERVERS=(~/.abra/servers/*)
-    shopt -u nullglob dotglob
-
-    for SERVER in "${SERVERS[@]}"; do
-        _abra_basename "${SERVER}"
-    done
-}
-
-_abra_complete_servers()
-{
-    mapfile -t COMPREPLY < <(compgen -W "$(_abra_servers)" -- "$1")
-}
-
-_abra_apps()
-{
-    shopt -s nullglob dotglob
-    local APPS=(~/.abra/servers/*/*.env)
-    shopt -u nullglob dotglob
-
-    for APP in "${APPS[@]}"; do
-        _abra_basename "${APP%.env}"
-    done
-}
-
-_abra_complete_apps()
-{
-    mapfile -t COMPREPLY < <(compgen -W "$(_abra_apps)" -- "$1")
-}
-
-_abra_complete()
-{
-    compopt +o default +o nospace
-    COMPREPLY=()
-
-    local -r cmds='
-        app
-        server
-    '
-    local -r short_opts='-e      -h     -s      -v'
-    local -r long_opts='--env --help --stack --version'
-
-    # Scan through the command line and find the abra command
-    # (if present), as well as its expected position.
-    local cmd
-    local cmd_index=1 # Expected index of the command token.
-    local i
-    for (( i = 1; i < ${#COMP_WORDS[@]}; i++ )); do
-        local word="${COMP_WORDS[i]}"
-        case "$word" in
-            -*)
-                ((cmd_index++))
-                ;;
-            *)
-                cmd="$word"
-                break
-                ;;
-        esac
-    done
-
-    local cur="${COMP_WORDS[COMP_CWORD]}"
-
-    if (( COMP_CWORD < cmd_index )); then
-        # Offer option completions.
-        case "$cur" in
-            --*)
-                mapfile -t COMPREPLY < <(compgen -W "$long_opts" -- "$cur")
-                ;;
-            -*)
-                mapfile -t COMPREPLY < <(compgen -W "$short_opts" -- "$cur")
-                ;;
-            *)
-                # Skip completion; we should never get here.
-                ;;
-        esac
-    elif (( COMP_CWORD == cmd_index )); then
-        # Offer command name completions.
-        mapfile -t COMPREPLY < <(compgen -W "$cmds" -- "$cur")
-    else
-        # Offer command argument completions.
-        case "$cmd" in
-            server)
-                # Offer exactly one server name completion.
-                if (( COMP_CWORD == cmd_index + 1 )); then
-                    _abra_complete_servers "$cur"
-                fi
-                ;;
-            app)
-                # Offer exactly one app completion.
-                if (( COMP_CWORD == cmd_index + 1 )); then
-                    _abra_complete_apps "$cur"
-                fi
-                ;;
-            #help)
-            #    # Offer exactly one command name completion.
-            #    if (( COMP_CWORD == cmd_index + 1 )); then
-            #        COMPREPLY=($(compgen -W "$cmds" -- "$cur"))
-            #    fi
-            #    ;;
-            *)
-                # Unknown command or unknowable argument.
-                ;;
-        esac
-    fi
-}
-
-complete -o default -F _abra_complete abra
--- a/deploy/apps.coopcloud.tech/apps.json
+++ b/deploy/apps.coopcloud.tech/apps.json
--- a/deploy/apps.coopcloud.tech/compose.yml
+++ b/deploy/apps.coopcloud.tech/compose.yml
@ -1,41 +0,0 @@
---
-version: "3.8"
-
-services:
-  app:
-    image: "nginx:stable"
-    configs:
-      - source: abra_conf
-        target: /etc/nginx/conf.d/abra.conf
-      - source: abra_apps_json
-        target: /var/www/abra-apps/apps.json
-    volumes:
-      - "public:/var/www/abra-apps"
-    networks:
-      - proxy
-    deploy:
-      update_config:
-        failure_action: rollback
-        order: start-first
-      labels:
-        - "traefik.enable=true"
-        - "traefik.http.services.abra-apps.loadbalancer.server.port=80"
-        - "traefik.http.routers.abra-apps.rule=Host(`apps.coopcloud.tech`, `abra-apps.cloud.autonomic.zone`)"
-        - "traefik.http.routers.abra-apps.entrypoints=web-secure"
-        - "traefik.http.routers.abra-apps.tls.certresolver=production"
-        - "traefik.http.routers.abra-apps.middlewares=abra-apps-redirect"
-        - "traefik.http.middlewares.abra-apps-redirect.headers.SSLForceHost=true"
-        - "traefik.http.middlewares.abra-apps-redirect.headers.SSLHost=apps.coopcloud.tech"
-
-configs:
-  abra_apps_json:
-    file: apps.json
-  abra_conf:
-    file: nginx.conf
-
-networks:
-  proxy:
-    external: true
-
-volumes:
-  public:
--- a/deploy/apps.coopcloud.tech/nginx.conf
+++ b/deploy/apps.coopcloud.tech/nginx.conf
@ -1,10 +0,0 @@
-server {
-  listen         80 default_server;
-  server_name    apps.coopcloud.tech;
-
-  location / {
-    root /var/www/abra-apps;
-    add_header Content-Type application/json;
-    index apps.json;
-  }
-}
--- a/deploy/install.abra.autonomic.zone/compose.yml
+++ b/deploy/install.abra.autonomic.zone/compose.yml
@ -1,38 +0,0 @@
---
-version: "3.8"
-
-services:
-  app:
-    image: "nginx:stable"
-    configs:
-      - source: abra_conf
-        target: /etc/nginx/conf.d/abra.conf
-      - source: abra_installer
-        target: /var/www/abra-installer/installer
-    volumes:
-      - "public:/var/www/abra-installer"
-    networks:
-      - proxy
-    deploy:
-      update_config:
-        failure_action: rollback
-        order: start-first
-      labels:
-        - "traefik.enable=true"
-        - "traefik.http.services.abra-installer.loadbalancer.server.port=80"
-        - "traefik.http.routers.abra-installer.rule=Host(`install.abra.autonomic.zone`)"
-        - "traefik.http.routers.abra-installer.entrypoints=web-secure"
-        - "traefik.http.routers.abra-installer.tls.certresolver=production"
-
-configs:
-  abra_installer:
-    file: installer
-  abra_conf:
-    file: nginx.conf
-
-networks:
-  proxy:
-    external: true
-
-volumes:
-  public:
--- a/deploy/install.abra.autonomic.zone/installer
+++ b/deploy/install.abra.autonomic.zone/installer
@ -1,35 +0,0 @@
-#!/bin/bash
-
-ABRA_VERSION="0.7.3"
-GIT_URL="https://git.autonomic.zone/coop-cloud/abra"
-ABRA_SRC="$GIT_URL/raw/tag/$ABRA_VERSION/abra"
-ABRA_DIR="${ABRA_DIR:-$HOME/.abra/}"
-
-function install_abra_release {
-  mkdir -p "$HOME/.local/bin"
-  curl "$ABRA_SRC" > "$HOME/.local/bin/abra"
-  chmod +x "$HOME/.local/bin/abra"
-  echo "abra installed to $HOME/.local/bin/abra"
-}
-
-function install_abra_dev {
-  mkdir -p "$ABRA_DIR/"
-  if [[ ! -d "$ABRA_DIR/src" ]]; then
-    git clone "$GIT_URL" "$ABRA_DIR/src"
-  fi
-  (cd "$ABRA_DIR/src" && git pull origin main && cd - || exit)
-  mkdir -p "$HOME/.local/bin"
-  ln -sf "$ABRA_DIR/src/abra" "$HOME/.local/bin/abra"
-  echo "abra installed to $HOME/.local/bin/abra (development bleeding edge)"
-}
-
-function run_installation {
-  if [ "$1" = "--dev" ]; then
-    install_abra_dev
-  else
-    install_abra_release
-  fi
-}
-
-run_installation "$@"
-exit 0
--- a/deploy/install.abra.autonomic.zone/nginx.conf
+++ b/deploy/install.abra.autonomic.zone/nginx.conf
@ -1,10 +0,0 @@
-server {
-  listen         80 default_server;
-  server_name    install.abra.autonomic.zone;
-
-  location / {
-    root /var/www/abra-installer;
-    add_header Content-Type text/plain;
-    index installer;
-  }
-}
--- a/18
+++ b/18
@ -0,0 +1,18 @@
+#!/bin/bash
+
+ABRA_VERSION="0.1.2"
+ABRA_SRC="https://git.autonomic.zone/autonomic-cooperative/abra/raw/tag/$ABRA_VERSION/abra"
+
+function install_abra {
+  mkdir -p "$HOME/.local/bin"
+	curl "$ABRA_SRC" > "$HOME/.local/bin/abra"
+  chmod +x "$HOME/.local/bin/abra"
+  echo "abra installed to $HOME/.loca/bin/abra"
+}
+
+function run_installation {
+  install_abra
+}
+
+run_installation
+exit 0
--- a/58
+++ b/58
@ -1,58 +0,0 @@
-.PHONY: test shellcheck docopt kcov codecov release-installer
-
-test:
-	@sudo DOCKER_CONTEXT=default docker run \
-		-v $$(pwd):/workdir \
-		--privileged \
-		-d \
-		--name=abra-test-dind \
-		-e DOCKER_TLS_CERTDIR="" \
-		decentral1se/docker-dind-bats-kcov
-	@DOCKER_CONTEXT=default docker exec \
-		-it \
-		abra-test-dind \
-		sh -c "cd /workdir && bats /workdir/tests"
-	@DOCKER_CONTEXT=default docker stop abra-test-dind
-	@DOCKER_CONTEXT=default docker rm abra-test-dind
-
-shellcheck:
-	@docker run \
-		-it \
-		--rm \
-		-v $$(pwd):/workdir \
-		koalaman/shellcheck-alpine \
-		shellcheck /workdir/abra && \
-		shellcheck /workdir/bin/*.sh
-
-docopt:
-	@if [ ! -d ".venv" ]; then \
-		python3 -m venv .venv && \
-		.venv/bin/pip install -U pip setuptools wheel && \
-		.venv/bin/pip install docopt-sh; \
-	fi
-	.venv/bin/docopt.sh abra
-
-kcov:
-	@docker run \
-		-it \
-		--rm \
-		-v $$(pwd):/workdir \
-		kcov/kcov:latest \
-		sh -c "kcov /workdir/coverage /workdir/abra || true"
-
-codecov: SHELL:=/bin/bash
-codecov:
-	@bash <(curl -s https://codecov.io/bash) \
-		-s coverage -t $$(pass show hosts/swarm.autonomic.zone/drone/codecov/token)
-
-release-installer:
-	@DOCKER_CONTEXT=swarm.autonomic.zone \
-		docker stack rm abra-installer-script && \
-		cd deploy/install.abra.autonomic.zone && \
-		DOCKER_CONTEXT=swarm.autonomic.zone docker stack deploy -c compose.yml abra-installer-script
-
-release-apps:
-	@DOCKER_CONTEXT=swarm.autonomic.zone \
-		docker stack rm abra-apps-json && \
-		cd deploy/apps.coopcloud.tech && \
-		DOCKER_CONTEXT=swarm.autonomic.zone docker stack deploy -c compose.yml abra-apps-json
--- a/test/abra.yml
+++ b/test/abra.yml
@ -0,0 +1,7 @@
+---
+name: mystack
+secrets:
+  foo:
+  bar:
+  baz:
+    length: 128
--- a/tests/01_environment.bats
+++ b/tests/01_environment.bats
@ -1,17 +0,0 @@
-#!/usr/bin/env bats
-
-@test "docker is available" {
-    command -v docker
-}
-
-@test "abra is executable" {
-    ./abra --help
-}
-
-@test "git is available" {
-    command -v git
-}
-
-@test "running in a container" {
-    ls /etc/alpine-release
-}
--- a/tests/02_abra.bats
+++ b/tests/02_abra.bats
@ -1,56 +0,0 @@
-#!/usr/bin/env bats
-
-setup() {
-    export ABRA_DIR=$(mktemp -d)
-    mkdir -p $ABRA_DIR/servers/default
-}
-
-teardown() {
-    rm -rf "$ABRA_DIR"
-}
-
-abra() {
-    ./abra -d $@
-}
-
-@test "abra server (add|rm)" {
-    abra server add swarm.test.com
-    docker context ls | grep swarm.test.com
-    [ -d $ABRA_DIR/servers/swarm.test.com ]
-    abra server swarm.test.com rm
-
-    abra server add swarm.test.com foobar 12345
-    [ -d $ABRA_DIR/servers/swarm.test.com ]
-    abra server swarm.test.com rm
-}
-
-@test "abra server init" {
-    abra server default init
-}
-
-@test "abra app (new|rm)" {
-    abra app new --server default --domain traefik.test.com --app-name traefik_test_com traefik
-    [ -f $ABRA_DIR/servers/default/traefik_test_com.env ]
-
-    # interactive prompt
-    echo "y" | abra app traefik_test_com delete
-    [ ! -f $ABRA_DIR/servers/default/traefik_test_com.env ]
-
-    # --no-prompt
-    abra app new --server default --domain traefik_test_com --app-name traefik_test_com traefik
-    abra --no-prompt app traefik_test_com delete
-    [ ! -f $ABRA_DIR/servers/default/traefik_test_com.env ]
-}
-
-@test "abra app <domain> secret (insert|generate|rm)" {
-    abra app new --server default --domain traefik_test_com --app-name traefik_test_com traefik
-
-    abra app traefik_test_com secret insert foobar v1 "foobar"
-
-    # interactive prompt
-    echo "y" | abra app traefik_test_com secret rm foobar
-
-    abra app traefik_test_com secret insert foobar v1 "foobar"
-    #  prompt
-    abra --no-prompt app traefik_test_com secret rm foobar
-}
Author	SHA1	Message	Date
3wc	66013f6ddb	Further tidy-up (+ add warning())	2020-09-23 11:53:34 +02:00
3wc	3d3d6b9b67	Merge branch 'main' into monorepo	2020-09-22 16:15:33 +02:00
3wc	eec1dcb0b2	Update README and tweak default STACK_DIR	2020-09-22 16:13:18 +02:00
3wc	1d52c0f5ed	Add -e, -c and ABRA_STACK_DIR options.. ..and tidy up a little	2020-09-22 16:04:28 +02:00