docs: new ssh consolidation changes

See coop-cloud/abra#255

Dockerfile

@@ -1,4 +1,4 @@
-FROM squidfunk/mkdocs-material:8.3.0
+FROM squidfunk/mkdocs-material:9.0.12
 
 EXPOSE 8000
 

docs/abra/cheat-sheet.md

@@ -1,5 +1,5 @@
 ---
-tags: coop-cloud
+title: Cheat sheet
 ---
 
 # Abra cheat sheet
@@ -28,8 +28,7 @@ flags: `-f/--force`, `-C/--chaos`
 flags: `-f/--force`, `-V/--volumes`
 
 ### add/remove server
-- `abra server add $SERVER $USERNAME $SSH_PORT`
-flags: `-p/--provision`, `-l/--local`
+- `abra server add $SERVER`
 - `abra server remove $SERVER`
 flags: `-s/--server`
 

docs/abra/hack.md

@@ -32,15 +32,15 @@ If there's no official release for the architecture you use, you can cross-compi
 
 ## Release management
 
-We use [goreleaser](https://goreleaser.com) to help us automate releases. We use [semver](https://semver.org) for versioning all releases of the tool. While we are still in the public alpha release phase, we will maintain a `0.y.z-alpha` format. Change logs are generated from our commit logs. We are still working this out and aim to refine our release praxis as we go.
+We use [goreleaser](https://goreleaser.com) to help us automate releases. We use [semver](https://semver.org) for versioning all releases of the tool. While we are still in the public beta release phase, we will maintain a `0.y.z-beta` format. Change logs are generated from our commit logs. We are still working this out and aim to refine our release praxis as we go.
 
-For developers, while using this `-alpha` format, the `y` part is the "major" version part. So, if you make breaking changes, you increment that and _not_ the `x` part. So, if you're on `0.1.0-alpha`, then you'd go to `0.1.1-alpha` for a backwards compatible change and `0.2.0-alpha` for a backwards incompatible change.
+For developers, while using this `-beta` format, the `y` part is the "major" version part. So, if you make breaking changes, you increment that and _not_ the `x` part. So, if you're on `0.1.0-beta`, then you'd go to `0.1.1-beta` for a backwards compatible change and `0.2.0-beta` for a backwards incompatible change.
 
 ### Making a new release
 
 - Change `ABRA_VERSION` to match the new tag in [`scripts`](./scripts/installer/installer) (use [semver](https://semver.org))
-- Commit that change (e.g. `git commit -m 'chore: publish next tag x.y.z-alpha'`)
-- Make a new tag (e.g. `git tag -a x.y.z-alpha`)
+- Commit that change (e.g. `git commit -m 'chore: publish next tag x.y.z-beta'`)
+- Make a new tag (e.g. `git tag -a x.y.z-beta`)
 - Push the new tag (e.g. `git push && git push --tags`)
 - Wait until the build finishes on [build.coopcloud.tech](https://build.coopcloud.tech/coop-cloud/abra)
 - Deploy the new installer script (e.g. `cd ./scripts/installer && make`)
@@ -55,3 +55,7 @@ We maintain a fork of [godotenv](https://github.com/Autonomic-Cooperative/godote
 ### `docker/client`
 
 A number of modules in [pkg/upstream](./pkg/upstream) are copy/pasta'd from the upstream [docker/docker/client](https://pkg.go.dev/github.com/docker/docker/client). We had to do this because upstream are not exposing their API as public.
+
+### `github.com/schultz-is/passgen`
+
+Due to [`coop-cloud/organising#358`](https://git.coopcloud.tech/coop-cloud/organising/issues/358).

docs/abra/index.md

@@ -4,7 +4,7 @@ title: Abra
 
 <a href="https://github.com/egonelbre/gophers"><img align="right" width="250" src="https://github.com/egonelbre/gophers/raw/master/.thumb/sketch/adventure/poking-fire.png"/></a>
 
-`abra` is our flagship client & command-line tool which has been developed specifically in the context of the Co-op Cloud project for the purpose of making the day-to-day operations of operators and maintainers pleasant & convenient. It is libre software, written in Go and maintained and extended by the community :heart:
+`abra` is the flagship client & command-line for Co-op Cloud. It has been developed specifically for the purpose of making the day-to-day operations of operators and maintainers pleasant & convenient. It is libre software, written in Go and maintained and extended by the community :heart:
 
 Once you've got `abra` installed, you can start your own Co-op Cloud deployment. `abra` allows you to create, deploy and maintain libre software apps. It supports working with existing servers or can create new servers (supported providers: [Servers.coop](https://servers.coop/) & [Hetzner](https://hetzner.com)). It can also help you manage your DNS configuration (supported providers: [Gandi](https://gandi.net)).
 

docs/abra/install.md

@@ -2,6 +2,10 @@
 title: Install
 ---
 
+!!! warning
+
+    We've seen reports that `abra` under [WSL](https://learn.microsoft.com/en-us/windows/wsl/about) doesn't work due to an underlying bug in Docker context handling. See [`coop-cloud/organising#406`](https://git.coopcloud.tech/coop-cloud/organising/issues/406) and [`docker/for-win#13180`](https://github.com/docker/for-win/issues/13180) for more.
+
 ## Stable release
 
 ```
@@ -17,3 +21,17 @@ curl https://install.abra.coopcloud.tech | bash -s -- --rc
 ## Installer script source
 
 You can view that [here](https://git.coopcloud.tech/coop-cloud/abra/src/branch/main/scripts/installer/installer).
+
+## Using Docker
+
+```
+docker run \
+	-v $HOME/.abra:/.abra \
+	git.coopcloud.tech/coop-cloud/abra app ls
+```
+
+!!! note
+	If you're using symlinks, e.g. for [sharing
+	`~/.abra`](/operators/handbook/#sharing-abra), add more `-v` options for each
+	directory you're symlinking to, e.g. `-v
+	$HOME/Projects/CoopCloud/apps:/home/user/Projects/CoopCloud/apps`

docs/abra/trouble.md

@@ -8,51 +8,21 @@ You can use [this issue tracker](https://git.coopcloud.tech/coop-cloud/abra/issu
 
 ## SSH connection issues?
 
-`abra` tries its best to learn from your system configuration or command-line input what the correct SSH connection details are for a given server. This doesn't always work out. Here are some things to try to fix it.
+When you run `abra server add <host>`, `abra` will read from your `~/.ssh/config` and try to match a `Host <host>` entry. If you can `ssh <host>` then you should be able to `abra server add <host>`.
 
-First, ensure that you can `ssh <my-server>` and things work. If you can't SSH to your server then neither can `abra`. If you have a password protected SSH key, then you'll need to make sure your `ssh-agent` is running and you've added your SSH key part:
+For example, if you do `abra server add example.com`, you should have a matching entry that looks like this:
 
 ```
-eval $(ssh-agent -k)
-ssh-add ~/.ssh/<my-secret-key-part>
-ssh-add -L # validate loaded keys
+Host example.com
+  Hostname example.com
+  User exampleUser
+  Port 12345
+  IdentityFile ~/.ssh/example@somewhere
 ```
 
-The first thing `abra` will check for is the connection details listed in `abra server ls`. Check those details are correct. If you haven't managed to `abra server add` your server yet, then no details will appear in that list. You may need to take a look at [this entry](/abra/trouble/#abra-server-ls-shows-the-wrong-details) to clean up old values depending on your situation.
-
-`abra` will then try to read your `~/.ssh/config` entries and match the server domain against a `Host` entry. So, if you do `ssh myserver.com` and you have:
-
-```
-Host myserver.com
-  Hostname myserver.com
-  User myuser
-  Port 222
-  IdentityFile ~/.ssh/my@myserver.com
-```
-
-Then `abra` should have all it needs to build a working SSH connection. You can validate this by passing `-d/--debug` to your commands.
-
-However, sometimes, you use an alias in your SSH configuration, say:
-
-```
-Host mys
-...
-```
-
-So that you can simply type `ssh mys`. `abra` won't be able to match against those entries to discover connection details. You can use aliases to remedy this:
-
-```
-Host mys, myserver.com
-...
-```
-
-`abra` will try to read the relevant `IdentityFile` entry from your `~/.ssh/config` but if it can't make a match, it will rely on your key being added to the `ssh-agent`.
-
-Due to a limitation in our implementation, `abra` uses 2 methods of making SSH connections, the main `abra` -> `remote docker` connection using `/usr/bin/ssh` which can seamlessly pick up loaded SSH keys. However, for SSH host key checking, `abra` uses an SSH library & Golang SSH internals. We're working on resolving this to a single implementation but it is tricky work.
-
 ## "abra server ls" shows the wrong details?
 
-You can use `abra server rm` to remove the incorrect details. Make sure to take a backup of your `~/.abra/servers/<domain>` first. You can then try to re-create by using `abra server add ...` again, making sure to take care if you need to use `<user> <port>`, see `abra server add -h` for more help on this.
+You can use `abra server rm` to remove the incorrect details. Make sure to take a backup of your `~/.abra/servers/<domain>` first. You can then try to re-create by using `abra server add ...` again.
 
 However, if you have Docker installed on the same machine you have `abra`, then there might be some confusion. If you run `docker context ls` you'll see that Docker uses context connection strings also. `abra` simply uses this approach. Sometimes, your Docker defined context details & your `abra` context details can get out of sync. You can use `docker context rm` to resolve this.
 
@@ -62,7 +32,7 @@ If you need to create a new context from Docker, you can do:
 docker context create <domain> --docker "host=ssh://<user>@<domain>:<port>"
 ```
 
-(This is what we used to before we wrote `abra` to make it more convenient.)
+This is what we used to before we wrote `abra` to make it more convenient.
 
 ## Command-line flag handling is weird?
 
@@ -89,7 +59,7 @@ We're still waiting for upstream patch which resovles this.
 
 We're sorry, it's an issue with an upstream dependency. See [`#291`](https://git.coopcloud.tech/coop-cloud/organising/issues/291) for more.
 
-## I need some feature from the old depreciated bash abra?
+## I need some feature from the old deprecated bash abra?
 
 There is an archive of the [old code here](https://git.coopcloud.tech/coop-cloud/abra-bash).
 
@@ -99,7 +69,3 @@ You can install it alongside the [supported version of Abra](https://git.coopclo
 git clone https://git.coopcloud.tech/coop-cloud/abra-bash ~/.abra/bash-src
 ln -s ~/.abra/bash-src/abra ~/.local/bin/babra
 ```
-
-## I am seeing very weird `lookup <domain> on <ip>: write udp <ip>: write: operation not permitted` errors
-
-You should turn off your VPN. `abra` has trouble dealing with it right now. We welcome change sets to make it work though!

docs/abra/upgrade.md

@@ -18,6 +18,49 @@ abra upgrade --rc
 
 ## Migration guides
 
+### `0.6.x-beta` -> `0.7.x-beta`
+
+> **ALERTA, ALERTA**: this is currently only available via the release
+> candidate channel, using `abra upgrade --rc`. There has been a lot of churn
+> and we're being cautious about releasing this one. Please help us test! We're
+> currently on `0.7.0-rc2-beta`.
+
+- `kadabra`, the app auto-updater is available for general alpha testing! See [these docs](https://docs.coopcloud.tech/operators/tutorial/#automatic-upgrades) for how to get started. Binaries can be found [here](https://git.coopcloud.tech/coop-cloud/abra/releases/tag/0.7.0-rc2-beta).
+
+- **ALERTA, ALERTA**, security related issue: all `$domain.env` env vars are now exposed to the deployment via the `app` service container. Each `FOO=BAR` is exported within the context of the container. If you have any privately committed secrets in your `.env` files, please migrate them to the `secrets: ...` configuration in the recipe. This change was made to facilitate tooling which can support auto-upgrading of apps in a deployment.
+
+- `abra` can no longer install Docker, initialise swarm mode and the proxy network. It will check if a Docker install exists and is in swarm mode or not and error out accordingly. We leave the provisioning to tools that are designed for that and reduce the command-line surface that we have to maintain going forward.
+
+- `abra server add <host> <args>` 👉 `abra server add <host>`. We have finally removed the custom SSH handling code and now solely rely on invoke `/usr/bin/ssh` directly and reading from the `~/.ssh/config`. The `<host>` argument should correspond to a `Host <host>` entry in your `~/.ssh/config` or in an `Include <file>` statement (hosts are retrieved via `ssh -G <host>`). This means "how does `abra` interact with SSH is 1) do you have an `~/.ssh/config` entry for `<host>` 2) can you `ssh <host>` successfully? 3) there is no 3. It's an easier mental model and also the way `abra-bash` works, hence, less weird obscure errors. `<host>` being public a domain name is still required.
+
+- `abra` no longer tries to do the TOFU host key verification prompt. We follow the praxis of the Docker CLI and just give up when host keys are not validated. We leave it to folks to SSH in and verify themselves.
+
+- On the way to [`kadabra`](https://git.coopcloud.tech/coop-cloud/abra/pulls/268), several changes regarding labelling deployments have been merged in this release. This will allow tooling to understand a deployment without having the context of a `~/.abra/...` configuration. This will pave the way for server-side tooling, like `kadabra` which can help operators with different kinds of maintenance tasks.
+
+- Welcome `abra recipe fetch`, which helps retrieve a recipe repository to your local work-station.
+
+- Also say hello to `abra app services <domain>`, which lists the in-deployment service names and corresponding image, e.g. `foo_example_com`.
+
+- Digests have been removed from the catalogue generation.
+
+- Backup files generated by `abra` have a much more human-friendly format.
+
+- Linting for domains is disabled when no `DOMAIN=...` is discovered in the `$odmain.env` file.
+
+### `0.5.x-beta` -> `0.6.x-beta`
+
+- Using `{{ .Domain }}` in recipe `.envrc.sample` files went away because it
+  was portable enough. We revert to replacing e.g `gitea.example.com` with the
+  domain. See
+  [`8fad34e`](https://git.coopcloud.tech/coop-cloud/abra/commit/8fad34e) for
+  more.
+
+- If your `abra.sh` scripts depend on `/bin/sh` and `/bin/bash` is available in
+  the container then `/bin/bash` will be used from now on. `/bin/sh` is only
+  now used if `/bin/bash` is not available. See
+  [`7f745ff`](https://git.coopcloud.tech/coop-cloud/abra/commit/7f745ff) for
+  more.
+
 ### `v0.4.x` -> `v0.5.x`
 
 - The only breaking change was making `abra` understand that the JSON dump for the recipes listing lives on [recipes.coopcloud.tech/recipes.json](https://recipes.coopcloud.tech) instead of [recipes.coopcloud.tech](https://recipes.coopcloud.tech).

docs/glossary/index.md

@@ -40,7 +40,7 @@ A [Docker](glossary#docker) related concept: a virtual network created on the se
 
 ## Recipe
 
-A recipe is what we call the configuration files that are used to deploy an [app](/glossary#app). When you run `abra app deploy <domain>`, `abra` is reading a recipe configuration, such as [the gitea recipe](https://git.coopcloud.tech/coop-cloud/gitea), in order to know how to deploy a new Gitea instance. When we speak of a "digital configuraiton commons", we're primarily referring to the [growing collection of recipes](https://git.coopcloud.tech/coop-cloud).
+A recipe is what we call the configuration files that are used to deploy an [app](/glossary#app). When you run `abra app deploy <domain>`, `abra` is reading a recipe configuration, such as [the gitea recipe](https://git.coopcloud.tech/coop-cloud/gitea), in order to know how to deploy a new Gitea instance. When we speak of a "digital configuration commons", we're primarily referring to the [growing collection of recipes](https://git.coopcloud.tech/coop-cloud).
 
 ## Secret
 

docs/index.md

@@ -16,9 +16,9 @@ We'd be happy to hear feedback about our documentation, if it was helpful, what
 
 !!! danger "Here be dragons"
 
-    This project is still [alpha quality software](https://en.wikipedia.org/wiki/Software_release_life_cycle#Alpha) :bomb: Please take that into consideration if you are thinking about using this system in production. We're working hard to make Co-op Cloud stable. In the meantime, this is a good time to help us out with initial testing, feedback, ideas or [join in with development](/get-involved/).
+    This project is still [beta quality software](https://en.wikipedia.org/wiki/Software_release_life_cycle#Beta) :bomb: Please take that into consideration if you are thinking about using this system in production. We're working hard to make Co-op Cloud stable. In the meantime, this is a good time to help us out with initial testing, feedback, ideas or [join in with development](/get-involved/).
 
-- [Operators guide](/operators/): You run a Co-op Cloud deployment or want to do so :computer:
+- [Operators guide](/operators/): You run a Co-op Cloud based deployment or want to do so :computer:
 
 - [Maintainers guide](/maintainers/): You maintain recipes and ensure things run smoothly for operators :tools:
 

docs/intro/bikemap.md

@@ -2,8 +2,8 @@
 title: Bike map
 ---
 
-- The project is currently in an [alpha quality](https://en.wikipedia.org/wiki/Software_release_life_cycle#Alpha) release state.
+- The project is currently in a [beta quality](https://en.wikipedia.org/wiki/Software_release_life_cycle#Beta) release state.
 
-- We are working towards a [beta release](https://en.wikipedia.org/wiki/Software_release_life_cycle#Beta). Our public Beta goals are listed in the following pad: [Beta Bike Map](https://pad.autonomic.zone/s/C3uuqfSCk).
+- We are working towards a stable `1.0.0` release.
 
 - What we're currently working on is listed on this issue tracker: [`coop-cloud/organising`](https://git.autonomic.zone/coop-cloud/organising/issues).

docs/intro/credits.md

@@ -9,5 +9,5 @@ Special thanks to:
 - [Doop Coop](mailto:cluck@doop.coop), for making a transparent version of the Co-op Cloud logo, and helping with OSX alpha testing.
 - [Social.coop](https://social.coop), for warmly welcoming us onto [`social.coop/@coopcloud`](https://social.coop/@coopcloud).
 - [Servers.coop](https://servers.coop), for hosting our digital infrastructure (website, builds, git hosting, etc.).
-- Every single last one of you heroic & patient alpha/beta testers, you are all comrades of the highest order of kropotkin :heart:
+- Every single last one of you heroic & patient beta testers, you are all comrades of the highest order of kropotkin :heart:
 - [`egonelbre/gophers`](https://github.com/egonelbre/gophers) for the rad gopher logos

docs/intro/managed.md

@@ -6,6 +6,8 @@ title: Managed hosting
 
     If you're a co-operative or a tech collective who wants to appear on this list, please [get in touch](/intro/contact/)! We want to expand the number of service providers using the Co-op Cloud so that project is more widely available to end-users and organisations who can influence the direction and co-fund the development.
 
-The Co-op Cloud is still [alpha quality software](https://en.wikipedia.org/wiki/Software_release_life_cycle#Alpha) :bomb: but you can still work with a tech co-op or collective to host some part or all of your online digital services with it. Organisations who want to support the project can get in touch with Co-op Cloud service providers via the following list for a quote on what they're looking for and how much it will cost. Service providers can then factor in some percentage of the cost to co-fund the development of this project.
+The Co-op Cloud is still [beta quality software](https://en.wikipedia.org/wiki/Software_release_life_cycle#Beta) :bomb: but you can still work with a tech co-op or collective to host some part or all of your online digital services with it. Organisations who want to support the project can get in touch with Co-op Cloud service providers via the following list for a quote on what they're looking for and how much it will cost. Service providers can then factor in some percentage of the cost to co-fund the development of this project.
 
 - [Autonomic Co-op](https://autonomic.zone) (contact: [`helo@autonomic.zone`](mailto:helo@autonomic.zone))
+- [Local-IT](https://local-it.org/) (contact [`info@local-it.org`](mailto:info@local-it.org))
+- [Solisoft](https://solisoft.top) (contact [`contact@solisoft.top`](mailto:contact@solisoft.top))

docs/maintainers/handbook.md

@@ -8,6 +8,15 @@ You can run `abra recipe new <recipe>` to generate a new `~/.abra/recipes/<recip
 
 ## Hacking on an existing recipe
 
+!!! warning
+
+    It is *very advisable* to disable any `healthcheck: ...` configuration
+    while hacking on new recipes. This is because it is very easy to mess up
+    and it will stop Traefik or other web proxies routing the app. You can
+    enable a specific healthcheck later when your recipe is stable. The default
+    "unconfigured" healthcheck behaviour is much less strict and it's faster to
+    get something up and running.
+
 If you want to make changes to an existing recipe then you can simply edit the files in `~/.abra/recipes/<recipe-name>` and run pass `--chaos` to the `deploy` command when deploying those changes. `abra` will not deploy unstaged changes to avoid instability but you can tell it to do so with `--chaos`. This means ou can simple hack away on the existing recipe files on your local file system and then when something is working, submit a change request to the recipe upstream.
 
 ## How is a recipe structured?
@@ -30,7 +39,7 @@ After docker creates the filesystem and copies files into a new container it run
 
 For a simple example check the [entrypoint.sh for `croc`](https://git.coopcloud.tech/coop-cloud/croc/src/commit/2f06e8aac52a3850d527434a26de0a242bea0c79/entrypoint.sh). In this case, `croc` needs the password to be exported as an environmental variable called `CROC_PASS`, and that is exactly what the entrypoint does before running vendor entrypoint.
 
-If you write your own entrypoint, it needs to be specified in the `config` section of compose.yml. See [this handbook entry](http://localhost:8000/maintainers/handbook/#entrypoints) for more.
+If you write your own entrypoint, it needs to be specified in the `config` section of compose.yml. See [this handbook entry](/maintainers/handbook/#how-do-i-set-a-custom-entrypoint) for more.
 
 ### `releases/` directory
 
@@ -224,11 +233,11 @@ file_env "DB_PASSWORD"
 Sometimes the containers don't even have Bash installed on them. You had better just use `/bin/sh` or, in your entrypoint script, install Bash :upside_down: The entrypoint secrets hack listed above doesn't work in this case (as it requires Bash), so instead you can just do `export FOO=$(cat /run/secrets/<secret-name>)`.
 
 
-## Reference services in configs?
+## How do I reference services in configs?
 
 When referencing an `app` service in a config file, you should prefix with the `STACK_NAME` to avoid namespace conflicts (because all these containers sit on the traefik overlay network). You might want to do something like this `{{ env "STACK_NAME" }}_app` (using  the often obscure dark magic of the Golang templating language). You can find examples of this approach used in the [Peertube recipe](https://git.coopcloud.tech/coop-cloud/peertube/src/commit/d1b297c5a6a23a06bf97bb954104ddfd7f736568/nginx.conf.tmpl#L9).
 
-## How are recipes are versioned?
+## How are recipes versioned?
 
 We'll use an example to work through this. Let's use [Gitea](https://hub.docker.com/r/gitea/gitea).
 
@@ -364,7 +373,7 @@ mkdir -p releases
 
 And then create a text file which corresponds to the version release, e.g. `1.1.0+5.9.0` and write some notes. `abra` will show these when another operator runs `abra app deploy` / `abra app upgrade`.
 
-## Generate the recipe catalogue
+## How do I generate the recipe catalogue
 
 To generate an entire new copy of the catalogue:
 
@@ -393,7 +402,7 @@ You can pass `--publish` to have `abra` automatically publish those changes.
 
     In order to have `abra` publish changes for you automatically, you'll have to have write permissons to the git.coopcloud.tech repository and your account must have a working SSH key configuration. `abra` will use the SSH based URL connection details for Git by automagically creating an `origin-ssh` remote in the repository and pushing to it.
 
-## Enable healthchecks
+## How do I enable healthchecks
 
 A healthcheck is an important and often overlooked part of the recipe configuration. It is part of the configuration that the runtime uses to figure out if a container is really up-and-running. You can tweak what command to run, how often and how many times to try until you assume the container is not up.
 
@@ -410,7 +419,7 @@ If you're just starting off with packaging a recipe, you can use `healthcheck: d
 
 `abra app errors -w <domain>` will show what errors are being reported from a failing healtcheck setup.
 
-## Tuning deploy configs
+## How do I tune deploy configs?
 
 A bit like healtchecks, there is no universal setup. A good default seems to be the following configuration:
 
@@ -431,13 +440,13 @@ Setting a restart policy is also good so that the runtime doesn't try to restart
 
 Best to [read](https://docs.docker.com/engine/reference/builder/#healthcheck) [the docs](https://docs.docker.com/compose/compose-file/compose-file-v3/#healthcheck) on this one.
 
-## Tuning resource limits
+## How do I tune resource limits?
 
 If you don't place resource limits on your app it will assume it can use the entire capacity of the server it is on. This can cause issues such as OOM eerors for your entire swarm.
 
 See the [Docker documentation](https://docs.docker.com/config/containers/resource_constraints/) to get into this topic and check the other recipes to see what other maintainers are doing.
 
-## Enable A+ SSL ratings
+## How do I enable A+ SSL ratings?
 
 If you want to get the highest rating on SSL certs, you can use the following traefik labels which use a tweaked Traefik configuration.
 
@@ -448,7 +457,7 @@ If you want to get the highest rating on SSL certs, you can use the following tr
 
 See [this PR](https://git.coopcloud.tech/coop-cloud/traefik/pulls/8/files) for the technical details
 
-## Tweaking secret generation length
+## How do I tweak secret generation length?
 
 It is possible to tell `abra` which length it should generate secrets with from your recipe config.
 
@@ -469,12 +478,13 @@ of passwords which admins have to type out in database shells.
 
 ## How are recipes added to the catalogue?
 
-> This is so far a manual process which requires a member of Autonomic. This is
-> a temporary situation, we want to open out this process & also introduce some
-> automation to support making thie process more convenient. Please nag us to
-> move things along.
+> This is so far a manual process which requires someone who's been added to the
+> `coop-cloud` "Organisation" on https://git.coopcloud.tech. This is a temporary
+> situation, we want to open out this process & also introduce some automation
+> to support making thie process more convenient. Please nag us to move things
+> along.
 
-- Publish your new recipe on the [git.coopcloud.tech](https://git.coopcloud.tech/coop-cloud) listing
+- Publish your new recipe on the [git.coopcloud.tech](https://git.coopcloud.tech/coop-cloud) "Organisation"
 - Run `abra catalogue generate <recipe> -p`
 - Run `cd ~/.abra/catalogue && make`
 
@@ -483,7 +493,7 @@ the [recipe release publishing dance](https://docs.coopcloud.tech/maintainers/ha
 which will then extend the `versions: [...]` section of the published JSON in the catalogue.
 
 Recipes that are not included in the catalogue can still be deployed. It is not
-required to add your recipes to the catalogue but this will improve the
+required to add your recipes to the catalogue, but this will improve the
 visibility for other co-op hosters & end-users.
 
 For now, it is best to [get in touch](https://docs.coopcloud.tech/intro/contact/) if you want to add your recipe to the catalogue.
@@ -502,14 +512,14 @@ Two of the current "blessed" options are
 [`backup-bot-two`](https://git.coopcloud.tech/coop-cloud/backup-bot-two) &
 [`abra`](https://git.coopcloud.tech/coop-cloud/abra).
 
-#### `abra`
-
-`abra` will read labels and store backups in `~/.abra/backups/...`.
-
 #### `backup-bot-two`
 
 Please see the [`README.md`](https://git.coopcloud.tech/coop-cloud/backup-bot-two#backupbot-ii) for the full docs.
 
+#### `abra`
+
+`abra` will read labels and store backups in `~/.abra/backups/...`.
+
 ### Backup
 
 For backup, here are the labels & some examples:
@@ -538,3 +548,91 @@ You can use [this `docker-compose` trick](https://docs.docker.com/compose/extend
 If you have a recipe that is using a `mysql` service and you'd like to use `postgresql` instead, you can create a `compose.psql.yml`!
 
 An example of this is the [`selfoss`](https://git.coopcloud.tech/coop-cloud/selfoss) recipe. The default is `sqlite` but there is a `postgresql` compose configuration there too.
+
+## How do I set a custom entrypoint?
+
+For more context, see the [`entrypoint.sh`](/maintainers/handbook/#entrypointsh) section. The following configuration example is ripped from the [`coop-cloud/peertube`](https://git.coopcloud.tech/coop-cloud/peertube) recipe but shortened down. Here are more or less the steps you need to take:
+
+Define a config:
+
+```yaml
+  app:
+    ...
+    configs:
+      - source: app_entrypoint
+        target: /docker-entrypoint.sh
+        mode: 0555
+    ...
+
+configs:
+  app_entrypoint:
+    name: ${STACK_NAME}_app_entrypoint_${APP_ENTRYPOINT_VERSION}
+    file: entrypoint.sh.tmpl
+    template_driver: golang
+```
+
+Define a `entrypoint.sh.tmpl`:
+
+```
+#!/bin/bash
+
+set -e
+
+file_env() {
+   local var="$1"
+   local fileVar="${var}_FILE"
+   local def="${2:-}"
+
+   if [ "${!var:-}" ] && [ "${!fileVar:-}" ]; then
+      echo >&2 "error: both $var and $fileVar are set (but are exclusive)"
+      exit 1
+   fi
+
+   local val="$def"
+
+   if [ "${!var:-}" ]; then
+      val="${!var}"
+   elif [ "${!fileVar:-}" ]; then
+      val="$(< "${!fileVar}")"
+   fi
+
+   export "$var"="$val"
+   unset "$fileVar"
+}
+
+file_env "PEERTUBE_DB_PASSWORD"
+
+{{ if eq (env "PEERTUBE_SMTP_ENABLED") "1" }}
+file_env "PEERTUBE_SMTP_PASSWORD"
+{{ end }}
+
+{{ if eq (env "PEERTUBE_LIVE_CHAT_ENABLED") "1" }}
+apt -y update && apt install -y prosody && apt -y clean
+mkdir -p /run/prosody && chown prosody:prosody /run/prosody
+{{ end }}
+
+# Copy the client files over to a named volume
+# so that they may be served by nginx directly
+cp -ar /app/client/dist /srv/client
+
+# upstream entrypoint
+# https://github.com/Chocobozzz/PeerTube/blob/66f77f63437c6774acbd72584a9839a7636ea167/support/docker/production/entrypoint.sh
+/usr/local/bin/entrypoint.sh "$@"
+```
+
+Please note:
+
+1. The `file_env` // `_FILE` hack is to pass secrets into the container runtime without exposing them in plaintext in the configuration. See [this entry](/maintainers/handbook/#exposing-secrets) for more.
+
+1. In order to pass execution back to the original entrypoint, it's a good idea to find the original entrypoint script and run it from your own entrypoint script. If there is none, you may want to reference the `CMD` definition or if that isn't working, try to actually specify `cmd: ...` in the `compose.yml` definition (there are other recipes which do this).
+
+1. If you're feeling reckless, you can also use the Golang templating engine to do things conditionally.
+
+Then, wire up the vendored config version:
+
+```
+# abra.sh
+export APP_ENTRYPOINT_VERSION=v5
+```
+
+You should be able to deploy this overriden configuration now.

docs/maintainers/tutorial.md

@@ -21,7 +21,7 @@ The idea scenario is when the upstream project provides both the packaged image
 - **Inspired**: Upstream image, someone else's compose file
 - **On fire**: Upstream image, upstream compose file
 
-### Writing the `compose.yml`
+### Writing / adapting the `compose.yml`
 
 Let's take a practical example, [Matomo web analytics](https://matomo.org/). We'll be making a Docker "swarm-mode" `compose.yml` file.
 
@@ -66,8 +66,8 @@ abra app new matomo --secrets \
  --server swarm.example.com
 ```
 
-Depending on whether you defined any extra environment variables, we didn't so
-far, in this example, you might want to run `abra app config swarm.example.com`
+Depending on whether you defined any extra environment variables -- we didn't so
+far, in this example -- you might want to run `abra app config swarm.example.com`
 to check the configuration.
 
 Otherwise, or once you've done that, go ahead and deploy the app:
@@ -80,4 +80,4 @@ Then, open the `DOMAIN` you configured (you might need to wait a while for Traef
 
 ### Finishing up
 
-You've probably got more questions, check out the [maintainers handbook](/maintainers/handbook)!
+You've probably got more questions, check out the [packaging handbook](/maintainers/handbook)!

docs/operators/handbook.md

@@ -82,6 +82,18 @@ Then, tell your collaborators (e.g. in the repository's `README.md`), to run `ma
 
     We don't currently recommend this, because it might set inaccurate expectations about the security model – remember that, by default, **any user who can deploy apps to a Docker Swarm can manage _any_ app in that swarm**.
 
+### Migrating a server into a repository
+
+Even if you've got your existing server configs in version control, by default, `abra server add` will define the server locally. To move it -- taking the example of `newserver.example.com`:
+
+```
+mv ~/.abra/servers/newserver.example.com ~/coop-cloud-apps/
+cd ~/coop-cloud-apps
+git add newserver.example.com
+git commit
+make link
+```
+
 ## Running abra server side
 
 If you're on an environment where it's hard to run Docker, or command-line programs in general, you might want to install `abra` on a server instead of your local work station.
@@ -231,15 +243,13 @@ The requirements are:
 wget -O- https://get.docker.com | bash
 
 # add user to docker group
-usermod -aG docker $YOURUSERNAMEHERE
+usermod -aG docker $USER
 
 # setup swarm
 docker swarm init
 docker network create -d overlay proxy
 ```
 
-`abra` will do this for you when you run `abra server add --provision`.
-
 ## Managing DNS entries
 
 `abra record ...` can help you manage your DNS entries if you have an account with a supported 3rd party provider. We currently support [Gandi](https://gandi.net). The process of managing DNS with `abra` usually goes like this:
@@ -312,6 +322,10 @@ Running `server add` with `-d/--debug` should help you debug what is going on un
 
 If you need to run a command within a running container you can use `abra app run <domain> <service> <command>`. For example, you could run `abra app run cloud.lumbung.space app bash` to open a new bash terminal session inside your remote container.
 
+## How do I attach on a non-running container?
+
+If you need to run a command on a container that won't start (eg. the container is stuck in a restart loop) you can temporarily disable its default entrypoint by setting it in `compose.yml` to something like ['tail', '-f', '/dev/null'], then redeploy the stack (with `--force --chaos` so you don't need to commit), then [get into the now running container](#how-do-i-attach-to-a-running-container), do your business, and when done revert the compose.yml change and redeploy again. 
+
 ## Can I run Co-op Cloud on ARM?
 
 `@Mayel`:
@@ -331,3 +345,43 @@ If you're app [supports backup/restore](/handbook/#how-do-i-configure-backuprest
 
 With `abra`, you can simply run `abra app backup ...` & `abra app restore ...`.
 Pass `-h` for more information on the specific flags & arguments.
+
+## How do I take a manual database backup?
+
+MySQL / MariaDB:
+
+```
+abra app run foo.bar.com db mysqldump -u root <database> | gzip > ~/.abra/backups/foo.bar.com_db_`date +%F`.sql.gz
+```
+
+Postgres:
+
+```
+abra app run foo.bar.com db pg_dump -u root <database> | gzip > ~/.abra/backups/foo.bar.com_db_`date +%F`.sql.gz
+```
+
+If you get errors about database access:
+- Make sure you've specified the right database user (`root` above) and db name
+- If you have a database password set, you might need to load it from a secret,
+    something like this:
+
+```
+abra app run foo.bar.com db bash -c 'mysqldump -u root -p"$(cat /run/secrets/db_oot_password)" <database>' | gzip > ~/.abra/backups/foo.bar.com_db_`date +%F`.sql.gz
+```
+
+## Can I deploy a recipe without `abra`?
+
+Yes! It's a design goal to keep the recipes not dependent on `abra` or any
+single tool that we develop. This means the configurationc commons can still be
+useful beyond this project. You can deploy a recipe with standard commands like
+so:
+
+```
+set -a
+source example.com.env
+cd ~/.abra/recipes/myrecipe
+docker stack deploy -c compose.yml example_com
+```
+
+`abra` makes all of this more cenvenient but other tooling could follow this
+approach.

docs/operators/tutorial.md

@@ -94,22 +94,22 @@ The tutorial tries to help you make choices about which server and which DNS set
 
 ### Server setup
 
-Co-op Cloud has itself near zero system requirements. You only need to worry about the system resource usage of your apps and the overhead of running containers with the docker runtime (often negligible. If you want to know more, see [this FAQ entry](/faq/#isnt-running-everything-in-containers-inefficient)).
+Co-op Cloud has itself near zero system requirements. You only need to worry about the system resource usage of your apps and the overhead of running containers with the docker runtime (often negligible. If you want to know more, see [this FAQ entry](/intro/faq/#isnt-running-everything-in-containers-inefficient)).
 
-We will deploy a new Nextcloud instance in this guide, so you will only need 1GB of RAM according to [their documentation](https://docs.nextcloud.com/server/latest/admin_manual/installation/system_requirements.html). You may also be interested in this [FAQ entry](/faq/#arent-containers-horrible-from-a-security-perspective) if you are curious about security in the context of containers.
+We will deploy a new Nextcloud instance in this guide, so you will only need 1GB of RAM according to [their documentation](https://docs.nextcloud.com/server/latest/admin_manual/installation/system_requirements.html). You may also be interested in this [FAQ entry](/intro/faq/#arent-containers-horrible-from-a-security-perspective) if you are curious about security in the context of containers.
 
 Most Co-op Cloud deployments have been run on Debian machines so far. Some experiments have been done on single board computers & servers with low resource capacities.
 
 You need to keep port `:80` and `:443` free on your server for web proxying to your apps. Typically, you don't need to keep any other ports free as the core web proxy ([Traefik](https://traefik.io)) keeps all app ports internal to its network. Sometimes however, you need to expose an app port when you need to use a transport which would perform better or more reliably without proxying.
 
-`abra` has support for both creating servers (`abra server new`) & provisioning them (passing `--provision` to `abra server add`) but those are more advanced automation options which are covered in the [handbook](/operators/handbook). For this tutorial, we'll focus on the basics. Assuming you've managed to create a testing VPS with some `$hosting_provider`, you'll need to install Docker, add your user to the Docker group & setup swarm mode:
+`abra` has support for creating servers (`abra server new`) but that is a more advanced automation feature which is covered in the [handbook](/operators/handbook). For this tutorial, we'll focus on the basics. Assuming you've managed to create a testing VPS with some `$hosting_provider`, you'll need to install Docker, add your user to the Docker group & setup swarm mode:
 
 ```
 # docker install convenience script
 wget -O- https://get.docker.com | bash
 
 # add user to docker group
-usermod -aG docker $YOURUSERNAMEHERE
+sudo usermod -aG docker $USER
 
 # setup swarm
 docker swarm init
@@ -162,14 +162,13 @@ If you run into issues during installation, [please report a ticket](https://git
 
 #### Add your server
 
-Now you can connect `abra` with your server. You need to have a working SSH configuration before you can do this. That means you can run `ssh <server-domain>` on your command-line and everything Works :tm:.
+Now you can connect `abra` with your server. You should have a working SSH configuration before you can do this (e.g. a matching `Host <server-domain>` entry in `~/.ssh/config` with the correct SSH connection details). That means you can run `ssh <server-domain>` on your command-line and everything Works :tm:.
 
 ```bash
-abra server add <server-domain> -p
+ssh <server-domain> # make sure it works
+abra server add <server-domain>
 ```
 
-The `-p` or `--provision` flag means that `abra` will install Docker and initialise the [new single-host swarm](https://docs.docker.com/engine/swarm/key-concepts/) on your server. If you've already followed the steps in [the server setup](/operators/tutorial/#server-setup) step, then `abra` should not need to do any work.
-
 It is important to note that `<domain>` here is a publicy accessible domain name which points to your server IP address. `abra` does make sure this is the case and this is done to avoid issues with HTTPS certificate rate limiting.
 
 You will now have a new `~/.abra/` folder on your local file system which stores all the configuration of your Co-op Cloud instance.
@@ -184,10 +183,6 @@ abra server ls
 
     `abra` uses plain 'ol SSH under the hood and aims to make use of your existing SSH configurations in `~/.ssh/config` and interfaces with your running `ssh-agent` for password protected secret key files.
 
-    The `server add` command listed above assumes that that you make SSH connections on port 22 using your current username. If that is not he case, pass the new values as positional arguments. See `abra server add -h` for more on this.
-
-        abra server add <domain> <user> <port> -p
-
     Running `server add` with `-d/--debug` should help you debug what is going on under the hood. It's best to take a moment to read [this troubleshooting entry](/abra/trouble/#ssh-connection-issues) if you're running into SSH connection issues with `abra`.
 
 !!! question "How do I share my configs in `~/.abra`?"
@@ -211,7 +206,7 @@ Choose your newly registered server and specify a domain name.
 You will want to take a look at your generated configuration and tweak the `LETS_ENCRYPT_EMAIL` value. You can do that by running `abra app config`:
 
 ```bash
-abra app config traefik
+abra app config <traefik-domain>
 ```
 
 Every app you deploy will have one of these `.env` files, which contains variables which will be injected into app configurations when deployed. Variables starting with `#` are optional, others are required.
@@ -252,6 +247,46 @@ abra app errors -w <nextcloud-domain> # error catcher
 
 Your new `traefik` instance will detect that a new app is coming up and generate SSL certificates for it. You can see what `traefik` is up to using the same commands above but replacing `<netcloud-domain>` with the `<traefik-domain>` you chose earlier (`abra app ls` will remind you what domains you chose :grinning:).
 
+### Upgrade Nextcloud
+
+To upgrade an app manually to the newest available version run:
+
+```bash
+abra app upgrade <nextcloud-domain>
+```
+
+### Automatic Upgrades
+
+`kadabra` the auto-updater is still under development, use it with care and don't use it in production environments.
+To setup the auto-updater copy the `kadabra` binary to the server and configure a cronjob for regular app upgrades.
+The following script will configure ssmtp for email notifications and setup a cronjob.
+This cronjob checks daily for new app versions, notifies if any kind of update is available and upgrades all apps to the latest patch/minor version.
+
+
+```bash
+apt install ssmtp
+
+cat > /etc/ssmtp/ssmtp.conf << EOF
+mailhub=$MAIL_SERVER:587
+hostname=$MAIL_DOMAIN
+AuthUser=$USER
+AuthPass=$PASSWORD
+FromLineOverride=yes
+UseSTARTTLS=yes
+EOF
+
+cat > /etc/cron.d/abra_updater << EOF
+MAILTO=admin@example.com
+MAILFROM=noreply@example.com
+
+0  6 * * *       root    ~/kadabra notify --major
+30 4 * * *       root    ~/kadabra upgrade --all
+EOF
+
+
+```
+
+
 ## Finishing up
 
 Hopefully you got something running! Well done! The [operators handbook](/operators/handbook) would probably be the next place to go check out if you're looking for more help. Especially on topics of ongoing maintenance.

docs/recipes/index.md

@@ -15,6 +15,65 @@ It aims to be a helpful place to understand the status of apps, who is taking ca
 
 The recipe catalogue is available on [recipes.coopcloud.tech](https://recipes.coopcloud.tech/).
 
+## Status / features / scoring
+
+Each recipe README has a "metadata" section, to help communicate the overall status of the recipe, and which features are supported. Here's an example, from [the Wordpress recipe](https://git.coopcloud.tech/coop-cloud/wordpress/):
+
+```
+<!-- metadata -->
+
+* **Category**: Apps
+* **Status**: 3, stable
+* **Image**: [`wordpress`](https://hub.docker.com/_/wordpress), 4, upstream
+* **Healthcheck**: Yes
+* **Backups**: Yes
+* **Email**: 3
+* **Tests**: 2
+* **SSO**: No
+
+<!-- endmetadata -->
+```
+
+Currently, recipe maintainers need to update the scores in this section manually. The specific meanings of the scores are:
+
+### Status (overall score)
+
+- 5: everything in 4 + Single-Sign-On
+- 4: upstream image, backups, email, healthcheck, integration testing
+- 3: upstream image, missing 1-2 items from 4
+- 2: missing 3-4 items from 4 or no upstream image
+- 1: alpha
+
+### Image
+
+- 4: official upstream image
+- 3: semi-official / actively-maintained image
+- 2: 3rd-party image
+- 1: our own custom image
+
+### Email
+
+- 3: automatic (using environment variables)
+- 2: mostly automatic
+- 1: manual
+- 0: none
+- N/A: app doesn't send email
+
+### CI
+
+- 3: as 2, plus healthcheck
+- 2: auto secrets + networks
+- 1: basic deployment using `stack-ssh-deploy`, manual secrets + networks
+- 0: none
+
+### Single-Sign-On
+
+- 3: automatic (using environment variables)
+- 2: mostly automatic
+- 1: manual
+- 0: none
+- N/A: app doesn't support SSO
+
 ## Wishlist
 
 If you'd like to see a new recipe packaged, make a request on the [recipes-wishlist](https://git.coopcloud.tech/coop-cloud/recipes-wishlist) repository issue tracker.

renovate.json

@@ -1,3 +1,7 @@
 {
-  "$schema": "https://docs.renovatebot.com/renovate-schema.json"
+  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
+  "packageRules": [{
+    "matchUpdateTypes": ["minor", "patch"],
+    "automerge": true
+  }]
 }

requirements.txt

@@ -1,4 +1,4 @@
-mkdocs-awesome-pages-plugin==2.7.0
-mkdocs-material-extensions==1.0.3
-mkdocs-material==8.3.0
-mkdocs==1.3.0
+mkdocs-awesome-pages-plugin==2.8.0
+mkdocs-material-extensions==1.1.1
+mkdocs-material==9.0.12
+mkdocs==1.4.2