You can run `abra recipe new <recipe>` to generate a new `~/.abra/recipes/<recipe>` repository. The generated repository is a copy of [`coop-cloud/example`](https://git.coopcloud.tech/coop-cloud/example).
If you want to make changes to an existing recipe then you can simply edit the files in `~/.abra/recipes/<recipe-name>` and run pass `--chaos` to the `deploy` command when deploying those changes. `abra` will not deploy unstaged changes to avoid instability but you can tell it to do so with `--chaos`. This means you can simply hack away on the existing recipe files on your local file system and then when something is working, submit a change request to the recipe upstream.
This is a [compose specification](https://compose-spec.io/) compliant file that contains a list of: services, secrets, networks, volumes and configs. It describe what is needed to run an app. Whenever you deploy an app, `abra` reads this file.
### `.env.sample`
This file is a skeleton for environmental variables that should be adjusted by the user. Examples include: domain or php extention list. Whenever you create a new app with `abra app new` this file gets copied to the `~/.abra/servers/<server-domain>/<app-domain>.env` and when you run `abra app config <app-domain>` you're editing this file.
### `abra.sh`
The `abra.sh` provides versions for configs that are vendored by the recipe maintainer. See [this handbook entry](/maintainers/handbook/#manage-configs) for more.
### `entrypoint.sh`
After docker creates the filesystem and copies files into a new container it runs what's called an entrypoint. This is usually a shell script that exports some variables and runs the application. Sometimes the vendor entrypoint doesn't do everything that we need it to do. In that case you can write your own entrypoint, do whatever you need to do and then run the vendor entrypoint.
For a simple example check the [entrypoint.sh for `croc`](https://git.coopcloud.tech/coop-cloud/croc/src/commit/2f06e8aac52a3850d527434a26de0a242bea0c79/entrypoint.sh). In this case, `croc` needs the password to be exported as an environmental variable called `CROC_PASS`, and that is exactly what the entrypoint does before running vendor entrypoint.
If you write your own entrypoint, it needs to be specified in the `config` section of compose.yml. See [this handbook entry](/maintainers/handbook/#how-do-i-set-a-custom-entrypoint) for more.
This directory contains text files whose names correspond to the recipe versions which have been released and contain useful tips for operators who are doing upgrade work. See [this handbook entry](/maintainers/handbook/#how-do-i-write-version-release-notes) for more.
### Optional compose files
I.e. `compose.smtp.yml`. These are used to provide non-essential functionality such as (registration) e-mails or single sign on. These are typically loaded by specifying `COMPOSE_FILE="compose.yml:compose.smtp.yml"` in your app `.env` configuration. Then `abra` learns to include these optional files at deploy time. `abra` uses the usual `docker-compose` configuration merging technique when merging all the `compose.**.yml` files together at deploy time.
### Additional configs
If you look at a `compose.yml` file and see a `configs` section, that means this compose file is putting files in the container. This might be used for changing default (vendor) configuration, such as this [fpm-tune.ini file](https://git.coopcloud.tech/coop-cloud/nextcloud/src/commit/28425b6138603067021757de28c639ad464e9cf8/fpm-tune.ini) used to adjust `php-fpm.` See [this handbook entry](/maintainers/handbook/#manage-configs) for more.
## Manage configs
To add additional files into the container, you can use [Docker configs](https://docs.docker.com/engine/swarm/configs/). This usually involves the following:
1. Create the file and add it to your recipe repository
1. Create an entry for this config in your `configs: ...` global stanza
1. Create an entry on the service configuration `configs: ...` stanza
1. Vendor a version in the `abra.sh` of the recipe
An example of a config is an [entrypoint](/maintainers/handbook/#entrypoints), a script run at container run time.
Because configurations are maintained in-repository by maintainers, we version them ourselves. This means that configs changes are seamless to operators unless they cause breaking changes which should be signalled in the new version and release notes. This is in distinction to secrets, which are managed by the operators. For example, operators may need to rotate secrets on a running deployment and should be able to do so at any time. We put the versions in the [`abra.sh`](/maintainers/handbook/#abrash) file.
This defines an env var which then needs to be added by an operator to their app env file. If you would like to add an env var which is optional, you can do:
```bash
#FOO=123
```
In order to expose this env var to recipe configuration, you pass this via the `environment` stanza of a service config in the recipe like so:
Then your environment variable will be threaded into the running app at deploy time. If you run `abra app run <domain> app env | grep FOO` then you'll see it exposed.
You can also access it in your configs using the following syntax:
-`ENABLE_AUTO_UPDATE`: if set to `true`, the auto-updater `kadabra` can update this app (see [this auto updater entry](/operators/tutorial/#automatic-upgrades) for more)
-`POST_DEPLOY_CMDS="<container> <command> <arguments>|<container> <command> <arguments>|... "` specifies commands that should be executed after each `abra app deploy`
-`POST_UPGRADE_CMDS="<container> <command> <arguments>|<container> <command> <arguments>|... "` specifies commands that should be executed after each `abra app upgrade`
Operators manage the secret versions themselves. So we provide a version hook in the environment variables which they control. This allows operators to deal with things like secret rotation without having to rely on recipe maintainers.
They can be useful to install additional dependencies or setup configuration that upstream doesn't have or want to have.
Here's a trimmed down config, the general idea is to create a new config and insert it into the container at a specific location and then have the compose configuration tell the underlying image to run this new script as the entrypoint.
You typically don't want to completely override the upstream entrypoint of the image you're using, so in the last line of your entrypoint, you can run the upstream entrypoint.
Sometimes apps expect to find a secret in their environment which is not possible with the default compose configuration approach. This requires a hack using an entrypoint. The hack is basically this (assume we want to expose a secret called `db_password`):
1. Setup the secret as per usual in `secrets: ...`
2. Pass a `DB_PASSWORD_FILE=/run/secrets/db_password` in via the `environment: ...`
3. Create an entrypoint and inside it, use the following boilerplate.
```bash
#!/bin/bash
set -e
file_env() {
local var="$1"
local fileVar="${var}_FILE"
local def="${2:-}"
if [ "${!var:-}" ] && [ "${!fileVar:-}" ]; then
echo >&2 "error: both $var and $fileVar are set (but are exclusive)"
exit 1
fi
local val="$def"
if [ "${!var:-}" ]; then
val="${!var}"
elif [ "${!fileVar:-}" ]; then
val="$(< "${!fileVar}")"
fi
export "$var"="$val"
unset "$fileVar"
}
```
And then to expose your secret to the container environment use the following in a line below this function:
```bash
file_env "DB_PASSWORD"
```
### `/bin/bash` is missing?
Sometimes the containers don't even have Bash installed on them. You had better just use `/bin/sh` or, in your entrypoint script, install Bash :upside_down: The entrypoint secrets hack listed above doesn't work in this case (as it requires Bash), so instead you can just do `export FOO=$(cat /run/secrets/<secret-name>)`.
When referencing an `app` service in a config file, you should prefix with the `STACK_NAME` to avoid namespace conflicts (because all these containers sit on the traefik overlay network). You might want to do something like this `{{ env "STACK_NAME" }}_app` (using the often obscure dark magic of the Golang templating language). You can find examples of this approach used in the [Peertube recipe](https://git.coopcloud.tech/coop-cloud/peertube/src/commit/d1b297c5a6a23a06bf97bb954104ddfd7f736568/nginx.conf.tmpl#L9).
We'll use an example to work through this. Let's use [Gitea](https://hub.docker.com/r/gitea/gitea).
The Gitea project maintains a version, e.g. `1.14.3`. This version uses the [semver](https://semver.org) strategy for communicating what type of changes are included in each version, i.e., if there is a breaking change, Gitea will release a new version as `2.0.0`.
However, there are other types of changes that can happen for a recipe. Perhaps the database image gets a breaking update or the actual recipe configuration changes some environment variable. This can mean that end-users of the recipe need to do some work to make sure their updates will deploy successfully.
Therefore, we maintain an additional version part, in front of the project version. So, the first release of the Gitea recipe in the Co-op Cloud project has the version of `1.0.0+1.14.3`. This `x.y.z+` is the version part that the recipe maintainer manages. If a new available Gitea version comes out as `1.15` then the recipe maintainer will publish `1.1.0+1.15` as this is a backwards compatible update, following semantic versioning.
In all cases, we follow the semver semantics. So, if we upgrade the Gitea recipe from `1.14.3` to `1.15.3`, we still publish `1.1.0+1.15.3` as our recipe version. In this case, we skipped a few patch releases but it was all backwards compatible, so we only increment the minor version part.
## How do I release a new recipe version?
The commands uses for dealing with recipe versioning in `abra` are:
-`abra recipe upgrade`: upgrade the image tags in the compose configs of a recipe
-`abra recipe sync`: upgrade the deploy labels to match the new recipe version
-`abra recipe release`: publish a git tag for the recipe repo
The `abra` recipe publishing commands have been designed to complement a semi-automatic workflow. If `abra` breaks or doesn't understand what is going on, you can always finish the process manually with a few Git commands and a bit of luck. We designed `abra` to support this way due to the chaotic nature of container publishing versioning schemes.
Let's take a practical example, publishing a new version of [Wordpress](https://git.coopcloud.tech/coop-cloud/wordpress).
If we run `abra recipe upgrade wordpress` (at time of running), we end up with a prompt to upgrade Wordpress to `5.9.0`. We can skip the database upgrade for now. Here is what that looks like:
```
➜ ~ abra recipe upgrade wordpress
? upgrade to which tag? (service: app, image: wordpress, tag: 5.8.3) 5.9.0
? upgrade to which tag? (service: db, image: mariadb, tag: 10.6) skip
WARN[0004] not upgrading mariadb, skipping as requested
```
Now, what happened? `abra` queried the upstream container repositories of all the images listed in the Wordpress recipe configuration and checked if there are new tags available. Once you make some choices on the prompt, `abra` will update the recipe configurations. Let's take a look by running `cd ~/.abra/recipes/wordpress && git diff`:
```diff
diff --git a/compose.yml b/compose.yml
index 1618ef5..6cd754d 100644
--- a/compose.yml
+++ b/compose.yml
@@ -3,7 +3,7 @@ version: "3.8"
services:
app:
- image: "wordpress:5.8.3"
+ image: "wordpress:5.9.0"
volumes:
- "wordpress_content:/var/www/html/wp-content/"
networks:
```
!!! warning "Here be versioning dragons"
`abra` doesn't understand all image tags unfortunately. There are limitations which we're still running into. You can pass `-a` to have `abra` list all available image tags from the upstream repository and then make a choice manually. See [`tagcmp`](https://git.coopcloud.tech/coop-cloud/tagcmp) for more info on how we implement image parsing.
Next, we need to update the label in the recipe, we can do that with `abra recipe sync wordpress`. You'll be prompted by a question asking what kind of upgrade this is. Take a moment to read the output and if it still doesn't make sense, read [this](/maintainers/handbook/#how-are-recipes-are-versioned). Since we're upgrading from `5.8.3` -> `5.9.0`, it is a minor release, so we choose `minor`:
```
➜ wordpress (master) ✗ abra recipe sync wordpress
...
INFO[0088] synced label coop-cloud.${STACK_NAME}.version=1.1.0+5.9.0 to service app
```
Once again, we can run `cd ~/.abra/recipes/wordpress && git diff` to see what `abra` has done for us:
In order to have `abra` publish changes for you automatically, you'll have to have write permissons to the git.coopcloud.tech repository and your account must have a working SSH key configuration. `abra` will use the SSH based URL connection details for Git by automagically creating an `origin-ssh` remote in the repository and pushing to it.
This is currently a manual process. Our best estimates are to do a backup and run a test deployment and see how things go.
Following the [entry above](/maintainers/handbook/#how-do-i-release-a-new-recipe-version), before running `abra recipe release --publish <recipe>`, you can deploy the new version of the recipe. You find an app that relies on this recipe and pass `-C/--chaos` to `ugrade` so that it accepts the locally unstaged changes.
In order to have `abra` publish changes for you automatically, you'll have to have write permissons to the git.coopcloud.tech repository and your account must have a working SSH key configuration. `abra` will use the SSH based URL connection details for Git by automagically creating an `origin-ssh` remote in the repository and pushing to it.
It is good practice to take note of all the issues you ran into and share them with other operators. See [this entry](/maintainers/handbook/#how-do-i-write-version-release-notes) for more.
If you don't have time or are not an operator, reach out on our communication channels for an operator willing to do some testing.
## How do I write version release notes?
In the root of your recipe repository, run the following (if the folder doesn't already exist):
And then create a text file which corresponds to the version release, e.g. `1.1.0+5.9.0` and write some notes. `abra` will show these when another operator runs `abra app deploy` / `abra app upgrade`.
You can pass `--publish` to have `abra` automatically publish those changes.
!!! warning "Here be more SSH dragons"
In order to have `abra` publish changes for you automatically, you'll have to have write permissons to the git.coopcloud.tech repository and your account must have a working SSH key configuration. `abra` will use the SSH based URL connection details for Git by automagically creating an `origin-ssh` remote in the repository and pushing to it.
A healthcheck is an important and often overlooked part of the recipe configuration. It is part of the configuration that the runtime uses to figure out if a container is really up-and-running. You can tweak what command to run, how often and how many times to try until you assume the container is not up.
There are no real univesal configs and most maintainers just pick up what others are doing and try to adapt. There is some testing involved to see what works well. You can browse the existing recipe repositories and see from there.
You'll often find the same one used for things like caches & supporting services, such as Redis:
```yaml
healthcheck:
test: ["CMD", "redis-cli", "ping"]
```
If you're just starting off with packaging a recipe, you can use `healthcheck: disable` until you have something working. It's definitely advised to work out your healthcheck as a last step, it can be a bit tricky.
`abra app errors -w <domain>` will show what errors are being reported from a failing healtcheck setup.
A bit like healtchecks, there is no universal setup. A good default seems to be the following configuration:
```yaml
deploy:
update_config:
failure_action: rollback
order: start-first
rollback_config:
order: start-first
restart_policy:
max_attempts: 3
```
The `start-first` setting ensures that the container runtime tries to start up the new container and get it running before switching over to it.
Setting a restart policy is also good so that the runtime doesn't try to restart the container forever.
Best to [read](https://docs.docker.com/engine/reference/builder/#healthcheck) [the docs](https://docs.docker.com/compose/compose-file/compose-file-v3/#healthcheck) on this one.
If you don't place resource limits on your app it will assume it can use the entire capacity of the server it is on. This can cause issues such as Out-Of Memory errors for your entire swarm.
See the [Docker documentation](https://docs.docker.com/config/containers/resource_constraints/) to get into this topic and check the other recipes to see what other maintainers are doing.
You can use [this `docker-compose` trick](https://docs.docker.com/compose/extends/#understanding-multiple-compose-files) to do this.
If you have a recipe that is using a `mysql` service and you'd like to use `postgresql` instead, you can create a `compose.psql.yml`!
An example of this is the [`selfoss`](https://git.coopcloud.tech/coop-cloud/selfoss) recipe. The default is `sqlite` but there is a `postgresql` compose configuration there too.
For more context, see the [`entrypoint.sh`](/maintainers/handbook/#entrypointsh) section. The following configuration example is ripped from the [`coop-cloud/peertube`](https://git.coopcloud.tech/coop-cloud/peertube) recipe but shortened down. Here are more or less the steps you need to take:
1. The `file_env` / `_FILE` hack is to pass secrets into the container runtime without exposing them in plaintext in the configuration. See [this entry](/maintainers/handbook/#exposing-secrets) for more.
1. In order to pass execution back to the original entrypoint, it's a good idea to find the original entrypoint script and run it from your own entrypoint script. If there is none, you may want to reference the `CMD` definition or if that isn't working, try to actually specify `cmd: ...` in the `compose.yml` definition (there are other recipes which do this).
1. If you're feeling reckless, you can also use the Golang templating engine to do things conditionally.
Then, wire up the vendored config version:
```
# abra.sh
export APP_ENTRYPOINT_VERSION=v5
```
You should be able to deploy this overriden configuration now.
