mayel
/
docs.coopcloud.tech
forked from coop-cloud/docs.coopcloud.tech
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

reorder + tweak for clarity

This commit is contained in:
decentral1se 2022-02-03 21:43:47 +01:00
parent 2cff469151
commit 53003edbed
Signed by untrusted user: decentral1se
GPG Key ID: 03789458B3D0C410
1 changed files with 212 additions and 139 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

351
docs/maintainers/handbook.md
View File

@ -6,6 +6,218 @@ title: Packaging handbook
You can run `abra recipe new <recipe>` to generate a new `~/.abra/recipes/<recipe>` repository. The generated repository is a copy of [`coop-cloud/example`](https://git.coopcloud.tech/coop-cloud/example).
## How is a recipe structured?
### `compose.yml`
This is a [compose specification](https://compose-spec.io/) compliant file that contains a list of: services, secrets, networks, volumes and configs. It describe what is needed to run an app. Whenever you deploy an app, `abra` reads this file.
### `.env.sample`
This file is a skeleton for environmental variables that should be adjusted by the user. Examples include: domain or php extention list. Whenever you create a new app with `abra app new` this file gets copied to the `~/.abra/servers/<server-domain>/<app-domain>.env` and when you run `abra app config <app-domain>` you're editing this file.
### `abra.sh`
The `abra.sh` provides versions for configs that are vendored by the recipe maintainer. See [this handbook entry](/maintainers/handbook/#manage-configs) for more.
### `entrypoint.sh`
After docker creates the filesystem and copies files into a new container it runs what's called an entrypoint. This is usually a shell script that exports some variables and runs the application. Sometimes the vendor entrypoint doesn't do everything that we need it to do. In that case you can write your own entrypoint, do whatever you need to do and then run the vendor entrypoint.
For a simple example check the [entrypoint.sh for `croc`](https://git.coopcloud.tech/coop-cloud/croc/src/commit/2f06e8aac52a3850d527434a26de0a242bea0c79/entrypoint.sh). In this case, `croc` needs the password to be exported as an environmental variable called `CROC_PASS`, and that is exactly what the entrypoint does before running vendor entrypoint.
If you write your own entrypoint, it needs to be specified in the `config` section of compose.yml. See [this handbook entry](http://localhost:8000/maintainers/handbook/#entrypoints) for more.
### `releases/` directory
This directory contains text files whose names correspond to the recipe versions which have been released and contain useful tips for operators who are doing upgrade work. See [this handbook entry](/maintainers/handbook/#how-do-i-write-version-release-notes) for more.
### Optional compose files
I.e. `compose.smtp.yml`. These are used to provide non-essential functionality such as (registration) e-mails or single sign on. These are typically loaded by specifying `COMPOSE_FILE="compose.yml:compose.smtp.yml"` in your app `.env` configuration. Then `abra` learns to include these optional files at deploy time. `abra` uses the usual `docker-compose` configuration merging technique when merging all the `compose.**.yml` files together at deploy time.
### Additional configs
If you look at a `compose.yml` file and see a `configs` section, that means this compose file is putting files in the container. This might be used for changing default (vendor) configuration, such as this [fpm-tune.ini file](https://git.coopcloud.tech/coop-cloud/nextcloud/src/commit/28425b6138603067021757de28c639ad464e9cf8/fpm-tune.ini) used to adjust `php-fpm.` See [this handbook entry](/maintainers/handbook/#manage-configs) for more.
## Manage configs
To add additional files into the container, you can use [Docker configs](https://docs.docker.com/engine/swarm/configs/). This usually involves the following:
1. Create the file and add it to your recipe repository
1. Create an entry for this config in your `configs: ...` global stanza
1. Create an entry on the service configuration `configs: ...` stanza
1. Vendor a version in the `abra.sh` of the recipe
An example of a config is an [entrypoint](/maintainers/handbook/#entrypoints), a script run at container run time.
```yaml
# compose.yml
services:
app:
configs:
- source: nginx_config
target: /etc/nginx/nginx.conf
configs:
nginx_config:
name: ${STACK_NAME}_nginx_config_${NGINX_CONFIG_VERSION}
file: nginx.conf.tmpl
template_driver: golang
```
```bash
# abra.sh
export NGINX_CONFIG_VERSION=v1
```
## Manage environment variables
When you define an environment variable in a `.env.sample` for a recipe, such as:
```bash
FOO=123
```
And you pass this via the `environment` stanza of a service config in the recipe like so:
```yaml
service:
app:
environment:
- FOO
```
Then your environment variable will be threaded into the running app at deploy time. If you run `abra app run <domain> app env | grep FOO` then you'll see it exposed.
You can also access it in your configs using the following syntax:
```go
{{ env "FOO" }}
```
## Manage secret data
Adding a secret to your recipe is done:
1. Create an entry in the `secrets: ...` global stanza
1. Add the `<SECRET-NAME>_VERSION=v1` to your `.env.sample`
1. Ensure that the secret is listed on the service configuration under `secrets: ...`
It might look something like this:
```yaml
# compose.yml
services:
app:
secrets:
- db_password
secrets:
db_password:
external: true
name: ${STACK_NAME}_db_password_${SECRET_DB_PASSWORD_VERSION}
```
```bash
# .env.sample
SECRET_DB_PASSWORD_VERSION=v1
```
If you need to access this secret in a config, say:
```yaml
configs:
someconfig:
name: ${STACK_NAME}_someconfig_${SOME_CONFIG_VERSION}
file: entrypoint.sh.tmpl
template_driver: golang
```
Don't forget the `template_driver: golang`, it won't work otherwise. Then you can use the following syntax to access the secret:
```go
# someconfig.conf
{{ secret "db_password"}}
```
## Entrypoints
### Custom entrypoints
They can be useful to install additional dependencies or setup configuration that upstream doesn't have or want to have.
Here's a trimmed down config, the general idea is to create a new config and insert it into the container at a specific location and then have the compose configuration tell the underlying image to run this new script as the entrypoint.
You typically don't want to completely override the upstream entrypoint of the image you're using, so in the last line of your entrypoint, you can run the upstream entrypoint.
```yaml
services:
app:
entrypoint: /docker-entrypoint.sh
configs:
- source: app_entrypoint
target: /docker-entrypoint.sh
mode: 0555
configs:
app_entrypoint:
name: ${STACK_NAME}_app_entrypoint_${APP_ENTRYPOINT_VERSION}
file: entrypoint.sh.tmpl
template_driver: golang
```
### Exposing secrets
Sometimes apps expect to find a secret in their environment which is not possible with the default compose configuration approach. This requires a hack using an entrypoint. The hack is basically this (assume we want to expose a secret called `db_password`):
1. Setup the secret as per usual in `secrets: ...`
2. Pass a `DB_PASSWORD_FILE=/run/secrets/db_password` in via the `environment: ...`
3. Create an entrypoint and inside it, use the following boilerplate.
```bash
#!/bin/bash
set -e
file_env() {
local var="$1"
local fileVar="${var}_FILE"
local def="${2:-}"
if [ "${!var:-}" ] && [ "${!fileVar:-}" ]; then
echo >&2 "error: both $var and $fileVar are set (but are exclusive)"
exit 1
fi
local val="$def"
if [ "${!var:-}" ]; then
val="${!var}"
elif [ "${!fileVar:-}" ]; then
val="$(< "${!fileVar}")"
fi
export "$var"="$val"
unset "$fileVar"
}
```
And then to expose your secret to the container environment use the following in a line below this function:
```bash
file_env "DB_PASSWORD"
```
### `/bin/bash` is missing?
Sometimes the containers don't even have Bash installed on them. You had better just use `/bin/sh` or, in your entrypoint script, install Bash :upside_down: The entrypoint secrets hack listed above doesn't work in this case (as it requires Bash), so instead you can just do `export FOO=$(cat /run/secrets/<secret-name>)`.
## Reference services in configs?
When referencing an `app` service in a config file, you should prefix with the `STACK_NAME` to avoid namespace conflicts (because all these containers sit on the traefik overlay network). You might want to do something like this `{{ env "STACK_NAME" }}_app` (using the often obscure dark magic of the Golang templating language). You can find examples of this approach used in the [Peertube recipe](https://git.coopcloud.tech/coop-cloud/peertube/src/commit/d1b297c5a6a23a06bf97bb954104ddfd7f736568/nginx.conf.tmpl#L9).
## How are recipes are versioned?
We'll use an example to work through this. Let's use [Gitea](https://hub.docker.com/r/gitea/gitea).
@ -171,44 +383,6 @@ You can pass `--publish` to have `abra` automatically publish those changes.
In order to have `abra` publish changes for you automatically, you'll have to have write permissons to the git.coopcloud.tech repository and your account must have a working SSH key configuration. `abra` will use the SSH based URL connection details for Git by automagically creating an `origin-ssh` remote in the repository and pushing to it.
## How is a recipe structured?
### `compose.yml`
This is a [compose specification](https://compose-spec.io/) compliant file that contains a list of:
- services
- secrets
- networks
- volumes
- configs
It describe what is needed to run an app. Whenever you deploy an app, `abra` reads this file.
### `.env.sample`
This file is a skeleton for environmental variables that should be adjusted by the user. Examples include: domain or php extention list. Whenever you create a new app with `abra app new` this file gets copied to the `~/.abra/servers/<server-domain>/<app-domain>.env` and when you run `abra app config <app-domain>` you're editing this file.
### `abra.sh`
The `abra.sh` provides versions for configs that are vendored by the recipe maintainer.
### `entrypoint.sh`
After docker creates the filesystem and copies files into a new container it runs what's called an entrypoint. This is usually a shell script that exports some variables and runs the application. Sometimes the vendor entrypoint doesn't do everything that we need it to do. In that case you can write your own entrypoint, do whatever you need to do and then run the vendor entrypoint.
For a simple example check the [entrypoint.sh for `croc`](https://git.coopcloud.tech/coop-cloud/croc/src/commit/2f06e8aac52a3850d527434a26de0a242bea0c79/entrypoint.sh). In this case, `croc` needs the password to be exported as an environmental variable called `CROC_PASS`, and that is exactly what the entrypoint does before running vendor entrypoint.
If you write your own entrypoint, it needs to be specified in the `config` section of compose.yml.
### Optional compose files
I.e. `compose.smtp.yml`. These are used to provide non-essential functionality such as (registration) e-mails or single sign on. These are typically loaded by specifying `COMPOSE_FILE="compose.yml:compose.smtp.yml"` in your app `.env` configuration. Then `abra` learns to include these optional files at deploy time. `abra` uses the usual `docker-compose` configuration merging technique when merging all the `compose.**.yml` files together at deploy time.
### Compose configs
If you look at a `compose.yml` file and see a `configs` section, that means this compose file is putting files in the container. This might be used for changing default (vendor) configuration, such as this [fpm-tune.ini file](https://git.coopcloud.tech/coop-cloud/nextcloud/src/commit/28425b6138603067021757de28c639ad464e9cf8/fpm-tune.ini) used to adjust `php-fpm.`
## Enable healthchecks
A healthcheck is an important and often overlooked part of the recipe configuration. It is part of the configuration that the runtime uses to figure out if a container is really up-and-running. You can tweak what command to run, how often and how many times to try until you assume the container is not up.
@ -263,104 +437,3 @@ If you want to get the highest rating on SSL certs, you can use the following tr
```
See [this PR](https://git.coopcloud.tech/coop-cloud/traefik/pulls/8/files) for the technical details
## Thread environment variables
When you define an environment variable in a `.env.sample` for a recipe, such as:
```bash
FOO=123
```
And you pass this via the `environment` stanza of a service config in the recipe like so:
```yaml
service:
app:
environment:
- FOO
```
Then your environment variable will be threaded into the running app at deploy time. If you run `abra app run <domain> app env | grep FOO` then you'll see it exposed.
You can also access it in your configs using the following syntax:
```go
{{ env "FOO" }}
```
## Entrypoints
### Custom entrypoints
They can be useful to install additional dependencies or setup configuration that upstream doesn't have or want to have.
Here's a trimmed down config, the general idea is to create a new config and insert it into the container at a specific location and then have the compose configuration tell the underlying image to run this new script as the entrypoint.
You typically don't want to completely override the upstream entrypoint of the image you're using, so in the last line of your entrypoint, you can run the upstream entrypoint.
```yaml
services:
app:
entrypoint: /docker-entrypoint.sh
configs:
- source: app_entrypoint
target: /docker-entrypoint.sh
mode: 0555
configs:
app_entrypoint:
name: ${STACK_NAME}_app_entrypoint_${APP_ENTRYPOINT_VERSION}
file: entrypoint.sh.tmpl
template_driver: golang
```
### Exposing secrets
Sometimes apps expect to find a secret in their environment which is not possible with the default compose configuration approach. This requires a hack using an entrypoint. The hack is basically this (assume we want to expose a secret called `db_password`):
1. Setup the secret as per usual in `secrets: ...`
2. Pass a `DB_PASSWORD_FILE=/run/secrets/db_password` in via the `environment: ...`
3. Create an entrypoint and inside it, use the following boilerplate.
```bash
#!/bin/bash
set -e
file_env() {
local var="$1"
local fileVar="${var}_FILE"
local def="${2:-}"
if [ "${!var:-}" ] && [ "${!fileVar:-}" ]; then
echo >&2 "error: both $var and $fileVar are set (but are exclusive)"
exit 1
fi
local val="$def"
if [ "${!var:-}" ]; then
val="${!var}"
elif [ "${!fileVar:-}" ]; then
val="$(< "${!fileVar}")"
fi
export "$var"="$val"
unset "$fileVar"
}
```
And then to expose your secret to the container environment use the following in a line below this function:
```bash
file_env "DB_PASSWORD"
```
### `/bin/bash` is missing?
Sometimes the containers don't even have Bash installed on them. You had better just use `/bin/sh` or, in your entrypoint script, install Bash :upside_down: The entrypoint secrets hack listed above doesn't work in this case (as it requires Bash), so instead you can just do `export FOO=$(cat /run/secrets/<secret-name>)`.
## Referening service names in configs?
When referencing an `app` service in a config file, you should prefix with the `STACK_NAME` to avoid namespace conflicts (because all these containers sit on the traefik overlay network). You might want to do something like this `{{ env "STACK_NAME" }}_app` (using the often obscure dark magic of the Golang templating language). You can find examples of this approach used in the [Peertube recipe](https://git.coopcloud.tech/coop-cloud/peertube/src/commit/d1b297c5a6a23a06bf97bb954104ddfd7f736568/nginx.conf.tmpl#L9).