fix: expose backup spec

docs/specs/backup/index.md

@@ -0,0 +1,3 @@
+---
+title: Backup
+---

docs/specs/backup/maintain.md

@@ -0,0 +1,166 @@
+# For Maintainers
+
+From the perspective of the recipe maintainer, backup/restore is just more
+`deploy: ...` labels. Tools can read these labels and then perform the
+backup/restore logic.
+
+## Tools
+
+Two of the current "blessed" options are, which both implement the [backupbot specification](link to spec)
+
+- [`backup-bot-two`](https://git.coopcloud.tech/coop-cloud/backup-bot-two)
+- [`abra`](https://git.coopcloud.tech/coop-cloud/abra)
+
+### `backup-bot-two`
+
+`backup-bot-two` is a recipe which gets deployed on the server, it can perform automatic backups and uses restic.
+Please see the [`README.md`](https://git.coopcloud.tech/coop-cloud/backup-bot-two#backupbot-ii) for the full docs.
+
+### `abra`
+
+`abra` will read labels and store backups in `~/.abra/backups/...` .
+It also provides an integration for `backup-bot-two`.
+
+## Backup
+
+### How to Configure backups
+
+Unless otherwise stated all labels should be added to the main service (which should be named `app`).
+
+1. Enable backups for the recipe:
+   You need to enable backups for the recipe by adding the following deploy label:
+
+```
+backupbot.backup=true
+```
+
+2. Decide wich volumes should be backed up:
+   By default all volumes will be backed up. To disable a certain volume you can add the following deploy label:
+
+```
+backupbot.backup.volumes.{volume_name}=false
+```
+
+3. Decide which path should be backed up on each volume
+   By default all files get backed up for a volume. To only include certain paths you can add the following deploy label:
+
+```
+backupbot.backup.volumes.{volume_name}.path=/mypath1/foo,/mypath2/bar
+```
+
+Note: You can include multiple paths by providing a comma seperated list
+Note: All paths are specified relativ to the volume root
+
+4. Run commands before the backup
+   For certain services like a database it is not reccomend to just backup files, because the backup might end up in a corrupted state. Instead it is reccomended to make a database dump. You can run arbitrary commands in any container before the files are backed up.
+   To do this add the following deploy label to the service on which you want the command being run:
+
+```
+backupbot.backup.pre-hook=mysqldump -u root -pghost ghost --tab /var/lib/foo
+```
+
+5. Run commands after the backup
+   Sometimes you want to clean up after the backup. You can run arbitrary commands in any container after the files were backed up.
+   To do this add the following deploy label to the service on which you want the command being run:
+
+```
+backupbot.backup.post-hook=rm -rf /var/lib/mysql-files/*
+```
+
+### Testing the backup
+
+To test that your backup is configured correctly you can deploy the recipe you are working on in a test app either [locally](link to local server deployment) or on a test server.
+
+After the deployment is succesfull run the backup and inspect its content
+
+```
+abra app backup myrecipe.example.com
+tar -tf ~/.abra/backups/mybackup
+```
+
+TODO: this is not complete yet
+
+## Restore
+
+When restoring an app, it takes the files from a backup and copies them to their correct location.
+In the case of restoring database tables, you can use the `pre-hook` & `post-hook` commands to run the insertion logic.
+
+## Pre and Post hooks
+
+To back up some services correctly it involves more than just copying a few files from one location to another. Some services already have specific backup tools that allow taking a coherent snapshot of its data like `mysqldump`.
+The pre and post hooks can be used to prepare the files which should get backed up and clean up afterwards.
+
+Here are some examples:
+
+### Example 1: Execute simple command
+
+```
+backupbot.backup.pre-hook: "echo 'foo' > /path/to/volume/bar.txt
+```
+
+### Example 2: Access environment variable
+
+```
+backupbot.backup.pre-hook: "cat $${POSTGRES_PASSWORD_FILE}"
+```
+
+### Example 3: Access secret
+
+```
+backupbot.backup.pre-hook: "cat /var/run/secrets/mysupersecret"
+```
+
+```
+backupbot.backup.pre-hook: 'mysqldump -p"$$(cat /run/secrets/mysupersecret)" mydatabase'
+```
+
+### Example 4: Complex script
+
+Sometimes the logic to backup up a service can get quite complex. In that case it might be easier to add a script (via mount or config) inside the container and call that from the pre and post hook:
+
+```
+backupbot.backup.pre-hook: "/scripts/my-pre-backup-scripts"
+backupbot.backup.post-hook: "/scripts/my-post-backup-scripts"
+```
+
+## Configuration Examples
+
+### Mariadb
+
+```
+services:
+  db:
+    image: mariadb
+    volumes:
+      - "mariadb:/var/lib/mysql"
+    deploy:
+      labels:
+        backupbot.backup: "true"
+        backupbot.backup.pre-hook: "sh -c 'mariadb-dump --single-transaction -u root -p\"$$(cat /run/secrets/db_root_password)\" wordpress | gzip > /var/lib/mysql/dump.sql.gz'"
+        backupbot.backup.volume.mariadb.path: "dump.sql.gz"
+        backupbot.backup.post-hook: "rm -f /var/lib/mysql/dump.sql.gz"
+        backupbot.restore.post-hook: "sh -c 'gzip -d /var/lib/mysql/dump.sql.gz && mariadb -u root -p\"$$(cat /run/secrets/db_root_password)\" wordpress < /var/lib/mysql/dump.sql && rm -f /var/lib/mysql/dump.sql'"
+```
+
+### Postgres
+
+```
+version: '3.8'
+
+services:
+  db:
+    image: "postgres"
+    volumes:
+      - "postgres:/var/lib/postgresql/data"
+    secrets:
+      - db_password
+    deploy:
+      labels:
+            backupbot.backup: "true"
+            backupbot.backup.pre-hook: "PGPASSWORD=$$(cat $${POSTGRES_PASSWORD_FILE}) pg_dump -U $${POSTGRES_USER} $${POSTGRES_DB} > /var/lib/postgresql/data/backup.sql"
+            backupbot.backup.post-hook: "rm -rf /var/lib/postgresql/data/backup.sql"
+            backupbot.backup.volume.postgres.path: "backup.sql"
+
+volumes:
+  postgres:
+```

docs/specs/backup/spec.md

@@ -0,0 +1,136 @@
+# Specification
+
+## Summary
+
+Creating automated backups of docker swarm services is an often needed task. This specification describes how backups can be configured via [service labels](https://docs.docker.com/compose/compose-file/compose-file-v3/#labels-1) in a standardised way.
+
+## Requirements
+
+The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this specification are to be interpreted as described in [RFC-2119](https://datatracker.ietf.org/doc/html/rfc2119).
+
+## Backup
+
+To enable backups for a docker stack, the `backupbot.backup=true` label MUST be set on one of its services. The label MUST NOT be set multiple times for a docker stack. Otherwise the implementation MUST show an error. The label SHOULD be declared on the main service.
+
+### Volumes and paths
+
+By default all volumes MUST be backed up. A volume MUST be excluded from the backup when `backupbot.backup.volumes.{volume_name}=false` is set, where `{volume_name}` is the name of the volume.
+By default all files MUST be backed up on a volume. `backupbot.backup.volumes.{volume_name}.path` MAY be set to limit the paths for that volume. The value MUST be a valid path relative to the volume root. It MAY contain multiple paths which get separated by a comma. When the label is set only the given paths MUST be backed up.
+
+### Pre/Post Hooks
+
+A `backupbot.backup.pre-hook` and `backupbot.backup.post-hook` MAY be set on a service. When set the command MUST be executed inside the running container of the service before/after backing up files.
+There is no guaranteed order in which different hooks MUST be executed.
+
+TODO: escaping
+
+### Output
+
+A backup implementation SHOULD provide the backup of one or multiple stacks in a `.tar.gz` format. In that case each volume MUST be in `/var/lib/docker/volumes/{stack_name}_{volume_name}`, where `{stack_name}` is the name of the docker stack and `{volume_name}` is the name of each volume that got backed up.
+
+## Restore
+
+By default all files MUST be restored into their volume. A volume or path MAY be excluded from restoring. When restoring a backup from a `.tar.gz` it expects the directory layout as described in the [backup output](#output) section.
+
+### Pre/Post Hooks
+
+A `backupbot.restore.pre-hook` and `backupbot.restore.post-hook` MAY be set on a service. When set the command MUST be executed inside the running container of the service before/after restoring the files.
+There is no guaranteed order in which different hooks MUST be executed.
+
+## Labels
+
+### `backupbot.backup`
+
+**Type:** boolean
+**Default:** false
+**Description:**
+Enables backups for this compose stack. The label should be added to the main service of the compose stack.
+
+**Example:**
+
+```
+backupbot.backup: true
+```
+
+### `backupbot.backup.volumes.{volume_name}`
+
+**Type:** boolean
+**Default:** true
+**Description:** When set to false the volume is excluded from backups.
+
+**Example:**
+
+```
+backupbot.backup.volumes.{volume_name}: false
+```
+
+### `backupbot.backup.volumes.{volume_name}.path`
+
+**Type:** string
+**Default:** ""
+**Description:**
+A comma seperated list of paths. When one or more paths are set, it only backs up those on the given volume instead of the whole volume.
+
+**Example 1:**
+
+```
+backupbot.backup.volumes.{volume_name}.path: '/var/lib/mariadb/dump.sql.gz'
+```
+
+**Example 2:**
+```
+backupbot.backup.volumes.{volume_name}.path: '/var/lib/myapp/foo,/var/lib/myapp/bar'
+```
+
+### `backupbot.backup.pre-hook`
+
+**Type:** string
+**Default:** ""
+**Description:**
+A command, that gets executed before the files are backed up.
+
+**Example:**
+
+```
+backupbot.backup.pre-hook: 'mysqldump -u root -p"$(cat /run/secrets/db_root_password)" -f /volume_path/dump.db'
+```
+
+### `backupbot.backup.post-hook`
+
+**Type:** string
+**Default:** ""
+**Description:**
+A command, that gets executed after the files are backed up.
+
+**Example:**
+
+```
+backupbot.backup.post-hook: "rm -rf /volume_path/dump.db"
+```
+
+### `backupbot.restore.pre-hook`
+
+**Type:** string
+**Default:** ""
+**Description:**
+A command, that gets executed before the files are restored.
+Note, that there is no guaranteed order in which multiple hooks get executed.
+
+**Example:**
+
+```
+backupbot.restore.pre-hook: "lock db"
+```
+
+### `backupbot.restore.post-hook`
+
+**Type:** string
+**Default:** ""
+**Description:**
+A command, that gets executed after the files are restored.
+
+**Example:**
+
+```
+backupbot.restore.post-hook: "sqldump dump.sql && unlock db && rm dump.sql"
+```

docs/specs/index.md

@@ -0,0 +1,3 @@
+---
+title: Specifications
+---