From 3f1f57f4ba746247e4db054556c3cb243931ee37 Mon Sep 17 00:00:00 2001 From: p4u1 Date: Wed, 3 Apr 2024 18:06:05 +0200 Subject: [PATCH] initial commit --- Backupbottwo.md | 298 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 298 insertions(+) create mode 100644 Backupbottwo.md diff --git a/Backupbottwo.md b/Backupbottwo.md new file mode 100644 index 0000000..3d968e8 --- /dev/null +++ b/Backupbottwo.md @@ -0,0 +1,298 @@ +--- +title: Backupbot Specification +--- + +# For Operators + +## Deployment + +**1. Create a new app:** + +``` +abra app new backup-bot-two +``` + +**2. Configure :** + +## Usage + +Run the cronjob that creates a backup, including the push notifications and docker logging: abra app cmd app run_cron + +### Create a backup of all apps: + +``` +abra app run app -- backup create +``` + + The apps to backup up need to be deployed + +### Create an individual backup: + +``` +abra app run app -- backup --host create +``` + +### Create a backup to a local repository: + +``` +abra app run app -- backup create -r /backups/restic +``` + +It is recommended to shutdown/undeploy an app before restoring the data + +### Restore the latest snapshot of all including apps: + +``` +abra app run app -- backup restore +``` + +### Restore a specific snapshot of an individual app: + +``` +abra app run app -- backup --host restore --snapshot +``` + +### Show all snapshots: + +``` +abra app run app -- backup snapshots +``` + +### Show all snapshots containing a specific app: + +``` +abra app run app -- backup --host snapshots +``` + +### Show all files inside the latest snapshot (can be very verbose): + +``` +abra app run app -- backup ls +``` + +### Show specific files inside a selected snapshot: + +``` +abra app run app -- backup ls --snapshot --path /var/lib/docker/volumes/ +``` + +### Download files from a snapshot: + +filename=$(abra app run app -- backup download --snapshot --path ) +abra app cp app:$filename . + +# For Maintainers + +TODO: +pre/post hooks +- Simple command +- accessing secrets +- more complex scripts mounted in the container + +From the perspective of the recipe maintainer, backup/restore is just more +`deploy: ...` labels. Tools can read these labels and then perform the +backup/restore logic. + +## Tools + +Two of the current "blessed" options are +[`backup-bot-two`](https://git.coopcloud.tech/coop-cloud/backup-bot-two) & +[`abra`](https://git.coopcloud.tech/coop-cloud/abra). + +### `backup-bot-two` + +`backup-bot-two` is a recipe which gets deployed on the server, it can perform automatic backups. +Please see the [`README.md`](https://git.coopcloud.tech/coop-cloud/backup-bot-two#backupbot-ii) for the full docs. + +### `abra` + +`abra` will read labels and store backups in `~/.abra/backups/...` . + +## Backup + +Unless otherwise stated all labels should be added to the main service (which should be named `app`). + +1. Enable backups for the recipe: +You need to enable backups for the recipe by adding the following label: +``` +backupbot.backup=true +``` + +2. Decide wich volumes should be backed up: +By default all volumes will be backed up. To disable a certain volume you can add the following label: +``` +backupbot.backup.volumes.{volume_name}=false +``` + +3. Decide which path should be backed up on each volume +By default all files get backed up for a volume. To only include certain paths you can add the following label: +``` +backupbot.backup.volumes.{volume_name}.path=/mypath1/foo,/mypath2/bar +``` + +Note: You cann include multiple paths by providing a comma seperated list +Note: All paths are specified relativ to the volume root + +4. Run commands before the backup +For certain services like a database it is not reccomend to just backup files, because the backup might end up in a corrupted state. Instead it is reccomended to make a database dump. You can run arbitrary commands in any container before the files are backed up. +To do this add the following label to the service on which you want the command being run: +``` +backupbot.backup.pre-hook=mysqldump -u root -pghost ghost --tab /var/lib/foo +``` + +5. Run commands after the backup +Sometimes you want to clean up after the backup. You can run arbitrary commands in any container after the files were backed up. +To do this add the following label to the service on which you want the command being run: +``` +backupbot.backup.post-hook=rm -rf /var/lib/mysql-files/* +``` + +### Testing the backup +That's it your recipe can now be backed up. But how can you make sure your configuration is correct? + +TODO: + +### Examples + +## Restore + +Restore, in this context means, "moving a compressed archive back to the +container backup paths". So, if you set +`backupbot.backup.path=/var/lib/foo,/var/lib/bar` and you have a backed up +archive, tooling will unzip files in the archive back to those paths. + +In the case of restoring database tables, you can use the `pre-hook` & +`post-hook` commands to run the insertion logic. + +## Configuration Examples + +### Mysql + +### Postgres + +# Specification + +Backups are enabled for the whole compose stack. +By default all volumes are backup up. +It is possible to disable backups for a volume. +By default the whole volume is backed up. +It is possible to define a path or a list of paths for a volume to backup only those. + +## Summary + +Creating automated backups of docker swarm services is an often needed task. This specification describes how backups can be configured via labels in a standardised way. + +## Requirements + +The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this specification are to be interpreted as described in [RFC-2119]. + +When backing up a docker stack you MUST first check if the `backupbot.backup`. It MUST be set to true, for the backup to happen. + +## Backup + +To enable backups for a docker stack, the `backupbot.backup=true` label MUST be on any of its services. It SHOULD be declared on the first service. +A `backupbot.backup.pre-hook` MAY be set on a service. When set the command MUST be executed inside the running container of the service before backup up files. +By default all volumes MUST be backed up. A volume MAY be excluded from backing up when `backupbot.backup.volumes.{volume_name}=false` is set, where `{volume_name}` is the name of the volume. +By default all files MUST be backed up on a volume. `backupbot.backup.volumes.{volume_name}.path` MAY be set to only include those paths. +A `backupbot.backup.post-hook` MAY be set on a service. When set the command MUST be executed inside the running container of the service after backing up all files. +A backup implementation SHOULD provide the backup of one or multiple stacks in a `.tar.gz` format. + +## Restore + +A `backupbot.restore.pre-hook` MAY be set on a service. When set the command MUST be executed inside the running container of the service before restoring backup files. +By default all files MUST be restored into their volume. A volume or path MAY be excluded from restoring. +A `backupbot.restore.post-hook` MAY be set on a service. When set the command MUST be executed inside the running container of the service after restoring backup files. + +## Labels + +### `backupbot.backup` + +**Type:** boolean +**Default:** false +**Description:** +Enables backupbot for this compose stack. It SHOULD be added to the first service. + +**Example:** + +``` +backupbot.backup: true +``` + +### `backupbot.backup.volumes.{volume_name}` + +**Type:** boolean +**Description:** false, true when the given volume has a path specified +Enables backups for the {volume_name} volume. + +**Example:** + +``` +backupbot.backup.volumes.{volume_name}: true +``` + +### `backupbot.backup.volumes.{volume_name}.path` + +**Type:** string +**Default:** "" +**Description:** +A comma seperated list of paths. When one or more paths are set, it only backups up those on the given volume instead of the whole volume. + +**Example:** + +``` +backupbot.backup.volumes.{volume_name}.path: '/var/lib/mariadb/dump.sql.gz' +``` + +### `backupbot.backup.pre-hook` + +**Type:** string +**Default:** "" +**Description:** +A command, that gets executed before the files are backuped. +TODO: multiple commands? +**Example:** + +``` +backupbot.backup.pre-hook: 'mysqldump -u root -p"$(cat /run/secrets/db_root_password)" -f /volume_path/dump.db' +``` + +### `backupbot.backup.post-hook` + +**Type:** string +**Default:** "" +**Description:** +A command, that gets executed after the files are backuped. +TODO: what todo if the backup fails? should this always run? + +**Example:** + +``` +backupbot.backup.post-hook: "rm -rf /volume_path/dump.db" +``` + +### `backupbot.restore.pre-hook` + +**Type:** string +**Default:** "" +**Description:** +A command, that gets executed before the files are restored. + +**Example:** + +``` +backupbot.restore.pre-hook: "lock db" +``` + +### `backupbot.restore.post-hook` + +**Type:** string +**Default:** "" +**Description:** +A command, that gets executed after the files are restored. +TODO: what todo if the restore fails? should this always run? + +**Example:** + +``` +backupbot.restore.post-hook: "sqldump dump.sql && unlock db && rm dump.sql" +```