forked from toolshed/docs.coopcloud.tech
		
	Compare commits
	
		
			1 Commits
		
	
	
		
			backupbott
			...
			main
		
	
	| Author | SHA1 | Date | |
|---|---|---|---|
| a42ab2e486 | 
							
								
								
									
										166
									
								
								Maintainers.md
									
									
									
									
									
								
							
							
						
						
									
										166
									
								
								Maintainers.md
									
									
									
									
									
								
							| @ -1,166 +0,0 @@ | |||||||
| # For Maintainers |  | ||||||
|  |  | ||||||
| From the perspective of the recipe maintainer, backup/restore is just more |  | ||||||
| `deploy: ...` labels. Tools can read these labels and then perform the |  | ||||||
| backup/restore logic. |  | ||||||
|  |  | ||||||
| ## Tools |  | ||||||
|  |  | ||||||
| Two of the current "blessed" options are, which both implement the [backupbot specification](link to spec) |  | ||||||
|  |  | ||||||
| - [`backup-bot-two`](https://git.coopcloud.tech/coop-cloud/backup-bot-two) |  | ||||||
| - [`abra`](https://git.coopcloud.tech/coop-cloud/abra) |  | ||||||
|  |  | ||||||
| ### `backup-bot-two` |  | ||||||
|  |  | ||||||
| `backup-bot-two` is a recipe which gets deployed on the server, it can perform automatic backups and uses restic. |  | ||||||
| Please see the [`README.md`](https://git.coopcloud.tech/coop-cloud/backup-bot-two#backupbot-ii) for the full docs. |  | ||||||
|  |  | ||||||
| ### `abra` |  | ||||||
|  |  | ||||||
| `abra` will read labels and store backups in `~/.abra/backups/...` . |  | ||||||
| It also provides an integration for `backup-bot-two`. |  | ||||||
|  |  | ||||||
| ## Backup |  | ||||||
|  |  | ||||||
| ### How to Configure backups |  | ||||||
|  |  | ||||||
| Unless otherwise stated all labels should be added to the main service (which should be named `app`). |  | ||||||
|  |  | ||||||
| 1. Enable backups for the recipe: |  | ||||||
|    You need to enable backups for the recipe by adding the following deploy label: |  | ||||||
|  |  | ||||||
| ``` |  | ||||||
| backupbot.backup=true |  | ||||||
| ``` |  | ||||||
|  |  | ||||||
| 2. Decide wich volumes should be backed up: |  | ||||||
|    By default all volumes will be backed up. To disable a certain volume you can add the following deploy label: |  | ||||||
|  |  | ||||||
| ``` |  | ||||||
| backupbot.backup.volumes.{volume_name}=false |  | ||||||
| ``` |  | ||||||
|  |  | ||||||
| 3. Decide which path should be backed up on each volume |  | ||||||
|    By default all files get backed up for a volume. To only include certain paths you can add the following deploy label: |  | ||||||
|  |  | ||||||
| ``` |  | ||||||
| backupbot.backup.volumes.{volume_name}.path=/mypath1/foo,/mypath2/bar |  | ||||||
| ``` |  | ||||||
|  |  | ||||||
| Note: You can include multiple paths by providing a comma seperated list |  | ||||||
| Note: All paths are specified relativ to the volume root |  | ||||||
|  |  | ||||||
| 4. Run commands before the backup |  | ||||||
|    For certain services like a database it is not reccomend to just backup files, because the backup might end up in a corrupted state. Instead it is reccomended to make a database dump. You can run arbitrary commands in any container before the files are backed up. |  | ||||||
|    To do this add the following deploy label to the service on which you want the command being run: |  | ||||||
|  |  | ||||||
| ``` |  | ||||||
| backupbot.backup.pre-hook=mysqldump -u root -pghost ghost --tab /var/lib/foo |  | ||||||
| ``` |  | ||||||
|  |  | ||||||
| 5. Run commands after the backup |  | ||||||
|    Sometimes you want to clean up after the backup. You can run arbitrary commands in any container after the files were backed up. |  | ||||||
|    To do this add the following deploy label to the service on which you want the command being run: |  | ||||||
|  |  | ||||||
| ``` |  | ||||||
| backupbot.backup.post-hook=rm -rf /var/lib/mysql-files/* |  | ||||||
| ``` |  | ||||||
|  |  | ||||||
| ### Testing the backup |  | ||||||
|  |  | ||||||
| To test that your backup is configured correctly you can deploy the recipe you are working on in a test app either [locally](link to local server deployment) or on a test server. |  | ||||||
|  |  | ||||||
| After the deployment is succesfull run the backup and inspect its content |  | ||||||
|  |  | ||||||
| ``` |  | ||||||
| abra app backup myrecipe.example.com |  | ||||||
| tar -tf ~/.abra/backups/mybackup |  | ||||||
| ``` |  | ||||||
|  |  | ||||||
| TODO: this is not complete yet |  | ||||||
|  |  | ||||||
| ## Restore |  | ||||||
|  |  | ||||||
| When restoring an app, it takes the files from a backup and copies them to their correct location. |  | ||||||
| In the case of restoring database tables, you can use the `pre-hook` & `post-hook` commands to run the insertion logic. |  | ||||||
|  |  | ||||||
| ## Pre and Post hooks |  | ||||||
|  |  | ||||||
| To back up some services correctly it involves more than just copying a few files from one location to another. Some services already have specific backup tools that allow taking a coherent snapshot of its data like `mysqldump`. |  | ||||||
| The pre and post hooks can be used to prepare the files which should get backed up and clean up afterwards. |  | ||||||
|  |  | ||||||
| Here are some examples: |  | ||||||
|  |  | ||||||
| ### Example 1: Execute simple command |  | ||||||
|  |  | ||||||
| ``` |  | ||||||
| backupbot.backup.pre-hook: "echo 'foo' > /path/to/volume/bar.txt |  | ||||||
| ``` |  | ||||||
|  |  | ||||||
| ### Example 2: Access environment variable |  | ||||||
|  |  | ||||||
| ``` |  | ||||||
| backupbot.backup.pre-hook: "cat $${POSTGRES_PASSWORD_FILE}" |  | ||||||
| ``` |  | ||||||
|  |  | ||||||
| ### Example 3: Access secret |  | ||||||
|  |  | ||||||
| ``` |  | ||||||
| backupbot.backup.pre-hook: "cat /var/run/secrets/mysupersecret" |  | ||||||
| ``` |  | ||||||
|  |  | ||||||
| ``` |  | ||||||
| backupbot.backup.pre-hook: 'mysqldump -p"$$(cat /run/secrets/mysupersecret)" mydatabase' |  | ||||||
| ``` |  | ||||||
|  |  | ||||||
| ### Example 4: Complex script |  | ||||||
|  |  | ||||||
| Sometimes the logic to backup up a service can get quite complex. In that case it might be easier to add a script (via mount or config) inside the container and call that from the pre and post hook: |  | ||||||
|  |  | ||||||
| ``` |  | ||||||
| backupbot.backup.pre-hook: "/scripts/my-pre-backup-scripts" |  | ||||||
| backupbot.backup.post-hook: "/scripts/my-post-backup-scripts" |  | ||||||
| ``` |  | ||||||
|  |  | ||||||
| ## Configuration Examples |  | ||||||
|  |  | ||||||
| ### Mariadb |  | ||||||
|  |  | ||||||
| ``` |  | ||||||
| services: |  | ||||||
|   db: |  | ||||||
|     image: mariadb |  | ||||||
|     volumes: |  | ||||||
|       - "mariadb:/var/lib/mysql" |  | ||||||
|     deploy: |  | ||||||
|       labels: |  | ||||||
|         backupbot.backup: "true" |  | ||||||
|         backupbot.backup.pre-hook: "sh -c 'mariadb-dump --single-transaction -u root -p\"$$(cat /run/secrets/db_root_password)\" wordpress | gzip > /var/lib/mysql/dump.sql.gz'" |  | ||||||
|         backupbot.backup.volume.mariadb.path: "dump.sql.gz" |  | ||||||
|         backupbot.backup.post-hook: "rm -f /var/lib/mysql/dump.sql.gz" |  | ||||||
|         backupbot.restore.post-hook: "sh -c 'gzip -d /var/lib/mysql/dump.sql.gz && mariadb -u root -p\"$$(cat /run/secrets/db_root_password)\" wordpress < /var/lib/mysql/dump.sql && rm -f /var/lib/mysql/dump.sql'" |  | ||||||
| ``` |  | ||||||
|  |  | ||||||
| ### Postgres |  | ||||||
|  |  | ||||||
| ``` |  | ||||||
| version: '3.8' |  | ||||||
|  |  | ||||||
| services: |  | ||||||
|   db: |  | ||||||
|     image: "postgres" |  | ||||||
|     volumes: |  | ||||||
|       - "postgres:/var/lib/postgresql/data" |  | ||||||
|     secrets: |  | ||||||
|       - db_password |  | ||||||
|     deploy: |  | ||||||
|       labels: |  | ||||||
|             backupbot.backup: "true" |  | ||||||
|             backupbot.backup.pre-hook: "PGPASSWORD=$$(cat $${POSTGRES_PASSWORD_FILE}) pg_dump -U $${POSTGRES_USER} $${POSTGRES_DB} > /var/lib/postgresql/data/backup.sql" |  | ||||||
|             backupbot.backup.post-hook: "rm -rf /var/lib/postgresql/data/backup.sql" |  | ||||||
|             backupbot.backup.volume.postgres.path: "backup.sql" |  | ||||||
|  |  | ||||||
| volumes: |  | ||||||
|   postgres: |  | ||||||
| ``` |  | ||||||
							
								
								
									
										136
									
								
								Specification.md
									
									
									
									
									
								
							
							
						
						
									
										136
									
								
								Specification.md
									
									
									
									
									
								
							| @ -1,136 +0,0 @@ | |||||||
| # Specification |  | ||||||
|  |  | ||||||
| ## Summary |  | ||||||
|  |  | ||||||
| Creating automated backups of docker swarm services is an often needed task. This specification describes how backups can be configured via [service labels](https://docs.docker.com/compose/compose-file/compose-file-v3/#labels-1) in a standardised way. |  | ||||||
|  |  | ||||||
| ## Requirements |  | ||||||
|  |  | ||||||
| The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this specification are to be interpreted as described in [RFC-2119](https://datatracker.ietf.org/doc/html/rfc2119). |  | ||||||
|  |  | ||||||
| ## Backup |  | ||||||
|  |  | ||||||
| To enable backups for a docker stack, the `backupbot.backup=true` label MUST be set on one of its services. The label MUST NOT be set multiple times for a docker stack. Otherwise the implementation MUST show an error. The label SHOULD be declared on the main service. |  | ||||||
|  |  | ||||||
| ### Volumes and paths |  | ||||||
|  |  | ||||||
| By default all volumes MUST be backed up. A volume MUST be excluded from the backup when `backupbot.backup.volumes.{volume_name}=false` is set, where `{volume_name}` is the name of the volume. |  | ||||||
| By default all files MUST be backed up on a volume. `backupbot.backup.volumes.{volume_name}.path` MAY be set to limit the paths for that volume. The value MUST be a valid path relative to the volume root. It MAY contain multiple paths which get separated by a comma. When the label is set only the given paths MUST be backed up. |  | ||||||
|  |  | ||||||
| ### Pre/Post Hooks |  | ||||||
|  |  | ||||||
| A `backupbot.backup.pre-hook` and `backupbot.backup.post-hook` MAY be set on a service. When set the command MUST be executed inside the running container of the service before/after backing up files. |  | ||||||
| There is no guaranteed order in which different hooks MUST be executed. |  | ||||||
|  |  | ||||||
| TODO: escaping |  | ||||||
|  |  | ||||||
| ### Output |  | ||||||
|  |  | ||||||
| A backup implementation SHOULD provide the backup of one or multiple stacks in a `.tar.gz` format. In that case each volume MUST be in `/var/lib/docker/volumes/{stack_name}_{volume_name}`, where `{stack_name}` is the name of the docker stack and `{volume_name}` is the name of each volume that got backed up. |  | ||||||
|  |  | ||||||
| ## Restore |  | ||||||
|  |  | ||||||
| By default all files MUST be restored into their volume. A volume or path MAY be excluded from restoring. When restoring a backup from a `.tar.gz` it expects the directory layout as described in the [backup output](#output) section. |  | ||||||
|  |  | ||||||
| ### Pre/Post Hooks |  | ||||||
|  |  | ||||||
| A `backupbot.restore.pre-hook` and `backupbot.restore.post-hook` MAY be set on a service. When set the command MUST be executed inside the running container of the service before/after restoring the files. |  | ||||||
| There is no guaranteed order in which different hooks MUST be executed. |  | ||||||
|  |  | ||||||
| ## Labels |  | ||||||
|  |  | ||||||
| ### `backupbot.backup` |  | ||||||
|  |  | ||||||
| **Type:** boolean |  | ||||||
| **Default:** false |  | ||||||
| **Description:** |  | ||||||
| Enables backups for this compose stack. The label should be added to the main service of the compose stack. |  | ||||||
|  |  | ||||||
| **Example:** |  | ||||||
|  |  | ||||||
| ``` |  | ||||||
| backupbot.backup: true |  | ||||||
| ``` |  | ||||||
|  |  | ||||||
| ### `backupbot.backup.volumes.{volume_name}` |  | ||||||
|  |  | ||||||
| **Type:** boolean |  | ||||||
| **Default:** true |  | ||||||
| **Description:** When set to false the volume is excluded from backups. |  | ||||||
|  |  | ||||||
| **Example:** |  | ||||||
|  |  | ||||||
| ``` |  | ||||||
| backupbot.backup.volumes.{volume_name}: false |  | ||||||
| ``` |  | ||||||
|  |  | ||||||
| ### `backupbot.backup.volumes.{volume_name}.path` |  | ||||||
|  |  | ||||||
| **Type:** string |  | ||||||
| **Default:** "" |  | ||||||
| **Description:** |  | ||||||
| A comma seperated list of paths. When one or more paths are set, it only backs up those on the given volume instead of the whole volume. |  | ||||||
|  |  | ||||||
| **Example 1:** |  | ||||||
|  |  | ||||||
| ``` |  | ||||||
| backupbot.backup.volumes.{volume_name}.path: '/var/lib/mariadb/dump.sql.gz' |  | ||||||
| ``` |  | ||||||
|  |  | ||||||
| **Example 2:** |  | ||||||
| ``` |  | ||||||
| backupbot.backup.volumes.{volume_name}.path: '/var/lib/myapp/foo,/var/lib/myapp/bar' |  | ||||||
| ``` |  | ||||||
|  |  | ||||||
| ### `backupbot.backup.pre-hook` |  | ||||||
|  |  | ||||||
| **Type:** string |  | ||||||
| **Default:** "" |  | ||||||
| **Description:** |  | ||||||
| A command, that gets executed before the files are backed up. |  | ||||||
|  |  | ||||||
| **Example:** |  | ||||||
|  |  | ||||||
| ``` |  | ||||||
| backupbot.backup.pre-hook: 'mysqldump -u root -p"$(cat /run/secrets/db_root_password)" -f /volume_path/dump.db' |  | ||||||
| ``` |  | ||||||
|  |  | ||||||
| ### `backupbot.backup.post-hook` |  | ||||||
|  |  | ||||||
| **Type:** string |  | ||||||
| **Default:** "" |  | ||||||
| **Description:** |  | ||||||
| A command, that gets executed after the files are backed up. |  | ||||||
|  |  | ||||||
| **Example:** |  | ||||||
|  |  | ||||||
| ``` |  | ||||||
| backupbot.backup.post-hook: "rm -rf /volume_path/dump.db" |  | ||||||
| ``` |  | ||||||
|  |  | ||||||
| ### `backupbot.restore.pre-hook` |  | ||||||
|  |  | ||||||
| **Type:** string |  | ||||||
| **Default:** "" |  | ||||||
| **Description:** |  | ||||||
| A command, that gets executed before the files are restored. |  | ||||||
| Note, that there is no guaranteed order in which multiple hooks get executed. |  | ||||||
|  |  | ||||||
| **Example:** |  | ||||||
|  |  | ||||||
| ``` |  | ||||||
| backupbot.restore.pre-hook: "lock db" |  | ||||||
| ``` |  | ||||||
|  |  | ||||||
| ### `backupbot.restore.post-hook` |  | ||||||
|  |  | ||||||
| **Type:** string |  | ||||||
| **Default:** "" |  | ||||||
| **Description:** |  | ||||||
| A command, that gets executed after the files are restored. |  | ||||||
|  |  | ||||||
| **Example:** |  | ||||||
|  |  | ||||||
| ``` |  | ||||||
| backupbot.restore.post-hook: "sqldump dump.sql && unlock db && rm dump.sql" |  | ||||||
| ``` |  | ||||||
| @ -12,7 +12,7 @@ title: Membership | |||||||
| | [Doop.coop](https://doop.coop) | - | - | `@yusf:gottsnack.net` | | | [Doop.coop](https://doop.coop) | - | - | `@yusf:gottsnack.net` | | ||||||
| | [EOTL](https://eotl.supply) | - | - | `@basebuilder:pub.solar` | | | [EOTL](https://eotl.supply) | - | - | `@basebuilder:pub.solar` | | ||||||
| | [Karrot](https://karrot.world) | - | - | `@nicksellen:matrix.org` | | | [Karrot](https://karrot.world) | - | - | `@nicksellen:matrix.org` | | ||||||
| | [Klasse & Methode](https://codeberg.org/Klasse-Methode) | - | - | `@p4u1_f4u1:matrix.org` | | | [Klasse & Methode](https://klasse-methode.it) | - | - | `@p4u1_f4u1:matrix.org` | | ||||||
| | [Local IT](https://local-it.org/)  | - | - | Philipp (`@yksflip:matrix.kaputt.cloud`) + `@moritz:matrix.local-it.org` | | | [Local IT](https://local-it.org/)  | - | - | Philipp (`@yksflip:matrix.kaputt.cloud`) + `@moritz:matrix.local-it.org` | | ||||||
| | Mirsal ™  | - | - | `@mirsal:1312.media` | | | Mirsal ™  | - | - | `@mirsal:1312.media` | | ||||||
| | [UTAW](https://utaw.tech) | -  | - | `@javielico:matrix.org` | | | [UTAW](https://utaw.tech) | -  | - | `@javielico:matrix.org` | | ||||||
|  | |||||||
		Reference in New Issue
	
	Block a user