Arriving at a workable generic backup solution #30

New Issue

2020-10-25T19:50:32Z

decentral1se commented

2020-10-25 19:50:32 +00:00

Current stab was https://git.autonomic.zone/coop-cloud/backup-bot.

Upsides:

Borgbackup, Borgbase, Borgmatic - pretty cutting edge radness
Kinda seems to work

Downsides:

It requires its own compose configs inside the repository
You need to remember to deploy it each time

Something more flexible and disconnected directly from the app repository would be ideal. There has been some research going on, so here comes the links:

Current stab was https://git.autonomic.zone/coop-cloud/backup-bot. Upsides: - Borgbackup, Borgbase, Borgmatic - pretty cutting edge radness - Kinda seems to work Downsides: - It requires its own compose configs inside the repository - You need to remember to deploy it each time Something more flexible and disconnected directly from the app repository would be ideal. There has been some research going on, so here comes the links: - https://github.com/futurice/docker-volume-backup - https://github.com/loomchild/volume-backup - https://github.com/blacklabelops/volumerize - https://janikvonrotz.ch/2020/08/18/backup-docker-volumes-with-this-script/ - https://github.com/ricardobranco777/docker-volumes.sh

decentral1se commented

2020-10-25 19:51:48 +00:00

backup-bot example working configs:

https://git.autonomic.zone/coop-cloud/wordpress/src/branch/master/backup.d

backup-bot example working configs: > https://git.autonomic.zone/coop-cloud/wordpress/src/branch/master/backup.d

3wordchant commented

2020-10-26 14:30:14 +00:00

Use-cases

Make & download an ad-hoc .tar.gz backup of all an app's volumes, like Yunohost / Cloudron
- Include stack config? I think it's available from docker inspect
- To not be super-frustrating I think the backups should be generated on the server and then downloadable in a separate command (like Yunhohost). We should keep the location of this configurable if possible; we don't currently hard-code any other paths except the Docker socket. So we might want commands to list and prune remote backups
Regular off-site automatic backups
- This is the case that's handled really well by backupbot currently

Features

Run custom pre- and post-backup commands (for DB backups)
Easy integration into stack definition (env vars, optional compose file if we really need to)
Easy per-app config (env vars)

### Use-cases - Make & download an ad-hoc .tar.gz backup of all an app's volumes, like Yunohost / Cloudron - Include stack config? I think it's available from `docker inspect` - To not be super-frustrating I think the backups should be generated on the server and then downloadable in a separate command (like Yunhohost). We should keep the location of this configurable if possible; we don't currently hard-code any other paths except the Docker socket. So we might want commands to list and prune remote backups - Regular off-site automatic backups - This is the case that's handled really well by `backupbot` currently ### Features - Run custom pre- and post-backup commands (for DB backups) - Easy integration into stack definition (env vars, optional compose file if we really need to) - Easy per-app config (env vars)

decentral1se added this to the (deleted) milestone 2020-10-27 08:04:48 +00:00

decentral1se added the design label 2020-10-27 08:09:14 +00:00

decentral1se added this to the (deleted) project 2020-12-27 11:48:53 +00:00

decentral1se commented

2021-01-01 21:42:08 +00:00

Make & download an ad-hoc .tar.gz backup of all an app's volumes

Coming down the tubes over at https://git.autonomic.zone/coop-cloud/abra/issues/70

> Make & download an ad-hoc .tar.gz backup of all an app's volumes Coming down the tubes over at https://git.autonomic.zone/coop-cloud/abra/issues/70

decentral1se removed this from the (deleted) milestone 2021-03-02 09:14:57 +00:00

decentral1se added this to the (deleted) milestone 2021-03-02 09:20:38 +00:00

3wordchant referenced this issue from a commit

2021-03-02 16:42:57 +00:00

Add backup/restore

3wordchant commented

2021-07-26 23:31:32 +00:00

July 2021: Some More Thoughts

Prototype (bash? 🙃) which uses a mounted Docker socket to scope the Swarm and look for services
- Ideally, a listener, but for the moment polling is fine
Find a set of coop-cloud.backup-bot label on any service
Use the data in those to schedule and run some backups

Based on restic-docker-swarm:

    deploy:
      labels:
        rds.backup: "true"
        rds.backup.repos: "postgres-1,postgres-2"
        rds.backup.at: "* * * * *"
        rds.backup.pre-hook: "pg_dumpall -U postgres -f /tmp/dump/dump.db"
        rds.backup.post-hook: "rm -rf /tmp/dump/dump.db"

Personally I would also love to see backup-bot support restic for those of us who aren't yet borgbase converts.

July 2021: Some More Thoughts - Prototype (bash? 🙃) which uses a mounted Docker socket to scope the Swarm and look for services - Ideally, a listener, but for the moment polling is fine - Find a set of `coop-cloud.backup-bot` label on any service - Use the data in those to schedule and run some backups Based on restic-docker-swarm: ``` deploy: labels: rds.backup: "true" rds.backup.repos: "postgres-1,postgres-2" rds.backup.at: "* * * * *" rds.backup.pre-hook: "pg_dumpall -U postgres -f /tmp/dump/dump.db" rds.backup.post-hook: "rm -rf /tmp/dump/dump.db" ``` Personally I would also love to see `backup-bot` support restic for those of us who aren't yet borgbase converts.

decentral1se added this to the (deleted) milestone 2021-09-09 14:32:32 +00:00

decentral1se commented

2021-09-13 09:38:39 +00:00

This looks promising: https://github.com/offen/docker-volume-backup

decentral1se added this to the Beta release (software) project 2021-10-25 11:42:27 +00:00

decentral1se commented

2021-10-25 11:43:00 +00:00

- https://git.coopcloud.tech/coop-cloud/ghost/commit/150c953c5136e9d25902fd8e137ba8e3b7188eec - https://git.coopcloud.tech/coop-cloud/backup-bot-two/src/branch/main/backup.sh

3wordchant commented

2021-11-08 11:44:57 +00:00

Another thought: abra could also read these labels and run the same commands as backup-bot-two, which could be the path to getting one-off backups & restores working again?

decentral1se added enhancement and removed design labels 2021-12-31 15:42:07 +00:00

3wordchant commented

2022-03-11 09:19:37 +00:00

Another thought: abra could also read these labels and run the same commands as backup-bot-two, which could be the path to getting one-off backups & restores working again?

Expanding on this:

abra app backup foobar app = read the backupbot labels off the app service, run pre/post commands, save backupbot.backup.path to ~/.abra/backups.

abra app backup foobar = read backupbot labels off all services, run as above.

abra app restore foobar app app.tar.gz = copy and extract app.tar.gz to backupbot.backup.path, and run restore-specific pre-/post- commands.

> Another thought: abra could also read these labels and run the same commands as backup-bot-two, which could be the path to getting one-off backups & restores working again? Expanding on this: `abra app backup foobar app` = read the `backupbot` labels off the `app` service, run pre/post commands, save `backupbot.backup.path` to `~/.abra/backups`. `abra app backup foobar` = read `backupbot` labels off all services, run as above. `abra app restore foobar app app.tar.gz` = copy and extract `app.tar.gz` to `backupbot.backup.path`, and run restore-specific pre-/post- commands.

decentral1se referenced this issue from toolshed/abra

2022-03-30 23:55:47 +00:00

feat: backup/restore #174

decentral1se commented

2022-04-03 15:30:06 +00:00

@3wordchant more interface confusions on the abra side...

Take the following labels:

service:
  app:
    ...
  deploy:
    ...
    - "backupbot.backup.path=/var/lib/ghost/content,/var/lib/ghost/otherthing"

When I run abra app backup ghost.com app, what should I get as output?

ghost_com_app_<timestamp>.tar.gz with both directories inside? Or ghost_com_app_var_lib_ghost_content_<timestamp>.tar.gz & ghost_com_app_var_lib_ghost_otherthing_<timestamp>.tar.gz? Or another format?

This causes an issue for restore, which paths should be read for the restoring of files? We have abra app restore <domain> <service> <file> but do we also need to add <path> 🙈

From what I remember from the last chat, we don't want to make the filename of the backup file correspond to how it is restored because that will make restoring non co-op cloud apps awkward?

The simplest implementation on abra side would be to generate a backup file per-path & add <path> to the restore UI. But that may be a really not fun backup/restore interface.

Got any ideas?

@3wordchant more interface confusions on the `abra` side... Take the following labels: ```yaml service: app: ... deploy: ... - "backupbot.backup.path=/var/lib/ghost/content,/var/lib/ghost/otherthing" ``` When I run `abra app backup ghost.com app`, what should I get as output? `ghost_com_app_<timestamp>.tar.gz` with both directories inside? Or `ghost_com_app_var_lib_ghost_content_<timestamp>.tar.gz` & `ghost_com_app_var_lib_ghost_otherthing_<timestamp>.tar.gz`? Or another format? This causes an issue for restore, which paths should be read for the restoring of files? We have `abra app restore <domain> <service> <file>` but do we also need to add `<path>` 🙈 From what I remember from the last chat, we don't want to make the filename of the backup file correspond to how it is restored because that will make restoring non co-op cloud apps awkward? The simplest implementation on `abra` side would be to generate a backup file per-path & add `<path>` to the `restore` UI. But that may be a really not fun backup/restore interface. Got any ideas?

decentral1se referenced this issue from a commit

2022-04-03 16:25:16 +00:00

feat: backup/restore [ci skip]

decentral1se referenced this issue from toolshed/abra

2022-04-03 17:05:42 +00:00

feat: backup/restore #174

decentral1se commented

2022-04-03 17:06:39 +00:00

The simplest implementation on abra side would be to generate a backup file per-path & add to the restore UI. But that may be a really not fun backup/restore interface.

Available for testing on latest HEAD.

> The simplest implementation on abra side would be to generate a backup file per-path & add <path> to the restore UI. But that may be a really not fun backup/restore interface. Available for testing on latest HEAD.

decentral1se commented

2022-04-05 10:26:18 +00:00

We have arrived 😅 Will be following up on fixes when feedback comes in. Thanks all!

decentral1se closed this issue

2022-04-05 10:26:19 +00:00

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: toolshed/organising#30