Arriving at a workable generic backup solution #30

Closed
opened 2 years ago by decentral1se · 11 comments
Owner

Current stab was https://git.autonomic.zone/coop-cloud/backup-bot.

Upsides:

  • Borgbackup, Borgbase, Borgmatic - pretty cutting edge radness
  • Kinda seems to work

Downsides:

  • It requires its own compose configs inside the repository
  • You need to remember to deploy it each time

Something more flexible and disconnected directly from the app repository would be ideal. There has been some research going on, so here comes the links:

Current stab was https://git.autonomic.zone/coop-cloud/backup-bot. Upsides: - Borgbackup, Borgbase, Borgmatic - pretty cutting edge radness - Kinda seems to work Downsides: - It requires its own compose configs inside the repository - You need to remember to deploy it each time Something more flexible and disconnected directly from the app repository would be ideal. There has been some research going on, so here comes the links: - https://github.com/futurice/docker-volume-backup - https://github.com/loomchild/volume-backup - https://github.com/blacklabelops/volumerize - https://janikvonrotz.ch/2020/08/18/backup-docker-volumes-with-this-script/ - https://github.com/ricardobranco777/docker-volumes.sh
Poster
Owner
backup-bot example working configs: > https://git.autonomic.zone/coop-cloud/wordpress/src/branch/master/backup.d
Owner

Use-cases

  • Make & download an ad-hoc .tar.gz backup of all an app's volumes, like Yunohost / Cloudron
    • Include stack config? I think it's available from docker inspect
    • To not be super-frustrating I think the backups should be generated on the server and then downloadable in a separate command (like Yunhohost). We should keep the location of this configurable if possible; we don't currently hard-code any other paths except the Docker socket. So we might want commands to list and prune remote backups
  • Regular off-site automatic backups
    • This is the case that's handled really well by backupbot currently

Features

  • Run custom pre- and post-backup commands (for DB backups)
  • Easy integration into stack definition (env vars, optional compose file if we really need to)
  • Easy per-app config (env vars)
### Use-cases - Make & download an ad-hoc .tar.gz backup of all an app's volumes, like Yunohost / Cloudron - Include stack config? I think it's available from `docker inspect` - To not be super-frustrating I think the backups should be generated on the server and then downloadable in a separate command (like Yunhohost). We should keep the location of this configurable if possible; we don't currently hard-code any other paths except the Docker socket. So we might want commands to list and prune remote backups - Regular off-site automatic backups - This is the case that's handled really well by `backupbot` currently ### Features - Run custom pre- and post-backup commands (for DB backups) - Easy integration into stack definition (env vars, optional compose file if we really need to) - Easy per-app config (env vars)
decentral1se added this to the Public mini-launch milestone 2 years ago
decentral1se added the
design
label 2 years ago
decentral1se added this to the (deleted) project 2 years ago
Poster
Owner

Make & download an ad-hoc .tar.gz backup of all an app's volumes

Coming down the tubes over at https://git.autonomic.zone/coop-cloud/abra/issues/70

> Make & download an ad-hoc .tar.gz backup of all an app's volumes Coming down the tubes over at https://git.autonomic.zone/coop-cloud/abra/issues/70
decentral1se removed this from the Public mini-launch milestone 1 year ago
decentral1se added this to the (deleted) milestone 1 year ago
3wordchant referenced this issue from a commit 1 year ago
Owner

July 2021: Some More Thoughts

  • Prototype (bash? 🙃) which uses a mounted Docker socket to scope the Swarm and look for services
    • Ideally, a listener, but for the moment polling is fine
  • Find a set of coop-cloud.backup-bot label on any service
  • Use the data in those to schedule and run some backups

Based on restic-docker-swarm:

    deploy:
      labels:
        rds.backup: "true"
        rds.backup.repos: "postgres-1,postgres-2"
        rds.backup.at: "* * * * *"
        rds.backup.pre-hook: "pg_dumpall -U postgres -f /tmp/dump/dump.db"
        rds.backup.post-hook: "rm -rf /tmp/dump/dump.db"

Personally I would also love to see backup-bot support restic for those of us who aren't yet borgbase converts.

July 2021: Some More Thoughts - Prototype (bash? 🙃) which uses a mounted Docker socket to scope the Swarm and look for services - Ideally, a listener, but for the moment polling is fine - Find a set of `coop-cloud.backup-bot` label on any service - Use the data in those to schedule and run some backups Based on restic-docker-swarm: ``` deploy: labels: rds.backup: "true" rds.backup.repos: "postgres-1,postgres-2" rds.backup.at: "* * * * *" rds.backup.pre-hook: "pg_dumpall -U postgres -f /tmp/dump/dump.db" rds.backup.post-hook: "rm -rf /tmp/dump/dump.db" ``` Personally I would also love to see `backup-bot` support restic for those of us who aren't yet borgbase converts.
decentral1se added this to the Backup and restore milestone 10 months ago
Poster
Owner
This looks promising: https://github.com/offen/docker-volume-backup
decentral1se added this to the Beta release (software) project 8 months ago
Poster
Owner
- https://git.coopcloud.tech/coop-cloud/ghost/commit/150c953c5136e9d25902fd8e137ba8e3b7188eec - https://git.coopcloud.tech/coop-cloud/backup-bot-two/src/branch/main/backup.sh
Owner

Another thought: abra could also read these labels and run the same commands as backup-bot-two, which could be the path to getting one-off backups & restores working again?

Another thought: abra could also read these labels and run the same commands as backup-bot-two, which could be the path to getting one-off backups & restores working again?
decentral1se added
enhancement
and removed
design
labels 6 months ago
Owner

Another thought: abra could also read these labels and run the same commands as backup-bot-two, which could be the path to getting one-off backups & restores working again?

Expanding on this:

abra app backup foobar app = read the backupbot labels off the app service, run pre/post commands, save backupbot.backup.path to ~/.abra/backups.

abra app backup foobar = read backupbot labels off all services, run as above.

abra app restore foobar app app.tar.gz = copy and extract app.tar.gz to backupbot.backup.path, and run restore-specific pre-/post- commands.

> Another thought: abra could also read these labels and run the same commands as backup-bot-two, which could be the path to getting one-off backups & restores working again? Expanding on this: `abra app backup foobar app` = read the `backupbot` labels off the `app` service, run pre/post commands, save `backupbot.backup.path` to `~/.abra/backups`. `abra app backup foobar` = read `backupbot` labels off all services, run as above. `abra app restore foobar app app.tar.gz` = copy and extract `app.tar.gz` to `backupbot.backup.path`, and run restore-specific pre-/post- commands.
Poster
Owner

@3wordchant more interface confusions on the abra side...

Take the following labels:

service:
  app:
    ...
  deploy:
    ...
    - "backupbot.backup.path=/var/lib/ghost/content,/var/lib/ghost/otherthing"

When I run abra app backup ghost.com app, what should I get as output?

ghost_com_app_<timestamp>.tar.gz with both directories inside? Or ghost_com_app_var_lib_ghost_content_<timestamp>.tar.gz & ghost_com_app_var_lib_ghost_otherthing_<timestamp>.tar.gz? Or another format?

This causes an issue for restore, which paths should be read for the restoring of files? We have abra app restore <domain> <service> <file> but do we also need to add <path> 🙈

From what I remember from the last chat, we don't want to make the filename of the backup file correspond to how it is restored because that will make restoring non co-op cloud apps awkward?

The simplest implementation on abra side would be to generate a backup file per-path & add <path> to the restore UI. But that may be a really not fun backup/restore interface.

Got any ideas?

@3wordchant more interface confusions on the `abra` side... Take the following labels: ```yaml service: app: ... deploy: ... - "backupbot.backup.path=/var/lib/ghost/content,/var/lib/ghost/otherthing" ``` When I run `abra app backup ghost.com app`, what should I get as output? `ghost_com_app_<timestamp>.tar.gz` with both directories inside? Or `ghost_com_app_var_lib_ghost_content_<timestamp>.tar.gz` & `ghost_com_app_var_lib_ghost_otherthing_<timestamp>.tar.gz`? Or another format? This causes an issue for restore, which paths should be read for the restoring of files? We have `abra app restore <domain> <service> <file>` but do we also need to add `<path>` 🙈 From what I remember from the last chat, we don't want to make the filename of the backup file correspond to how it is restored because that will make restoring non co-op cloud apps awkward? The simplest implementation on `abra` side would be to generate a backup file per-path & add `<path>` to the `restore` UI. But that may be a really not fun backup/restore interface. Got any ideas?
decentral1se referenced this issue from a commit 3 months ago
Poster
Owner

The simplest implementation on abra side would be to generate a backup file per-path & add to the restore UI. But that may be a really not fun backup/restore interface.

Available for testing on latest HEAD.

> The simplest implementation on abra side would be to generate a backup file per-path & add <path> to the restore UI. But that may be a really not fun backup/restore interface. Available for testing on latest HEAD.
Poster
Owner

We have arrived 😅 Will be following up on fixes when feedback comes in. Thanks all!

We have arrived 😅 Will be following up on fixes when feedback comes in. Thanks all!
decentral1se closed this issue 3 months ago
Sign in to join this conversation.
Loading…
There is no content yet.