toolshed/organising
toolshed/organising
7
1
Fork 0
Code Issues 124 Pull Requests Releases Activity

Arriving at a workable generic backup solution #30

New Issue
Closed
opened 2020-10-25 19:50:32 +00:00 by decentral1se · 11 comments
decentral1se commented 2020-10-25 19:50:32 +00:00
Owner
Copy Link

Current stab was https://git.autonomic.zone/coop-cloud/backup-bot.

Upsides:

  • Borgbackup, Borgbase, Borgmatic - pretty cutting edge radness
  • Kinda seems to work

Downsides:

  • It requires its own compose configs inside the repository
  • You need to remember to deploy it each time

Something more flexible and disconnected directly from the app repository would be ideal. There has been some research going on, so here comes the links:

Current stab was https://git.autonomic.zone/coop-cloud/backup-bot. Upsides: - Borgbackup, Borgbase, Borgmatic - pretty cutting edge radness - Kinda seems to work Downsides: - It requires its own compose configs inside the repository - You need to remember to deploy it each time Something more flexible and disconnected directly from the app repository would be ideal. There has been some research going on, so here comes the links: - https://github.com/futurice/docker-volume-backup - https://github.com/loomchild/volume-backup - https://github.com/blacklabelops/volumerize - https://janikvonrotz.ch/2020/08/18/backup-docker-volumes-with-this-script/ - https://github.com/ricardobranco777/docker-volumes.sh
decentral1se commented 2020-10-25 19:51:48 +00:00
Author
Owner
Copy Link

backup-bot example working configs:

https://git.autonomic.zone/coop-cloud/wordpress/src/branch/master/backup.d

backup-bot example working configs: > https://git.autonomic.zone/coop-cloud/wordpress/src/branch/master/backup.d
3wordchant commented 2020-10-26 14:30:14 +00:00
Owner
Copy Link

Use-cases

  • Make & download an ad-hoc .tar.gz backup of all an app's volumes, like Yunohost / Cloudron
    • Include stack config? I think it's available from docker inspect
    • To not be super-frustrating I think the backups should be generated on the server and then downloadable in a separate command (like Yunhohost). We should keep the location of this configurable if possible; we don't currently hard-code any other paths except the Docker socket. So we might want commands to list and prune remote backups
  • Regular off-site automatic backups
    • This is the case that's handled really well by backupbot currently

Features

  • Run custom pre- and post-backup commands (for DB backups)
  • Easy integration into stack definition (env vars, optional compose file if we really need to)
  • Easy per-app config (env vars)
### Use-cases - Make & download an ad-hoc .tar.gz backup of all an app's volumes, like Yunohost / Cloudron - Include stack config? I think it's available from `docker inspect` - To not be super-frustrating I think the backups should be generated on the server and then downloadable in a separate command (like Yunhohost). We should keep the location of this configurable if possible; we don't currently hard-code any other paths except the Docker socket. So we might want commands to list and prune remote backups - Regular off-site automatic backups - This is the case that's handled really well by `backupbot` currently ### Features - Run custom pre- and post-backup commands (for DB backups) - Easy integration into stack definition (env vars, optional compose file if we really need to) - Easy per-app config (env vars)
decentral1se added this to the (deleted) milestone 2020-10-27 08:04:48 +00:00
decentral1se added the
design
 label 2020-10-27 08:09:14 +00:00
decentral1se added this to the (deleted) project 2020-12-27 11:48:53 +00:00
decentral1se commented 2021-01-01 21:42:08 +00:00
Author
Owner
Copy Link

Make & download an ad-hoc .tar.gz backup of all an app's volumes

Coming down the tubes over at https://git.autonomic.zone/coop-cloud/abra/issues/70

> Make & download an ad-hoc .tar.gz backup of all an app's volumes Coming down the tubes over at https://git.autonomic.zone/coop-cloud/abra/issues/70
decentral1se removed this from the (deleted) milestone 2021-03-02 09:14:57 +00:00
decentral1se added this to the (deleted) milestone 2021-03-02 09:20:38 +00:00
3wordchant referenced this issue from a commit 2021-03-02 16:42:57 +00:00
Add backup/restore
3wordchant commented 2021-07-26 23:31:32 +00:00
Owner
Copy Link

July 2021: Some More Thoughts

  • Prototype (bash? 🙃) which uses a mounted Docker socket to scope the Swarm and look for services
    • Ideally, a listener, but for the moment polling is fine
  • Find a set of coop-cloud.backup-bot label on any service
  • Use the data in those to schedule and run some backups

Based on restic-docker-swarm:

    deploy:
      labels:
        rds.backup: "true"
        rds.backup.repos: "postgres-1,postgres-2"
        rds.backup.at: "* * * * *"
        rds.backup.pre-hook: "pg_dumpall -U postgres -f /tmp/dump/dump.db"
        rds.backup.post-hook: "rm -rf /tmp/dump/dump.db"

Personally I would also love to see backup-bot support restic for those of us who aren't yet borgbase converts.

July 2021: Some More Thoughts - Prototype (bash? 🙃) which uses a mounted Docker socket to scope the Swarm and look for services - Ideally, a listener, but for the moment polling is fine - Find a set of `coop-cloud.backup-bot` label on any service - Use the data in those to schedule and run some backups Based on restic-docker-swarm: ``` deploy: labels: rds.backup: "true" rds.backup.repos: "postgres-1,postgres-2" rds.backup.at: "* * * * *" rds.backup.pre-hook: "pg_dumpall -U postgres -f /tmp/dump/dump.db" rds.backup.post-hook: "rm -rf /tmp/dump/dump.db" ``` Personally I would also love to see `backup-bot` support restic for those of us who aren't yet borgbase converts.
decentral1se added this to the (deleted) milestone 2021-09-09 14:32:32 +00:00
decentral1se commented 2021-09-13 09:38:39 +00:00
Author
Owner
Copy Link

This looks promising: https://github.com/offen/docker-volume-backup

This looks promising: https://github.com/offen/docker-volume-backup
decentral1se added this to the Beta release (software) project 2021-10-25 11:42:27 +00:00
decentral1se commented 2021-10-25 11:43:00 +00:00
Author
Owner
Copy Link
- https://git.coopcloud.tech/coop-cloud/ghost/commit/150c953c5136e9d25902fd8e137ba8e3b7188eec - https://git.coopcloud.tech/coop-cloud/backup-bot-two/src/branch/main/backup.sh
3wordchant commented 2021-11-08 11:44:57 +00:00
Owner
Copy Link

Another thought: abra could also read these labels and run the same commands as backup-bot-two, which could be the path to getting one-off backups & restores working again?

Another thought: abra could also read these labels and run the same commands as backup-bot-two, which could be the path to getting one-off backups & restores working again?
decentral1se added
enhancement
 and removed
design
 labels 2021-12-31 15:42:07 +00:00
3wordchant commented 2022-03-11 09:19:37 +00:00
Owner
Copy Link

Another thought: abra could also read these labels and run the same commands as backup-bot-two, which could be the path to getting one-off backups & restores working again?

Expanding on this:

abra app backup foobar app = read the backupbot labels off the app service, run pre/post commands, save backupbot.backup.path to ~/.abra/backups.

abra app backup foobar = read backupbot labels off all services, run as above.

abra app restore foobar app app.tar.gz = copy and extract app.tar.gz to backupbot.backup.path, and run restore-specific pre-/post- commands.

> Another thought: abra could also read these labels and run the same commands as backup-bot-two, which could be the path to getting one-off backups & restores working again? Expanding on this: `abra app backup foobar app` = read the `backupbot` labels off the `app` service, run pre/post commands, save `backupbot.backup.path` to `~/.abra/backups`. `abra app backup foobar` = read `backupbot` labels off all services, run as above. `abra app restore foobar app app.tar.gz` = copy and extract `app.tar.gz` to `backupbot.backup.path`, and run restore-specific pre-/post- commands.
decentral1se commented 2022-04-03 15:30:06 +00:00
Author
Owner
Copy Link

@3wordchant more interface confusions on the abra side...

Take the following labels:

service:
  app:
    ...
  deploy:
    ...
    - "backupbot.backup.path=/var/lib/ghost/content,/var/lib/ghost/otherthing"

When I run abra app backup ghost.com app, what should I get as output?

ghost_com_app_<timestamp>.tar.gz with both directories inside? Or ghost_com_app_var_lib_ghost_content_<timestamp>.tar.gz & ghost_com_app_var_lib_ghost_otherthing_<timestamp>.tar.gz? Or another format?

This causes an issue for restore, which paths should be read for the restoring of files? We have abra app restore <domain> <service> <file> but do we also need to add <path> 🙈

From what I remember from the last chat, we don't want to make the filename of the backup file correspond to how it is restored because that will make restoring non co-op cloud apps awkward?

The simplest implementation on abra side would be to generate a backup file per-path & add <path> to the restore UI. But that may be a really not fun backup/restore interface.

Got any ideas?

@3wordchant more interface confusions on the `abra` side... Take the following labels: ```yaml service: app: ... deploy: ... - "backupbot.backup.path=/var/lib/ghost/content,/var/lib/ghost/otherthing" ``` When I run `abra app backup ghost.com app`, what should I get as output? `ghost_com_app_<timestamp>.tar.gz` with both directories inside? Or `ghost_com_app_var_lib_ghost_content_<timestamp>.tar.gz` & `ghost_com_app_var_lib_ghost_otherthing_<timestamp>.tar.gz`? Or another format? This causes an issue for restore, which paths should be read for the restoring of files? We have `abra app restore <domain> <service> <file>` but do we also need to add `<path>` 🙈 From what I remember from the last chat, we don't want to make the filename of the backup file correspond to how it is restored because that will make restoring non co-op cloud apps awkward? The simplest implementation on `abra` side would be to generate a backup file per-path & add `<path>` to the `restore` UI. But that may be a really not fun backup/restore interface. Got any ideas?
decentral1se commented 2022-04-03 17:06:39 +00:00
Author
Owner
Copy Link

The simplest implementation on abra side would be to generate a backup file per-path & add to the restore UI. But that may be a really not fun backup/restore interface.

Available for testing on latest HEAD.

> The simplest implementation on abra side would be to generate a backup file per-path & add <path> to the restore UI. But that may be a really not fun backup/restore interface. Available for testing on latest HEAD.
decentral1se commented 2022-04-05 10:26:18 +00:00
Author
Owner
Copy Link

We have arrived 😅 Will be following up on fixes when feedback comes in. Thanks all!

We have arrived 😅 Will be following up on fixes when feedback comes in. Thanks all!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
abra
Everything to do with abra
abra-gandi
Abra Gandi plugin
awaiting-feedback
Ping/pong on comms
backups
bug
Something is not working
build
Go build related issues
ci/cd
Getting the robots into the mix
community organising
Opening this thing up
contributing
Contributors stuff
coopcloud.tech
Our main website
democracy
Democratic decision making
design
Design thinking required
documentation
Let's write things together
duplicate
This issue or pull request already exists
enhancement
New feature
finance
Money things
funding
Anything related to grant funding
good first issue
Easy start with development
help wanted
Need some help
installer
Installation related issues
kadabra
performance
Performance related
proposal
Large change which requires feedback & decisin making
question
More information is needed
recipes.coopcloud.tech
Our recipes catalogue
security
Securing our shit
test
Unit or integration test suite
wontfix
This won't be fixed
No Label
enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project Beta release (software)
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: toolshed/organising#30
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?