Email Alert for failed backups #24
Closed
opened 2023-09-07 21:00:58 +00:00 by moritz
·
9 comments
No Branch/Tag Specified
main
renovate/docker-29.x
feature/makeDeleteDefault
restic-exporter
cmd-setup-restic-s3
prom-mon
feature/dockerfile
bb2-classic
feature/selective_paths
list
enable-label
backupbot_revolution
backup_volumes
multi_path
2.4.0+2.4.0
2.3.0+2.3.0-beta
2.2.0+2.2.1-beta
2.1.0+2.2.0-beta
2.0.1+2.1.1-beta
1.0.0+2.0.0-beta
0.2.0+1.0.0
No labels
enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No projects
backupbot revolution
Done
Assignees
3wordchant
aadil (Aadil Ayub)
abra-bot (Abra Bot)
ammaratef45
amras (Sarma)
Apfelwurm
appletalk
arjan
basebuilder
BornDeleuze
Brooke
carla
cas (Cassowary)
codegod100
coopcloud
cyrnel
decentral1se (d1)
dede
devydave
fauno (fauno)
flancian
Frando
iexos
jade (Jade Ambrose)
javielico (Javielico)
jjsfunhouse
jmakdah2 (Jackie Makdah)
joe-irving (Joe Irving)
kawaiipunk (KawaiiPunk)
knoflook
kolaente
lambdabundesverband
linnealovespie (April)
marlon (marlon)
mayel
mirsal
moosemower
moritz
nicksellen (Nick Sellen)
notplants
oxaliq (sorrel)
p4u1
pau
pharaohgraphy (Andrew 🐦🔥❤️🔥✴️)
PhiNatalie
renovate-bot (Comrade Renovate Bot)
ripclap
rix
rscmbbng
sef (sef)
simon
sixsmith (Sixsmith)
stevensting
tobias
trav (Trav Fryer)
val (val (he/him))
vaznasty
virtualboys
wolcen (Chris Thompson)
wykwit
xynosis
yksflip
Clear assignees
No Assignees
moritz
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: coop-cloud/backup-bot-two#24
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
There should be the option to get notified for failed backups. I would simply recommend the same technique like kadabra with ssmpt and cronjobs:
https://docs.coopcloud.tech/operators/tutorial/#automatic-upgrades
We use healthchecks.io at autonomic, and I'm also using it personally, for backup notifications.
It supports receiving pings by HTTP and email, so if we want to stick with email then all we'd need for healthchecks.io is to be able to configure different addresses for successful and failed backups.
For bonus points, it would be nice to have:
I really like the idea of healthchecks.io and I wonder if there is something similar that could be self hosted?
I think this should be an additional way of notification besides email.
Haven't found anything yet 😞
Sounds fine. "different addresses for successful and failed backups" is my only recommended addition to your proposal, but also fine to start with just one address.
We decided to use the json log output in grafana to create alerts if the last backup is either too long ago or an error message was produces. We'll publish the grafana setup soon.
But anyway I like the idea to additionally have the option for webhooks and email alerts.
Maybe we could also publish backup metrics to prometheus, then we could set up alerts this way
That's the way we try to do it currently.
https://git.coopcloud.tech/coop-cloud/monitoring-ng
Unfortunately it's not possible to create Alerts in Grafana for non existing logs. Therefore we now use the Push API of Uptime Kuma to get alerts of failed or missing backups. I created a commit to add different kinds of Push URLs.
We for don't need email notifications any more, as push notification are much more reliable. Uptime Kuma will not only notify on a failed backup but also if there was no successful backup for more than 24 hours.
I might close this issue if there is no one who would like to have email notifications.
folks can always set up email notifications using Healthchecks.io or a custom automation (using e.g. n8n) if they like. Closing!