Email Alert for failed backups #24

New Issue

moritz · 2023-09-07T21:00:58Z

moritz commented

2023-09-07 21:00:58 +00:00

There should be the option to get notified for failed backups. I would simply recommend the same technique like kadabra with ssmpt and cronjobs:
https://docs.coopcloud.tech/operators/tutorial/#automatic-upgrades

There should be the option to get notified for failed backups. I would simply recommend the same technique like kadabra with ssmpt and cronjobs: https://docs.coopcloud.tech/operators/tutorial/#automatic-upgrades

👍 1

moritz self-assigned this 2023-09-07 21:00:59 +00:00

moritz added this to the backupbot revolution project 2023-09-07 21:01:06 +00:00

3wordchant commented

2023-11-12 00:56:50 +00:00

We use healthchecks.io at autonomic, and I'm also using it personally, for backup notifications.

It supports receiving pings by HTTP and email, so if we want to stick with email then all we'd need for healthchecks.io is to be able to configure different addresses for successful and failed backups.

For bonus points, it would be nice to have:

HTTP / webhooks as well as email notifications
Sending a notification when a backup starts (healthchecks.io can use this to report on how long a task takes)

We use healthchecks.io at autonomic, and I'm also using it personally, for backup notifications. It supports receiving pings by HTTP and email, so if we want to stick with email then all we'd need for healthchecks.io is to be able to configure different addresses for successful and failed backups. For bonus points, it would be nice to have: * HTTP / webhooks as well as email notifications * Sending a notification when a backup starts (healthchecks.io can use this to report on how long a task takes)

moritz commented

2023-11-14 18:17:38 +00:00

I really like the idea of healthchecks.io and I wonder if there is something similar that could be self hosted?
I think this should be an additional way of notification besides email.

I really like the idea of healthchecks.io and I wonder if there is something similar that could be self hosted? I think this should be an additional way of notification besides email.

3wordchant commented

2023-12-02 15:34:46 +00:00

I really like the idea of healthchecks.io and I wonder if there is something similar that could be self hosted?

Haven't found anything yet 😞

I think this should be an additional way of notification besides email.

Sounds fine. "different addresses for successful and failed backups" is my only recommended addition to your proposal, but also fine to start with just one address.

> I really like the idea of healthchecks.io and I wonder if there is something similar that could be self hosted? Haven't found anything yet 😞 > I think this should be an additional way of notification besides email. Sounds fine. "different addresses for successful and failed backups" is my only recommended addition to your proposal, but also fine to start with just one address.

moritz commented

2023-12-07 15:59:51 +00:00

We decided to use the json log output in grafana to create alerts if the last backup is either too long ago or an error message was produces. We'll publish the grafana setup soon.
But anyway I like the idea to additionally have the option for webhooks and email alerts.

We decided to use the json log output in grafana to create alerts if the last backup is either too long ago or an error message was produces. We'll publish the grafana setup soon. But anyway I like the idea to additionally have the option for webhooks and email alerts.

p4u1 commented

2023-12-07 17:43:14 +00:00

Maybe we could also publish backup metrics to prometheus, then we could set up alerts this way

moritz commented

2023-12-12 11:47:28 +00:00

That's the way we try to do it currently.
https://git.coopcloud.tech/coop-cloud/monitoring-ng

That's the way we try to do it currently. https://git.coopcloud.tech/coop-cloud/monitoring-ng

decentral1se added the

enhancement

label 2024-01-11 13:34:31 +00:00

moritz referenced this issue from a commit

2024-01-16 19:44:23 +00:00

Push Notifications #24

moritz commented

2024-01-16 19:45:40 +00:00

Maybe we could also publish backup metrics to prometheus, then we could set up alerts this way

Unfortunately it's not possible to create Alerts in Grafana for non existing logs. Therefore we now use the Push API of Uptime Kuma to get alerts of failed or missing backups. I created a commit to add different kinds of Push URLs.

> Maybe we could also publish backup metrics to prometheus, then we could set up alerts this way Unfortunately it's not possible to create Alerts in Grafana for non existing logs. Therefore we now use the Push API of Uptime Kuma to get alerts of failed or missing backups. I created a commit to add different kinds of Push URLs.

moritz commented

2024-01-17 10:34:17 +00:00

We for don't need email notifications any more, as push notification are much more reliable. Uptime Kuma will not only notify on a failed backup but also if there was no successful backup for more than 24 hours.
I might close this issue if there is no one who would like to have email notifications.

We for don't need email notifications any more, as push notification are much more reliable. Uptime Kuma will not only notify on a failed backup but also if there was no successful backup for more than 24 hours. I might close this issue if there is no one who would like to have email notifications.

❤️ 1

3wordchant commented

2024-02-10 21:18:11 +00:00

I might close this issue if there is no one who would like to have email notifications.

folks can always set up email notifications using Healthchecks.io or a custom automation (using e.g. n8n) if they like. Closing!

> I might close this issue if there is no one who would like to have email notifications. folks can always set up email notifications using Healthchecks.io or a custom automation (using e.g. n8n) if they like. Closing!

3wordchant closed this issue

2024-02-10 21:18:11 +00:00

Sign in to join this conversation.