coop-cloud/monitoring-ng
3
0
Fork 4
Code Issues 5 Pull Requests 5 Packages Projects Releases Wiki Activity
121 Commits 31 Branches 16 Tags
smartctl-dev
Commit Graph
30 Commits
Author SHA1 Message Date
fauno d0d600ca4e Merge branch 'main' into loki-debugging 2026-10-30 10:52:10 -03:00
ammaratef45 8c7afe5d8f increase grafana healthcheck start_period and reduce error logs 2026-10-26 17:40:03 +00:00
p4u1 01111b992d fix(alert): invert memory usage alert 2026-10-12 13:02:52 +02:00
p4u1 f8973c11fc feat: improve loki debugging 2026-10-10 14:44:30 +02:00
simon e5f57c97f1 Split image tags, relabel journald logs (#30) 
Reviewed-on: #30
Reviewed-by: ammaratef45 <ammaratef45@proton.me>
Co-authored-by: Simon <s.thiessen@local-it.org>
 2026-09-17 13:38:45 +00:00
simon f48aa64193 make disk usage per container metric optional (#29) 
Following the discussion from the matrix channel, I made the disk usage metrics optional, as it currently causes ~35% of cpu usage on our system.

[Old cadvisor-config](02b01e5c23/compose.yml) from this repo had a `- "--housekeeping_interval=120s"` option enabled which seems currently not configurable for alloy - maybe this allows us reenabling disk metrics at a later point again.

dropped from 35% to 5% when removing `disk` from
`  enabled_metrics = ["cpu", "cpuLoad", "diskIO", "memory", "network"]` for cadvisor.

go profiling:

```
File: alloy
Build ID: de7ba8978f6753c25c13f6886ab3b896d3b05d3f
Type: cpu
Time: Sep 7, 2026 at 12:53pm (CEST)
Duration: 120s, Total samples = 37.23s (31.02%)
Showing nodes accounting for 32.20s, 86.49% of 37.23s total
Dropped 790 nodes (cum <= 0.19s)
      flat  flat%   sum%        cum   cum%
    28.72s 77.14% 77.14%     28.72s 77.14%  internal/runtime/syscall/linux.Syscall6
     0.47s  1.26% 78.40%      0.48s  1.29%  internal/filepathlite.(*lazybuf).append (inline)
     0.30s  0.81% 79.21%      0.94s  2.52%  internal/filepathlite.Clean
     0.26s   0.7% 79.91%      0.26s   0.7%  runtime.nextFreeFast (inline)
     0.23s  0.62% 80.53%      0.23s  0.62%  runtime.futex
     0.21s  0.56% 81.09%      0.21s  0.56%  runtime.memclrNoHeapPointers
     0.21s  0.56% 81.65%      0.21s  0.56%  runtime.memmove
     0.19s  0.51% 82.16%      6.19s 16.63%  os.(*File).readdir
     0.17s  0.46% 82.62%     31.24s 83.91%  path/filepath.walk
     0.12s  0.32% 82.94%      0.20s  0.54%  runtime.exitsyscall
     0.08s  0.21% 83.16%      0.24s  0.64%  runtime.scanObject
     0.08s  0.21% 83.37%      0.24s  0.64%  runtime.sweepone
     0.08s  0.21% 83.59%      0.29s  0.78%  slices.pdqsortOrdered[go.shape.string]
     0.07s  0.19% 83.78%      0.27s  0.73%  runtime.makeslicecopy
     0.06s  0.16% 83.94%     28.48s 76.50%  syscall.RawSyscall6
     0.05s  0.13% 84.07%      0.27s  0.73%  github.com/google/cadvisor/fs.GetDirUsage.func1
     0.05s  0.13% 84.21%     19.99s 53.69%  os.lstatNolog
     0.05s  0.13% 84.34%      1.26s  3.38%  runtime.mallocgc
     0.05s  0.13% 84.47%      0.43s  1.15%  runtime.mallocgcSmallNoscan
     0.04s  0.11% 84.58%      0.46s  1.24%  runtime.newobject
     0.04s  0.11% 84.69%      0.19s  0.51%  runtime.selectgo
     0.04s  0.11% 84.80%      1.18s  3.17%  runtime.systemstack
```

Reviewed-on: #29
Reviewed-by: Danny Groenewegen <247+dannygroenewegen@noreply.git.coopcloud.tech>
Co-authored-by: Simon <s.thiessen@local-it.org>
 2026-09-08 14:25:47 +00:00
faunoanddannygroenewegen ff56438d7c BREAKING CHANGES: replace promtail and cadvisor for alloy (#21) 
closes #20

---------

Co-authored-by: Danny Groenewegen <danny@ecommons.space>
Reviewed-on: #21
Reviewed-by: ammaratef45 <ammaratef45@proton.me>
Reviewed-by: Danny Groenewegen <247+dannygroenewegen@noreply.git.coopcloud.tech>
Co-authored-by: f <f@sutty.nl>
 2026-09-02 22:57:39 +00:00
p4u1 02b01e5c23 feat: make alerts configurable (#19) 
Alert threshholds can now be configured via env variables

I also seperated the basic auth, so that it is possible to deploy only prometheus without promtail

Reviewed-on: #19
Reviewed-by: ammaratef45 <ammaratef45@proton.me>
Reviewed-by: Danny Groenewegen <dannygroenewegen@noreply.git.coopcloud.tech>
Co-authored-by: p4u1 <p4u1_f4u1@riseup.net>
Co-committed-by: p4u1 <p4u1_f4u1@riseup.net>
 2026-06-12 06:49:17 +00:00
moritz dd320e9f1c fix: Shorten all secret and config names to max 14 characters (#13) 
Reviewed-on: #13
Reviewed-by: p4u1 <p4u1@noreply.git.coopcloud.tech>
Reviewed-by: moritz <moritz@noreply.git.coopcloud.tech>
 2026-05-11 15:38:15 +00:00
ammaratef45 48d137d194 update loki config file 2026-04-09 04:36:03 +00:00
dannygroenewegen 682f30cef1 Add migrate_secret_names() to abra.sh to reinsert all secrets with shortened names in docker 2026-03-25 16:11:37 +01:00
dannygroenewegen 9dfa9cad2a Shortened all the secret and config names to max 14 characters to prevent running into Docker's 64 character limit when STACK_NAME is appended to it. 2026-03-25 15:58:28 +01:00
dannygroenewegen 99f8790ec4 fix: Update scape-config example to use HTTPS for Traefik metrics (#17) 
This fixes the insecure Traefik metrics endpoint. See coop-cloud/traefik#94 for details.

Reviewed-on: #17
Co-authored-by: Danny Groenewegen <mail@dannygroenewegen.nl>
Co-committed-by: Danny Groenewegen <mail@dannygroenewegen.nl>
 2026-03-24 09:37:05 +00:00
p4u1 310c28e735 refactor: provision alerts instead of putting them in the /var/lib folder (#16) 
Note that I did not copy the backupbot alert since this one gets a rework soon

Reviewed-on: #16
Co-authored-by: p4u1 <p4u1_f4u1@riseup.net>
Co-committed-by: p4u1 <p4u1_f4u1@riseup.net>
 2026-03-20 14:10:10 +00:00
p4u1 8900ace6a2 feat: Adds matrix contact point for grafana alerts 2025-02-18 16:03:47 +01:00
p4u1 0352a393de feat: Adds dashboard and alerts for backupbot 2024-12-30 14:43:30 +01:00
moritz c06dda7205 chore: publish 1.0.0+v1.7.0 release 2023-12-21 02:24:47 +01:00
moritz b497b4ce4e shorten config names 2023-12-21 01:43:44 +01:00
philippr 961a17f1ad chore: publish 0.3.0+v1.6.0 release 2023-07-11 14:59:37 +02:00
philippr 9a791ef794 feat: add hostname label to promtail logs 2023-06-26 11:43:26 +02:00
philippr 705934abc1 fix: increase loki limits 2023-06-20 16:03:57 +02:00
philippr de1819521b remove prometheus alertmanager 
grafana has basic alert mechanisms included, might be enough for most
usecases?
 2023-06-01 15:10:53 +02:00
philippr 8c82943289 fix: config version 2023-05-24 12:28:35 +02:00
philippr a8e94af0cf chore: publish 120 release 2023-05-23 17:08:21 +02:00
philippr 6e02886e51 fix scrape config 2023-05-23 12:22:22 +02:00
philippr d31d56ed03 add command to create prometheus scrape config 2023-05-23 12:19:16 +02:00
philippr cc8a0b2905 wip loki stuff 2023-02-13 16:10:33 +01:00
philippr 15cd881356 wip loki stuff 2023-02-12 19:06:30 +01:00
philippr 7fa22a1350 add grafana 2023-02-11 17:17:50 +01:00
philippr d5a34436f9 wip 2023-02-09 10:07:29 +01:00