feat: implement basic rate limiting by ip address

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [traefik](https://github.com/containous/traefik) | patch | `v3.7.12` -> `v3.7.13` |

> ❗ **Important**
>
> Release Notes retrieval for this PR were skipped because no github.com credentials were available.
> If you are self-hosted, please see [this instruction](https://github.com/renovatebot/renovate/blob/master/docs/usage/examples/self-hosting.md#githubcom-token-for-release-notes).

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MS4xNzMuMSIsInVwZGF0ZWRJblZlciI6IjQxLjE3My4xIiwidGFyZ2V0QnJhbmNoIjoibWFzdGVyIiwibGFiZWxzIjpbXX0=-->

Reviewed-on: #141
Co-authored-by: Renovate Bot <renovate@coopcloud.tech>

.env.sample

@@ -139,10 +139,14 @@ WRITE_TIMEOUT=0s
 #####################################################################
 
 ## Enable prometheus metrics collection
-## used used by the coop-cloud monitoring stack
-## BASIC_AUTH should also be enabled
+## Metrics are served unauthenticated on :8082, reachable only from
+## other services on the proxy network (e.g. monitoring-ng's Alloy,
+## which auto-discovers it via the prometheus.io/scrape label)
 #COMPOSE_FILE="$COMPOSE_FILE:compose.metrics.yml"
 #METRICS_ENABLED=1
+
+## Setting METRICS_FQDN also adds a public metrics endpoint (behind
+## basic auth). BASIC_AUTH should be enabled for this.
 #METRICS_FQDN=metrics.traefik.example.com
 
 #####################################################################
@@ -159,6 +163,10 @@ WRITE_TIMEOUT=0s
 #COMPOSE_FILE="$COMPOSE_FILE:compose.smtp.yml"
 #SMTP_ENABLED=1
 
+## PGSQL recipe open port 5432
+#COMPOSE_FILE="$COMPOSE_FILE:compose.pgsql.yml"
+#PGSQL_ENABLED=1
+
 ## Compy
 #COMPOSE_FILE="$COMPOSE_FILE:compose.compy.yml"
 #COMPY_ENABLED=1
@@ -207,6 +215,8 @@ WRITE_TIMEOUT=0s
 ## Garage
 #COMPOSE_FILE="$COMPOSE_FILE:compose.garage.yml"
 #GARAGE_RPC_ENABLED=1
+#COMPOSE_FILE="$COMPOSE_FILE:compose.garage-admin.yml"
+#GARAGE_ADMIN_ENABLED=1
 
 ## Nextcloud Talk HPB
 #COMPOSE_FILE="$COMPOSE_FILE:compose.nextcloud-talk-hpb.yml"
@@ -223,5 +233,28 @@ WRITE_TIMEOUT=0s
 #ANUBIS_SERVE_ROBOTS_TXT=true
 #ANUBIS_SLOG_LEVEL=INFO
 
+## Anubis metrics
+#COMPOSE_FILE="$COMPOSE_FILE:compose.anubis-metrics.yml"
+
 ## Enable onion service support
 #ONION_ENABLED=1
+
+## Access logs
+#COMPOSE_FILE="$COMPOSE_FILE:compose.access-log.yml"
+
+## Behind a reverse proxy
+#
+# YAML array of subnets from which Traefik's trusts the x-real-ip
+# header when behind a reverse proxy.
+#
+# https://doc.traefik.io/traefik/reference/install-configuration/entrypoints/#opt-forwardedHeaders-trustedIPs
+#TRUSTED_IPS="['10.13.12.1']" # 10.13.12.1 is an example
+
+## Rate limits
+# You can enable universal rate limits by setting RATE_LIMIT_EVERYTHING
+# to true, otherwise each app needs to export their own middleware labels.
+#
+# https://doc.traefik.io/traefik/reference/routing-configuration/http/middlewares/ratelimit/
+#RATE_LIMIT_EVERYTHING=false
+#RATE_LIMIT_AVERAGE=100
+#RATE_LIMIT_BURST=200

MAINTENANCE.md

@@ -29,3 +29,4 @@ Everyone can apply to be a recipe maintainer:
 1. Watch the repository to always get updates
 2. Simply add your self to the list in the [README.md](./README.md) and open a new pull request with the change.
 3. Once the pull request gets merged you will be added to the [traefik maintainers team](https://git.coopcloud.tech/org/coop-cloud/teams/traefik-maintainers).
+4. Join the room [#cc-|-traefik-maintenance:matrix.org](#cc-|-traefik-maintenance:matrix.org) and chat to other maintainers.

README.md

@@ -5,7 +5,7 @@
 > https://docs.traefik.io
 
 <!-- metadata -->
-* **Maintainer**: [@p4u1](https://git.coopcloud.tech/p4u1), [@decentral1se](https://git.coopcloud.tech/decentral1se), [@javielico](https://git.coopcloud.tech/javielico), Local-IT: [@moritz](https://git.coopcloud.tech/moritz), [@msimon](https://git.coopcloud.tech/simon), [@carla](https://git.coopcloud.tech/carla)
+* **Maintainer**: [@p4u1](https://git.coopcloud.tech/p4u1), [@javielico](https://git.coopcloud.tech/javielico), Local-IT: [@moritz](https://git.coopcloud.tech/moritz), [@msimon](https://git.coopcloud.tech/simon), [@carla](https://git.coopcloud.tech/carla)
 * **Status**: `stable`
 * **Category**: Utilities
 * **Features**: ?
@@ -72,8 +72,27 @@ After deploying these changes, go to each recipe that supports Anubis
 and follow the process there.  **Enabling Anubis here is not enough for
 protection your apps.**
 
+If you want to collect Prometheus metrics for Anubis, for instance with
+[monitoring-ng](/monitoring-ng), uncomment the "Anubis metrics" section.
+
 ## Enabling onion service
 
 Uncomment the line in the config setting `ONION_ENABLED=1`. This will create a new entrypoint on port 9052 which can be used to bypass forced SSL. For more details, see the [onion recipe](https://recipes.coopcloud.tech/onion).
 
 [`abra`]: https://git.autonomic.zone/autonomic-cooperative/abra
+
+## Access logs
+
+To keep access logs on a volume, uncomment the "Access logs" section.
+
+You'll need to setup rotation yourself and send the USR1 signal to
+Traefik for reloading.  For instance, with `logrotate:
+
+```
+# /etc/logrotate.d/traefik.conf
+/var/lib/docker/volumes/traefik_SERVICE_NAME_access-logs/_data/access_log.jsonl {
+  postrotate
+    pkill -USR1 traefik
+  endscript
+}
+```

abra.sh

@@ -1,3 +1,4 @@
-export TRAEFIK_YML_VERSION=v31a
-export FILE_PROVIDER_YML_VERSION=v12
+export TRAEFIK_YML_VERSION=v37
+export FILE_PROVIDER_YML_VERSION=v14
 export ENTRYPOINT_VERSION=v5
+export ANUBIS_YML_VERSION=v1

anubis.yml.tmpl

@@ -0,0 +1,7 @@
+bots:
+- import: (data)/meta/default-config.yaml
+{{ if eq (env "ANUBIS_METRICS_ENABLED") "true" }}
+metrics:
+  bind: ":9090"
+  network: "tcp"
+{{ end }}

compose.access-log.yml

@@ -0,0 +1,10 @@
+---
+version: "3.8"
+services:
+  app:
+    environment:
+      ACCESS_LOG_ENABLED: "true"
+    volumes:
+      - "access-logs:/var/log/"
+volumes:
+  access-logs:

compose.anubis-metrics.yml

@@ -0,0 +1,11 @@
+---
+version: "3.8"
+services:
+  anubis:
+    environment:
+      ANUBIS_METRICS_ENABLED: "true"
+    deploy:
+      labels:
+      - "prometheus.io/scrape=true"
+      - "prometheus.io/port=9090"
+      - "prometheus.io/path=/metrics"

compose.anubis.yml

@@ -5,8 +5,9 @@ services:
     deploy:
       labels:
       - "traefik.http.middlewares.anubis.forwardauth.address=http://anubis:8080/.within.website/x/cmd/anubis/api/check"
+      - "traefik.http.middlewares.anubis.forwardauth.trustForwardHeader=true"
   anubis:
-    image: "ghcr.io/techarohq/anubis:v1.25.0"
+    image: "ghcr.io/techarohq/anubis:v1.27.0"
     environment:
       BIND: ":8080"
       TARGET: " "
@@ -18,6 +19,10 @@ services:
       OG_CACHE_CONSIDER_HOST: "${ANUBIS_OG_CACHE_CONSIDER_HOST}"
       SERVE_ROBOTS_TXT: "${ANUBIS_SERVE_ROBOTS_TXT}"
       SLOG_LEVEL: "${ANUBIS_SLOG_LEVEL:-INFO}"
+      POLICY_FNAME: "/data/cfg/botPolicy.yaml"
+    configs:
+    - source: anubis_yml
+      target: /data/cfg/botPolicy.yaml
     networks:
     - proxy
     deploy:
@@ -28,3 +33,8 @@ services:
       - "traefik.http.routers.anubis.entrypoints=web-secure"
       - "traefik.http.services.anubis.loadbalancer.server.port=8080"
       - "traefik.http.routers.anubis.service=anubis"
+configs:
+  anubis_yml:
+    name: ${STACK_NAME}_anubis_yml_${ANUBIS_YML_VERSION}
+    file: anubis.yml.tmpl
+    template_driver: golang

compose.garage-admin.yml

@@ -0,0 +1,10 @@
+version: "3.8"
+services:
+  app:
+    environment:
+      - GARAGE_ADMIN_ENABLED
+    ports:
+      - target: 3903
+        published: 3903
+        protocol: tcp
+        mode: host

compose.metrics.yml

@@ -3,3 +3,9 @@ services:
   app:
     environment:
       - METRICS_ENABLED
+    deploy:
+      labels:
+        # lets monitoring-ng's Alloy auto-discover and scrape metrics-internal
+        # via the proxy network.
+        - "prometheus.io/scrape=true"
+        - "prometheus.io/port=8082"

compose.pgsql.yml

@@ -0,0 +1,12 @@
+---
+version: "3.8"
+
+services:
+  app:
+    environment:
+      - PGSQL_ENABLED
+    ports:
+      - target: 5432
+        published: 5432
+        protocol: tcp
+        mode: host

compose.yml

@@ -3,7 +3,7 @@ version: "3.8"
 
 services:
   app:
-    image: "traefik:v3.7.5"
+    image: "traefik:v3.7.13"
     # Note(decentral1se): *please do not* add any additional ports here.
     # Doing so could break new installs with port conflicts. Please use
     # the usual `compose.$app.yml` approach for any additional ports
@@ -57,12 +57,12 @@ services:
         - "traefik.http.routers.${STACK_NAME}.tls.certresolver=${LETS_ENCRYPT_ENV}"
         - "traefik.http.routers.${STACK_NAME}.service=api@internal"
         - "traefik.http.routers.${STACK_NAME}.middlewares=security@file"
-        - "coop-cloud.${STACK_NAME}.version=5.1.1+v3.6.15"
+        - "coop-cloud.${STACK_NAME}.version=6.2.0+v3.7.12"
         - "coop-cloud.${STACK_NAME}.timeout=${TIMEOUT}"
         - "backupbot.backup=${ENABLE_BACKUPS:-true}"
 
   socket-proxy:
-    image: lscr.io/linuxserver/socket-proxy:3.4.0
+    image: lscr.io/linuxserver/socket-proxy:3.4.4
     deploy:
       endpoint_mode: dnsrr
     environment:

file-provider.yml.tmpl

@@ -22,6 +22,11 @@ http:
       basicAuth:
         usersFile: "/run/secrets/usersfile"
     {{ end }}
+    ip-rate-limit:
+      rateLimit:
+        average: {{ or (env "RATE_LIMIT_AVERAGE") "100" }}
+        burst: {{ or (env "RATE_LIMIT_BURST") "200" }}
+        period: "1s"
     security:
       headers:
         frameDeny: true
@@ -32,6 +37,7 @@ http:
         stsSeconds: "31536000"
   {{ if eq (env "METRICS_ENABLED") "1" }}
   routers:
+    {{ if ne (env "METRICS_FQDN") "" }}
     traefik-metrics:
       rule: "Host(`{{ env "METRICS_FQDN" }}`)"
       entrypoints:
@@ -41,6 +47,14 @@ http:
       middlewares:
         - basicauth@file
       service: prometheus@internal
+    {{ end }}
+    # reachable from other services on the proxy network only (this port
+    # isn't published to the host), without auth
+    traefik-metrics-internal:
+      rule: "PathPrefix(`/`)"
+      entrypoints:
+        - metrics-internal
+      service: prometheus@internal
   {{ end }}
 
 tls:
@@ -61,4 +75,4 @@ tls:
   certificates:
     - certFile: /run/secrets/ssl_cert
       keyFile: /run/secrets/ssl_key
-  {{ end }}
+  {{ end }}

release/6.0.0+v3.7.7

@@ -0,0 +1,13 @@
+!Breaking: Starting with v3.6.16, the Docker provider requires Docker API version v1.40 or above (Docker Engine v19.03). Users running older (end of life) versions of Docker Engine should update their Docker Engine or use the DOCKER_API_VERSION environment variable to override the API version used by Traefik.
+
+letsencrypt: Avoid HTTP-01 challenge if `LETS_ENCRYPT_DNS_CHALLENGE_ENABLED` is set, in order to rely on DNS-01 challenges for servers not exposed to the internet.
+
+matrix-federation: Entrypoint was changed to :8448 to match published port
+
+fix: ensure large uploads work. You can now set the following env vars:
+- READ_TIMEOUT
+- WRITE_TIMEOUT
+
+cloudflare: Add Cloudflare as DNS provider
+
+For more information take a look at the migration guide: https://doc.traefik.io/traefik/v3.7/migrate/v3/#v377

release/6.1.0+v3.7.10

@@ -0,0 +1 @@
+Adds option ability to open ports for PGSQL recipe, new version of anubis, and ability to add anubis metrics on this release.

release/6.2.0+v3.7.12

@@ -0,0 +1 @@
+1. compose.metrics.yml now adds prometheus.io/scrape labels so services like monitoring-ng can automatically discover and scrape Traefik's metrics.

release/next

@@ -1 +0,0 @@
-letsencrypt: Avoid HTTP-01 challenge if `LETS_ENCRYPT_DNS_CHALLENGE_ENABLED` is set, in order to rely on DNS-01 challenges for servers not exposed to the internet.

traefik.yml.tmpl

@@ -6,6 +6,21 @@ log:
   level: {{ env "LOG_LEVEL" }}
   maxAge: {{ env "LOG_MAX_AGE" }}
 
+{{ if eq (env "ACCESS_LOG_ENABLED") "true" }}
+accessLog:
+  format: "json"
+  filePath: "/var/log/access_log.jsonl"
+  fields:
+    defaultMode: "keep"
+    headers:
+      defaultMode: "keep"
+      names:
+        Authorization: "drop"
+        Cookie: "drop"
+    queryParameters:
+      defaultMode: "keep"
+{{ end }}
+
 providers:
   swarm:
     endpoint: "tcp://socket-proxy:2375"
@@ -33,6 +48,14 @@ entrypoints:
           to: web-secure
   web-secure:
     address: ":443"
+{{ if ne (env "TRUSTED_IPS") "" }}
+    forwardedHeaders:
+      trustedIPs: {{ env "TRUSTED_IPS" }}
+{{ end }}
+{{ if eq (env "RATE_LIMIT_EVERYTHING") "true" }}
+    middlewares:
+      - "ip-rate-limit@file"
+{{ end }}
     transport:
       respondingTimeouts:
         readTimeout: {{ env "READ_TIMEOUT" }}
@@ -60,6 +83,10 @@ entrypoints:
   garage-rpc:
     address: ":3901"
   {{- end }}
+  {{- if eq (env "GARAGE_ADMIN_ENABLED") "1" }}
+  garage-admin:
+    address: ":3903"
+  {{- end }}
   {{- if eq (env "FOODSOFT_SMTP_ENABLED") "1" }}
   foodsoft-smtp:
     address: ":2525"
@@ -100,7 +127,7 @@ entrypoints:
   {{- end }}
   {{- if eq (env "MATRIX_FEDERATION_ENABLED") "1" }}
   matrix-federation:
-    address: ":9001"
+    address: ":8448"
   {{- end }}
   {{- if eq (env "NEXTCLOUD_TALK_HPB_ENABLED") "1" }}
   nextcloud-talk-hpb:
@@ -112,6 +139,10 @@ entrypoints:
   onion:
     address: ":9052"
   {{- end }}
+  {{- if eq (env "METRICS_ENABLED") "1" }}
+  metrics-internal:
+    address: ":8082"
+  {{- end }}
 
 ping:
   entryPoint: web