improve documentation of federation

Adds a second Traefik router on the web (nginx) service that listens on
the `matrix-federation` entrypoint (host port 8448 via the traefik
recipe's compose.matrix.yml) and uses Host(${SERVER_NAME}).

Lets remote homeservers reach this server's federation/key endpoints via
the standard `<server_name>:8448` fallback when no .well-known or SRV
delegation exists for SERVER_NAME — useful when SERVER_NAME differs from
DOMAIN and no external service serves /.well-known/matrix/server.

Requires MATRIX_FEDERATION_ENABLED=1 on the traefik stack.

.env.sample

@@ -26,6 +26,10 @@ SECRET_REGISTRATION_VERSION=v1
 # Set "true" to enable federation endpoint on $DOMAIN/.well-known/matrix/server
 SERVE_SERVER_WELLKNOWN=false
 
+# Serve /.well-known/matrix/{server,client} on SERVER_NAME via Traefik.
+# Can be used when SERVER_NAME != DOMAIN and SERVER_NAME is served by Traefik.
+#COMPOSE_FILE="$COMPOSE_FILE:compose.wellknown.yml"
+
 ALLOW_PUBLIC_ROOMS_FEDERATION=false
 
 ## Registration
@@ -39,9 +43,7 @@ PASSWORD_LOGIN_ENABLED=true
 ## Room auto-join
 
 #AUTO_JOIN_ROOM_ENABLED=1
-#AUTO_JOIN_ROOM is deprecated, but kept for backward compatibility. Please use only one, and prefer AUTO_JOIN_ROOM_LIST.
 #AUTO_JOIN_ROOM="#example:example.com"
-#AUTO_JOIN_ROOM_LIST="[\"#room1:example.com\",\"#room2:example.com\"]"
 
 ## Logging
 
@@ -88,8 +90,6 @@ RETENTION_MAX_LIFETIME=4w
 #MEDIA_RETENTION_LOCAL_LIFETIME=30d
 #MEDIA_RETENTION_REMOTE_LIFETIME=14d
 
-MAX_UPLOAD_SIZE=50M
-
 ## Old Signing Key
 #OLD_SIGNING_KEY_ID=a_OLDKEYID
 #OLD_SIGNING_KEY=base64string
@@ -203,6 +203,14 @@ MAX_UPLOAD_SIZE=50M
 #WEB_CLIENT_LOCATION=https://element-web.example.com
 
 
+## State compression (reduces database bloat from federation)
+## Runs synapse_auto_compressor daily, built from source on first start
+#COMPOSE_FILE="$COMPOSE_FILE:compose.compress-state.yml"
+# See https://github.com/matrix-org/rust-synapse-compress-state#running-options
+#STATE_COMPRESS_CHUNK_SIZE=500
+#STATE_COMPRESS_CHUNKS=100
+#STATE_COMPRESS_SCHEDULE=0 3 * * *
+
 ## Admin interface at /admin
 #COMPOSE_FILE="$COMPOSE_FILE:compose.admin.yml"
 #ADMIN_INTERFACE_ENABLED=1

README.md

@@ -39,11 +39,95 @@
 
 ### Enabling federation
 
-See [`#27`](https://git.coopcloud.tech/coop-cloud/matrix-synapse/pulls/27) for more. Depending on your setup, using `SERVE_SERVER_WELLKNOWN=true` might work to start federating. Make sure you don't leave `DISABLE_FEDERATION=1` set!
+Federation is on by default (`DISABLE_FEDERATION=0`). Remote homeservers need a way to discover the host:port that serves your `SERVER_NAME`. There are three supported approaches.
+
+#### Option 1: built-in well-known (`SERVER_NAME` = `DOMAIN`)
+
+Set `SERVE_SERVER_WELLKNOWN=true` and leave `SERVER_NAME` unset (defaults to `DOMAIN`). The recipe's nginx serves `/.well-known/matrix/server` and `/.well-known/matrix/client` on `DOMAIN`. 
+
+Suitable when users are e.g. `@alice:matrix.example.com`.
+
+#### Option 2: external well-known on `SERVER_NAME`
+
+Use when you want users to be e.g. `@alice:example.com` while Synapse runs at `matrix.example.com`. Set:
+
+```
+SERVER_NAME=example.com
+DOMAIN=matrix.example.com
+SERVE_SERVER_WELLKNOWN=false
+```
+
+The two paths that must be served on `SERVER_NAME` are:
+
+- `https://example.com/.well-known/matrix/server` → `{"m.server": "matrix.example.com:443"}`
+- `https://example.com/.well-known/matrix/client` → `{"m.homeserver": {"base_url": "https://matrix.example.com"}}`
+
+**Recommended — let this recipe serve them via Traefik** by enabling `compose.wellknown.yml`:
+
+```
+COMPOSE_FILE="$COMPOSE_FILE:compose.wellknown.yml"
+```
+
+This publishes a Traefik router `Host(${SERVER_NAME}) && PathPrefix(/.well-known/matrix)`
+pointing at the matrix nginx, which already serves both files. The path-scoped, high-priority
+rule coexists with any apex website that also serves `Host(${SERVER_NAME})` — that site keeps
+serving everything except `/.well-known/matrix`. `SERVER_NAME` must resolve to this Traefik so
+ACME can issue its certificate.
+
+**Alternative** — serve the two files yourself from whatever already hosts `example.com`.
+
+
+#### Option 3: Traefik `matrix-federation` entrypoint (port 8448)
+
+Use when `SERVER_NAME` ≠ `DOMAIN` but you have no separate web service at `SERVER_NAME`. Remote homeservers fall back to `SERVER_NAME:8448` when there's no delegation.
+
+Requirements:
+
+- [traefik](https://git.coopcloud.tech/coop-cloud/traefik) `>= 5.1.2+v3.6.15` with `MATRIX_FEDERATION_ENABLED=1` and `compose.matrix.yml` enabled.
+- `SERVER_NAME` set in your matrix-synapse env (used by the federation router's Host rule).
+
+With these in place, the recipe publishes a Traefik router on `Host(${SERVER_NAME})` via the `matrix-federation` entrypoint, reusing the existing matrix nginx → synapse path.
+
+#### Option 4: DNS SRV records (usually not viable here)
+
+Federation can also be delegated with a DNS `SRV` record on `SERVER_NAME` instead of well-known:
+
+```
+_matrix-fed._tcp.example.com.  3600 IN SRV 10 0 8448 matrix.example.com.   # modern
+_matrix._tcp.example.com.      3600 IN SRV 10 0 8448 matrix.example.com.   # deprecated, for older peers
+```
+
+The catch is TLS: on the SRV path a remote validates the certificate against **`SERVER_NAME`**, *not* the SRV target. This recipe's Traefik only issues a cert for **`DOMAIN`**, so:
+
+- **SRV → `DOMAIN`:443 fails** — the presented cert is for `DOMAIN`, but the peer requires one for `SERVER_NAME`.
+- **SRV → `SERVER_NAME`:443 collides** — Traefik routes TLS by SNI, and `Host(SERVER_NAME)` on `:443` is already owned by whatever apex site serves `SERVER_NAME`.
+- **SRV → `SERVER_NAME`:8448 works** — the Option 3 `matrix-federation` router holds a cert for `SERVER_NAME` — but that's just Option 3 made explicit (the `:8448` fallback already works with no SRV record).
+
+So I think SRV does little here. Probably prefer Option 2 (well-known).
+
+#### Verifying
+
+The canonical test:
+
+- https://federationtester.matrix.org/#YOUR_SERVER_NAME
+
+Or check the underlying paths directly. They should all return JSON:
+
+```bash
+# Options 1 & 2 — delegation
+curl https://SERVER_NAME/.well-known/matrix/server
+
+# Option 3 — federation endpoint via 8448
+curl https://SERVER_NAME:8448/_matrix/key/v2/server
+
+# Confirms Synapse itself is healthy (independent of the path remote servers use)
+curl https://DOMAIN/_matrix/key/v2/server
+```
 
 ### Getting client discovery on a custom domain
 
-You'll need to deploy something like [this](https://git.autonomic.zone/ruangrupa/well-known-uris). This could be implemented in this recipe but we haven't merged it in yet. Change sets are welcome.
+Enable `compose.wellknown.yml` (see Option 2 above) — it serves `/.well-known/matrix/client`
+on `SERVER_NAME` too, so clients signing in as `@alice:example.com` auto-discover the homeserver.
 
 ## Bridges
 For all Bridges:
@@ -52,6 +136,9 @@ For all Bridges:
 - include the registration in synapse, e.g. `APP_SERVICE_CONFIGS="[\"/telegram-data/registration.yaml\"]"`
 - and set yourself as admin, e.g.: `TELEGRAM_BRIDGE_PERMISSIONS="{ \"*\": \"relaybot\", \"@akadmin:example.com\": \"admin\"}"`
 
+> [!IMPORTANT]
+> The shared secret authenticator may break when matrix-synapse uses a newer python version with an error stating something like "module not found". You have to fix the path in the compose.shared_secret_auth.yml like [here](https://git.coopcloud.tech/coop-cloud/matrix-synapse/commit/3d1350533079ce1ad3bea92039fe003684589b95)
+
 ### Telegram bridging
 
 You need to get your bot setup on the telegram side first by creating a [telegram app](https://my.telegram.org/apps) and a [telegram bot](https://docs.mau.fi/bridges/python/telegram/relay-bot.html#setup) and have these values:

abra.sh

@@ -1,15 +1,188 @@
 export DISCORD_BRIDGE_YAML_VERSION=v2
 export ENTRYPOINT_CONF_VERSION=v3
-export HOMESERVER_YAML_VERSION=v34
+export HOMESERVER_YAML_VERSION=v35
 export LOG_CONFIG_VERSION=v2
 export SHARED_SECRET_AUTH_VERSION=v2
 export SIGNAL_BRIDGE_YAML_VERSION=v6
 export TELEGRAM_BRIDGE_YAML_VERSION=v6
-export NGINX_CONFIG_VERSION=v11
+export NGINX_CONFIG_VERSION=v12
 export WK_SERVER_VERSION=v1
 export WK_CLIENT_VERSION=v1
-export PG_BACKUP_VERSION=v1
+export PG_BACKUP_VERSION=v2
 export ADMIN_CONFIG_VERSION=v1
+export COMPRESS_STATE_ENTRYPOINT_VERSION=v5
+
+# See https://levans.fr/shrink-synapse-database.html
+db_size() {
+    echo "=== Database size ==="
+    psql -U synapse -d synapse -c "SELECT pg_size_pretty(pg_database_size('synapse')) AS db_size;"
+    echo ""
+    echo "=== Top 10 largest tables ==="
+    psql -U synapse -d synapse -c "
+        SELECT nspname || '.' || relname AS table,
+               pg_size_pretty(pg_total_relation_size(C.oid)) AS total_size
+        FROM pg_class C
+        LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
+        WHERE nspname NOT IN ('pg_catalog', 'information_schema')
+        ORDER BY pg_total_relation_size(C.oid) DESC
+        LIMIT 10;"
+}
+
+state_bloat() {
+    echo "=== Rooms with most state bloat ==="
+    psql -U synapse -d synapse -c "
+        SELECT room_id, count(*) AS state_entries
+        FROM state_groups_state
+        GROUP BY room_id
+        ORDER BY state_entries DESC
+        LIMIT 20;"
+}
+
+empty_rooms() {
+    echo "=== Rooms with no local members ==="
+    psql -U synapse -d synapse -c "
+        SELECT room_id, room_version
+        FROM rooms
+        WHERE room_id NOT IN (
+            SELECT room_id FROM local_current_membership WHERE membership = 'join'
+        );"
+}
+
+reindex() {
+    echo "WARNING: REINDEX locks tables. Synapse should be stopped before running this."
+    echo "Running REINDEX on synapse database..."
+    psql -U synapse -d synapse -c "REINDEX (VERBOSE) DATABASE synapse;"
+    echo "REINDEX complete."
+    psql -U synapse -d synapse -c "SELECT pg_size_pretty(pg_database_size('synapse')) AS db_size;"
+}
+
+# Run via: abra app cmd <domain> compress-state run_compressor
+run_compressor() {
+    CHUNK_SIZE="${1:-${STATE_COMPRESS_CHUNK_SIZE:-500}}"
+    CHUNKS="${2:-${STATE_COMPRESS_CHUNKS:-100}}"
+    DB_PASS=$(cat /run/secrets/db_password)
+    echo "Running synapse_auto_compressor (chunk_size=$CHUNK_SIZE, chunks=$CHUNKS)..."
+    /build/synapse_auto_compressor \
+        -p "postgresql://synapse:${DB_PASS}@db:5432/synapse" \
+        -c "$CHUNK_SIZE" -n "$CHUNKS"
+}
+
+vacuum_full() {
+    echo "WARNING: VACUUM FULL locks tables and requires temporary disk space."
+    echo "Synapse should be stopped before running this."
+    echo "Running VACUUM FULL on synapse database..."
+    psql -U synapse -d synapse -c "VACUUM FULL;"
+    echo "VACUUM FULL complete."
+    psql -U synapse -d synapse -c "SELECT pg_size_pretty(pg_database_size('synapse')) AS db_size;"
+}
+
+# Purge commands — run via: abra app cmd <domain> app <command>
+# These use the Synapse admin API and require an admin access token.
+
+register_admin() {
+    USER="${1}"
+    PASS="${2}"
+    if [ -z "$USER" ] || [ -z "$PASS" ]; then
+        echo "Usage: register_admin <username> <password>"
+        return 1
+    fi
+    register_new_matrix_user -u "$USER" -p "$PASS" -a -c /data/homeserver.yaml http://localhost:8008
+}
+
+get_token() {
+    USER="${1}"
+    PASS="${2}"
+    if [ -z "$USER" ] || [ -z "$PASS" ]; then
+        echo "Usage: get_token <username> <password>"
+        echo "Returns an admin access token for use with purge commands."
+        return 1
+    fi
+    curl -s -X POST "http://localhost:8008/_matrix/client/r0/login" \
+        -H "Content-Type: application/json" \
+        -d "{\"type\":\"m.login.password\",\"user\":\"$USER\",\"password\":\"$PASS\"}" \
+        | python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('access_token', d.get('error', 'unknown error')))"
+}
+
+purge_remote_media() {
+    DAYS="${1:-30}"
+    TOKEN="${2}"
+    if [ -z "$TOKEN" ]; then
+        echo "Usage: purge_remote_media <days> <admin_token>"
+        return 1
+    fi
+    BEFORE_TS=$(( $(date +%s) * 1000 - DAYS * 86400000 ))
+    echo "Purging remote media older than $DAYS days..."
+    curl -s -X POST "http://localhost:8008/_synapse/admin/v1/purge_media_cache?before_ts=$BEFORE_TS" \
+        -H "Authorization: Bearer $TOKEN"
+    echo ""
+}
+
+purge_room() {
+    ROOM_ID="${1}"
+    TOKEN="${2}"
+    if [ -z "$ROOM_ID" ] || [ -z "$TOKEN" ]; then
+        echo "Usage: purge_room <room_id> <admin_token>"
+        return 1
+    fi
+    echo "Purging room $ROOM_ID..."
+    curl -s -X DELETE "http://localhost:8008/_synapse/admin/v1/rooms/$ROOM_ID" \
+        -H "Authorization: Bearer $TOKEN" \
+        -H "Content-Type: application/json" \
+        -d '{"purge": true}'
+    echo ""
+}
+
+purge_history() {
+    ROOM_ID="${1}"
+    DAYS="${2:-90}"
+    TOKEN="${3}"
+    if [ -z "$ROOM_ID" ] || [ -z "$TOKEN" ]; then
+        echo "Usage: purge_history <room_id> <days> <admin_token>"
+        return 1
+    fi
+    BEFORE_TS=$(( $(date +%s) * 1000 - DAYS * 86400000 ))
+    echo "Purging history older than $DAYS days from $ROOM_ID..."
+    curl -s -X POST "http://localhost:8008/_synapse/admin/v1/purge_history/$ROOM_ID" \
+        -H "Authorization: Bearer $TOKEN" \
+        -H "Content-Type: application/json" \
+        -d "{\"purge_up_to_ts\": $BEFORE_TS}"
+    echo ""
+}
+
+purge_empty_rooms() {
+    TOKEN="${1}"
+    if [ -z "$TOKEN" ]; then
+        echo "Usage: purge_empty_rooms <admin_token>"
+        return 1
+    fi
+    echo "Fetching rooms with no local members..."
+    ROOMS=$(curl -s "http://localhost:8008/_synapse/admin/v1/rooms?limit=1000" \
+        -H "Authorization: Bearer $TOKEN" \
+        | python3 -c "
+import sys, json
+data = json.load(sys.stdin)
+for r in data.get('rooms', []):
+    if r.get('joined_local_members', 0) == 0:
+        print(r['room_id'])
+")
+    COUNT=$(echo "$ROOMS" | grep -c '.' || true)
+    echo "Found $COUNT empty rooms."
+    if [ "$COUNT" -eq 0 ]; then
+        echo "Nothing to purge."
+        return 0
+    fi
+    echo "$ROOMS"
+    echo ""
+    echo "Purging..."
+    for ROOM_ID in $ROOMS; do
+        echo "  Purging $ROOM_ID"
+        curl -s -X DELETE "http://localhost:8008/_synapse/admin/v1/rooms/$ROOM_ID" \
+            -H "Authorization: Bearer $TOKEN" \
+            -H "Content-Type: application/json" \
+            -d '{"purge": true}' > /dev/null
+    done
+    echo "Done."
+}
 
 set_admin () {
     admin=akadmin

compose.admin.yml

@@ -3,13 +3,13 @@ version: "3.8"
 
 services:
   admin:
-    image: awesometechnologies/synapse-admin:0.11.1
+    image: awesometechnologies/synapse-admin:0.11.4
     networks:
       - proxy
     deploy:
       labels:
         - "traefik.enable=true"
-        - "traefik.docker.network=proxy"
+        - "traefik.swarm.network=proxy"
         - "traefik.http.services.${STACK_NAME}_admin.loadbalancer.server.port=80"
         - "traefik.http.routers.${STACK_NAME}_admin.rule=Host(`${DOMAIN}`${EXTRA_DOMAINS})&&PathPrefix(`/admin`)"
         - "traefik.http.routers.${STACK_NAME}_admin.entrypoints=web-secure"

compose.compress-state.yml

@@ -0,0 +1,31 @@
+version: "3.8"
+
+services:
+  compress-state:
+    image: rust:1-alpine
+    entrypoint: /compress_state_entrypoint.sh
+    environment:
+      - STATE_COMPRESS_CHUNK_SIZE=${STATE_COMPRESS_CHUNK_SIZE:-500}
+      - STATE_COMPRESS_CHUNKS=${STATE_COMPRESS_CHUNKS:-100}
+      - STATE_COMPRESS_SCHEDULE=${STATE_COMPRESS_SCHEDULE:-0 3 * * *}
+    secrets:
+      - db_password
+    configs:
+      - source: compress_entrypoint
+        target: /compress_state_entrypoint.sh
+        mode: 0555
+    volumes:
+      - compress_state_build:/build
+    networks:
+      - internal
+    deploy:
+      restart_policy:
+        condition: on-failure
+
+volumes:
+  compress_state_build:
+
+configs:
+  compress_entrypoint:
+    name: ${STACK_NAME}_compress_ep_${COMPRESS_STATE_ENTRYPOINT_VERSION}
+    file: compress_state_entrypoint.sh

compose.shared_secret_auth.yml

@@ -9,7 +9,7 @@ services:
       - shared_secret_auth
     configs:
       - source: shared_secret_auth
-        target: /usr/local/lib/python3.12/site-packages/shared_secret_authenticator.py
+        target: /usr/local/lib/python3.13/site-packages/shared_secret_authenticator.py
 
 configs:
   shared_secret_auth:

compose.wellknown.yml

@@ -0,0 +1,24 @@
+---
+version: "3.8"
+
+# Serve /.well-known/matrix/{server,client} on SERVER_NAME via Traefik, routed to
+# the matrix nginx (`web`) — so server/client delegation works without hand-placing
+# files on whatever else hosts SERVER_NAME.
+#
+# Enable when SERVER_NAME != DOMAIN (users are @alice:example.com, Synapse runs at
+# matrix.example.com). The PathPrefix rule is more specific than a bare Host()
+# router, and the explicit high priority guarantees it wins over any apex website
+# that also serves Host(SERVER_NAME) — so the two coexist, the apex site keeps
+# serving everything except /.well-known/matrix.
+#
+# Requires SERVER_NAME to resolve to this Traefik so ACME can issue its cert.
+services:
+  web:
+    deploy:
+      labels:
+        - "traefik.http.routers.${STACK_NAME}-wellknown.rule=Host(`${SERVER_NAME}`) && PathPrefix(`/.well-known/matrix`)"
+        - "traefik.http.routers.${STACK_NAME}-wellknown.entrypoints=web-secure"
+        - "traefik.http.routers.${STACK_NAME}-wellknown.tls=true"
+        - "traefik.http.routers.${STACK_NAME}-wellknown.tls.certresolver=${LETS_ENCRYPT_ENV}"
+        - "traefik.http.routers.${STACK_NAME}-wellknown.service=${STACK_NAME}"
+        - "traefik.http.routers.${STACK_NAME}-wellknown.priority=1000"

compose.yml

@@ -3,7 +3,7 @@ version: "3.8"
 
 services:
   web:
-    image: nginx:1.29.2
+    image: nginx:1.29.6
     networks:
       - proxy
       - internal
@@ -22,21 +22,27 @@ services:
         target: /var/www/.well-known/matrix/client
     deploy:
       restart_policy:
-        condition: on-failure
+        condition: any
       labels:
         - "traefik.enable=true"
         - "traefik.http.services.${STACK_NAME}.loadbalancer.server.port=80"
         - "traefik.http.routers.${STACK_NAME}.rule=Host(`${DOMAIN}`)"
         - "traefik.http.routers.${STACK_NAME}.entrypoints=web-secure"
         - "traefik.http.routers.${STACK_NAME}.tls.certresolver=${LETS_ENCRYPT_ENV}"
+        - "traefik.http.routers.${STACK_NAME}-federation.rule=Host(`${SERVER_NAME}`)"
+        - "traefik.http.routers.${STACK_NAME}-federation.entrypoints=matrix-federation"
+        - "traefik.http.routers.${STACK_NAME}-federation.tls=true"
+        - "traefik.http.routers.${STACK_NAME}-federation.tls.certresolver=${LETS_ENCRYPT_ENV}"
+        - "traefik.http.routers.${STACK_NAME}-federation.service=${STACK_NAME}"
     healthcheck:
       test: curl -f http://${STACK_NAME}_app:8008/health || exit 1
-      interval: 20s
+      interval: 30s
       timeout: 15s
-      retries: 20
+      retries: 90
+      start_period: 2m
 
   app:
-    image: "matrixdotorg/synapse:v1.139.2"
+    image: "matrixdotorg/synapse:v1.149.1"
     volumes:
       - "data:/data"
     secrets:
@@ -48,7 +54,6 @@ services:
       - ALLOWED_LIFETIME_MAX
       - ALLOW_PUBLIC_ROOMS_FEDERATION
       - AUTO_JOIN_ROOM
-      - AUTO_JOIN_ROOM_LIST
       - AUTO_JOIN_ROOM_ENABLED
       - DISABLE_FEDERATION
       - DOMAIN
@@ -106,34 +111,33 @@ services:
       restart_policy:
         condition: on-failure
       labels:
-        - "coop-cloud.${STACK_NAME}.version=6.8.0+v1.139.2"
+        - "coop-cloud.${STACK_NAME}.version=7.0.2+v1.149.1"
         - "coop-cloud.${STACK_NAME}.timeout=${TIMEOUT}"
     healthcheck:
       test: ["CMD", "curl", "-f", "http://localhost:8008/health"]
       interval: 30s
       timeout: 10s
-      retries: 10
+      retries: 30
       start_period: 1m
 
   db:
-    image: postgres:13-alpine
+    image: pgautoupgrade/pgautoupgrade:17-alpine
     secrets:
       - db_password
     environment:
       - LC_COLLATE=C
       - LC_CTYPE=C
       - POSTGRES_DB=synapse
-      - POSTGRES_INITDB_ARGS="-E \"UTF8\""
+      - POSTGRES_INITDB_ARGS=-E UTF8
       - POSTGRES_PASSWORD_FILE=/run/secrets/db_password
       - POSTGRES_USER=synapse
       - DOMAIN
     networks:
       - internal
     healthcheck:
-      test: ["CMD", "pg_isready", "-U", "synapse"]
       interval: 30s
       timeout: 10s
-      retries: 10
+      retries: 20
       start_period: 1m
     volumes:
       - postgres:/var/lib/postgresql/data
@@ -142,11 +146,11 @@ services:
         backupbot.backup: "${ENABLE_BACKUPS:-true}"
         backupbot.backup.pre-hook: "/pg_backup.sh backup"
         backupbot.backup.volumes.postgres.path: "backup.sql"
-        backupbot.restore.post-hook: '/pg_backup.sh restore'
+        backupbot.restore.post-hook: "/pg_backup.sh restore"
     configs:
-        - source: pg_backup
-          target: /pg_backup.sh
-          mode: 0555
+      - source: pg_backup
+        target: /pg_backup.sh
+        mode: 0555
 
 volumes:
   data:

compress_state_entrypoint.sh

@@ -0,0 +1,46 @@
+#!/bin/sh
+set -e
+
+BINARY="/build/synapse_auto_compressor"
+REPO_DIR="/build/rust-synapse-compress-state"
+DB_PASS=$(cat /run/secrets/db_password)
+CONN="postgresql://synapse:${DB_PASS}@db:5432/synapse"
+CHUNK_SIZE="${STATE_COMPRESS_CHUNK_SIZE:-500}"
+CHUNKS="${STATE_COMPRESS_CHUNKS:-100}"
+SCHEDULE="${STATE_COMPRESS_SCHEDULE:-0 3 * * *}"
+
+# Build from source if binary doesn't exist
+if [ ! -f "$BINARY" ]; then
+  echo "[compress-state] Binary not found, building from source..."
+  apk add --no-cache git openssl-dev openssl-libs-static perl make musl-dev jemalloc-dev
+  rm -rf "$REPO_DIR"
+  git clone https://github.com/matrix-org/rust-synapse-compress-state "$REPO_DIR"
+  cd "$REPO_DIR"
+  cargo build --release -p synapse_auto_compressor
+  cp target/release/synapse_auto_compressor "$BINARY"
+  echo "[compress-state] Build complete"
+  # Clean up source to save space
+  rm -rf "$REPO_DIR"
+else
+  echo "[compress-state] Using cached binary"
+fi
+
+# Run once at startup
+echo "[compress-state] Running initial compression at $(date)"
+"$BINARY" -p "$CONN" -c "$CHUNK_SIZE" -n "$CHUNKS" || echo "[compress-state] Error: $?"
+
+# Set up cron job
+CRON_SCRIPT="/build/run_compressor.sh"
+cat > "$CRON_SCRIPT" <<EOF
+#!/bin/sh
+echo "[compress-state] Running at \$(date)"
+$BINARY -p "$CONN" -c $CHUNK_SIZE -n $CHUNKS || echo "[compress-state] Error: \$?"
+echo "[compress-state] Done at \$(date)"
+EOF
+chmod +x "$CRON_SCRIPT"
+
+echo "$SCHEDULE $CRON_SCRIPT" | crontab -
+echo "[compress-state] Cron scheduled: $SCHEDULE"
+
+# Run crond in the foreground
+exec crond -f -l 2

nginx.conf.tmpl

@@ -28,7 +28,7 @@ http {
       proxy_set_header X-Forwarded-For $remote_addr;
       proxy_set_header X-Forwarded-Proto https;
       proxy_set_header Host $host;
-      client_max_body_size {{ or (env "MAX_UPLOAD_SIZE") "50M" }};
+      client_max_body_size 50M;
       proxy_http_version 1.1;
     }
     
@@ -37,7 +37,7 @@ http {
       proxy_set_header X-Forwarded-For $remote_addr;
       proxy_set_header X-Forwarded-Proto https;
       proxy_set_header Host $host;
-      client_max_body_size {{ or (env "MAX_UPLOAD_SIZE") "50M" }};
+      client_max_body_size 50M;
       proxy_http_version 1.1;
     }
 
@@ -56,7 +56,7 @@ http {
       proxy_set_header X-Forwarded-For $remote_addr;
       proxy_set_header X-Forwarded-Proto https;
       proxy_set_header Host $host;
-      client_max_body_size {{ or (env "MAX_UPLOAD_SIZE") "50M" }};
+      client_max_body_size 50M;
       proxy_http_version 1.1;
     }
 {{ end }}

pg_backup.sh

@@ -6,7 +6,7 @@ BACKUP_FILE='/var/lib/postgresql/data/backup.sql'
 
 function backup {
   export PGPASSWORD=$(cat $POSTGRES_PASSWORD_FILE)
-  pg_dump -U ${POSTGRES_USER} ${POSTGRES_DB} > $BACKUP_FILE
+  pg_dump -U ${POSTGRES_USER} ${POSTGRES_DB} | gzip > $BACKUP_FILE
 }
 
 function restore {
@@ -25,7 +25,7 @@ function restore {
     # Recreate Database
     psql -U ${POSTGRES_USER} -d postgres -c "DROP DATABASE ${POSTGRES_DB} WITH (FORCE);" 
     createdb -U ${POSTGRES_USER} ${POSTGRES_DB}
-    psql -U ${POSTGRES_USER} -d ${POSTGRES_DB} -1 -f $BACKUP_FILE
+    gunzip -c $BACKUP_FILE | psql -U ${POSTGRES_USER} -d ${POSTGRES_DB} -1 -f -
 
     trap - EXIT INT TERM
     restore_config

release/6.8.2+v1.139.2

@@ -0,0 +1 @@
+this patch is a reset to the state of the last known deploying version 6.8.0 – so better skip 6.8.1

release/7.0.0+v1.149.1

@@ -0,0 +1,2 @@
+WARNING: Backup your database!
+This upgrade switches the database image from postgres to pgautoupgrade and performs an in-place database upgrades from version 13 to 17.