add comrade backupbot

This config allows matrix-synapse to create a endpoint on
$DOMAIN/.well-known/matrix/server used for federation.
It's a straight forward way (no port opening required)
to enable federation if you are okay having a subdomain
(matrix.example.org) as your homeserver address.

.env.sample

@@ -0,0 +1,64 @@
+TYPE=matrix-synapse
+
+DOMAIN=matrix.example.com
+LETS_ENCRYPT_ENV=production
+
+SECRET_DB_PASSWORD_VERSION=v1
+
+SYNAPSE_ADMIN_EMAIL=admin@example.com
+
+SECRET_REGISTRATION_SHARED_SECRET_VERSION=v1
+SECRET_MACAROON_SECRET_KEY_VERSION=v1
+SECRET_FORM_SECRET_VERSION=v1
+
+COMPOSE_FILE="compose.yml"
+
+#DISABLE_FEDERATION=1
+
+# Set "true" to enable federation endpoint on $DOMAIN/.well-known/matrix/server
+SERVE_SERVER_WELLKNOWN=false 
+
+ENABLE_REGISTRATION=false
+PASSWORD_LOGIN_ENABLED=true
+
+#AUTO_JOIN_ROOM_ENABLED=1
+#AUTO_JOIN_ROOM="#example:example.com"
+
+SQL_LOG_LEVEL=WARN
+ROOT_LOG_LEVEL=WARN
+
+REDACTION_RETENTION_PERIOD=7d
+
+RETENTION_MAX_LIFETIME=1m
+
+ENABLE_3PID_LOOKUP=true
+
+USER_IPS_MAX_AGE=1d
+
+ENCRYPTED_BY_DEFAULT=all
+
+#ENABLE_ALLOWLIST=1
+#FEDERATION_ALLOWLIST="[]"
+
+#COMPOSE_FILE="compose.yml:compose.keycloak.yml"
+#KEYCLOAK_ENABLED=1
+#KEYCLOAK_NAME=
+#KEYCLOAK_URL=
+#KEYCLOAK_CLIENT_ID=
+#KEYCLOAK_CLIENT_DOMAIN=
+#SECRET_KEYCLOAK_CLIENT_SECRET_VERSION=v1
+
+#COMPOSE_FILE="compose.yml:compose.turn.yml"
+#TURN_ENABLED=1
+#TURN_URIS="[\"turns:coturn.foo.zone?transport=udp\", \"turns:coturn.foo.zone?transport=tcp\"]"
+#TURN_ALLOW_GUESTS=true
+#SECRET_TURN_SHARED_SECRET_VERSION=v1
+
+#COMPOSE_FILE="compose.yml:compose.smtp.yml"
+#SMTP_ENABLED=1
+#SMTP_APP_NAME=
+#SMTP_FROM=
+#SMTP_HOST=
+#SMTP_PORT=
+#SMTP_USER=
+#SECRET_SMTP_PASSWORD_VERSION=v1

.envrc.sample

@@ -1,6 +0,0 @@
-export DOMAIN=matrix.example.com
-export STACK_NAME=matrix
-
-export LETS_ENCRYPT_ENV=production
-
-export ENTRYPOINT_CONF_VERSION=v1

.gitignore

@@ -1 +1,2 @@
-/.envrc
+.envrc
+synapse

README.md

@@ -1,18 +1,81 @@
-# Matrix Synapse
+# Matrix (Synapse)
 
-Matrix Synapse, based on the [official `matrixdotorg/synapse`
-image][synapse-docker].
+<!-- metadata -->
 
-1. Set up Docker Swarm and [`abra`][abra]
-2. Deploy [`compose-stacks/traefik`][compose-traefik]
-2. `cp .envrc.sample .envrc`
-3. Edit `.envrc` - be sure to change `$DOMAIN` to something that resolves to
-   your Docker swarm box
-4. `direnv allow` (or `. .envrc`)
-5. `abra deploy`
-6. `abra service_run synapse` to open a shell
-7. `register_new_matrix_user -c /data/homeserver.yaml http://localhost:8008`
+* **Category**: Apps
+* **Status**: 0, work-in-progress
+* **Image**: [`matrixdotorg/synapse`](https://hub.docker.com/r/matrixdotorg/synapse), 4, upstream
+* **Healthcheck**: Yes
+* **Backups**: No
+* **Email**: Yes
+* **Tests**: No
+* **SSO**: Yes
 
-[synapse-docker]: https://hub.docker.com/r/matrixdotorg/synapse
-[abra]: https://git.autonomic.zone/autonomic-cooperative/abra
-[compose-traefik]: https://git.autonomic.zone/compose-stacks/traefik
+<!-- endmetadata -->
+
+## Basic usage
+
+1. Set up Docker Swarm and [`abra`](https://docs.coopcloud.tech/abra/)
+2. Deploy [`coop-cloud/traefik`](https://git.coopcloud.tech/coop-cloud/traefik)
+3. `abra app new matrix-synapse --secrets` (optionally with `--pass` if you'd like to save secrets in `pass`)
+4. `abra app YOURAPPDOMAIN config` - be sure to change `$DOMAIN` to something that resolves to your Docker swarm box
+5. `abra app YOURAPPDOMAIN deploy`
+6. Create an initial user: `abra app YOURAPPDOMAIN run app register_new_matrix_user -c /data/homeserver.yaml http://localhost:8008`
+
+## Tips & Tricks
+
+### Disabling federation
+
+> We're not sure this does it exactly and there is still a discussion running
+> upstrem about whether this is the right way to do it & whether it could be
+> more convenient. We welcome issues / change sets to close up more federation
+> functionality.
+
+- use `DISABLE_FEDERATION=1` to turn off federation listeners
+- don't use [`compose.matrix.yml`](https://git.coopcloud.tech/coop-cloud/traefik/src/branch/master/compose.matrix.yml) in your traefik config to keep the federation ports closed
+
+### Enabling federation
+
+See [`#27`](https://git.coopcloud.tech/coop-cloud/matrix-synapse/pulls/27) for more.
+
+Depending on your setup, using `SERVE_SERVER_WELLKNOWN=true` might work to start federating.
+
+### Seeing what changed in `homeserver.yaml` between versions
+
+Change the version range to suit your needs.
+
+```
+git clone https://github.com/matrix-org/synapse
+cd synapse/docs
+git log --follow -p v1.48.0..v1.51.0 sample_config.yaml
+```
+
+### Generating a new `homeserver.yaml`
+
+The default is also available to see [here](https://matrix-org.github.io/synapse/latest/usage/configuration/homeserver_sample_config.html).
+
+```
+docker run -it \
+  --entrypoint="" \
+  -e SYNAPSE_SERVER_NAME=foo.com \
+  -e SYNAPSE_REPORT_STATS=no \
+  matrixdotorg/synapse:v1.48.0 \
+  sh -c '/start.py generate; cat /data/homeserver.yaml' > homeserver.yaml.tmpl`
+```
+
+### Generating a new `<server>.log.config`
+
+```
+docker run -it \
+  --entrypoint="" \
+  -e SYNAPSE_SERVER_NAME=foo.com \
+  -e SYNAPSE_REPORT_STATS=no \
+  matrixdotorg/synapse:v1.48.0 \
+  sh -c '/start.py generate; cat /data/foo.com.log.config' > log.config
+```
+
+### Getting client discovery on a custom domain
+
+You'll need to deploy something like [this](https://git.autonomic.zone/ruangrupa/well-known-uris).
+
+This could be implemented in this recipe but we haven't merged it in yet. Chang sets are welcome.

abra.sh

@@ -0,0 +1,3 @@
+export ENTRYPOINT_CONF_VERSION=v1
+export HOMESERVER_YAML_VERSION=v8
+export LOG_CONFIG_VERSION=v2

compose.keycloak.yml

@@ -0,0 +1,22 @@
+---
+version: "3.8"
+
+services:
+  app:
+    secrets:
+      - db_password
+      - form_secret
+      - keycloak_client_secret
+      - macaroon_secret_key
+      - registration_shared_secret
+    environment:
+      - KEYCLOAK_CLIENT_DOMAIN
+      - KEYCLOAK_CLIENT_ID
+      - KEYCLOAK_ENABLED
+      - KEYCLOAK_NAME
+      - KEYCLOAK_URL
+
+secrets:
+  keycloak_client_secret:
+    external: true
+    name: ${STACK_NAME}_keycloak_client_secret_${SECRET_KEYCLOAK_CLIENT_SECRET_VERSION}

compose.keycloak2.yml

@@ -0,0 +1,17 @@
+---
+version: "3.8"
+
+services:
+  app:
+    secrets:
+      - keycloak2_client_secret
+    environment:
+      - KEYCLOAK2_CLIENT_ID
+      - KEYCLOAK2_ENABLED
+      - KEYCLOAK2_NAME
+      - KEYCLOAK2_URL
+
+secrets:
+  keycloak2_client_secret:
+    external: true
+    name: ${STACK_NAME}_keycloak2_client_secret_${SECRET_KEYCLOAK2_CLIENT_SECRET_VERSION}

compose.smtp.yml

@@ -0,0 +1,23 @@
+---
+version: "3.8"
+
+services:
+  app:
+    secrets:
+      - db_password
+      - form_secret
+      - macaroon_secret_key
+      - registration_shared_secret
+      - smtp_password
+    environment:
+      - SMTP_APP_NAME
+      - SMTP_ENABLED
+      - SMTP_FROM
+      - SMTP_HOST
+      - SMTP_PORT
+      - SMTP_USER
+
+secrets:
+  smtp_password:
+    external: true
+    name: ${STACK_NAME}_smtp_password_${SECRET_SMTP_PASSWORD_VERSION}

compose.turn.yml

@@ -0,0 +1,20 @@
+---
+version: "3.8"
+
+services:
+  app:
+    secrets:
+      - db_password
+      - form_secret
+      - macaroon_secret_key
+      - registration_shared_secret
+      - turn_shared_secret
+    environment:
+      - TURN_ALLOW_GUESTS
+      - TURN_ENABLED
+      - TURN_URIS
+
+secrets:
+  turn_shared_secret:
+    external: true
+    name: ${STACK_NAME}_turn_shared_secret_${SECRET_TURN_SHARED_SECRET_VERSION}

compose.yml

@@ -2,38 +2,88 @@
 version: "3.8"
 
 services:
-  synapse:
-    image: "matrixdotorg/synapse:latest"
+  app:
+    image: "matrixdotorg/synapse:v1.55.2"
     volumes:
       - "data:/data"
+    secrets:
+      - db_password
+      - registration_shared_secret
+      - macaroon_secret_key
+      - form_secret
     environment:
+      - ENCRYPTED_BY_DEFAULT
+      - AUTO_JOIN_ROOM
+      - AUTO_JOIN_ROOM_ENABLED
+      - DISABLE_FEDERATION
+      - DOMAIN
+      - ENABLE_3PID_LOOKUP
+      - ENABLE_ALLOWLIST
+      - ENABLE_REGISTRATION
+      - FEDERATION_ALLOWLIST
+      - LETSENCRYPT_HOST=${DOMAIN}
+      - PASSWORD_LOGIN_ENABLED
+      - REDACTION_RETENTION_PERIOD
+      - ROOT_LOG_LEVEL
+      - SERVE_SERVER_WELLKNOWN
+      - SQL_LOG_LEVEL
+      - STACK_NAME
+      - SYNAPSE_ADMIN_EMAIL
+      - SYNAPSE_REPORT_STATS=no
+      - SYNAPSE_SERVER_NAME=${DOMAIN}
+      - USER_IPS_MAX_AGE
       - VIRTUAL_HOST=${DOMAIN}
       - VIRTUAL_PORT=8008
-      - LETSENCRYPT_HOST=${DOMAIN}
-      - SYNAPSE_SERVER_NAME=${DOMAIN}
-      - SYNAPSE_REPORT_STATS=no
     networks:
       - proxy
+      - internal
+    entrypoint: /docker-entrypoint.sh
+    configs:
+      - source: homeserver_yaml
+        target: /data/homeserver.yaml
+      - source: log_config
+        target: /data/log.config
+      - source: entrypoint_conf
+        target: /docker-entrypoint.sh
+        mode: 0555
     deploy:
+      restart_policy:
+        condition: on-failure
       labels:
         - "traefik.enable=true"
         - "traefik.http.services.${STACK_NAME}.loadbalancer.server.port=8008"
         - "traefik.http.routers.${STACK_NAME}.rule=Host(`${DOMAIN}`)"
         - "traefik.http.routers.${STACK_NAME}.entrypoints=web-secure"
         - "traefik.http.routers.${STACK_NAME}.tls.certresolver=${LETS_ENCRYPT_ENV}"
-      restart_policy:
-        condition: on-failure
-        delay: "60s"
-        max_attempts: 3
-        window: 120s
-    entrypoint: /docker-entrypoint.sh
-    configs:
-      - source: entrypoint_conf
-        target: /docker-entrypoint.sh
-        mode: 0555
+        - "coop-cloud.${STACK_NAME}.version=1.3.0+v1.55.2"
+
+  db:
+    image: postgres:13-alpine
+    secrets:
+      - db_password
+    environment:
+      - LC_COLLATE=C
+      - LC_CTYPE=C
+      - POSTGRES_DB=synapse
+      - POSTGRES_INITDB_ARGS="-E \"UTF8\""
+      - POSTGRES_PASSWORD_FILE=/run/secrets/db_password
+      - POSTGRES_USER=synapse
+    networks:
+      - internal
+    healthcheck:
+      test: ["CMD", "pg_isready", "-U", "synapse"]
+    volumes:
+      - postgres:/var/lib/postgresql/data
+    deploy:
+      labels:
+          backupbot.backup: "true"
+          backupbot.backup.pre-hook: "mkdir -p /tmp/backup/ && PGPASSWORD=$$(cat $${POSTGRES_PASSWORD_FILE}) pg_dump -U $${POSTGRES_USER} $${POSTGRES_DB} > /tmp/backup/backup.sql"
+          backupbot.backup.post-hook: "rm -rf /tmp/backup"
+          backupbot.backup.path: "/tmp/backup/"
 
 volumes:
   data:
+  postgres:
 
 networks:
   proxy:
@@ -45,3 +95,25 @@ configs:
     name: ${STACK_NAME}_entrypoint_${ENTRYPOINT_CONF_VERSION}
     file: entrypoint.sh.tmpl
     template_driver: golang
+  homeserver_yaml:
+    name: ${STACK_NAME}_homserver_yaml_${HOMESERVER_YAML_VERSION}
+    file: homeserver.yaml.tmpl
+    template_driver: golang
+  log_config:
+    name: ${STACK_NAME}_log_config_${LOG_CONFIG_VERSION}
+    file: log.config.tmpl
+    template_driver: golang
+
+secrets:
+  db_password:
+    external: true
+    name: ${STACK_NAME}_db_password_${SECRET_DB_PASSWORD_VERSION}
+  registration_shared_secret:
+    external: true
+    name: ${STACK_NAME}_db_password_${SECRET_REGISTRATION_SHARED_SECRET_VERSION}
+  macaroon_secret_key:
+    external: true
+    name: ${STACK_NAME}_db_password_${SECRET_MACAROON_SECRET_KEY_VERSION}
+  form_secret:
+    external: true
+    name: ${STACK_NAME}_db_password_${SECRET_FORM_SECRET_VERSION}

entrypoint.sh.tmpl

@@ -1,7 +1,11 @@
 #!/bin/bash
 
-if [[ ! -f /data/homeserver.yaml ]]; then
-	/start.py generate
+# https://github.com/matrix-org/synapse/tree/develop/docker#running-synapse
+# default user permissions for the synapse user
+chown 991:991 /data
+
+if [[ ! -f /data/{{ env "DOMAIN" }}.signing.key ]]; then
+    /start.py generate
 fi
 
 /start.py

log.config.tmpl

@@ -0,0 +1,20 @@
+version: 1
+
+formatters:
+  precise:
+    format: '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s - %(message)s'
+
+handlers:
+  console:
+    class: logging.StreamHandler
+    formatter: precise
+
+loggers:
+    synapse.storage.SQL:
+      level: {{ env "SQL_LOG_LEVEL" }}
+
+root:
+    level: {{ env "ROOT_LOG_LEVEL" }}
+    handlers: [console]
+
+disable_existing_loggers: false

release/1.3.0+v1.55.2

@@ -0,0 +1,6 @@
+The deployment failed due to the app/db getting confused. I think this is just
+due to the recipe not having good healthcheck config. After the app container
+flapped a bit, everything came up nicely. d1 @ autonomic co-op.
+
+Same thing happened to me when deploying this for another instance. Also d1 @
+autonomic co-op.