Adapt recipe to Docs v6 (yhub collaboration server)

- the new yhub service needs new initialisation and RSA keys
- SOFT_MIGRATION on by default, add migrate_documents command
- back up the yhub database too
- document the v5 -> v6 upgrade in README and release/next

Following the [C4DT AI Policy](https://git.coopcloud.tech/c4dt/ai-policy/src/branch/main/AI-POLICY.md):

 #LLM_vibe
 #LLM_model Claude Opus 5.5

- [x] I take full responsibility for this code
- [x] I understand every line of this code
- [x] I tested this code thoroughly on my own computer / servers
- [x] None of the commit messages were created using LLMs
- [x] I will participate in any discussion on my own and will not use an LLM for the discussion

.env.sample

@@ -21,14 +21,16 @@ SECRET_DJANGO_SP_VERSION=v1
 SECRET_MINIO_RP_VERSION=v1
 # MINIO_ROOT_USER
 SECRET_MINIO_RU_VERSION=v1
-# COLLABORATION_SERVER_SECRET
-SECRET_COLLAB_SS_VERSION=v1
 # POSTGRES_PASSWORD
 SECRET_POSTGRES_P_VERSION=v1
 # Y_PROVIDER_API_KEY
 SECRET_Y_API_KEY_VERSION=v1
 # DJANGO_HOST_EMAIL_PASSWORD
 SECRET_EMAIL_PASS_VERSION=v1
+# JWT_PRIVATE_KEY of the backend (RSA PEM, must be inserted manually, see README)
+SECRET_JWT_PK_VERSION=v1 # generate=false
+# YHUB_JWT_PRIVATE_KEY of the collaboration server (RSA PEM, must be inserted manually, see README)
+SECRET_YHUB_JWT_PK_VERSION=v1 # generate=false
 
 ##############################################################################
 # EMAIL
@@ -66,3 +68,53 @@ OIDC_AUTH_REQUEST_EXTRA_PARAMS='{"acr_values": "eidas1"}'
 LOGGING_LEVEL_HANDLERS_CONSOLE=INFO
 LOGGING_LEVEL_LOGGERS_ROOT=INFO
 LOGGING_LEVEL_LOGGERS_APP=INFO
+
+##############################################################################
+# MIGRATIONS
+##############################################################################
+# Set to false to disable automatic migrations on backend startup
+# AUTO_MIGRATIONS=true
+
+##############################################################################
+# CONVERSION (file upload → Yjs via y-provider)
+##############################################################################
+# Allow uploading .docx/.md files for conversion to Yjs.
+# Recipe sets this to true (upstream default is false).
+# CONVERSION_UPLOAD_ENABLED=true
+# Upstream defaults (uncomment to override):
+# CONVERSION_FILE_MAX_SIZE=20971520
+# CONVERSION_FILE_EXTENSIONS_ALLOWED=.docx,.md
+# CONVERSION_API_ENDPOINT=convert
+# CONVERSION_API_CONTENT_FIELD=content
+# CONVERSION_API_TIMEOUT=30
+# CONVERSION_API_SECURE=false
+
+##############################################################################
+# DATABASE CONNECTION POOL
+##############################################################################
+# Upstream default (uncomment to override):
+# DB_PSYCOPG_POOL_ENABLED=false
+
+##############################################################################
+# EMAIL APP URL
+##############################################################################
+# DJANGO_EMAIL_URL_APP is set automatically in compose.yml to https://${DOMAIN}.
+# Override is not exposed via .env — edit compose.yml directly if needed.
+# (Upstream default falls back to Django's Site framework, not configured here.)
+
+##############################################################################
+# MEDIA AUTH HEADER
+##############################################################################
+# Header the backend reads for media auth subrequests.
+# Upstream default matches the recipe's nginx config (X-Original-URL).
+# MEDIA_AUTH_ORIGINAL_URL_HEADER=HTTP_X_ORIGINAL_URL
+
+##############################################################################
+# COLLABORATION SERVER (yhub) - SOFT MIGRATION
+##############################################################################
+# Since v6 the document content lives in the collaboration server, not in the
+# media bucket. With SOFT_MIGRATION, a pre-v6 document is copied over the first
+# time it is opened. Only set to false once
+# `abra app cmd <app-name> backend migrate_documents` has covered all documents
+# (or on a fresh v6 install); unmigrated documents would otherwise open empty.
+# SOFT_MIGRATION=true

MAINTENANCE.md

@@ -0,0 +1,30 @@
+# Lasuite-Docs Recipe Maintenance
+
+All contributions should be made via pull requests.
+
+## Pull Requests
+
+A pull request can be merged directly into `main` by any maintainer. 
+No approval is needed, but pull requests should be tested as working before merge.
+
+## Dependency Updates
+
+- patch updates proposed from renovate bot are merged directly - 
+  if there is a CVE, also release a new patch version of this recipe
+- minor updates proposed from renovate bot are tested by one maintainer 
+  and merged if they don't break, then a new minor version of the recipe 
+  is released
+- major updates proposed form renovate bot are tested by two maintainers 
+  and merged if they don't break, then a major version of the recipe is 
+  released. 
+  If the renovate bot PR is older than one week, one maintainer can decide
+  to merge it anyway. 
+
+## Become a maintainer
+
+Everyone can apply to be a recipe maintainer:
+
+1. Watch the repository to get updates.
+2. Add yourself to the list in [README.md](./README.md) and open a pull request with the change.
+3. Once the pull request is merged, you will be added to the 
+  [La Suite maintainers team](https://git.coopcloud.tech/org/coop-cloud/teams/lasuite-maintainers).

README.md

@@ -4,6 +4,7 @@
 
 <!-- metadata -->
 
+* **Maintainer**: [@trav](https://git.coopcloud.tech/trav), [@notplants](https://git.coopcloud.tech/notplants), [@ineiti](https://git.coopcloud.tech/ineiti)
 * **Category**: Apps
 * **Status**: 2
 * **Image**: [`lasuite/docs`](https://hub.docker.com/r/lasuite/docs), 4, upstream
@@ -18,13 +19,35 @@
 ## Quick start
 
 * `abra app new lasuite-docs --secrets`
+* Insert the two RSA keys, which abra cannot generate (see below)
 * `abra app config <app-name>`
 * `abra app deploy <app-name>`
-* `abra app cmd <app-name> backend migrate`
-* `abra app restart <app-name> minio-bootstrap` (Note: this will appear to fail, but probably worked! Check `abra app logs <app-name> minio-bootstrap`)
 
-You should then be able to visit the landing page of your app, but not yet to login. To login, you need to deploy and integrate single sign on (described below in the "Configure Authentication" section). 
+You should then be able to visit the landing page of your app, but not yet to login. To login, you need to deploy and integrate single sign on (described below in the "Configure Authentication" section).
 
+* Migrations run automatically on backend startup. To trigger manually: `abra app cmd <app-name> backend migrate`
+* Minio buckets are created automatically on first deploy. To manually trigger: `abra app cmd <app-name> minio minio_initialize`
+
+## RSA keys
+
+Since v6, the backend and the collaboration server (`yhub`) authenticate each other with signed tokens. Each needs its own RSA private key, never the same one:
+
+```
+openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out jwt.pem
+openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out yhub-jwt.pem
+abra app secret insert <app-name> jwt_pk v1 jwt.pem -f
+abra app secret insert <app-name> yhub_jwt_pk v1 yhub-jwt.pem -f
+rm jwt.pem yhub-jwt.pem
+```
+
+## Upgrading from v5 to v6
+
+v6 replaces the collaboration server. Document content moves from the minio bucket to the new `yhub` service, which stores it in its own `yhub` database. If you skip the migration, every existing document opens **empty**. See [UPGRADE.md](https://github.com/suitenumerique/docs/blob/main/UPGRADE.md) upstream for details.
+
+1. Insert the two RSA keys (see "RSA keys" above). The `collab_ss` secret is no longer used.
+2. Leave `SOFT_MIGRATION` at its default (`true`) and deploy. The `yhub` schema is created on startup. A document is copied to yhub the first time someone opens it.
+3. Copy all remaining documents: `abra app cmd <app-name> backend migrate_documents`. You can pass arguments, e.g. `abra app cmd <app-name> backend migrate_documents -- --dry-run`. Re-run it with `--retry-failed` if some documents failed.
+4. Only after that, you may set `SOFT_MIGRATION=false`. Keep the minio bucket and its object versions: they hold the document history from before v6.
 
 ## Configure Authentication
 

abra-entrypoint.sh

@@ -7,7 +7,6 @@ set -e
 [ -f /run/secrets/minio_ru ] && export AWS_S3_ACCESS_KEY_ID="$(cat /run/secrets/minio_ru)"
 [ -f /run/secrets/django_sp ] && export DJANGO_SUPERUSER_PASSWORD="$(cat /run/secrets/django_sp)"
 [ -f /run/secrets/oidc_rpcs ] && export OIDC_RP_CLIENT_SECRET="$(cat /run/secrets/oidc_rpcs)"
-[ -f /run/secrets/collab_ss ] && export COLLABORATION_SERVER_SECRET="$(cat /run/secrets/collab_ss)"
 [ -f /run/secrets/y_api_key ] && export Y_PROVIDER_API_KEY="$(cat /run/secrets/y_api_key)"
 [ -f /run/secrets/email_pass ] && export DJANGO_EMAIL_HOST_PASSWORD="$(cat /run/secrets/email_pass)"
 

abra.sh

@@ -1,8 +1,10 @@
 # Set any config versions here
 # Docs: https://docs.coopcloud.tech/maintainers/handbook/#manage-configs
-export ABRA_ENTRYPOINT_VERSION=v5
-export NGINX_CONF_VERSION=v3
-export PG_BACKUP_VERSION=v3
+export ABRA_ENTRYPOINT_VERSION=v6
+export NGINX_CONF_VERSION=v4
+export PG_BACKUP_VERSION=v5
+export MINIO_INITIALIZE_VERSION=v2
+export MIGRATE_VERSION=v1
 
 environment() {
   # this exports all the secrets as environment variables
@@ -10,6 +12,17 @@ environment() {
 }
 
 migrate() {
-  environment
-  python manage.py migrate --noinput
+  /migrate.sh
+}
+
+minio_initialize() {
+  /minio-initialize.sh
+}
+
+# Hand all pre-v6 documents from the media bucket over to the collaboration
+# server (yhub). Resumable and safe to re-run; extra args are passed through,
+# e.g. --dry-run, --retry-failed, --concurrency N
+migrate_documents() {
+  source /abra-entrypoint.sh -e
+  python manage.py migrate_documents "$@"
 }

compose.yml

@@ -49,13 +49,34 @@ x-common-env: &common-env
   LOGOUT_REDIRECT_URL:
   OIDC_REDIRECT_ALLOWED_HOSTS:
   OIDC_AUTH_REQUEST_EXTRA_PARAMS:
-  # AI (Fixme: remove?)
+  # AI
   AI_FEATURE_ENABLED: "false"
-  AI_BASE_URL: https://openaiendpoint.com
-  AI_API_KEY: password
+  OPENAI_SDK_BASE_URL: https://openaiendpoint.com
+  OPENAI_SDK_API_KEY: password
   AI_MODEL: llama
-  # Collaboration
-  COLLABORATION_API_URL: https://$DOMAIN/collaboration/api/
+  # Collaboration server (yhub), reached in-stack
+  YHUB_API_BASE_URL: http://yhub:3002
+  # RSA key signing the tokens the backend presents to yhub and y-provider
+  JWT_PRIVATE_KEY_FILE: /run/secrets/jwt_pk
+  # Email app URL (used in email templates; matches recipe DOMAIN)
+  DJANGO_EMAIL_URL_APP: "https://${DOMAIN}"
+  # Conversion (file upload → Yjs via y-provider)
+  CONVERSION_UPLOAD_ENABLED: "${CONVERSION_UPLOAD_ENABLED:-true}"
+  CONVERSION_FILE_MAX_SIZE:
+  CONVERSION_FILE_EXTENSIONS_ALLOWED:
+  CONVERSION_API_ENDPOINT:
+  CONVERSION_API_CONTENT_FIELD:
+  CONVERSION_API_TIMEOUT:
+  CONVERSION_API_SECURE:
+  # Y-Provider base URL + API key (needed when CONVERSION_UPLOAD_ENABLED is true)
+  Y_PROVIDER_API_BASE_URL: http://y-provider:4444/api/
+  Y_PROVIDER_API_KEY_FILE: /run/secrets/y_api_key
+  # DocSpec — converts .docx → BlockNote JSON, then y-provider turns that into Yjs
+  DOCSPEC_API_URL: http://docspec:4000/conversion
+  # Database connection pool
+  DB_PSYCOPG_POOL_ENABLED:
+  # Media auth header (matches recipe nginx by default)
+  MEDIA_AUTH_ORIGINAL_URL_HEADER:
 
 x-postgres-env: &postgres-env
   # Postgresql db container configuration
@@ -72,11 +93,26 @@ x-postgres-env: &postgres-env
 x-yprovider-env: &yprovider-env
   COLLABORATION_LOGGING: "true"
   # Y_PROVIDER_API_KEY supplied via secrets
-  COLLABORATION_API_URL: http://y-provider:4444/api/
   COLLABORATION_SERVER_ORIGIN: https://${DOMAIN}
-  # COLLABORATION_SERVER_SECRET supplied via secrets
   COLLABORATION_BACKEND_BASE_URL: https://${DOMAIN}
-  COLLABORATION_WS_URL: wss://${DOMAIN}/collaboration/ws/
+  # The last segment is YHUB_ORG (default: docs)
+  COLLABORATION_WS_URL: wss://${DOMAIN}/collaboration/ws/v1/docs
+
+x-yhub-env: &yhub-env
+  PORT: 3002
+  REDIS: redis://yhub-valkey:6379
+  # POSTGRES is built from the postgres_p secret in the entrypoint
+  COLLABORATION_BACKEND_BASE_URL: https://${DOMAIN}
+  COLLABORATION_SERVER_ORIGIN: https://${DOMAIN}
+  Y_PROVIDER_API_KEY_FILE: /run/secrets/y_api_key
+  YHUB_JWT_PRIVATE_KEY_FILE: /run/secrets/yhub_jwt_pk
+  # Seed documents from the pre-v6 media bucket the first time they are opened.
+  # Only turn off once `abra app cmd <app> backend migrate_documents` covered all documents.
+  SOFT_MIGRATION: "${SOFT_MIGRATION:-true}"
+  LEGACY_S3_ENDPOINT_URL: http://minio:9000
+  LEGACY_S3_ACCESS_KEY_ID_FILE: /run/secrets/minio_ru
+  LEGACY_S3_SECRET_ACCESS_KEY_FILE: /run/secrets/minio_rp
+  LEGACY_S3_BUCKET_NAME: docs-media-storage
 
 x-minio-env: &minio-env
   MINIO_ROOT_USER_FILE: /run/secrets/minio_ru
@@ -84,14 +120,14 @@ x-minio-env: &minio-env
 
 services:
   app:
-    image: lasuite/impress-frontend:v4.4.0
+    image: lasuite/impress-frontend:v6.0.1
     networks:
       - backend
     deploy:
       labels:
         - "traefik.enable=false"
         - "coop-cloud.${STACK_NAME}.timeout=${TIMEOUT:-120}"
-        - "coop-cloud.${STACK_NAME}.version=0.2.5+v4.4.0"
+        - "coop-cloud.${STACK_NAME}.version=0.4.4+v5.6.1"
     user: "${DOCKER_USER:-1000}"
     healthcheck:
       test: ["CMD", "curl", "-f", "http://localhost:8080"]
@@ -101,11 +137,12 @@ services:
       start_period: 10s
 
   backend:
-    image: lasuite/impress-backend:v4.4.0
+    image: lasuite/impress-backend:v6.0.1
     networks:
-      - backend 
+      - backend
     environment:
       <<: [*common-env, *postgres-env, *yprovider-env]
+      AUTO_MIGRATIONS: "${AUTO_MIGRATIONS:-true}"
     healthcheck:
       test: ["CMD", "/abra-entrypoint.sh", "python", "manage.py", "check"]
       interval: 15s
@@ -114,26 +151,36 @@ services:
       start_period: 10s
     user: "${DOCKER_USER:-1000}"
     command: ["gunicorn", "-c", "/usr/local/etc/gunicorn/impress.py", "impress.wsgi:application"]
-    entrypoint: ["/abra-entrypoint.sh", "/usr/local/bin/entrypoint"]
+    entrypoint: >
+      sh -c "if [ \"$$AUTO_MIGRATIONS\" = \"true\" ]; then /migrate.sh; fi && exec /abra-entrypoint.sh /usr/local/bin/entrypoint \"$$@\"" --
     configs:
       - source: abra_entrypoint
         target: /abra-entrypoint.sh
         mode: 0555
+      - source: migrate
+        target: /migrate.sh
+        mode: 0555
     secrets:
       - django_sk
       - django_sp
       - oidc_rpcs
-      - collab_ss
       - minio_rp
       - minio_ru
       - postgres_p
       - y_api_key
       - email_pass
+      - jwt_pk
 
   celery:
-    image: lasuite/impress-backend:v4.4.0
+    image: lasuite/impress-backend:v6.0.1
     networks:
       - backend
+    healthcheck:
+      test: ["CMD", "celery", "-A", "impress.celery_app", "inspect", "ping", "--timeout", "5"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 30s
     user: "${DOCKER_USER:-1000}"
     command: ["celery", "-A", "impress.celery_app", "worker", "-l", "INFO"]
     environment:
@@ -147,28 +194,88 @@ services:
       - django_sk
       - django_sp
       - oidc_rpcs
-      - collab_ss
       - minio_rp
+      - minio_ru
       - postgres_p
       - y_api_key
       - email_pass
+      - jwt_pk
 
-
-  y-provider:
-    image: lasuite/impress-y-provider:v4.4.0
+  # Collaboration server: serves the websocket and holds the document content
+  yhub:
+    image: lasuite/impress-yhub:v6.0.1
     networks:
-      - backend 
+      - backend
+    environment: *yhub-env
+    user: "${DOCKER_USER:-1000}"
+    healthcheck:
+      test: ["CMD", "node", "-e", "fetch('http://localhost:3002/collaboration/ready/v1').then(r=>process.exit(r.ok?0:1),()=>process.exit(1))"]
+      interval: 15s
+      timeout: 5s
+      retries: 20
+      start_period: 30s
+    # The server never creates its schema: run the idempotent init-db before every start
+    entrypoint: >
+      sh -c "export POSTGRES=\"postgres://docs:$$(node -e 'process.stdout.write(encodeURIComponent(require(\"fs\").readFileSync(\"/run/secrets/postgres_p\",\"utf8\").trim()))')@db:5432/yhub\" && node node_modules/@y/hub/bin/init-db.js && exec /usr/local/bin/entrypoint \"$$@\"" --
+    command: ["node", "--import", "./dist/sentry.js", "dist/server.js"]
+    secrets:
+      - postgres_p
+      - y_api_key
+      - yhub_jwt_pk
+      - minio_ru
+      - minio_rp
+
+  # Valkey of yhub: holds updates not yet persisted to postgres, so it is a store, not a cache
+  yhub-valkey:
+    image: valkey/valkey:8.1.10-alpine
+    command: ["valkey-server", "--maxmemory-policy", "volatile-lru", "--appendonly", "yes", "--appendfsync", "everysec"]
+    healthcheck:
+      test: ["CMD", "valkey-cli", "ping"]
+      interval: 15s
+      timeout: 5s
+      retries: 3
+    networks:
+      - backend
+    volumes:
+      - yhub_valkey:/data
+    deploy:
+      labels:
+        backupbot.backup: "${ENABLE_BACKUPS:-true}"
+
+  # Conversion service only since v6 (no longer serves the collaboration)
+  y-provider:
+    image: lasuite/impress-y-provider:v6.0.1
+    networks:
+      - backend
+    healthcheck:
+      # y-provider returns 403 on unauthenticated requests; wget exit 4 = network error (server down), anything else = server is responding
+      test: ["CMD-SHELL", "wget -qO /dev/null http://localhost:4444/ 2>/dev/null; test $$? -ne 4"]
+      interval: 15s
+      timeout: 5s
+      retries: 3
+      start_period: 10s
     environment: *yprovider-env
     user: "${DOCKER_USER:-1000}"
     entrypoint: >
       sh -c "export Y_PROVIDER_API_KEY=\"$$(cat /run/secrets/y_api_key)\" && exec /usr/local/bin/entrypoint \"$$@\"" --
     command: ["yarn", "start"]
-    # NOTE: healthcheck - `wget` is available in the container, but `wget http://localhost:4444` gives a 403
     secrets:
       - y_api_key
 
+  docspec:
+    image: ghcr.io/docspecio/api:3.0.2
+    networks:
+      - backend
+    healthcheck:
+      # Use 127.0.0.1, which is required instead of localhost for this healtcheck because of elixir/busybox bindings
+      test: ["CMD", "wget", "-q", "-O", "/dev/null", "http://127.0.0.1:4000/health"]
+      interval: 15s
+      timeout: 5s
+      retries: 3
+      start_period: 10s
+
   db:
-    image: postgres:16
+    image: pgautoupgrade/pgautoupgrade:18-debian
     networks:
       - backend 
     healthcheck:
@@ -178,14 +285,14 @@ services:
       retries: 300
     environment:
       <<: *postgres-env
-      PGDATA: var/lib/postgresql/data/pgdata
+      PGDATA: /var/lib/postgresql/data/pgdata
     volumes:
       - postgres:/var/lib/postgresql/data/pgdata
     deploy:
       labels:
         backupbot.backup: "${ENABLE_BACKUPS:-true}"
         backupbot.backup.pre-hook: "/pg_backup.sh backup"
-        backupbot.backup.volumes.postgres.path: "backup.sql"
+        backupbot.backup.volumes.postgres.path: "backup.sql,backup-yhub.sql"
         backupbot.restore.post-hook: '/pg_backup.sh restore'
     configs:
         - source: pg_backup
@@ -195,35 +302,18 @@ services:
       - postgres_p
 
   redis:
-    image: redis:8
-    networks:
-      - backend 
-
-  minio-bootstrap:
-    # NOTE: Not started by default, only run with a manual `abra app restart` / `docker service scale`
-    image: minio/mc:RELEASE.2025-05-21T01-59-54Z
-    environment: *minio-env
+    image: redis:8.10.2
+    healthcheck:
+      test: ["CMD", "redis-cli", "ping"]
+      interval: 15s
+      timeout: 5s
+      retries: 3
     networks:
       - backend
-    entrypoint: >
-      sh -c "
-      MINIO_ROOT_USER=\"\$$(cat /run/secrets/minio_ru)\" &&   
-      MINIO_ROOT_PASSWORD=\"\$$(cat /run/secrets/minio_rp)\" &&   
-      /usr/bin/mc alias set docs http://minio:9000 \$${MINIO_ROOT_USER} \"\$${MINIO_ROOT_PASSWORD}\" &&
-      /usr/bin/mc mb --ignore-existing docs/docs-media-storage &&
-      /usr/bin/mc version enable docs/docs-media-storage &&
-      exit 0"
-    deploy:
-      mode: replicated
-      replicas: 0
-      restart_policy:
-        condition: none
-    secrets:
-      - minio_rp
-      - minio_ru
 
   minio:
-    image: minio/minio:RELEASE.2025-05-24T17-08-30Z
+    image: bitnamilegacy/minio:2025.7.23-debian-12-r5
+    user: root
     environment: *minio-env
     healthcheck:
       test: ["CMD", "mc", "ready", "local"]
@@ -232,10 +322,11 @@ services:
       retries: 300
     networks:
       - backend
-    command: minio server /data
-    entrypoint: ["/usr/bin/docker-entrypoint.sh"]
+    command: /opt/bitnami/scripts/minio/run.sh
+    entrypoint: >
+      sh -c "/minio-initialize.sh & exec /opt/bitnami/scripts/minio/entrypoint.sh \"$$@\"" --
     volumes:
-      - minio:/data
+      - minio:/bitnami/minio/data
     deploy:
       labels:
         backupbot.backup: "${ENABLE_BACKUPS:-true}"
@@ -244,12 +335,21 @@ services:
       - source: abra_entrypoint
         target: /abra-entrypoint.sh
         mode: 0555
+      - source: minio_initialize
+        target: /minio-initialize.sh
+        mode: 0555
     secrets:
       - minio_rp
       - minio_ru
 
   web:
-    image: nginx:1.29
+    image: nginx:1.31.7
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:8083"]
+      interval: 15s
+      timeout: 5s
+      retries: 3
+      start_period: 10s
     configs:
       - source: nginx_conf
         target: /etc/nginx/conf.d/default.conf
@@ -279,6 +379,7 @@ networks:
 volumes:
   postgres:
   minio:
+  yhub_valkey:
 
 configs:
   nginx_conf:
@@ -291,6 +392,12 @@ configs:
   abra_entrypoint:
     name: ${STACK_NAME}_entrypoint_${ABRA_ENTRYPOINT_VERSION}
     file: abra-entrypoint.sh
+  minio_initialize:
+    name: ${STACK_NAME}_minio_initialize_${MINIO_INITIALIZE_VERSION}
+    file: minio-initialize.sh
+  migrate:
+    name: ${STACK_NAME}_migrate_${MIGRATE_VERSION}
+    file: migrate.sh
 
 secrets:
   django_sk:
@@ -305,9 +412,12 @@ secrets:
   postgres_p:
     external: true
     name: ${STACK_NAME}_postgres_p_${SECRET_POSTGRES_P_VERSION}
-  collab_ss:
+  jwt_pk:
     external: true
-    name: ${STACK_NAME}_collab_ss_${SECRET_COLLAB_SS_VERSION}
+    name: ${STACK_NAME}_jwt_pk_${SECRET_JWT_PK_VERSION}
+  yhub_jwt_pk:
+    external: true
+    name: ${STACK_NAME}_yhub_jwt_pk_${SECRET_YHUB_JWT_PK_VERSION}
   minio_rp:
     external: true
     name: ${STACK_NAME}_minio_rp_${SECRET_MINIO_RP_VERSION}

migrate.sh

@@ -0,0 +1,26 @@
+#!/bin/sh
+set -e
+
+# Load secrets into environment
+source /abra-entrypoint.sh -e
+
+# Wait for database to be ready (up to 30 seconds)
+i=0
+while ! python manage.py check --database default 2>/dev/null; do
+  i=$((i+1))
+  if [ "$i" -ge 30 ]; then
+    echo "migrate: timed out waiting for database" >&2
+    exit 1
+  fi
+  sleep 1
+done
+
+# Idempotent: skip if no pending migrations
+if python manage.py migrate --check > /dev/null 2>&1; then
+  echo "migrate: no pending migrations, skipping"
+  exit 0
+fi
+
+echo "migrate: applying pending migrations..."
+python manage.py migrate --noinput
+echo "migrate: done"

minio-initialize.sh

@@ -0,0 +1,34 @@
+#!/bin/sh
+set -e
+
+# Use an isolated mc config dir: the bundled Bitnami image runs its own
+# internal mc client (as the "minio" user) against the default /.mc path,
+# which collides with this script running as root against the same path.
+export MC_CONFIG_DIR=/tmp/mc-docs-init
+
+# Wait for minio to be ready (up to 60 seconds)
+i=0
+while ! mc ready local 2>/dev/null; do
+  i=$((i+1))
+  if [ "$i" -ge 60 ]; then
+    echo "minio-initialize: timed out waiting for minio to be ready" >&2
+    exit 1
+  fi
+  sleep 1
+done
+
+MINIO_ROOT_USER="$(cat /run/secrets/minio_ru)"
+MINIO_ROOT_PASSWORD="$(cat /run/secrets/minio_rp)"
+
+mc alias set docs http://localhost:9000 "${MINIO_ROOT_USER}" "${MINIO_ROOT_PASSWORD}"
+
+# Idempotent: skip if bucket already exists
+if mc ls docs/docs-media-storage > /dev/null 2>&1; then
+  echo "minio-initialize: bucket 'docs-media-storage' already exists, skipping"
+  exit 0
+fi
+
+echo "minio-initialize: creating bucket 'docs-media-storage'..."
+mc mb docs/docs-media-storage
+mc version enable docs/docs-media-storage
+echo "minio-initialize: done"

nginx.conf.tmpl

@@ -47,7 +47,14 @@ server {
         try_files $uri @proxy_to_docs_backend;
     }
     
-    # Proxy auth for collaboration server
+    location /external_api {
+        try_files $uri @proxy_to_docs_backend;
+    }
+
+    # Collaboration server (yhub). Only the routes below are published: it also
+    # serves prune, create-ydoc, reset-connections, migrate, restore-ydoc and
+    # reset-ydoc, which the backend calls in-stack and which must not be
+    # reachable from the outside.
     location /collaboration/ws/ {
         # Ensure WebSocket upgrade
         proxy_http_version 1.1;
@@ -55,7 +62,7 @@ server {
         proxy_set_header Connection "Upgrade";
 
         # Collaboration server
-        proxy_pass http://y-provider:4444;
+        proxy_pass http://{{ env "STACK_NAME" }}_yhub:3002;
 
         # Set appropriate timeout for WebSocket
         proxy_read_timeout 86400;
@@ -67,9 +74,11 @@ server {
         proxy_set_header Host $host;
     }
 
-    location  /collaboration/api/ {
-        # Collaboration server
-        proxy_pass http://y-provider:4444;
+    # Document routes (http fallback, history) and the public keys of yhub
+    location ~ ^/collaboration/(ydoc|activity|changeset|rollback|jwks)/ {
+        proxy_pass http://{{ env "STACK_NAME" }}_yhub:3002;
+        proxy_set_header X-Forwarded-Proto https;
+        proxy_set_header Origin $http_origin;
         proxy_set_header Host $host;
     }
 

pg_backup.sh

@@ -3,14 +3,20 @@
 set -e
 
 BACKUP_FILE='/var/lib/postgresql/data/pgdata/backup.sql'
+# database of the collaboration server (yhub), since v6
+YHUB_DB='yhub'
+YHUB_BACKUP_FILE='/var/lib/postgresql/data/pgdata/backup-yhub.sql'
 
 function backup {
   export PGPASSWORD=$(cat $POSTGRES_PASSWORD_FILE)
   pg_dump -U ${POSTGRES_USER} ${POSTGRES_DB} > $BACKUP_FILE
+  if psql -U ${POSTGRES_USER} -d postgres -tAc "SELECT 1 FROM pg_database WHERE datname='${YHUB_DB}'" | grep -q 1; then
+    pg_dump -U ${POSTGRES_USER} ${YHUB_DB} > $YHUB_BACKUP_FILE
+  fi
 }
 
 function restore {
-    cd /var/lib/postgresql/data/
+    cd /var/lib/postgresql/data/pgdata/
     restore_config(){
         # Restore allowed connections
         cat pg_hba.conf.bak > pg_hba.conf
@@ -27,6 +33,12 @@ function restore {
     createdb -U ${POSTGRES_USER} ${POSTGRES_DB}
     psql -U ${POSTGRES_USER} -d ${POSTGRES_DB} -1 -f $BACKUP_FILE
 
+    if [ -f $YHUB_BACKUP_FILE ]; then
+        psql -U ${POSTGRES_USER} -d postgres -c "DROP DATABASE IF EXISTS ${YHUB_DB} WITH (FORCE);"
+        createdb -U ${POSTGRES_USER} ${YHUB_DB}
+        psql -U ${POSTGRES_USER} -d ${YHUB_DB} -1 -f $YHUB_BACKUP_FILE
+    fi
+
     trap - EXIT INT TERM
     restore_config
 }

release/0.2.6+v4.5.0

@@ -0,0 +1,4 @@
+upgraded to v4.5.0, and also switched from postgres:16 to pgautoupgrade/pgautoupgrade:18-bookworm
+for automatic major version upgrades
+
+no actions by operator should be necessary

release/next

@@ -0,0 +1,22 @@
+⚠️ Major upgrade to Docs v6: the collaboration server is replaced by the new `yhub`
+service, and existing documents have to be migrated to it, or they open EMPTY.
+
+Before deploying, generate and insert two RSA keys:
+
+  openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out jwt.pem
+  openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out yhub-jwt.pem
+  abra app secret insert <app-name> jwt_pk v1 jwt.pem -f
+  abra app secret insert <app-name> yhub_jwt_pk v1 yhub-jwt.pem -f
+
+and add to your app config:
+
+  SECRET_JWT_PK_VERSION=v1 # generate=false
+  SECRET_YHUB_JWT_PK_VERSION=v1 # generate=false
+
+`SECRET_COLLAB_SS_VERSION` and the `collab_ss` secret are no longer used.
+
+After deploying (SOFT_MIGRATION is on by default), migrate all documents:
+
+  abra app cmd <app-name> backend migrate_documents
+
+Only then may you set SOFT_MIGRATION=false. See the README for details.

renovate.json

@@ -0,0 +1,6 @@
+{
+  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
+  "extends": [
+    "config:recommended"
+  ]
+}