coop-cloud/matrix-synapse
3
0
Fork 1
Code Issues 6 Pull Requests 6 Releases Activity

Support Matrix 2.0 and Element X #57

New Issue
Open
opened 2026-03-27 12:08:41 +00:00 by simon · 9 comments
simon commented 2026-03-27 12:08:41 +00:00
Owner
Copy Link
Copy Source

Recipe needs some upgrade to support element call, matrix authentication service, qr code base login etc.

Recipe needs some upgrade to support element call, matrix authentication service, qr code base login etc.
🚀 1
simon self-assigned this 2026-03-27 12:08:44 +00:00
simon commented 2026-03-27 12:52:12 +00:00
Author
Owner
Copy Link
Copy Source

Currently we have those containers:

  • compose.yml
    • web: nginx
    • app: synapse
    • db: pgautoupgrade
    • admin: synapse-admin
    • element-web
    • bridges

For 2.0 we're missing:

Sources

maaaaybe later:

Questions

  • Keep element-web and matrix-synapse separate recipes y/n?
  • If not - put element-call, livekit etc. into element-web y/n?
  • rename element-web to just "element" y/n?
Currently we have those containers: - compose.yml - web: nginx - app: synapse - db: pgautoupgrade - admin: synapse-admin - [element-web](https://git.coopcloud.tech/coop-cloud/element-web/src/branch/main/compose.yml) - **bridges** For 2.0 we're missing: - ~~redis?~~ - ~~synapse-workers?~~ [probably not](https://matrix-org.github.io/synapse/latest/workers.html#scaling-synapse-via-workers) - mas: matrix-authentication-service - [element-call ](https://github.com/element-hq/element-call/blob/livekit/docs/self-hosting.md) - -> in [element-web recipe](https://git.coopcloud.tech/coop-cloud/element-web/)? - includes livekit & livekit-jwt containers - [Docs](https://github.com/element-hq/element-call/blob/livekit/docs/self-hosting.md) ## Sources - https://github.com/element-hq/element-docker-demo/blob/main/compose.yml - https://github.com/spantaleev/matrix-docker-ansible-deploy - Backlink to old internal issue for LIT folks [here](https://git.local-it.org/KolliCloud-intern/development/issues/282) ## maaaaybe later: - https://matrix-org.github.io/matrix-hookshot/latest/hookshot.html # Questions - [ ] Keep `element-web` and `matrix-synapse` separate recipes y/n? - [ ] If not - put `element-call`, livekit etc. into `element-web` y/n? - [ ] rename element-web to just "element" y/n?
❤️ 1
val commented 2026-03-30 14:48:15 +00:00
Owner
Copy Link
Copy Source

Hey,

thanks for the initiative. I like the idea to restructure the recipes, with the following goals:

  1. improve the maintainability (this means: don't having to fear that small changes bring the whole complex recipe to explode, making it easier to collaborate)
  2. facilitate usage of server-alternatives to matrix-synapse

So I would vote in favor of seperating in three recipes:

1. matrix-synapse

  • Only what is necessary for synapse-server (+ admin?)
  • -> other servers, seperate recipes
  • Open question:
    • Does mas belong here in this structure? How deeply is it coupled to synapse? In my proposed logic it would go to "matrix-extensions", although it feels like it directly starts getting messy again :/

2. element

  • element-web
  • And as optional compose its existing extension: element-call
  • -> other clients, seperate recipes

3. matrix-extensions

  • Preferibly one recipe for a lot of stuff that can be (potentially) used with several servers, with optional compose-files.

  • One because of standardization for maintainers and operators, no extensive duplication of envs, secrets.

  • I hope so to build a little team of maintainers, which could support each other instead of splitting up in several recipes

  • unites:

  • Open questions:

    • what is "base" compose container of matrix-extensions-recipe? Simply no? But "base-envs" & secrets
    • Do bridges technically work separated from homeserver?
    • Is this the root of a new "messiness" because it combines a lot of better sperated services?

So as to your conrete questions @simon, my vote:

  • Keep element-web and matrix-synapse separate recipes y/n?

yes

  • If not - put element-call, livekit etc. into element-web y/n?

yes, put element-call into element (although it directly starts getting messy probably, because you have to configure matrix-synapse in a certain way to be able to use element-call, right?)

  • rename element-web to just "element" y/n?

either rename, or create a new recipe "element" to not blow existing deployments.

docs against the new messiness

as already mentioned, this structure will probably create a new "messiness", as it would put things like mas in the extensions-recipe, although you will probably want to use it in every deployment of matrix-synapse. And you would need to change configs in matrix-synapse to use element-call.
But at the moment I would vote for solving this by good documentation.

The alternative would be keeping and extending a "huge" matrix-synapse recipe, and probably do the same for tuwunel, continuwuity ... or to getting lost in dozens of recipes...

What do you think?

Hey, thanks for the initiative. I like the idea to restructure the recipes, with the following goals: 1. improve the maintainability (this means: don't having to fear that small changes bring the whole complex recipe to explode, making it easier to collaborate) 2. facilitate usage of server-alternatives to matrix-synapse So I would vote in favor of seperating in three recipes: **1. `matrix-synapse`** - Only what is necessary for synapse-server (+ admin?) - -> other servers, seperate recipes - **Open question:** - [ ] Does `mas` belong here in this structure? How deeply is it coupled to synapse? In my proposed logic it would go to "matrix-extensions", although it feels like it directly starts getting messy again :/ **2. `element`** - `element-web` - And as optional compose its existing extension: `element-call` - -> other clients, seperate recipes **3. `matrix-extensions`** - Preferibly one recipe for a lot of stuff that can be (potentially) used with several servers, with optional compose-files. - One because of standardization for maintainers and operators, no extensive duplication of envs, secrets. - I hope so to build a little team of maintainers, which could support each other instead of splitting up in several recipes - unites: - `mas` - bridges (as of today: signal, telegram, discord) - bots ([maubot](https://git.coopcloud.tech/coop-cloud/maubot), [draupnir](https://git.coopcloud.tech/coop-cloud/draupnir), later: [hookshot](https://matrix-org.github.io/matrix-hookshot/latest/hookshot.html)) - **Open questions:** - [ ] what is "base" compose container of matrix-extensions-recipe? Simply no? But "base-envs" & secrets - [ ] Do bridges technically work separated from homeserver? - [ ] Is this the root of a new "messiness" because it combines a lot of better sperated services? So as to your conrete questions @simon, my vote: > - [ ] Keep `element-web` and `matrix-synapse` separate recipes y/n? yes > - [ ] If not - put `element-call`, livekit etc. into `element-web` y/n? yes, put element-call into element (although it directly starts getting messy probably, because you have to configure matrix-synapse in a certain way to be able to use element-call, right?) > - [ ] rename element-web to just "element" y/n? either rename, or create a new recipe "element" to not blow existing deployments. ## docs against the new messiness as already mentioned, this structure will probably create a new "messiness", as it would put things like `mas` in the extensions-recipe, although you will probably want to use it in every deployment of matrix-synapse. And you would need to change configs in `matrix-synapse` to use element-call. But at the moment I would vote for solving this by good documentation. The alternative would be keeping and extending a "huge" matrix-synapse recipe, and probably do the same for `tuwunel`, `continuwuity` ... or to getting lost in dozens of recipes... What do you think?
❤️ 1
moritz commented 2026-03-31 00:05:52 +00:00
Owner
Copy Link
Copy Source

I like the idea of splitting the extensions and the core. But I think the biggest problem to solve is how to ensure the compatibility between different versions. My matrix update today unfortunately broke the signal/telegram bridges, but I'm not able to fix them, as I'm not deploying any of these bridges. Maybe the extension recipe could specify the last stable matrix recipe version it was still working with. Then people can see if an update might break the extensions and if the extensions need to be tested and upgraded for a the current matrix version.

I like the idea of splitting the extensions and the core. But I think the biggest problem to solve is how to ensure the compatibility between different versions. My matrix update today unfortunately broke the signal/telegram bridges, but I'm not able to fix them, as I'm not deploying any of these bridges. Maybe the extension recipe could specify the last stable matrix recipe version it was still working with. Then people can see if an update might break the extensions and if the extensions need to be tested and upgraded for a the current matrix version.
👀 1
simon commented 2026-03-31 07:13:21 +00:00
Author
Owner
Copy Link
Copy Source

Yes, I also like the proposed setup with matrix-synapse matrix-extensions and element.

I feel like the mas is so close to matrix itself that I'd put it in the core recipe. We can always allow to switch those things off with separate compose.yml files

also the extension-recipe with all the bridges, potentially maubot (?) etc. will already feature a number of extensions, too

Yes, I also like the proposed setup with matrix-synapse matrix-extensions and element. I feel like the mas is so close to matrix itself that I'd put it in the core recipe. We can always allow to switch those things off with separate compose.yml files also the extension-recipe with all the bridges, potentially maubot (?) etc. will already feature a number of extensions, too
👀 1
decentral1se commented 2026-03-31 12:58:50 +00:00
Owner
Copy Link
Copy Source

Off-topic but @p4u1, this makes me think of your "sub recipes" ideas 🤔

Off-topic but @p4u1, this makes me think of your "sub recipes" ideas 🤔
👍 1
val commented 2026-03-31 18:12:40 +00:00
Owner
Copy Link
Copy Source

Off-topic but @p4u1, this makes me think of your "sub recipes" ideas 🤔

Don't know what it's exactly about, but sounds like we could need this here ;)

I haven't thought about the problem, that we give up some kind of stability/guarantee that the various components work together. But nonetheless it seems a good objective to me, to "reduce" the matrix-synapse recipe a bit and to gather together various extensions of the ecosystem in one extra recipe.

The matrix-extensions should probably at least mention in the docs "last known matrix-synapse version".

Another thought I had would be to provide some kind of "meta-configs of a known-to-work set of recipes" with alakazam if it's somehow adopted in abra. This would allow to maintain working/tested combinations of recipe-versions nicely, without having to put everything in one recipe.

In any case we probably should make the image-versions flexible via env-vars in matrix-extensions.

As to mas @simon seems a good plan to me, including it in matrix-synapse for the moment.

> Off-topic but @p4u1, this makes me think of your "sub recipes" ideas 🤔 Don't know what it's exactly about, but sounds like we could need this here ;) I haven't thought about the problem, that we give up some kind of stability/guarantee that the various components work together. But nonetheless it seems a good objective to me, to "reduce" the matrix-synapse recipe a bit and to gather together various extensions of the ecosystem in one extra recipe. The `matrix-extensions` should probably at least mention in the docs "last known matrix-synapse version". Another thought I had would be to provide some kind of "meta-configs of a known-to-work set of recipes" with `alakazam` if it's somehow adopted in `abra`. This would allow to maintain working/tested combinations of recipe-versions nicely, without having to put everything in one recipe. In any case we probably should make the image-versions flexible via env-vars in `matrix-extensions`. As to `mas` @simon seems a good plan to me, including it in matrix-synapse for the moment.
simon commented 2026-04-01 12:18:18 +00:00
Author
Owner
Copy Link
Copy Source

Another thing aside regarding the general recipe layout - the env list in the compose.yml is a bit frightening. I imagine such things can keep people from touching stuff in a recipe

matrix-synapse/compose.yml Lines 49 to 93 in 8a7978b388
- ALLOWED_LIFETIME_MAX
- ALLOW_PUBLIC_ROOMS_FEDERATION
- AUTO_JOIN_ROOM
- AUTO_JOIN_ROOM_ENABLED
- DISABLE_FEDERATION
- DOMAIN
- ENABLE_3PID_LOOKUP
- ENABLE_ALLOWLIST
- ENABLE_REGISTRATION
- REGISTRATION_REQUIRES_TOKEN
- ENCRYPTED_BY_DEFAULT
- OLD_SIGNING_KEY
- OLD_SIGNING_KEY_ID
- OLD_SIGNING_KEY_EXPIRES
- USER_DIRECTORY_ENABLED=${USER_DIRECTORY_ENABLED:-true}
- USER_DIRECTORY_SEARCH_ALL_USERS=${USER_DIRECTORY_SEARCH_ALL_USERS:-true}
- USER_DIRECTORY_PREFER_LOCAL_USERS=${USER_DIRECTORY_PREFER_LOCAL_USERS:-true}
- USER_DIRECTORY_SHOW_LOCKED_USERS=${USER_DIRECTORY_SHOW_LOCKED_USERS:-false}
- FEDERATION_ALLOWLIST
- REQUIRE_AUTH_FOR_PROFILE_REQUESTS=${REQUIRE_AUTH_FOR_PROFILE_REQUESTS:-false}
- LIMIT_PROFILE_REQUESTS_TO_USERS_WHO_SHARE_ROOMS=${LIMIT_PROFILE_REQUESTS_TO_USERS_WHO_SHARE_ROOMS:-false}
- DELETE_STALE_DEVICES_AFTER
- SESSION_LIFETIME
- TRACK_PUPPETED_USER_IPS=${TRACK_PUPPETED_USER_IPS:-false}
- LETSENCRYPT_HOST=${DOMAIN}
- MEDIA_RETENTION_LOCAL_LIFETIME
- MEDIA_RETENTION_REMOTE_LIFETIME
- PASSWORD_LOGIN_ENABLED
- REDACTION_RETENTION_PERIOD
- RETENTION_MAX_LIFETIME
- ROOT_LOG_LEVEL
- SERVE_SERVER_WELLKNOWN
- SQL_LOG_LEVEL
- STACK_NAME
- SYNAPSE_ADMIN_EMAIL
- SYNAPSE_REPORT_STATS=no
- SYNAPSE_SERVER_NAME=${DOMAIN}
- USER_IPS_MAX_AGE
- VIRTUAL_HOST=${DOMAIN}
- VIRTUAL_PORT=8008
- LOGIN_LIMIT_IP_PER_SECOND=${LOGIN_LIMIT_IP_PER_SECOND:-0.003}
- LOGIN_LIMIT_IP_BURST=${LOGIN_LIMIT_IP_BURST:-5}
- LOGIN_LIMIT_ACCOUNT_PER_SECOND=${LOGIN_LIMIT_ACCOUNT_PER_SECOND:-0.003}
- LOGIN_LIMIT_ACCOUNT_BURST=${LOGIN_LIMIT_ACCOUNT_BURST:-5}
- WEB_CLIENT_LOCATION

This also touches the point of detailed configurations of such config files. I've wondered about different approaches for supplying configs to apps aside from only .envs - e.g. just provide a path for a configmap and edit configs in the file itself.

Config files may change between versions, sometimes they need to be adapted and with all the templating to fill in values via envs, this can get quite complicated. And we have to deal with all the different escaping strategies for parentheses, quotes etc.

but I'm also sorta leaving the actual topic here 😆

Another thing aside regarding the general recipe layout - the env list in the compose.yml is a bit frightening. I imagine such things can keep people from touching stuff in a recipe https://git.coopcloud.tech/coop-cloud/matrix-synapse/src/commit/8a7978b388322d26a40b1ac83e64d4c8ee9b72b3/compose.yml#L49-L93 This also touches the point of detailed configurations of such config files. I've wondered about different approaches for supplying configs to apps aside from only .envs - e.g. just provide a path for a configmap and edit configs in the file itself. Config files may change between versions, sometimes they need to be adapted and with all the templating to fill in values via envs, this can get quite complicated. And we have to deal with all the different escaping strategies for parentheses, quotes etc. but I'm also sorta leaving the actual topic here 😆
😆 1
simon commented 2026-04-09 10:12:32 +00:00
Author
Owner
Copy Link
Copy Source

Soo, mas service integration is done, rfc #58

QR-Code login looked easy on the surface but revealed deeper problems:

  • Looks like we have a problem with our $STACK_NAME convention with all the underscores, e.g. in the service matrix_dev_kolli_cloud_mas
  • When Synapse (Python) loads MAS OIDC metadata, it builds a URL from matrix_authentication_service.endpoint and runs IDNA encoding on the hostname.
  • Underscores (_) are not allowed in IDNA labels → idna.core.InvalidCodepoint → AuthMetadataServlet returns 500 → clients (e.g. Element X) cannot use MAS/OIDC.
  • I manually changed the config to the IP of the service and at least got a step further before I ran into a different error.

So I wonder how we should solve this..
I guess that's a similar reason behind all these kinds of (ignoreable) error logs in various recipes?

�2026-04-08T13:10:33.363248980Z      psycopg2.OperationalError: could not translate host name "matrix_dev_kolli_cloud_db" to address: Name or service not know

@decentral1se have you got any ideas? Do we really need to change the stack name naming convention?

This is also relevant I guess.

  • especially the reference to RFC-1123:

    The syntax of a legal Internet host name was specified in RFC-952 [DNS:4].

  • and RFC-952

    A "name" (Net, Host, Gateway, or Domain name) is a text string up
    to 24 characters drawn from the alphabet (A-Z), digits (0-9), minus
    sign (-), and period (.).

Soo, `mas` service integration is done, rfc #58 QR-Code login looked easy on the surface but revealed deeper problems: - Looks like we have a problem with our `$STACK_NAME` convention with all the underscores, e.g. in the service `matrix_dev_kolli_cloud_mas` - When **Synapse** (Python) loads **MAS OIDC metadata**, it builds a URL from `matrix_authentication_service.endpoint` and runs **IDNA encoding** on the hostname. - **Underscores (`_`) are not allowed in IDNA labels** → `idna.core.InvalidCodepoint` → **`AuthMetadataServlet` returns 500** → clients (e.g. Element X) **cannot use MAS/OIDC**. - I manually changed the config to the IP of the service and at least got a step further before I ran into a different error. So I wonder how we should solve this.. I guess that's a similar reason behind all these kinds of (ignoreable) error logs in various recipes? ``` �2026-04-08T13:10:33.363248980Z psycopg2.OperationalError: could not translate host name "matrix_dev_kolli_cloud_db" to address: Name or service not know ``` @decentral1se have you got any ideas? Do we really need to change the stack name naming convention? [This](https://stackoverflow.com/questions/2180465/can-domain-name-subdomains-have-an-underscore-in-it) is also relevant I guess. - especially the reference to [RFC-1123](https://www.ietf.org/rfc/rfc1123.txt): > The syntax of a legal Internet host name was specified in RFC-952 [DNS:4]. - and [RFC-952](https://www.rfc-editor.org/rfc/rfc952.txt) > A "name" (Net, Host, Gateway, or Domain name) is a text string up to 24 characters drawn from the alphabet (A-Z), digits (0-9), minus sign (-), and period (.).
decentral1se commented 2026-04-09 14:15:21 +00:00
Owner
Copy Link
Copy Source

Gnarly! I believe you can hack it and just do STACK_NAME=IReallyKnowWhatImDoing in your .env? Test with care 🙃 But yeh, if you really need to get rid of _, then you still get caught with IReallyKnowWhatImDoing_db or whatever. You could patch this in abra but it sounds like a recipe for disaster 😆 Good luck!

Gnarly! I believe you can hack it and just do `STACK_NAME=IReallyKnowWhatImDoing` in your `.env`? Test with care 🙃 But yeh, if you really need to get rid of `_`, then you still get caught with `IReallyKnowWhatImDoing_db` or whatever. You could patch this in `abra` but it sounds like a recipe for disaster 😆 Good luck!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No labels
Milestone
No items
No Milestone
Assignees
Clear assignees
3wordchant aadil (Aadil Ayub) abra-bot (Abra Bot) ammaratef45 amras (Sarma) Apfelwurm appletalk arjan basebuilder BornDeleuze Brooke carla cas (Cassowary) codegod100 coopcloud cyrnel decentral1se (d1) dede devydave fauno (fauno) flancian Frando iexos jade (Jade Ambrose) javielico (Javielico) jjsfunhouse jmakdah2 (Jackie Makdah) joe-irving (Joe Irving) kawaiipunk (KawaiiPunk) knoflook kolaente lambdabundesverband linnealovespie (April) marlon (marlon) mayel mirsal moosemower moritz nicksellen (Nick Sellen) notplants oxaliq (sorrel) p4u1 pau pharaohgraphy (Andrew 🐦‍🔥❤️‍🔥✴️) PhiNatalie renovate-bot (Comrade Renovate Bot) ripclap rix rscmbbng sef (sef) simon sixsmith (Sixsmith) stevensting tobias trav (Trav Fryer) val (val (he/him)) vaznasty virtualboys wolcen (Chris Thompson) wykwit xynosis yksflip
No Assignees simon
4 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: coop-cloud/matrix-synapse#57