Add first-class support to co-op cloud for reverse proxies other than traefik? #686

New Issue

notplants · 2026-02-23T15:18:21Z

notplants commented

2026-02-23 15:18:21 +00:00

This issue is highly related to "Caddy as a Traefik replacement?"

Currently, most co-op cloud apps expect coop-cloud/traefik is deployed on the server. But there are two main issues:

traefik has some limitations that caddy does not have (example)
if you have a server already running another reverse proxy (such as nginx), you cannot install coop cloud without taking over the server

Using this thread to brainstorm and discuss ideas for solutions

This issue is highly related to ["Caddy as a Traefik replacement?" ](https://git.coopcloud.tech/toolshed/organising/issues/388#issuecomment-15327) Currently, most co-op cloud apps expect [coop-cloud/traefik](https://git.coopcloud.tech/coop-cloud/traefik) is deployed on the server. But there are two main issues: 1. traefik has some limitations that caddy does not have ([example](https://matrix.to/#/!DfXPgKLoYCvjHithgS:autonomic.zone/$lMdysyTYZjZeEPJSyxdtqrS9mCeB6L6ezY45_gknALs?via=autonomic.zone&via=matrix.org&via=pub.solar)) 2. if you have a server already running another reverse proxy (such as nginx), you cannot install coop cloud without taking over the server Using this thread to brainstorm and discuss ideas for solutions

🚀 1

notplants commented

2026-02-23 15:36:24 +00:00

on the topic of (1), looking more through #388 "Caddy as Traefik replacement", I'd be tempted to fully migrate to caddy instead of traefik, rather than supporting both. Curious where that issue got to or if there were blockers to fully switching over all recipes to caddy instead of traefik?

on the topic of (2), was brainstorming a bit the case of a server that already has nginx installed and thinking through some ideas based on the recent tls passthrough configuration for the bluesky-pds recipe

* nginx-shim-idea *

add an additional optional compose file to traefik recipe which mounts /etc/nginx on the host server

in this additional compose file (lets call it nginx-shim), it does the following:

it modifies the host nginx config to add tls passthrough from nginx to traefik for the domains that co-op cloud manages.

traefik could itself listen on port 8443 and the host nginx could stream whatever requests coop cloud needs to 8443 by sni, and all other requests get handled by nginx

downside: this means cc is slightly editing the host nginx config, even if its just plumbing. I could imagine times where this could create issues

* inverse-nginx-shim idea *

the inverse of the above. coop cloud traefik proxy takes over the host and binds to port 443.

the inverse-nginx-shim recipe binds /etc/nginx and watches it, and whenever a new domain is added, it asks traefik to route that domain to nginx on port 8443 using tls passthrough (very similar to what bluesky-pds recipe does, but asking traefik to passthrough requests to nginx for the needed domains)

downside: still slightly modifies the host nginx to listen on port 8443 instead of 443, and takes over binding to the host at the edge

on the topic of (1), looking more through #388 "Caddy as Traefik replacement", I'd be tempted to fully migrate to caddy instead of traefik, rather than supporting both. Curious where that issue got to or if there were blockers to fully switching over all recipes to caddy instead of traefik? on the topic of (2), was brainstorming a bit the case of a server that already has nginx installed and thinking through some ideas based on the recent tls passthrough configuration for the [bluesky-pds recipe](https://git.coopcloud.tech/coop-cloud/bluesky-pds#tls-architecture-caddy-sidecar) ------ _* nginx-shim-idea *_ add an additional optional compose file to traefik recipe which mounts /etc/nginx on the host server in this additional compose file (lets call it nginx-shim), it does the following: it modifies the host nginx config to add tls passthrough from nginx to traefik for the domains that co-op cloud manages. traefik could itself listen on port 8443 and the host nginx could stream whatever requests coop cloud needs to 8443 by sni, and all other requests get handled by nginx downside: this means cc is slightly editing the host nginx config, even if its just plumbing. I could imagine times where this could create issues ------ _* inverse-nginx-shim idea *_ the inverse of the above. coop cloud traefik proxy takes over the host and binds to port 443. the inverse-nginx-shim recipe binds /etc/nginx and watches it, and whenever a new domain is added, it asks traefik to route that domain to nginx on port 8443 using tls passthrough (very similar to what bluesky-pds recipe does, but asking traefik to passthrough requests to nginx for the needed domains) downside: still slightly modifies the host nginx to listen on port 8443 instead of 443, and takes over binding to the host at the edge

notplants commented

2026-02-23 15:51:20 +00:00

* no magic idea / documentation-only *

both nginx-shim and inverse-nginx-shim magically modify the host nginx configuration, which seems dangerous and admins might not like that

instead, we could just create documentation, explaining how to have a traefik configuration for coop cloud where it runs on port 8443, and then documentation+script that an admin can run to modify their own nginx config on the host to add streaming support to get their nginx config to pass requests to port 8443 via tls-passthrough for the domains that are managed by co-op cloud

_* no magic idea / documentation-only *_ both nginx-shim and inverse-nginx-shim magically modify the host nginx configuration, which seems dangerous and admins might not like that instead, we could just create documentation, explaining how to have a traefik configuration for coop cloud where it runs on port 8443, and then documentation+script that an admin can run to modify their own nginx config on the host to add streaming support to get their nginx config to pass requests to port 8443 via tls-passthrough for the domains that are managed by co-op cloud

mayel commented

2026-02-23 16:26:33 +00:00

tempted to fully migrate to caddy instead of traefik, rather than supporting both. Curious where that issue got to or if there were blockers to fully switching over all recipes to caddy instead of traefik?

I'd say that question would be better discussed once/if we have caddy working as an alternative and all recipes migrated/compatible with it, (as in should we now deprecated traefik support since caddy works better for all current recipes), so we can maintain backwards compatibility / progressive enhancement if that makes more sense than multi-proxy support...

> tempted to fully migrate to caddy instead of traefik, rather than supporting both. Curious where that issue got to or if there were blockers to fully switching over all recipes to caddy instead of traefik? I'd say that question would be better discussed once/if we have caddy working as an alternative _and all recipes migrated/compatible with it_, (as in should we now deprecated traefik support since caddy works better for all current recipes), so we can maintain backwards compatibility / progressive enhancement if that makes more sense than multi-proxy support...

👍 1

decentral1se commented

2026-02-23 16:57:06 +00:00

Thanks for opening!

Yes, I think the question of deprecating Traefik should be left out until we have Caddy proving itself in a few stable deployments first. Tbh I think we should probably always support Traefik because there are sooooo many deployments but that discussion is for another day. In general, I'm curious about how to make progress with more Caddy adoption because this will likely be needed for a post-swarm Co-op Cloud and dealing with zero downtime deploys, according to the uncloud.dev folx: https://github.com/psviderski/uncloud/discussions/255
Can Co-op Cloud directly support Nginx and not have to shim in/out with Traefik? Then peoply don't deploy Traefik and we facilitate a convenient way to generate configurations for an existing Nginx installation? This seems like a good starting point: nginx-proxy/nginx-proxy We can generate from the abra side or ask people to install something on the server side? Some experimentation is necessary

Thanks for opening! 1) Yes, I think the question of deprecating Traefik should be left out until we have Caddy proving itself in a few stable deployments first. Tbh I think we should probably always support Traefik because there are sooooo many deployments but that discussion is for another day. In general, I'm curious about how to make progress with more Caddy adoption because this will likely be needed for a post-swarm Co-op Cloud and dealing with zero downtime deploys, according to the uncloud.dev folx: https://github.com/psviderski/uncloud/discussions/255 2) Can Co-op Cloud directly support Nginx and not have to shim in/out with Traefik? Then peoply don't deploy Traefik and we facilitate a convenient way to generate configurations for an existing Nginx installation? This seems like a good starting point: [`nginx-proxy/nginx-proxy`](https://github.com/nginx-proxy/nginx-proxy?tab=readme-ov-file) We can generate from the `abra` side or ask people to install something on the server side? Some experimentation is necessary

👍 1

notplants commented

2026-02-28 19:12:07 +00:00

maintain backwards compatibility / progressive enhancement

hear you both on that

Can Co-op Cloud directly support Nginx and not have to shim in/out with Traefik?

a part of me also feels direct support for nginx would feel more elegant, but it does also have downsides

if we have direct support for nginx, caddy and traefik, then either:

A. recipe maintainers need to add labels to their recipes for all three of these proxies (and many won't)

B. in a sense, we are creating a sort of unified DSL for specifying reverse proxy configurations as docker labels (or treating Traefik labels as that), which different adaptors use to generate reverse proxy configs for their specific reverse proxy (traefik already has a built-in adaptor, nginx-proxy for nginx, and caddy-docker-proxy for caddy, are all ways of generating reverse proxy configs)

for common configurations (expose port 443 here) I imagine a unified DSL/labels/syntax could easily be adapted to all three reverse proxies. I imagine with apps with weird routing there could be some edge cases where things are handled differently between the three proxies and then recipe maintainers are left to fallback to maintaining specific labels for each of the three proxies (although these edge cases need to be confirmed?)

the unified reverse proxy spec is kind of a cool problem though... and feels more permacomputing in one sense (more direct)

vs. if you just officially support one proxy, then recipe maintainers just write labels for that proxy, and any another proxy is assumed to shim in/out of that vis TLS passthrough (less elegant, but less recipe maintenance)

just trying to give a break down of the problem here as I see it (maybe missing things)

> maintain backwards compatibility / progressive enhancement hear you both on that > Can Co-op Cloud directly support Nginx and not have to shim in/out with Traefik? a part of me also feels direct support for nginx would feel more elegant, but it does also have downsides if we have direct support for nginx, caddy and traefik, then either: A. recipe maintainers need to add labels to their recipes for all three of these proxies (and many won't) B. in a sense, we are creating a sort of unified DSL for specifying reverse proxy configurations as docker labels (or treating Traefik labels as that), which different adaptors use to generate reverse proxy configs for their specific reverse proxy (traefik already has a built-in adaptor, nginx-proxy for nginx, and caddy-docker-proxy for caddy, are all ways of generating reverse proxy configs) for common configurations (expose port 443 here) I imagine a unified DSL/labels/syntax could easily be adapted to all three reverse proxies. I imagine with apps with weird routing there could be some edge cases where things are handled differently between the three proxies and then recipe maintainers are left to fallback to maintaining specific labels for each of the three proxies (although these edge cases need to be confirmed?) the unified reverse proxy spec is kind of a cool problem though... and feels more permacomputing in one sense (more direct) vs. if you just officially support one proxy, then recipe maintainers just write labels for that proxy, and any another proxy is assumed to shim in/out of that vis TLS passthrough (less elegant, but less recipe maintenance) just trying to give a break down of the problem here as I see it (maybe missing things)

👀 1

decentral1se commented

2026-03-01 16:13:49 +00:00

Another data point from the uncloud folx in the category of focusing on a single proxy and having a really tight integration: https://uncloud.run/docs/concepts/ingress/overview (in this case, Caddy wins the day again 🙃).

The custom compose stanza for extending the proxy from within the recipe config is pretty wild: https://uncloud.run/docs/concepts/ingress/publishing-services#custom-caddy-configuration 🤯

Another data point from the uncloud folx in the category of focusing on a single proxy and having a really tight integration: https://uncloud.run/docs/concepts/ingress/overview (in this case, Caddy wins the day again 🙃). The custom compose stanza for extending the proxy from within the recipe config is pretty wild: https://uncloud.run/docs/concepts/ingress/publishing-services#custom-caddy-configuration 🤯

decentral1se commented

2026-03-01 16:19:41 +00:00

Also, just so we have it all in one place, the Pretty Damn Gnarly "Proxying apps outside of Co-op Cloud with Traefik?" part of our docs: https://docs.coopcloud.tech/operators/handbook/#proxying-apps-outside-of-co-op-cloud-with-traefik