compose: Switch to host-mode port publishing by default #88

mirsal · 2026-02-11T04:11:14Z

mirsal commented

2026-02-11 04:11:14 +00:00

By default, swarm services use ingress mode port publishing, which is
not ideal for traefik (it breaks IPv6 ingress and there is no need to
load-balance traffic between multiple traefik instances or to route it
from multiple swarm nodes)

This PR switches traefik's port publishing mode to host for all of
its exposed ports as well as:

change traefik's update order to stop-first (there cannot be multiple
containers exposing the same port when using host-mode publishing)
use endpoint_mode: dnsrr instead of the default vip
remove all overrides from compose.host.yml, leaving the file empty
for backwards compatibility

/!\ This is a breaking change

Closes: #52

I have deployed and tested my changes
I have added a release note entry

By default, swarm services use ingress mode port publishing, which is not ideal for traefik (it breaks IPv6 ingress and there is no need to load-balance traffic between multiple traefik instances or to route it from multiple swarm nodes) This PR switches traefik's port publishing mode to `host` for all of its exposed ports as well as: * change traefik's update order to stop-first (there cannot be multiple containers exposing the same port when using host-mode publishing) * use `endpoint_mode: dnsrr` instead of the default `vip` * remove all overrides from `compose.host.yml`, leaving the file empty for backwards compatibility /!\ This is a breaking change Closes: #52 * [x] I have deployed and tested my changes * [x] I have added a [release note entry](https://docs.coopcloud.tech/maintainers/upgrade/#creating-new-release-notes)

❤️ 2

mirsal added 1 commit 2026-02-11 04:11:15 +00:00

compose: Switch to host-mode port publishing by default

continuous-integration/drone/pr Build is failing

Details

4e22350410

By default, swarm services use ingress mode port publishing, which is
not ideal for traefik (it breaks IPv6 ingress and there is no need to
load-balance traffic between multiple traefik instances or to route it
from multiple swarm nodes)

This commit switches traefik's port publishing mode to `host` for all of
its exposed ports as well as:

 * change traefik's update order to stop-first (there cannot be multiple
   containers exposing the same port when using host-mode publishing)
 * use `endpoint_mode: dnsrr` instead of the default `vip`
 * remove all overrides from `compose.host.yml`, leaving the file empty
   for backwards compatibility

/!\ This is a breaking change

Closes: #52

mirsal requested review from Apfelwurm 2026-02-11 04:11:15 +00:00

mirsal requested review from decentral1se 2026-02-11 04:11:15 +00:00

Apfelwurm commented

2026-02-11 18:25:55 +00:00

looks good to me, but maybe we should also add a compose.no-host.yml that overwrites endpoint-mode to vip and the mode of the ports 80/443 to ingress or do you think we don't need that? :)

looks good to me, but maybe we should also add a `compose.no-host.yml` that overwrites `endpoint-mode` to `vip` and the `mode` of the `port`s 80/443 to `ingress` or do you think we don't need that? :)

👀 1

dannygroenewegen commented

2026-02-11 18:49:41 +00:00

First-time contributor

I had Traefik deployed with compose.host.yml and compose.nextcloud-talk-hpb.yml. Upgrading to this PR worked mostly ok, except for Nextcloud Talk ports:

From the nextcloud readme:

Due to a bug in compose that deletes duplacted ports without checking for the protocol

I had previously run
docker service update --publish-add published=3478,target=3478,protocol=udp traefik_XXX_XXX_app

After upgrading to this PR, docker service inspect traefik_XXX_XXX_app | grep 3478 -a2 no longer showed the UDP binding (I believe this also happens when redeploying the current version, so nothing new), but I couldn't add the UDP port binding again; it didn't work.

I noticed docker service inspect traefik_XXX_XXX_app still showed a vip mode under EndpointSpec with ingress mode for 3478. I guess this manual --publish-add was still hanging somewhere and removed it:
docker service update --publish-rm 3478/udp traefik_XXX_XXX_app

And then I could add the UDP port again by specifying host mode:
docker service update --publish-add published=3478,target=3478,protocol=udp,mode=host traefik_XXX_XXX_app

I had Traefik deployed with compose.host.yml and compose.nextcloud-talk-hpb.yml. Upgrading to this PR worked mostly ok, except for Nextcloud Talk ports: From the nextcloud readme: > Due to a bug in compose that deletes duplacted ports without checking for the protocol I had previously run `docker service update --publish-add published=3478,target=3478,protocol=udp traefik_XXX_XXX_app` After upgrading to this PR, `docker service inspect traefik_XXX_XXX_app | grep 3478 -a2` no longer showed the UDP binding (I believe this also happens when redeploying the current version, so nothing new), but I couldn't add the UDP port binding again; it didn't work. I noticed `docker service inspect traefik_XXX_XXX_app` still showed a vip mode under EndpointSpec with ingress mode for 3478. I guess this manual --publish-add was still hanging somewhere and removed it: `docker service update --publish-rm 3478/udp traefik_XXX_XXX_app` And then I could add the UDP port again by specifying host mode: `docker service update --publish-add published=3478,target=3478,protocol=udp,mode=host traefik_XXX_XXX_app`

❤️ 3

decentral1se commented

2026-02-11 19:10:59 +00:00

LGTM! tysm @mirsal ❤️‍🔥

Testers welcome! Great stuff @dannygroenewegen!

LGTM! tysm @mirsal ❤️‍🔥 Testers welcome! Great stuff @dannygroenewegen!

Apfelwurm commented

2026-02-11 19:11:25 +00:00

I had Traefik deployed with compose.host.yml and compose.nextcloud-talk-hpb.yml. Upgrading to this PR worked mostly ok, except for Nextcloud Talk ports:

From the nextcloud readme:

Due to a bug in compose that deletes duplacted ports without checking for the protocol

Thanks a lot for testing this, since this will hit us as soon as we make the changes as well.
this bug is so bad and i think we can't do much about it, since currently compose is still broken when handling the port merging :'D @p4u1 and i thought about forking the compose library, to get it under control. Not sure if we ever made the PR to the upstream library? (or was there one and it's just not getting merged?)

> I had Traefik deployed with compose.host.yml and compose.nextcloud-talk-hpb.yml. Upgrading to this PR worked mostly ok, except for Nextcloud Talk ports: > > From the nextcloud readme: > > Due to a bug in compose that deletes duplacted ports without checking for the protocol Thanks a lot for testing this, since this will hit us as soon as we make the changes as well. this bug is so bad and i think we can't do much about it, since currently compose is still broken when handling the port merging :'D @p4u1 and i thought about forking the compose library, to get it under control. Not sure if we ever made the PR to the upstream library? (or was there one and it's just not getting merged?)

mirsal added 1 commit 2026-02-11 20:23:02 +00:00

compose: Allow overriding traefik's endpoint_mode back to vip

continuous-integration/drone/pr Build is failing

Details

41ce894cc1

mirsal commented

2026-02-11 20:24:12 +00:00

maybe we should also add a compose.no-host.yml that overwrites endpoint-mode to vip and the mode of the ports 80/443 to ingress or do you think we don't need that? :)

I intentionally left that part out originally because I am not sure that the publish mode would merge properly into existing port definitions.

Additionally, there's a big gotcha because setting the endpoint_mode to vip bars from using any of the compose overrides which add published ports to the traefik service.

I'm adding compose.no-host.yml as a separate commit, so that we can decide whether to include it or not (if we do, then I'll squash them)

See: 41ce894cc1

> maybe we should also add a `compose.no-host.yml` that overwrites `endpoint-mode` to `vip` and the `mode` of the `port`s 80/443 to `ingress` or do you think we don't need that? :) I intentionally left that part out originally because I am not sure that the publish mode would merge properly into existing port definitions. Additionally, there's a big gotcha because setting the endpoint_mode to vip bars from using any of the compose overrides which add published ports to the traefik service. I'm adding compose.no-host.yml as a separate commit, so that we can decide whether to include it or not (if we do, then I'll squash them) See: 41ce894cc15cbf229a92ada0f934dc87f1c087b8

❤️ 1

mirsal commented

2026-02-11 20:28:20 +00:00

I had Traefik deployed with compose.host.yml and compose.nextcloud-talk-hpb.yml. Upgrading to this PR worked mostly ok, except for Nextcloud Talk ports

Thanks a lot for testing this!

I am not sure about what to do about that bug, maybe documenting the issue in the release notes is enough?

afaict, compose.mumble.yml is also affected

> I had Traefik deployed with compose.host.yml and compose.nextcloud-talk-hpb.yml. Upgrading to this PR worked mostly ok, except for Nextcloud Talk ports Thanks a lot for testing this! I am not sure about what to do about that bug, maybe documenting the issue in the release notes is enough? afaict, `compose.mumble.yml` is also affected

👍 3

p4u1 commented

2026-02-11 21:30:06 +00:00

The nextcloud talk bug will be fixed with toolshed/abra#741

The nextcloud talk bug will be fixed with https://git.coopcloud.tech/toolshed/abra/pulls/741

❤️ 3

decentral1se requested review from p4u1 2026-02-12 08:19:03 +00:00

decentral1se requested review from javielico 2026-02-12 08:19:15 +00:00

mirsal changed title from ~~WIP: compose: Switch to host-mode port publishing by default~~ to compose: Switch to host-mode port publishing by default

2026-02-12 08:29:37 +00:00

Apfelwurm commented

2026-02-12 14:13:17 +00:00

I intentionally left that part out originally because I am not sure that the publish mode would merge properly into existing port definitions.

I already thought you had a reason for that, but wanted to make sure :D

Additionally, there's a big gotcha because setting the endpoint_mode to vip bars from using any of the compose overrides which add published ports to the traefik service.

i don't really understand what you mean by that, since this worked as of right now or didn't it? To be honest, i still have a bit of a hard time to fully wrap my head around the different concepts in swarm networking.

We don't inevitably need to add the no-host override yml from my pov, i just thought this might leave the door open for the few cases someone maybe needs it and we don't know yet, but i'm fine to go without it :)

> I intentionally left that part out originally because I am not sure that the publish mode would merge properly into existing port definitions. I already thought you had a reason for that, but wanted to make sure :D > Additionally, there's a big gotcha because setting the endpoint_mode to vip bars from using any of the compose overrides which add published ports to the traefik service. i don't really understand what you mean by that, since this worked as of right now or didn't it? To be honest, i still have a bit of a hard time to fully wrap my head around the different concepts in swarm networking. We don't inevitably need to add the no-host override yml from my pov, i just thought this might leave the door open for the few cases someone maybe needs it and we don't know yet, but i'm fine to go without it :)

mirsal commented

2026-02-12 18:00:21 +00:00

Additionally, there's a big gotcha because setting the endpoint_mode to vip bars from using any of the compose overrides which add published ports to the traefik service.

i don't really understand what you mean by that, since this worked as of right now or didn't it? To be honest, i still have a bit of a hard time to fully wrap my head around the different concepts in swarm networking.

Basically, if someone uses:

COMPOSE_FILE=$COMPOSE_FILE:compose.no-host.yml
COMPOSE_FILE=$COMPOSE_FILE:compose.gitea.yml

then port 2222 will still be published in host mode, which would make the switch to ingress-mode pointless because our override files (compose.gitea.yml in this case) use host-mode port publishing anyway.

That's the reason why I added the big fat warning in .env and didn't switch update_config back to start-first in compose.no-host.yml (unfortunately, there are other gun-foots when mixing ingress and host mode port publishing)

We don't inevitably need to add the no-host override yml from my pov, i just thought this might leave the door open for the few cases someone maybe needs it and we don't know yet, but i'm fine to go without it :)

I think having it is ok, it's just a bit confusing.
It's fine as long as users don't expect it to work intuitively with overrides.

> > Additionally, there's a big gotcha because setting the endpoint_mode to vip bars from using any of the compose overrides which add published ports to the traefik service. > > i don't really understand what you mean by that, since this worked as of right now or didn't it? To be honest, i still have a bit of a hard time to fully wrap my head around the different concepts in swarm networking. Basically, if someone uses: ```sh COMPOSE_FILE=$COMPOSE_FILE:compose.no-host.yml COMPOSE_FILE=$COMPOSE_FILE:compose.gitea.yml ``` then port 2222 will still be published in host mode, which would make the switch to ingress-mode pointless because our override files (compose.gitea.yml in this case) use host-mode port publishing anyway. That's the reason why I added the big fat warning in `.env` and didn't switch `update_config` back to `start-first` in `compose.no-host.yml` (unfortunately, there are other gun-foots when mixing ingress and host mode port publishing) > We don't inevitably need to add the no-host override yml from my pov, i just thought this might leave the door open for the few cases someone maybe needs it and we don't know yet, but i'm fine to go without it :) I think having it is ok, it's just a bit confusing. It's fine as long as users don't expect it to work intuitively with overrides.

p4u1 approved these changes 2026-02-12 20:38:48 +00:00

decentral1se approved these changes 2026-02-13 08:52:50 +00:00

decentral1se left a comment

🥇