Please clarify: "Mailu requires host-mode networking" #8

New Issue

amras · 2023-09-10T14:50:11Z

amras commented

2023-09-10 14:50:11 +00:00

The way I understand it, there are two options for exposing IMAP/SMTP ports.

In the host-mode variant, IMAP/SMTP ports are visible only on the host machine or via the webmail client.
External clients are unable to connect.
When exposing ports using entrypoints, IMAP/SMTP ports are visible on $WEB_DOMAIN. External clients can connect using the
client setup specified in https://$WEB_DOMAIN/admin/client. The webmail client also works without issue.

What is the rationale behind exposing the ports on the host only?
The README specifies "so that containers are aware of remote IP addresses" - but I'm unclear how that affects functionality.

IMAP configuration examples for context:

Host-mode

# mailu/compose.yml
ports:
  - target: 993
     published: 993
     mode: host

Entrypoints

# mailu/compose.yml
deploy:
  labels:
    - "traefik.tcp.routers.${STACK_NAME}-imap.entrypoints=imap-tls"                                                            
    - "traefik.tcp.routers.${STACK_NAME}-imap.service=${STACK_NAME}-imap-service"
    - "traefik.tcp.routers.${STACK_NAME}-imap.rule=HostSNI(`${WEB_DOMAIN}`)"
    - "traefik.tcp.routers.${STACK_NAME}-imap.tls.passthrough=true"
    - "traefik.tcp.services.${STACK_NAME}-imap-service.loadbalancer.server.port=993"

# traefik/compose.imap.yml
services:
  app:
    environment:
      - IMAP_ENABLED
    ports:
      - "993:993/tcp"

# traefik/traefik.yml.tmpl
{{ if eq (env "IMAP_ENABLED") "1" }}
imap-tls:
  address: ":993"
{{ end }}

The way I understand it, there are two options for exposing IMAP/SMTP ports. - In the host-mode variant, IMAP/SMTP ports are visible only on the host machine or via the webmail client. **External clients are unable to connect.** - When exposing ports using entrypoints, IMAP/SMTP ports are visible on `$WEB_DOMAIN`. External clients can connect using the client setup specified in `https://$WEB_DOMAIN/admin/client`. The webmail client also works without issue. What is the rationale behind exposing the ports on the host only? The README specifies "so that containers are aware of remote IP addresses" - but I'm unclear how that affects functionality. ## IMAP configuration examples for context: ### Host-mode ```yaml # mailu/compose.yml ports: - target: 993 published: 993 mode: host ``` ### Entrypoints ```yaml # mailu/compose.yml deploy: labels: - "traefik.tcp.routers.${STACK_NAME}-imap.entrypoints=imap-tls" - "traefik.tcp.routers.${STACK_NAME}-imap.service=${STACK_NAME}-imap-service" - "traefik.tcp.routers.${STACK_NAME}-imap.rule=HostSNI(`${WEB_DOMAIN}`)" - "traefik.tcp.routers.${STACK_NAME}-imap.tls.passthrough=true" - "traefik.tcp.services.${STACK_NAME}-imap-service.loadbalancer.server.port=993" ``` ```yaml # traefik/compose.imap.yml services: app: environment: - IMAP_ENABLED ports: - "993:993/tcp" ``` ```yaml # traefik/traefik.yml.tmpl {{ if eq (env "IMAP_ENABLED") "1" }} imap-tls: address: ":993" {{ end }} ```

3wordchant commented

2023-09-10 23:14:05 +00:00

Hey @amras, thanks so much for trying out co-op cloud, and for opening this ticket! 🙏

In the host-mode variant, IMAP/SMTP ports are visible only on the host machine or via the webmail client.
External clients are unable to connect.

We've got a few Mailu instances running at Autonomic that use "host mode", and connections from the internet to the host-mode ports do work – I wonder if there's some problematic interaction between Docker and the firewall or networking configuration on your server which is preventing external clients from being able to connect?

The intention is definitely not to restrict those ports to local-only connections, anyway.

The README specifies "so that containers are aware of remote IP addresses" - but I'm unclear how that affects functionality.

Mailu includes some IP-based rate limiting, and unless either Mailu's "front" container (or, if we were using Traefik entrypoints, Traefik itself) is in "host" mode, then docker swarm doesn't give access to the real originating IP, and all connections seem to be treated as coming from the same source IP, which in practice led to frequent accidental denial-of-service-style outages.

"Host mode" has seemed like an acceptable limitation because the main drawbacks seem to be that (1) it's not compatible with multi-node docker swarms and (2) it doesn't allow multiple Mailu or Traefik instances on the same host.

(1) seems reasonable because there are other unsolved blockers to multi-node (mainly, lack of multi-node SSL ingress in Traefik's community edition as of Traefik 2)

(2) likewise seemed like an acceptable down-side because Traefik requires HostSNI(*) to route protocols which don't do SNI (server name indication), and that includes "opportunistic TLS" SMTP – which seems to be required for mail servers:

Yes HostSNI() would work with SMTPS(465). However I don't believe that SMTPS is widely used and you also run the risk of other MTAs not supporting tls at all. So you could artificially restrict mail flow.

https://community.traefik.io/t/tcp-proxy-with-sni-support/6976

So, there's effectively a maximum of one Mailu (or SMTP-routing Traefik) instance per swarm node anyway.

I'd personally be fine with additional entrypoints in the Traefik recipe – especially if that gives any advantages for anyone over "host mode" – in that case, I wonder what would happen if a second Mailu instance were deployed on a server (currently, I think the use of host-mode ports would make abra app deploy fail, and containers fail to launch which, while not elegant, might be clearer and more deterministic than having two services with a HostSNI(*) routing rule).

Hey @amras, thanks so much for trying out co-op cloud, and for opening this ticket! 🙏 > In the host-mode variant, IMAP/SMTP ports are visible only on the host machine or via the webmail client. External clients are unable to connect. We've got a few Mailu instances running at Autonomic that use "host mode", and connections from the internet to the [host-mode ports](https://git.coopcloud.tech/coop-cloud/mailu/src/branch/main/compose.yml#L63-L83) do work – I wonder if there's some problematic interaction between Docker and the firewall or networking configuration on your server which is preventing external clients from being able to connect? The intention is definitely not to restrict those ports to local-only connections, anyway. > The README specifies "so that containers are aware of remote IP addresses" - but I'm unclear how that affects functionality. Mailu includes some IP-based rate limiting, and unless either Mailu's "front" container (or, if we were using Traefik entrypoints, Traefik itself) is in "host" mode, then docker swarm doesn't give access to the real originating IP, and all connections seem to be treated as coming from the same source IP, which in practice led to frequent accidental denial-of-service-style outages. "Host mode" has seemed like an acceptable limitation because the main drawbacks seem to be that (1) it's not compatible with multi-node docker swarms and (2) it doesn't allow multiple Mailu or Traefik instances on the same host. (1) seems reasonable because there are other unsolved blockers to multi-node (mainly, lack of multi-node SSL ingress in Traefik's community edition as of Traefik 2) (2) likewise seemed like an acceptable down-side because Traefik requires `HostSNI(*)` to route protocols which don't do SNI (server name indication), and that includes "opportunistic TLS" SMTP – which seems to be required for mail servers: > Yes HostSNI() would work with SMTPS(465). However I don't believe that SMTPS is widely used and you also run the risk of other MTAs not supporting tls at all. So you could artificially restrict mail flow. https://community.traefik.io/t/tcp-proxy-with-sni-support/6976 So, there's effectively a maximum of one Mailu (or SMTP-routing Traefik) instance per swarm node anyway. I'd personally be fine with additional entrypoints in the Traefik recipe – especially if that gives any advantages for anyone over "host mode" – in that case, I wonder what would happen if a second Mailu instance were deployed on a server (currently, I _think_ the use of host-mode ports would make `abra app deploy` fail, and containers fail to launch which, while not elegant, might be clearer and more deterministic than having two services with a `HostSNI(*)` routing rule).

amras commented

2023-09-24 08:20:06 +00:00

I wonder if there's some problematic interaction between Docker and the firewall or networking configuration on your server which is preventing external clients from being able to connect?

This seems to be correct. I'm not certain which interaction was the problem, but I've switched back to host-mode with no further issues.

Mailu includes some IP-based rate limiting

This answers my question. The additional context was also helpful. Thank you so much for the explanation!

> I wonder if there's some problematic interaction between Docker and the firewall or networking configuration on your server which is preventing external clients from being able to connect? This seems to be correct. I'm not certain which interaction was the problem, but I've switched back to host-mode with no further issues. > Mailu includes some IP-based rate limiting This answers my question. The additional context was also helpful. Thank you so much for the explanation!

❤️ 2

amras closed this issue

2023-09-24 08:20:14 +00:00

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: coop-cloud/mailu#8