Prometheus metrics endpoint exposes credentials over plaintext HTTP #94

New Issue

2026-03-13T12:28:58Z

dannygroenewegen commented

2026-03-13 12:28:58 +00:00

The metrics endpoint is currently configured on a dedicated entrypoint on port 8082 with basicauth middleware. Because this entrypoint has no TLS, basic authentication credentials are transmitted in cleartext over HTTP. This is particularly relevant in multi-host setups where metrics are scraped across hosts.

A solution would be to expose the metrics endpoint through a HTTPS router using manualRouting with these snippets:

metrics:
  prometheus:
    entryPoint: web-secure
    manualRouting: true
    addRoutersLabels: true
    addServicesLabels: true

http:
  routers:
    traefik-metrics:
      rule: "Host(`${PROMETHEUS_METRICS_HOSTNAME`)"
      entrypoints:
        - web-secure
      tls:
        certResolver: ${LETS_ENCRYPT_ENV}
      middlewares:
        - basicauth@file
      service: prometheus@internal

I'm happy to open a PR for this. Two approaches are possible and I'd like input on which direction to take:

Non-breaking: add a new secure metrics configuration alongside the existing one, so current installations are unaffected. Users can opt in to the secure setup.
Breaking: replace the current plaintext configuration with the secure approach. This would require users to update their setup (both Traefik .env and hostname configuration in monitoring-ng scrape config).

I lean toward option 2, given that the current configuration transmits credentials in cleartext. I think defaults should be secure, even if that means a breaking change.

The metrics endpoint is currently configured on a dedicated entrypoint on port 8082 with basicauth middleware. Because this entrypoint has no TLS, basic authentication credentials are transmitted in cleartext over HTTP. This is particularly relevant in multi-host setups where metrics are scraped across hosts. A solution would be to expose the metrics endpoint through a HTTPS router using `manualRouting` with these snippets: ```yaml metrics: prometheus: entryPoint: web-secure manualRouting: true addRoutersLabels: true addServicesLabels: true ``` ```yaml http: routers: traefik-metrics: rule: "Host(`${PROMETHEUS_METRICS_HOSTNAME`)" entrypoints: - web-secure tls: certResolver: ${LETS_ENCRYPT_ENV} middlewares: - basicauth@file service: prometheus@internal ``` I'm happy to open a PR for this. Two approaches are possible and I'd like input on which direction to take: 1. Non-breaking: add a new secure metrics configuration alongside the existing one, so current installations are unaffected. Users can opt in to the secure setup. 2. Breaking: replace the current plaintext configuration with the secure approach. This would require users to update their setup (both Traefik .env and hostname configuration in monitoring-ng scrape config). I lean toward option 2, given that the current configuration transmits credentials in cleartext. I think defaults should be secure, even if that means a breaking change.

👀 1 ❤️ 1

decentral1se commented

2026-03-13 14:41:43 +00:00

Oh shiet, good catch. I'm happy to see a PR for option 2. I don't quite understand the config implications and also this is then connected to monitoring-ng? If you can provide links to the specific configuration lines in the recipes, that would help.

Oh shiet, good catch. I'm happy to see a PR for option 2. I don't quite understand the config implications and also this is then connected to `monitoring-ng`? If you can provide links to the specific configuration lines in the recipes, that would help.

👍 1

stevensting commented

2026-03-19 07:57:36 +00:00

Thanks for reporting. I would also prefer solution 2, and if we do it right it isn't even that breaking I guess:

make modifications to traefik recipe as outline above
make modifications to monitoring-ng as outlined above
at the same time release new versions of traefik and monitoring-ng
tell people in the releasenotes that they also need to update the other app
all people need to do is update both at the same time to prevent stuff from breaking

some more hints:

I did not need to do anything special in the scrape config for https. I think prometheus just accepts being rerouted to https
I added a https endpoint for prometheus monitoring to backup bot two yesterday, the implementation is probably the quiet similar: https://git.coopcloud.tech/coop-cloud/backup-bot-two/pulls/84/files

@p4u1 did I overlook something?

Thanks for reporting. I would also prefer solution 2, and if we do it right it isn't even that breaking I guess: - make modifications to traefik recipe as outline above - make modifications to monitoring-ng as outlined above - at the same time release new versions of traefik and monitoring-ng - tell people in the releasenotes that they also need to update the other app - all people need to do is update both at the same time to prevent stuff from breaking some more hints: - I did not need to do anything special in the scrape config for https. I think prometheus just accepts being rerouted to https - I added a https endpoint for prometheus monitoring to backup bot two yesterday, the implementation is probably the quiet similar: https://git.coopcloud.tech/coop-cloud/backup-bot-two/pulls/84/files @p4u1 did I overlook something?

❤️ 1

dannygroenewegen referenced a pull request that will close this issue

2026-03-20 19:26:35 +00:00

fix: Change metrics endpoint to use https instead of http 8082 to prevent sending BASIC_AUTH in plaintext. #95

dannygroenewegen commented

2026-03-20 19:34:55 +00:00

If you can provide links to the specific configuration lines in the recipes, that would help.

@decentral1se I made #95 for the changes in Traefik recipe and coop-cloud/monitoring-ng#17 for the changed in monitoring-ng

I did not need to do anything special in the scrape config for https. I think prometheus just accepts being rerouted to https

@stevensting See coop-cloud/monitoring-ng#17 for the changes in monitoring-ng. How do you think we can prevent stuff from breaking? The PR updates the scrape-config example and abra.sh function for adding new domains to the scrape config. But existing installs will have example.org:8082 in the container at one or more /prometheus/scrape_configs/example.org.yml files. Those are not updated with a new recipe version and need manual updating, right?

> If you can provide links to the specific configuration lines in the recipes, that would help. @decentral1se I made #95 for the changes in Traefik recipe and coop-cloud/monitoring-ng#17 for the changed in monitoring-ng > I did not need to do anything special in the scrape config for https. I think prometheus just accepts being rerouted to https @stevensting See coop-cloud/monitoring-ng#17 for the changes in monitoring-ng. How do you think we can prevent stuff from breaking? The PR updates the scrape-config example and abra.sh function for adding new domains to the scrape config. But existing installs will have example.org:8082 in the container at one or more /prometheus/scrape_configs/example.org.yml files. Those are not updated with a new recipe version and need manual updating, right?

❤️ 1

dannygroenewegen referenced this issue from coop-cloud/monitoring-ng

2026-03-20 20:26:20 +00:00

fix: Update scape-config example to use HTTPS for Traefik metrics #17

decentral1se closed this issue

2026-03-21 15:44:36 +00:00

decentral1se referenced this issue from a commit

2026-03-21 15:44:38 +00:00

fix: Change metrics endpoint to use https instead of http 8082 to prevent sending BASIC_AUTH in plaintext. (#95)

p4u1 referenced this issue from a commit

2026-03-24 09:37:07 +00:00

fix: Update scape-config example to use HTTPS for Traefik metrics (#17)