The console serves nothing, not even the certificate ask, until Temporal and the identity provider are reachable #141

New Issue

2026-09-25T08:20:00Z

cgalo5758 commented

2026-09-25 08:20:00 +00:00

What happens

At boot the console opens its HTTP port only after two outside services answer. It connects to Temporal, first fetching an OAuth token from the identity provider when Temporal authorization is configured, and it reads the identity provider's discovery document. Until both succeed, no route is served. If Temporal is not reached within the connect budget, the process exits and the orchestrator restarts it into the same wait.

That includes GET /domains/ask, the endpoint a TLS-terminating proxy calls before it issues or loads an on-demand certificate (the certificate ask). The ask needs only the database.

When the identity provider's host is itself issued on demand by the same proxy, a proxy restart with an empty certificate cache deadlocks. The proxy will not present a certificate for the identity provider until the ask answers, and the console will not answer the ask until it has reached the identity provider through that proxy. Every on-demand host stays without TLS until an operator points the proxy's ask somewhere else.

What should happen

Routes whose only dependency is the database, /domains/ask first among them, are served as soon as migrations have run, whatever the state of Temporal and the identity provider. Routes that need either answer 503 until it is reachable, and the console keeps retrying in the background instead of exiting.

Where

cmd/start.go (the Temporal connection before server.Start), internal/auth/auth.go (oidc.NewProvider, called from server.Start before the listener opens), internal/server/domains.go

Steps

Put the identity provider behind a proxy whose on-demand TLS ask points at the console's /domains/ask, with the identity provider's host issued on demand.
Restart the proxy and the console together; a Docker daemon restart does this.
The proxy refuses TLS for every on-demand host. The console logs Temporal not ready yet, retrying until it exits, and restarts into the same state.

Why it matters

A routine host restart can take every on-demand site offline, and the console, which exists to authorize those certificates, is what keeps them offline. Until this is fixed, a deployment avoids it by giving the identity provider's host an ordinary managed certificate, which the proxy loads at startup without asking.

### What happens At boot the console opens its HTTP port only after two outside services answer. It connects to Temporal, first fetching an OAuth token from the identity provider when Temporal authorization is configured, and it reads the identity provider's discovery document. Until both succeed, no route is served. If Temporal is not reached within the connect budget, the process exits and the orchestrator restarts it into the same wait. That includes `GET /domains/ask`, the endpoint a TLS-terminating proxy calls before it issues or loads an on-demand certificate (the certificate ask). The ask needs only the database. When the identity provider's host is itself issued on demand by the same proxy, a proxy restart with an empty certificate cache deadlocks. The proxy will not present a certificate for the identity provider until the ask answers, and the console will not answer the ask until it has reached the identity provider through that proxy. Every on-demand host stays without TLS until an operator points the proxy's ask somewhere else. ### What should happen Routes whose only dependency is the database, `/domains/ask` first among them, are served as soon as migrations have run, whatever the state of Temporal and the identity provider. Routes that need either answer 503 until it is reachable, and the console keeps retrying in the background instead of exiting. ### Where `cmd/start.go` (the Temporal connection before `server.Start`), `internal/auth/auth.go` (`oidc.NewProvider`, called from `server.Start` before the listener opens), `internal/server/domains.go` ### Steps 1. Put the identity provider behind a proxy whose on-demand TLS ask points at the console's `/domains/ask`, with the identity provider's host issued on demand. 2. Restart the proxy and the console together; a Docker daemon restart does this. 3. The proxy refuses TLS for every on-demand host. The console logs `Temporal not ready yet, retrying` until it exits, and restarts into the same state. ### Why it matters A routine host restart can take every on-demand site offline, and the console, which exists to authorize those certificates, is what keeps them offline. Until this is fixed, a deployment avoids it by giving the identity provider's host an ordinary managed certificate, which the proxy loads at startup without asking.

cgalo5758 added the

kind

bug

area/domains area/ops labels 2026-09-25 08:20:00 +00:00

cgalo5758 referenced this issue

2026-10-09 16:03:03 +00:00

`member-console start` exits with status 0 when the web server fails to start #209

Sign in to join this conversation.