Describe the boot that serves before its dependencies
Seventh section of #141, the docs. docs/production-deployment.md's first boot section lists the steps in the order start runs them, says what serves from the moment the listener opens, and describes the two loops that run beside it: the identity provider's discovery, with sign-in at 503 until a usable document is published and roles not re-derived before a document with the configured issuer is read, and Temporal, retried for the life of the process with its log turning to Error after temporal-connect-timeout. The service list says that Stripe and Discourse need Temporal. README.md and docs/plan-management.md lose their description of a bounded retry before the server comes up. docs/environment-reference.md rewrites the temporal-connect-timeout and temporal-host rows; docs/stripe.md and docs/discourse-setup.md say why temporal-host is required; docs/building-an-integration.md documents NeedsTemporal. docs/fedwiki-setup.md says the console answers the certificate ask as soon as it serves. docs/identity-provider-setup.md and internal/auth/README.md replace "refuses to start" with what the console does, and docs/temporal-authorization-setup.md explains the token URL. Comments in the FedWiki and Stripe integrations and the FedWiki workflows README that still described the old boot are corrected. Every quoted log line is the one the code writes. No code changes. make test passes with the test stack up, and go test -race -p 1 over cmd, auth, server, retryloop, workflows and the three integrations after reset-test-db.sh finds no data race.
This commit is contained in:
@@ -125,9 +125,10 @@ in the same file). The console runs fine with neither.
|
||||
console is at **http://member-console.localhost:9481**; sign in to the operator
|
||||
panel as `alice` / `password`.
|
||||
|
||||
> On a fresh stack the console rides out Temporal's cold-start with a bounded
|
||||
> connect retry, so the first `start` may pause briefly before the server comes
|
||||
> up — no re-run needed.
|
||||
> On a fresh stack the console serves before Temporal is up and connects to
|
||||
> it in the background, so the first `start` needs no re-run. Until it
|
||||
> connects, the actions that need Temporal answer that they are not
|
||||
> available.
|
||||
|
||||
See [`test/AGENTS.md`](test/AGENTS.md) for the full stack contract (idempotency,
|
||||
Stripe webhook constraints, teardown, running multiple worktrees concurrently).
|
||||
|
||||
@@ -599,6 +599,7 @@ type ConfigKey struct {
|
||||
Secret bool
|
||||
RequiredGroup string
|
||||
DeploymentOnly bool
|
||||
NeedsTemporal bool
|
||||
}
|
||||
```
|
||||
|
||||
@@ -630,6 +631,14 @@ type ConfigKey struct {
|
||||
`fedwiki-admin-token`, `discourse-base-url` beside `discourse-api-key`. An
|
||||
operator who cannot read the token must not be able to choose its
|
||||
recipient. `Secret` implies it; declaring both is a declaration error.
|
||||
- **`NeedsTemporal`** — goes on a key of the `RequiredGroup` of an integration
|
||||
that receives events processed only on Temporal. `config.ValidateStart`
|
||||
refuses the configured group without `temporal-host`, naming the group and
|
||||
its keys (`member-console config validate` runs the same check);
|
||||
`config.CheckSpecs` refuses the declaration on a key that has no
|
||||
`RequiredGroup`. An integration whose actions run on Temporal but that
|
||||
receives no events does not declare it. Stripe declares it on
|
||||
`stripe-api-key`, Discourse on `discourse-base-url`.
|
||||
|
||||
The composition root binds flags/env/defaults for every declared key
|
||||
generically (`cmd/start.go`'s `registerIntegrationConfigFlags`, looping
|
||||
|
||||
@@ -77,6 +77,10 @@ applied, no forum calls, and the operator page says so). Both are set with
|
||||
the deployment: the settings page shows the URL and offers no control,
|
||||
because the API key is sent to whatever that URL names.
|
||||
|
||||
A configured Discourse also requires `temporal-host`: a workflow reconciles
|
||||
its webhook events, and a Temporal schedule restores what a failed event
|
||||
missed, so `start` refuses the configuration without it.
|
||||
|
||||
## 4. Map entitlements to groups
|
||||
|
||||
Operator panel → Integrations → Discourse:
|
||||
|
||||
@@ -86,14 +86,14 @@ See [identity-provider-setup.md](identity-provider-setup.md) for configuring the
|
||||
|
||||
| Key | Type | Env override | Purpose | Default | Required |
|
||||
|---|---|---|---|---|---|
|
||||
| `temporal-host` | string | `MC_TEMPORAL_HOST` | Temporal server `host:port`, e.g. `localhost:7233`. Leaving it unset disables Temporal, the worker, integration workflows, and every scheduled sweep below. | none | Optional (functionally required for provisioning and scheduled jobs) |
|
||||
| `temporal-host` | string | `MC_TEMPORAL_HOST` | Temporal server `host:port`, e.g. `localhost:7233`. Leaving it unset disables Temporal, the worker, integration workflows, and every scheduled sweep below. A configured Stripe or Discourse needs it: both receive events that only Temporal processes, and `start` refuses the configuration without it. | none | Required when Stripe or Discourse is configured; otherwise optional (functionally required for provisioning and scheduled jobs) |
|
||||
| `temporal-namespace` | string | `MC_TEMPORAL_NAMESPACE` | Temporal namespace. | `default` | Required once `temporal-host` is set |
|
||||
| `temporal-oauth-token-url` | url | `MC_TEMPORAL_OAUTH_TOKEN_URL` | OAuth2 token endpoint for Temporal client authentication. | none | Optional; if any `temporal-oauth-*` key below is set, all three must be |
|
||||
| `temporal-oauth-client-id` | string | `MC_TEMPORAL_OAUTH_CLIENT_ID` | OAuth2 client ID for Temporal authentication. | none | Optional; see above |
|
||||
| `temporal-oauth-client-secret` (secret) | string | `MC_TEMPORAL_OAUTH_CLIENT_SECRET` | OAuth2 client secret for Temporal authentication. | none | Optional; see above |
|
||||
| `temporal-oauth-client-secret-file` | string | `MC_TEMPORAL_OAUTH_CLIENT_SECRET_FILE` | File path holding `temporal-oauth-client-secret`. | none | Optional |
|
||||
| `temporal-oauth-scopes` | list | `MC_TEMPORAL_OAUTH_SCOPES` | OAuth2 scopes for Temporal authentication (comma-separated). | none | Optional |
|
||||
| `temporal-connect-timeout` | duration | `MC_TEMPORAL_CONNECT_TIMEOUT` | Max time to retry connecting to Temporal at boot before failing. | `90s` | Optional |
|
||||
| `temporal-connect-timeout` | duration | `MC_TEMPORAL_CONNECT_TIMEOUT` | Time after startup from which failed Temporal connection attempts log at Error. The console keeps retrying for the life of the process, so the value sets when the log turns to Error and never stops the retries. | `90s` | Optional |
|
||||
|
||||
See [temporal-authorization-setup.md](temporal-authorization-setup.md) for configuring the IdP side of Temporal's own JWT authorization.
|
||||
|
||||
@@ -145,7 +145,7 @@ See [stripe.md](stripe.md) for the webhook endpoint, subscribed events, and cata
|
||||
| `fedwiki-site-scheme` | http\|https | `MC_FEDWIKI_SITE_SCHEME` | URL scheme for generated FedWiki site links: `http` or `https`. | `https` | Optional |
|
||||
| `fedwiki-sync-enabled` | bool | `MC_FEDWIKI_SYNC_ENABLED` | Enable periodic sync of sites from the FedWiki farm. | `false` | Optional |
|
||||
| `fedwiki-sync-interval` | duration, greater than zero | `MC_FEDWIKI_SYNC_INTERVAL` | Interval between FedWiki site syncs. Inert while sync is disabled. | `1h` | Optional |
|
||||
| `fedwiki-sync-trigger-immediately` | bool | `MC_FEDWIKI_SYNC_TRIGGER_IMMEDIATELY` | Run a sync immediately at startup, in addition to the interval. | `true` | Optional |
|
||||
| `fedwiki-sync-trigger-immediately` | bool | `MC_FEDWIKI_SYNC_TRIGGER_IMMEDIATELY` | Run a sync immediately once the console connects to Temporal, in addition to the interval. | `true` | Optional |
|
||||
| `fedwiki-swap-cooldown` | duration | `MC_FEDWIKI_SWAP_COOLDOWN` | Cooldown window between FedWiki active-site rotations (swap or archive-then-restore). `0` disables it. | `720h` (30 days) | Optional |
|
||||
|
||||
See [fedwiki-setup.md](fedwiki-setup.md) for the FarmManager security stack and local dev setup.
|
||||
@@ -163,7 +163,7 @@ See [fedwiki-setup.md](fedwiki-setup.md) for the FarmManager security stack and
|
||||
| `discourse-webhook-secret` (secret) | string | `MC_DISCOURSE_WEBHOOK_SECRET` | Shared secret verifying Discourse webhook signatures. Leave empty to disable the webhook endpoint. | none | Optional |
|
||||
| `discourse-webhook-secret-file` | string | `MC_DISCOURSE_WEBHOOK_SECRET_FILE` | File path holding `discourse-webhook-secret`. | none | Optional |
|
||||
| `discourse-sync-interval` | duration, greater than zero | `MC_DISCOURSE_SYNC_INTERVAL` | Interval between Discourse group-sync sweeps. | `15m` | Optional |
|
||||
| `discourse-sync-trigger-immediately` | bool | `MC_DISCOURSE_SYNC_TRIGGER_IMMEDIATELY` | Run a group-sync sweep immediately at startup. | `false` | Optional |
|
||||
| `discourse-sync-trigger-immediately` | bool | `MC_DISCOURSE_SYNC_TRIGGER_IMMEDIATELY` | Run a group-sync sweep immediately once the console connects to Temporal. | `false` | Optional |
|
||||
|
||||
See [discourse-setup.md](discourse-setup.md) for forum-side prerequisites and linkage-mode selection.
|
||||
|
||||
|
||||
@@ -269,10 +269,12 @@ other answer refuses it. Point Caddy at it:
|
||||
}
|
||||
```
|
||||
|
||||
While the console is unreachable the proxy can issue no new certificate,
|
||||
because every ask goes unanswered; names whose certificates were already
|
||||
issued keep being served. The deployment accepts that bound rather than
|
||||
materializing an allowlist in the proxy.
|
||||
The console answers the ask as soon as its listener opens, which needs only
|
||||
the database: Temporal, the identity provider and the session store can each
|
||||
be down. While the console is down the
|
||||
proxy can issue no new certificate, because every ask goes unanswered; names
|
||||
whose certificates were already issued keep being served. The deployment
|
||||
accepts that bound rather than materializing an allowlist in the proxy.
|
||||
|
||||
The endpoint is unauthenticated (TLS proxies send no credentials) — expose
|
||||
it to the proxy's network only. This replaces filesystem-based ask
|
||||
|
||||
@@ -13,8 +13,8 @@ This guide covers configuring your OIDC identity provider to work with the membe
|
||||
- An OIDC identity provider with a client configured for Authorization Code + PKCE flow
|
||||
- The client ID and secret configured in `mc-config.yaml` (or equivalent flags)
|
||||
- Refresh tokens issued to the client (Keycloak's default). Two things use them: every five minutes a signed-in session re-derives its identity and roles from a fresh ID token, so a role removed, an account disabled, or a session ended at the provider takes effect here within that bound; and sign-out mints a fresh ID token for a valid `id_token_hint`, so the provider ends its session without asking. Without a refresh token a session keeps its sign-in roles for its lifetime and the provider prompts at sign-out. The roles claim must be present in the ID token a refresh returns, not only the one sign-in returns (Keycloak's mappers do both).
|
||||
- `end_session_endpoint` published in the provider's discovery document (OpenID Connect RP-Initiated Logout 1.0). The console refuses to start without it.
|
||||
- `revocation_endpoint` published in the discovery document (RFC 7009), so the console can revoke the refresh token at sign-out. Optional; without it the console warns once at startup and skips revocation.
|
||||
- `end_session_endpoint` published in the provider's discovery document (OpenID Connect RP-Initiated Logout 1.0). Without it the console serves but keeps sign-in unavailable, logging at Error, until the provider publishes the endpoint. A document that names the configured issuer but has no `end_session_endpoint` leaves sign-in unavailable and does not stop the five-minute re-derivation of signed-in sessions. No session is re-derived until the console has read a document that names the configured issuer. After a start, that lasts until the first fetch succeeds. While the document names another issuer, it lasts until the issuer is fixed, and for that time a role removed or an account disabled at the provider keeps working in the console.
|
||||
- `revocation_endpoint` published in the discovery document (RFC 7009), so the console can revoke the refresh token at sign-out. Optional; without it the console warns once when it reads the discovery document and skips revocation.
|
||||
|
||||
## Standard OIDC Claims
|
||||
|
||||
@@ -109,4 +109,4 @@ console's operator login (`alice` / `password`) is covered in the
|
||||
|
||||
> **Run app OIDC on a dedicated realm, not Keycloak's `master` admin realm.** The member-facing "Identity and Access" link opens Keycloak's account console; on the `master` admin realm that console errors (*"Something went wrong"*) because the minted token carries no `aud: account` claim. The test stack therefore creates and runs on a dedicated `test` realm, where the account console works out of the box — do the same in production. Fixed by the `keycloak-dedicated-test-realm` change.
|
||||
|
||||
> **Note:** The first `member-console start` after a fresh `docker compose up -d` used to fail with a Temporal auth error. This is fixed — compose now gates Temporal on its database and Keycloak being healthy, and member-console retries the initial Temporal connection. Fixed by the `temporal-first-boot-retry` change.
|
||||
> The first `member-console start` after a fresh `docker compose up -d` serves before Temporal is ready and connects once it is, so it needs no re-run.
|
||||
|
||||
@@ -162,6 +162,12 @@ The system:
|
||||
- Creates a grant with the chosen reason and the resumption mark, then calls the conferral boundary (`internal/entitlements/conferral.go`) through the operator handler in `internal/server/operator_enrollment.go`: it writes the grant decree, confers the position onto the organization's default pool, then materializes the organization's effective entitlements
|
||||
- Registers a **Temporal workflow** to expire the grant at `valid_until` when one is set
|
||||
|
||||
While Temporal is not connected, which on a deployment without
|
||||
`temporal-host` is always, a grant with **Valid until** is refused before
|
||||
it is written, with "An end date needs the expiration scheduler, which is
|
||||
not connected." under the field. **Manage** refuses a new or changed end
|
||||
date the same way and saves a cleared one.
|
||||
|
||||
A grant with no `valid_until` is permanent. A grant with `valid_until` expires at that moment; when it expires, the pool returns to what it held before (see [Downgrades](#downgrades-and-the-tier-reduction-policy)).
|
||||
|
||||
### Grant Revocation
|
||||
|
||||
@@ -43,13 +43,14 @@ Decide now which services member-console needs for your offering:
|
||||
- **An OIDC identity provider** — always required (any OIDC-compliant
|
||||
provider; see step 4).
|
||||
- **Temporal** — required for provisioning, integration sync, and billing
|
||||
sweeps to actually run; the console starts without it but logs
|
||||
`Temporal not configured` and every scheduled job stays off.
|
||||
sweeps to actually run, and required whenever Stripe or Discourse is
|
||||
configured; without it the console starts, logs `Temporal not
|
||||
configured`, and every scheduled job stays off.
|
||||
- **A FedWiki farm** — required only if you offer FedWiki sites, the
|
||||
product's first conforming integration; see step 6.
|
||||
- **Stripe** — required only for paid products; a deployment offering only
|
||||
free/unmetered products can skip it entirely.
|
||||
- **Discourse** — optional, for forum-membership sync.
|
||||
- **Stripe** — required only for paid products, and needs Temporal; a
|
||||
deployment offering only free/unmetered products can skip it entirely.
|
||||
- **Discourse** — optional, for forum-membership sync; needs Temporal.
|
||||
|
||||
The test stack's `test/compose.yaml` is a useful reference for how these
|
||||
services relate to each other and to member-console (ports, environment
|
||||
@@ -368,14 +369,83 @@ they live, what the console shows members, and what the API key can read.
|
||||
|
||||
With the database reachable, `base-url` set, the session store's password
|
||||
configured, and the OIDC keys pointed at your realm, start the process (`member-console
|
||||
start`, or your image's equivalent entrypoint). At startup it, in order:
|
||||
opens the runtime connection; when `db-migrate-dsn` is set, opens the
|
||||
migration connection, runs pending migrations, grants `console_runtime` to
|
||||
the runtime user and closes the migration connection; checks the runtime user
|
||||
and logs `runtime database user checked`; validates the aggregated configuration (failing
|
||||
fast with every problem named at once if something required is missing —
|
||||
see [environment-reference.md](environment-reference.md) for what's
|
||||
required), connects to Temporal if configured, and starts serving.
|
||||
start`, or your image's equivalent entrypoint). It runs these steps in this
|
||||
order, and exits 1 at the first one that fails:
|
||||
|
||||
1. Checks the integrations' configuration declarations, resolves each
|
||||
`*-file` secret, and validates the aggregated configuration, failing fast
|
||||
with every problem named at once if something required is missing (see
|
||||
[environment-reference.md](environment-reference.md) for what is
|
||||
required). A configured Stripe or Discourse without `temporal-host` fails
|
||||
here, because both receive events that only Temporal processes. Stripe's
|
||||
mode is derived from its API key here, and a key with an unknown prefix
|
||||
fails.
|
||||
2. Opens the runtime connection; when `db-migrate-dsn` is set, opens the
|
||||
migration connection, runs pending migrations, grants `console_runtime` to
|
||||
the runtime user and closes the migration connection; checks the runtime
|
||||
user and logs `runtime database user checked`.
|
||||
3. Applies the configuration overrides stored in the database, ensures the
|
||||
boot data every deployment needs, and binds each integration's provider
|
||||
connection. When `temporal-host` is set, it then builds the Temporal
|
||||
authorization from the `temporal-oauth-*` keys. A value that no
|
||||
connection attempt could fix, such as a token URL that does not parse,
|
||||
fails here and is logged as `failed to connect to Temporal`, before any
|
||||
connection is tried.
|
||||
4. Probes the session store once. A store that refuses the console's
|
||||
credentials fails the start, logged as `server failed to start` with the
|
||||
error `failed to connect to Valkey: <error>`. When the store does not
|
||||
answer, or answers with any other error, the console logs `session store
|
||||
not ready at startup; requests that need a session answer 503` at Error
|
||||
and continues. Those requests answer 503 until the store answers; the
|
||||
console needs no restart.
|
||||
5. Opens the listener. A port already in use fails the start.
|
||||
|
||||
From the moment the listener is open the console serves, whether or not
|
||||
Temporal and the identity provider have answered. The certificate ask
|
||||
(`/domains/ask`, see
|
||||
[fedwiki-setup.md](fedwiki-setup.md#serving-authorization-domainsask)), the
|
||||
static assets, the Stripe and Discourse webhook endpoints and the signed-in
|
||||
pages all answer. Two background loops run
|
||||
alongside and have no effect on whether the process stays up:
|
||||
|
||||
- **Identity provider.** The console fetches the provider's discovery
|
||||
document in the background, retrying with a growing wait. Until it has a
|
||||
usable document, `/login`, `/register` and `/callback` answer 503 with a
|
||||
page headed "Sign-in unavailable" or "Registration unavailable" and a
|
||||
`Retry-After: 5` header. The first failed attempt logs `identity provider
|
||||
discovery failed; sign-in is unavailable`. Later failures log again when
|
||||
the error changes, at most once every 30 seconds, and otherwise once every
|
||||
five minutes. These lines are Warn for the first 90 seconds and Error
|
||||
after. A document that names an issuer other than `oidc-idp-issuer-url`,
|
||||
or whose `end_session_endpoint` is missing or not an absolute URL, cannot
|
||||
be used and logs `identity provider discovery document cannot be used;
|
||||
sign-in is unavailable` at Error, and the loop fetches it again until the
|
||||
provider publishes a usable one. While no document names the configured
|
||||
issuer, signed-in sessions keep the roles they have. Once a usable document
|
||||
is read, sign-in works without a restart and the console logs `identity
|
||||
provider discovery succeeded; sign-in is available`.
|
||||
- **Temporal** (when `temporal-host` is set). The console connects, checks
|
||||
that the worker may start, starts the worker, and only then starts the
|
||||
integrations' workflows and schedules. A failed connection attempt logs
|
||||
`Temporal not ready yet, retrying`, spaced as the identity provider's lines
|
||||
are, at Warn until `temporal-connect-timeout` has passed since the
|
||||
connector began and at Error after. A worker that cannot be built or
|
||||
started logs `failed to create Temporal worker` or `failed to start
|
||||
Temporal worker` at Error from the first line. When FedWiki finds an open
|
||||
run of a workflow type the release removed, the console logs `refused to
|
||||
start the Temporal worker` at Error, starts no worker, and checks again
|
||||
until no such run is open (see [the FedWiki workflows
|
||||
README](../internal/integrations/fedwiki/workflows/README.md)). Once
|
||||
connected, the console logs `connected to Temporal server`. The connection
|
||||
is retried for the life of the process: the timeout sets when the log turns
|
||||
to Error and does not stop the retries. Until it connects, each Temporal-backed action says it
|
||||
is not available, and the Stripe webhook answers 503 so that Stripe
|
||||
redelivers the event. Without `temporal-host` the console logs a Warn that
|
||||
integration workflows and scheduled jobs are unavailable and never
|
||||
connects.
|
||||
|
||||
When the server fails, the process exits 1, so an orchestrator that restarts
|
||||
only on failure restarts it.
|
||||
|
||||
Sign in as an operator (the account holding the `operator-member` role you
|
||||
configured in step 4) and go to `/operator/setup`. It is a checklist of
|
||||
|
||||
+5
-1
@@ -25,6 +25,10 @@ This guide covers connecting the member console to Stripe for payment processing
|
||||
|
||||
All flags support `_FILE` variants for secret injection from mounted files.
|
||||
|
||||
Stripe also requires `temporal-host`: renewals, cancellations and payment
|
||||
failures arrive as webhook events that only Temporal applies, so `start`
|
||||
refuses a configured Stripe without it.
|
||||
|
||||
There is no mode setting. The console derives test or live from the API
|
||||
key's prefix at boot (`sk_live_` and `rk_live_` are live, `sk_test_` and
|
||||
`rk_test_` are test); any other prefix fails boot. "View in Stripe" links
|
||||
@@ -135,7 +139,7 @@ The sweep only reads invoices the console has heard of. An invoice for which no
|
||||
1. Grant **Invoices** read and **Charges** read to the restricted key, for every key the deployment uses, before deploying.
|
||||
2. Deploy. The Stripe store migration adds the columns the read-back needs and changes no data. A core migration adds `core.invoices.issued_at` and `core.payments.paid_at` and fills them from each row's own times, so no list loses a date: an invoice's issue time from its `created_at`, a succeeded payment's paid time from its invoice's `paid_at` or its own `created_at`, and a failed payment's missing `failed_at` from its `created_at`. It then requires an issue time on every invoice that is not a draft.
|
||||
3. If the deployment uses the `console_reader` role ([database-management.md](database-management.md#the-console_reader-role)), re-run `scripts/db-reader-role.sql` as the schema owner, so the role can read the new columns. None of them holds personal data.
|
||||
4. The sweep's first run, at boot, reads every invoice projected so far back once, one at a time. Invoices projected from later-version payloads gain their lines, their subscription link, and a payment for each settled charge, with its payment method; numbers stay as they are. Each invoice read back takes Stripe's issue time over the migration's, and Stripe's paid and void times over the ones the console recorded when the events arrived. Each charge's payment takes its time as the list above sets it: a failed one its charge's creation time, the one whose charge paid the invoice the invoice's paid time, any other succeeded one its charge's creation time. The payment of an invoice paid without a charge takes the invoice's paid time. Invoice and payment lists can therefore reorder; `created_at` stays as it was. A payment recorded from an event payload is not matched to a charge: it keeps its times, and each charge is recorded as a payment of its own beside it. A failed payment Stripe made no charge for (an authentication the customer never completed) keeps its `failed_at`, the time the console received its event, and an invoice Stripe no longer has keeps the issue time the migration gave it. Invoices left open because a first attempt failed and a later attempt on the same payment intent succeeded move to paid, so open-invoice counts and receivables can fall after the upgrade.
|
||||
4. The sweep's first run, once the console connects to Temporal, reads every invoice projected so far back once, one at a time. Invoices projected from later-version payloads gain their lines, their subscription link, and a payment for each settled charge, with its payment method; numbers stay as they are. Each invoice read back takes Stripe's issue time over the migration's, and Stripe's paid and void times over the ones the console recorded when the events arrived. Each charge's payment takes its time as the list above sets it: a failed one its charge's creation time, the one whose charge paid the invoice the invoice's paid time, any other succeeded one its charge's creation time. The payment of an invoice paid without a charge takes the invoice's paid time. Invoice and payment lists can therefore reorder; `created_at` stays as it was. A payment recorded from an event payload is not matched to a charge: it keeps its times, and each charge is recorded as a payment of its own beside it. A failed payment Stripe made no charge for (an authentication the customer never completed) keeps its `failed_at`, the time the console received its event, and an invoice Stripe no longer has keeps the issue time the migration gave it. Invoices left open because a first attempt failed and a later attempt on the same payment intent succeeded move to paid, so open-invoice counts and receivables can fall after the upgrade.
|
||||
|
||||
## What members see
|
||||
|
||||
|
||||
@@ -40,143 +40,6 @@ Therefore, the entire Keycloak setup is about **reliably producing a `permission
|
||||
|
||||
---
|
||||
|
||||
## Design Overview
|
||||
|
||||
We use three Keycloak concepts together:
|
||||
|
||||
1. **A dedicated “authorization-only” client** to own Temporal roles
|
||||
2. **A reusable Client Scope** to emit the `permissions` claim
|
||||
3. **Role assignments** on users, groups, and service accounts
|
||||
|
||||
This design works whether you:
|
||||
|
||||
- use separate Keycloak clients for your web UI vs worker, or
|
||||
- reuse the same Keycloak client for both (by enabling service accounts on it)
|
||||
|
||||
### Why this design?
|
||||
|
||||
| Problem | Why this solves it |
|
||||
| -------------------------- | --------------------------------------------- |
|
||||
| Realm roles feel cluttered | Temporal roles live in one obvious place |
|
||||
| Multiple Temporal clients | Roles are shared, not duplicated |
|
||||
| Service accounts | Work naturally via service-account users |
|
||||
| Temporal defaults | No custom authorizer or claim mapper required |
|
||||
|
||||
---
|
||||
|
||||
## Step 1: Create a Temporal Authorization Client
|
||||
|
||||
Create a new Keycloak client whose only purpose is to **hold Temporal permission roles**.
|
||||
|
||||
Example:
|
||||
|
||||
* **Client ID:** `temporal-authz`
|
||||
* **Access Type:** Confidential (details don’t matter much)
|
||||
* No login flows needed
|
||||
|
||||
### Create Client Roles on `temporal-authz`
|
||||
|
||||
Create client roles whose names exactly match Temporal’s expected format:
|
||||
|
||||
```
|
||||
default:read
|
||||
default:write
|
||||
default:worker
|
||||
default:admin
|
||||
payments:worker
|
||||
payments:admin
|
||||
```
|
||||
|
||||
> Tip: You may use **composite roles** (e.g. `namespace:admin` composed of read/write/worker) to reduce assignment complexity.
|
||||
|
||||
This client is now the **single source of truth** for Temporal permissions.
|
||||
|
||||
---
|
||||
|
||||
## Step 2: Assign Temporal Roles
|
||||
|
||||
### Human Users
|
||||
|
||||
* Prefer assigning roles via **groups**
|
||||
* Groups can be named however you like (`temporal-default-writers`, etc.)
|
||||
* Groups receive `temporal-authz` client roles
|
||||
* Users inherit them automatically
|
||||
|
||||
### Service Accounts (client_credentials)
|
||||
|
||||
For each client that uses service accounts:
|
||||
|
||||
1. Enable **Service Accounts**
|
||||
2. Go to **Service Account Roles**
|
||||
3. Assign roles from `temporal-authz`
|
||||
|
||||
Keycloak will treat the service account as a user, and role mapping works the same way.
|
||||
|
||||
#### Reusing a single client for web login + service account
|
||||
|
||||
If you want to reuse the same Keycloak client for both:
|
||||
|
||||
- web login (Authorization Code + PKCE) and
|
||||
- the Temporal worker (Client Credentials / service account)
|
||||
|
||||
the critical rule is: **assign Temporal worker permissions to the service account user, not to regular users**.
|
||||
|
||||
This way:
|
||||
|
||||
- the service account token includes `permissions: ["default:worker", ...]` (Temporal can poll task queues), but
|
||||
- user tokens do not accidentally include `default:worker` (reduces the impact of a leaked user token).
|
||||
|
||||
---
|
||||
|
||||
## Step 3: Create a Client Scope to Emit Permissions
|
||||
|
||||
Create a **Client Scope**:
|
||||
|
||||
* **Name:** `temporal-permissions`
|
||||
* **Type:** Optional
|
||||
* **Display on consent screen:** Off
|
||||
* **Include in token scope:** Off
|
||||
|
||||
This scope is responsible for producing the `permissions` claim.
|
||||
|
||||
---
|
||||
|
||||
## Step 4: Add a Protocol Mapper
|
||||
|
||||
Inside the `temporal-permissions` client scope, add a mapper:
|
||||
|
||||
### Mapper Configuration
|
||||
|
||||
* **Mapper Type:** User Client Role (or similar, depending on Keycloak version)
|
||||
* **Client ID:** `temporal-authz`
|
||||
* **Token Claim Name:** `permissions`
|
||||
* **Claim JSON Type:** `String`
|
||||
* **Multivalued:** On
|
||||
* **Add to access token:** On
|
||||
* **Add to ID token:** Off (usually)
|
||||
* **Add to userinfo:** Off (usually)
|
||||
|
||||
This mapper:
|
||||
|
||||
* Reads roles assigned from the `temporal-authz` client
|
||||
* Emits them as a multi-valued `permissions` claim
|
||||
|
||||
---
|
||||
|
||||
## Step 5: Attach the Client Scope
|
||||
|
||||
For every client that should be allowed to call Temporal:
|
||||
|
||||
1. Go to **Clients → (client) → Client scopes**
|
||||
2. Add `temporal-permissions` as a **Default Client Scope**
|
||||
|
||||
Now any token issued for that client from a user login or service account will include the `permissions[]` claim (if roles are assigned)
|
||||
|
||||
Gotchas:
|
||||
- Make sure that the dedicated client scope has either "Full Scope Allowed" enabled, or disable it and assign the `temporal-authz` client roles to it. Otherwise, the mapper won't see any roles to map.
|
||||
|
||||
---
|
||||
|
||||
## Resulting Token Example
|
||||
|
||||
```json
|
||||
@@ -192,3 +55,16 @@ Gotchas:
|
||||
```
|
||||
|
||||
Temporal’s default authorizer can consume this **without any customization**.
|
||||
|
||||
---
|
||||
|
||||
## Token URL
|
||||
|
||||
Set `temporal-oauth-token-url` to an address the console reaches without
|
||||
passing through the TLS proxy in front of the deployment, such as the
|
||||
identity provider's internal service address. The Temporal connection then
|
||||
works while the proxy is down or still obtaining certificates. The console
|
||||
does not wait for this connection before it serves, so a proxy that asks
|
||||
the console's `/domains/ask` before issuing a certificate
|
||||
([fedwiki-setup.md](fedwiki-setup.md#serving-authorization-domainsask))
|
||||
gets its answer either way.
|
||||
|
||||
@@ -102,13 +102,13 @@ The callback hands the verified claims to `provisioning.ResolveSignIn`, which re
|
||||
|
||||
## Identity Re-derivation
|
||||
|
||||
- Every five minutes, on the first authenticated request after the interval, the middleware re-derives the session's identity and roles with a refresh-token grant: the new ID token is verified as at sign-in, its roles replace the session's, and the new ID token and refresh token replace the stored ones.
|
||||
- Every five minutes, on the first authenticated request after the interval, the middleware re-derives the session's identity and roles with a refresh-token grant: the new ID token is verified as at sign-in, its roles replace the session's, and the new ID token and refresh token replace the stored ones. No session is re-derived before the background loop has read a discovery document with the configured issuer; until then the middleware keeps the session's identity and roles.
|
||||
- `invalid_grant` (session ended at the IdP, account disabled, token revoked) ends the console session and bounces to `/login`. No answer keeps the session and retries after a minute. A token naming a different subject ends the session.
|
||||
- Concurrent requests crossing the interval share one refresh (`singleflight`, keyed by session token).
|
||||
|
||||
## Logout Flow
|
||||
|
||||
- Sign-out is a POST, made by the account menu's button; the cross-origin protection covers it and a GET at `/logout` only redirects to `/`. `LogoutHandler` destroys the console session first, then sends the browser to the IdP's `end_session_endpoint`, read from discovery at startup (`HX-Redirect` to the htmx request, since fetch cannot follow a redirect to another origin). A provider that publishes none fails startup; there is no fallback path.
|
||||
- Sign-out is a POST, made by the account menu's button; the cross-origin protection covers it and a GET at `/logout` only redirects to `/`. `LogoutHandler` destroys the console session first, then sends the browser to the IdP's `end_session_endpoint`, read from the discovery document the background loop fetches (`HX-Redirect` to the htmx request, since fetch cannot follow a redirect to another origin). A provider that publishes none leaves sign-in unavailable, and the console logs at Error until the provider publishes one; the console still serves. Before the endpoint is known, `POST /logout` ends the console session, revokes the stored refresh token when the verifier is published and the document carries a revocation endpoint, mints no `id_token_hint`, and lands on `/`. The provider's own session stays.
|
||||
- The request carries `id_token_hint` whenever a valid ID token can be had: the stored one while it is still valid, otherwise a fresh one minted from the stored refresh token in one bounded call to the token endpoint. A valid hint is what lets the IdP end its session without asking the person to confirm. When no hint can be had the IdP asks; the console session is gone either way.
|
||||
- Between the two, every refresh token that was live (the stored one, and a rotated one when the refresh returned it) is revoked at the IdP's `revocation_endpoint` (RFC 7009) when discovery publishes one, so no copy can mint tokens after sign-out. Best effort: a failure is logged and the sign-out proceeds.
|
||||
- `LogoutCallbackHandler` destroys the session once more (expiring a cookie the browser may still carry) and redirects to `/login`.
|
||||
|
||||
@@ -430,9 +430,9 @@ type workflowLister interface {
|
||||
ListWorkflow(ctx context.Context, request *workflowservice.ListWorkflowExecutionsRequest) (*workflowservice.ListWorkflowExecutionsResponse, error)
|
||||
}
|
||||
|
||||
// openRunsError is the refusal to start while runs of removed types are open;
|
||||
// a type, not an fmt.Errorf, because its text is the sentence an operator
|
||||
// reads in the boot log.
|
||||
// openRunsError is the refusal to start the worker while runs of removed
|
||||
// types are open; a type, not an fmt.Errorf, because its text is the
|
||||
// sentence an operator reads in the connector's log line.
|
||||
type openRunsError struct {
|
||||
total int
|
||||
counts []string // "type (count)", in removedWorkflowTypes order
|
||||
@@ -449,7 +449,9 @@ func (e *openRunsError) Error() string {
|
||||
e.total, strings.Join(e.counts, ", "))
|
||||
}
|
||||
|
||||
// listRunsError is the refusal to start when the list cannot be read.
|
||||
// listRunsError is the precondition's failure to read the list: it reports
|
||||
// workflows.ErrCannotCheck, so the connector closes the client and connects
|
||||
// again.
|
||||
type listRunsError struct{ err error }
|
||||
|
||||
func (e *listRunsError) Error() string {
|
||||
@@ -463,11 +465,11 @@ func (e *listRunsError) Unwrap() error { return e.err }
|
||||
// not report it.
|
||||
func (e *listRunsError) Is(target error) bool { return target == corework.ErrCannotCheck }
|
||||
|
||||
// CheckBeforeWorker refuses the start while a run of a workflow type this
|
||||
// release removed is open (workflows.WorkerPrecondition): a worker without
|
||||
// the type fails every task of such a run and leaves its farm call and row
|
||||
// write unfinished. A list that cannot be read refuses too, since it cannot
|
||||
// say that none is open.
|
||||
// CheckBeforeWorker refuses the worker's start while a run of a workflow type
|
||||
// this release removed is open (workflows.WorkerPrecondition): a worker
|
||||
// without the type fails every task of such a run and leaves its farm call
|
||||
// and row write unfinished. A list that cannot be read is returned as
|
||||
// workflows.ErrCannotCheck, since it cannot say that none is open.
|
||||
func (Adapter) CheckBeforeWorker(ctx context.Context, c client.Client) error {
|
||||
return checkNoRemovedRuns(ctx, c)
|
||||
}
|
||||
|
||||
@@ -207,4 +207,4 @@ Delete operations treat HTTP 404 (Not Found) as success - if the site doesn't ex
|
||||
|
||||
HTTP handlers do not wait for these workflows. A handler records a lifecycle request in `core.lifecycle_requests`, commits, starts the workflow under the request's workflow id, and answers as soon as the start is confirmed. The workflow ends by recording the request's status and reason code, and the site list shows an open request as Pending on its row and an undismissed declined or failed one as a notice above the list. A request whose workflow never recorded an outcome is settled by the boot pass in `settle.go`.
|
||||
|
||||
The four workflow types that predate lifecycle requests (`CreateFedWikiSiteWorkflow`, `DeleteFedWikiSiteWorkflow`, `SetSiteStatusWorkflow`, `SwapActiveSiteWorkflow`) are not registered. The console refuses to start while a run of one of them is open: after it connects to Temporal and before its worker starts, `Adapter.CheckBeforeWorker` lists the running executions of those types and the process exits with status 1 when any is open or the list cannot be read. Let an open run finish on the previous release, or terminate it, before starting this release.
|
||||
The four workflow types that predate lifecycle requests (`CreateFedWikiSiteWorkflow`, `DeleteFedWikiSiteWorkflow`, `SetSiteStatusWorkflow`, `SwapActiveSiteWorkflow`) are not registered. After the console connects to Temporal and before its worker starts, `Adapter.CheckBeforeWorker` lists the running executions of those types. While any is open the worker does not start: the console serves, every Temporal-backed action answers that it is not available, the console logs `refused to start the Temporal worker` at Error with the open runs counted per type, and the list is read again every 15 to 22.5 seconds. A list that cannot be read is retried on a new connection. Let an open run finish on the previous release, or terminate it; the worker starts once none is open.
|
||||
|
||||
@@ -211,12 +211,12 @@ func (Adapter) RegisterWorkflows(w worker.Worker, database *sql.DB, logger *slog
|
||||
w.RegisterActivity(workflows.NewPublicDetailsActivities(database, logger, mode, FingerprintForKey(apiKey)))
|
||||
}
|
||||
|
||||
// Startup hand-starts Stripe's long-running webhook-processor and
|
||||
// outbox-poller workflows, replacing the two ExecuteWorkflow calls that used
|
||||
// to live in cmd/start.go. It does not assign the stripe-go API key: start
|
||||
// sets it once at boot through UseKey, before any goroutine runs. Each
|
||||
// workflow start is independently non-fatal: a failure is logged and boot
|
||||
// continues, matching today's convention.
|
||||
// Startup terminates a retired webhook-processor workflow, sweeps unfinished
|
||||
// webhook events and outbox entries into their per-item workflows, and
|
||||
// ensures the invoice sweep schedule. It does not assign the stripe-go API key: start
|
||||
// sets it once at boot through UseKey, before the listener and before any
|
||||
// goroutine that reads the key. Each workflow start is independently
|
||||
// non-fatal: a failure is logged and the next step runs.
|
||||
func (Adapter) Startup(ctx context.Context, c client.Client, taskQueue string, database *sql.DB, logger *slog.Logger) error {
|
||||
// Webhook events are processed one workflow per event, started by the
|
||||
// endpoint. The polling processor that preceded them (through 2026-09)
|
||||
|
||||
@@ -187,22 +187,23 @@ Sections 1 to 7 are one commit each, and their items are that commit's parts; se
|
||||
|
||||
## 7. Docs
|
||||
|
||||
- [ ] 7.1 `docs/production-deployment.md` section 7, as design D20 writes it: the steps in the order `start` runs them, configuration validation first; what serves before Temporal and the identity provider answer; sign-in at 503 until a usable discovery document is published; Temporal retried with an Error log after `temporal-connect-timeout`; a configured Stripe or Discourse needs `temporal-host`. Placeholders only, and no deployment's names.
|
||||
- [ ] 7.2 `docs/environment-reference.md`: the `temporal-connect-timeout` row says what the key now sets; the `temporal-host` row says that a configured Stripe or Discourse needs it.
|
||||
- [ ] 7.3 `docs/fedwiki-setup.md`, "Serving authorization": the paragraph about an unreachable console says that the console answers the ask as soon as it serves, whatever Temporal, the identity provider and the session store do, and keeps the bound for a console that is down.
|
||||
- [ ] 7.4 `docs/identity-provider-setup.md` and `internal/auth/README.md`: "refuses to start" and "fails startup" become what the code does: the console serves and keeps sign-in unavailable, logging at Error, until the provider publishes the endpoint. The logout section says what sign-out does without an end-session endpoint, and the provider setup says that a document with the configured issuer and no `end_session_endpoint` leaves sign-in unavailable but does not stop session re-derivation.
|
||||
- [ ] 7.5 `docs/temporal-authorization-setup.md`: one paragraph on the token URL: an internal address keeps the Temporal connection off the TLS proxy; the console no longer needs it before it serves.
|
||||
- [ ] 7.6 `docs/stripe.md` and `docs/discourse-setup.md`: one sentence each that `temporal-host` is required, and why. `docs/building-an-integration.md` section 6 gains a `NeedsTemporal` bullet: it goes on a key of the RequiredGroup of an integration that receives events processed only on Temporal, and boot refuses the configured group without `temporal-host`.
|
||||
- [ ] 7.7 Four texts that still describe the boot section 6 replaced:
|
||||
- [x] 7.1 `docs/production-deployment.md` section 7, as design D20 writes it: the steps in the order `start` runs them, configuration validation first; what serves before Temporal and the identity provider answer; sign-in at 503 until a usable discovery document is published; Temporal retried with an Error log after `temporal-connect-timeout`; a configured Stripe or Discourse needs `temporal-host`. Placeholders only, and no deployment's names. The quickstart note in `README.md` and the first-boot note in `docs/identity-provider-setup.md` describe the same boot and change with it. `docs/plan-management.md` states that a grant with **Valid until** is refused while Temporal is not connected.
|
||||
- [x] 7.2 `docs/environment-reference.md` (the two immediate-sync rows and `docs/stripe.md`'s first-run sentence say "once the console connects to Temporal" instead of "at startup" or "at boot"): the `temporal-connect-timeout` row says what the key now sets; the `temporal-host` row says that a configured Stripe or Discourse needs it.
|
||||
- [x] 7.3 `docs/fedwiki-setup.md`, "Serving authorization": the paragraph about an unreachable console says that the console answers the ask as soon as it serves, whatever Temporal, the identity provider and the session store do, and keeps the bound for a console that is down.
|
||||
- [x] 7.4 `docs/identity-provider-setup.md` and `internal/auth/README.md`: "refuses to start" and "fails startup" become what the code does: the console serves and keeps sign-in unavailable, logging at Error, until the provider publishes the endpoint. The logout section says what sign-out does without an end-session endpoint, and the provider setup says that a document with the configured issuer and no `end_session_endpoint` leaves sign-in unavailable but does not stop session re-derivation.
|
||||
- [x] 7.5 `docs/temporal-authorization-setup.md`: one paragraph on the token URL: an internal address keeps the Temporal connection off the TLS proxy; the console no longer needs it before it serves.
|
||||
- [x] 7.6 `docs/stripe.md` and `docs/discourse-setup.md`: one sentence each that `temporal-host` is required, and why. `docs/building-an-integration.md` section 6 gains a `NeedsTemporal` bullet: it goes on a key of the RequiredGroup of an integration that receives events processed only on Temporal, and boot refuses the configured group without `temporal-host`.
|
||||
- [x] 7.7 Four texts that still describe the boot section 6 replaced:
|
||||
1. `internal/integrations/fedwiki/workflows/README.md` (around line 210): "the process exits with status 1 when any is open or the list cannot be read. Let an open run finish on the previous release, or terminate it, before starting this release." becomes "...lists the running executions of those types. While any is open the worker does not start: the console serves, every Temporal-backed action answers that it is not available, and the list is read again every fifteen seconds. A list that cannot be read is retried on a new connection. Let an open run finish on the previous release, or terminate it; the worker starts once none is open."
|
||||
2. `internal/integrations/fedwiki/fedwiki.go` (around line 452): "listRunsError is the refusal to start when the list cannot be read" becomes "listRunsError is the precondition's failure to read the list: it reports workflows.ErrCannotCheck, so the connector closes the client and connects again."
|
||||
3. `internal/integrations/fedwiki/fedwiki.go` (around lines 469 and 470): "A list that cannot be read refuses too" becomes "A list that cannot be read is returned as workflows.ErrCannotCheck, since it cannot say that none is open."
|
||||
4. `internal/integrations/stripe/stripe.go` (around line 216): `Startup`'s doc says the key is set "before any goroutine runs"; it becomes "...through UseKey, before the listener and before any goroutine that reads the key."
|
||||
- [ ] 7.8 No code changes other than the comments of 7.7; `make test` passes.
|
||||
Also comment-only: `openRunsError` and `CheckBeforeWorker` in `fedwiki.go` name the worker's start, not the process start, as what is refused; `Startup`'s non-fatal sentence in `stripe.go` drops "boot continues", and its doc describes what the body does.
|
||||
- [x] 7.8 No code changes other than the comments of 7.7; `make test` passes.
|
||||
|
||||
## 8. Landing
|
||||
|
||||
- [ ] 8.1 With the test stack up, `make test` passes (it runs `make lint` first), item 6.11's race run passes (after `test/reset-test-db.sh`, with `-p 1`; narrowed as the race baseline says, if it came to that), and `openspec validate serve-before-dependencies --strict` passes.
|
||||
- [x] 8.1 With the test stack up, `make test` passes (it runs `make lint` first), item 6.11's race run passes (after `test/reset-test-db.sh`, with `-p 1`; narrowed as the race baseline says, if it came to that), and `openspec validate serve-before-dependencies --strict` passes.
|
||||
- [ ] 8.2 Post a comment on #141 under CONTRIBUTING.md's rules for forge text, and close it. The comment states that the console now opens its port once the database steps have run and answers the certificate ask whatever Temporal, the identity provider and the session store do; that sign-in answers 503 until a usable discovery document is published, which a background loop fetches from the identity provider, and recovers without a restart; that Temporal is retried for the life of the process and `temporal-connect-timeout` now sets when the log turns to Error; that the Stripe webhook answers 503 until Temporal connects, so Stripe redelivers; that the Stripe library's key is set at boot and Checkout and the plan moves read the mode boot derives; that a configured Stripe or Discourse without `temporal-host` no longer starts, which a deployment changes its configuration for; that a bounded grant, or a new end date on a grant, is refused while Temporal is not connected; and that `start` exits non-zero when it cannot serve. It names the commits of sections 1 to 7.
|
||||
- [ ] 8.3 At archive, two edits a delta cannot make are made in the same commit: the Purpose of `openspec/specs/temporal-connection/spec.md` stops saying that start retries within `temporal-connect-timeout` and exits non-zero, and says that start retries for the life of the process; the Purpose of `openspec/specs/error-pages/spec.md` names the 503 as well as the 404 and the 500.
|
||||
- [ ] 8.4 Nothing in this change touches a production host. The candidate issues in the proposal's "Out of scope" (graceful shutdown, a backstop for grant expiry) and the session that cannot be decoded (design F27) are handed to the maintainer; none is filed without his go.
|
||||
|
||||
Reference in New Issue
Block a user