member-console/test/AGENTS.md

# test/ — instructions for LLM agents

If you are an automated agent working in this repository, read this before
starting Docker. The test stack uses fixed default host ports; multiple
worktrees running it concurrently will collide unless each generates its own
isolated configuration.

All commands in this doc assume your working directory is `test/` —
that's where compose, the env file, and the binary's expected secret
paths all resolve from.

> **All credentials in the test stack are throwaway test-only values** — the Keycloak
> `admin/admin`, seeded client secrets, database passwords, and generated `.env`
> secrets exist only for local development. Never reuse any of them in a real deployment.

## Required first step in any worktree

```
cd test
./bootstrap-stack.sh
```

The script:

- Derives a slug from this worktree's directory name (override with
  `./test/bootstrap-stack.sh my-slug`).
- Allocates a deterministic port slot via `FNV-32(slug) % 200`. Probes for
  collisions and advances if any candidate port is already bound on the host.
- Copies `test/secrets/` from the main worktree if missing locally (this
  directory is gitignored, so fresh worktrees do not inherit it).
- Writes `test/.env` with `COMPOSE_PROJECT_NAME`, `COMPOSE_PROFILES`, host port
  assignments, and `MC_*` env vars that override `test/mc-config.yaml` via Viper.
- Prints the URL/port table the developer should use, and which integration
  profiles are active.
- **Idempotent**: re-running with `test/.env` already in place is a no-op
  that reprints the summary.

## Which integrations the stack composes

The stack ships **core only**: Postgres, Valkey, Keycloak, Temporal. Every
integration service is behind a compose profile named after it, and
`COMPOSE_PROFILES` in `test/.env` is the knob that selects them:

| Value | Composes |
|-------|----------|
| *(empty, the default)* | core services only |
| `fedwiki` | the wiki farm chain (`fedwiki-init`, `fedwiki-render`, `fedwiki`) and the Caddy TLS proxy on `443` |
| `discourse` | the live forum, its Postgres, and its Redis |
| `fedwiki,discourse` | a full demo composition |

Docker Compose reads the variable natively, so `up`, `ps`, and `down` all
resolve the same composition without flags, and `seed-stack.sh` /
`teardown-stack.sh` follow along automatically.

**Composing is not configuring.** Turning a profile on starts containers;
the app still ignores that integration until you uncomment its `MC_*` block in
the same `test/.env`. Both halves are needed for a FedWiki or Discourse flow.

Bake a composition in at generation time when you know you want it:

```
COMPOSE_PROFILES=fedwiki ./bootstrap-stack.sh
```

To change it later, edit the one `COMPOSE_PROFILES` line in `test/.env` and
re-run `docker compose up -d`. This is the **only** hand-edit `test/.env`
sanctions (see Constraints): re-running bootstrap would not pick up a change,
and a teardown would drop the database.

## After bootstrap

```
docker compose up -d
set -a; . .env; set +a
go run .. start --config mc-config.yaml
```

`keycloak-seed` (creates realm users) runs automatically as a one-shot on
`up`. With the `fedwiki` profile selected, `fedwiki-render` does too
(it resolves real KC UUIDs and templates `seed/fedwiki/*.tpl` into
`testdata/fedwiki/`), and `fedwiki` waits for both to complete before
starting. Keycloak silently regenerates the `id` field on POST /users,
so FedWiki identity cannot be pinned at commit time — it's rendered
after each realm seed.

The member-console binary must run with `test/` as its working directory
because `mc-config.yaml` references `secrets/stripe-*` via relative paths.
Compose auto-loads `.env` from the current directory, so no
`-f`/`--env-file` flags are needed.

Use the URLs printed by bootstrap, not the defaults from
`test/mc-config.yaml` — the YAML still says `port: 8081`, but the running
process listens on whatever `MC_PORT` is in `test/.env`.

## Walking the operator UI with data

The default workflow produces a clean, empty environment — good for fresh-user
flows. To populate the operator panel (products, plan ladder, entitlement set,
one active grant) for UX walkthroughs or operator-panel exploration, run the
opt-in demo seeder from the repo root after `bootstrap-stack.sh`:

```
./test/seed-demo.sh
```

Idempotent — re-running is a no-op. Person rows for bob/carlos/diana are
seeded through the app's own signup path, each carrying as its OIDC subject
the literal `sub` the Keycloak seed pins for that username; Alice is left to
lazy-create on her first login. See
[`seed/member-console-demo/README.md`](seed/member-console-demo/README.md) for
the catalog and rationale.

For sample data at deployment scale (thousands of members, subscriptions,
invoices — used for screenshots), run `test/mockshot-seed.sql` and reverse it
with `test/mockshot-nuke.sql`.

## Seed identity

The database generates every ID (Decision 30), so a seed never supplies one;
it addresses the rows it owns by `key` (grammar `^[a-z][a-z0-9_]*$`) and
upserts on that key, which is what makes re-running a seed a no-op. The
mockshot pair finds its rows by the `mockshot_` organization key prefix and
the `mockshot:` OIDC subject prefix, and the nuke deletes from exactly those
predicates. The one place an ID is written as a literal is
[`seed/keycloak/seed-keycloak.sh`](seed/keycloak/seed-keycloak.sh), which pins
the four demo users' Keycloak ids; the demo seeder mirrors those ids as its
persons' OIDC subjects.

## Running the Go test suite

```
make test        # from the repo root, with the stack up
```

That target runs `test/reset-test-db.sh` and then `go test -count=1 -p 1
./...` with `test/.env` sourced. The reset drops, recreates, and migrates the
tests' own databases, so every run starts from migrations only. `-p 1` runs
packages one at a time: several of them commit rows others read (an org
type's default plan ladder, a provider's group mappings), and interleaving
those makes roughly half of parallel runs fail somewhere. `-count=1` is there
because Go's test cache keys on code and environment, not database state, so
after a reset a cached `ok` would describe a database that no longer exists.
The whole suite takes about 26 seconds.

The stack's Postgres carries three databases:

| Database              | Who writes it                          | Env var              |
|-----------------------|----------------------------------------|----------------------|
| `member_console`      | the running app, seeders, browser walkthroughs | `MC_DB_DSN`   |
| `member_console_test` | every DB-backed package test           | `TEST_DATABASE_URL`  |
| `member_console_e2e`  | `test/e2e/plan-management`             | `E2E_DATABASE_URL`   |

The split is what makes the suite trustworthy, and each boundary earns its
place. Tests are off the app database because a running `member-console
start` executes boot reconcilers (domains backfill, orphaned-placement
sweep, entitlement materialization) that adopt leftover test fixtures. The
in-process e2e suite is off the package-test database because it commits an
org type's default plan ladder, which `internal/provisioning` asserts the
absence of. Both were live failure modes, not hypotheticals.

Running `go test ./...` directly still works, but skips both the reset and
the serialization, so it is flaky by construction. Prefer `make test`, and
reach for a bare `go test ./<pkg>` only when iterating on one package.

Migrating during the reset is deliberate. Each package calls
`db.RunMigrations` itself and goose takes no advisory lock, so against a
genuinely empty database a dozen parallel packages would race to create the
same schemas. Pre-migrating makes each of those calls a no-op.

One caveat: `./...` includes `test/e2e/operator-walkthroughs`, which drives a
browser against the live app. Those skip wholesale when no app is listening
(the common case for `make test`), but with an app up they run against the
**app** database and need it seeded — see the next section. A walkthrough
failure there is about that database's contents, not about the isolation the
reset provides.

If `reset-test-db.sh` reports a missing key, `test/.env` predates the split:
`cd test && rm .env && ./bootstrap-stack.sh`. Containers and volumes survive.

## Browser walkthroughs (e2e) and stack age

`test/e2e/operator-walkthroughs/` runs against whatever state the stack DB
has accumulated, and adapts rather than assumes:

- Walkthroughs **skip** (never fail) when their environment or subject is
  missing — app down, a profile-gated integration service unreachable, no org
  with the state they need. A skip message always names the missing
  precondition, including which profile would supply it.
- State-dependent walkthroughs (e.g. grant-extend) **scan** for a valid
  subject instead of trusting listing order, so orgs left behind by other
  walkthroughs or purchase-flow testing can't poison a run.

Skips are honest but they are still coverage gaps: for a full-exercise run,
respin first (`make stack-fresh` from the repo root, then boot the app and
run `./test/seed-demo.sh`). These walkthroughs drive the live app over a
browser, so they legitimately read the **app** database and stay subject to
its accumulated state; a respin is their freshness mechanism. The Go suite
no longer needs one — `make test` resets its own databases.

## Tearing down

```
./teardown-stack.sh
```

Brings the compose stack down with volumes, removes `test/.env`,
`test/secrets/`, and `test/testdata/`. Returns the worktree to a clean
state for the next experiment. It enumerates the profiles `compose.yaml`
declares rather than reading `COMPOSE_PROFILES`, so gated services come down
even if the knob was turned off after they started, and it reclaims ownership
of root-owned runtime data (Postgres, Temporal, Caddy, Discourse's Postgres)
through a short-lived container before deleting it.

## Keycloak user seeding

`seed/keycloak/seed-keycloak.sh` creates users via `POST /admin/realms/{realm}/partialImport`, not `POST /admin/realms/{realm}/users`. Keycloak 26.x silently drops the `id` field on the latter (server-side ID assignment for least-privilege admin operations); `partialImport` preserves pinned IDs and is the supported escape hatch. See OpenSpec change `2026-05-11-keycloak-id-pinning-fix` for the empirical confirmation and design rationale.

## Temporal namespace

The compose stack sets `SKIP_DEFAULT_NAMESPACE_CREATION=true` on the
`temporal` service, so a one-shot `temporal-seed` service registers the
`default` namespace via the internal frontend on first `up`. If
member-console fails with `Namespace default is not found`, the seed
service didn't run yet — `docker compose ... up -d temporal-seed` and
re-check `docker compose ... logs temporal-seed`.

## Stripe webhook forwarding (local activation)

Stripe cannot POST to `localhost`, so subscription activation (the
`checkout.session.completed` / `customer.subscription.created` →
pool-provision path) only fires locally if you forward events:

```
stripe login                                              # one-time, interactive
stripe listen --forward-to localhost:$MC_PORT/webhooks/stripe
```

`stripe listen` prints a signing secret; `test/secrets/stripe-webhook-secret`
must equal it. The app's restricted key (`rk_`) cannot run `stripe listen`
(it lacks `rak_stripecli_session_write`), hence the `stripe login` session.
Replay a past event with `stripe events resend <evt_id>`. Forwarding is
account-global — see the parallel-stack constraint below.

The app's restricted key (`rk_`) must include **read** scope for *Subscriptions*
(plus *Customers* and *Prices*). Subscription fulfillment refetches authoritative
state from the Stripe API rather than trusting the webhook payload (OpenSpec
change `stripe-fulfillment-reconcile`), so a missing read scope surfaces as a
`403` during reconcile — not at checkout.

## Constraints

- **Do not run Stripe webhook scenarios in parallel across stacks.** All
  stacks share the same Stripe test account via `test/secrets/stripe-*`.
  Webhook delivery to one stack is global to that account; running two
  stacks through webhook flows simultaneously gives undefined results.
- **Do not edit `test/.env` by hand**, with one exception: the
  `COMPOSE_PROFILES` line and the commented-out `MC_FEDWIKI_*` /
  `MC_DISCOURSE_*` opt-in blocks are meant to be edited in place. For
  anything else — ports above all — re-run bootstrap after deleting the file.
- **Do not point `TEST_DATABASE_URL` or `E2E_DATABASE_URL` at the app
  database.** That reinstates every pollution vector the split closes, and
  `reset-test-db.sh` refuses to drop `member_console` precisely so a
  mistake here fails loudly instead of destroying the running stack's data.
- **Do not remove the `${VAR:-default}` fallbacks from `compose.yaml`.**
  They preserve the single-stack default workflow when no `.env` is
  present.
- **The HTTPS farm (Caddy) binds host `443`, so only one stack at a time
  can serve it** — but only among stacks that select the `fedwiki` profile;
  a core-only stack binds nothing there. Browsers must reach FedWiki sites
  over HTTPS because
  `wiki-security-social` sets Secure cookies (see
  [[project_fedwiki_security_stack]]); member-console's admin API still uses
  plain HTTP on `FEDWIKI_PORT` (bearer token, no cookies), so provisioning is
  unaffected and parallel stacks still work for everything except browser
  login. Caddy serves a wildcard cert from its **internal CA** — for manual
  browsing, trust the root once:
  `docker compose cp caddy:/data/caddy/pki/authorities/local/root.crt ./caddy-root.crt`.
  Override the port with `FEDWIKI_HTTPS_PORT` only if you also give
  member-console a matching site port (`buildSiteURL` emits no port).
- **FedWiki OAuth needs Keycloak reachable at the same host:port from both
  the browser and the fedwiki container.** `config.json`'s `oauth2_discoveryUrl`
  is rendered to `${KC_HOSTNAME}:${KEYCLOAK_PORT}` (browser-facing), and the
  fedwiki service gets `extra_hosts: ${KC_HOSTNAME}:host-gateway` so the
  container resolves that name to the host. Don't drop either half.

## What `bootstrap-stack.sh` writes to `test/.env`

```
COMPOSE_PROJECT_NAME=<slug>
SLOT=<0..199>
COMPOSE_PROFILES=            # composition knob; empty = core stack only

POSTGRES_PORT, VALKEY_PORT, KEYCLOAK_PORT,
TEMPORAL_PORT, TEMPORAL_UI_PORT, FEDWIKI_PORT, DISCOURSE_PORT

KC_HOSTNAME, DISCOURSE_HOSTNAME, MC_BASE_URL, TEMPORAL_UI_URL

MC_PORT, MC_DB_DSN, MC_VALKEY_ADDR,
MC_OIDC_IDP_ISSUER_URL, MC_TEMPORAL_HOST,
MC_TEMPORAL_OAUTH_TOKEN_URL

#MC_FEDWIKI_*                # commented-out opt-in, see below
#MC_DISCOURSE_*              # commented-out opt-in, see below

TEST_DATABASE_URL, E2E_DATABASE_URL, POSTGRES_ADMIN_DSN
```

Compose reads the first three blocks. Viper reads the `MC_*` block via the
existing `SetEnvPrefix("MC")` + `AutomaticEnv()` configuration in
`cmd/root.go`. The last block is read by `test/reset-test-db.sh` and the Go
test suites; `POSTGRES_ADMIN_DSN` points at the cluster's maintenance
database, the only place a `DROP DATABASE` can run from.

Both integrations' `MC_*` blocks ship commented out. Each is a
required-together config group (farm URL + admin token; forum base URL + API
key file), so uncommenting half of one fails boot validation on purpose. Port
assignments for both are always written, whether or not their profile is on —
they are harmless when nothing consumes them, and it keeps flipping the knob a
one-line edit.