cgalo5758
c4bb1ba585
Sweep stranded pending claims at boot and on a Temporal schedule while preserving evidence-based abandonment semantics. Apply occupancy and name-policy checks to carves by operator-root owners without affecting direct operator placements.
264 lines
14 KiB
Markdown
264 lines
14 KiB
Markdown
---
|
||
title: "Testing"
|
||
audience: [developer]
|
||
summary: "The test taxonomy (unit, DB-gated integration, Stripe-mocked, HTTP render, lint, e2e, OpenSpec validation), how to run each, and conventions for new tests."
|
||
---
|
||
|
||
# Testing guide
|
||
|
||
How testing works in member-console: the kinds of tests, what each covers, how
|
||
to run them, and the conventions to follow when writing new ones.
|
||
|
||
## TL;DR
|
||
|
||
```bash
|
||
# Fast feedback — pure unit tests only (integration tests skip without a DB):
|
||
go test ./...
|
||
go vet ./...
|
||
|
||
# Full suite — needs a migrated Postgres (use the test stack):
|
||
cd test && ./bootstrap-stack.sh && docker compose up -d # one-time per worktree
|
||
set -a && . test/.env && set +a
|
||
TEST_DATABASE_URL="$MC_DB_DSN" go test ./... # from repo root
|
||
|
||
# Route/template integrity + render discipline:
|
||
go run . lint # 65 routes / URL refs resolve (cmd/lint.go)
|
||
make lint-templates # no handler bypasses SafeTemplates
|
||
```
|
||
|
||
`go test ./...` is always safe to run: every database- or Stripe-backed test
|
||
**skips itself** when `TEST_DATABASE_URL` is unset.
|
||
|
||
## The `test/` directory — the local stack
|
||
|
||
Most Go test files sit next to the code they cover (`internal/.../<name>_test.go`).
|
||
`test/` is the exception, and serves two roles: it's the **integration/e2e
|
||
environment** — a Dockerized stack of every backing service the app needs
|
||
(Postgres, Valkey, Keycloak/OIDC, Temporal, FedWiki) that the database-gated
|
||
tests (types 2–3) and the e2e suites (type 6) run against — **and** it houses the
|
||
higher-level **end-to-end Go test suites** under `test/e2e/`.
|
||
|
||
Its role:
|
||
|
||
- **Provides the backing services.** `test/compose.yaml` brings up Postgres,
|
||
Valkey, Keycloak, Temporal, and FedWiki, plus one-shot seeders (realm users,
|
||
FedWiki identity, the Temporal default namespace).
|
||
- **Isolates each worktree.** `./bootstrap-stack.sh` derives collision-free host
|
||
ports and writes `test/.env` (`COMPOSE_PROJECT_NAME`, port assignments, and the
|
||
`MC_*` config overrides), so several worktrees can run their own stacks
|
||
concurrently without colliding.
|
||
- **Is where `TEST_DATABASE_URL` comes from.** After bootstrap, `test/.env`
|
||
defines `MC_DB_DSN` (Postgres on this worktree's port); integration tests point
|
||
at it via `TEST_DATABASE_URL="$MC_DB_DSN"`.
|
||
- **Houses the end-to-end Go suites** under `test/e2e/`: `plan-management/`
|
||
(in-process `httptest` over the real handlers + a migrated DB) and
|
||
`operator-walkthroughs/` (Go tests that drive a browser against the running
|
||
stack via `MC_BASE_URL`; `WALKTHROUGH_HEADFUL=1` to watch). Both run under
|
||
`go test` (see type 6).
|
||
- **Holds the stack's run-only assets:** `mc-config.yaml`, `secrets/`
|
||
(gitignored Stripe keys + webhook secret), the opt-in demo seeder
|
||
(`./test/seed-demo.sh` + `seed/`), and `teardown-stack.sh` (compose down +
|
||
removes `.env`, `secrets/`, `testdata/`).
|
||
|
||
Don't edit `test/.env` by hand — re-run `bootstrap-stack.sh`. The full contract
|
||
(idempotency, the account-global Stripe webhook constraint, teardown) lives in
|
||
[`test/AGENTS.md`](../test/AGENTS.md); read it before running the stack.
|
||
|
||
## The kinds of tests
|
||
|
||
### 1. Pure unit tests — no external dependencies
|
||
|
||
Plain Go tests of pure logic: parsing, classification, formatting, guard
|
||
conditions. They run on every `go test` invocation, need nothing external, and
|
||
are fast.
|
||
|
||
- **Cover:** webhook payload parsing, status classification, value mapping,
|
||
small helpers, input guards.
|
||
- **Examples:** `internal/fulfillment` `TestClassifyStatus`,
|
||
`TestUnixToNullTime`; `internal/workflows/stripe` `TestCheckoutSessionPayloadParsing`,
|
||
`TestInvoicePayloadParsing`, `TestCardStringField_*`.
|
||
- **Write one when:** the behavior is a function of its inputs with no DB/HTTP/
|
||
Stripe dependency. Prefer pushing logic into such functions so it can be
|
||
tested this way.
|
||
|
||
### 2. Database-gated integration tests
|
||
|
||
Run against a real, migrated Postgres. They exercise queries, constraints
|
||
(e.g. the `pool_provision_ladders` GiST exclusion, `chk_pool_provisions_source`),
|
||
transactions, and cross-schema flows that unit tests can't reach.
|
||
|
||
- **Gated on `TEST_DATABASE_URL`** — skip when unset, so they never block a
|
||
driverless `go test ./...`. Driver is **pgx** (`_ "github.com/jackc/pgx/v5/stdlib"`).
|
||
- **Cover:** the entitlement graph, provisioning, fulfillment reconcile, webhook
|
||
projection, materialization, operator queries.
|
||
- **Examples:** `internal/provisioning` (`TestAutoProvision…`), `internal/fulfillment`
|
||
(`TestReconcile_*`), `internal/entitlements/materialize_test.go`,
|
||
`internal/workflows/stripe` (`*_Integration`, transition/product-catalog tests).
|
||
|
||
Two sub-patterns for the `testDB(t)` helper — know which one you're following:
|
||
|
||
| Sub-pattern | `testDB` behavior | Used by |
|
||
|---|---|---|
|
||
| **Self-migrating** | calls `db.RunMigrations(...)` to build the schema from scratch | `provisioning`, `fulfillment`, `entitlements/materialize`, `workflows/entitlements`, `server/operator_plan_ladders` |
|
||
| **Assume-migrated** | just `sql.Open` against an already-migrated DB | `workflows/stripe` |
|
||
|
||
> **Use the canonical source list.** Self-migrating tests should build their
|
||
> sources via `migrate.Sources()` rather than hand-rolling a subset. Each
|
||
> source migrates against its own goose ledger (`goose_db_version_<name>`),
|
||
> so source order doesn't affect version numbering — but `core` must come
|
||
> first, since integration schemas' FKs point at `core` and can't apply
|
||
> before it exists. See `docs/database-management.md`.
|
||
|
||
**Fixtures:** `provisioning.AutoProvision(ctx, db, claims)` bootstraps a fresh
|
||
org + default pool + billing account in one call — the cheapest way to get a
|
||
clean tenant. Layer the plan graph (entitlement set → product → price → ladder →
|
||
tier, plus stripe mappings) on top with the sqlc `Create*` helpers. Use unique
|
||
suffixes (`uuid.New().String()[:8]`) so repeated runs against the shared stack DB
|
||
don't collide; rows are left behind in the disposable test database.
|
||
|
||
### 3. Stripe-mocked tests
|
||
|
||
A specialization of the integration tests: the Stripe SDK is pointed at an
|
||
in-process fake so fulfillment's "refetch from the Stripe API" path runs
|
||
deterministically and offline, with **no live Stripe account**.
|
||
|
||
- **Helper:** `internal/stripetest` — `MockBackend{Sub, List, Err}` implements
|
||
`stripe.Backend`; `stripetest.Install(m)` swaps the global backend + a dummy
|
||
key and returns a restore func (`t.Cleanup(stripetest.Install(m))`).
|
||
- **Cover:** `fulfillment.ReconcileSubscription` scenarios (empty-payload
|
||
provisioning, idempotent replay, ordering, cancellation, multi-ladder guard)
|
||
and the webhook→`plan-transitions` end-to-end tests.
|
||
- **Gotcha:** `Install` mutates **global** stripe-go state, so these tests must
|
||
**not** call `t.Parallel()`.
|
||
|
||
### 4. HTTP handler / render tests
|
||
|
||
Exercise handlers through `net/http/httptest` plus the real `SafeTemplates`
|
||
renderer, session manager (`scs`), and CSRF middleware.
|
||
|
||
- **Cover:** rendered output, template wiring, auth guards (e.g. 401 for
|
||
unauthenticated), CSRF behavior.
|
||
- **Examples:** `internal/server/member_upgrade_render_test.go`
|
||
(`TestMemberPlansUpgradeControlRendering`, `TestDashboardCheckoutBanner`,
|
||
`TestGetPlansUnauthenticatedReturns401`); `internal/middleware/tests/decompress_test.go`.
|
||
|
||
### 5. Route & template lint (static)
|
||
|
||
Not Go tests, but part of the green-bar definition:
|
||
|
||
- **`go run . lint`** (`cmd/lint.go`) — operator-UI integrity: every registered
|
||
route and templated URL reference resolves. Reports e.g. `lint: ok (65 routes, 29 URL refs)`.
|
||
- **`make lint-templates`** — guards that no handler bypasses `SafeTemplates` by
|
||
calling `.ExecuteTemplate(w, …)` directly (CSP/render discipline; see
|
||
`internal/server/render.go`).
|
||
|
||
### 6. End-to-end tests
|
||
|
||
Exercise whole flows against a running stack. Three flavors:
|
||
|
||
- **Go HTTP-level e2e** — `test/e2e/plan-management/`: an in-process `httptest`
|
||
harness over the real handlers + a migrated DB (`TEST_DATABASE_URL`). Runs under
|
||
`go test`; covers operator plan/ladder/grant flows end-to-end at the HTTP layer.
|
||
- **Go browser walkthroughs** — `test/e2e/operator-walkthroughs/`: Go tests that
|
||
drive a real browser against the running stack (reads `MC_BASE_URL` from
|
||
`test/.env`; set `WALKTHROUGH_HEADFUL=1` to watch). One walkthrough per operator
|
||
route group, each asserting the four-bullet UI contract (empty/invalid → 422,
|
||
valid → success + DOM update, …). Skips unless the stack is up and seeded.
|
||
- **Manual / agent-driven live loop** — the `chrome-devtools` MCP (browser on
|
||
`127.0.0.1:9222` via `chrome-debug &`), the Stripe MCP, and `stripe listen`
|
||
webhook forwarding, for flows not yet codified — e.g. a real Stripe Checkout →
|
||
`checkout.session.completed` → reconcile → the member UI reflecting the new
|
||
plan. Not part of `go test`.
|
||
|
||
Setup for all three lives in [`test/AGENTS.md`](../test/AGENTS.md) (worktree stack
|
||
contract; the "Stripe webhook forwarding" section for the live loop).
|
||
|
||
### 7. Live DNS verification (domain claims, manual)
|
||
|
||
Some of the domains registry can only be proven against a real zone: that a
|
||
published challenge record activates a claim, and that publishing the challenge
|
||
*prefix* with the wrong token latches `evidence_at` (exempting the claim from the
|
||
abandonment ledger) without verifying. DB tests cover the branching; only real DNS
|
||
covers the resolver path.
|
||
|
||
You need a **disposable zone you control** — any domain whose records you can add
|
||
and remove, at any registrar or DNS host with an API. Keep it separate from
|
||
anything the deployment actually serves, and never commit credentials for it;
|
||
supply them from the environment at the moment you use them. If the harness blocks
|
||
outbound HTTP from the shell (the context-mode routing rules do), drive the DNS
|
||
provider's API from a sandboxed script instead.
|
||
|
||
The loop is: start a claim in the console, read its token, publish
|
||
`_member-console-challenge.<name>` as TXT, wait for a probe, assert, then delete the
|
||
record. Two cases are worth proving because they diverge — a TXT carrying the
|
||
challenge **prefix with a wrong token** must latch evidence *without* activating the
|
||
claim, while the **correct token** must activate it.
|
||
|
||
**The gotcha that will cost you ten minutes.** If the zone has a **wildcard record**
|
||
(`*.example` CNAME/ALIAS/TXT), resolvers synthesise it for any challenge name that
|
||
has no explicit record — so the probe reports "Found, but doesn't match" against
|
||
whatever the wildcard points at, long before you did anything wrong. Publishing an
|
||
explicit record stops the synthesis (RFC 4592 closest-encloser), but any cached
|
||
wildcard answer still wins until its TTL expires, so probes keep reporting
|
||
`mismatch` for minutes after DNS is already correct. Before concluding the prober is
|
||
broken, compare authoritative against cached:
|
||
|
||
```bash
|
||
dig +short @<authoritative-ns> TXT _member-console-challenge.<name> # what the zone says
|
||
dig +noall +answer TXT _member-console-challenge.<name> # what the console sees
|
||
```
|
||
|
||
Clean up the records you added, and leave alone any challenge record that backs a
|
||
claim you intend to keep verified.
|
||
|
||
### 8. OpenSpec validation (change workflow)
|
||
|
||
`openspec validate <change>` is not a Go test but is part of the definition of
|
||
done for a spec-driven change: it checks delta specs are well-formed (MODIFIED
|
||
requirement headers match the main spec, every requirement has a `####` scenario,
|
||
etc.). Run it before archiving a change.
|
||
|
||
## Choosing what to write
|
||
|
||
```
|
||
Is the behavior a pure function of inputs? → unit test (1)
|
||
Does it touch the DB / constraints / multi-table? → DB integration test (2)
|
||
Does it call the Stripe API? → add a stripetest mock (3)
|
||
Is it an HTTP handler / rendered page? → httptest render test (4)
|
||
Is it route/template wiring or render discipline? → go run . lint / lint-templates (5)
|
||
Is it a whole user flow (HTTP or operator UI)? → e2e (6): a test/e2e/ Go suite
|
||
Is it a flow only real infra can prove (live Stripe)? → the live MCP loop (6), see test/AGENTS.md
|
||
Does it depend on what a resolver actually returns? → live DNS on a zone you control (7)
|
||
```
|
||
|
||
Prefer the cheapest test that genuinely covers the behavior. Reach for an e2e
|
||
suite or the live loop (6) to *prove a flow works end-to-end*, then capture the
|
||
regression-proofing parts as (2)/(3) so they run in CI without a live account.
|
||
|
||
## Conventions & gotchas
|
||
|
||
- **Gate DB/Stripe tests on `TEST_DATABASE_URL`** and `t.Skip` when unset — keep
|
||
`go test ./...` green with no infrastructure. (This is the one convention; the
|
||
legacy `DB_DSN` + `postgres`-driver variant has been retired.)
|
||
- **Match `cmd/migrate.go` source order** in self-migrating tests (see above).
|
||
- **Don't trust webhook payloads in tests** the way production doesn't: a Stripe
|
||
test should drive behavior through `stripetest.MockBackend` returning canned
|
||
*API* state, not by hand-building the event payload's fields.
|
||
- **No `t.Parallel()` in Stripe-mocked tests** — they share global SDK state.
|
||
- **Unique fixture ids** for tests that write to the shared stack DB; tear the
|
||
stack down with `test/teardown-stack.sh` when you want a clean slate.
|
||
- **A wildcard CNAME shadows challenge lookups** — see (7). A domain-verification
|
||
probe reporting `mismatch` against a record you just published is usually a
|
||
cached wildcard, not a defect.
|
||
- **Fixture pollution breaks the browser walkthroughs.** DB-gated suites seed rows
|
||
they never clean up, so repeated full-suite runs against the shared stack can
|
||
grow `core.plan_ladders` / `core.org_types` into the thousands until
|
||
`test/e2e/operator-walkthroughs` times out rendering them. Check row counts
|
||
before believing a walkthrough regression.
|
||
|
||
## Related docs
|
||
|
||
- [`test/AGENTS.md`](../test/AGENTS.md) — the test stack (ports, webhook forwarding, demo seed).
|
||
- [`docs/database-management.md`](database-management.md) — migrations & sqlc.
|
||
- [`docs/stripe.md`](stripe.md) — the member-console ↔ Stripe responsibility split.
|