member-console/docs/testing.md

title, audience, summary

title	audience	summary
Testing	developer	The test taxonomy (unit, DB-gated integration, Stripe-mocked, HTTP render, lint, e2e, OpenSpec validation), the test kit, how to run each, and conventions for new tests.

Testing guide

How testing works in member-console: the kinds of tests, what each covers, how to run them, and the conventions to follow when writing new ones.

TL;DR

# The whole suite: lint, a reset of the test databases, then every package
# one at a time (needs the test stack up):
cd test && ./bootstrap-stack.sh && docker compose up -d        # one-time per worktree
make test                                                      # from repo root

# One package while iterating, on the same footing as make test:
(cd test && ./reset-test-db.sh)
set -a && . test/.env && set +a
go test -count=1 ./internal/<package>/

# Static checks and render discipline:
make lint                # project lint, golangci-lint, deadcode, unused sqlc queries
make lint-templates      # no handler bypasses SafeTemplates

go test ./... with no database configured is always safe to run: every database- or Stripe-backed test skips itself when TEST_DATABASE_URL is unset. With it set, a database that reset-test-db.sh has not migrated fails the test instead of passing on a stale schema.

The test/ directory — the local stack

Most Go test files sit next to the code they cover (internal/.../<name>_test.go). test/ is the exception, and serves two roles: it's the integration/e2e environment — a Dockerized stack of every backing service the app needs (Postgres, Valkey, Keycloak/OIDC, Temporal, FedWiki) that the database-gated tests (types 2–3) and the e2e suites (type 7) run against — and it houses the higher-level end-to-end Go test suites under test/e2e/.

Its role:

• Provides the backing services. test/compose.yaml brings up Postgres, Valkey, Keycloak, Temporal, and FedWiki, plus one-shot seeders (realm users, FedWiki identity, the Temporal default namespace).
• Isolates each worktree. ./bootstrap-stack.sh derives collision-free host ports and writes test/.env (COMPOSE_PROJECT_NAME, port assignments, and the MC_* config overrides), so several worktrees can run their own stacks concurrently without colliding.
• Is where the test databases come from. ./reset-test-db.sh drops, recreates and migrates member_console_test (TEST_DATABASE_URL) and member_console_e2e (E2E_DATABASE_URL); make test runs it first. test/.env holds both addresses beside MC_DB_DSN, the running app's database, which only the browser tests write.
• Houses the end-to-end Go suites under test/e2e/: plan-management/ (in-process httptest over the real handlers + a migrated DB) and operator-walkthroughs/ (Go tests that drive a browser against the running stack via MC_BASE_URL; WALKTHROUGH_HEADFUL=1 to watch). Both run under go test (see type 7).
• Holds the stack's run-only assets: mc-config.yaml, secrets/ (gitignored Stripe keys + webhook secret), the opt-in demo seeder (./test/seed-demo.sh + seed/), and teardown-stack.sh (compose down + removes .env, secrets/, testdata/).

Don't edit test/.env by hand — re-run bootstrap-stack.sh. The full contract (idempotency, the account-global Stripe webhook constraint, teardown) lives in test/AGENTS.md; read it before running the stack.

The kinds of tests

1. Pure unit tests — no external dependencies

Plain Go tests of pure logic: parsing, classification, formatting, guard conditions. They run on every go test invocation, need nothing external, and are fast.

• Cover: webhook payload parsing, status classification, value mapping, small helpers, input guards.
• Examples: internal/fulfillment TestClassifyStatus, TestUnixToNullTime; internal/integrations/stripe/workflows TestCheckoutSessionPayloadParsing, TestInvoicePayloadParsing, TestCardStringField_*.
• Write one when: the behavior is a function of its inputs with no DB/HTTP/ Stripe dependency. Prefer pushing logic into such functions so it can be tested this way.

2. Database-gated integration tests

Run against a real Postgres that test/reset-test-db.sh has migrated. They exercise queries, constraints (e.g. the pool_provision_ladders GiST exclusion, chk_pool_provisions_source), transactions, and cross-schema flows that unit tests can't reach.

• Cover: the entitlement graph, provisioning, fulfillment reconcile, webhook projection, materialization, operator queries.
• Examples: internal/provisioning (TestAutoProvision…), internal/fulfillment (TestReconcile_*), internal/entitlements/materialize_test.go, internal/integrations/stripe/workflows (*_Integration, transition/product-catalog tests).

Opening the database. Every test reaches the shared databases through dbtest (internal/testkit/dbtest):

• dbtest.Open(t) opens TEST_DATABASE_URL, closes it when the test ends, and skips the test when the variable is unset. It never migrates: the reset script is the only migrator of the shared test database. It fails the test, naming the script, when the core ledger lacks the highest version in db.BaseSources(), so a stale database cannot pass a test. The driver is pgx.
• dbtest.OpenE2E(t) does the same for E2E_DATABASE_URL, the database test/e2e/plan-management commits its fixtures to. dbtest.DSN(t) returns the address (and skips) for a test that needs it itself.
• A test whose subject is a migration or the migrator gets an empty database from dbtest.Scratch(t), created in the same cluster and dropped when the test ends, and applies a stream to it with dbtest.UpTo or dbtest.ApplyVersion, or all streams with db.RunMigrations on a dbtest.Connect to its address. It skips where the login role cannot create databases.
• dbtest.Counted(t) opens TEST_DATABASE_URL as Open does, on a handle that records the name of every statement run on it (ByName, Total, Reset; Names lists them in the order they ran, a transaction's begin, commit and rollback included; Note puts a line of the test's own among them; Diff prints what changed between two records). Hand it to the code under test only and build fixtures on a handle from Open, so the record holds the code's statements and nothing else.
• dbtest.Q runs statements and fails the test on any error (Exec, Scan, Int, String, Refuse for a statement that must fail with a given SQLSTATE; Rows prints a query's rows as text, in the order the query names; a rows golden names the fixture's ids with the test's normaliser and numbers the rest with golden.Ordinals). dbtest.Tx(t, d) opens a transaction the test rolls back; dbtest.On(t, x) binds a statement runner to a test. A Q belongs to the test or subtest that built it: a subtest that uses its parent's transaction rebinds with dbtest.On(t, q.Tx()).
• dbtest.AwaitLockWait polls until a backend waits on a lock, for a test that holds one transaction open while a second one blocks.

Fixtures. world (internal/testkit/world) builds the rows a test needs through the functions the console itself calls: the identity and organization queries (Person, Org, which also creates the default pool, and DefaultLadder), the catalog queries (Product, EditProduct, Ladder, AddTier), the rule-change function (Rule, the system's path, and RuleChange, a person's commit that settles the carrying pools), the conferral queries (Confer), the provisioning step that gives a workspace its pool (Workspace) and the provider registration boot runs (Providers). Billing accounts, invoices and their lines, and payments (BillingAccount, Invoice, LineItem, Payment), the Stripe mappings of billing rows (MapAccount, MapSubscription, MapInvoice and MapPayment record no environment; the ...In forms record one), Stripe product mappings (MapProduct, MapProductIn), a product without its set (WithoutSet), a subscription row with no items (BareSubscription) and the grant, tier and subscription acts (Extend, ReorderTiers, RemoveTier, EndSubscription) go through the same queries and functions the console calls. OrgType is the one raw INSERT, since no console act creates an organization type. world.New(t, db) takes any *sql.DB the migrations built, so one builder fills the test database or the application database. Every method fails the test on any error. Every organization, product, set, ladder name, subject and email carries the World's suffix, so repeated runs against the shared database do not collide, and nothing is deleted: rows stay in the disposable database until the next reset. A World holds the testing.TB it was built with, so a subtest builds its own with world.New(t, db). The first sign-in's rows (user, person, organization with its membership, workspace) come from provisioning.AutoProvision(ctx, db, claims).

A test of the schema itself writes its rows by SQL: a constraint, a trigger or a migration is the subject, and a builder that went through the console's functions would not exercise it.

3. Stripe-mocked tests

A specialization of the integration tests: the Stripe SDK is pointed at an in-process fake so fulfillment's "refetch from the Stripe API" path runs deterministically and offline, with no live Stripe account.

• Helper: internal/stripetest — MockBackend{Sub, List, Err} implements stripe.Backend; stripetest.Install(m) swaps the global backend + a dummy key and returns a restore func (t.Cleanup(stripetest.Install(m))). m.OnCall sees each request as it is recorded, and stripetest.Line(c) prints one as text, so m.OnCall = func(c stripetest.Call) { stmts.Note(stripetest.Line(c)) } puts each Stripe call among the statements of a dbtest.Counted handle. stripetest.KeyMode answers the mode of the key the SDK holds when it is called ("test" under Install); a test passes it where the console passes its configured key mode.
• Cover: fulfillment.ReconcileSubscription scenarios (empty-payload provisioning, idempotent replay, ordering, cancellation, multi-ladder guard) and the webhook→plan-transitions end-to-end tests.
• Gotcha: Install mutates global stripe-go state, so these tests must not call t.Parallel().

4. HTTP handler / render tests

Exercise handlers through net/http/httptest plus the real SafeTemplates renderer, session manager (scs), and CSRF middleware.

• Cover: rendered output, template wiring, auth guards (e.g. 401 for unauthenticated), CSRF behavior.
• Examples: internal/server/member_upgrade_render_test.go (TestMemberPlansUpgradeControlRendering, TestDashboardCheckoutBanner, TestGetPlansUnauthenticatedReturns401); internal/middleware/tests/security_headers_test.go.
• Template sets come from the constructors production uses. The server package's test harness (internal/server/harness_test.go) builds them: OperatorSet(t) is web.OperatorTemplates and MemberSet(t) is web.MemberTemplates, both over web.CoreTemplates(), and Render(t, set, name, data) executes one template. A test never builds a stand-in set or registers template functions itself.
• Query the output with htmltest (internal/testkit/htmltest): htmltest.Parse(t, body) parses a page or a fragment, and One (exactly one match), All, Count, None, Text, Attr, HasClass, Before and InOrder select and compare with CSS selectors, the vocabulary the browser walkthroughs use. Select by role and by aria-*, hx-* and data-* attributes (nav[aria-label=Account], [role="menu"] button[hx-post="/logout"]); a class that only styles an element is not what a test asserts. A failure quotes the selector and a snippet of the markup. The harness adds ShellBar, ShellRail and LinkNamed for the shell.
• Sign in through authtest (internal/testkit/authtest): authtest.NewSessionManager() is the in-memory session manager, and authtest.SignIn(t, sm, signIn) writes the session through auth.PutSignIn, the function the callback handler calls, so a test session has the keys and values a real sign-in produces. authtest.Get and authtest.Post run a handler in process with that session on the request; authtest.PathValue sets a path parameter as the mux would. Get and Post send no htmx headers, so a handler answers as it does to a native request. authtest.Cookie commits that session and returns its cookie, for a handler wrapped in the session middleware, such as the whole router.
• Compare with a golden file through golden (internal/testkit/golden): golden.Compare(t, file, got) fails unless got is the file's content, and go test -update writes the file. A golden file is written once, with -update, and read against its fixture before it is committed; after that it changes only in a commit that names the change. A golden.Normalizer turns the ids a fixture made into names (Name) and golden.Ordinals numbers the ids and times an act made. A test package that defines its own update flag cannot import golden, because the flag would be defined twice.

5. Static checks (make lint)

Not Go tests, but part of the green-bar definition. make lint runs five checks, and make lint-templates is a sixth that runs on its own:

• go run . lint (cmd/lint.go) — operator-UI integrity: every registered route and templated URL reference resolves, swap targets exist, pages follow the page anatomy, copy follows the vocabulary rules, and every page route has a screen. It parses each template in the set its surface builds in production (web.MemberTemplates and web.OperatorTemplates, with each integration's directory as a mount), so a template it cannot parse fails it, naming the file, and so does a file that no surface parses.
• scripts/notebook-citations.sh — no tracked file cites a local notebook path.
• scripts/golangci-lint.sh — golangci-lint with .golangci.yml: the standard set (vet, staticcheck, unchecked errors with errcheck, dead assignments with ineffassign, unused code with unused), plus error comparison (errorlint), rows and sql handling (rowserrcheck, sqlclosecheck), unused parameters (unparam), duplicated code (dupl), function complexity (gocognit, limit 30) and forbidden identifiers (forbidigo), among them the rod waits the browser tests refuse (type 7).
• scripts/deadcode.sh — production functions no program reaches, except those listed in scripts/deadcode-testonly.txt.
• go run ./scripts/sqlc-unused — generated queries with no caller outside generated and test files.
• make lint-templates — guards that no handler bypasses SafeTemplates by calling .ExecuteTemplate(w, …) directly (CSP/render discipline; see internal/server/render.go).

.golangci.yml lists the findings a later change removes, one rule per finding, pinned to the function's current score and commented with the issue or #166 step that removes it. A rule that stops matching fails the lint, so the list only shrinks.

6. Security scanners (static, offline)

A security audit runs two passes, in this order: the deterministic scanners (gosec, govulncheck, semgrep, staticcheck, gitleaks) run first and offline, and a model then triages their hits. The order is deliberate — a model adjudicating scanner output cuts semgrep's false-positive rate from about 0.42 to about 0.05 at a cost of about 3% relative recall, while a model reading code unaided over-reports. The method the audit follows is docs/agent-runner.md.

7. End-to-end tests

Exercise whole flows against a running stack. Three flavors:

• Go HTTP-level e2e — test/e2e/plan-management/: an in-process httptest harness over the real handlers + a migrated DB (E2E_DATABASE_URL), with fixtures from world and requests from authtest. Runs under go test; covers operator plan/ladder/grant flows end-to-end at the HTTP layer.
• Go browser walkthroughs — test/e2e/operator-walkthroughs/: Go tests that drive a real browser against the running stack (reads MC_BASE_URL from test/.env; set WALKTHROUGH_HEADFUL=1 to watch). One walkthrough per operator route group, each asserting the four-bullet UI contract (empty/invalid → 422, valid → success + DOM update, …). Skips unless the stack is up and seeded. The browser tests (the walkthroughs, test/e2e/screens and test/e2e/merchant-disclosure) share test/e2e/browsertest, whose rules are:
  • Waits are boolean predicates. WaitTrue, TryWaitTrue, WaitLoaded and WaitReady poll a JavaScript function that returns a boolean, and a missing or destroyed execution context counts as "not yet". Eventually polls a Go probe that returns a boolean. golangci-lint refuses rod's WaitLoad and WaitStable (MustWaitLoad, MustWaitStable) on a page or element, with the message "wait on a predicate that returns a boolean (browsertest)". The one wait without a predicate is rod's WaitDOMStable, for a page that may never stop changing and is captured as it stands.
  • ClickAndLand for an answer that is a new document (an HX-Redirect, a native redirect): it anchors the wait to the new document, not to a lifecycle event. AwaitSwap for an htmx swap: it runs the action and waits for htmx to settle.
  • Recover. Each walkthrough defers browsertest.Recover(t) first, so a rod panic fails that walkthrough with its stack and the package's other walkthroughs still run.
  • AppDB opens the running app's database (MC_DB_DSN in test/.env), never migrates it, and skips when the key is missing.
  • Free functions that take t, and no helper stores it, so a subtest never calls a parent's.
  • A long walkthrough is phases: named functions, or methods on a struct that holds the walkthrough's state, called in order.
• Manual / agent-driven live loop — the chrome-devtools MCP (browser on 127.0.0.1:9222, started with --remote-debugging-port=9222), the Stripe MCP, and stripe listen webhook forwarding, for flows not yet codified — e.g. a real Stripe Checkout → checkout.session.completed → reconcile → the member UI reflecting the new plan. Not part of go test.

Setup for all three lives in test/AGENTS.md (worktree stack contract; the "Stripe webhook forwarding" section for the live loop).

8. Live DNS verification (domain claims, manual)

Some of the domains registry can only be proven against a real zone: that a published challenge record activates a claim, and that publishing the challenge prefix with the wrong token latches evidence_at (exempting the claim from the abandonment ledger) without verifying. DB tests cover the branching; only real DNS covers the resolver path.

You need a disposable zone you control — any domain whose records you can add and remove, at any registrar or DNS host with an API. Keep it separate from anything the deployment actually serves, and never commit credentials for it; supply them from the environment at the moment you use them. If the harness blocks outbound HTTP from the shell (the context-mode routing rules do), drive the DNS provider's API from a sandboxed script instead.

The loop is: start a claim in the console, read its token, publish _member-console-challenge.<name> as TXT, wait for a probe, assert, then delete the record. Two cases are worth proving because they diverge — a TXT carrying the challenge prefix with a wrong token must latch evidence without activating the claim, while the correct token must activate it.

The gotcha that will cost you ten minutes. If the zone has a wildcard record (*.example CNAME/ALIAS/TXT), resolvers synthesise it for any challenge name that has no explicit record — so the probe reports "Found, but doesn't match" against whatever the wildcard points at, long before you did anything wrong. Publishing an explicit record stops the synthesis (RFC 4592 closest-encloser), but any cached wildcard answer still wins until its TTL expires, so probes keep reporting mismatch for minutes after DNS is already correct. Before concluding the prober is broken, compare authoritative against cached:

dig +short @<authoritative-ns> TXT _member-console-challenge.<name>   # what the zone says
dig +noall +answer             TXT _member-console-challenge.<name>   # what the console sees

Clean up the records you added, and leave alone any challenge record that backs a claim you intend to keep verified.

9. OpenSpec validation (change workflow)

openspec validate <change> is not a Go test but is part of the definition of done for a spec-driven change: it checks delta specs are well-formed (MODIFIED requirement headers match the main spec, every requirement has a #### scenario, etc.). Run it before archiving a change.

The test kit

internal/testkit and its sub-packages hold what tests share. Each package imports only what its row allows, so any package's tests can use it without an import cycle; internal/testkit/imports_test.go lists the dependencies of each with go list -deps and fails when one reaches a package it forbids.

Package	Holds	Imports
testkit	Close, which closes an io.Closer and fails the test on its error	no console package
testkit/dbtest	opening the test databases, scratch databases, counting the statements a request runs, Q, AwaitLockWait	internal/db and internal/testkit only (internal/migrate reaches nearly every package)
testkit/htmltest	parsing HTML and selecting from it with CSS	no console package
testkit/golden	comparing a test's output with a file under testdata, -update writing it, and naming the ids and times a run makes	no console package
testkit/world	the fixture builder, plan and billing fixtures and the grant, tier and subscription acts included	the console's data packages; never internal/migrate, internal/server, internal/integrations, an integration's web or workflows, or internal/domains
testkit/authtest	signed-in sessions, their cookies, and in-process requests	as world
integrations/fedwiki/fedwikitest	FedWiki's fixtures and fake farm, built on dbtest and world	never internal/migrate, internal/server, or FedWiki's own module, web and workflows packages

Choosing what to write

Is the behavior a pure function of inputs?            → unit test (1)
Does it touch the DB / constraints / multi-table?     → DB integration test (2)
Does it call the Stripe API?                          → add a stripetest mock (3)
Is it an HTTP handler / rendered page?                → httptest render test (4)
Is it route/template wiring or render discipline?     → make lint / lint-templates (5)
Is it a whole user flow (HTTP or operator UI)?        → e2e (7): a test/e2e/ Go suite
Is it a flow only real infra can prove (live Stripe)? → the live MCP loop (7), see test/AGENTS.md
Does it depend on what a resolver actually returns?   → live DNS on a zone you control (8)

Prefer the cheapest test that genuinely covers the behavior. Reach for an e2e suite or the live loop (7) to prove a flow works end-to-end, then capture the regression-proofing parts as (2)/(3) so they run in CI without a live account.

Conventions & gotchas

• Tests meet the same complexity limit as production code. gocognit counts a closure passed to t.Run toward the function that encloses it, so splitting a test into subtests does not lower its score. A test over the limit splits into named functions, runs its table rows through a named function, or uses the kit's helpers.
• Open the database through dbtest (Open, OpenE2E), which skips the test when TEST_DATABASE_URL or E2E_DATABASE_URL is unset, so go test ./... stays green with no infrastructure. No test migrates the shared databases; test/reset-test-db.sh does, and dbtest.Open refuses one it has not.
• A test of a migration applies it to a dbtest.Scratch database, never to the shared one.
• Prove a converted assertion with a deliberate break. When a check moves onto the kit (a substring onto htmltest, a hand-written wait onto browsertest, a fixture onto world), apply one break that violates what the assertion says it checks, not its incidental markup, and see the test fail with that message. scripts/mutation-check runs a JSON list of breaks (file, exact text to replace, replacement, package, test pattern, wanted message): it runs each test unbroken first, applies one break at a time, restores the file, and reports CAUGHT, MISSED or WRONG-MESSAGE. Run it with -side old on a checkout of the parent commit and with -side new on yours, so a break the old test missed shows up as a weaker assertion. A list for browser tests carries a prepare command that resets and restarts the app before each break, and a command record covers a break that only the lint shows. The header of scripts/mutation-check/main.go gives the format.
• Don't trust webhook payloads in tests the way production doesn't: a Stripe test should drive behavior through stripetest.MockBackend returning canned API state, not by hand-building the event payload's fields.
• No t.Parallel() in Stripe-mocked tests — they share global SDK state.
• Unique fixture names for tests that write to the shared test databases: world suffixes every name it writes, and dbtest.Name returns a unique lowercase name for the rest. make test resets the databases; tear the stack down with test/teardown-stack.sh when you want a clean slate.
• A wildcard CNAME shadows challenge lookups — see (8). A domain-verification probe reporting mismatch against a record you just published is usually a cached wildcard, not a defect.
• Accumulated rows can break the browser walkthroughs. The DB-gated suites write to member_console_test and member_console_e2e and never clean up, but make test resets both, and no package test writes the app database. The walkthroughs read the app database, so rows that pile up there (core.plan_ladders, core.org_types) come from the walkthroughs themselves or from manual use. Check row counts before believing a walkthrough regression.

Lint and screens (the UI quality gate)

A UX walk cannot hold the console to one shape: a page that differs from its siblings costs the walker nothing they report, so the walk's severity scale rates a consistency finding minor and triage tracks it. Sameness holds only when the build fails on it and the maintainer reviews a capture of every screen, which is what these two commands are for.

• make lint runs member-console lint: dead routes and swap targets, retired resource keys, raw error rendering, walkthrough coverage, and the page-anatomy rules (design-system.md §6: raw titles, badges, and empty states outside the shared parts, hx-confirm, load-triggered fetches, em dashes in copy, heading skips) plus screen coverage (every GET page route in test/e2e/screens/manifest.go). Templates not yet rebuilt on the parts are listed in internal/lint/anatomy_allowlist.txt, which only shrinks; a stale entry fails. A template the lint cannot parse fails it, naming the file. make test runs it first; the Go checks are in (5).
• make screens first restores the application database from a demo snapshot (test/reset-app-db.sh; the one deliberate exception to "nothing under test/ touches member_console"). The snapshot is built once: migrate, seed-demo --with-operator, boot data state, row clocks pinned by test/pin-timestamps.sql to 2026-01-01 09:00 UTC plus one second per distinct original instant in the column's own order (a dense rank), so distinct instants stay distinct, each table keeps the chronology its rows were written in, and ties stay ties (the two rows one conferral act writes keep one instant, which the Tier changes fold relies on; the ledger query's id tiebreak orders inside a tie); every reset recreates the application database from it, so rows and their IDs are identical between captures and a page nobody touched never reads as changed (only relative ages tick, once a day). The snapshot rebuilds itself when the migrations, the seed, or the clock pin change (their fingerprint is the snapshot's database comment); ./reset-app-db.sh --rebuild forces it., then captures every manifest entry at 1600×1000 and 390×844 (rod, logged in as alice for the operator surface and bob for the member surface, one incognito context each) into test/screens/out/, writes operator.html and member.html contact sheets in rail order, and compares each PNG with test/screens/baseline/, listing changed and new screens in changes.txt with the differing region's size and position. Parameterised routes resolve their instance from the seeded list page (the first record link, or the one a manifest entry prefers by its link text) and report "no instance" when the stack has none. make screens-accept promotes the output to the baseline. Both directories are gitignored; the stack in test/.env must be up and seeded. A change touching templates or app.css is not done until its sheet is reviewed (first-contact-ux-process.md, "Definition of done").

Related docs

• test/AGENTS.md — the test stack (ports, webhook forwarding, demo seed).
• docs/database-management.md — migrations & sqlc.
• docs/stripe.md — the member-console ↔ Stripe responsibility split.