world gains MapProduct and MapProductIn, which give a product a synced Stripe mapping through the console's own upsert (Stripe id prod_<product id>, no environment or the one asked for), and WithoutSet, which clears a product's entitlement set through EditProduct the way the operator's edit form saves a product with no set chosen. TestCatalogBuilders reads both back through the console's queries. No builder writes a dead-lettered outbox entry; a test that needs one makes it through the console's MarkOutboxDeadLetter. No production file changes and no existing builder writes anything new. Names and results over ./internal/testkit/..., internal/entitlements, internal/fulfillment, internal/server, internal/integrations/discourse/web, internal/integrations/fedwiki/web and internal/integrations/stripe/workflows, taken with the test database reset before each side, are equal at the parent and here except the new test and its two subtests, which pass. The package test/e2e/plan-management, which also builds through world, compiles under go vet; it calls none of the new builders and no existing builder changed. 5 breaks, each caught by the new test; none dropped. Unprobed: MapProductIn(p, false), which is the same statement with the flag false; the test reads the live case.
499 lines
30 KiB
Markdown
499 lines
30 KiB
Markdown
---
|
||
title: "Testing"
|
||
audience: [developer]
|
||
summary: "The test taxonomy (unit, DB-gated integration, Stripe-mocked, HTTP render, lint, e2e, OpenSpec validation), the test kit, how to run each, and conventions for new tests."
|
||
---
|
||
|
||
# Testing guide
|
||
|
||
How testing works in member-console: the kinds of tests, what each covers, how
|
||
to run them, and the conventions to follow when writing new ones.
|
||
|
||
## TL;DR
|
||
|
||
```bash
|
||
# The whole suite: lint, a reset of the test databases, then every package
|
||
# one at a time (needs the test stack up):
|
||
cd test && ./bootstrap-stack.sh && docker compose up -d # one-time per worktree
|
||
make test # from repo root
|
||
|
||
# One package while iterating, on the same footing as make test:
|
||
(cd test && ./reset-test-db.sh)
|
||
set -a && . test/.env && set +a
|
||
go test -count=1 ./internal/<package>/
|
||
|
||
# Static checks and render discipline:
|
||
make lint # project lint, golangci-lint, deadcode, unused sqlc queries
|
||
make lint-templates # no handler bypasses SafeTemplates
|
||
```
|
||
|
||
`go test ./...` with no database configured is always safe to run: every
|
||
database- or Stripe-backed test **skips itself** when `TEST_DATABASE_URL` is
|
||
unset. With it set, a database that `reset-test-db.sh` has not migrated fails
|
||
the test instead of passing on a stale schema.
|
||
|
||
## The `test/` directory — the local stack
|
||
|
||
Most Go test files sit next to the code they cover (`internal/.../<name>_test.go`).
|
||
`test/` is the exception, and serves two roles: it's the **integration/e2e
|
||
environment** — a Dockerized stack of every backing service the app needs
|
||
(Postgres, Valkey, Keycloak/OIDC, Temporal, FedWiki) that the database-gated
|
||
tests (types 2–3) and the e2e suites (type 7) run against — **and** it houses the
|
||
higher-level **end-to-end Go test suites** under `test/e2e/`.
|
||
|
||
Its role:
|
||
|
||
- **Provides the backing services.** `test/compose.yaml` brings up Postgres,
|
||
Valkey, Keycloak, Temporal, and FedWiki, plus one-shot seeders (realm users,
|
||
FedWiki identity, the Temporal default namespace).
|
||
- **Isolates each worktree.** `./bootstrap-stack.sh` derives collision-free host
|
||
ports and writes `test/.env` (`COMPOSE_PROJECT_NAME`, port assignments, and the
|
||
`MC_*` config overrides), so several worktrees can run their own stacks
|
||
concurrently without colliding.
|
||
- **Is where the test databases come from.** `./reset-test-db.sh` drops,
|
||
recreates and migrates `member_console_test` (`TEST_DATABASE_URL`) and
|
||
`member_console_e2e` (`E2E_DATABASE_URL`); `make test` runs it first.
|
||
`test/.env` holds both addresses beside `MC_DB_DSN`, the running app's
|
||
database, which only the browser tests write.
|
||
- **Houses the end-to-end Go suites** under `test/e2e/`: `plan-management/`
|
||
(in-process `httptest` over the real handlers + a migrated DB) and
|
||
`operator-walkthroughs/` (Go tests that drive a browser against the running
|
||
stack via `MC_BASE_URL`; `WALKTHROUGH_HEADFUL=1` to watch). Both run under
|
||
`go test` (see type 7).
|
||
- **Holds the stack's run-only assets:** `mc-config.yaml`, `secrets/`
|
||
(gitignored Stripe keys + webhook secret), the opt-in demo seeder
|
||
(`./test/seed-demo.sh` + `seed/`), and `teardown-stack.sh` (compose down +
|
||
removes `.env`, `secrets/`, `testdata/`).
|
||
|
||
Don't edit `test/.env` by hand — re-run `bootstrap-stack.sh`. The full contract
|
||
(idempotency, the account-global Stripe webhook constraint, teardown) lives in
|
||
[`test/AGENTS.md`](../test/AGENTS.md); read it before running the stack.
|
||
|
||
## The kinds of tests
|
||
|
||
### 1. Pure unit tests — no external dependencies
|
||
|
||
Plain Go tests of pure logic: parsing, classification, formatting, guard
|
||
conditions. They run on every `go test` invocation, need nothing external, and
|
||
are fast.
|
||
|
||
- **Cover:** webhook payload parsing, status classification, value mapping,
|
||
small helpers, input guards.
|
||
- **Examples:** `internal/fulfillment` `TestClassifyStatus`,
|
||
`TestUnixToNullTime`; `internal/integrations/stripe/workflows` `TestCheckoutSessionPayloadParsing`,
|
||
`TestInvoicePayloadParsing`, `TestCardStringField_*`.
|
||
- **Write one when:** the behavior is a function of its inputs with no DB/HTTP/
|
||
Stripe dependency. Prefer pushing logic into such functions so it can be
|
||
tested this way.
|
||
|
||
### 2. Database-gated integration tests
|
||
|
||
Run against a real Postgres that `test/reset-test-db.sh` has migrated. They
|
||
exercise queries, constraints (e.g. the `pool_provision_ladders` GiST
|
||
exclusion, `chk_pool_provisions_source`), transactions, and cross-schema flows
|
||
that unit tests can't reach.
|
||
|
||
- **Cover:** the entitlement graph, provisioning, fulfillment reconcile, webhook
|
||
projection, materialization, operator queries.
|
||
- **Examples:** `internal/provisioning` (`TestAutoProvision…`), `internal/fulfillment`
|
||
(`TestReconcile_*`), `internal/entitlements/materialize_test.go`,
|
||
`internal/integrations/stripe/workflows` (`*_Integration`, transition/product-catalog tests).
|
||
|
||
**Opening the database.** Every test reaches the shared databases through
|
||
`dbtest` (`internal/testkit/dbtest`):
|
||
|
||
- `dbtest.Open(t)` opens `TEST_DATABASE_URL`, closes it when the test ends, and
|
||
skips the test when the variable is unset. It **never migrates**: the reset
|
||
script is the only migrator of the shared test database. It fails the test,
|
||
naming the script, when the core ledger lacks the highest version in
|
||
`db.BaseSources()`, so a stale database cannot pass a test. The driver is
|
||
pgx.
|
||
- `dbtest.OpenE2E(t)` does the same for `E2E_DATABASE_URL`, the database
|
||
`test/e2e/plan-management` commits its fixtures to. `dbtest.DSN(t)` returns
|
||
the address (and skips) for a test that needs it itself.
|
||
- A test whose subject is a migration or the migrator gets an empty database
|
||
from `dbtest.Scratch(t)`, created in the same cluster and dropped when the
|
||
test ends, and applies a stream to it with `dbtest.UpTo` or
|
||
`dbtest.ApplyVersion`, or all streams with `db.RunMigrations` on a
|
||
`dbtest.Connect` to its address. It skips where the login role cannot create
|
||
databases.
|
||
- `dbtest.Counted(t)` opens `TEST_DATABASE_URL` as `Open` does, on a handle
|
||
that records the name of every statement run on it (`ByName`, `Total`,
|
||
`Reset`; `Names` lists them in the order they ran, a transaction's `begin`,
|
||
`commit` and `rollback` included; `Note` puts a line of the test's own among
|
||
them; `Diff` prints what changed between two records). Hand it to the code
|
||
under test only and build fixtures on a handle from `Open`, so the record
|
||
holds the code's statements and nothing else.
|
||
- `dbtest.Q` runs statements and fails the test on any error (`Exec`, `Scan`,
|
||
`Int`, `String`, `Refuse` for a statement that must fail with a given
|
||
SQLSTATE; `Rows` prints a query's rows as text, in the order the query
|
||
names; a rows golden names the fixture's ids with the test's normaliser and
|
||
numbers the rest with `golden.Ordinals`). `dbtest.Tx(t, d)` opens a
|
||
transaction the test rolls back; `dbtest.On(t, x)` binds a statement runner to
|
||
a test. A `Q` belongs to the test or subtest that built it: a subtest that
|
||
uses its parent's transaction rebinds with `dbtest.On(t, q.Tx())`.
|
||
- `dbtest.AwaitLockWait` polls until a backend waits on a lock, for a test that
|
||
holds one transaction open while a second one blocks.
|
||
|
||
**Fixtures.** `world` (`internal/testkit/world`) builds the rows a test needs
|
||
through the functions the console itself calls: the identity and organization
|
||
queries (`Person`, `Org`, which also creates the default pool, and
|
||
`DefaultLadder`), the catalog queries (`Product`, `EditProduct`, `Ladder`,
|
||
`AddTier`), the rule-change function (`Rule`, the system's path, and
|
||
`RuleChange`, a person's commit that settles the carrying pools), the conferral
|
||
queries (`Confer`), the provisioning step that gives a workspace its pool
|
||
(`Workspace`) and the provider registration boot runs (`Providers`). Billing
|
||
accounts, invoices and their lines, and payments (`BillingAccount`, `Invoice`,
|
||
`LineItem`, `Payment`), the Stripe mappings of billing rows (`MapAccount`,
|
||
`MapSubscription`, `MapInvoice` and `MapPayment` record no environment; the
|
||
`...In` forms record one), Stripe product mappings (`MapProduct`,
|
||
`MapProductIn`), a product without its set (`WithoutSet`), a subscription row
|
||
with no items (`BareSubscription`) and the grant, tier and subscription acts
|
||
(`Extend`, `ReorderTiers`, `RemoveTier`, `EndSubscription`) go through the same
|
||
queries and functions the console calls. `OrgType` is the one raw INSERT, since
|
||
no console act creates an organization type. `world.New(t, db)` takes any
|
||
`*sql.DB` the migrations built, so one builder fills the test database or the
|
||
application database. Every method fails the test on any error. Every
|
||
organization, product, set, ladder name, subject and email carries the World's
|
||
suffix, so repeated runs against the shared database do not collide, and nothing
|
||
is deleted: rows stay in the disposable database until the next reset. A World
|
||
holds the `testing.TB` it was built with, so a subtest builds its own with
|
||
`world.New(t, db)`. The first sign-in's rows (user, person, organization with
|
||
its membership, workspace) come from
|
||
`provisioning.AutoProvision(ctx, db, claims)`.
|
||
|
||
A test of the schema itself writes its rows by SQL: a constraint, a trigger or a
|
||
migration is the subject, and a builder that went through the console's
|
||
functions would not exercise it.
|
||
|
||
### 3. Stripe-mocked tests
|
||
|
||
A specialization of the integration tests: the Stripe SDK is pointed at an
|
||
in-process fake so fulfillment's "refetch from the Stripe API" path runs
|
||
deterministically and offline, with **no live Stripe account**.
|
||
|
||
- **Helper:** `internal/stripetest` — `MockBackend{Sub, List, Err}` implements
|
||
`stripe.Backend`; `stripetest.Install(m)` swaps the global backend + a dummy
|
||
key and returns a restore func (`t.Cleanup(stripetest.Install(m))`).
|
||
`m.OnCall` sees each request as it is recorded, and `stripetest.Line(c)`
|
||
prints one as text, so
|
||
`m.OnCall = func(c stripetest.Call) { stmts.Note(stripetest.Line(c)) }` puts
|
||
each Stripe call among the statements of a `dbtest.Counted` handle.
|
||
`stripetest.KeyMode` answers the mode of the key the SDK holds when it is
|
||
called (`"test"` under `Install`); a test passes it where the console passes
|
||
its configured key mode.
|
||
- **Cover:** `fulfillment.ReconcileSubscription` scenarios (empty-payload
|
||
provisioning, idempotent replay, ordering, cancellation, multi-ladder guard)
|
||
and the webhook→`plan-transitions` end-to-end tests.
|
||
- **Gotcha:** `Install` mutates **global** stripe-go state, so these tests must
|
||
**not** call `t.Parallel()`.
|
||
|
||
### 4. HTTP handler / render tests
|
||
|
||
Exercise handlers through `net/http/httptest` plus the real `SafeTemplates`
|
||
renderer, session manager (`scs`), and CSRF middleware.
|
||
|
||
- **Cover:** rendered output, template wiring, auth guards (e.g. 401 for
|
||
unauthenticated), CSRF behavior.
|
||
- **Examples:** `internal/server/member_upgrade_render_test.go`
|
||
(`TestMemberPlansUpgradeControlRendering`, `TestDashboardCheckoutBanner`,
|
||
`TestGetPlansUnauthenticatedReturns401`); `internal/middleware/tests/security_headers_test.go`.
|
||
- **Template sets come from the constructors production uses.** The server
|
||
package's test harness (`internal/server/harness_test.go`) builds them:
|
||
`OperatorSet(t)` is `web.OperatorTemplates` and `MemberSet(t)` is
|
||
`web.MemberTemplates`, both over `web.CoreTemplates()`, and `Render(t, set,
|
||
name, data)` executes one template. A test never builds a stand-in set or
|
||
registers template functions itself.
|
||
- **Query the output with `htmltest`** (`internal/testkit/htmltest`):
|
||
`htmltest.Parse(t, body)` parses a page or a fragment, and `One` (exactly
|
||
one match), `All`, `Count`, `None`, `Text`, `Attr`, `HasClass`, `Before` and
|
||
`InOrder` select and compare with CSS selectors, the vocabulary the browser
|
||
walkthroughs use. Select by role and by `aria-*`, `hx-*` and `data-*`
|
||
attributes (`nav[aria-label=Account]`, `[role="menu"] button[hx-post="/logout"]`);
|
||
a class that only styles an element is not what a test asserts. A failure
|
||
quotes the selector and a snippet of the markup. The harness adds
|
||
`ShellBar`, `ShellRail` and `LinkNamed` for the shell.
|
||
- **Sign in through `authtest`** (`internal/testkit/authtest`):
|
||
`authtest.NewSessionManager()` is the in-memory session manager, and
|
||
`authtest.SignIn(t, sm, signIn)` writes the session through `auth.PutSignIn`,
|
||
the function the callback handler calls, so a test session has the keys and
|
||
values a real sign-in produces. `authtest.Get` and `authtest.Post` run a
|
||
handler in process with that session on the request; `authtest.PathValue`
|
||
sets a path parameter as the mux would. `Get` and `Post` send no htmx
|
||
headers, so a handler answers as it does to a native request.
|
||
`authtest.Cookie` commits that session and returns its cookie, for a handler
|
||
wrapped in the session middleware, such as the whole router.
|
||
- **Compare with a golden file through `golden`** (`internal/testkit/golden`):
|
||
`golden.Compare(t, file, got)` fails unless `got` is the file's content, and
|
||
`go test -update` writes the file. A golden file is written once, with
|
||
`-update`, and read against its fixture before it is committed; after that it
|
||
changes only in a commit that names the change. A `golden.Normalizer` turns
|
||
the ids a fixture made into names (`Name`) and `golden.Ordinals` numbers the
|
||
ids and times an act made. A test package that defines its own `update` flag
|
||
cannot import `golden`, because the flag would be defined twice.
|
||
|
||
### 5. Static checks (`make lint`)
|
||
|
||
Not Go tests, but part of the green-bar definition. `make lint` runs five
|
||
checks, and `make lint-templates` is a sixth that runs on its own:
|
||
|
||
- **`go run . lint`** (`cmd/lint.go`) — operator-UI integrity: every registered
|
||
route and templated URL reference resolves, swap targets exist, pages follow
|
||
the page anatomy, copy follows the vocabulary rules, and every page route has
|
||
a screen. It parses each template in the set its surface builds in
|
||
production (`web.MemberTemplates` and `web.OperatorTemplates`, with each
|
||
integration's directory as a mount), so a template it cannot parse fails it,
|
||
naming the file, and so does a file that no surface parses.
|
||
- **`scripts/notebook-citations.sh`** — no tracked file cites a local notebook
|
||
path.
|
||
- **`scripts/golangci-lint.sh`** — golangci-lint with `.golangci.yml`: the
|
||
standard set (vet, staticcheck, unchecked errors with errcheck, dead
|
||
assignments with ineffassign, unused code with unused), plus error
|
||
comparison (errorlint), `rows` and `sql` handling (rowserrcheck,
|
||
sqlclosecheck), unused parameters (unparam), duplicated code (dupl),
|
||
function complexity (gocognit, limit 30) and forbidden identifiers
|
||
(forbidigo), among them the rod waits the browser tests refuse (type 7).
|
||
- **`scripts/deadcode.sh`** — production functions no program reaches, except
|
||
those listed in `scripts/deadcode-testonly.txt`.
|
||
- **`go run ./scripts/sqlc-unused`** — generated queries with no caller outside
|
||
generated and test files.
|
||
- **`make lint-templates`** — guards that no handler bypasses `SafeTemplates` by
|
||
calling `.ExecuteTemplate(w, …)` directly (CSP/render discipline; see
|
||
`internal/server/render.go`).
|
||
|
||
`.golangci.yml` lists the findings a later change removes, one rule per
|
||
finding, pinned to the function's current score and commented with the issue or
|
||
#166 step that removes it. A rule that stops matching fails the lint, so the
|
||
list only shrinks.
|
||
|
||
### 6. Security scanners (static, offline)
|
||
|
||
A security audit runs two passes, in this order: the deterministic scanners
|
||
(gosec, govulncheck, semgrep, staticcheck, gitleaks) run first and offline,
|
||
and a model then triages their hits. The order is deliberate — a model
|
||
adjudicating scanner output cuts semgrep's false-positive rate from about
|
||
0.42 to about 0.05 at a cost of about 3% relative recall, while a model
|
||
reading code unaided over-reports. The method the audit follows is
|
||
[`docs/agent-runner.md`](agent-runner.md).
|
||
|
||
### 7. End-to-end tests
|
||
|
||
Exercise whole flows against a running stack. Three flavors:
|
||
|
||
- **Go HTTP-level e2e** — `test/e2e/plan-management/`: an in-process `httptest`
|
||
harness over the real handlers + a migrated DB (`E2E_DATABASE_URL`), with
|
||
fixtures from `world` and requests from `authtest`. Runs under `go test`;
|
||
covers operator plan/ladder/grant flows end-to-end at the HTTP layer.
|
||
- **Go browser walkthroughs** — `test/e2e/operator-walkthroughs/`: Go tests that
|
||
drive a real browser against the running stack (reads `MC_BASE_URL` from
|
||
`test/.env`; set `WALKTHROUGH_HEADFUL=1` to watch). One walkthrough per operator
|
||
route group, each asserting the four-bullet UI contract (empty/invalid → 422,
|
||
valid → success + DOM update, …). Skips unless the stack is up and seeded.
|
||
The browser tests (the walkthroughs, `test/e2e/screens` and
|
||
`test/e2e/merchant-disclosure`) share `test/e2e/browsertest`, whose rules are:
|
||
- **Waits are boolean predicates.** `WaitTrue`, `TryWaitTrue`, `WaitLoaded` and
|
||
`WaitReady` poll a JavaScript function that returns a boolean, and a
|
||
missing or destroyed execution context counts as "not yet". `Eventually`
|
||
polls a Go probe that returns a boolean.
|
||
golangci-lint refuses rod's `WaitLoad` and `WaitStable` (`MustWaitLoad`,
|
||
`MustWaitStable`) on a page or element, with the message "wait on a
|
||
predicate that returns a boolean (browsertest)". The one wait without a
|
||
predicate is rod's `WaitDOMStable`, for a page that may never stop changing
|
||
and is captured as it stands.
|
||
- **`ClickAndLand` for an answer that is a new document** (an `HX-Redirect`, a
|
||
native redirect): it anchors the wait to the new document, not to a
|
||
lifecycle event. **`AwaitSwap` for an htmx swap**: it runs the action and
|
||
waits for htmx to settle.
|
||
- **`Recover`.** Each walkthrough defers `browsertest.Recover(t)` first, so a
|
||
rod panic fails that walkthrough with its stack and the package's other
|
||
walkthroughs still run.
|
||
- **`AppDB`** opens the running app's database (`MC_DB_DSN` in `test/.env`),
|
||
never migrates it, and skips when the key is missing.
|
||
- **Free functions that take `t`**, and no helper stores it, so a subtest
|
||
never calls a parent's.
|
||
- **A long walkthrough is phases**: named functions, or methods on a struct
|
||
that holds the walkthrough's state, called in order.
|
||
- **Manual / agent-driven live loop** — the `chrome-devtools` MCP (browser on
|
||
`127.0.0.1:9222`, started with `--remote-debugging-port=9222`), the Stripe MCP, and `stripe listen`
|
||
webhook forwarding, for flows not yet codified — e.g. a real Stripe Checkout →
|
||
`checkout.session.completed` → reconcile → the member UI reflecting the new
|
||
plan. Not part of `go test`.
|
||
|
||
Setup for all three lives in [`test/AGENTS.md`](../test/AGENTS.md) (worktree stack
|
||
contract; the "Stripe webhook forwarding" section for the live loop).
|
||
|
||
### 8. Live DNS verification (domain claims, manual)
|
||
|
||
Some of the domains registry can only be proven against a real zone: that a
|
||
published challenge record activates a claim, and that publishing the challenge
|
||
*prefix* with the wrong token latches `evidence_at` (exempting the claim from the
|
||
abandonment ledger) without verifying. DB tests cover the branching; only real DNS
|
||
covers the resolver path.
|
||
|
||
You need a **disposable zone you control** — any domain whose records you can add
|
||
and remove, at any registrar or DNS host with an API. Keep it separate from
|
||
anything the deployment actually serves, and never commit credentials for it;
|
||
supply them from the environment at the moment you use them. If the harness blocks
|
||
outbound HTTP from the shell (the context-mode routing rules do), drive the DNS
|
||
provider's API from a sandboxed script instead.
|
||
|
||
The loop is: start a claim in the console, read its token, publish
|
||
`_member-console-challenge.<name>` as TXT, wait for a probe, assert, then delete the
|
||
record. Two cases are worth proving because they diverge — a TXT carrying the
|
||
challenge **prefix with a wrong token** must latch evidence *without* activating the
|
||
claim, while the **correct token** must activate it.
|
||
|
||
**The gotcha that will cost you ten minutes.** If the zone has a **wildcard record**
|
||
(`*.example` CNAME/ALIAS/TXT), resolvers synthesise it for any challenge name that
|
||
has no explicit record — so the probe reports "Found, but doesn't match" against
|
||
whatever the wildcard points at, long before you did anything wrong. Publishing an
|
||
explicit record stops the synthesis (RFC 4592 closest-encloser), but any cached
|
||
wildcard answer still wins until its TTL expires, so probes keep reporting
|
||
`mismatch` for minutes after DNS is already correct. Before concluding the prober is
|
||
broken, compare authoritative against cached:
|
||
|
||
```bash
|
||
dig +short @<authoritative-ns> TXT _member-console-challenge.<name> # what the zone says
|
||
dig +noall +answer TXT _member-console-challenge.<name> # what the console sees
|
||
```
|
||
|
||
Clean up the records you added, and leave alone any challenge record that backs a
|
||
claim you intend to keep verified.
|
||
|
||
### 9. OpenSpec validation (change workflow)
|
||
|
||
`openspec validate <change>` is not a Go test but is part of the definition of
|
||
done for a spec-driven change: it checks delta specs are well-formed (MODIFIED
|
||
requirement headers match the main spec, every requirement has a `####` scenario,
|
||
etc.). Run it before archiving a change.
|
||
|
||
## The test kit
|
||
|
||
`internal/testkit` and its sub-packages hold what tests share. Each package
|
||
imports only what its row allows, so any package's tests can use it without an
|
||
import cycle; `internal/testkit/imports_test.go` lists the dependencies of each
|
||
with `go list -deps` and fails when one reaches a package it forbids.
|
||
|
||
| Package | Holds | Imports |
|
||
|---|---|---|
|
||
| `testkit` | `Close`, which closes an `io.Closer` and fails the test on its error | no console package |
|
||
| `testkit/dbtest` | opening the test databases, scratch databases, counting the statements a request runs, `Q`, `AwaitLockWait` | `internal/db` and `internal/testkit` only (`internal/migrate` reaches nearly every package) |
|
||
| `testkit/htmltest` | parsing HTML and selecting from it with CSS | no console package |
|
||
| `testkit/golden` | comparing a test's output with a file under testdata, `-update` writing it, and naming the ids and times a run makes | no console package |
|
||
| `testkit/world` | the fixture builder, plan and billing fixtures and the grant, tier and subscription acts included | the console's data packages; never `internal/migrate`, `internal/server`, `internal/integrations`, an integration's `web` or `workflows`, or `internal/domains` |
|
||
| `testkit/authtest` | signed-in sessions, their cookies, and in-process requests | as `world` |
|
||
| `integrations/fedwiki/fedwikitest` | FedWiki's fixtures and fake farm, built on `dbtest` and `world` | never `internal/migrate`, `internal/server`, or FedWiki's own module, `web` and `workflows` packages |
|
||
|
||
## Choosing what to write
|
||
|
||
```
|
||
Is the behavior a pure function of inputs? → unit test (1)
|
||
Does it touch the DB / constraints / multi-table? → DB integration test (2)
|
||
Does it call the Stripe API? → add a stripetest mock (3)
|
||
Is it an HTTP handler / rendered page? → httptest render test (4)
|
||
Is it route/template wiring or render discipline? → make lint / lint-templates (5)
|
||
Is it a whole user flow (HTTP or operator UI)? → e2e (7): a test/e2e/ Go suite
|
||
Is it a flow only real infra can prove (live Stripe)? → the live MCP loop (7), see test/AGENTS.md
|
||
Does it depend on what a resolver actually returns? → live DNS on a zone you control (8)
|
||
```
|
||
|
||
Prefer the cheapest test that genuinely covers the behavior. Reach for an e2e
|
||
suite or the live loop (7) to *prove a flow works end-to-end*, then capture the
|
||
regression-proofing parts as (2)/(3) so they run in CI without a live account.
|
||
|
||
## Conventions & gotchas
|
||
|
||
- **Tests meet the same complexity limit as production code.** gocognit counts a
|
||
closure passed to `t.Run` toward the function that encloses it, so splitting a
|
||
test into subtests does not lower its score. A test over the limit splits into
|
||
named functions, runs its table rows through a named function, or uses the
|
||
kit's helpers.
|
||
- **Open the database through `dbtest`** (`Open`, `OpenE2E`), which skips the
|
||
test when `TEST_DATABASE_URL` or `E2E_DATABASE_URL` is unset, so `go test ./...`
|
||
stays green with no infrastructure. No test migrates the shared databases;
|
||
`test/reset-test-db.sh` does, and `dbtest.Open` refuses one it has not.
|
||
- **A test of a migration** applies it to a `dbtest.Scratch` database, never to
|
||
the shared one.
|
||
- **Prove a converted assertion with a deliberate break.** When a check moves
|
||
onto the kit (a substring onto `htmltest`, a hand-written wait onto
|
||
`browsertest`, a fixture onto `world`), apply one break that violates what the
|
||
assertion says it checks, not its incidental markup, and see the test fail
|
||
with that message. `scripts/mutation-check` runs a JSON list of breaks (file,
|
||
exact text to replace, replacement, package, test pattern, wanted message): it
|
||
runs each test unbroken first, applies one break at a time, restores the file,
|
||
and reports CAUGHT, MISSED or WRONG-MESSAGE. Run it with `-side old` on a
|
||
checkout of the parent commit and with `-side new` on yours, so a break the
|
||
old test missed shows up as a weaker assertion. A list for browser tests carries a `prepare` command
|
||
that resets and restarts the app before each break, and a command record
|
||
covers a break that only the lint shows. The header of
|
||
`scripts/mutation-check/main.go` gives the format.
|
||
- **Don't trust webhook payloads in tests** the way production doesn't: a Stripe
|
||
test should drive behavior through `stripetest.MockBackend` returning canned
|
||
*API* state, not by hand-building the event payload's fields.
|
||
- **No `t.Parallel()` in Stripe-mocked tests** — they share global SDK state.
|
||
- **Unique fixture names** for tests that write to the shared test databases:
|
||
`world` suffixes every name it writes, and `dbtest.Name` returns a unique
|
||
lowercase name for the rest. `make test` resets the databases; tear the stack
|
||
down with `test/teardown-stack.sh` when you want a clean slate.
|
||
- **A wildcard CNAME shadows challenge lookups** — see (8). A domain-verification
|
||
probe reporting `mismatch` against a record you just published is usually a
|
||
cached wildcard, not a defect.
|
||
- **Accumulated rows can break the browser walkthroughs.** The DB-gated suites
|
||
write to `member_console_test` and `member_console_e2e` and never clean up, but
|
||
`make test` resets both, and no package test writes the app database. The
|
||
walkthroughs read the app database, so rows that pile up there
|
||
(`core.plan_ladders`, `core.org_types`) come from the walkthroughs themselves
|
||
or from manual use.
|
||
Check row counts before believing a walkthrough regression.
|
||
|
||
## Lint and screens (the UI quality gate)
|
||
|
||
A UX walk cannot hold the console to one shape: a page that differs from its
|
||
siblings costs the walker nothing they report, so the walk's severity scale
|
||
rates a consistency finding minor and triage tracks it. Sameness holds only
|
||
when the build fails on it and the maintainer reviews a capture of every
|
||
screen, which is what these two commands are for.
|
||
|
||
- **`make lint`** runs `member-console lint`: dead routes and swap targets,
|
||
retired resource keys, raw error rendering,
|
||
walkthrough coverage, and the page-anatomy rules (`design-system.md` §6:
|
||
raw titles, badges, and empty states outside the shared parts,
|
||
`hx-confirm`, load-triggered fetches, em dashes in copy, heading skips) plus screen coverage
|
||
(every GET page route in `test/e2e/screens/manifest.go`). Templates not
|
||
yet rebuilt on the parts are listed in `internal/lint/anatomy_allowlist.txt`,
|
||
which only shrinks; a stale entry fails. A template the lint cannot parse
|
||
fails it, naming the file. `make test` runs it first; the Go checks are in (5).
|
||
- **`make screens`** first restores the application database from a demo
|
||
snapshot (`test/reset-app-db.sh`; the one deliberate exception to "nothing
|
||
under `test/` touches `member_console`"). The snapshot is built once:
|
||
migrate, `seed-demo --with-operator`, boot data state, row clocks pinned by
|
||
`test/pin-timestamps.sql` to 2026-01-01 09:00 UTC plus one second per
|
||
distinct original instant in the column's own order (a dense rank), so
|
||
distinct instants stay distinct, each table keeps the chronology its rows
|
||
were written in, and ties stay ties (the two rows one conferral act writes
|
||
keep one instant, which the Tier changes fold relies on; the ledger query's
|
||
id tiebreak orders inside a tie);
|
||
every reset recreates the
|
||
application database from it, so rows and their IDs are identical between
|
||
captures and a page nobody touched never reads as changed (only relative
|
||
ages tick, once a day). The snapshot rebuilds itself when the migrations,
|
||
the seed, or the clock pin change (their fingerprint is the snapshot's
|
||
database comment); `./reset-app-db.sh --rebuild` forces it., then captures every manifest entry at 1600×1000 and 390×844
|
||
(rod, logged in as alice for the operator surface and bob for the member
|
||
surface, one incognito context each) into `test/screens/out/`, writes
|
||
`operator.html` and `member.html` contact sheets in rail order, and
|
||
compares each PNG with `test/screens/baseline/`, listing changed and new
|
||
screens in `changes.txt` with the differing region's size and position. Parameterised routes resolve their instance from
|
||
the seeded list page (the first record link, or the one a manifest entry
|
||
prefers by its link text) and report "no instance" when the stack has none.
|
||
`make screens-accept` promotes the output to the baseline. Both directories
|
||
are gitignored; the stack in `test/.env` must be up and seeded. A change
|
||
touching templates or `app.css` is not done until its sheet is reviewed
|
||
(`first-contact-ux-process.md`, "Definition of done").
|
||
|
||
## Related docs
|
||
|
||
- [`test/AGENTS.md`](../test/AGENTS.md) — the test stack (ports, webhook forwarding, demo seed).
|
||
- [`docs/database-management.md`](database-management.md) — migrations & sqlc.
|
||
- [`docs/stripe.md`](stripe.md) — the member-console ↔ Stripe responsibility split.
|