docs/go-conventions.md says a failed read is never an answer, and nothing checked it. nilerr refuses one form of that: a function that returns an error catches one and returns nil. On the tree it found 15 such returns. 12 were the project lint's walkers, which the parent commit replaced. The other three: - CheckQuotaActivity answers a failed read of the sites entitlement as a quota of 0, which tells the member they have reached their site limit. That is #231; the return is pinned in .golangci.yml by its source line until #231 removes it. - paymentMethodID in the invoice reconcile returned no id when the mapped payment method id did not parse. The column is UUID NOT NULL, so that cannot happen with the data the schema allows; the function now returns the error, so a row that breaks the rule is reported instead of read as unlinked. Not probed: no test can store a value the column refuses. - The domain claim's connect probe answers a failed lookup of the operator's own target as ProbeStateError, the state that says the record could not be checked. That is the probe's answer, so the return is exempt with that reason. With the pins, golangci-lint reports none. 2 breaks, each caught by scripts/golangci-lint.sh: a read error returned as nil in the lint's readTree, and the payment method id's parse error returned as nil. The rule's page names nilerr as its check and says what it leaves to review: a failed read turned into an empty list or a zero by a function that returns a value. docs/testing.md lists nilerr among the linters. make lint passes; the Stripe workflows and lint packages pass with the database tests skipped (the test stack was down).
34 KiB
title, audience, summary
| title | audience | summary | |
|---|---|---|---|
| Testing |
|
The test taxonomy (unit, DB-gated integration, Stripe-mocked, HTTP render, lint, e2e, OpenSpec validation), the test kit, how to run each, and conventions for new tests. |
Testing guide
How testing works in member-console: the kinds of tests, what each covers, how to run them, and the conventions to follow when writing new ones.
TL;DR
# The whole suite: lint, a reset of the test databases, then every package
# one at a time (needs the test stack up):
cd test && ./bootstrap-stack.sh && docker compose up -d # one-time per worktree
make test # from repo root
# One package while iterating, on the same footing as make test:
(cd test && ./reset-test-db.sh)
set -a && . test/.env && set +a
go test -count=1 ./internal/<package>/
# Static checks and render discipline:
make lint # every static check (type 5 lists them)
make lint-templates # no handler bypasses SafeTemplates
go test ./... with no database configured is always safe to run: every
database- or Stripe-backed test skips itself when TEST_DATABASE_URL is
unset. With it set, a database that reset-test-db.sh has not migrated fails
the test instead of passing on a stale schema.
The test/ directory — the local stack
Most Go test files sit next to the code they cover (internal/.../<name>_test.go).
test/ is the exception, and serves two roles: it's the integration/e2e
environment — a Dockerized stack of every backing service the app needs
(Postgres, Valkey, Keycloak/OIDC, Temporal, FedWiki) that the database-gated
tests (types 2–3) and the e2e suites (type 7) run against — and it houses the
higher-level end-to-end Go test suites under test/e2e/.
Its role:
- Provides the backing services.
test/compose.yamlbrings up Postgres, Valkey, Keycloak, Temporal, and FedWiki, plus one-shot seeders (realm users, FedWiki identity, the Temporal default namespace). - Isolates each worktree.
./bootstrap-stack.shderives collision-free host ports and writestest/.env(COMPOSE_PROJECT_NAME, port assignments, and theMC_*config overrides), so several worktrees can run their own stacks concurrently without colliding. - Is where the test databases come from.
./reset-test-db.shdrops, recreates and migratesmember_console_test(TEST_DATABASE_URL) andmember_console_e2e(E2E_DATABASE_URL);make testruns it first.test/.envholds both addresses besideMC_DB_DSN, the running app's database, which only the browser tests write. - Houses the end-to-end Go suites under
test/e2e/:plan-management/(in-processhttptestover the real handlers + a migrated DB) andoperator-walkthroughs/(Go tests that drive a browser against the running stack viaMC_BASE_URL;WALKTHROUGH_HEADFUL=1to watch). Both run undergo test(see type 7). - Holds the stack's run-only assets:
mc-config.yaml,secrets/(gitignored Stripe keys + webhook secret), the opt-in demo seeder (./test/seed-demo.sh+seed/), andteardown-stack.sh(compose down + removes.env,secrets/,testdata/).
Don't edit test/.env by hand — re-run bootstrap-stack.sh. The full contract
(idempotency, the account-global Stripe webhook constraint, teardown) lives in
test/AGENTS.md; read it before running the stack.
The kinds of tests
1. Pure unit tests — no external dependencies
Plain Go tests of pure logic: parsing, classification, formatting, guard
conditions. They run on every go test invocation, need nothing external, and
are fast.
- Cover: webhook payload parsing, status classification, value mapping, small helpers, input guards.
- Examples:
internal/fulfillmentTestClassifyStatus,TestUnixToNullTime;internal/integrations/stripe/workflowsTestCheckoutSessionPayloadParsing,TestSubscriptionPayloadParsing,TestCardStringField_*. - Write one when: the behavior is a function of its inputs with no DB/HTTP/ Stripe dependency. Prefer pushing logic into such functions so it can be tested this way.
2. Database-gated integration tests
Run against a real Postgres that test/reset-test-db.sh has migrated. They
exercise queries, constraints (e.g. the pool_provision_ladders GiST
exclusion, chk_pool_provisions_source), transactions, and cross-schema flows
that unit tests can't reach.
- Cover: the entitlement graph, provisioning, fulfillment reconcile, webhook projection, materialization, operator queries.
- Examples:
internal/provisioning(TestAutoProvision…),internal/fulfillment(TestReconcile_*),internal/entitlements/materialize_test.go,internal/integrations/stripe/workflows(*_Integration, transition/product-catalog tests).
Opening the database. Every test reaches the shared databases through
dbtest (internal/testkit/dbtest):
dbtest.Open(t)opensTEST_DATABASE_URL, closes it when the test ends, and skips the test when the variable is unset. It never migrates: the reset script is the only migrator of the shared test database. It fails the test, naming the script, when the core ledger lacks the highest version indb.BaseSources(), so a stale database cannot pass a test. The driver is pgx.dbtest.OpenE2E(t)does the same forE2E_DATABASE_URL, the databasetest/e2e/plan-managementcommits its fixtures to.dbtest.DSN(t)returns the address (and skips) for a test that needs it itself.- A test whose subject is a migration or the migrator gets an empty database
from
dbtest.Scratch(t), created in the same cluster and dropped when the test ends, and applies a stream to it withdbtest.UpToordbtest.ApplyVersion, or all streams withdb.RunMigrationson adbtest.Connectto its address. It skips where the login role cannot create databases. dbtest.Counted(t)opensTEST_DATABASE_URLasOpendoes, on a handle that records the name of every statement run on it (ByName,Total,Reset;Nameslists them in the order they ran, a transaction'sbegin,commitandrollbackincluded;Noteputs a line of the test's own among them;Diffprints what changed between two records). Hand it to the code under test only and build fixtures on a handle fromOpen, so the record holds the code's statements and nothing else.dbtest.Qruns statements and fails the test on any error (Exec,Scan,Int,String,Refusefor a statement that must fail with a given SQLSTATE;Rowsprints a query's rows as text, in the order the query names; a rows golden names the fixture's ids with the test's normaliser and numbers the rest withgolden.Ordinals).dbtest.Tx(t, d)opens a transaction the test rolls back;dbtest.On(t, x)binds a statement runner to a test. AQbelongs to the test or subtest that built it: a subtest that uses its parent's transaction rebinds withdbtest.On(t, q.Tx()).dbtest.AwaitLockWaitpolls until a backend waits on a lock (Waitersfor a number of backends queued at once), for a test that holds one transaction open while a second one blocks.
Fixtures. world (internal/testkit/world) builds the rows a test needs
through the functions the console itself calls. Its builders cover people,
organizations (Org also creates the default pool), members and pools; the
catalog: products and their states, prices, ladders and tiers; rules, through
the system's path (Rule) or a person's commit that settles the carrying pools
(RuleChange); grants, conferred, issued, extended or revoked, and the floor;
billing: accounts, subscriptions and their scheduled changes, invoices and
their lines, payments, and the Stripe mapping of each (the ...In forms record
an environment, the others record none); and workspaces, usage and the
provider registration boot runs. OrgType is the one raw INSERT, since no
console act creates an organization type. world.New(t, db) takes any
*sql.DB the migrations built, so one builder fills the test database or the
application database. Every method fails the test on any error. Every
organization, product, set, ladder name, subject and email carries the World's
suffix, so repeated runs against the shared database do not collide, and nothing
is deleted: rows stay in the disposable database until the next reset. A World
holds the testing.TB it was built with, so a subtest builds its own with
world.New(t, db). The first sign-in's rows (user, person, organization with
its membership, workspace) come from
provisioning.AutoProvision(ctx, db, claims).
A test of the schema itself writes its rows by SQL: a constraint, a trigger or a migration is the subject, and a builder that went through the console's functions would not exercise it.
3. Stripe-mocked tests
A specialization of the integration tests: the Stripe SDK is pointed at an in-process fake so fulfillment's "refetch from the Stripe API" path runs deterministically and offline, with no live Stripe account.
- Helper:
internal/stripetest—MockBackendimplementsstripe.Backendand answers each read from its fields (Sub,List,Invoices,Products,Prices,Scheduleand others; each field's comment names the call it answers).Errfails every call,ObjectErrsfails one Stripe id andPathErrsfails one API path.stripetest.Install(m)swaps the global backend + a dummy key and returns a restore func (t.Cleanup(stripetest.Install(m))).m.OnCallsees each request as it is recorded, andstripetest.Line(c)prints one as text, som.OnCall = func(c stripetest.Call) { stmts.Note(stripetest.Line(c)) }puts each Stripe call among the statements of adbtest.Countedhandle.stripetest.KeyModeanswers the mode of the key the SDK holds when it is called ("test"underInstall); a test passes it where the console passes its configured key mode. - Cover:
fulfillment.ReconcileSubscriptionscenarios (empty-payload provisioning, idempotent replay, ordering, cancellation, multi-ladder guard) and the webhook→plan-transitionsend-to-end tests. - Trace tests:
TestReconcileTrace,TestScheduledChangeTraceandTestSweepTracerecord a reconcile's or the sweep's statements in order with the Stripe calls between them, its log lines and the rows it leaves, in goldens underinternal/fulfillment/testdata.TestSweepTrace's cases run on a scratch database each, because the sweep fires every due row in its database. - Gotcha:
Installmutates global stripe-go state, so these tests must not callt.Parallel().
4. HTTP handler / render tests
Exercise handlers through net/http/httptest plus the real SafeTemplates
renderer, session manager (scs), and CSRF middleware.
- Cover: rendered output, template wiring, auth guards (e.g. 401 for unauthenticated), CSRF behavior.
- Examples:
internal/server/member_upgrade_render_test.go(TestMemberPlansUpgradeControlRendering,TestDashboardCheckoutBanner,TestGetPlansUnauthenticatedReturns401);internal/middleware/tests/security_headers_test.go. - Template sets come from the constructors production uses. The server
package's test harness (
internal/server/harness_test.go) builds them:OperatorSet(t)isweb.OperatorTemplatesandMemberSet(t)isweb.MemberTemplates, both overweb.CoreTemplates(), andRender(t, set, name, data)executes one template. A test never builds a stand-in set or registers template functions itself. - Query the output with
htmltest(internal/testkit/htmltest):htmltest.Parse(t, body)parses a page or a fragment, andOne(exactly one match),All,Count,None,Text,Attr,HasClass,BeforeandInOrderselect and compare with CSS selectors, the vocabulary the browser walkthroughs use. Select by role and byaria-*,hx-*anddata-*attributes (nav[aria-label=Account],[role="menu"] button[hx-post="/logout"]); a class that only styles an element is not what a test asserts. A failure quotes the selector and a snippet of the markup. The harness addsShellBar,ShellRailandLinkNamedfor the shell. - Sign in through
authtest(internal/testkit/authtest):authtest.NewSessionManager()is the in-memory session manager, andauthtest.SignIn(t, sm, signIn)writes the session throughauth.PutSignIn, the function the callback handler calls, so a test session has the keys and values a real sign-in produces.authtest.Getandauthtest.Postrun a handler in process with that session on the request;authtest.PathValuesets a path parameter as the mux would.GetandPostsend no htmx headers, so a handler answers as it does to a native request.authtest.Cookiecommits that session and returns its cookie, for a handler wrapped in the session middleware, such as the whole router. - Compare with a golden file through
golden(internal/testkit/golden):golden.Compare(t, file, got)fails unlessgotis the file's content, andgo test -updatewrites the file. A golden file is written once, with-update, and read against its fixture before it is committed; after that it changes only in a commit that names the change. Agolden.Normalizerturns the ids a fixture made into names (Name) andgolden.Ordinalsnumbers the ids and times an act made. A test package that defines its ownupdateflag cannot importgolden, because the flag would be defined twice.- A golden holds output in a defined order: its query ends on a unique key and its Go code ranges over sorted map keys (go-conventions.md, "Output order"). On the shared test database it holds only the rows of the test's own fixture.
- When a change should move a golden, edit the expected lines into the file
by hand first; the test then passes without
-update, which shows the change was intended. Use-updateto write a new golden, or read its diff line by line. - Pass
-updateone package at a time: a package that does not define the flag fails to start with it, sogo test ./... -updatefails.
5. Static checks (make lint)
Not Go tests, but part of the green-bar definition. make lint runs five
checks, and make lint-templates is a sixth that runs on its own:
go run . lint(cmd/lint.go) — operator-UI integrity. It parses each template in the set its surface builds in production (web.MemberTemplatesandweb.OperatorTemplates, with each integration's directory as a mount), so a template it cannot parse fails it, naming the file, and so does a file that no surface parses. Its rules, by the id each finding carries:- routes and URLs:
dead-route,dead-swap-target,interpolated-literal-url,retired-resource-key; - coverage:
walkthrough-coverage(every operator mutation route has a walkthrough claim) andscreen-coverage(every GET page route has an entry intest/e2e/screens/manifest.go); - page anatomy (
design-system.md§6):page-without-header,raw-page-title,raw-section-title,raw-badge,raw-empty-state,raw-form-control,form-without-part,disabled-outside-part,table-without-list-controls,hx-confirm,load-fetch,heading-skip. Templates not yet rebuilt on the parts are listed ininternal/lint/anatomy_allowlist.txt, which only shrinks:allowlist-stalefails an entry that no longer matches, andallowlist-unreadablea list it cannot read; - copy:
em-dash-copy,filler-copy; - htmx:
htmx-2-compat,htmx-old-event,hx-disinherit; - error text:
raw-error-render; - reading:
unreadable-file, a directory the lint cannot list or a file it cannot read, so that nothing passes the lint unread.
- routes and URLs:
scripts/notebook-citations.sh— no tracked file cites a local notebook path.scripts/golangci-lint.sh— golangci-lint with.golangci.yml: the standard set (vet, staticcheck, unchecked errors with errcheck, dead assignments with ineffassign, unused code with unused), plus error comparison (errorlint), an error caught and then returned as nil (nilerr),rowsandsqlhandling (rowserrcheck, sqlclosecheck), unused parameters (unparam), duplicated code (dupl), function complexity (gocognit, limit 30) and forbidden identifiers (forbidigo), among them the rod waits the browser tests refuse (type 7).scripts/deadcode.sh— production functions no program reaches, except those listed inscripts/deadcode-testonly.txt.go run ./scripts/sqlc-unused— generated queries with no caller outside generated and test files.make lint-templates— guards that no handler bypassesSafeTemplatesby calling.ExecuteTemplate(w, …)directly (CSP/render discipline; seeinternal/server/render.go).
.golangci.yml lists the findings a later change removes, one rule per
finding, pinned to the function's current score and commented with the issue
that removes it. A rule that stops matching fails the lint, so the list only
shrinks.
6. Security scanners (static, offline)
A security audit runs two passes, in this order: the deterministic scanners
(gosec, govulncheck, semgrep, staticcheck, gitleaks) run first and offline,
and a model then triages their hits. The order is deliberate — a model
adjudicating scanner output cuts semgrep's false-positive rate from about
0.42 to about 0.05 at a cost of about 3% relative recall, while a model
reading code unaided over-reports. The method the audit follows is
docs/agent-runner.md.
7. End-to-end tests
Exercise whole flows against a running stack. Three flavors:
- Go HTTP-level e2e —
test/e2e/plan-management/: an in-processhttptestharness over the real handlers + a migrated DB (E2E_DATABASE_URL), with fixtures fromworldand requests fromauthtest. Runs undergo test; covers operator plan/ladder/grant flows end-to-end at the HTTP layer. - Go browser walkthroughs —
test/e2e/operator-walkthroughs/: Go tests that drive a real browser against the running stack (readsMC_BASE_URLfromtest/.env; setWALKTHROUGH_HEADFUL=1to watch). One walkthrough per operator route group, each asserting the four-bullet UI contract (empty/invalid → 422, valid → success + DOM update, …). Skips unless the stack is up and seeded. The browser tests (the walkthroughs,test/e2e/screensandtest/e2e/merchant-disclosure) sharetest/e2e/browsertest, whose rules are:- Waits are boolean predicates.
WaitTrue,TryWaitTrue,WaitLoadedandWaitReadypoll a JavaScript function that returns a boolean, and a missing or destroyed execution context counts as "not yet".Eventuallypolls a Go probe that returns a boolean, and rod's ownMustWaittakes a JavaScript predicate the same way. golangci-lint refuses rod'sWaitLoadandWaitStable(MustWaitLoad,MustWaitStable) on a page or element, with the message "wait on a predicate that returns a boolean (browsertest)". The one wait without a predicate is rod'sWaitDOMStable, for a page that may never stop changing and is captured as it stands. ClickAndLandfor an answer that is a new document (anHX-Redirect, a native redirect): it anchors the wait to the new document, not to a lifecycle event.AwaitSwapfor an htmx swap: it runs the action and waits for htmx to settle.Recover. Each walkthrough defersbrowsertest.Recover(t)first, so a rod panic fails that walkthrough with its stack and the package's other walkthroughs still run.AppDBopens the running app's database (MC_DB_DSNintest/.env), never migrates it, and skips when the key is missing.- Free functions that take
t, and no helper stores it, so a subtest never calls a parent's. - A long walkthrough is phases: named functions, or methods on a struct that holds the walkthrough's state, called in order.
- Waits are boolean predicates.
- Manual / agent-driven live loop — the
chrome-devtoolsMCP (browser on127.0.0.1:9222, started with--remote-debugging-port=9222), the Stripe MCP, andstripe listenwebhook forwarding, for flows not yet codified — e.g. a real Stripe Checkout →checkout.session.completed→ reconcile → the member UI reflecting the new plan. Not part ofgo test.
Setup for all three lives in test/AGENTS.md (worktree stack
contract; the "Stripe webhook forwarding" section for the live loop).
8. Live DNS verification (domain claims, manual)
Some of the domains registry can only be proven against a real zone: that a
published challenge record activates a claim, and that publishing the challenge
prefix with the wrong token latches evidence_at (exempting the claim from the
abandonment ledger) without verifying. DB tests cover the branching; only real DNS
covers the resolver path.
You need a disposable zone you control — any domain whose records you can add and remove, at any registrar or DNS host with an API. Keep it separate from anything the deployment actually serves, and never commit credentials for it; supply them from the environment at the moment you use them. If your harness blocks outbound HTTP from the shell, drive the DNS provider's API from a sandboxed script instead.
The loop is: start a claim in the console, read its token, publish
_member-console-challenge.<name> as TXT, wait for a probe, assert, then delete the
record. Two cases are worth proving because they diverge — a TXT carrying the
challenge prefix with a wrong token must latch evidence without activating the
claim, while the correct token must activate it.
The gotcha that will cost you ten minutes. If the zone has a wildcard record
(*.example CNAME/ALIAS/TXT), resolvers synthesise it for any challenge name that
has no explicit record — so the probe reports "Found, but doesn't match" against
whatever the wildcard points at, long before you did anything wrong. Publishing an
explicit record stops the synthesis (RFC 4592 closest-encloser), but any cached
wildcard answer still wins until its TTL expires, so probes keep reporting
mismatch for minutes after DNS is already correct. Before concluding the prober is
broken, compare authoritative against cached:
dig +short @<authoritative-ns> TXT _member-console-challenge.<name> # what the zone says
dig +noall +answer TXT _member-console-challenge.<name> # what the console sees
Clean up the records you added, and leave alone any challenge record that backs a claim you intend to keep verified.
9. OpenSpec validation (change workflow)
openspec validate <change> is not a Go test but is part of the definition of
done for a spec-driven change: it checks delta specs are well-formed (MODIFIED
requirement headers match the main spec, every requirement has a #### scenario,
etc.). Run it before archiving a change.
The test kit
internal/testkit and its sub-packages hold what tests share. Each package
imports only what its row allows, so any package's tests can use it without an
import cycle; internal/testkit/imports_test.go lists the dependencies of each
with go list -deps and fails when one reaches a package it forbids.
This page says what each package is for and names the helpers a new test
reaches for first. go doc -all ./internal/testkit/<package> lists every
exported name with its comment, and go doc -all ./internal/stripetest does
the same for the Stripe mock. A helper joins the kit with its first caller and
a test of its own.
| Package | Holds | Imports |
|---|---|---|
testkit |
Close, which closes an io.Closer and fails the test on its error |
no console package |
testkit/dbtest |
opening the test databases, scratch databases, counting the statements a request runs, Q, AwaitLockWait |
internal/db and internal/testkit only (internal/migrate reaches nearly every package) |
testkit/htmltest |
parsing HTML and selecting from it with CSS | no console package |
testkit/golden |
comparing a test's output with a file under testdata, -update writing it, and naming the ids and times a run makes |
no console package |
testkit/replay |
replaying a package's recorded Temporal histories against its workflows, and recording them | no console package |
testkit/world |
the fixture builder, plan and billing fixtures and the grant, tier and subscription acts included | the console's data packages; never internal/migrate, internal/server, internal/integrations, an integration's web or workflows, or internal/domains |
testkit/authtest |
signed-in sessions, their cookies, and in-process requests | as world |
integrations/fedwiki/fedwikitest |
FedWiki's fixtures and fake farm, built on dbtest and world |
never internal/migrate, internal/server, or FedWiki's own module, web and workflows packages |
Recorded workflow histories. A package whose workflows are replayed keeps recorded histories under testdata/replay (today claim verification, the desired-state push and FedWiki's workflows), and its TestWorkflowHistoriesReplay replays each against the workflows the worker registers (replay.Run). A replay fails when a change would break a run in flight at deploy. It compares the commands a workflow issues, not their inputs, the timer durations or the result, so a trace or golden test pins those beside it. The package's TestRecordReplayHistories writes the histories with stand-in activities when TEMPORAL_REPLAY_RECORD names a temporal CLI binary (a private dev server) or a running server's host:port. It writes only the files that do not exist; TEMPORAL_REPLAY_ONLY=<prefix> records the files with that prefix again. Record from the code a release shipped, never from edited code.
Choosing what to write
Is the behavior a pure function of inputs? → unit test (1)
Does it touch the DB / constraints / multi-table? → DB integration test (2)
Does it call the Stripe API? → add a stripetest mock (3)
Is it an HTTP handler / rendered page? → httptest render test (4)
Is it route/template wiring or render discipline? → make lint / lint-templates (5)
Is it a whole user flow (HTTP or operator UI)? → e2e (7): a test/e2e/ Go suite
Is it a flow only real infra can prove (live Stripe)? → the live MCP loop (7), see test/AGENTS.md
Does it depend on what a resolver actually returns? → live DNS on a zone you control (8)
Prefer the cheapest test that genuinely covers the behavior. Reach for an e2e suite or the live loop (7) to prove a flow works end-to-end, then capture the regression-proofing parts as (2)/(3) so they run in CI without a live account.
Conventions & gotchas
- Characterize before a refactor. A change that reorganizes code without
changing what it does first commits tests that pin what the code does today:
a golden of the view model and the HTML, the statements a request runs
(
dbtest.Counted), a table of what each failed read answers. Those tests pin today's defects too, and the commit message names them. The refactor commits that follow change no golden, screen, test name or result; a change in any of them is a finding to explain, not a file to update. A behaviour fix found on the way lands afterwards as its own commit with its own test, and names the golden it moves. - Tests meet the same complexity limit as production code. gocognit counts a
closure passed to
t.Runtoward the function that encloses it, so splitting a test into subtests does not lower its score. A test over the limit splits into named functions, runs its table rows through a named function, or uses the kit's helpers. - Open the database through
dbtest(Open,OpenE2E), which skips the test whenTEST_DATABASE_URLorE2E_DATABASE_URLis unset, sogo test ./...stays green with no infrastructure. No test migrates the shared databases;test/reset-test-db.shdoes, anddbtest.Openrefuses one it has not. - A test of a migration applies it to a
dbtest.Scratchdatabase, never to the shared one. - Prove a test with deliberate breaks. A new test, a test whose name or
comment states a value, and a check that moves onto the kit (a substring
onto
htmltest, a hand-written wait ontobrowsertest, a fixture ontoworld) each get one break per fact they claim: a change to the code that violates what the assertion says it checks, not its incidental markup, and makes the test fail with that message.scripts/mutation-checkruns a JSON list of breaks (file, exact text to replace, replacement, package, test pattern, wanted message): it runs each test unbroken first, applies one break at a time, restores the file, and reports each break as CAUGHT, MISSED, WRONG-MESSAGE, NOT-FOUND (the text does not occur exactly once), SKIPPED (the test skipped) or CONTROL-FAILED (the unbroken run failed). For a converted check, run it with-side oldon a checkout of the parent commit and with-side newon yours, so a break the old test missed shows up as a weaker assertion; a new test has only a new side. A list for browser tests carries apreparecommand that resets and restarts the app before each break, and a command record covers a break that only the lint shows. The header ofscripts/mutation-check/main.gogives the format. List the breaks in the commit message. - Read a MISSED break before acting on it. It means one of three things: the assertion is weaker than its message (tighten it, or name what it does not check); the break changes nothing a caller can observe, so no test can catch it (say so, with the reason); or the harness cannot see the effect, as a mobile-width screen cannot see a page wider than its viewport. Only the first is a weak test.
- Tests of locks and of order are deterministic. A test that holds one
transaction open while a second blocks waits for the block with
dbtest.AwaitLockWait, never with a sleep. A test of an order feeds its input in more than one order, because input that is already sorted cannot fail. - A flaky test is a suspected defect until a diagnosis says otherwise. Two
flaky tests in this repository turned out to be a race in
core.conferand a query with no unique sort key. Rerun a failing test alone and record the result; never update its expected output to match one run. - A database test passes twice in a row without a reset, and never asserts on an unfiltered list: other tests' rows share the database (#192), so a test filters to the rows its own fixture made.
- Don't trust webhook payloads in tests the way production doesn't: a Stripe
test should drive behavior through
stripetest.MockBackendreturning canned API state, not by hand-building the event payload's fields. - No
t.Parallel()in Stripe-mocked tests — they share global SDK state. - Unique fixture names for tests that write to the shared test databases:
worldsuffixes every name it writes, anddbtest.Namereturns a unique lowercase name for the rest.make testresets the databases; tear the stack down withtest/teardown-stack.shwhen you want a clean slate. - A wildcard CNAME shadows challenge lookups — see (8). A domain-verification
probe reporting
mismatchagainst a record you just published is usually a cached wildcard, not a defect. - Accumulated rows can break the browser walkthroughs. The DB-gated suites
write to
member_console_testandmember_console_e2eand never clean up, butmake testresets both, and no package test writes the app database. The walkthroughs read the app database, so rows that pile up there (core.plan_ladders,core.org_types) come from the walkthroughs themselves or from manual use. Check row counts before believing a walkthrough regression.
Lint and screens (the UI quality gate)
A UX walk cannot hold the console to one shape: a page that differs from its siblings costs the walker nothing they report, so the walk's severity scale rates a consistency finding minor and triage tracks it. Sameness holds only when the build fails on it and the maintainer reviews a capture of every screen, which is what these two commands are for.
make lintholds every page to one shape through the project lint's page-anatomy, copy and screen-coverage rules; type 5 lists every rule and the other checksmake lintruns.make testruns it first.make screensfirst restores the application database from a demo snapshot (test/reset-app-db.sh; the one deliberate exception to "nothing undertest/touchesmember_console"). The snapshot is built once: migrate,seed-demo --with-operator, boot data state, row clocks pinned bytest/pin-timestamps.sqlto 2026-01-01 09:00 UTC plus one second per distinct original instant in the column's own order (a dense rank), so distinct instants stay distinct, each table keeps the chronology its rows were written in, and ties stay ties (the two rows one conferral act writes keep one instant, whichcore.superseded_endspairs them by; the ledger query's id tiebreak orders inside a tie); every reset recreates the application database from it, so rows and their IDs are identical between captures and a page nobody touched never reads as changed (only relative ages tick, once a day). The snapshot rebuilds itself when the migrations, the seed, or the clock pin change (their fingerprint is the snapshot's database comment);./reset-app-db.sh --rebuildforces it.make screensthen captures every manifest entry at 1600×1000 and 390×844 (rod, logged in as alice for the operator surface and bob for the member surface, one incognito context each) intotest/screens/out/, writesoperator.htmlandmember.htmlcontact sheets in rail order, and compares each PNG withtest/screens/baseline/, listing changed and new screens inchanges.txtwith the differing region's size and position. Parameterised routes resolve their instance from the seeded list page (the first record link, or the one a manifest entry prefers by its link text) and report "no instance" when the stack has none.make screens-acceptpromotes the output to the baseline. Both directories are gitignored; the stack intest/.envmust be up and seeded. A change touching templates orapp.cssis not done until its sheet is reviewed (first-contact-ux-process.md, "Definition of done").
Related docs
test/AGENTS.md— the test stack (ports, webhook forwarding, demo seed).docs/database-management.md— migrations & sqlc.docs/stripe.md— the member-console ↔ Stripe responsibility split.