Files

T

cgalo5758 b7447bea28 Isolate tests in per-purpose databases

Guard cluster-global CREATE ROLE in all five migration streams with
pg_roles checks so multiple databases can migrate in one cluster, and
tolerate still-referenced roles on Down.

Add test/reset-test-db.sh to drop and recreate member_console_test and
member_console_e2e per run, emit their DSNs from bootstrap, and add a
make test target that resets then runs the suite serialized; parallel
unit packages sharing one database still interfered even after the e2e
split.

Fix customdomain_db_test.go, stale since 0affda7 and previously passing
only through pollution. Bootstrap and the Makefile carry small forward
references to the compose-profile knob introduced next.

Archives the test-db-isolation change.

2026-08-01 04:13:57 -05:00

5.3 KiB

Raw Blame History

test-database-isolation

Purpose

Defines how automated tests get pristine, dedicated databases per run, isolated from the application database and from each other's committing suites, so a suite result reflects the code under test rather than accumulated state.

Requirements

Requirement: Tests run against dedicated databases, never the application database

Automated tests SHALL read their DSN from environment variables that name databases distinct from the application's db-dsn database. Package tests SHALL use TEST_DATABASE_URL; the in-process end-to-end suites, which commit fixtures that package tests assert on, SHALL use E2E_DATABASE_URL. Neither SHALL default to the application database. A test that finds its variable unset SHALL skip with a message naming the variable, not fail and not fall back to another database.

Scenario: Live application cannot perturb package tests

WHEN the application is running against its own database and its boot reconcilers commit changes (domain backfill, orphaned-placement sweep, entitlement materialization)
THEN no row visible to a package test SHALL change
AND DB-backed package tests SHALL pass unchanged while the application is running

Scenario: Committing e2e suite cannot perturb package tests

WHEN an in-process e2e suite commits fixtures that a package test asserts the absence of, such as an org type's default plan ladder
THEN the package test SHALL be unaffected, because the two suites hold separate databases

Scenario: Unset variable skips rather than fails

WHEN a DB-backed test runs with its DSN variable unset
THEN the test SHALL skip
AND the skip message SHALL name the missing variable

Requirement: The harness runs test packages serially

The harness SHALL run package test binaries one at a time rather than concurrently, and SHALL disable the language toolchain's test result cache, whose key does not include database state. Several packages commit rows that other packages read and assert on (an org type's default plan ladder, a provider's group mappings), and interleaving them fails intermittently. Serialization is not a substitute for the database split: ordering packages does not undo a commit that outlives the package which made it.

Scenario: Packages that mutate shared rows do not interleave

WHEN one package temporarily sets a committed row that another package asserts is unset, and restores it before finishing
THEN the asserting package SHALL never observe the intermediate state

Scenario: A reset invalidates cached results

WHEN the harness resets the databases and runs the suite again with no source change
THEN every test SHALL execute again rather than reporting a cached result from before the reset

Requirement: The test harness resets its databases at the start of a run

The repository SHALL provide a harness step that, before the suite runs, connects to the cluster's maintenance database and drops and recreates each test database, then applies migrations to it. The reset SHALL terminate existing sessions on the target so a leftover connection cannot block it. It SHALL NOT touch the application's database. Migrating as part of the reset is load-bearing: it leaves every parallel test package's own migration call a no-op, so concurrent packages do not race to create the same objects.

Scenario: Every run starts from migrations only

WHEN the harness reset runs
THEN each test database SHALL contain exactly what the migration streams create, with no rows committed by a previous run

Scenario: Repeated suite runs are independent

WHEN the full suite is run twice back to back through the harness
THEN both runs SHALL produce the same result
AND no test SHALL fail on a uniqueness violation or a precondition that a previous run's committed fixture invalidated

Scenario: Reset leaves the application database alone

WHEN the harness reset runs against a stack whose application database holds seeded demo data
THEN the application database SHALL be untouched
AND an application already connected to it SHALL keep running

Scenario: Open connections do not block the reset

WHEN a previous run left a connection open against a test database
THEN the reset SHALL terminate that session and complete

Requirement: Test database names and DSNs are per-worktree generated

test/bootstrap-stack.sh SHALL write the test-database DSNs and the maintenance DSN into the generated test/.env, derived from the same worktree-allocated Postgres host port as MC_DB_DSN, so concurrent worktree stacks address their own cluster. The harness reset and the suite SHALL both read these values from test/.env rather than hardcoding a port or database name.

Scenario: Bootstrap emits the test DSNs

WHEN test/bootstrap-stack.sh generates test/.env
THEN the file SHALL contain TEST_DATABASE_URL, E2E_DATABASE_URL, and a maintenance DSN
AND each SHALL use the same host port as MC_DB_DSN
AND the test DSNs SHALL name databases other than the one in MC_DB_DSN

Scenario: Two worktrees reset independently

WHEN two worktree stacks are up and each runs the harness reset
THEN each SHALL act only on its own cluster's databases

5.3 KiB Raw Blame History