member-console/docs/database-management.md

title, audience, summary

title	audience	summary
Database Management	developer admin	goose migrations, sqlc code generation, and the core + per-integration schema and migration-stream conventions.

Database Management

This project uses pressly/goose for database migrations and sqlc for type-safe SQL code generation. Requires PostgreSQL 18+ (for native uuidv7()).

Schema layout

The database has three PostgreSQL schemas:

• core — every domain table (identity, organization, billing, entitlements, and the provider registry that integrations register against). This is the single source of business truth; there is no per-Go-package schema split. internal/db/migrations/ owns this stream.
• stripe — the Stripe provider's own mapping tables (customers, subscriptions, invoices, etc.), FK'd into core. internal/integrations/stripe/store/migrations/ owns this stream.
• fedwiki — the FedWiki provider's own tables (sites, site swap policy), FK'd into core. internal/integrations/fedwiki/store/migrations/ owns this stream.

The Go package layout does not mirror the schema layout: internal/identity, internal/organization, internal/billing, internal/entitlements, and internal/integration are separate packages that all read and write tables in the single core schema. Schema boundaries track core domain vs. provider integration, not Go package boundaries — see the schema-consolidation change record under openspec/changes/archive/ for the rationale.

Per-Source Migrations

Each schema is owned by exactly one embedded migrations/ directory:

internal/db/migrations/                          # core schema (identity, organization, billing, entitlements, provider registry)
internal/integrations/fedwiki/store/migrations/   # fedwiki schema (sites, site_swap_policy)
internal/integrations/stripe/store/migrations/    # stripe schema (mapping tables)

Each source runs against its own goose ledger table, goose_db_version_<name> (goose_db_version_core, goose_db_version_fedwiki, goose_db_version_stripe), and keeps its native 00001… file numbering. There is no shared goose_db_version table and no global version namespace — a source's version numbers are independent of its position in the dependency-ordered source list returned by internal/migrate.Sources(). internal/db/migrations.go loops the sources in order, pointing goose at each source's embedded FS and ledger table in turn.

Source order matters only weakly: core must run first, because the FK DAG is one-directional — integration schemas reference core, never the reverse and never each other — so core's tables must exist before any integration's migrations apply. Beyond that, integration sources are mutually order-independent. Appending a new integration source, or reordering integration sources relative to each other, does not renumber or otherwise disturb any other stream's ledger.

Rollback (migrate down) walks the sources in reverse and rolls back one step of the most-recently-populated stream — integrations before core. Status (migrate status) prints each stream's ledger under its own == <name> == header.

Migrations run automatically on startup. The CLI also provides migrate up, migrate down, and migrate status commands.

Creating New Migrations

# Install goose CLI tool
go install github.com/pressly/goose/v3/cmd/goose@latest

# Create a new migration in the appropriate source directory
cd internal/db/migrations       # or internal/integrations/stripe/store/migrations,
                                 # internal/integrations/fedwiki/store/migrations
goose create your_migration_name sql

Always Use StatementBegin / StatementEnd

Every migration with more than one SQL statement must wrap both Up and Down sections in -- +goose StatementBegin / -- +goose StatementEnd. Without these directives, goose v3 splits statements and runs them individually, which can silently skip failed statements while still marking the migration as applied.

Migrations Run Without search_path

The ConnectAndMigrate function runs migrations on a separate database connection that does not include the custom search_path (core,public). This is intentional — a pgx/PostgreSQL 18 interaction causes multi-statement DDL sent via the simple query protocol to silently lose tables when a non-default schema appears in the connection-level search_path. After migrations complete, the application connection uses the full search_path as normal.

-- +goose Up
-- +goose StatementBegin

CREATE TABLE core.example (...);
CREATE INDEX idx_example ON core.example(...);
GRANT ALL ON ALL TABLES IN SCHEMA core TO core_owner;

-- +goose StatementEnd

-- +goose Down
-- +goose StatementBegin

DROP TABLE IF EXISTS core.example;

-- +goose StatementEnd

This ensures PostgreSQL receives the entire block as a single unit, so failures are atomic — either all statements succeed or none are applied.

Roles

Each schema has an owner/writer/reader role triple: core_owner / core_writer / core_reader, stripe_owner / stripe_writer / stripe_reader, fedwiki_owner / fedwiki_writer / fedwiki_reader. Each triple is created by its owning stream's own baseline migration, not by any other stream: the core baseline (internal/db/migrations/00001_init.sql) creates only the core_* triple, and each integration's baseline (internal/integrations/<slug>/store/migrations/00001_init.sql) creates its own {slug}_owner / {slug}_writer / {slug}_reader triple. Because integration tables FK into core, each integration's baseline also grants core_reader to its own {slug}_writer; this is safe because core always migrates first (see internal/migrate.Sources()), so core_reader already exists by the time an integration's baseline runs. The member_console login role (the app's DSN user) ends up a member of core_writer and every {slug}_writer — membership in core_writer is granted by the core baseline, and membership in each {slug}_writer is granted by that integration's own baseline. Owner roles are reserved for running migrations. Each schema's authoring stream is solely responsible for its own roles, grants, and member_console membership — there is no privilege wiring performed from outside a schema's own migration stream.

sqlc Code Generation

Each Go package has its own sqlc.yaml that generates type-safe Go code from SQL queries.

# Regenerate sqlc code after schema or query changes
cd internal/entitlements && sqlc generate

internal/db/sqlc_schemas.sql

This file is not a goose migration — it is a sqlc-only helper that declares all PostgreSQL schemas (CREATE SCHEMA IF NOT EXISTS ...) so that sqlc can resolve schema-qualified table names (e.g., core.accounts) across packages. It declares core, stripe, and fedwiki, and should be listed in every package's sqlc.yaml schema paths alongside the migration directories it needs for FK resolution.

Naming Convention

Table names must not repeat the schema name:

• Use core.accounts, not core.billing_accounts
• Use core.persons, not core.identity_persons

Generated Go type names carry no schema prefix — the package itself provides the namespace:

• billing.Account, not billing.CoreAccount
• identity.Person, not identity.CorePerson

This follows Go's own convention: http.Request not http.HttpRequest.

Standard sqlc.yaml Template

Every package follows this pattern:

schema:
  - "../db/sqlc_schemas.sql"   # always first — declares all schemas for cross-package resolution
  - "../db/migrations/"        # the core baseline (most packages read/write core tables)
  # add the provider's own migrations only for stripe/fedwiki packages
  # e.g. - "migrations/"       # (internal/integrations/stripe/store, internal/integrations/fedwiki/store only)
queries: "queries/"

An integration's sqlc.yaml sits two directories deeper than a typical package's — internal/integrations/<slug>/store/sqlc.yaml, with store/ nested under the integration's own tree root — so its relative paths climb two extra levels: ../../../db/migrations/ and ../../../db/sqlc_schemas.sql, alongside its own migrations/ for the provider's tables.

rename: Block Placement

The rename: block goes inside gen.go:, after emit_empty_slices and before overrides:. Keys use the {schema}_{singular_table} format — with a single domain schema, most keys are core_* regardless of which Go package owns the query:

gen:
  go:
    package: "billing"
    emit_empty_slices: true
    rename:
      core_account: Account    # core.accounts → Account
      core_price: Price
    overrides:
      - db_type: "uuid"
        go_type: "string"

Rename maps are pinned per package so that generated type names do not churn when the underlying schema is shared — e.g. internal/billing/sqlc.yaml and internal/identity/sqlc.yaml both see core.persons but only one may need to generate a Person type, depending on which package's queries touch it.