member-console/openspec/changes/runtime-database-user/proposal.md at 2d71aa8db09f2fb206ced884ff99ce250fc59b6e

Files

T

cgalo5758 028e2ef448 Grant every writer to one runtime role, and leave writers only the four writes

First section of #23. The console still connects as before; this section
puts in place the role its runtime user will hold.

Each stream's new migration grants its writer, and SELECT on its goose
ledger, to console_runtime, a NOLOGIN role core creates, and revokes the
writer from member_console where an earlier release granted it there.
The five baselines no longer grant a writer to any login, so a fresh
install needs no role of a particular name.

The baselines granted ALL on their tables to the writers, which gave the
runtime path TRUNCATE, REFERENCES, TRIGGER and MAINTAIN as well. With
TRIGGER, a user holding the writer could put a trigger on an ordinary
table; a SECURITY DEFINER function's write to that table then ran the
trigger as the function's owner, which wrote the enclosed tables
directly. Each stream's writer_dml_only migration revokes the four, and
TestRuntimeRolePrivileges fails, naming the relation and the privilege,
when console_runtime holds any of them, holds a column grant that reopens
a write, or is denied a write it needs.

The enclosure tests now drop to console_runtime instead of core_writer
or a copied role, and the conferral enclosure test probes TRUNCATE,
CREATE TRIGGER and LOCK TABLE on the three position tables.

Full suite on reset databases: 4195 pass, 15 skip; one failure,
TestIntegrationSettingsWalkthrough's missing toast (#222), passed when
run alone. make lint passes.

Deliberate breaks, scripts/mutation-check -side new, all CAUGHT (39):
  core 00029 without GRANT core_writer TO console_runtime
  fedwiki 00005 grants domains_writer in place of fedwiki_writer
  stripe 00006 without its revoke from member_console
  discourse 00004 without its ledger grant (two tests)
  domains 00003 without GRANT domains_writer TO console_runtime
  domains 00003's revoke without its guard
  core and discourse baselines with the login grant restored
  core 00029's Down revokes core_writer from console_runtime
  domains 00003's Down revokes domains_writer from console_runtime
  core 00029's Down drops the role without its dependency handler
  core 00029's Down grants core_writer back to member_console
  core 00030 emptied (catalog test, three probes)
  core 00030 leaves TRIGGER (catalog test, probe)
  core 00030 revokes from two position tables only
  the domains, fedwiki, stripe and discourse writer_dml_only emptied
  GRANT ALL on integration_config_overrides appended after the revoke
  00009's grant loses core_writer
  00027's SELECT on acts becomes SELECT, INSERT (two tests)
  00005's revoke loses INSERT; 00024's SELECT becomes SELECT, INSERT
  core 00029 also grants INSERT, or TRIGGER, on pool_provisions (four)
  column UPDATE and REFERENCES grants on pool_provisions, column
  UPDATE on the outbox's subject_id
  test side: the membership assertion without its fedwiki_writer
  exception; a locked entry for a table that does not exist
The two converted checks were not run on the old side.

2026-10-23 05:56:52 -04:00

21 KiB

Raw Blame History

Why

The migrations enclose several tables behind SECURITY DEFINER functions by revoking direct writes from core_writer: the three position tables (core.pool_provisions, core.pool_provision_ladders, core.pool_provision_transitions), core.entitlement_set_rules, the rule-change ledger, the act record, core.person_cut_offs and the insert enclosure of core.desired_state_outbox. None of these revokes binds the console itself, for four reasons.

The console connects as the bootstrap superuser. The official postgres image creates POSTGRES_USER as the cluster's bootstrap superuser, and the test stack (test/compose.yaml, test/bootstrap-stack.sh) and the documented deployment both connect as that user. A superuser passes every privilege check except the right to log in, so the revokes never apply to it. Postgres refuses to remove SUPERUSER from the bootstrap user, so the fix is a second login, not an ALTER ROLE.
The same login owns the objects. It owns every schema, table, view, sequence and goose ledger the migrations create; only the definer functions belong to core_owner. An owner holds every privilege on its own tables, so a login that stopped being a superuser but still owned the tables would still bypass the revokes. The login the console runs as must own nothing and must not be able to act as an owner.
The runtime path runs DDL. db.EnsureWebhookEventPartitions (internal/db/partitions.go) creates the monthly partitions of core.webhook_events at boot (cmd/bootstate.go) and from the Temporal maintenance schedule (internal/workflows/maintenance/activities.go). Creating a partition needs CREATE on schema core and ownership of the parent table. start also runs every migration at boot on the connection string it then serves from (db.ConnectAndMigrate, internal/db/database.go).
The writers hold more than the four DML privileges. GRANT ALL ON ALL TABLES ... TO {key}_writer also gives TRUNCATE, REFERENCES, TRIGGER and MAINTAIN. With TRIGGER on a table, a role can attach a trigger to it; Postgres runs a trigger with the privileges of the user whose statement fired it, so a SECURITY DEFINER function owned by core_owner that writes that table runs the trigger as core_owner, which writes the locked tables directly. TRUNCATE and MAINTAIN also empty or lock a table.
One runtime query takes a row lock on an enclosed table. LockPlanActObligation (internal/entitlements/queries/plan_acts.sql) runs SELECT ... FROM core.plan_act_obligations ... FOR UPDATE, and Postgres requires UPDATE on at least one column of a table for any locking read. The writer holds SELECT only on that table, because the obligations are enclosed rather than given column UPDATE grants, so the query works today only because the login is a superuser, and every plan-act obligation move would fail with a privilege error once the console serves as the runtime user.

A non-superuser login that owns nothing would therefore fail its first partition, its first plan-act obligation move and its first migration, and it would hold no privileges at all, because the five migration streams grant their writer roles to the login by the name member_console (GRANT core_writer TO member_console in core's 00001_init.sql, and the same in the domains, FedWiki, Stripe and Discourse baselines), and the group of privileges it would get through them is wider than the console uses.

What Changes

Two Postgres users and two connection strings. db-dsn (MC_DB_DSN, db-dsn-file, MC_DB_DSN_FILE) names the runtime user, the user the console serves from. A new db-migrate-dsn (MC_DB_MIGRATE_DSN, db-migrate-dsn-file, MC_DB_MIGRATE_DSN_FILE) names the migration user and feeds only the migration connection. start migrates on the migration connection, grants the runtime group role to the runtime user there, closes it, and then runs only on the runtime connection. migrate up|down|status use db-migrate-dsn and refuse to run without it.
A separate-step mode. With db-migrate-dsn left out, start runs no migrations and refuses to start while the schema is behind its code, naming the command that brings it up to date. A host that wants the running process never to hold the migration password runs member-console migrate up as its own step before each deploy.
A runtime group role. A new core migration creates the NOLOGIN role console_runtime. Each migration stream grants its own {key}_writer role, and SELECT on its own goose ledger, to console_runtime in a new migration (core, domains, FedWiki, Stripe, Discourse). The five baselines stop granting their writer to member_console (design D6), and the same migrations revoke each writer from member_console on a database that already holds the membership, when that role exists. docs/building-an-integration.md teaches GRANT {key}_writer TO console_runtime;.
Writers hold only the four DML privileges. A second new migration in each stream (writer_dml_only) revokes TRUNCATE, REFERENCES, TRIGGER and MAINTAIN on all tables of the stream's schema from its writer (design D13). It runs after every earlier grant, so it corrects fresh and existing databases alike, and the GRANT ALL lines of the earlier migrations are not edited.
The boot grant. At every start, on the migration connection, the console grants console_runtime to the user the runtime connection logs in as. It learns that name from the runtime connection's current_user and builds the statement in the database with format('%I'), so any login name works and no deployment needs a hand step per release or per integration.
A start-up check, always on. The console refuses to start when the runtime user is a superuser or a member of a superuser role, is the same user as the migration user, connects to another database than the migration user, is not a member of console_runtime, owns the database or is a member of the role that does, is a member of a console owner role (core_owner, domains_owner, fedwiki_owner, stripe_owner, discourse_owner), owns (or is a member of a role that owns) any schema, relation or function in the console's schemas, is a direct member of any role other than console_runtime, or holds INSERT on core.pool_provisions, or TRUNCATE, REFERENCES, TRIGGER or MAINTAIN on any relation of the console's schemas, or can create objects in public or in a console schema. The checks that do not depend on the schema run before any migration, so a wrong db-dsn never migrates the database, and a runtime user that owns the database, or can create objects where the migration run resolves names, is refused before the migration run that it could affect. SQL that the console runs on the migration connection, or prints as a statement for the operator to run, names every function and catalog relation with its schema (pg_catalog.<name>), so an object the runtime user planted never runs in a privileged session, and a structural test fails when a check query names one without it. Before the database-ownership check passes, the migration connection runs only a query with qualified catalog names. Every connection opens with startup parameters that override database-level and role-level settings (search_path, and role=none so that current_user is the login), and every migrate command and start refuse a database that is owned by neither the migration user nor a superuser role, naming the statement that fixes it. There is no exemption, no opt-out setting, and nothing keyed on env. Each refusal names what to fix.
Ownership stays where it is, except for the partitioned table. The migration user keeps owning what it owns today. core.webhook_events and its partitions move to core_owner, and partitions are created by a new function, core.ensure_webhook_event_partitions, SECURITY DEFINER, owned by core_owner, with a pinned search_path, EXECUTE revoked from PUBLIC and granted to console_runtime. The boot pass and the Temporal schedule both call it. The per-schema-roles spec is corrected to say who owns what. The core baseline's initial-partition block computes its month bounds in UTC, as the function does.
The obligation lock goes through a function owned by core_owner. A new core function, core.lock_plan_act_obligation, SECURITY DEFINER, owned by core_owner, with a pinned search_path, EXECUTE revoked from PUBLIC and granted to core_writer, takes the row lock LockPlanActObligation takes today and returns the same columns. The sqlc query becomes a call to it, the lock order in docs/database-locks.md is unchanged, and core_writer and console_runtime still hold no UPDATE on the obligations. It lands in the same section as the partition function, before the section that switches the application and the test stack to the runtime user.
The test stack runs on two users. A one-shot compose service, postgres-seed, creates the runtime login member_console_runtime in new and existing volumes. test/bootstrap-stack.sh writes MC_DB_DSN as the runtime user and MC_DB_MIGRATE_DSN as member_console, today's user. The stack scripts refuse a test/.env that lacks MC_DB_MIGRATE_DSN and print the command that regenerates it. The work that needs a superuser on the application database (test/reset-app-db.sh, the screens' timestamp pinning, test/mockshot-*.sql) uses the migration user. TEST_DATABASE_URL stays the superuser.
The tests that prove the locks run as console_runtime. The enclosure tests drop to console_runtime with SET LOCAL ROLE, and the applicationRole fixture, which copied the login's memberships, goes.
A catalog test of the runtime role. As console_runtime, every table and view in the console's schemas is readable, every table outside an explicit list of locked tables is writable, each locked table refuses the privileges its enclosure revokes, and no table, view or partition grants console_runtime TRUNCATE, REFERENCES, TRIGGER or MAINTAIN, so a later GRANT ALL fails make test. Grants stay explicit in each migration; no ALTER DEFAULT PRIVILEGES is added.
Docs. docs/production-deployment.md describes the two users, the separate-step mode, and a procedure that moves an existing deployment onto two users and back. docs/environment-reference.md, docs/hosting.md, docs/database-management.md, docs/building-an-integration.md, docs/testing.md, test/AGENTS.md and test/README.md follow.
BREAKING (deployers): a deployment that upgrades without a runtime user and db-migrate-dsn does not start: its db-dsn names a superuser, and the start-up check refuses it. The procedure in docs/production-deployment.md is the upgrade path.

Quick fixes inside this change

Listed here because each is small and was found while investigating #23.

Writers hold only SELECT, INSERT, UPDATE and DELETE. Migration 00005 revoked INSERT, UPDATE and DELETE from core_writer on the three position tables and left what GRANT ALL gave it: TRUNCATE, REFERENCES, TRIGGER and MAINTAIN. Postgres lets a role holding TRIGGER on a table create a trigger there, and runs a trigger with the privileges of the user whose statement fired it, so a trigger on a table that a SECURITY DEFINER function writes runs as the function's owner; TRUNCATE empties a table, and TRUNCATE or MAINTAIN allows LOCK TABLE ... ACCESS EXCLUSIVE. A new migration in each stream revokes the four, TestConferralEnclosure_WriterRole probes TRUNCATE, CREATE TRIGGER and LOCK TABLE on the position tables, and the catalog test fails for any table that grants console_runtime one of the four. It lands with the group role.
The core baseline's initial partitions are computed in UTC. The block in 00001_init.sql took the first of the month in the session's time zone, while the Go code and the new function use UTC; the block now agrees with them. Databases that applied the baseline are untouched, and on a UTC server the result is the same as before.
The plan-act obligation move takes its row lock through a function. LockPlanActObligation is a code path that works only because the console's login is a superuser: a locking read of core.plan_act_obligations needs UPDATE, which the enclosure withholds from the writer on purpose. A new migration, core 00032_lock_plan_act_obligation.sql, adds core.lock_plan_act_obligation(act_id, pool_id), a SECURITY DEFINER function owned by core_owner, SET search_path = core, pg_temp, EXECUTE revoked from PUBLIC and granted to core_writer as the other definer functions of 00028 are, which runs the same single-table SELECT ... FOR UPDATE and returns the same seven columns; the sqlc query becomes SELECT ... FROM core.lock_plan_act_obligation(...). It is not its own change because it is the defect class this change removes (a path that a superuser login hides), because the partition function handles the other such path the same way, and because it must land before the switch to the runtime user for the switch to work. A test runs the move's lock under SET LOCAL ROLE console_runtime, and the bare SELECT ... FOR UPDATE on the table under the same role is refused with 42501, which keeps the enclosure honest. No grant is added to the table.
Every database command reads the -file keys. Only start and config validate load db-dsn-file today; migrate, config list, config set, config clear and seed-demo ignore it, so a container that mounts the connection string as a file cannot run them. Each command now resolves the database's -file keys the same way, and a command that finds no connection string refuses instead of letting the driver fall back to its defaults. It lands as its own commit with its own test.

Capabilities

New Capabilities

None.

Modified Capabilities

per-schema-roles: modified "Owner role grants full DDL and data access" (who owns what), "Writer role grants read-write data access" (the writer holds the four DML privileges and no other; the writer reaches the application through console_runtime), "Application login role wiring is explicit" (the runtime group role in place of the login's name) and "Entitlements position-machinery enclosure" (TRUNCATE, TRIGGER and MAINTAIN revoked; the tables' real names). Added "The runtime group role's privileges are checked against the catalog" and "A locking read of an enclosed table goes through a function owned by core_owner".
module-migrations: modified "Baseline migration creates core schema and roles" (the baselines grant no login; a later migration revokes an earlier release's by-name grant).
webhook-partition-maintenance: modified "Webhook event partitions are provisioned ahead of need" (the function owned by core_owner in place of a CREATE grant to the runtime role; the baseline's initial partitions in UTC).
startup-configuration: modified "Boot-time configuration validation" (db-migrate-dsn) and "Connection strings and secrets never reach the logs" (db-migrate-dsn-file; every database-opening command reads the -file keys). Added "Start migrates on the migration connection and serves from the runtime connection" (both connections open with search_path and role=none; the migration connection reads only qualified names before the ownership check; a database owned by neither the migration user nor a superuser is refused by every migrate command and start) and "The console refuses a runtime user that can bypass the grants" (the four privileges, as well as INSERT on core.pool_provisions; CREATE on public or a console schema, before migrating).
postgres-database: modified "Database migrations via goose" (migrations at startup only on the migration connection).
worktree-test-stacks: modified "Bootstrap script generates a per-worktree .env" and "Application reads MC_-prefixed env vars to override mc-config.yaml". Added "The stack's Postgres carries a runtime login", "Stack scripts refuse an env file without the migration connection string" and "Superuser work on the application database uses the migration connection string".
test-database-isolation: added "Tests of the grants run as the runtime group role".
deployment-docs: modified "A production deployment guide exists under docs/" (two users, the runtime login owning neither the database nor any role but console_runtime, the separate-step mode, the procedure for an existing deployment and its verification of the database's settings and of public).
discourse-integration-infrastructure: modified "Conformant integration registration" (discourse_writer membership for console_runtime; the baseline grants no login).

Impact

Schema: twelve migrations: core 00029_runtime_group_role.sql, domains 00003_runtime_group_role.sql, FedWiki 00005_runtime_group_role.sql, Stripe 00006_runtime_group_role.sql and Discourse 00004_runtime_group_role.sql (the group role and its grants); core 00030_writer_dml_only.sql, domains 00004_writer_dml_only.sql, FedWiki 00006_writer_dml_only.sql, Stripe 00007_writer_dml_only.sql and Discourse 00005_writer_dml_only.sql (the privilege revoke); core 00031_webhook_partition_function.sql (the ownership move and the function); core 00032_lock_plan_act_obligation.sql (the obligation lock's function). The five baselines lose their GRANT {key}_writer TO member_console statement and its Down REVOKE, and the core baseline's initial-partition block is computed in UTC. No table or column is added.
Roles: console_runtime (NOLOGIN) is created, cluster-wide, guarded like the stream triples. The five writer roles become its members, lose TRUNCATE, REFERENCES, TRIGGER and MAINTAIN, and stop being members of member_console on every database that had the membership. core_owner comes to own core.webhook_events, its partitions and the two new functions. No login is created by a migration.
Config keys: db-migrate-dsn and db-migrate-dsn-file (MC_DB_MIGRATE_DSN, MC_DB_MIGRATE_DSN_FILE) are new; db-dsn now names the runtime user. env is unchanged and still picks only the log format.
Code: cmd/database.go, cmd/start.go, cmd/migrate.go, cmd/config.go, cmd/seed_demo.go, internal/config/validate.go, internal/db/database.go (ConnectAndMigrate is deleted), internal/db/migrations.go (the pending check), internal/db/runtime_user.go (new: the boot grant and the start-up check, with every name pg_catalog-qualified), internal/db/partitions.go, internal/entitlements/queries/plan_acts.sql and its generated plan_acts.sql.go, internal/embeds/mc-config.yaml.
Tests: new tests of the group role's grants, the catalog, the partition function, the obligation lock run as the runtime role, the pending check, the boot grant, the start-up check, a scan of the check and boot-grant queries for an unqualified function or catalog relation, the -file resolution and the stack scripts' guard. The enclosure tests move from core_writer or applicationRole to console_runtime; applicationRole is deleted. No golden file and no screen changes.
Test stack: test/compose.yaml (postgres-seed, a health check on postgres), test/seed/postgres/seed-runtime-user.sh (new), test/bootstrap-stack.sh, test/reset-test-db.sh, test/reset-app-db.sh, test/seed-demo.sh, test/mc-config.yaml, test/mockshot-nuke.sql (a false comment corrected), test/e2e/browsertest/session.go, test/e2e/screens/screens_test.go, test/seed/discourse/seed-discourse.sh (its header), and the script tests in test/. Every existing worktree regenerates its test/.env once.
Reader role: scripts/db-reader-role.sql does not change, and internal/migrate/testdata/console-reader-columns.txt does not change: no column is added. The script still runs as the migration user, which owns the schemas. A deployment re-runs it after deploying, as after every migration, though nothing this change adds is a column the role reads. The new functions revoke EXECUTE from PUBLIC, which the script requires.
Docs: docs/database-locks.md (the obligation row's lock is taken by the new function), docs/production-deployment.md, docs/environment-reference.md, docs/hosting.md, docs/database-management.md, docs/building-an-integration.md, docs/testing.md, test/AGENTS.md, test/README.md.
Deployments: each deployment creates a runtime login and adds a second secret before it deploys this release; docs/production-deployment.md gives the procedure and its roll back. The procedure is run by the host's operator, not by this change.

Out of scope

Renaming env (#251). It keeps picking the log format, and nothing in this change reads it.
Moving every object to its schema's owner role (#252). This change moves core.webhook_events and its partitions only; design D3 says what it leaves for that issue.
Switching any running deployment. The procedure is documented and handed to the deployment's operator.
A migration user that is not a superuser, as on managed Postgres. The migrations already need a superuser, or an owner with CREATE on the database, CREATEROLE and SET membership in every owner role; this change adds the ownership move of core.webhook_events, which needs the same SET membership in core_owner.

21 KiB Raw Blame History

Why