Fifth section of #23: the start-up check. Before any migration runs, on both connections, the console refuses when the two strings log in as one user or reach different databases, when the runtime user is a superuser or a member of a superuser role, when it owns or is a member of the owner of the database, when the database is owned by neither the migration user nor a superuser, when the runtime user can create objects in the database, in public or in a console schema, or when it owns an object in those schemas that a name lookup could resolve (relations, functions, operators, types, and operator classes, families and extensions database-wide). After the migrations and the boot grant, on the runtime connection, it refuses when console_runtime is missing, when the runtime user does not inherit it or holds it with SET, when the schema is behind the code, when the runtime user reaches an owner role, owns an object, holds any direct membership but console_runtime, or holds a privilege an enclosure withholds. Each refusal names the statement that fixes it, with names quoted by the database; a check query that fails refuses too. migrate up, down and status refuse a database whose owner is neither user before goose runs. The boot grant is now WITH INHERIT TRUE, SET FALSE: the runtime user needs only to inherit console_runtime, and without SET on that first membership no SET ROLE reaches a role through it. Granting again updates an existing membership in place; this worktree's runtime user held SET from the previous section and lost it at the suite's boot. Every check runs in a read-only REPEATABLE READ transaction that begins with SET LOCAL search_path = pg_catalog, pg_temp, and names every function and catalog relation with pg_catalog., so nothing the runtime user planted (a function such as public.quote_ident(name), or an operator such as public.=(oid, integer), which qualified function names alone did not exclude) runs in the migration user's session or writes a statement the console prints. TestCheckQueriesQualifyEveryName scans the query constants and fails on an unqualified function call or catalog relation. Full suite on reset databases: 4370 pass, 15 skip, none failed. The console booted on the test stack and logged "runtime database user checked" with the runtime and migration users. make lint passes. Break runs that disable an early refusal or a test's cleanup can leave a role or a membership in the shared test cluster; those were removed after the run. Deliberate breaks, scripts/mutation-check -side new, all CAUGHT (76): checks 3a to 3c run before checks 1 to 3 checks 1 to 4 run after the migrations the boot grant runs before the superuser check the database-ownership check removed (TestCheckBeforeMigrating) the database-ownership check removed (TestOpenRuntimeDatabase) check 4 tests datdba = current_user in place of MEMBER check 4 reads the first database in pg_database check 4 without its datname = current_database() filter the migration identity read outside the pinned read-only transaction, with an unqualified quote_ident the runtime identity read outside the pinned read-only transaction, with an unqualified quote_ident the migration pool opened without role=none (owner set its own role) check 12 reads has_schema_privilege in place of the ACL entries check 12's schema CREATE query removed (TestCheckBeforeMigrating) check 12's schema CREATE query removed (TestOpenRuntimeDatabase) check 12's ownership query removed (TestCheckBeforeMigrating) check 12's ownership query removed (TestOpenRuntimeDatabase) check 12 runs after the migrations the boot grant's pg_catalog.format written format check 12's queries read outside the pinned read-only transaction (TestCheckBeforeMigrating) check 12's queries read outside the pinned read-only transaction (TestOpenRuntimeDatabase) CheckRuntimeUser's transaction without the search-path pin the second query of checks 9 and 12 casts the login with regrole CheckMigrationDatabaseOwner removed from dbMigration dbMigration checks before up only CheckMigrationDatabaseOwner removed from dbRuntime the owner half refuses a superuser-owned database copy 3d prints the database name unquoted check 11 without the four-privilege read copy 9 prints the user in place of the grantee check 11's grantee filter dropped check 11's predefined-role query removed copy 3c prints the direct role unquoted copy 6 prints the direct role unquoted check 6 tests MEMBER in place of USAGE check 8 tests USAGE in place of MEMBER check 9 without functions check 9 without public check 9 tests owner = current_user in place of MEMBER the direct-membership check removed (TestCheckRuntimeUser) the direct-membership check removed (TestOpenRuntimeDatabase) the direct-membership check counts console_runtime the direct-membership check runs before check 8 check 11 removed check 3 without its membership half copy 5 quotes the name in Go (%q) an exemption when env is development a failed check query passes the scratch database is dropped after the login that owns it (test-side) an unqualified quote_ident( in the identity query an unqualified pg_roles in the owner check PendingMigrations calls to_regclass without pg_catalog. the query constants renamed so none begins query (test-side) the search-path pin removed from the pinned read-only transaction the search-path pin removed from the savepoint a test runs a check in check 5 removed check 6 does not read the SET option check 6 does not read the SET option (separate-step start) the boot grant leaves the SET option (TestGrantRuntimeRole) the boot grant leaves the SET option (TestOpenRuntimeDatabase) the database CREATE query removed (TestCheckBeforeMigrating) the database CREATE query removed (TestOpenRuntimeDatabase) the database CREATE query reads has_database_privilege in place of the ACL entries check 12's ownership query reads public only check 12's ownership query reads public only (TestOpenRuntimeDatabase) check 9 without operators check 9 without types check 9 without operator classes check 9 without operator families check 9 without extensions start's refusal log lacks the owner refusal the scan folds no case (test-side) the scan reads a quoted call as no call (test-side) the scan reads no line comment (test-side) the scan skips a concatenated constant (test-side) the scan passes a query constant without a value (test-side) copy 5a prints only the GRANT, not the REVOKE by the grantor
256 lines
9.4 KiB
Go
256 lines
9.4 KiB
Go
// SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Commercial
|
|
// SPDX-FileCopyrightText: 2025-2026 Christian Galo
|
|
|
|
package cmd
|
|
|
|
import (
|
|
"context"
|
|
"database/sql"
|
|
"errors"
|
|
"fmt"
|
|
"log/slog"
|
|
"strings"
|
|
|
|
"git.coopcloud.tech/wiki-cafe/member-console/internal/db"
|
|
"github.com/spf13/viper"
|
|
)
|
|
|
|
// dbMode says which connection openDB opens and what it runs before it
|
|
// returns.
|
|
type dbMode int
|
|
|
|
const (
|
|
// dbRuntime opens the pool the console serves from, on db-dsn.
|
|
dbRuntime dbMode = iota
|
|
// dbMigration opens the migration pool on db-migrate-dsn and nothing else.
|
|
dbMigration
|
|
// dbConfig opens db-dsn plain and leaves the schema as it is.
|
|
dbConfig
|
|
)
|
|
|
|
// openDB opens the database in the given mode. Each command uses one:
|
|
//
|
|
// - start and seed-demo open dbRuntime: they serve from, or write through,
|
|
// the runtime user. When db-migrate-dsn is set the open migrates and
|
|
// grants the runtime group role on that connection, then closes it; when
|
|
// it is not set the open runs no migration and refuses a schema that is
|
|
// behind the code.
|
|
// - migrate up, down and status open dbMigration: they run DDL, which the
|
|
// runtime user cannot, and they refuse to fall back to db-dsn.
|
|
// - config list, set, clear and validate open dbConfig: they are tools an
|
|
// operator runs by hand on the runtime user's string, read and write
|
|
// core.integration_config_overrides, and take no part in the boot.
|
|
//
|
|
// The returned closer is for the caller to defer; it logs a failed Close at
|
|
// Warn, because by then the command has nothing left to do with the error.
|
|
//
|
|
// It does not resolve the -file keys: a second resolution after
|
|
// resolveSecretFiles would report the pair as ambiguous, so each command
|
|
// resolves once before it calls openDB. It refuses a mode whose connection
|
|
// string is empty, so a command that forgot to resolve fails here and pgx
|
|
// never connects with libpq's defaults.
|
|
func openDB(ctx context.Context, logger *slog.Logger, mode dbMode) (*sql.DB, func(), error) {
|
|
var database *sql.DB
|
|
var err error
|
|
switch mode {
|
|
case dbMigration:
|
|
dsn := migrationDSN()
|
|
if dsn == "" {
|
|
return nil, nil, errors.New("member-console migrate needs db-migrate-dsn (MC_DB_MIGRATE_DSN or MC_DB_MIGRATE_DSN_FILE)")
|
|
}
|
|
database, err = openMigration(ctx, logger, dsn)
|
|
case dbConfig:
|
|
dsn, derr := runtimeDSN()
|
|
if derr != nil {
|
|
return nil, nil, derr
|
|
}
|
|
database, err = db.ConnectPlain(ctx, logger, db.DefaultDBConfig(dsn))
|
|
case dbRuntime:
|
|
dsn, derr := runtimeDSN()
|
|
if derr != nil {
|
|
return nil, nil, derr
|
|
}
|
|
database, err = openRuntimeDatabase(ctx, logger, dsn, migrationDSN())
|
|
default:
|
|
return nil, nil, fmt.Errorf("unknown database mode %d", mode)
|
|
}
|
|
if err != nil {
|
|
return nil, nil, err
|
|
}
|
|
return database, func() {
|
|
if err := database.Close(); err != nil {
|
|
logger.Warn("closing the database", slog.Any("error", err))
|
|
}
|
|
}, nil
|
|
}
|
|
|
|
// migrationDSN returns db-migrate-dsn without surrounding whitespace, as
|
|
// ValidateStart reads it: a value of only whitespace is unset, never a
|
|
// connection string for pgx to fill with libpq's defaults.
|
|
func migrationDSN() string {
|
|
return strings.TrimSpace(viper.GetString("db-migrate-dsn"))
|
|
}
|
|
|
|
// runtimeDSN returns db-dsn, or the refusal for a command that has none. A
|
|
// value of only whitespace is none.
|
|
func runtimeDSN() (string, error) {
|
|
dsn := strings.TrimSpace(viper.GetString("db-dsn"))
|
|
if dsn == "" {
|
|
return "", errors.New("db-dsn is not set (MC_DB_DSN or MC_DB_DSN_FILE)")
|
|
}
|
|
return dsn, nil
|
|
}
|
|
|
|
// logOpenFailure logs why start could not open the database: a refusal is
|
|
// "refused to start" with the sentence as its error, anything else is a
|
|
// failure to initialize.
|
|
func logOpenFailure(logger *slog.Logger, err error) {
|
|
var refusal *db.RuntimeUserRefusal
|
|
var ownerRefusal *db.DatabaseOwnerRefusal
|
|
if errors.As(err, &refusal) || errors.As(err, &ownerRefusal) {
|
|
logger.Error("refused to start", slog.Any("error", err))
|
|
return
|
|
}
|
|
logger.Error("failed to initialize database", slog.Any("error", err))
|
|
}
|
|
|
|
// openRuntimeDatabase is the boot sequence of dbRuntime (design, "The boot
|
|
// sequence"): it opens the runtime pool; with migrateDSN set it opens the
|
|
// migration pool, refuses a configuration that would reopen the bypass,
|
|
// runs every stream's migrations there, grants the runtime group role to the
|
|
// runtime user there, and closes it; then it checks the runtime user and the
|
|
// schema it meets. The runtime user runs no DDL: a pending migration is the
|
|
// migration connection's, or the operator's `migrate up`.
|
|
func openRuntimeDatabase(ctx context.Context, logger *slog.Logger, dsn, migrateDSN string) (*sql.DB, error) {
|
|
runtime, err := db.Connect(ctx, logger, db.DefaultDBConfig(dsn))
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
if migrateDSN == "" {
|
|
err = bootSeparateStep(ctx, logger, runtime)
|
|
} else {
|
|
err = bootWithMigration(ctx, logger, runtime, migrateDSN)
|
|
}
|
|
if err != nil {
|
|
return nil, errors.Join(err, runtime.Close())
|
|
}
|
|
return runtime, nil
|
|
}
|
|
|
|
// bootSeparateStep is the boot with db-migrate-dsn left out: the checks that
|
|
// need no schema run on the runtime connection, no migration runs, and the
|
|
// runtime user is checked against the schema as it is.
|
|
func bootSeparateStep(ctx context.Context, logger *slog.Logger, runtime *sql.DB) error {
|
|
check := db.NewRuntimeCheck(migrationSources())
|
|
id, err := db.ReadRuntimeIdentity(ctx, runtime)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
if err := db.CheckIdentities(id, nil); err != nil {
|
|
return err
|
|
}
|
|
if err := db.CheckBeforeMigrating(ctx, runtime); err != nil {
|
|
return err
|
|
}
|
|
if err := db.CheckSchemasBeforeMigrating(ctx, runtime, id.User, check.CreateSchemas()); err != nil {
|
|
return err
|
|
}
|
|
logger.Info("migrations not run: db-migrate-dsn is not set")
|
|
if err := db.CheckRuntimeUser(ctx, runtime, check); err != nil {
|
|
return err
|
|
}
|
|
logger.Info("runtime database user checked", slog.String("db_user", id.User))
|
|
return nil
|
|
}
|
|
|
|
// bootWithMigration is the boot with db-migrate-dsn set. It closes the
|
|
// migration pool before it checks the runtime user, so a check meets the
|
|
// schema as the runtime connection will serve it.
|
|
func bootWithMigration(ctx context.Context, logger *slog.Logger, runtime *sql.DB, migrateDSN string) error {
|
|
migration, err := db.ConnectMigration(ctx, logger, db.DefaultDBConfig(migrateDSN))
|
|
if err != nil {
|
|
return err
|
|
}
|
|
id, migrationUser, err := migrateAndGrant(ctx, runtime, migration)
|
|
if err != nil {
|
|
return errors.Join(err, migration.Close())
|
|
}
|
|
if err := migration.Close(); err != nil {
|
|
logger.Warn("closing the migration connection", slog.Any("error", err))
|
|
}
|
|
logger.Info("database migrations applied")
|
|
if err := db.CheckRuntimeUser(ctx, runtime, db.NewRuntimeCheck(migrationSources())); err != nil {
|
|
return err
|
|
}
|
|
logger.Info("runtime database user checked",
|
|
slog.String("db_user", id.User), slog.String("migration_user", migrationUser))
|
|
return nil
|
|
}
|
|
|
|
// migrateAndGrant refuses the configurations that must not migrate (checks 1
|
|
// to 4 and 12), runs every stream's migrations on migration, and grants the
|
|
// runtime group role to the user runtime logs in as. It returns the runtime
|
|
// identity and the migration user's name.
|
|
func migrateAndGrant(ctx context.Context, runtime, migration *sql.DB) (db.RuntimeIdentity, string, error) {
|
|
id, migrationUser, err := refuseBeforeMigrating(ctx, runtime, migration)
|
|
if err != nil {
|
|
return id, "", err
|
|
}
|
|
if err := db.RunMigrations(migration, migrationSources()); err != nil {
|
|
return id, "", fmt.Errorf("failed to run migrations: %w", err)
|
|
}
|
|
err = db.InTx(ctx, migration, db.Plain, func(tx *sql.Tx) error {
|
|
return db.GrantRuntimeRole(ctx, tx, id.User)
|
|
})
|
|
if err != nil {
|
|
return id, "", &db.RuntimeUserRefusal{Message: fmt.Sprintf(
|
|
"could not grant %s to %s on the db-migrate-dsn connection: %v", db.RuntimeGroupRole, id.User, err)}
|
|
}
|
|
return id, migrationUser, nil
|
|
}
|
|
|
|
// refuseBeforeMigrating runs the checks that need no schema, in the order of
|
|
// the design: both identities, checks 1 to 3, the membership and ownership
|
|
// of the runtime user (3a to 3c), the database's owner on the migration
|
|
// connection (3d), and what the runtime user can create or owns where the
|
|
// migrations resolve names (3e, 3f, 7, 8).
|
|
func refuseBeforeMigrating(ctx context.Context, runtime, migration *sql.DB) (db.RuntimeIdentity, string, error) {
|
|
id, err := db.ReadRuntimeIdentity(ctx, runtime)
|
|
if err != nil {
|
|
return id, "", err
|
|
}
|
|
mid, err := db.ReadMigrationIdentity(ctx, migration)
|
|
if err != nil {
|
|
return id, "", err
|
|
}
|
|
if err := db.CheckIdentities(id, &mid); err != nil {
|
|
return id, "", err
|
|
}
|
|
if err := db.CheckBeforeMigrating(ctx, runtime); err != nil {
|
|
return id, "", err
|
|
}
|
|
if err := db.CheckMigrationDatabaseOwner(ctx, migration); err != nil {
|
|
return id, "", err
|
|
}
|
|
check := db.NewRuntimeCheck(migrationSources())
|
|
if err := db.CheckSchemasBeforeMigrating(ctx, migration, id.User, check.CreateSchemas()); err != nil {
|
|
return id, "", err
|
|
}
|
|
return id, mid.User, nil
|
|
}
|
|
|
|
// openMigration opens the migration pool and refuses a database that is owned
|
|
// by neither the migration user nor a superuser role before any migrate
|
|
// command runs goose, because goose's Status creates a missing ledger in
|
|
// public (check 4, owner half).
|
|
func openMigration(ctx context.Context, logger *slog.Logger, dsn string) (*sql.DB, error) {
|
|
database, err := db.ConnectMigration(ctx, logger, db.DefaultDBConfig(dsn))
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
if err := db.CheckMigrationDatabaseOwner(ctx, database); err != nil {
|
|
return nil, errors.Join(err, database.Close())
|
|
}
|
|
return database, nil
|
|
}
|