Fourth section of #23. db-dsn now names the console's runtime user, and the new db-migrate-dsn (or db-migrate-dsn-file) names the user that runs migrations. start opens both, migrates on the migration connection, grants console_runtime to the runtime user there (WITH INHERIT TRUE, the name quoted by the database), closes it, and serves on the runtime pool alone. With db-migrate-dsn left out, start runs no migration and refuses when the schema is behind its code, which is how a host runs migrations as a separate step; migrate up, down and status use the migration string and refuse without it, with no fallback to db-dsn. Both pools are opened from one pgx.ParseConfig: the migration pool's search_path is public and both pools' role is none, set in the startup parameters after every key of either name is removed case-insensitively, so neither a setting the database or role carries nor a key the string repeats in another case moves a session off its login or its path. PendingMigrations reads each stream's ledger by qualified name inside a transaction pinned to search_path pg_catalog, pg_temp. The test stack runs on two users: postgres-seed creates member_console_runtime in new and existing volumes, bootstrap-stack.sh writes MC_DB_DSN as that user and MC_DB_MIGRATE_DSN as member_console, and the four scripts refuse a .env without the migration string, with the command that regenerates it. Scripts and fixtures that need a superuser on the application database (the reset, the screens' timestamp pinning) use the migration string. This worktree's stack was regenerated with that command and the console booted as member_console_runtime: its sessions were the runtime user's only, no migration-user session stayed open, and the runtime user could not insert into core.pool_provisions. Full suite on reset databases: 4277 pass, 15 skip, none failed. The walkthroughs' names, results and skip messages match the previous section's run. make screens lists three changed screens, all data: regenerated ids on operator-person-detail and operator-billing-invoice-detail (the demo snapshot was rebuilt) and a domain's age on operator-domains (294d to 295d); the baseline was not accepted. make lint passes. Deliberate breaks, scripts/mutation-check -side new, all CAUGHT (36): dbRuntime runs the migrations on the runtime pool dbRuntime skips the boot grant dbRuntime leaves the migration pool open dbRuntime skips the pending check when db-migrate-dsn is unset dbRuntime migrates on db-dsn when db-migrate-dsn is unset GrantRuntimeRole without WITH INHERIT TRUE GrantRuntimeRole with %s in place of %I PendingMigrations takes a missing ledger as current PendingMigrations resolves the ledger without the public. qualifier the migration pool opened without search_path=public the migration pool opened without role=none the runtime pool opened without role=none ConnectMigration keeps the search_path and role the string carries dbMigration falls back to db-dsn ValidateStart accepts a malformed db-migrate-dsn db-migrate-dsn-file is not a database secret pair db-migrate-dsn-file is not a database secret pair (the migrate commands read db-dsn-file only) the bootstrap guard removed reset-test-db.sh guard removed seed-demo.sh guard removed reset-app-db.sh guard after its first psql startup parameters match their keys case-sensitively the migration pool opened without role=none (keys and options in the string) the runtime pool opened without role=none (keys and options in the string) the runtime pool replaces the search_path the string carries the string's spaces are written as plus signs ValidateStart reads db-migrate-dsn as set whatever it holds PendingMigrations lists only the versions the ledger shows when it is missing migrationDSN keeps whitespace migrate up accepts a whitespace-only db-migrate-dsn runtimeDSN accepts a whitespace-only db-dsn openDB runs the migrating boot for an unknown mode bootstrap-stack.sh trusts the exported MC_DB_MIGRATE_DSN reset-test-db.sh trusts the exported MC_DB_MIGRATE_DSN reset-app-db.sh trusts the exported MC_DB_MIGRATE_DSN seed-demo.sh trusts the exported MC_DB_MIGRATE_DSN
219 lines
7.9 KiB
Go
219 lines
7.9 KiB
Go
// SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Commercial
|
|
// SPDX-FileCopyrightText: 2025-2026 Christian Galo
|
|
|
|
package db
|
|
|
|
import (
|
|
"context"
|
|
"database/sql"
|
|
"errors"
|
|
"fmt"
|
|
"log/slog"
|
|
"net/url"
|
|
"slices"
|
|
"strings"
|
|
"time"
|
|
|
|
"github.com/jackc/pgx/v5"
|
|
"github.com/jackc/pgx/v5/stdlib"
|
|
)
|
|
|
|
// DBConfig holds database configuration.
|
|
type DBConfig struct {
|
|
DSN string // Data Source Name (PostgreSQL connection string)
|
|
MaxOpenConns int // Maximum number of open connections
|
|
MaxIdleConns int // Maximum number of idle connections
|
|
ConnMaxLifetime time.Duration // Maximum lifetime of connections
|
|
ConnMaxIdleTime time.Duration // Maximum idle time for connections
|
|
}
|
|
|
|
// DefaultDBConfig returns a DBConfig with sensible defaults for PostgreSQL.
|
|
func DefaultDBConfig(dsn string) *DBConfig {
|
|
return &DBConfig{
|
|
DSN: dsn,
|
|
MaxOpenConns: 25,
|
|
MaxIdleConns: 10,
|
|
ConnMaxLifetime: 30 * time.Minute,
|
|
ConnMaxIdleTime: 5 * time.Minute,
|
|
}
|
|
}
|
|
|
|
// startupParam is one connection parameter that the pool sets in the
|
|
// connection's startup message.
|
|
type startupParam struct{ key, value string }
|
|
|
|
// The parameters the pools open with. A startup parameter outranks
|
|
// ALTER DATABASE ... SET and ALTER ROLE ... SET, so a setting that a database
|
|
// or a role carries cannot change where names resolve or who current_user is.
|
|
// It also outranks a -c switch in the string's options, which Postgres applies
|
|
// before the startup parameters.
|
|
var (
|
|
// roleNone keeps current_user the login, whatever role a database or a
|
|
// role setting names.
|
|
roleNone = startupParam{"role", "none"}
|
|
// migrationSearchPath is where goose's ledgers live: the schema every
|
|
// migration connection resolves unqualified names in.
|
|
migrationSearchPath = startupParam{"search_path", "public"}
|
|
// moduleSearchPath is where the runtime pool resolves names (Decision
|
|
// 113). Provider schemas (stripe, fedwiki) stay out of it and their
|
|
// queries stay schema-qualified.
|
|
moduleSearchPath = startupParam{"search_path", "core,public"}
|
|
)
|
|
|
|
// deleteParam removes every runtime parameter whose key matches key without
|
|
// regard to case, as Postgres matches parameter names, and returns the value
|
|
// of the last one in key order (empty when there is none).
|
|
func deleteParam(params map[string]string, key string) string {
|
|
var matches []string
|
|
for k := range params {
|
|
if strings.EqualFold(k, key) {
|
|
matches = append(matches, k)
|
|
}
|
|
}
|
|
slices.Sort(matches)
|
|
var value string
|
|
for _, k := range matches {
|
|
value = params[k]
|
|
delete(params, k)
|
|
}
|
|
return value
|
|
}
|
|
|
|
// setParams replaces each parameter's value, whatever case its key has in the
|
|
// string.
|
|
func setParams(params map[string]string, set ...startupParam) {
|
|
for _, p := range set {
|
|
deleteParam(params, p.key)
|
|
params[p.key] = p.value
|
|
}
|
|
}
|
|
|
|
// keepSearchPath keeps the search_path the string carries, and sets
|
|
// core,public when it carries none or an empty one.
|
|
func keepSearchPath(params map[string]string) {
|
|
if value := deleteParam(params, moduleSearchPath.key); value != "" {
|
|
params[moduleSearchPath.key] = value
|
|
return
|
|
}
|
|
setParams(params, moduleSearchPath)
|
|
}
|
|
|
|
// openPool parses config.DSN once with the parser that connects, lets shape
|
|
// set the startup parameters on the parsed parameters, and opens the pool with
|
|
// the settings of config. The string is never edited as text, so no spelling
|
|
// of a key, quoting or fragment can survive beside the values shape sets.
|
|
func openPool(config *DBConfig, shape func(params map[string]string)) (*sql.DB, error) {
|
|
cfg, err := pgx.ParseConfig(config.DSN)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("failed to open database: %w", err)
|
|
}
|
|
shape(cfg.RuntimeParams)
|
|
return configurePool(stdlib.OpenDB(*cfg), config), nil
|
|
}
|
|
|
|
// configurePool applies the pool settings of config.
|
|
func configurePool(db *sql.DB, config *DBConfig) *sql.DB {
|
|
db.SetMaxOpenConns(config.MaxOpenConns)
|
|
db.SetMaxIdleConns(config.MaxIdleConns)
|
|
db.SetConnMaxLifetime(config.ConnMaxLifetime)
|
|
db.SetConnMaxIdleTime(config.ConnMaxIdleTime)
|
|
return db
|
|
}
|
|
|
|
// dsnLogAttrs parses dsn as a PostgreSQL connection URL and returns only the
|
|
// parts an operator needs to recognise the connection target: host, port,
|
|
// database and user (design D1, security-audit-remediation-2 — the DSN is a
|
|
// secret, not a string to redact at each call site). It never returns the
|
|
// password or the query string. A DSN that fails to parse yields a single
|
|
// attribute naming the failure, never the input, since url.Error.Error()
|
|
// echoes the string it failed to parse.
|
|
func dsnLogAttrs(dsn string) []slog.Attr {
|
|
u, err := url.Parse(dsn)
|
|
if err != nil || u.Scheme == "" || u.Host == "" {
|
|
return []slog.Attr{slog.String("dsn_parse_error", "unparseable")}
|
|
}
|
|
database := strings.TrimPrefix(u.Path, "/")
|
|
var user string
|
|
if u.User != nil {
|
|
user = u.User.Username()
|
|
}
|
|
return []slog.Attr{
|
|
slog.String("db_host", u.Hostname()),
|
|
slog.String("db_port", u.Port()),
|
|
slog.String("db_name", database),
|
|
slog.String("db_user", user),
|
|
}
|
|
}
|
|
|
|
// openAndConfigureDB opens the runtime pool: the application's search_path
|
|
// (kept when the string carries one) and role=none.
|
|
func openAndConfigureDB(config *DBConfig) (*sql.DB, error) {
|
|
return openPool(config, func(params map[string]string) {
|
|
keepSearchPath(params)
|
|
setParams(params, roleNone)
|
|
})
|
|
}
|
|
|
|
// ConnectPlain opens a database connection without the custom search_path
|
|
// and without the startup parameters of the other two pools. The config
|
|
// commands use it on db-dsn: they are tools an operator runs by hand, not the
|
|
// serving process or the migration run.
|
|
func ConnectPlain(ctx context.Context, logger *slog.Logger, config *DBConfig) (*sql.DB, error) {
|
|
db, err := sql.Open("pgx", config.DSN)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("failed to open database: %w", err)
|
|
}
|
|
configurePool(db, config)
|
|
if err := db.PingContext(ctx); err != nil {
|
|
return nil, errors.Join(fmt.Errorf("failed to connect to database: %w", err), db.Close())
|
|
}
|
|
logger.LogAttrs(ctx, slog.LevelInfo, "database connection established (plain)",
|
|
dsnLogAttrs(config.DSN)...)
|
|
return db, nil
|
|
}
|
|
|
|
// ConnectMigration opens the migration connection: the pool the migrations,
|
|
// the boot grant and the migrate commands run on. It opens with the startup
|
|
// parameters search_path=public and role=none, replacing any search_path or
|
|
// role the string carries, whatever case its keys have. The runtime pool's
|
|
// rule, which keeps an operator's search_path, is not reused: a
|
|
// db-migrate-dsn copied from the runtime string would otherwise open with
|
|
// core,public, and a database-level or role-level search_path or role setting
|
|
// would decide where goose's ledgers land and who runs the DDL (design D2,
|
|
// rule 2).
|
|
func ConnectMigration(ctx context.Context, logger *slog.Logger, config *DBConfig) (*sql.DB, error) {
|
|
db, err := openPool(config, func(params map[string]string) {
|
|
setParams(params, migrationSearchPath, roleNone)
|
|
})
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
if err := db.PingContext(ctx); err != nil {
|
|
return nil, errors.Join(fmt.Errorf("failed to connect to database: %w", err), db.Close())
|
|
}
|
|
logger.LogAttrs(ctx, slog.LevelInfo, "database connection established (migration)",
|
|
dsnLogAttrs(config.DSN)...)
|
|
return db, nil
|
|
}
|
|
|
|
// Connect initializes and returns the runtime connection pool: the
|
|
// application's search_path (core,public, kept when the string carries one)
|
|
// and role=none, always replacing the string's value. It runs no migration.
|
|
func Connect(ctx context.Context, logger *slog.Logger, config *DBConfig) (*sql.DB, error) {
|
|
db, err := openAndConfigureDB(config)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
if err := db.PingContext(ctx); err != nil {
|
|
return nil, errors.Join(fmt.Errorf("failed to connect to database: %w", err), db.Close())
|
|
}
|
|
|
|
attrs := append(dsnLogAttrs(config.DSN),
|
|
slog.Int("max_open_conns", config.MaxOpenConns),
|
|
slog.Int("max_idle_conns", config.MaxIdleConns))
|
|
logger.LogAttrs(ctx, slog.LevelInfo, "database connection established", attrs...)
|
|
|
|
return db, nil
|
|
}
|