From e52fdb37c895e503c0560ae4ce9942dbf04da896 Mon Sep 17 00:00:00 2001 From: Christian Galo Date: Sat, 31 Oct 2026 04:03:32 -0500 Subject: [PATCH] Build every advisory key from its family, and take locks only through the helpers Second of two changes for #186; closes it. Each kind of object the console serializes on now has a family, and every advisory key is built one way: hashtextextended(':', 0), with a UUID written in canonical form. db.Lock and db.LockShared (internal/db/locks.go) build the key and take the lock on a *sql.Tx only, and each family's statement begins with a name line, so pg_stat_activity and the test traces show which family a backend waits on. Families, in the order a transaction takes them: materialization, domains-registry, fedwiki-workspace, lifecycle, stripe-subscription, stripe-invoice, stripe-product-sync, desired-state. The lifecycle and desired-state keys are unchanged byte for byte; the other six change. Migration 00040 replaces core.assert_rendezvous to look for the new materialization key (owner, grants, SECURITY DEFINER and search_path kept; Down restores migration 18's body), and both seeds take the new key right after BEGIN. docs/database-locks.md lists the families in a table and writes one order every path keeps: the materialization lock first, then the families in table order, then rows. The project lint gains advisory-lock-site (an advisory lock function named in a Go string or in SQL or shell code outside the helpers, the test kit's probe, the migrations that name the rendezvous and the rule's own tests; a seed taking any other key, or the rendezvous anywhere but first after BEGIN), pool-lock-site (a core.resource_pools row lock outside LockResourcePool and LockGrantPool, or a call or method value of those outside entitlements/tx.go) and lock-families (the doc's table against the code). The walk covers cmd/, scripts/, internal/, test/ and the root's Go files, and skips testdata and directories whose name begins with a dot, as the go tool does. Tests take a family's lock through the same helpers and probe one with dbtest.TryLock. Recording a lifecycle request comes in two forms: RecordLifecycleRequest opens its own transaction on a database or a pinned connection, and RecordLifecycleRequestInTx runs in the caller's *sql.Tx, so a connection is never used outside a transaction (which released the instance lock at once) and does not compile in the second. The same holds for RecordFirstLifecycleRequest. docs/production-deployment.md now states the upgrade order for every upgrade: stop every process of the running release, run migrate up if migrations run separately, then start the new release; two releases running at once can build different keys for one lock and share none. docs/go-conventions.md, docs/testing.md and docs/building-an-integration.md follow the code. The lint command's help text names the files the lock rules read. 202 golden lines in 109 files change their statement text. Deploy: stop every process of the previous release first. Rollback: stop this release; as the migration user, in one transaction, run 00040's Down section and DELETE FROM goose_db_version_core WHERE version_id = 40; start the previous release (migrate down rolls back the last integration stream first). A deployment using console_reader re-runs scripts/db-reader-role.sql. Break list: 35 records, 35 CAUGHT. Named without a record: the *sql.Tx parameters of the helpers and the -InTx forms (a break that passes a connection does not compile, and the build is the check), and the comments. Full suite on reset databases: 5727 passed, 16 skipped, none failed. --- cmd/lint.go | 15 +- docs/building-an-integration.md | 2 +- docs/database-locks.md | 73 +- docs/go-conventions.md | 20 +- docs/production-deployment.md | 23 +- docs/testing.md | 7 +- internal/db/acts_test.go | 4 +- internal/db/locks.go | 139 +++ internal/db/locks_db_test.go | 163 +++ internal/db/locks_test.go | 95 ++ .../00040_rendezvous_family_key.sql | 80 ++ internal/db/rendezvous_family_key_test.go | 103 ++ internal/db/settlement_pool_first_test.go | 6 +- internal/domains/registry.go | 34 +- internal/domains/registry_db_test.go | 2 +- .../entitlements/plan_act_drain_db_test.go | 2 +- .../entitlements/plan_act_queries_db_test.go | 9 +- .../subscription_delivery_db_test.go | 3 +- .../subscription_resume_db_test.go | 3 +- .../testdata/materialize/pool.golden | 2 +- .../testdata/rule_change/atomic.golden | 2 +- .../testdata/rule_change/deferred.golden | 6 +- internal/entitlements/tx.go | 25 +- internal/fulfillment/reconcile.go | 10 +- .../release_schedule_after_commit_test.go | 5 +- .../reconcile_trace/active_again.golden | 4 +- .../reconcile_trace/addon_added.golden | 4 +- .../reconcile_trace/addon_removed.golden | 4 +- .../reconcile_trace/canceled_replay.golden | 4 +- .../canceled_resumes_grant.golden | 4 +- .../canceled_to_default.golden | 4 +- .../reconcile_trace/first_sight.golden | 4 +- .../grant_over_subscription.golden | 4 +- .../grant_over_suspended_recovery.golden | 4 +- .../reconcile_trace/incomplete.golden | 4 +- .../reconcile_trace/item_replaced.golden | 4 +- .../reconcile_trace/item_without_id.golden | 4 +- .../reconcile_trace/no_customer.golden | 4 +- .../testdata/reconcile_trace/past_due.golden | 4 +- .../reconcile_trace/price_swap.golden | 4 +- .../product_without_set.golden | 4 +- .../testdata/reconcile_trace/quantity.golden | 4 +- .../reconcile_trace/quantity_zero.golden | 4 +- .../testdata/reconcile_trace/replay.golden | 4 +- .../second_subscription_over_first.golden | 4 +- .../three_ladders_first_sight.golden | 4 +- .../two_tiers_one_ladder.golden | 4 +- .../reconcile_trace/unmapped_price.golden | 4 +- .../cancellation_cleared.golden | 4 +- .../dashboard_schedule.golden | 4 +- .../downgrade_landed.golden | 4 +- .../downgrade_lands_while_displaced.golden | 4 +- .../downgrade_pending.golden | 4 +- .../later_upgrade.golden | 6 +- .../release_refused.golden | 4 +- .../schedule_released_outside.golden | 4 +- .../term_boundary_switch_pending.golden | 4 +- .../sweep_trace/already_canceled.golden | 4 +- .../testdata/sweep_trace/already_ended.golden | 4 +- .../sweep_trace/cancellation_refused.golden | 4 +- .../sweep_trace/due_cancellation.golden | 4 +- .../testdata/sweep_trace/lost_claim.golden | 4 +- .../sweep_trace/period_end_landed.golden | 4 +- .../sweep_trace/stale_downgrade.golden | 4 +- .../sweep_trace/term_boundary_switch.golden | 4 +- internal/fulfillment/trace_db_test.go | 4 +- .../integration/delivery_queries_db_test.go | 6 +- internal/integration/desired_state.sql.go | 37 +- internal/integration/export_test.go | 5 + internal/integration/lifecycle.go | 129 ++- .../lifecycle_answers_golden_db_test.go | 4 +- internal/integration/lifecycle_db_test.go | 60 +- .../integration/lifecycle_requests.sql.go | 26 +- internal/integration/querier.go | 30 +- .../integration/queries/desired_state.sql | 17 +- .../queries/lifecycle_requests.sql | 20 +- .../fedwiki/store/request_intake.go | 17 +- .../fedwiki/store/request_intake_db_test.go | 8 +- internal/integrations/fedwiki/usage/usage.go | 28 +- .../fedwiki/usage/usage_db_test.go | 2 +- .../fedwiki/web/request_identity_db_test.go | 2 +- .../integrations/fedwiki/web/site_actions.go | 11 +- .../fedwiki/web/workspace_lock_db_test.go | 50 + .../workflows/purge_failures_db_test.go | 4 +- .../fedwiki/workflows/reconcile.go | 3 +- .../fedwiki/workflows/request_activities.go | 2 +- .../stripe/catalogsync/catalogsync.go | 15 +- .../stripe/workflows/invoice_reconcile.go | 8 +- .../a-charge-without-a-currency.golden | 2 +- .../a-failure-later-reported-succeeded.golden | 2 +- .../a-method-that-appears-later.golden | 2 +- ...er-payments-kept-beside-the-charges.golden | 2 +- .../paid-without-a-charge-read-again.golden | 2 +- .../paid-without-a-charge.golden | 2 +- .../the-same-read-again.golden | 2 +- .../two-charges-in-the-same-second.golden | 2 +- .../two-declines-then-a-success.golden | 2 +- .../every-kind-of-candidate.golden | 6 +- internal/lint/lint.go | 10 + internal/lint/locks.go | 437 ++++++++ internal/lint/locks_test.go | 190 ++++ .../server/operator_org_type_change_test.go | 3 +- .../server/operator_sync_stripe_db_test.go | 3 +- .../server/plan_acts_concurrency_db_test.go | 2 +- .../default_price_with_nothing_mapped.golden | 2 +- .../product_create_in_flight.golden | 2 +- ...uested_price_of_a_reachable_product.golden | 2 +- .../add-above-the-limit-default-sourced.json | 2 +- .../plan_acts/add-above-the-limit.json | 2 +- .../add-aligns-a-live-provision.json | 2 +- .../add-collides-with-a-held-rung.json | 4 +- .../plan_acts/add-product-held-twice.json | 2 +- .../plan_acts/add-product-unread.json | 2 +- .../plan_acts/default-already-at.json | 6 +- .../default-gate-true-preview-empty.json | 2 +- .../plan_acts/default-grandfather.json | 12 +- .../plan_acts/default-initiate-fails.json | 6 +- .../plan_acts/default-listing-fails.json | 4 +- .../default-migrate-restore-fails.json | 6 +- .../testdata/plan_acts/default-migrate.json | 8 +- .../plan_acts/default-needs-disposition.json | 2 +- .../plan_acts/default-none-migrate.json | 10 +- .../plan_acts/default-preview-capped.json | 2 +- .../plan_acts/default-request-cap.json | 6 +- .../plan_acts/default-retry-nothing.json | 2 +- .../testdata/plan_acts/default-retry.json | 6 +- .../plan_acts/default-save-fails.json | 2 +- .../plan_acts/default-superseded.json | 10 +- .../plan_acts/default-untouched-outgoing.json | 6 +- .../plan_acts/delete-bottom-tier.json | 2 +- .../testdata/plan_acts/delete-held-tier.json | 2 +- .../delete-last-tier-of-a-default.json | 2 +- .../plan_acts/delete-org-types-unread.json | 2 +- .../plan_acts/delete-suspended-holder.json | 2 +- .../testdata/plan_acts/delete-top-tier.json | 2 +- .../plan_acts/remove-above-the-limit.json | 2 +- .../plan_acts/remove-keep-rank-0.json | 2 +- .../plan_acts/remove-last-tier-keep.json | 2 +- .../plan_acts/remove-last-tier-migrate.json | 2 +- .../remove-migrate-pool-held-elsewhere.json | 2 +- .../plan_acts/remove-migrate-rank-0.json | 2 +- .../plan_acts/remove-needs-disposition.json | 2 +- .../plan_acts/remove-no-holders-left.json | 2 +- .../plan_acts/remove-org-types-unread.json | 2 +- .../plan_acts/remove-other-source.json | 2 +- .../plan_acts/remove-product-name-unread.json | 2 +- .../plan_acts/remove-second-pool-fails.json | 2 +- .../testdata/plan_acts/remove-two-pools.json | 2 +- .../reorder-an-organization-fails.json | 8 +- .../testdata/plan_acts/reorder-cosmetic.json | 2 +- .../plan_acts/reorder-grandfather.json | 12 +- .../plan_acts/reorder-listing-fails.json | 4 +- .../testdata/plan_acts/reorder-migrate.json | 8 +- .../plan_acts/reorder-needs-disposition.json | 2 +- .../plan_acts/reorder-rank-0-no-default.json | 2 +- .../plan_acts/reorder-record-fails.json | 2 +- .../testdata/plan_acts/reorder-redrop.json | 2 +- .../testdata/plan_acts/reorder-retry.json | 6 +- .../testdata/plan_acts/reorder-stale.json | 2 +- .../testdata/plan_acts/reorder-two-types.json | 10 +- internal/testkit/dbtest/locks.go | 25 + internal/testkit/dbtest/locks_test.go | 38 + .../workflows/desiredstate/export_test.go | 8 + internal/workflows/desiredstate/push.go | 2 +- internal/workflows/desiredstate/readback.go | 2 +- internal/workflows/desiredstate/record.go | 8 +- .../desiredstate/record_lock_db_test.go | 40 + .../.openspec.yaml | 2 + .../advisory-keys-name-their-family/design.md | 233 +++++ .../proposal.md | 55 + .../specs/database-locks/spec.md | 69 ++ .../specs/deployment-docs/spec.md | 15 + .../specs/entitlement-set-management/spec.md | 89 ++ .../specs/entitlements/spec.md | 84 ++ .../specs/go-static-analysis/spec.md | 50 + .../advisory-keys-name-their-family/tasks.md | 947 ++++++++++++++++++ test/mockshot-seed.sql | 12 +- test/seed/discourse/seed-discourse.sh | 12 +- 178 files changed, 3520 insertions(+), 531 deletions(-) create mode 100644 internal/db/locks.go create mode 100644 internal/db/locks_db_test.go create mode 100644 internal/db/locks_test.go create mode 100644 internal/db/migrations/00040_rendezvous_family_key.sql create mode 100644 internal/db/rendezvous_family_key_test.go create mode 100644 internal/integrations/fedwiki/web/workspace_lock_db_test.go create mode 100644 internal/lint/locks.go create mode 100644 internal/lint/locks_test.go create mode 100644 internal/testkit/dbtest/locks.go create mode 100644 internal/testkit/dbtest/locks_test.go create mode 100644 internal/workflows/desiredstate/record_lock_db_test.go create mode 100644 openspec/changes/advisory-keys-name-their-family/.openspec.yaml create mode 100644 openspec/changes/advisory-keys-name-their-family/design.md create mode 100644 openspec/changes/advisory-keys-name-their-family/proposal.md create mode 100644 openspec/changes/advisory-keys-name-their-family/specs/database-locks/spec.md create mode 100644 openspec/changes/advisory-keys-name-their-family/specs/deployment-docs/spec.md create mode 100644 openspec/changes/advisory-keys-name-their-family/specs/entitlement-set-management/spec.md create mode 100644 openspec/changes/advisory-keys-name-their-family/specs/entitlements/spec.md create mode 100644 openspec/changes/advisory-keys-name-their-family/specs/go-static-analysis/spec.md create mode 100644 openspec/changes/advisory-keys-name-their-family/tasks.md diff --git a/cmd/lint.go b/cmd/lint.go index 68521c13..bb4dca77 100644 --- a/cmd/lint.go +++ b/cmd/lint.go @@ -15,10 +15,11 @@ import ( var lintCmd = &cobra.Command{ Use: "lint", Short: "Project lint: routes, swap targets, page anatomy and copy", - Long: `lint walks the Go source under internal/, the templates under -internal/embeds/templates/, the scripts under internal/embeds/static/ and the -walkthroughs and screens manifest under test/e2e/ and the specs under -openspec/specs/, and reports: + Long: `lint walks the Go source under internal/ (and, for the lock rules, the Go, +SQL and shell files under cmd/, scripts/, internal/ and test/ and the root's +Go files), the templates under internal/embeds/templates/, the scripts under +internal/embeds/static/ and the walkthroughs and screens manifest under +test/e2e/ and the specs under openspec/specs/, and reports: - routes: routeURL patterns with no registered handler - swap targets: hx-target="#X" with no matching id="X" @@ -34,6 +35,9 @@ openspec/specs/, and reports: - spec purposes: a spec whose Purpose is the archive's placeholder or empty - templates: a template that does not parse in its surface, or that no surface parses + - lock sites: an advisory or pool row lock taken outside its helper, + a seed's other key, or a family table that disagrees + with internal/db Exits 0 on a clean tree, 1 on violations, 2 on internal errors. See docs/testing.md, "Static checks".`, @@ -45,6 +49,7 @@ docs/testing.md, "Static checks".`, allowlistPath, _ := cmd.Flags().GetString("anatomy-allowlist") manifestPath, _ := cmd.Flags().GetString("screens-manifest") specsDir, _ := cmd.Flags().GetString("specs-dir") + lockRoot, _ := cmd.Flags().GetString("lock-root") res, err := lint.Run(lint.Config{ ServerDir: serverDir, @@ -53,6 +58,7 @@ docs/testing.md, "Static checks".`, AllowlistPath: allowlistPath, ManifestPath: manifestPath, SpecsDir: specsDir, + LockRoot: lockRoot, }) if err != nil { fmt.Fprintf(os.Stderr, "lint: %v\n", err) @@ -77,5 +83,6 @@ func init() { lintCmd.Flags().String("screens-manifest", "test/e2e/screens/manifest.go", `Screens manifest for the screen-coverage rule ("-" disables it)`) lintCmd.Flags().String("walkthrough-dir", "test/e2e/operator-walkthroughs", `Walkthrough directory for coverage rule (set "-" to disable)`) lintCmd.Flags().String("specs-dir", "openspec/specs", `Specs directory for the spec-purpose-placeholder rule (set "-" to disable)`) + lintCmd.Flags().String("lock-root", ".", `Repository root the lock rules walk ("-" disables the lock rules)`) rootCmd.AddCommand(lintCmd) } diff --git a/docs/building-an-integration.md b/docs/building-an-integration.md index d54d2bde..05e52aa4 100644 --- a/docs/building-an-integration.md +++ b/docs/building-an-integration.md @@ -589,7 +589,7 @@ Use this decision rule when designing a new integration's mutating verbs: FarmManager and write `fedwiki.sites` in the same workflow) — **dispatch directly** via your own `RegisterWorkflows`-registered workflows. When a person starts the operation, record it first as a lifecycle request - through `internal/integration`'s writers (`RecordLifecycleRequest` in the + through `internal/integration`'s writers (`RecordLifecycleRequestInTx` in the same transaction as your own rows, then start the workflow under the request's workflow id, and finish the request from the workflow), as FedWiki does: the handler then answers when the workflow starts, and a diff --git a/docs/database-locks.md b/docs/database-locks.md index de9b331e..70333a77 100644 --- a/docs/database-locks.md +++ b/docs/database-locks.md @@ -8,9 +8,7 @@ summary: "Every advisory lock and serializing row lock the console takes, how ea This page describes the locks as the code takes them today. It records what each lock serializes, who takes it, and the order each path takes several. -It does not describe a target design. One piece of planned work changes the -page: #186 gives every advisory key one family prefix and writes one lock -order. Whoever lands it edits this page in the same change. +It does not describe a target design. The code names functions and sqlc queries; so does this page. Line numbers are left out because they move. @@ -27,14 +25,14 @@ creators lock exactly one site `FOR SHARE` (see Registry below). | Lock | Key | Mode | Taken by | Serializes | |---|---|---|---|---| -| Materialization rendezvous | `hashtextextended('entitlement_materialization', 0)` | shared for a transaction that materializes a pool; exclusive for a rule change, for a change to a ladder's tiers (tier add, tier delete, tier removal, the reorder's renumbering) and for a default change's T0 | the `entitlements.Materializing` and `entitlements.RuleChange` transaction kinds (`BeginMaterializing`, `BeginRuleChange`), as the transaction's first statement | A rule change or a tier change against every materializing transaction: each one finishes before the change or starts after it, so a conferral reads one set of rules and one order of each ladder. Materializing transactions do not wait on each other here. | -| Subscription | `hashtext()::bigint` | exclusive | `fulfillment.ReconcileSubscription` | Two reconciles of one subscription (the checkout return and a webhook), so they converge instead of racing the exclusion constraint on subscription periods. | -| Invoice | `hashtext()::bigint` | exclusive | `invoiceReconcile.converge` in `internal/integrations/stripe/workflows` | Two reconciles of one invoice (two events, or an event and the sweep), so the second finds what the first wrote. | -| Product sync | `hashtext()::bigint` | exclusive | `catalogsync.Sync` in `internal/integrations/stripe/catalogsync` | Two sync clicks for one product, so the mapping writes and the outbox enqueue happen once. | -| FedWiki workspace | `hashtext()::bigint` | exclusive | `withWorkspaceTx` in `internal/integrations/fedwiki/web`, for the status-change and swap handlers | One workspace's rotations of its active site, so two tabs cannot interleave the cooldown check and stamp or the choice of the site to park. | -| Lifecycle instance | `hashtextextended('lifecycle:' \|\| , 0)` | exclusive | the `LockLifecycleInstance` query, run by `integration.RecordLifecycleRequest`, `RecordFirstLifecycleRequest` and the FedWiki activity helpers `finishAllWith` and `purgeArchivedSite` | One instance's requests: the latest-generation check and the insert are one step, and an outcome is recorded against a request that is still accepted; and retention's check that no request is open, held through its farm delete and local deletion. | -| Desired-state address | `hashtextextended('desired-state:' \|\| \|\| ':' \|\| \|\| ':' \|\| , 0)` | exclusive | the `LockDesiredStateAddress` query, run by `lockAddress` in `internal/workflows/desiredstate` | Every write to the content or generation of the record at one (connection, recipient) address, and to its projection, including an address that has no record yet. | -| Domains registry | the constant `registryAllocationLock` (the ASCII bytes of "domains") | exclusive | `domains.Registry.WithLock` | Every mutating allocation in the registry, one lock for the whole namespace, because the disjointness rules span rows no database constraint can compare. | +| Materialization rendezvous | `hashtextextended('materialization:', 0)` | shared for a transaction that materializes a pool; exclusive for a rule change, for a change to a ladder's tiers (tier add, tier delete, tier removal, the reorder's renumbering) and for a default change's T0 | the `entitlements.Materializing` and `entitlements.RuleChange` transaction kinds (`BeginMaterializing`, `BeginRuleChange`), as the transaction's first statement; `test/mockshot-seed.sql` and `test/seed/discourse/seed-discourse.sh`, as their first statement after `BEGIN` | A rule change or a tier change against every materializing transaction: each one finishes before the change or starts after it, so a conferral reads one set of rules and one order of each ladder. Materializing transactions do not wait on each other here. | +| Subscription | `hashtextextended('stripe-subscription:', 0)` | exclusive | `fulfillment.ReconcileSubscription` | Two reconciles of one subscription (the checkout return and a webhook), so they converge instead of racing the exclusion constraint on subscription periods. | +| Invoice | `hashtextextended('stripe-invoice:', 0)` | exclusive | `invoiceReconcile.converge` in `internal/integrations/stripe/workflows` | Two reconciles of one invoice (two events, or an event and the sweep), so the second finds what the first wrote. | +| Product sync | `hashtextextended('stripe-product-sync:', 0)` | exclusive | `catalogsync.Sync` in `internal/integrations/stripe/catalogsync` | Two sync clicks for one product, so the mapping writes and the outbox enqueue happen once. | +| FedWiki workspace | `hashtextextended('fedwiki-workspace:', 0)` | exclusive | `withWorkspaceTx` in `internal/integrations/fedwiki/web`, for the status-change and swap handlers | One workspace's rotations of its active site, so two tabs cannot interleave the cooldown check and stamp or the choice of the site to park. | +| Lifecycle instance | `hashtextextended('lifecycle:', 0)` | exclusive | `db.Lock` on the lifecycle family, run by `integration.RecordLifecycleRequestInTx` and `RecordFirstLifecycleRequestInTx`, FedWiki's `RecordSiteRequestInTx` and `CheckSwapRequestSites`, `usage.Hold`, and the activity helpers `finishAllWith` and `purgeArchivedSite` | One instance's requests: the latest-generation check and the insert are one step, and an outcome is recorded against a request that is still accepted; and retention's check that no request is open, held through its farm delete and local deletion. | +| Desired-state address | `hashtextextended('desired-state:::', 0)` | exclusive | `lockAddress` in `internal/workflows/desiredstate` | Every write to the content or generation of the record at one (connection, recipient) address, and to its projection, including an address that has no record yet. | +| Domains registry | `hashtextextended('domains-registry:', 0)` | exclusive | `registryKind`, the kind of `domains.Registry.WithLock` | Every mutating allocation in the registry, one lock for the whole namespace, because the disjointness rules span rows no database constraint can compare. | ### Row locks @@ -60,25 +58,30 @@ flag-only writes change neither claims nor allocation counts. | Desired-state notice | `core.desired_state_outbox` rows | `FOR UPDATE SKIP LOCKED` | the `ClaimDesiredStateOutbox` query | Two relay passes, so each claims different rows; a pass skips rows another still holds. Triggers write the rows through `core.insert_desired_state_notice` inside the transaction that changed the source, and they take no lock beyond the insert. | | Resource-key row | `core.resource_keys` row of one key | the update lock of `SetResourceKeyProvider` | `integration.RegisterProviders` at boot, stamping keys in byte order | A stamp that changes a key's provider against an open entitlement insert of that key, which holds the row's `FOR KEY SHARE` through the foreign key. The materializer writes a pool's keys in byte order too, so the two wait in one direction. | -## How advisory keys are built +## Advisory key families -Postgres takes one 64-bit key for an advisory lock, so every lock above maps -its identity onto a bigint. The code builds those keys four ways. +Every advisory key is `hashtextextended(':', 0)`: the family's name, a colon and the id, whose parts are joined by colons. `db.Lock` and `db.LockShared` (`internal/db/locks.go`) build it and take it, on a transaction, and nothing else names an advisory lock function (the project lint's `advisory-lock-site`). A UUID part is written in canonical form, as `uuid::text` writes it, and a text part holds no colon; a family with one lock for the whole database has no id. Each family's statement begins with a name line (`-- name: LockSubscription :exec`), which `pg_stat_activity` shows for a waiter and the statement traces record. The table lists the families in the order a transaction takes them; the lint fails when it and `db.Families` differ (`lock-families`). -1. **`hashtextextended` of a fixed name.** The rendezvous. One key for the - whole database. -2. **`hashtextextended` of a prefixed string.** The lifecycle instance - (`lifecycle:` and the instance id) and the desired-state address - (`desired-state:`, the connection, the recipient kind and the recipient - id). The prefix keeps the two families from sharing keys with each other. -3. **`hashtext` of a bare id, widened to bigint.** The subscription, the - invoice, the product sync and the FedWiki workspace. `hashtext` returns 32 - bits, so these four families share one 32-bit key space with no prefix - between them; a collision serializes two unrelated objects briefly and - cannot lose a write. The ids differ in shape (Stripe ids against UUIDs), - which makes a cross-family collision unlikely, not impossible. -4. **A literal constant.** The domains registry. Its value is larger than any - 32-bit hash, so it cannot collide with family 3. +| Family | Id | Statement | Taken by | +|---|---|---|---| +| `materialization` | none | `LockMaterialization`, `LockMaterializationShared` | the `entitlements.Materializing` and `entitlements.RuleChange` kinds; `test/mockshot-seed.sql` and `test/seed/discourse/seed-discourse.sh`, as their first statement; `core.assert_rendezvous` checks its key | +| `domains-registry` | none | `LockDomainsRegistry` | `registryKind`, the kind of `domains.Registry.WithLock` | +| `fedwiki-workspace` | workspace id | `LockFedWikiWorkspace` | `withWorkspaceTx` | +| `lifecycle` | instance id | `LockLifecycleInstance` | the lifecycle recording, FedWiki's intake, the slot gate, the outcome steps and the retention purge | +| `stripe-subscription` | Stripe subscription id | `LockSubscription` | `fulfillment.ReconcileSubscription`, `supersedeReleasedDowngrades` | +| `stripe-invoice` | Stripe invoice id | `LockInvoice` | `invoiceReconcile.converge` | +| `stripe-product-sync` | product id | `LockProductSync` | `catalogsync.Sync` | +| `desired-state` | connection id, recipient kind, recipient id | `LockDesiredStateAddress` | `lockAddress` in `internal/workflows/desiredstate` | + +A change to a family's name or to the shape of its id changes its keys. Two processes that build different keys for one object hold no common lock, so a release that changes a key is deployed as [production-deployment.md](production-deployment.md), section 5, orders an upgrade. + +## The order every path keeps + +1. A transaction that takes the materialization rendezvous takes it as its first statement. No other advisory lock comes before it. +2. The advisory families the transaction needs, in the order of the table above. Several locks of one family follow the order this page gives for that family: lifecycle instances in the order their requests were recorded. +3. Rows, by the rules below: a rule change's set and rule rows before pool rows; pool rows in ascending `pool_id`, before a grant's lineage head, obligation rows and a tier's product row; FedWiki's site rows, then their placement rows through the trigger, then the usage row, then request rows; a desired-state record after its address. + +No path waits on an advisory lock after it takes a row lock: a recording that takes an instance lock its transaction already holds, as the swap's second recording does, takes it without waiting. The sections below list each path's locks in this order. ## The order each path takes locks @@ -539,9 +542,11 @@ writes no count; the insert, the status write and the delete commit. generation, then inserts. `RecordFirstLifecycleRequest` takes the same lock once and reads the latest generation once: it refuses a later generation of an instance that already holds a request, then decides as -`RecordLifecycleRequest` does. Called with a `*sql.DB`, each opens its own -transaction; called with a transaction, each uses the caller's, which is how -the handler records under the workspace lock. +`RecordLifecycleRequest` does. `RecordLifecycleRequest` and +`RecordFirstLifecycleRequest` open their own transaction on a database or a +pinned connection; `RecordLifecycleRequestInTx` and +`RecordFirstLifecycleRequestInTx` run in the caller's, which is how the handler +records under the workspace lock. `DeleteUnstartedLifecycleRequest`, which removes a request whose workflow never started, takes no instance lock, so a repeat that races it may be answered from either side of the delete. @@ -586,7 +591,7 @@ registry operations; skipped sites do not incur this wait. `registryKind`, whose first statement is the registry lock. Every composite operation, including the FedWiki create, release and sync projections, runs on the `*Tx` it passes, so the lock is taken once. The comment on -`registryAllocationLock` states the rule for a transaction that would touch both +`registryKind` states the rule for a transaction that would touch both the registry and a usage row: registry lock first. No transaction does both today: the slot gate and the recounts are workflow activities that run their own transactions, apart from the transaction that allocates or releases the name. @@ -603,7 +608,7 @@ transactions, apart from the transaction that allocates or releases the name. adds its ladder's tiers on the plain handle, outside the rendezvous. `core.assert_rendezvous` raises `materialization_rendezvous_missing` when a function that materializes or - settles runs without it. + settles runs without it. No other advisory lock precedes it. 2. The pool row is locked before positions or entitlements change. The five conferral and settlement functions lock the row themselves, so a caller that goes straight to one of them still holds it. @@ -642,5 +647,3 @@ transactions, apart from the transaction that allocates or releases the name. inside `core.end_conferral`, and takes it again for resumption and before the floor restoration. Each path then runs resumption, the floor restoration and materialization in the same order. -- **Advisory keys are built four ways** (#186). The keys follow the four - conventions above. diff --git a/docs/go-conventions.md b/docs/go-conventions.md index 861c1663..3abbf99b 100644 --- a/docs/go-conventions.md +++ b/docs/go-conventions.md @@ -146,12 +146,20 @@ are those two calls). A failed begin, a failed commit and `fn`'s own error come back unwrapped; the caller adds its own context once around the whole act. -A helper that takes a lock and then runs more statements under it, its own -or its caller's, takes a `*sql.Tx` rather than sqlc's `DBTX` (`usage.Recount`, -`usage.Hold`, and FedWiki's intake helpers `RecordSiteRequestInTx` and -`CheckSwapRequestSites`). Every advisory lock is transaction-scoped, so given -a `*sql.DB` each statement runs in a transaction of its own, the lock ends -with the statement that took it, and nothing reports the loss. +A helper that takes a lock and then runs more statements under it, its own or +its caller's, takes a `*sql.Tx` rather than sqlc's `DBTX`: an advisory lock +(`usage.Hold`, `integration.RecordLifecycleRequestInTx`, and FedWiki's intake +helpers `RecordSiteRequestInTx` and `CheckSwapRequestSites`) or a row lock +(`usage.Recount`, the usage row). Both kinds last until the transaction ends, +so given a `*sql.DB` each statement runs in a transaction of its own, the lock +ends with the statement that took it, and nothing reports the loss. An +advisory lock goes through `db.Lock` or `db.LockShared` +(`internal/db/locks.go`) with its family and id; the project lint refuses an +advisory lock function anywhere else, and a pool row lock outside `LockPools` +and `LockPoolOfGrant`. Those two are the one exception to the `*sql.Tx` rule: +they stay methods of `*entitlements.Queries`, because the plan act runner's +querier seam wraps them in tests and every caller builds `Queries` on a +transaction. No call to Stripe runs while a transaction is open. A path reads from Stripe before it opens its transaction, and writes to Stripe after the commit or diff --git a/docs/production-deployment.md b/docs/production-deployment.md index f7c70100..98cae04d 100644 --- a/docs/production-deployment.md +++ b/docs/production-deployment.md @@ -160,12 +160,13 @@ also uses `db-migrate-dsn`. **Run migrations as a separate step.** To keep the migration password out of the running process, leave `db-migrate-dsn` off the service and run -`member-console migrate up` with it set as a one-shot step before each -deploy. With `db-migrate-dsn` unset, `start` runs no migration, and it -refuses to start while the schema is behind the code. After the first run of -`migrate up`, run, as the migration user, the `GRANT console_runtime TO - WITH INHERIT TRUE, SET FALSE;` statement that the refusal names, because -only a boot with `db-migrate-dsn` set grants it. +`member-console migrate up` with it set as a one-shot step after the old +release stops and before the new one starts (below). With `db-migrate-dsn` +unset, `start` runs no migration, and it refuses to start while the schema is +behind the code. After the first run of `migrate up`, run, as the migration +user, the `GRANT console_runtime TO WITH INHERIT TRUE, SET +FALSE;` statement that the refusal names, because only a boot with +`db-migrate-dsn` set grants it. **Check configuration before restarting.** `member-console config validate` connects to the database, parses every stored override and the resolved @@ -176,6 +177,16 @@ upgrade's restart so a bad value (an environment variable edited by hand, or an override saved before this check existed) surfaces in your terminal instead of the service manager's restart loop. +**Stop the old release before starting the new one.** Stop every process of +the running release, run `member-console migrate up` if you run migrations as +a separate step, then start the new release. Two releases running at once can +build different keys for one advisory lock and then hold no common lock, so +writes the lock serializes can interleave +([database-locks.md](database-locks.md#advisory-key-families)); and a +migration can change what the old release's writes need. An orchestrator's +rolling update that starts the new task before it stops the old one, or a +service with more than one replica, is stopped or scaled to zero first. + **Manage stored overrides from the terminal.** `member-console config` offers the operator panel's three override actions from the command line: `config list [integration]` prints each declared key's environment or diff --git a/docs/testing.md b/docs/testing.md index e851ca3b..c7010cb8 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -136,6 +136,10 @@ that unit tests can't reach. - `dbtest.AwaitLockWait` polls until a backend waits on a lock (`Waiters` for a number of backends queued at once), for a test that holds one transaction open while a second one blocks. +- `dbtest.TryLock` probes whether a family's lock is free, under the key the + code builds. A test that holds a family's lock takes it with `db.Lock`, and + one that waits on it gives `dbtest.Backend` the family's `Statement()`, so + the test meets the key and the statement the code uses. **Fixtures.** `world` (`internal/testkit/world`) builds the rows a test needs through the functions the console itself calls. Its builders cover people, @@ -313,6 +317,7 @@ checks, and `make lint-templates` is a sixth that runs on its own: - copy: `em-dash-copy`, `filler-copy`; - specs: `spec-purpose-placeholder`, a spec under `openspec/specs/` whose Purpose is the archive's placeholder or empty; + - locks: `advisory-lock-site`, `pool-lock-site`, `lock-families`; - htmx: `htmx-2-compat`, `htmx-old-event`, `hx-disinherit`; - error text: `raw-error-render`; - reading: `unreadable-file`, a directory the lint cannot list or a file it @@ -491,7 +496,7 @@ a test of its own. | Package | Holds | Imports | |---|---|---| | `testkit` | `Close`, which closes an `io.Closer` and fails the test on its error | no console package | -| `testkit/dbtest` | opening the test databases, scratch databases, counting the statements a request runs, `Q`, `AwaitLockWait` | `internal/db` and `internal/testkit` only (`internal/migrate` reaches nearly every package) | +| `testkit/dbtest` | opening the test databases, scratch databases, counting the statements a request runs, `Q`, `AwaitLockWait`, `TryLock` | `internal/db` and `internal/testkit` only (`internal/migrate` reaches nearly every package) | | `testkit/htmltest` | parsing HTML and selecting from it with CSS | no console package | | `testkit/golden` | comparing a test's output with a file under testdata, `-update` writing it, and naming the ids and times a run makes | no console package | | `testkit/replay` | replaying a package's recorded Temporal histories against its workflows, and recording them | no console package | diff --git a/internal/db/acts_test.go b/internal/db/acts_test.go index 73f4f83a..49b62923 100644 --- a/internal/db/acts_test.go +++ b/internal/db/acts_test.go @@ -31,8 +31,8 @@ const ( // refusal, materialization_rendezvous_missing. notInPrerequisiteState = "55000" - exclusiveRendezvous = `SELECT pg_advisory_xact_lock(hashtextextended('entitlement_materialization', 0))` - sharedRendezvous = `SELECT pg_advisory_xact_lock_shared(hashtextextended('entitlement_materialization', 0))` + exclusiveRendezvous = `SELECT pg_advisory_xact_lock(hashtextextended('materialization:', 0))` + sharedRendezvous = `SELECT pg_advisory_xact_lock_shared(hashtextextended('materialization:', 0))` // recordTierRemoved calls core.record_plan_act for a tier removal by a // person: $1 the person, $2 the ladder, $3 the product. diff --git a/internal/db/locks.go b/internal/db/locks.go new file mode 100644 index 00000000..409c1f4d --- /dev/null +++ b/internal/db/locks.go @@ -0,0 +1,139 @@ +// SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Commercial +// SPDX-FileCopyrightText: 2025-2026 Christian Galo + +package db + +import ( + "context" + "database/sql" + "errors" + "fmt" + "strings" + + "github.com/google/uuid" +) + +// idPart is the shape of one part of a family's id. +type idPart int + +const ( + // textPart is used as given and may be neither empty nor hold ':', which + // would run into the next part: a Stripe id, a kind. + textPart idPart = iota + // uuidPart is parsed and written in canonical form, as Postgres writes + // uuid::text, so two spellings of one id take one key. + uuidPart +) + +// Family is one advisory lock family. Its key for an id is +// hashtextextended(name || ':' || id, 0), the id's parts joined by ':'; a +// family with one lock for the whole database has an id of no parts. Only +// this file declares one. docs/database-locks.md lists the families in the +// order a transaction takes them, and the project lint holds its table to +// Families. +type Family struct { + name string + label string + parts []idPart +} + +var ( + // Materialization is the rendezvous: every transaction that materializes + // a pool holds it shared, every rule change and operator tier change + // exclusive (entitlements.Materializing, entitlements.RuleChange). + // core.assert_rendezvous checks its key. + Materialization = Family{name: "materialization", label: "LockMaterialization"} + // DomainsRegistry serializes every mutating allocation of a name + // (domains.Registry.WithLock). + DomainsRegistry = Family{name: "domains-registry", label: "LockDomainsRegistry"} + // FedWikiWorkspace serializes one workspace's rotations of its active + // site. + FedWikiWorkspace = Family{name: "fedwiki-workspace", label: "LockFedWikiWorkspace", parts: []idPart{uuidPart}} + // Lifecycle serializes one instance's lifecycle requests. + Lifecycle = Family{name: "lifecycle", label: "LockLifecycleInstance", parts: []idPart{uuidPart}} + // Subscription serializes two reconciles of one Stripe subscription. + Subscription = Family{name: "stripe-subscription", label: "LockSubscription", parts: []idPart{textPart}} + // Invoice serializes two reconciles of one Stripe invoice. + Invoice = Family{name: "stripe-invoice", label: "LockInvoice", parts: []idPart{textPart}} + // ProductSync serializes two Stripe syncs of one product. + ProductSync = Family{name: "stripe-product-sync", label: "LockProductSync", parts: []idPart{uuidPart}} + // DesiredState serializes every write at one address: a connection, a + // recipient kind and a recipient. + DesiredState = Family{name: "desired-state", label: "LockDesiredStateAddress", parts: []idPart{uuidPart, textPart, uuidPart}} +) + +// Families is every family, in the order a transaction takes them. +var Families = []Family{Materialization, DomainsRegistry, FedWikiWorkspace, Lifecycle, Subscription, Invoice, ProductSync, DesiredState} + +// Name is the family's name, the prefix of its keys. +func (f Family) Name() string { return f.name } + +// Key is the text whose hashtextextended is f's key for the id's parts. It +// refuses a family this file did not declare, a number of parts other than +// the family's, an empty text part, a text part that holds ':' and a uuid +// part that does not parse. +func (f Family) Key(id ...string) (string, error) { + if f.name == "" { + return "", errors.New("advisory lock: no family") + } + if len(id) != len(f.parts) { + return "", fmt.Errorf("advisory lock %s: %d id parts, want %d", f.name, len(id), len(f.parts)) + } + parts := make([]string, len(id)) + for i, p := range id { + switch f.parts[i] { + case uuidPart: + u, err := uuid.Parse(p) + if err != nil { + return "", fmt.Errorf("advisory lock %s: id part %d: %w", f.name, i+1, err) + } + parts[i] = u.String() + default: + if p == "" { + return "", fmt.Errorf("advisory lock %s: id part %d is empty", f.name, i+1) + } + if strings.Contains(p, ":") { + return "", fmt.Errorf("advisory lock %s: id part %d holds ':'", f.name, i+1) + } + parts[i] = p + } + } + return f.name + ":" + strings.Join(parts, ":"), nil +} + +// Statement is the statement Lock runs for f, the key as $1. Its first line +// names the family, which pg_stat_activity shows for a waiter and the +// statement traces record; a test waits on it by its text. +func (f Family) Statement() string { + return "-- name: " + f.label + " :exec\nSELECT pg_advisory_xact_lock(hashtextextended($1, 0))" +} + +// SharedStatement is the statement LockShared runs for f. +func (f Family) SharedStatement() string { + return "-- name: " + f.label + "Shared :exec\nSELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))" +} + +// Lock takes f's exclusive transaction-scoped advisory lock on the id, +// released at commit or rollback. It takes a *sql.Tx because on a database +// or a connection outside a transaction the lock would end with the +// statement that took it. An id Key refuses sends no statement; the +// driver's error comes back as it is. +func Lock(ctx context.Context, tx *sql.Tx, f Family, id ...string) error { + return take(ctx, tx, f.Statement(), f, id) +} + +// LockShared takes f's shared lock on the id, as Lock takes the exclusive +// one. +func LockShared(ctx context.Context, tx *sql.Tx, f Family, id ...string) error { + return take(ctx, tx, f.SharedStatement(), f, id) +} + +// take builds f's key for id and runs stmt with it on tx. +func take(ctx context.Context, tx *sql.Tx, stmt string, f Family, id []string) error { + key, err := f.Key(id...) + if err != nil { + return err + } + _, err = tx.ExecContext(ctx, stmt, key) + return err +} diff --git a/internal/db/locks_db_test.go b/internal/db/locks_db_test.go new file mode 100644 index 00000000..65e06da1 --- /dev/null +++ b/internal/db/locks_db_test.go @@ -0,0 +1,163 @@ +// SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Commercial +// SPDX-FileCopyrightText: 2025-2026 Christian Galo + +package db_test + +import ( + "context" + "database/sql" + "strings" + "testing" + + "github.com/google/uuid" + + "git.coopcloud.tech/wiki-cafe/member-console/internal/db" + "git.coopcloud.tech/wiki-cafe/member-console/internal/testkit/dbtest" +) + +// lockRow is one family, an id it locks and another id of the same family; +// a family with one lock for the whole database has neither. +type lockRow struct { + family db.Family + id []string + other []string +} + +// lockRows builds a row for each family from ids no other test locks. +func lockRows() []lockRow { + u := func() string { return uuid.NewString() } + return []lockRow{ + {family: db.Materialization}, + {family: db.DomainsRegistry}, + {family: db.FedWikiWorkspace, id: []string{u()}, other: []string{u()}}, + {family: db.Lifecycle, id: []string{u()}, other: []string{u()}}, + {family: db.Subscription, id: []string{"sub_" + u()}, other: []string{"sub_" + u()}}, + {family: db.Invoice, id: []string{"in_" + u()}, other: []string{"in_" + u()}}, + {family: db.ProductSync, id: []string{u()}, other: []string{u()}}, + {family: db.DesiredState, id: []string{u(), "person", u()}, other: []string{u(), "person", u()}}, + } +} + +// holderOf begins a transaction on d that the test rolls back when it ends +// and returns it with its backend. +func holderOf(t *testing.T, d *sql.DB) (*sql.Tx, int) { + t.Helper() + q := dbtest.Tx(t, d) + return q.Tx(), int(q.Int(`SELECT pg_backend_pid()`)) +} + +// TestLockTakesOneKeyPerId holds that a second transaction asking for a +// family's lock on an id waits on the first, that the statement it runs +// names the family, that the key granted is hashtextextended of the family's +// key text, and that another id of the family is free. +func TestLockTakesOneKeyPerId(t *testing.T) { + d := dbtest.Open(t) + for _, row := range lockRows() { + t.Run(row.family.Name(), func(t *testing.T) { checkLockTakesOneKey(t, d, row) }) + } +} + +// checkLockTakesOneKey is one row of TestLockTakesOneKeyPerId. +func checkLockTakesOneKey(t *testing.T, d *sql.DB, row lockRow) { + t.Helper() + ctx := context.Background() + name := row.family.Name() + key, err := row.family.Key(row.id...) + if err != nil { + t.Fatalf("%s: key: %v", name, err) + } + holder, holderPID := holderOf(t, d) + if err := db.Lock(ctx, holder, row.family, row.id...); err != nil { + t.Fatalf("%s: the holder's lock: %v", name, err) + } + waiterDone := make(chan error, 1) + go func() { + tx, err := d.BeginTx(ctx, nil) + if err != nil { + waiterDone <- err + return + } + defer tx.Rollback() + waiterDone <- db.Lock(ctx, tx, row.family, row.id...) + }() + dbtest.AwaitLockWait(t, d, dbtest.Backend{BlockedBy: holderPID}) + + var query string + if err := d.QueryRowContext(ctx, `SELECT query FROM pg_stat_activity WHERE $1 = ANY(pg_blocking_pids(pid))`, holderPID).Scan(&query); err != nil { + t.Fatalf("%s: read the waiter's statement: %v", name, err) + } + first, _, _ := strings.Cut(row.family.Statement(), "\n") + if !strings.HasPrefix(query, first) { + t.Errorf("%s: the waiter's statement reads %q, want it to begin with its family's name line", name, query) + } + var granted bool + if err := d.QueryRowContext(ctx, `SELECT EXISTS ( + SELECT 1 FROM pg_locks l, (SELECT hashtextextended($2, 0) AS k) h + WHERE l.locktype = 'advisory' AND l.pid = $1 AND l.granted AND l.objsubid = 1 + AND l.classid = ((h.k >> 32) & 4294967295)::oid AND l.objid = (h.k & 4294967295)::oid)`, holderPID, key).Scan(&granted); err != nil { + t.Fatalf("%s: read pg_locks: %v", name, err) + } + if !granted { + t.Errorf("%s: no granted lock on hashtextextended(%q, 0)", name, key) + } + if row.other != nil { + side := dbtest.Tx(t, d).Tx() + if free, err := dbtest.TryLock(ctx, side, row.family, row.other...); err != nil || !free { + t.Errorf("%s: the lock of another id was held (%v)", name, err) + } + } + if err := holder.Rollback(); err != nil { + t.Fatalf("%s: roll back the holder: %v", name, err) + } + if err := <-waiterDone; err != nil { + t.Errorf("%s: the waiter returned %v", name, err) + } +} + +// TestSharedHoldersDoNotWait holds that two transactions take the +// materialization family's shared lock without waiting for each other. +func TestSharedHoldersDoNotWait(t *testing.T) { + d := dbtest.Open(t) + ctx := context.Background() + first := dbtest.Tx(t, d).Tx() + if err := db.LockShared(ctx, first, db.Materialization); err != nil { + t.Fatalf("the first shared holder: %v", err) + } + second := dbtest.Tx(t, d) + second.Exec(`SET LOCAL lock_timeout = '1s'`) + if err := db.LockShared(ctx, second.Tx(), db.Materialization); err != nil { + t.Errorf("two shared holders of materialization waited for each other: %v", err) + } +} + +// TestUnchangedFamiliesKeepTheirKeys holds that the two families whose keys +// the change kept build the keys the console took before it. +func TestUnchangedFamiliesKeepTheirKeys(t *testing.T) { + d := dbtest.Open(t) + address, err := db.DesiredState.Key(lockUUIDCapitals, "organization", lockUUIDCapitals) + if err != nil { + t.Fatal(err) + } + instance, err := db.Lifecycle.Key(lockUUIDCapitals) + if err != nil { + t.Fatal(err) + } + rows := []struct { + name string + query string + args []any + }{ + {"lifecycle", `SELECT hashtextextended('lifecycle:' || $1::uuid::text, 0) = hashtextextended($2, 0)`, []any{lockUUIDCapitals, instance}}, + {"desired-state", `SELECT hashtextextended('desired-state:' || $1::uuid::text || ':' || $2 || ':' || $3::uuid::text, 0) = hashtextextended($4, 0)`, + []any{lockUUIDCapitals, "organization", lockUUIDCapitals, address}}, + } + for _, row := range rows { + var same bool + if err := d.QueryRowContext(context.Background(), row.query, row.args...).Scan(&same); err != nil { + t.Fatalf("%s: %v", row.name, err) + } + if !same { + t.Errorf("%s: the family's key differs from the key the console took before", row.name) + } + } +} diff --git a/internal/db/locks_test.go b/internal/db/locks_test.go new file mode 100644 index 00000000..eea2a8d5 --- /dev/null +++ b/internal/db/locks_test.go @@ -0,0 +1,95 @@ +// SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Commercial +// SPDX-FileCopyrightText: 2025-2026 Christian Galo + +package db_test + +import ( + "strings" + "testing" + + "git.coopcloud.tech/wiki-cafe/member-console/internal/db" +) + +const ( + // lockUUID and lockUUIDCapitals are one UUID in two spellings. + lockUUID = "a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11" + lockUUIDCapitals = "A0EEBC99-9C0B-4EF8-BB6D-6BB9BD380A11" +) + +// keyRow is one family, the parts of an id and either the key the family +// builds or the text its error holds. +type keyRow struct { + name string + family db.Family + id []string + key string + wantErr string +} + +// checkKey builds the row's key and compares it, or its error, with what the +// row expects. +func checkKey(t *testing.T, row keyRow) { + t.Helper() + key, err := row.family.Key(row.id...) + switch { + case row.wantErr == "" && err != nil: + t.Errorf("%s: the error is %v, want the key %q", row.name, err, row.key) + case row.wantErr == "" && key != row.key: + t.Errorf("%s: the key is %q, want %q", row.name, key, row.key) + case row.wantErr != "" && err == nil: + t.Errorf("%s: the key is %q with no error, want an error containing %q", row.name, key, row.wantErr) + case row.wantErr != "" && !strings.Contains(err.Error(), row.wantErr): + t.Errorf("%s: the error is %v, want one containing %q", row.name, err, row.wantErr) + } +} + +// TestFamilyKeys holds every family's key for its id, and every id a family +// refuses. +func TestFamilyKeys(t *testing.T) { + rows := []keyRow{ + {name: "the rendezvous", family: db.Materialization, key: "materialization:"}, + {name: "the registry", family: db.DomainsRegistry, key: "domains-registry:"}, + {name: "a workspace", family: db.FedWikiWorkspace, id: []string{lockUUID}, key: "fedwiki-workspace:" + lockUUID}, + {name: "an instance id in capitals", family: db.Lifecycle, id: []string{lockUUIDCapitals}, key: "lifecycle:" + lockUUID}, + {name: "a subscription", family: db.Subscription, id: []string{"sub_123"}, key: "stripe-subscription:sub_123"}, + {name: "an invoice", family: db.Invoice, id: []string{"in_123"}, key: "stripe-invoice:in_123"}, + {name: "a product", family: db.ProductSync, id: []string{lockUUIDCapitals}, key: "stripe-product-sync:" + lockUUID}, + {name: "an address", family: db.DesiredState, id: []string{lockUUIDCapitals, "person", lockUUID}, key: "desired-state:" + lockUUID + ":person:" + lockUUID}, + {name: "an empty subscription id", family: db.Subscription, id: []string{""}, wantErr: "id part 1 is empty"}, + {name: "a malformed product id", family: db.ProductSync, id: []string{"p1"}, wantErr: "stripe-product-sync: id part 1"}, + {name: "a colon in a subscription id", family: db.Subscription, id: []string{"sub:1"}, wantErr: "id part 1 holds ':'"}, + {name: "two parts of an address", family: db.DesiredState, id: []string{lockUUID, "person"}, wantErr: "2 id parts, want 3"}, + {name: "an id given to the rendezvous", family: db.Materialization, id: []string{lockUUID}, wantErr: "1 id parts, want 0"}, + {name: "no family", family: db.Family{}, id: []string{lockUUID}, wantErr: "no family"}, + } + for _, row := range rows { + t.Run(strings.ReplaceAll(row.name, " ", "_"), func(t *testing.T) { checkKey(t, row) }) + } +} + +// TestStatementsNameTheirFamily holds that each family's two statements begin +// with a name line no other family shares, and that Families lists eight +// distinct families. +func TestStatementsNameTheirFamily(t *testing.T) { + const ( + exclusive = "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))" + shared = "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))" + ) + firstLines := map[string]string{} + names := map[string]bool{} + for _, f := range db.Families { + names[f.Name()] = true + stmt, sharedStmt := f.Statement(), f.SharedStatement() + if !strings.HasPrefix(stmt, "-- name: ") || !strings.HasSuffix(stmt, exclusive) || !strings.HasSuffix(sharedStmt, shared) { + t.Errorf("%s's statements read %q and %q, want a name line and the exclusive and shared locks", f.Name(), stmt, sharedStmt) + } + first, _, _ := strings.Cut(stmt, "\n") + if other, ok := firstLines[first]; ok { + t.Errorf("%s and %s share the name line %q", other, f.Name(), first) + } + firstLines[first] = f.Name() + } + if len(db.Families) != 8 || len(names) != 8 { + t.Errorf("Families holds %v, want eight distinct names", db.Families) + } +} diff --git a/internal/db/migrations/00040_rendezvous_family_key.sql b/internal/db/migrations/00040_rendezvous_family_key.sql new file mode 100644 index 00000000..ff07e6fb --- /dev/null +++ b/internal/db/migrations/00040_rendezvous_family_key.sql @@ -0,0 +1,80 @@ +-- SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Commercial +-- SPDX-FileCopyrightText: 2025-2026 Christian Galo + +-- +goose Up +-- core.assert_rendezvous checks the materialization family's key, +-- hashtextextended('materialization:', 0), which internal/db/locks.go builds +-- for every rendezvous. A backend holding the key this function checked +-- before, a process of the previous release, is refused. A later replacement +-- keeps `v_key` equal to the key `db.Materialization` builds. +-- +goose StatementBegin +CREATE OR REPLACE FUNCTION core.assert_rendezvous(p_exclusive BOOLEAN) RETURNS void +LANGUAGE plpgsql +SECURITY DEFINER +SET search_path = core, pg_temp +AS $$ +DECLARE + v_key BIGINT := hashtextextended('materialization:', 0); + v_held BOOLEAN; +BEGIN + SELECT EXISTS ( + SELECT 1 + FROM pg_catalog.pg_locks l + WHERE l.locktype = 'advisory' + AND l.pid = pg_backend_pid() + AND l.granted + AND l.objsubid = 1 + AND l.classid = ((v_key >> 32) & 4294967295)::oid + AND l.objid = (v_key & 4294967295)::oid + AND (NOT p_exclusive OR l.mode = 'ExclusiveLock') + ) INTO v_held; + IF NOT v_held THEN + RAISE EXCEPTION 'materialization_rendezvous_missing' + USING ERRCODE = '55000', + DETAIL = CASE WHEN p_exclusive + THEN 'the calling transaction does not hold the exclusive materialization rendezvous' + ELSE 'the calling transaction does not hold the materialization rendezvous' END; + END IF; +END; +$$; +ALTER FUNCTION core.assert_rendezvous(BOOLEAN) OWNER TO core_owner; +REVOKE EXECUTE ON FUNCTION core.assert_rendezvous(BOOLEAN) FROM PUBLIC; +GRANT EXECUTE ON FUNCTION core.assert_rendezvous(BOOLEAN) TO core_writer; +-- +goose StatementEnd + +-- +goose Down +-- The key the function checked before the materialization family. +-- +goose StatementBegin +CREATE OR REPLACE FUNCTION core.assert_rendezvous(p_exclusive BOOLEAN) RETURNS void +LANGUAGE plpgsql +SECURITY DEFINER +SET search_path = core, pg_temp +AS $$ +DECLARE + v_key BIGINT := hashtextextended('entitlement_materialization', 0); + v_held BOOLEAN; +BEGIN + SELECT EXISTS ( + SELECT 1 + FROM pg_catalog.pg_locks l + WHERE l.locktype = 'advisory' + AND l.pid = pg_backend_pid() + AND l.granted + AND l.objsubid = 1 + AND l.classid = ((v_key >> 32) & 4294967295)::oid + AND l.objid = (v_key & 4294967295)::oid + AND (NOT p_exclusive OR l.mode = 'ExclusiveLock') + ) INTO v_held; + IF NOT v_held THEN + RAISE EXCEPTION 'materialization_rendezvous_missing' + USING ERRCODE = '55000', + DETAIL = CASE WHEN p_exclusive + THEN 'the calling transaction does not hold the exclusive materialization rendezvous' + ELSE 'the calling transaction does not hold the materialization rendezvous' END; + END IF; +END; +$$; +ALTER FUNCTION core.assert_rendezvous(BOOLEAN) OWNER TO core_owner; +REVOKE EXECUTE ON FUNCTION core.assert_rendezvous(BOOLEAN) FROM PUBLIC; +GRANT EXECUTE ON FUNCTION core.assert_rendezvous(BOOLEAN) TO core_writer; +-- +goose StatementEnd diff --git a/internal/db/rendezvous_family_key_test.go b/internal/db/rendezvous_family_key_test.go new file mode 100644 index 00000000..94ab437f --- /dev/null +++ b/internal/db/rendezvous_family_key_test.go @@ -0,0 +1,103 @@ +// SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Commercial +// SPDX-FileCopyrightText: 2025-2026 Christian Galo + +package db_test + +import ( + "context" + "database/sql" + "strings" + "testing" + + "git.coopcloud.tech/wiki-cafe/member-console/internal/db" + "git.coopcloud.tech/wiki-cafe/member-console/internal/testkit/dbtest" +) + +const ( + rendezvousFamilyKeyVersion = 40 + rendezvousFamilyKeyPrevious = 39 + rendezvousFunction = "core.assert_rendezvous(boolean)" +) + +// TestRendezvousFamilyKeyMigration checks, before the migration, after it and +// after its down, which key core.assert_rendezvous accepts, that the +// function's owner and grants never change, and that down restores the +// definition from before the migration. +func TestRendezvousFamilyKeyMigration(t *testing.T) { + scratch := dbtest.Connect(t, dbtest.Scratch(t)) + core := db.BaseSources()[0] + queries := dbtest.On(t, scratch) + dbtest.UpTo(t, scratch, core, rendezvousFamilyKeyPrevious) + definition := queries.String(`SELECT pg_get_functiondef($1::regprocedure)`, rendezvousFunction) + privileges := rendezvousFunctionPrivileges(queries) + + steps := []struct { + name string + apply bool + up bool + oldKey string + newKey string + sameBody bool + }{ + {name: "before the migration", oldKey: "passed", newKey: "refused"}, + {name: "after the migration", apply: true, up: true, oldKey: "refused", newKey: "passed"}, + {name: "after its down", apply: true, oldKey: "passed", newKey: "refused", sameBody: true}, + } + for _, step := range steps { + if step.apply { + dbtest.ApplyVersion(t, scratch, core, rendezvousFamilyKeyVersion, step.up) + } + if got := rendezvousAssertion(t, scratch, func(ctx context.Context, tx *sql.Tx) error { + _, err := tx.ExecContext(ctx, sharedRendezvousBeforeFamilies) + return err + }); got != step.oldKey { + t.Errorf("%s, the assertion %s the old key, want it to be %s", step.name, got, step.oldKey) + } + if got := rendezvousAssertion(t, scratch, func(ctx context.Context, tx *sql.Tx) error { + return db.LockShared(ctx, tx, db.Materialization) + }); got != step.newKey { + t.Errorf("%s, the assertion %s the new key, want it to be %s", step.name, got, step.newKey) + } + if got := rendezvousFunctionPrivileges(queries); got != privileges { + t.Errorf("%s, core.assert_rendezvous's owner or grants read %s, want %s", step.name, got, privileges) + } + if step.sameBody { + if got := queries.String(`SELECT pg_get_functiondef($1::regprocedure)`, rendezvousFunction); got != definition { + t.Errorf("after its down, core.assert_rendezvous reads %q, want the definition before the migration", got) + } + } + } +} + +// rendezvousFunctionPrivileges is the owner and the ACL of +// core.assert_rendezvous, which CREATE OR REPLACE must keep. +func rendezvousFunctionPrivileges(queries *dbtest.Q) string { + return queries.String(`SELECT pg_get_userbyid(proowner) || ' ' || COALESCE(proacl::text, '') + FROM pg_proc WHERE oid = $1::regprocedure`, rendezvousFunction) +} + +// rendezvousAssertion takes a lock with take in a transaction of its own, +// runs core.assert_rendezvous(false) and reports whether it passed or +// refused with materialization_rendezvous_missing. Any other outcome fails +// the test. +func rendezvousAssertion(t *testing.T, database *sql.DB, take func(context.Context, *sql.Tx) error) string { + t.Helper() + ctx := context.Background() + tx, err := database.BeginTx(ctx, nil) + if err != nil { + t.Fatal(err) + } + defer tx.Rollback() + if err := take(ctx, tx); err != nil { + t.Fatalf("take the lock: %v", err) + } + _, err = tx.ExecContext(ctx, `SELECT core.assert_rendezvous(false)`) + switch { + case err == nil: + return "passed" + case strings.Contains(err.Error(), "materialization_rendezvous_missing"): + return "refused" + } + t.Fatalf("core.assert_rendezvous: %v", err) + return "" +} diff --git a/internal/db/settlement_pool_first_test.go b/internal/db/settlement_pool_first_test.go index f6488943..a9150641 100644 --- a/internal/db/settlement_pool_first_test.go +++ b/internal/db/settlement_pool_first_test.go @@ -15,6 +15,10 @@ const ( settlementPoolFirstVersion = 38 settlementPoolFirstPrevious = 37 settlementFunction = "core.settle_obligation(uuid,uuid,jsonb,text,integer)" + + // sharedRendezvousBeforeFamilies is the key core.assert_rendezvous checks + // at core versions before the materialization family. + sharedRendezvousBeforeFamilies = `SELECT pg_advisory_xact_lock_shared(hashtextextended('entitlement_materialization', 0))` ) // TestSettlementPoolFirstMigrationUpDownUp checks the restored body, the @@ -65,7 +69,7 @@ func checkRuntimeSettlement(t *testing.T, database *sql.DB, step string, failure t.Helper() queries := dbtest.Tx(t, database) defer queries.Tx().Rollback() - queries.Exec(sharedRendezvous) + queries.Exec(sharedRendezvousBeforeFamilies) person := fxPerson(t, queries.Tx()) org := fxOrg(t, queries.Tx(), person) pool := fxPool(t, queries.Tx(), org, "default") diff --git a/internal/domains/registry.go b/internal/domains/registry.go index f0f008ac..c2312dea 100644 --- a/internal/domains/registry.go +++ b/internal/domains/registry.go @@ -54,29 +54,19 @@ const ( ProbeStateError = "error" ) -// registryAllocationLock is the key for the single registry-wide -// pg_advisory_xact_lock every mutating allocation takes (design D3). -// Allocation is human-paced, so one lock for the whole namespace buys the -// subtree-disjointness invariants — which span rows the database cannot -// constrain against each other — at no practical cost. -// -// The value is ASCII "domains", chosen to keep this key space visibly -// distinct from the per-object `hashtext($1)::bigint` locks (see -// docs/database-locks.md): hashtext produces int32-range values, so a 56-bit -// constant can never collide with one. +// registryKind opens a registry allocation's transaction: the registry's +// advisory lock, the domains-registry family's (internal/db/locks.go), is +// its first statement. Allocation is human-paced, so one lock for the whole +// namespace buys the subtree-disjointness invariants, which span rows the +// database cannot constrain against each other, at no practical cost +// (design D3). // // LOCK ORDERING INVARIANT: a transaction that touches both the registry and -// entitlement-usage rows MUST take this lock FIRST — the usage row's lock -// (LockNumericUsage, taken by a recount) is always second. A transaction that -// took a usage row and then allocated, and one that allocated and then took -// the usage row, would otherwise grab the two in opposite orders and deadlock. -const registryAllocationLock int64 = 0x646f6d61696e73 - -// registryKind opens a registry allocation's transaction: the registry's -// advisory lock is its first statement, so nothing in the transaction runs -// before the lock (docs/database-locks.md, Registry). +// entitlement-usage rows MUST take this lock FIRST, as docs/database-locks.md +// orders every advisory family before rows: the usage row's lock +// (LockNumericUsage, taken by a recount) is always second. var registryKind = db.FirstRuns(func(ctx context.Context, tx *sql.Tx) error { - if _, err := tx.ExecContext(ctx, `SELECT pg_advisory_xact_lock($1)`, registryAllocationLock); err != nil { + if err := db.Lock(ctx, tx, db.DomainsRegistry); err != nil { return fmt.Errorf("domains: registry advisory lock: %w", err) } return nil @@ -476,8 +466,8 @@ func (t *Tx) PlacementByResource(ctx context.Context, provider, resourceRef stri // (released on commit or rollback), opened through the runner with // registryKind. Returning an error rolls the whole composite back; returning // nil commits. fn's error, a failed begin and a failed commit come back as -// the runner returns them. See registryAllocationLock for the lock-ordering -// invariant against entitlement usage rows. +// the runner returns them. See registryKind for the lock-ordering invariant +// against entitlement usage rows. func (r *Registry) WithLock(ctx context.Context, fn func(context.Context, *Tx) error) error { if r.db == nil { return ErrNoDatabase diff --git a/internal/domains/registry_db_test.go b/internal/domains/registry_db_test.go index 77e65a55..04ac6e83 100644 --- a/internal/domains/registry_db_test.go +++ b/internal/domains/registry_db_test.go @@ -759,7 +759,7 @@ func TestCheckAvailabilityOutsideEveryClaim(t *testing.T) { // registryLockStatement is the statement the registry's kind runs first, // named as dbtest.Counted names a raw statement: its first line. -const registryLockStatement = `SELECT pg_advisory_xact_lock($1)` +const registryLockStatement = `LockDomainsRegistry` // TestRegistryLockIsTheFirstStatement proves a registry transaction runs the // registry lock before any statement of its composite: the statements of one diff --git a/internal/entitlements/plan_act_drain_db_test.go b/internal/entitlements/plan_act_drain_db_test.go index 729b4db4..4b96c172 100644 --- a/internal/entitlements/plan_act_drain_db_test.go +++ b/internal/entitlements/plan_act_drain_db_test.go @@ -836,7 +836,7 @@ func TestDefaultChangeWaitsForAMoveInFlight(t *testing.T) { }) changed <- err }() - dbtest.AwaitLockWait(t, f.w.DB, dbtest.Backend{Query: "pg_advisory_xact_lock(hashtextextended", Waiters: 1}) + dbtest.AwaitLockWait(t, f.w.DB, dbtest.Backend{Query: db.Materialization.Statement(), Waiters: 1}) if got := f.q.String(`SELECT default_plan_ladder_id::text FROM core.org_types WHERE org_type = $1`, f.orgType); got != f.next.ID { t.Errorf("the default is %s while a move is in flight, want the T0 to have saved nothing (Next %s)", got, f.next.ID) } diff --git a/internal/entitlements/plan_act_queries_db_test.go b/internal/entitlements/plan_act_queries_db_test.go index 056b77ed..9d558470 100644 --- a/internal/entitlements/plan_act_queries_db_test.go +++ b/internal/entitlements/plan_act_queries_db_test.go @@ -10,6 +10,7 @@ package entitlements_test // exclusive rendezvous the act's functions assert. import ( + "context" "database/sql" "errors" "maps" @@ -48,7 +49,9 @@ func newPlanActRows(t *testing.T) planActRows { // when a test needs the rows visible to another connection. func newPlanActRowsOn(t *testing.T, q *dbtest.Q) planActRows { t.Helper() - q.Exec(`SELECT pg_advisory_xact_lock(hashtextextended('entitlement_materialization', 0))`) + if err := db.Lock(context.Background(), q.Tx(), db.Materialization); err != nil { + t.Fatalf("take the rendezvous: %v", err) + } user := q.ID(`INSERT INTO core.users (oidc_subject) VALUES ($1) RETURNING user_id`, dbtest.Name("pa-subject-")) person := q.ID(`INSERT INTO core.persons (user_id, display_name, primary_email) VALUES ($1, 'Plan Act', $2) RETURNING person_id`, user, dbtest.Name("pa-")+"@example.test") @@ -366,7 +369,9 @@ func TestLockPlanActObligationSurvivesARequeueItWaitedOn(t *testing.T) { } defer retrying.Rollback() rq := dbtest.On(t, retrying) - rq.Exec(`SELECT pg_advisory_xact_lock(hashtextextended('entitlement_materialization', 0))`) + if err := db.Lock(context.Background(), retrying, db.Materialization); err != nil { + t.Fatalf("take the rendezvous: %v", err) + } retry := planActRows{t: t, q: rq, tx: retrying, e: entitlements.New(retrying), person: r.person}.retry(act) if n, err := entitlements.New(retrying).RequeuePlanActObligations(ctx, retry); err != nil || n != 1 { t.Fatalf("requeue: %d, %v", n, err) diff --git a/internal/entitlements/subscription_delivery_db_test.go b/internal/entitlements/subscription_delivery_db_test.go index b0cbe640..f750e9ab 100644 --- a/internal/entitlements/subscription_delivery_db_test.go +++ b/internal/entitlements/subscription_delivery_db_test.go @@ -23,6 +23,7 @@ import ( "github.com/google/uuid" + "git.coopcloud.tech/wiki-cafe/member-console/internal/db" "git.coopcloud.tech/wiki-cafe/member-console/internal/entitlements" "git.coopcloud.tech/wiki-cafe/member-console/internal/testkit/dbtest" ) @@ -496,7 +497,7 @@ func TestDeliverDisplaced_WaitsForThePoolRow(t *testing.T) { t.Fatal(err) } defer subLock.Rollback() - if _, err := subLock.ExecContext(ctx, `SELECT pg_advisory_xact_lock(hashtext($1)::bigint)`, fx.stripeID); err != nil { + if err := db.Lock(ctx, subLock, db.Subscription, fx.stripeID); err != nil { t.Fatalf("take S's advisory lock: %v", err) } if _, err := subLock.ExecContext(ctx, `UPDATE core.subscriptions SET status = status WHERE subscription_id = $1`, fx.sub); err != nil { diff --git a/internal/entitlements/subscription_resume_db_test.go b/internal/entitlements/subscription_resume_db_test.go index 01b16231..f5fa47a8 100644 --- a/internal/entitlements/subscription_resume_db_test.go +++ b/internal/entitlements/subscription_resume_db_test.go @@ -20,6 +20,7 @@ import ( "time" "git.coopcloud.tech/wiki-cafe/member-console/internal/billing" + "git.coopcloud.tech/wiki-cafe/member-console/internal/db" "git.coopcloud.tech/wiki-cafe/member-console/internal/entitlements" "git.coopcloud.tech/wiki-cafe/member-console/internal/testkit/dbtest" "git.coopcloud.tech/wiki-cafe/member-console/internal/testkit/world" @@ -341,7 +342,7 @@ func checkEndingActFirst(t *testing.T, name string, end func(f *resumeFixture, t t.Fatalf("begin the subscription's lock holder: %v", err) } defer holder.Rollback() - if _, err := holder.ExecContext(f.w.Ctx, `SELECT pg_advisory_xact_lock(hashtext($1)::bigint)`, "sub_"+f.sub.ID); err != nil { + if err := db.Lock(f.w.Ctx, holder, db.Subscription, "sub_"+f.sub.ID); err != nil { t.Fatalf("take the subscription's advisory lock: %v", err) } if _, err := holder.ExecContext(f.w.Ctx, `UPDATE core.subscriptions SET status = status WHERE subscription_id = $1`, f.sub.ID); err != nil { diff --git a/internal/entitlements/testdata/materialize/pool.golden b/internal/entitlements/testdata/materialize/pool.golden index 90ec6723..ccddc041 100644 --- a/internal/entitlements/testdata/materialize/pool.golden +++ b/internal/entitlements/testdata/materialize/pool.golden @@ -1,7 +1,7 @@ statements: begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) +LockMaterializationShared GetActivePoolProvisionsByPoolID GetActiveRulesBySetID SELECT core.assert_rendezvous(false) diff --git a/internal/entitlements/testdata/rule_change/atomic.golden b/internal/entitlements/testdata/rule_change/atomic.golden index 70f4009a..819e88a9 100644 --- a/internal/entitlements/testdata/rule_change/atomic.golden +++ b/internal/entitlements/testdata/rule_change/atomic.golden @@ -1,7 +1,7 @@ statements: begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock(hashtextextended($1, 0)) +LockMaterialization LockEntitlementSet GetEntitlementSetRule GetEntitlementSetRule diff --git a/internal/entitlements/testdata/rule_change/deferred.golden b/internal/entitlements/testdata/rule_change/deferred.golden index 3e299b0f..e05242a7 100644 --- a/internal/entitlements/testdata/rule_change/deferred.golden +++ b/internal/entitlements/testdata/rule_change/deferred.golden @@ -1,7 +1,7 @@ statements: begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock(hashtextextended($1, 0)) +LockMaterialization LockEntitlementSet GetEntitlementSetRule GetEntitlementSetRule @@ -18,7 +18,7 @@ ListPendingObligations GetObligationStatus begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) +LockMaterializationShared GetObligationStatus GetObligationStatus GetObligationStatus @@ -63,7 +63,7 @@ commit GetObligationStatus begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) +LockMaterializationShared GetObligationStatus GetObligationStatus GetObligationStatus diff --git a/internal/entitlements/tx.go b/internal/entitlements/tx.go index 3a040fb5..49e2c0a3 100644 --- a/internal/entitlements/tx.go +++ b/internal/entitlements/tx.go @@ -17,13 +17,12 @@ import ( ) // The materialization rendezvous (Decision 144): one transaction-scoped -// advisory lock that every transaction materializing a pool holds in shared -// mode and every rule change and operator tier change holds in exclusive -// mode, taken as the transaction's first statement so no holder ever waits -// on something a would-be holder already holds. The demonstration seed adds -// its ladder's tiers on the plain handle, outside it. The key is -// hashtextextended of this name. -const rendezvousName = "entitlement_materialization" +// advisory lock, the materialization family's (internal/db/locks.go), that +// every transaction materializing a pool holds in shared mode and every rule +// change and operator tier change holds in exclusive mode, taken as the +// transaction's first statement so no holder ever waits on something a +// would-be holder already holds. The demonstration seed adds its ladder's +// tiers on the plain handle, outside it. // rendezvousLockTimeout bounds every wait on the rendezvous; a wait past it // surfaces as SQLSTATE 55P03 and renders as the refusal copy. It is the lock @@ -54,7 +53,7 @@ var Materializing = MaterializingWithin(rendezvousLockTimeout, 0) // transaction timeout of zero sets none. Materializing is the case of 5 s and // none, so there is one opener for each side of the rendezvous. func MaterializingWithin(lockTimeout, txTimeout time.Duration) db.Kind { - return db.FirstRuns(rendezvous("pg_advisory_xact_lock_shared", lockTimeout, txTimeout)) + return db.FirstRuns(rendezvous(db.LockShared, lockTimeout, txTimeout)) } // RuleChange opens a transaction that changes a rule or carries an @@ -65,7 +64,7 @@ func MaterializingWithin(lockTimeout, txTimeout time.Duration) db.Kind { // and a product's shape in separate statements, so a tier change that // committed between them would have it classify a tier change against two // orders of the ladder (#213). -var RuleChange = db.FirstRuns(rendezvous("pg_advisory_xact_lock", rendezvousLockTimeout, 0)) +var RuleChange = db.FirstRuns(rendezvous(db.Lock, rendezvousLockTimeout, 0)) // BeginMaterializing opens a Materializing transaction on b. func BeginMaterializing(ctx context.Context, b db.Beginner) (*sql.Tx, error) { @@ -78,9 +77,9 @@ func BeginRuleChange(ctx context.Context, b db.Beginner) (*sql.Tx, error) { } // rendezvous is the first statements of a transaction that takes the -// materialization rendezvous with lockFn: the lock timeout, the transaction -// timeout when there is one, then the lock. -func rendezvous(lockFn string, lockTimeout, txTimeout time.Duration) func(context.Context, *sql.Tx) error { +// materialization rendezvous with lock, db.Lock or db.LockShared: the lock +// timeout, the transaction timeout when there is one, then the lock. +func rendezvous(lock func(context.Context, *sql.Tx, db.Family, ...string) error, lockTimeout, txTimeout time.Duration) func(context.Context, *sql.Tx) error { return func(ctx context.Context, tx *sql.Tx) error { if txTimeout > 0 { if _, err := tx.ExecContext(ctx, "SET LOCAL transaction_timeout = '"+milliseconds(txTimeout)+"'"); err != nil { @@ -90,7 +89,7 @@ func rendezvous(lockFn string, lockTimeout, txTimeout time.Duration) func(contex if _, err := tx.ExecContext(ctx, "SET LOCAL lock_timeout = '"+milliseconds(lockTimeout)+"'"); err != nil { return fmt.Errorf("set lock_timeout: %w", err) } - if _, err := tx.ExecContext(ctx, "SELECT "+lockFn+"(hashtextextended($1, 0))", rendezvousName); err != nil { + if err := lock(ctx, tx, db.Materialization); err != nil { return fmt.Errorf("take materialization rendezvous: %w", err) } return nil diff --git a/internal/fulfillment/reconcile.go b/internal/fulfillment/reconcile.go index 39e6c4af..035e9754 100644 --- a/internal/fulfillment/reconcile.go +++ b/internal/fulfillment/reconcile.go @@ -119,9 +119,9 @@ type afterCommit struct { // reconcileInTx runs the reconcile's transaction on the fetched subscription // and commits it. Without applyCollection, a collection action the rule asks // for writes the outbox entry in the transaction. -func reconcileInTx(ctx context.Context, db *sql.DB, logger *slog.Logger, sub *stripe.Subscription, stripeSubscriptionID, reason string, applyCollection bool) (afterCommit, error) { +func reconcileInTx(ctx context.Context, database *sql.DB, logger *slog.Logger, sub *stripe.Subscription, stripeSubscriptionID, reason string, applyCollection bool) (afterCommit, error) { var plan afterCommit - tx, err := entitlements.BeginMaterializing(ctx, db) + tx, err := entitlements.BeginMaterializing(ctx, database) if err != nil { return plan, fmt.Errorf("reconcile: begin tx: %w", err) } @@ -129,8 +129,7 @@ func reconcileInTx(ctx context.Context, db *sql.DB, logger *slog.Logger, sub *st // 1.2 Serialize concurrent reconciles (eager return + webhook) of the same // subscription so they converge instead of racing the GiST exclusion. - if _, err := tx.ExecContext(ctx, - `SELECT pg_advisory_xact_lock(hashtext($1)::bigint)`, stripeSubscriptionID); err != nil { + if err := db.Lock(ctx, tx, db.Subscription, stripeSubscriptionID); err != nil { return plan, fmt.Errorf("reconcile: advisory lock: %w", err) } @@ -1329,8 +1328,7 @@ func supersedeReleasedDowngrades(ctx context.Context, database *sql.DB, stripeSu return fmt.Errorf("reconcile: begin tx after releasing the schedule: %w", err) } defer tx.Rollback() - if _, err := tx.ExecContext(ctx, - `SELECT pg_advisory_xact_lock(hashtext($1)::bigint)`, stripeSubscriptionID); err != nil { + if err := db.Lock(ctx, tx, db.Subscription, stripeSubscriptionID); err != nil { return fmt.Errorf("reconcile: advisory lock after releasing the schedule: %w", err) } billingQ := billing.New(tx) diff --git a/internal/fulfillment/release_schedule_after_commit_test.go b/internal/fulfillment/release_schedule_after_commit_test.go index 1f6e6de3..2090926e 100644 --- a/internal/fulfillment/release_schedule_after_commit_test.go +++ b/internal/fulfillment/release_schedule_after_commit_test.go @@ -19,6 +19,7 @@ import ( stripe "github.com/stripe/stripe-go/v81" + "git.coopcloud.tech/wiki-cafe/member-console/internal/db" "git.coopcloud.tech/wiki-cafe/member-console/internal/entitlements" "git.coopcloud.tech/wiki-cafe/member-console/internal/fulfillment" "git.coopcloud.tech/wiki-cafe/member-console/internal/stripetest" @@ -246,8 +247,8 @@ func trackSide(t *testing.T, side *sql.Tx) (*sql.Tx, int) { // holdSubscriptionLock takes the reconcile's subscription lock on side // without waiting, and fails when another transaction holds it. func holdSubscriptionLock(ctx context.Context, side *sql.Tx, stripeSubID string) error { - var free bool - if err := side.QueryRowContext(ctx, `SELECT pg_try_advisory_xact_lock(hashtext($1)::bigint)`, stripeSubID).Scan(&free); err != nil { + free, err := dbtest.TryLock(ctx, side, db.Subscription, stripeSubID) + if err != nil { return fmt.Errorf("try the subscription lock: %w", err) } if !free { diff --git a/internal/fulfillment/testdata/reconcile_trace/active_again.golden b/internal/fulfillment/testdata/reconcile_trace/active_again.golden index e9b1b353..37bf796c 100644 --- a/internal/fulfillment/testdata/reconcile_trace/active_again.golden +++ b/internal/fulfillment/testdata/reconcile_trace/active_again.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/reconcile_trace/addon_added.golden b/internal/fulfillment/testdata/reconcile_trace/addon_added.golden index 1666c7f2..f98680f3 100644 --- a/internal/fulfillment/testdata/reconcile_trace/addon_added.golden +++ b/internal/fulfillment/testdata/reconcile_trace/addon_added.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/reconcile_trace/addon_removed.golden b/internal/fulfillment/testdata/reconcile_trace/addon_removed.golden index bdfc6be9..ba5ecce1 100644 --- a/internal/fulfillment/testdata/reconcile_trace/addon_removed.golden +++ b/internal/fulfillment/testdata/reconcile_trace/addon_removed.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/reconcile_trace/canceled_replay.golden b/internal/fulfillment/testdata/reconcile_trace/canceled_replay.golden index ac391735..e167eec4 100644 --- a/internal/fulfillment/testdata/reconcile_trace/canceled_replay.golden +++ b/internal/fulfillment/testdata/reconcile_trace/canceled_replay.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/reconcile_trace/canceled_resumes_grant.golden b/internal/fulfillment/testdata/reconcile_trace/canceled_resumes_grant.golden index 0b44475d..06859329 100644 --- a/internal/fulfillment/testdata/reconcile_trace/canceled_resumes_grant.golden +++ b/internal/fulfillment/testdata/reconcile_trace/canceled_resumes_grant.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/reconcile_trace/canceled_to_default.golden b/internal/fulfillment/testdata/reconcile_trace/canceled_to_default.golden index 45544ee7..02f7d7e5 100644 --- a/internal/fulfillment/testdata/reconcile_trace/canceled_to_default.golden +++ b/internal/fulfillment/testdata/reconcile_trace/canceled_to_default.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/reconcile_trace/first_sight.golden b/internal/fulfillment/testdata/reconcile_trace/first_sight.golden index 0a4d54f0..ab13da6d 100644 --- a/internal/fulfillment/testdata/reconcile_trace/first_sight.golden +++ b/internal/fulfillment/testdata/reconcile_trace/first_sight.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/reconcile_trace/grant_over_subscription.golden b/internal/fulfillment/testdata/reconcile_trace/grant_over_subscription.golden index df21073e..379f7bc8 100644 --- a/internal/fulfillment/testdata/reconcile_trace/grant_over_subscription.golden +++ b/internal/fulfillment/testdata/reconcile_trace/grant_over_subscription.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/reconcile_trace/grant_over_suspended_recovery.golden b/internal/fulfillment/testdata/reconcile_trace/grant_over_suspended_recovery.golden index 1fe39463..081e299f 100644 --- a/internal/fulfillment/testdata/reconcile_trace/grant_over_suspended_recovery.golden +++ b/internal/fulfillment/testdata/reconcile_trace/grant_over_suspended_recovery.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/reconcile_trace/incomplete.golden b/internal/fulfillment/testdata/reconcile_trace/incomplete.golden index 90eb041f..f540ffa7 100644 --- a/internal/fulfillment/testdata/reconcile_trace/incomplete.golden +++ b/internal/fulfillment/testdata/reconcile_trace/incomplete.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/reconcile_trace/item_replaced.golden b/internal/fulfillment/testdata/reconcile_trace/item_replaced.golden index 3cfaa36f..a3b0c51b 100644 --- a/internal/fulfillment/testdata/reconcile_trace/item_replaced.golden +++ b/internal/fulfillment/testdata/reconcile_trace/item_replaced.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/reconcile_trace/item_without_id.golden b/internal/fulfillment/testdata/reconcile_trace/item_without_id.golden index 89bd1924..59e9c890 100644 --- a/internal/fulfillment/testdata/reconcile_trace/item_without_id.golden +++ b/internal/fulfillment/testdata/reconcile_trace/item_without_id.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/reconcile_trace/no_customer.golden b/internal/fulfillment/testdata/reconcile_trace/no_customer.golden index 62c7f30e..149649a4 100644 --- a/internal/fulfillment/testdata/reconcile_trace/no_customer.golden +++ b/internal/fulfillment/testdata/reconcile_trace/no_customer.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription rollback == logs == rows diff --git a/internal/fulfillment/testdata/reconcile_trace/past_due.golden b/internal/fulfillment/testdata/reconcile_trace/past_due.golden index d2925d8e..85a396cb 100644 --- a/internal/fulfillment/testdata/reconcile_trace/past_due.golden +++ b/internal/fulfillment/testdata/reconcile_trace/past_due.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/reconcile_trace/price_swap.golden b/internal/fulfillment/testdata/reconcile_trace/price_swap.golden index 348890b8..fc0ad474 100644 --- a/internal/fulfillment/testdata/reconcile_trace/price_swap.golden +++ b/internal/fulfillment/testdata/reconcile_trace/price_swap.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/reconcile_trace/product_without_set.golden b/internal/fulfillment/testdata/reconcile_trace/product_without_set.golden index e0da6fef..ca3231c5 100644 --- a/internal/fulfillment/testdata/reconcile_trace/product_without_set.golden +++ b/internal/fulfillment/testdata/reconcile_trace/product_without_set.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/reconcile_trace/quantity.golden b/internal/fulfillment/testdata/reconcile_trace/quantity.golden index 4918363e..6d45d830 100644 --- a/internal/fulfillment/testdata/reconcile_trace/quantity.golden +++ b/internal/fulfillment/testdata/reconcile_trace/quantity.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/reconcile_trace/quantity_zero.golden b/internal/fulfillment/testdata/reconcile_trace/quantity_zero.golden index 0a4d54f0..ab13da6d 100644 --- a/internal/fulfillment/testdata/reconcile_trace/quantity_zero.golden +++ b/internal/fulfillment/testdata/reconcile_trace/quantity_zero.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/reconcile_trace/replay.golden b/internal/fulfillment/testdata/reconcile_trace/replay.golden index b4d46fe3..1491d6d4 100644 --- a/internal/fulfillment/testdata/reconcile_trace/replay.golden +++ b/internal/fulfillment/testdata/reconcile_trace/replay.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/reconcile_trace/second_subscription_over_first.golden b/internal/fulfillment/testdata/reconcile_trace/second_subscription_over_first.golden index 3ec1ce5e..2851764b 100644 --- a/internal/fulfillment/testdata/reconcile_trace/second_subscription_over_first.golden +++ b/internal/fulfillment/testdata/reconcile_trace/second_subscription_over_first.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/reconcile_trace/three_ladders_first_sight.golden b/internal/fulfillment/testdata/reconcile_trace/three_ladders_first_sight.golden index eb56b2bb..505ba181 100644 --- a/internal/fulfillment/testdata/reconcile_trace/three_ladders_first_sight.golden +++ b/internal/fulfillment/testdata/reconcile_trace/three_ladders_first_sight.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/reconcile_trace/two_tiers_one_ladder.golden b/internal/fulfillment/testdata/reconcile_trace/two_tiers_one_ladder.golden index 9489359c..7ad118f7 100644 --- a/internal/fulfillment/testdata/reconcile_trace/two_tiers_one_ladder.golden +++ b/internal/fulfillment/testdata/reconcile_trace/two_tiers_one_ladder.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/reconcile_trace/unmapped_price.golden b/internal/fulfillment/testdata/reconcile_trace/unmapped_price.golden index b5488a32..d3eb1509 100644 --- a/internal/fulfillment/testdata/reconcile_trace/unmapped_price.golden +++ b/internal/fulfillment/testdata/reconcile_trace/unmapped_price.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/scheduled_change_trace/cancellation_cleared.golden b/internal/fulfillment/testdata/scheduled_change_trace/cancellation_cleared.golden index 0d48ad9b..8ca5890b 100644 --- a/internal/fulfillment/testdata/scheduled_change_trace/cancellation_cleared.golden +++ b/internal/fulfillment/testdata/scheduled_change_trace/cancellation_cleared.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/scheduled_change_trace/dashboard_schedule.golden b/internal/fulfillment/testdata/scheduled_change_trace/dashboard_schedule.golden index 9edec66b..af479312 100644 --- a/internal/fulfillment/testdata/scheduled_change_trace/dashboard_schedule.golden +++ b/internal/fulfillment/testdata/scheduled_change_trace/dashboard_schedule.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/scheduled_change_trace/downgrade_landed.golden b/internal/fulfillment/testdata/scheduled_change_trace/downgrade_landed.golden index 6c65741e..6c778449 100644 --- a/internal/fulfillment/testdata/scheduled_change_trace/downgrade_landed.golden +++ b/internal/fulfillment/testdata/scheduled_change_trace/downgrade_landed.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/scheduled_change_trace/downgrade_lands_while_displaced.golden b/internal/fulfillment/testdata/scheduled_change_trace/downgrade_lands_while_displaced.golden index 5efa6031..83d759c0 100644 --- a/internal/fulfillment/testdata/scheduled_change_trace/downgrade_lands_while_displaced.golden +++ b/internal/fulfillment/testdata/scheduled_change_trace/downgrade_lands_while_displaced.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/scheduled_change_trace/downgrade_pending.golden b/internal/fulfillment/testdata/scheduled_change_trace/downgrade_pending.golden index 2066946c..c46e36d6 100644 --- a/internal/fulfillment/testdata/scheduled_change_trace/downgrade_pending.golden +++ b/internal/fulfillment/testdata/scheduled_change_trace/downgrade_pending.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/scheduled_change_trace/later_upgrade.golden b/internal/fulfillment/testdata/scheduled_change_trace/later_upgrade.golden index bda5d4ff..1493b2e1 100644 --- a/internal/fulfillment/testdata/scheduled_change_trace/later_upgrade.golden +++ b/internal/fulfillment/testdata/scheduled_change_trace/later_upgrade.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID @@ -35,7 +35,7 @@ commit stripe POST /v1/subscription_schedules/sub_sched_mock/release probe: the subscription lock is free begin -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockSubscription SupersedeScheduledChange commit == logs diff --git a/internal/fulfillment/testdata/scheduled_change_trace/release_refused.golden b/internal/fulfillment/testdata/scheduled_change_trace/release_refused.golden index 8523b6ec..11c3d22e 100644 --- a/internal/fulfillment/testdata/scheduled_change_trace/release_refused.golden +++ b/internal/fulfillment/testdata/scheduled_change_trace/release_refused.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/scheduled_change_trace/schedule_released_outside.golden b/internal/fulfillment/testdata/scheduled_change_trace/schedule_released_outside.golden index 2635a177..b4e98677 100644 --- a/internal/fulfillment/testdata/scheduled_change_trace/schedule_released_outside.golden +++ b/internal/fulfillment/testdata/scheduled_change_trace/schedule_released_outside.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/scheduled_change_trace/term_boundary_switch_pending.golden b/internal/fulfillment/testdata/scheduled_change_trace/term_boundary_switch_pending.golden index d650a61c..b1e83ab3 100644 --- a/internal/fulfillment/testdata/scheduled_change_trace/term_boundary_switch_pending.golden +++ b/internal/fulfillment/testdata/scheduled_change_trace/term_boundary_switch_pending.golden @@ -5,8 +5,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/sweep_trace/already_canceled.golden b/internal/fulfillment/testdata/sweep_trace/already_canceled.golden index 54ec6a9c..8b9b8752 100644 --- a/internal/fulfillment/testdata/sweep_trace/already_canceled.golden +++ b/internal/fulfillment/testdata/sweep_trace/already_canceled.golden @@ -9,8 +9,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/sweep_trace/already_ended.golden b/internal/fulfillment/testdata/sweep_trace/already_ended.golden index 022dd522..f03f883d 100644 --- a/internal/fulfillment/testdata/sweep_trace/already_ended.golden +++ b/internal/fulfillment/testdata/sweep_trace/already_ended.golden @@ -9,8 +9,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/sweep_trace/cancellation_refused.golden b/internal/fulfillment/testdata/sweep_trace/cancellation_refused.golden index 277e4ed5..6c97036f 100644 --- a/internal/fulfillment/testdata/sweep_trace/cancellation_refused.golden +++ b/internal/fulfillment/testdata/sweep_trace/cancellation_refused.golden @@ -10,8 +10,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/sweep_trace/due_cancellation.golden b/internal/fulfillment/testdata/sweep_trace/due_cancellation.golden index 5aef3b03..4c2eb621 100644 --- a/internal/fulfillment/testdata/sweep_trace/due_cancellation.golden +++ b/internal/fulfillment/testdata/sweep_trace/due_cancellation.golden @@ -10,8 +10,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/sweep_trace/lost_claim.golden b/internal/fulfillment/testdata/sweep_trace/lost_claim.golden index 8fdcc6d4..2ba5874b 100644 --- a/internal/fulfillment/testdata/sweep_trace/lost_claim.golden +++ b/internal/fulfillment/testdata/sweep_trace/lost_claim.golden @@ -10,8 +10,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/sweep_trace/period_end_landed.golden b/internal/fulfillment/testdata/sweep_trace/period_end_landed.golden index 78f28351..a2d2f3af 100644 --- a/internal/fulfillment/testdata/sweep_trace/period_end_landed.golden +++ b/internal/fulfillment/testdata/sweep_trace/period_end_landed.golden @@ -10,8 +10,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/sweep_trace/stale_downgrade.golden b/internal/fulfillment/testdata/sweep_trace/stale_downgrade.golden index 09e6de88..fc521cb4 100644 --- a/internal/fulfillment/testdata/sweep_trace/stale_downgrade.golden +++ b/internal/fulfillment/testdata/sweep_trace/stale_downgrade.golden @@ -10,8 +10,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/testdata/sweep_trace/term_boundary_switch.golden b/internal/fulfillment/testdata/sweep_trace/term_boundary_switch.golden index 56fd34ae..6cc72a02 100644 --- a/internal/fulfillment/testdata/sweep_trace/term_boundary_switch.golden +++ b/internal/fulfillment/testdata/sweep_trace/term_boundary_switch.golden @@ -15,8 +15,8 @@ GetSubscriptionMappingByStripeID stripe GET /v1/subscriptions/ expand[0]=items.data.price&expand[1]=schedule begin SET LOCAL lock_timeout = '5s' -SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0)) -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockMaterializationShared +LockSubscription GetCustomerMappingByStripeCustomerID GetBillingAccountByID GetDefaultPoolByOrgID diff --git a/internal/fulfillment/trace_db_test.go b/internal/fulfillment/trace_db_test.go index d5a3ddf3..281d00ee 100644 --- a/internal/fulfillment/trace_db_test.go +++ b/internal/fulfillment/trace_db_test.go @@ -228,8 +228,8 @@ func (e *traceEnv) probeSubscriptionLock() string { e.t.Fatalf("probe: begin: %v", err) } defer func() { _ = tx.Rollback() }() - var free bool - if err := tx.QueryRowContext(e.ctx, `SELECT pg_try_advisory_xact_lock(hashtext($1)::bigint)`, e.subID).Scan(&free); err != nil { + free, err := dbtest.TryLock(e.ctx, tx, db.Subscription, e.subID) + if err != nil { e.t.Fatalf("probe: try the lock: %v", err) } if free { diff --git a/internal/integration/delivery_queries_db_test.go b/internal/integration/delivery_queries_db_test.go index 7f50f6c2..5c4161e1 100644 --- a/internal/integration/delivery_queries_db_test.go +++ b/internal/integration/delivery_queries_db_test.go @@ -20,6 +20,7 @@ import ( "github.com/google/uuid" + "git.coopcloud.tech/wiki-cafe/member-console/internal/db" "git.coopcloud.tech/wiki-cafe/member-console/internal/integration" "git.coopcloud.tech/wiki-cafe/member-console/internal/testkit/dbtest" ) @@ -468,8 +469,7 @@ func TestDesiredStateRecordQueries_DB(t *testing.T) { pool := d.pool(org) d.Exec(`INSERT INTO core.numeric_entitlements (pool_id, resource_key, entitlement_type, resource_limit) VALUES ($1, $2, 'limit', 3)`, pool, r.sites) - if err := d.q.LockDesiredStateAddress(d.ctx, integration.LockDesiredStateAddressParams{ - ConnectionID: r.conn, RecipientKind: "organization", RecipientID: org}); err != nil { + if err := db.Lock(d.ctx, d.tx, db.DesiredState, r.conn, "organization", org); err != nil { t.Fatal(err) } addr := integration.LockDesiredStateRecordParams{ConnectionID: r.conn, OrgID: nullUUID(org)} @@ -665,7 +665,7 @@ func newLRQ(t *testing.T) *lrq { // insert takes the instance's lock and inserts a request at generation g. func (l *lrq) insert(g int64, op, key, params string) (string, error) { l.t.Helper() - if err := l.q.LockLifecycleInstance(l.ctx, l.instance); err != nil { + if err := db.Lock(l.ctx, l.tx, db.Lifecycle, l.instance); err != nil { l.t.Fatal(err) } return l.q.InsertLifecycleRequest(l.ctx, integration.InsertLifecycleRequestParams{ diff --git a/internal/integration/desired_state.sql.go b/internal/integration/desired_state.sql.go index f358c771..e5cdf195 100644 --- a/internal/integration/desired_state.sql.go +++ b/internal/integration/desired_state.sql.go @@ -836,35 +836,8 @@ func (q *Queries) ListLaggingDesiredStateRecords(ctx context.Context, arg ListLa return items, nil } -const lockDesiredStateAddress = `-- name: LockDesiredStateAddress :exec - -SELECT pg_advisory_xact_lock(hashtextextended( - 'desired-state:' || $1::uuid::text - || ':' || $2::text - || ':' || $3::uuid::text, 0)) -` - -type LockDesiredStateAddressParams struct { - ConnectionID string `json:"connection_id"` - RecipientKind string `json:"recipient_kind"` - RecipientID string `json:"recipient_id"` -} - -// --------------------------------------------------------------------------- -// Recomputing one record (design D9), in one READ COMMITTED transaction, in -// this order: LockDesiredStateAddress, LockDesiredStateRecord, then -// DesiredStateContent in a new statement, then UpdateDesiredStateRecord or -// InsertDesiredStateRecord. Every transaction that writes a record's content -// or generation, or a projection, takes the address lock first. -// --------------------------------------------------------------------------- -// The serialization point for one (connection, recipient) address, including -// one that has no record yet. recipient_kind is 'person' or 'organization'. -func (q *Queries) LockDesiredStateAddress(ctx context.Context, arg LockDesiredStateAddressParams) error { - _, err := q.db.ExecContext(ctx, lockDesiredStateAddress, arg.ConnectionID, arg.RecipientKind, arg.RecipientID) - return err -} - const lockDesiredStateRecord = `-- name: LockDesiredStateRecord :one + SELECT record_id, connection_id, person_id, org_id, content, content_hash, generation, generation_at, last_dispatched_at, last_push_error, last_push_failed_at, created_at, updated_at FROM core.desired_state_records r WHERE (r.connection_id = $1::uuid AND r.person_id = $2::uuid) OR (r.connection_id = $1::uuid AND r.org_id = $3::uuid) @@ -877,6 +850,14 @@ type LockDesiredStateRecordParams struct { OrgID uuid.NullUUID `json:"org_id"` } +// --------------------------------------------------------------------------- +// Recomputing one record (design D9), in one READ COMMITTED transaction, in +// this order: the desired-state family's lock (db.Lock), +// LockDesiredStateRecord, then DesiredStateContent in a new statement, then +// UpdateDesiredStateRecord or InsertDesiredStateRecord. Every transaction that +// writes a record's content or generation, or a projection, takes the address +// lock first. +// --------------------------------------------------------------------------- // The record at one address, locked; exactly one of person_id and org_id is // set. sql.ErrNoRows when the address has no record. func (q *Queries) LockDesiredStateRecord(ctx context.Context, arg LockDesiredStateRecordParams) (DesiredStateRecord, error) { diff --git a/internal/integration/export_test.go b/internal/integration/export_test.go index 9b26c055..47756339 100644 --- a/internal/integration/export_test.go +++ b/internal/integration/export_test.go @@ -25,3 +25,8 @@ func (k LifecycleAnswerKind) String() string { return fmt.Sprintf("LifecycleAnswerKind(%d)", int(k)) } } + +// SetAfterInstanceLock sets a hook that runs inside a recording right after +// its instance lock is granted, and clears it when fn is nil; a test that +// sets it resets it in cleanup. +func SetAfterInstanceLock(fn func(string)) { afterInstanceLock = fn } diff --git a/internal/integration/lifecycle.go b/internal/integration/lifecycle.go index d27d1633..c675773c 100644 --- a/internal/integration/lifecycle.go +++ b/internal/integration/lifecycle.go @@ -114,10 +114,19 @@ func NewLifecycleGeneration(ctx context.Context, db DBTX) (int64, error) { return New(db).MintLifecycleGeneration(ctx) } -// RecordLifecycleRequest records spec or answers from what holds its address -// (design D26). It serializes per instance through a transaction-scoped -// advisory lock, so dbtx must be a transaction for the answer to hold until -// the caller commits; given a *sql.DB it runs in a transaction of its own. +// RecordLifecycleRequest records spec, or answers from what holds its +// address, in a transaction of its own on b (a *sql.DB, or a *sql.Conn a +// test pins), committed when it answers. RecordLifecycleRequestInTx records +// in the caller's transaction. +func RecordLifecycleRequest(ctx context.Context, b db.Beginner, spec LifecycleRequestSpec) (LifecycleAnswer, error) { + return inOwnTx(ctx, b, func(tx *sql.Tx) (LifecycleAnswer, error) { + return RecordLifecycleRequestInTx(ctx, tx, spec) + }) +} + +// RecordLifecycleRequestInTx records spec or answers from what holds its +// address (design D26), in tx, which the caller commits; the lifecycle +// family's lock serializes it per instance until then. // // Under the lock it reads the instance's latest generation. A generation // above it is new: the request is recorded unless another request on the @@ -125,51 +134,57 @@ func NewLifecycleGeneration(ctx context.Context, db DBTX) (int64, error) { // core.generation_ceiling() (LifecycleConflict). A generation at or below it is a repeat // when the address holds the same content, and a conflict otherwise, which // also covers an older tab's generation that was never recorded. -func RecordLifecycleRequest(ctx context.Context, dbtx DBTX, spec LifecycleRequestSpec) (LifecycleAnswer, error) { - return inOwnTx(ctx, dbtx, func(tx DBTX) (LifecycleAnswer, error) { - if err := spec.checkAddress("record lifecycle request"); err != nil { - return LifecycleAnswer{}, err - } - content, err := spec.content() - if err != nil { - return LifecycleAnswer{}, err - } - q := New(tx) - latest, err := lockLatest(ctx, q, spec.InstanceID) - if err != nil { - return LifecycleAnswer{}, err - } - return decide(ctx, q, spec, content, latest) - }) +func RecordLifecycleRequestInTx(ctx context.Context, tx *sql.Tx, spec LifecycleRequestSpec) (LifecycleAnswer, error) { + if err := spec.checkAddress("record lifecycle request"); err != nil { + return LifecycleAnswer{}, err + } + content, err := spec.content() + if err != nil { + return LifecycleAnswer{}, err + } + q := New(tx) + latest, err := lockLatest(ctx, tx, q, spec.InstanceID) + if err != nil { + return LifecycleAnswer{}, err + } + return decide(ctx, q, spec, content, latest) } // RecordFirstLifecycleRequest records a request that must be its instance's -// first, such as a create, or answers from what holds its address. It runs -// as RecordLifecycleRequest does, except that a generation above the -// instance's latest is a conflict when the instance already holds a request: -// a create posted again under another generation must not become a second -// create of the same instance. -func RecordFirstLifecycleRequest(ctx context.Context, dbtx DBTX, spec LifecycleRequestSpec) (LifecycleAnswer, error) { - return inOwnTx(ctx, dbtx, func(tx DBTX) (LifecycleAnswer, error) { - if err := spec.checkAddress("record first lifecycle request"); err != nil { - return LifecycleAnswer{}, err - } - q := New(tx) - latest, err := lockLatest(ctx, q, spec.InstanceID) - if err != nil { - return LifecycleAnswer{}, err - } - if latest != 0 && spec.Generation > latest { - return LifecycleAnswer{Kind: LifecycleConflict}, nil - } - content, err := spec.content() - if err != nil { - return LifecycleAnswer{}, err - } - return decide(ctx, q, spec, content, latest) +// first, such as a create, or answers from what holds its address, in a +// transaction of its own on b, as RecordLifecycleRequest does. +// RecordFirstLifecycleRequestInTx records in the caller's transaction. +func RecordFirstLifecycleRequest(ctx context.Context, b db.Beginner, spec LifecycleRequestSpec) (LifecycleAnswer, error) { + return inOwnTx(ctx, b, func(tx *sql.Tx) (LifecycleAnswer, error) { + return RecordFirstLifecycleRequestInTx(ctx, tx, spec) }) } +// RecordFirstLifecycleRequestInTx records a request that must be its +// instance's first, such as a create, or answers from what holds its +// address. It runs as RecordLifecycleRequestInTx does, except that a +// generation above the instance's latest is a conflict when the instance +// already holds a request: a create posted again under another generation +// must not become a second create of the same instance. +func RecordFirstLifecycleRequestInTx(ctx context.Context, tx *sql.Tx, spec LifecycleRequestSpec) (LifecycleAnswer, error) { + if err := spec.checkAddress("record first lifecycle request"); err != nil { + return LifecycleAnswer{}, err + } + q := New(tx) + latest, err := lockLatest(ctx, tx, q, spec.InstanceID) + if err != nil { + return LifecycleAnswer{}, err + } + if latest != 0 && spec.Generation > latest { + return LifecycleAnswer{Kind: LifecycleConflict}, nil + } + content, err := spec.content() + if err != nil { + return LifecycleAnswer{}, err + } + return decide(ctx, q, spec, content, latest) +} + // decide answers a spec under its instance's lock, given the latest // generation read under it. func decide(ctx context.Context, q *Queries, spec LifecycleRequestSpec, c requestContent, latest int64) (LifecycleAnswer, error) { @@ -179,16 +194,12 @@ func decide(ctx context.Context, q *Queries, spec LifecycleRequestSpec, c reques return answerAddress(ctx, q, spec, c) } -// inOwnTx runs fn on dbtx, or, when dbtx is a *sql.DB, in a transaction of -// its own that commits when fn succeeds. The answer comes back with a -// failed commit's error, as each caller returned it before. -func inOwnTx(ctx context.Context, dbtx DBTX, fn func(DBTX) (LifecycleAnswer, error)) (LifecycleAnswer, error) { - sqlDB, ok := dbtx.(*sql.DB) - if !ok { - return fn(dbtx) - } +// inOwnTx runs fn in a transaction of its own on b, committed when fn +// succeeds. The answer comes back with a failed commit's error, as each +// caller returned it before. +func inOwnTx(ctx context.Context, b db.Beginner, fn func(*sql.Tx) (LifecycleAnswer, error)) (LifecycleAnswer, error) { var answer LifecycleAnswer - err := db.InTx(ctx, sqlDB, db.Plain, func(tx *sql.Tx) error { + err := db.InTx(ctx, b, db.Plain, func(tx *sql.Tx) error { var err error answer, err = fn(tx) return err @@ -229,11 +240,19 @@ func (spec LifecycleRequestSpec) content() (requestContent, error) { return c, nil } -// lockLatest takes the instance's lock and reads its latest generation. -func lockLatest(ctx context.Context, q *Queries, instanceID string) (int64, error) { - if err := q.LockLifecycleInstance(ctx, instanceID); err != nil { +// afterInstanceLock, when set by a test, runs inside a recording right after +// the instance's lock is granted and before its latest generation is read. +var afterInstanceLock func(instanceID string) + +// lockLatest takes the instance's lock on tx and reads its latest +// generation. +func lockLatest(ctx context.Context, tx *sql.Tx, q *Queries, instanceID string) (int64, error) { + if err := db.Lock(ctx, tx, db.Lifecycle, instanceID); err != nil { return 0, fmt.Errorf("lock lifecycle instance: %w", err) } + if afterInstanceLock != nil { + afterInstanceLock(instanceID) + } latest, err := q.LatestLifecycleGeneration(ctx, instanceID) if err != nil { return 0, fmt.Errorf("latest lifecycle generation: %w", err) diff --git a/internal/integration/lifecycle_answers_golden_db_test.go b/internal/integration/lifecycle_answers_golden_db_test.go index ba4df5df..4aa2ab4f 100644 --- a/internal/integration/lifecycle_answers_golden_db_test.go +++ b/internal/integration/lifecycle_answers_golden_db_test.go @@ -145,9 +145,9 @@ func recordCase(d *dq, f lifecycleFixture, statements *dbtest.Statements, norm * if c.edit != nil { c.edit(f, d, &spec) } - record := integration.RecordLifecycleRequest + record := integration.RecordLifecycleRequestInTx if c.first { - record = integration.RecordFirstLifecycleRequest + record = integration.RecordFirstLifecycleRequestInTx } statements.Reset() a, err := record(d.ctx, d.tx, spec) diff --git a/internal/integration/lifecycle_db_test.go b/internal/integration/lifecycle_db_test.go index 7e23fa98..d6326b42 100644 --- a/internal/integration/lifecycle_db_test.go +++ b/internal/integration/lifecycle_db_test.go @@ -21,6 +21,7 @@ import ( "github.com/google/uuid" + "git.coopcloud.tech/wiki-cafe/member-console/internal/db" "git.coopcloud.tech/wiki-cafe/member-console/internal/integration" "git.coopcloud.tech/wiki-cafe/member-console/internal/testkit" "git.coopcloud.tech/wiki-cafe/member-console/internal/testkit/dbtest" @@ -59,7 +60,7 @@ func (d *dq) generation() int64 { func (d *dq) record(spec integration.LifecycleRequestSpec) integration.LifecycleAnswer { d.t.Helper() - a, err := integration.RecordLifecycleRequest(d.ctx, d.tx, spec) + a, err := integration.RecordLifecycleRequestInTx(d.ctx, d.tx, spec) if err != nil { d.t.Fatalf("record %+v: %v", spec, err) } @@ -197,7 +198,7 @@ func TestRecordLifecycleRequestOlderTabAndFloors_DB(t *testing.T) { foreignKey.ResourceKey = "external_domain_claims" for name, s := range map[string]integration.LifecycleRequestSpec{"operation": undeclared, "key": foreignKey} { d.Exec("SAVEPOINT refused") - if _, err := integration.RecordLifecycleRequest(d.ctx, d.tx, s); !errors.Is(err, integration.ErrLifecycleOperationRefused) { + if _, err := integration.RecordLifecycleRequestInTx(d.ctx, d.tx, s); !errors.Is(err, integration.ErrLifecycleOperationRefused) { t.Errorf("undeclared %s: %v", name, err) } d.Exec("ROLLBACK TO SAVEPOINT refused") @@ -330,7 +331,7 @@ func TestRecordLifecycleRequestConcurrentOlderTab_DB(t *testing.T) { t.Fatal(err) } defer newer.Rollback() - if a, err := integration.RecordLifecycleRequest(ctx, newer, fx.spec(instance, g2, integration.LifecycleDelete, `{}`)); err != nil || a.Kind != integration.LifecycleRecorded { + if a, err := integration.RecordLifecycleRequestInTx(ctx, newer, fx.spec(instance, g2, integration.LifecycleDelete, `{}`)); err != nil || a.Kind != integration.LifecycleRecorded { t.Fatalf("newer = %+v, %v", a, err) } @@ -354,7 +355,7 @@ func TestRecordLifecycleRequestConcurrentOlderTab_DB(t *testing.T) { done <- result{err: err} return } - a, err := integration.RecordLifecycleRequest(ctx, tx, fx.spec(instance, g1, integration.LifecycleDelete, `{}`)) + a, err := integration.RecordLifecycleRequestInTx(ctx, tx, fx.spec(instance, g1, integration.LifecycleDelete, `{}`)) if err == nil { err = tx.Commit() } else { @@ -392,7 +393,7 @@ func TestRecordLifecycleRequestOwnTransaction_DB(t *testing.T) { counted, statements := dbtest.Counted(t) ctx := context.Background() fx := committedLifecycleFixture(t, database) - record := map[string]func(context.Context, integration.DBTX, integration.LifecycleRequestSpec) (integration.LifecycleAnswer, error){ + record := map[string]func(context.Context, db.Beginner, integration.LifecycleRequestSpec) (integration.LifecycleAnswer, error){ "RecordLifecycleRequest": integration.RecordLifecycleRequest, "RecordFirstLifecycleRequest": integration.RecordFirstLifecycleRequest, } @@ -557,7 +558,7 @@ func TestFirstLifecycleRequestRepeatAndWorkspace_DB(t *testing.T) { instance := uuid.New().String() spec := f.spec(instance, d.generation(), integration.LifecycleCreate, `{"domain": "first.example.test"}`) for i, want := range []integration.LifecycleAnswerKind{integration.LifecycleRecorded, integration.LifecycleRepeatOpen} { - a, err := integration.RecordFirstLifecycleRequest(d.ctx, d.tx, spec) + a, err := integration.RecordFirstLifecycleRequestInTx(d.ctx, d.tx, spec) if err != nil || a.Kind != want { t.Fatalf("first request, call %d = %v, %v; want %v", i+1, a.Kind, err, want) } @@ -584,7 +585,7 @@ func TestLifecycleGenerationAboveTheCeilingIsRefused_DB(t *testing.T) { if a := d.record(f.spec(instance, math.MaxInt64, integration.LifecycleCreate, `{"domain": "top.example.test"}`)); a.Kind != integration.LifecycleConflict { t.Errorf("a generation of MaxInt64 = %+v, want a conflict", a) } - if a, err := integration.RecordFirstLifecycleRequest(d.ctx, d.tx, f.spec(instance, math.MaxInt64, integration.LifecycleCreate, `{}`)); err != nil || a.Kind != integration.LifecycleConflict { + if a, err := integration.RecordFirstLifecycleRequestInTx(d.ctx, d.tx, f.spec(instance, math.MaxInt64, integration.LifecycleCreate, `{}`)); err != nil || a.Kind != integration.LifecycleConflict { t.Errorf("a first request at MaxInt64 = %+v, %v; want a conflict", a, err) } if n := d.requestRows(instance); n != 0 { @@ -603,3 +604,48 @@ func TestLifecycleGenerationAboveTheCeilingIsRefused_DB(t *testing.T) { } d.Exec(`ROLLBACK TO SAVEPOINT above_ceiling`) } + +// TestARecordingOnAConnectionHoldsTheInstanceLock records on a connection a +// test pins, which the recording function used to run on without a +// transaction, so the instance lock ended with the statement that took it. A +// hook that runs right after the lock is granted probes the lock from another +// transaction. +func TestARecordingOnAConnectionHoldsTheInstanceLock(t *testing.T) { + database := dbtest.Open(t) + ctx := context.Background() + fx := committedLifecycleFixture(t, database) + conn, err := database.Conn(ctx) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = conn.Close() }) + instance := uuid.New().String() + gen, err := integration.NewLifecycleGeneration(ctx, database) + if err != nil { + t.Fatal(err) + } + var probed, free bool + var probeErr error + integration.SetAfterInstanceLock(func(id string) { + side, err := database.BeginTx(ctx, nil) + if err != nil { + probeErr = err + return + } + defer side.Rollback() + probed = true + free, probeErr = dbtest.TryLock(ctx, side, db.Lifecycle, id) + }) + t.Cleanup(func() { integration.SetAfterInstanceLock(nil) }) + + a, err := integration.RecordLifecycleRequest(ctx, conn, fx.spec(instance, gen, integration.LifecycleCreate, `{}`)) + if err != nil || a.Kind != integration.LifecycleRecorded { + t.Errorf("the recording on a pinned connection answered %v, %v, want accepted", a, err) + } + if probeErr != nil || !probed { + t.Fatalf("the probe after the instance lock ran %v, failed with %v", probed, probeErr) + } + if free { + t.Errorf("a recording on a pinned connection left the instance lock free after taking it") + } +} diff --git a/internal/integration/lifecycle_requests.sql.go b/internal/integration/lifecycle_requests.sql.go index e1a04486..2310f098 100644 --- a/internal/integration/lifecycle_requests.sql.go +++ b/internal/integration/lifecycle_requests.sql.go @@ -262,6 +262,12 @@ FROM core.lifecycle_requests WHERE instance_id = $1::uuid ` +// Lifecycle requests (design D24 to D28). Recording a request runs, in one +// transaction: the lifecycle family's lock (db.Lock), +// LatestLifecycleGeneration, then for a generation above the latest +// GenerationCeiling, OpenLifecycleGeneration and InsertLifecycleRequest, and +// otherwise GetLifecycleRequestByAddress, so the latest-generation check and +// the insert are one step per instance. // The highest generation recorded for an instance, 0 when it has none. func (q *Queries) LatestLifecycleGeneration(ctx context.Context, instanceID string) (int64, error) { row := q.db.QueryRowContext(ctx, latestLifecycleGeneration, instanceID) @@ -326,20 +332,6 @@ func (q *Queries) ListAcceptedLifecycleRequestsOlderThan(ctx context.Context, ar return items, nil } -const lockLifecycleInstance = `-- name: LockLifecycleInstance :exec -SELECT pg_advisory_xact_lock(hashtextextended('lifecycle:' || $1::uuid::text, 0)) -` - -// Lifecycle requests (design D24 to D28). Recording a request runs, in one -// transaction: LockLifecycleInstance, LatestLifecycleGeneration, then for a -// generation above the latest GenerationCeiling, OpenLifecycleGeneration and -// InsertLifecycleRequest, and otherwise GetLifecycleRequestByAddress, so the -// latest-generation check and the insert are one step per instance. -func (q *Queries) LockLifecycleInstance(ctx context.Context, instanceID string) error { - _, err := q.db.ExecContext(ctx, lockLifecycleInstance, instanceID) - return err -} - const mintInstanceID = `-- name: MintInstanceID :one SELECT uuidv7()::uuid AS instance_id ` @@ -411,9 +403,9 @@ WHERE instance_id = $1::uuid AND status = 'accepted' ` // The generation of the instance's accepted request, 0 when none is open. -// Read under LockLifecycleInstance, so the one-open rule is answered before -// the insert instead of by a unique violation that would abort the caller's -// transaction. +// Read under the lifecycle family's lock (db.Lock), so the one-open rule is +// answered before the insert instead of by a unique violation that would +// abort the caller's transaction. func (q *Queries) OpenLifecycleGeneration(ctx context.Context, instanceID string) (int64, error) { row := q.db.QueryRowContext(ctx, openLifecycleGeneration, instanceID) var open_generation int64 diff --git a/internal/integration/querier.go b/internal/integration/querier.go index 9fdfc915..2300c6ca 100644 --- a/internal/integration/querier.go +++ b/internal/integration/querier.go @@ -145,6 +145,12 @@ type Querier interface { // accepted request on the instance under another generation raises a unique // violation on uq_lifecycle_requests_one_open. InsertLifecycleRequest(ctx context.Context, arg InsertLifecycleRequestParams) (string, error) + // Lifecycle requests (design D24 to D28). Recording a request runs, in one + // transaction: the lifecycle family's lock (db.Lock), + // LatestLifecycleGeneration, then for a generation above the latest + // GenerationCeiling, OpenLifecycleGeneration and InsertLifecycleRequest, and + // otherwise GetLifecycleRequestByAddress, so the latest-generation check and + // the insert are one step per instance. // The highest generation recorded for an instance, 0 when it has none. LatestLifecycleGeneration(ctx context.Context, instanceID string) (int64, error) // The boot pass (design D27): accepted requests at one connection older than @@ -191,23 +197,15 @@ type Querier interface { ListSubscriptionsWithRecentReconcileEntries(ctx context.Context, subscriptionIds []string) ([]string, error) // --------------------------------------------------------------------------- // Recomputing one record (design D9), in one READ COMMITTED transaction, in - // this order: LockDesiredStateAddress, LockDesiredStateRecord, then - // DesiredStateContent in a new statement, then UpdateDesiredStateRecord or - // InsertDesiredStateRecord. Every transaction that writes a record's content - // or generation, or a projection, takes the address lock first. + // this order: the desired-state family's lock (db.Lock), + // LockDesiredStateRecord, then DesiredStateContent in a new statement, then + // UpdateDesiredStateRecord or InsertDesiredStateRecord. Every transaction that + // writes a record's content or generation, or a projection, takes the address + // lock first. // --------------------------------------------------------------------------- - // The serialization point for one (connection, recipient) address, including - // one that has no record yet. recipient_kind is 'person' or 'organization'. - LockDesiredStateAddress(ctx context.Context, arg LockDesiredStateAddressParams) error // The record at one address, locked; exactly one of person_id and org_id is // set. sql.ErrNoRows when the address has no record. LockDesiredStateRecord(ctx context.Context, arg LockDesiredStateRecordParams) (DesiredStateRecord, error) - // Lifecycle requests (design D24 to D28). Recording a request runs, in one - // transaction: LockLifecycleInstance, LatestLifecycleGeneration, then for a - // generation above the latest GenerationCeiling, OpenLifecycleGeneration and - // InsertLifecycleRequest, and otherwise GetLifecycleRequestByAddress, so the - // latest-generation check and the insert are one step per instance. - LockLifecycleInstance(ctx context.Context, instanceID string) error // A failed attempt on a row: for a wide row, its expansion query failed; for // a recipient_only row, its recompute failed. The row becomes failed (the // dead letter) when attempts reaches max_attempts and otherwise stays @@ -229,9 +227,9 @@ type Querier interface { // request and at least the clock, one row per instance asked for. NextLifecycleGenerations(ctx context.Context, instanceIds []string) ([]NextLifecycleGenerationsRow, error) // The generation of the instance's accepted request, 0 when none is open. - // Read under LockLifecycleInstance, so the one-open rule is answered before - // the insert instead of by a unique violation that would abort the caller's - // transaction. + // Read under the lifecycle family's lock (db.Lock), so the one-open rule is + // answered before the insert instead of by a unique violation that would + // abort the caller's transaction. OpenLifecycleGeneration(ctx context.Context, instanceID string) (int64, error) // What a workspace's list shows for one connection: every accepted request, // and per instance the latest request when it is declined or failed and not diff --git a/internal/integration/queries/desired_state.sql b/internal/integration/queries/desired_state.sql index 2d422f5a..7a6ed463 100644 --- a/internal/integration/queries/desired_state.sql +++ b/internal/integration/queries/desired_state.sql @@ -252,20 +252,13 @@ LIMIT sqlc.arg(page_size)::int; -- --------------------------------------------------------------------------- -- Recomputing one record (design D9), in one READ COMMITTED transaction, in --- this order: LockDesiredStateAddress, LockDesiredStateRecord, then --- DesiredStateContent in a new statement, then UpdateDesiredStateRecord or --- InsertDesiredStateRecord. Every transaction that writes a record's content --- or generation, or a projection, takes the address lock first. +-- this order: the desired-state family's lock (db.Lock), +-- LockDesiredStateRecord, then DesiredStateContent in a new statement, then +-- UpdateDesiredStateRecord or InsertDesiredStateRecord. Every transaction that +-- writes a record's content or generation, or a projection, takes the address +-- lock first. -- --------------------------------------------------------------------------- --- name: LockDesiredStateAddress :exec --- The serialization point for one (connection, recipient) address, including --- one that has no record yet. recipient_kind is 'person' or 'organization'. -SELECT pg_advisory_xact_lock(hashtextextended( - 'desired-state:' || sqlc.arg(connection_id)::uuid::text - || ':' || sqlc.arg(recipient_kind)::text - || ':' || sqlc.arg(recipient_id)::uuid::text, 0)); - -- name: LockDesiredStateRecord :one -- The record at one address, locked; exactly one of person_id and org_id is -- set. sql.ErrNoRows when the address has no record. diff --git a/internal/integration/queries/lifecycle_requests.sql b/internal/integration/queries/lifecycle_requests.sql index 0ae20eef..a4d7550f 100644 --- a/internal/integration/queries/lifecycle_requests.sql +++ b/internal/integration/queries/lifecycle_requests.sql @@ -1,12 +1,10 @@ --- name: LockLifecycleInstance :exec --- Lifecycle requests (design D24 to D28). Recording a request runs, in one --- transaction: LockLifecycleInstance, LatestLifecycleGeneration, then for a --- generation above the latest GenerationCeiling, OpenLifecycleGeneration and --- InsertLifecycleRequest, and otherwise GetLifecycleRequestByAddress, so the --- latest-generation check and the insert are one step per instance. -SELECT pg_advisory_xact_lock(hashtextextended('lifecycle:' || sqlc.arg(instance_id)::uuid::text, 0)); - -- name: LatestLifecycleGeneration :one +-- Lifecycle requests (design D24 to D28). Recording a request runs, in one +-- transaction: the lifecycle family's lock (db.Lock), +-- LatestLifecycleGeneration, then for a generation above the latest +-- GenerationCeiling, OpenLifecycleGeneration and InsertLifecycleRequest, and +-- otherwise GetLifecycleRequestByAddress, so the latest-generation check and +-- the insert are one step per instance. -- The highest generation recorded for an instance, 0 when it has none. SELECT COALESCE(MAX(generation), 0)::bigint AS latest_generation FROM core.lifecycle_requests @@ -128,9 +126,9 @@ GROUP BY ids.instance_id; -- name: OpenLifecycleGeneration :one -- The generation of the instance's accepted request, 0 when none is open. --- Read under LockLifecycleInstance, so the one-open rule is answered before --- the insert instead of by a unique violation that would abort the caller's --- transaction. +-- Read under the lifecycle family's lock (db.Lock), so the one-open rule is +-- answered before the insert instead of by a unique violation that would +-- abort the caller's transaction. SELECT COALESCE(MAX(generation), 0)::bigint AS open_generation FROM core.lifecycle_requests WHERE instance_id = sqlc.arg(instance_id)::uuid AND status = 'accepted'; diff --git a/internal/integrations/fedwiki/store/request_intake.go b/internal/integrations/fedwiki/store/request_intake.go index 789daab5..2c8fac4e 100644 --- a/internal/integrations/fedwiki/store/request_intake.go +++ b/internal/integrations/fedwiki/store/request_intake.go @@ -33,24 +33,25 @@ func RecordSiteRequest(ctx context.Context, database db.Beginner, spec integrati // RecordSiteRequestInTx validates new status intent under the instance lock. // The caller supplies a READ COMMITTED transaction and rolls back on refusal. func RecordSiteRequestInTx(ctx context.Context, tx *sql.Tx, spec integration.LifecycleRequestSpec, domain string) (integration.LifecycleAnswer, error) { - return recordSiteRequestInTx(ctx, tx, spec, domain) + return recordSiteRequestInTx(ctx, tx, tx, spec, domain) } -func recordSiteRequestInTx(ctx context.Context, tx DBTX, spec integration.LifecycleRequestSpec, domain string) (integration.LifecycleAnswer, error) { - queries := integration.New(tx) - if err := queries.LockLifecycleInstance(ctx, spec.InstanceID); err != nil { +// recordSiteRequestInTx takes the instance lock and records on tx, and runs +// every read on q, a handle a test wraps to make one of them fail. +func recordSiteRequestInTx(ctx context.Context, tx *sql.Tx, q DBTX, spec integration.LifecycleRequestSpec, domain string) (integration.LifecycleAnswer, error) { + if err := db.Lock(ctx, tx, db.Lifecycle, spec.InstanceID); err != nil { return integration.LifecycleAnswer{}, fmt.Errorf("lock lifecycle instance: %w", err) } - _, _, found, err := integration.LookupLifecycleRequest(ctx, tx, spec.WorkspaceID, spec.InstanceID, spec.Generation) + _, _, found, err := integration.LookupLifecycleRequest(ctx, q, spec.WorkspaceID, spec.InstanceID, spec.Generation) if err != nil { return integration.LifecycleAnswer{}, err } if !found { - if err := checkRequestSite(ctx, tx, spec.InstanceID, spec.WorkspaceID, domain); err != nil { + if err := checkRequestSite(ctx, q, spec.InstanceID, spec.WorkspaceID, domain); err != nil { return integration.LifecycleAnswer{}, err } } - return integration.RecordLifecycleRequest(ctx, tx, spec) + return integration.RecordLifecycleRequestInTx(ctx, tx, spec) } // checkRequestSite reads an original site under an instance lock the caller holds. @@ -72,7 +73,7 @@ func checkRequestSite(ctx context.Context, tx DBTX, instanceID, workspaceID, dom // The caller supplies a READ COMMITTED transaction and records activation then park. func CheckSwapRequestSites(ctx context.Context, tx *sql.Tx, workspaceID, activateID, activateDomain, parkID, parkDomain string) error { for _, instanceID := range []string{activateID, parkID} { - if err := integration.New(tx).LockLifecycleInstance(ctx, instanceID); err != nil { + if err := db.Lock(ctx, tx, db.Lifecycle, instanceID); err != nil { return err } } diff --git a/internal/integrations/fedwiki/store/request_intake_db_test.go b/internal/integrations/fedwiki/store/request_intake_db_test.go index ee5f7ac6..1fedc386 100644 --- a/internal/integrations/fedwiki/store/request_intake_db_test.go +++ b/internal/integrations/fedwiki/store/request_intake_db_test.go @@ -88,7 +88,7 @@ func checkIntakeOrder(t *testing.T, row intakeOrderCase) { } defer holder.Rollback() pid := int(dbtest.On(t, holder).Int("SELECT pg_backend_pid()")) - if err := integration.New(holder).LockLifecycleInstance(env.Ctx, instance); err != nil { + if err := db.Lock(env.Ctx, holder, db.Lifecycle, instance); err != nil { t.Fatal(err) } ctx, cancel := context.WithTimeout(env.Ctx, 10*time.Second) @@ -154,7 +154,7 @@ func checkProtectedReadError(t *testing.T, database *sql.DB, spec integration.Li t.Fatal(err) } defer tx.Rollback() - _, err = fwmod.RecordSiteRequestOn(t.Context(), failedSiteRead{tx}, spec, domain) + _, err = fwmod.RecordSiteRequestOn(t.Context(), tx, failedSiteRead{tx}, spec, domain) var missing *fwmod.MissingSiteError if err == nil || errors.As(err, &missing) { t.Errorf("%s: protected read hid a database error as not found", name) @@ -192,7 +192,7 @@ func TestSiteRequestRepeats_DB(t *testing.T) { // deleteWithoutRequest checks that the locked read sees committed intake. func deleteWithoutRequest(ctx context.Context, intakeDB *sql.DB, instance string) error { return db.InTx(ctx, intakeDB, db.ReadCommitted, func(tx *sql.Tx) error { - if err := integration.New(tx).LockLifecycleInstance(ctx, instance); err != nil { + if err := db.Lock(ctx, tx, db.Lifecycle, instance); err != nil { return err } open, err := fwmod.New(tx).SiteHasOpenRequest(ctx, instance) @@ -216,7 +216,7 @@ func recordIntakePair(ctx context.Context, database *sql.DB, spec integration.Li if err := fwmod.CheckSwapRequestSites(ctx, tx, spec.WorkspaceID, spec.InstanceID, domain, peerID, peerDomain); err != nil { return err } - _, err := integration.RecordLifecycleRequest(ctx, tx, spec) + _, err := integration.RecordLifecycleRequestInTx(ctx, tx, spec) return err }) } diff --git a/internal/integrations/fedwiki/usage/usage.go b/internal/integrations/fedwiki/usage/usage.go index 6c110a9a..7504ae48 100644 --- a/internal/integrations/fedwiki/usage/usage.go +++ b/internal/integrations/fedwiki/usage/usage.go @@ -219,15 +219,17 @@ type HoldRequest struct { // other caller of the package checks errors.Is(err, ErrNoUsageRow) and goes // on. func Hold(ctx context.Context, tx *sql.Tx, req HoldRequest) (held bool, err error) { - return hold(ctx, tx, req) + return hold(ctx, tx, tx, req) } -func hold(ctx context.Context, tx entitlements.DBTX, req HoldRequest) (held bool, err error) { +// hold is Hold with the lock and its read on tx and every other statement on +// q, a handle a test wraps to make one of them fail. +func hold(ctx context.Context, tx *sql.Tx, q entitlements.DBTX, req HoldRequest) (held bool, err error) { row, err := lockRequest(ctx, tx, req) if err != nil { return false, err } - siteQ := fwmod.New(tx) + siteQ := fwmod.New(q) alreadyActive, err := activeSite(ctx, siteQ, req.InstanceID, row.Operation) if err != nil { return false, err @@ -237,35 +239,35 @@ func hold(ctx context.Context, tx entitlements.DBTX, req HoldRequest) (held bool return false, fmt.Errorf("read hold of request %s: %w", row.RequestID, err) } if existing { - return recountHeld(ctx, tx, row.WorkspaceID) + return recountHeld(ctx, q, row.WorkspaceID) } - poolID, err := primaryPool(ctx, tx, row.WorkspaceID) + poolID, err := primaryPool(ctx, q, row.WorkspaceID) if err != nil { return false, err } - limit, found, err := Limit(ctx, entitlements.New(tx), poolID) + limit, found, err := Limit(ctx, entitlements.New(q), poolID) if err != nil { return false, err } if !found { // No entitlement is a full limit; the pool's figure, when it has a // row, is still written. - if _, err := recount(ctx, tx, poolID); err != nil && !errors.Is(err, ErrNoUsageRow) { + if _, err := recount(ctx, q, poolID); err != nil && !errors.Is(err, ErrNoUsageRow) { return false, err } return false, nil } - slots, err := lockAndCount(ctx, tx, poolID) + slots, err := lockAndCount(ctx, q, poolID) if err != nil { return false, err } if !alreadyActive && slots.Total() >= limit { - return false, slots.write(ctx, tx, poolID) + return false, slots.write(ctx, q, poolID) } if err := siteQ.InsertSlotHold(ctx, row.RequestID); err != nil { return false, fmt.Errorf("hold a slot for request %s: %w", row.RequestID, err) } - if _, err := recount(ctx, tx, poolID); err != nil { + if _, err := recount(ctx, q, poolID); err != nil { return false, err } return true, nil @@ -287,11 +289,11 @@ func recountHeld(ctx context.Context, tx entitlements.DBTX, workspaceID string) // lockRequest takes the request's instance lock and reads the request, which // must still be accepted. -func lockRequest(ctx context.Context, tx entitlements.DBTX, req HoldRequest) (integration.LifecycleRequest, error) { - q := integration.New(tx) - if err := q.LockLifecycleInstance(ctx, req.InstanceID); err != nil { +func lockRequest(ctx context.Context, tx *sql.Tx, req HoldRequest) (integration.LifecycleRequest, error) { + if err := db.Lock(ctx, tx, db.Lifecycle, req.InstanceID); err != nil { return integration.LifecycleRequest{}, err } + q := integration.New(tx) row, err := q.GetLifecycleRequestByAddress(ctx, integration.GetLifecycleRequestByAddressParams{ InstanceID: req.InstanceID, Generation: req.Generation, }) diff --git a/internal/integrations/fedwiki/usage/usage_db_test.go b/internal/integrations/fedwiki/usage/usage_db_test.go index a44d4ef3..7b66fdce 100644 --- a/internal/integrations/fedwiki/usage/usage_db_test.go +++ b/internal/integrations/fedwiki/usage/usage_db_test.go @@ -355,7 +355,7 @@ func TestHoldReturnsAFailedLimitRead(t *testing.T) { var held bool err := db.InTx(f.ctx, f.db, db.ReadCommitted, func(tx *sql.Tx) error { var err error - held, err = usage.HoldOn(f.ctx, failingLimitRead{tx}, usage.HoldRequest{InstanceID: instance, Generation: gen}) + held, err = usage.HoldOn(f.ctx, tx, failingLimitRead{tx}, usage.HoldRequest{InstanceID: instance, Generation: gen}) return err }) if err == nil || held { diff --git a/internal/integrations/fedwiki/web/request_identity_db_test.go b/internal/integrations/fedwiki/web/request_identity_db_test.go index 5f939be6..af500944 100644 --- a/internal/integrations/fedwiki/web/request_identity_db_test.go +++ b/internal/integrations/fedwiki/web/request_identity_db_test.go @@ -270,7 +270,7 @@ func holdIntakeDeletion(t *testing.T, env *siteEnv, target, domain string, repla t.Fatal(err) } holderPID := int(dbtest.On(t, holder).Int("SELECT pg_backend_pid()")) - if err := integration.New(holder).LockLifecycleInstance(env.ctx, target); err != nil { + if err := db.Lock(env.ctx, holder, db.Lifecycle, target); err != nil { t.Fatal(err) } dbtest.On(t, holder).Exec("DELETE FROM fedwiki.sites WHERE site_id = $1", target) diff --git a/internal/integrations/fedwiki/web/site_actions.go b/internal/integrations/fedwiki/web/site_actions.go index efbac234..f9f20b07 100644 --- a/internal/integrations/fedwiki/web/site_actions.go +++ b/internal/integrations/fedwiki/web/site_actions.go @@ -615,7 +615,7 @@ func (h *FedWikiPartialsHandler) swapActiveSite(w http.ResponseWriter, r *http.R return err } - answer, err = integration.RecordLifecycleRequest(ctx, tx, newRequestSpec(conn, ws, act.session.PersonID, + answer, err = integration.RecordLifecycleRequestInTx(ctx, tx, newRequestSpec(conn, ws, act.session.PersonID, integration.LifecycleSetStatus, act.addr, workflowID, requestParams{Domain: act.site.Domain, Status: "active", SwapParkSiteID: parked.SiteID})) if err != nil { @@ -787,7 +787,7 @@ func (h *FedWikiPartialsHandler) siteActionConflict(w http.ResponseWriter) { } // withWorkspaceTx runs fn inside a transaction that holds a per-workspace -// Postgres advisory lock (pg_advisory_xact_lock, released on commit or +// Postgres advisory lock (the fedwiki-workspace family's, released on commit or // rollback), serializing a workspace's active-site rotations so two tabs // cannot interleave the cooldown check-and-stamp or the park-target // selection (audit #51). fn receives the transaction and a querier on it; @@ -798,10 +798,7 @@ func (h *FedWikiPartialsHandler) withWorkspaceTx(ctx context.Context, workspaceI return errors.New("no database for the workspace lock") } return db.InTx(ctx, h.Database, db.ReadCommitted, func(tx *sql.Tx) error { - // hashtext maps the workspace UUID text onto the bigint the advisory-lock - // API takes; an occasional hash collision only serializes two unrelated - // workspaces briefly, which is harmless. - if _, err := tx.ExecContext(ctx, `SELECT pg_advisory_xact_lock(hashtext($1)::bigint)`, workspaceID); err != nil { + if err := db.Lock(ctx, tx, db.FedWikiWorkspace, workspaceID); err != nil { return err } return fn(ctx, tx, fwmod.New(tx)) @@ -1030,7 +1027,7 @@ func (h *FedWikiPartialsHandler) recordParkRequest(ctx context.Context, tx *sql. return requestAddress{}, integration.LifecycleAnswer{}, err } park := requestAddress{Instance: instance, Generation: gens[instance]} - answer, err := integration.RecordLifecycleRequest(ctx, tx, newRequestSpec(h.connectionID(), act.session.WorkspaceID, act.session.PersonID, + answer, err := integration.RecordLifecycleRequestInTx(ctx, tx, newRequestSpec(h.connectionID(), act.session.WorkspaceID, act.session.PersonID, integration.LifecycleSetStatus, park, workflowID, requestParams{Domain: domain, Status: "readonly", SwapActivateSiteID: act.site.SiteID})) return park, answer, err diff --git a/internal/integrations/fedwiki/web/workspace_lock_db_test.go b/internal/integrations/fedwiki/web/workspace_lock_db_test.go new file mode 100644 index 00000000..87b34817 --- /dev/null +++ b/internal/integrations/fedwiki/web/workspace_lock_db_test.go @@ -0,0 +1,50 @@ +// SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Commercial +// SPDX-FileCopyrightText: 2025-2026 Christian Galo + +package web + +import ( + "context" + "database/sql" + "testing" + "time" + + "github.com/google/uuid" + + "git.coopcloud.tech/wiki-cafe/member-console/internal/db" + fwmod "git.coopcloud.tech/wiki-cafe/member-console/internal/integrations/fedwiki/store" + "git.coopcloud.tech/wiki-cafe/member-console/internal/testkit/dbtest" +) + +// TestWorkspaceTxTakesTheWorkspaceLock holds that a rotation's transaction +// waits for the lock of its own workspace and for no other: a side +// transaction holds one workspace's lock, the rotation of that workspace +// waits, and the rotation of another workspace goes through. +func TestWorkspaceTxTakesTheWorkspaceLock(t *testing.T) { + database := dbtest.Open(t) + h := &FedWikiPartialsHandler{Database: database} + ctx := context.Background() + held, other := uuid.NewString(), uuid.NewString() + noop := func(context.Context, *sql.Tx, fwmod.Querier) error { return nil } + + q := dbtest.Tx(t, database) + sidePID := int(q.Int(`SELECT pg_backend_pid()`)) + if err := db.Lock(ctx, q.Tx(), db.FedWikiWorkspace, held); err != nil { + t.Fatalf("take the workspace lock on the side: %v", err) + } + rotation := make(chan error, 1) + go func() { rotation <- h.withWorkspaceTx(ctx, held, noop) }() + dbtest.AwaitLockWait(t, database, dbtest.Backend{BlockedBy: sidePID}) + + otherCtx, cancel := context.WithTimeout(ctx, 5*time.Second) + defer cancel() + if err := h.withWorkspaceTx(otherCtx, other, noop); err != nil { + t.Errorf("a rotation of another workspace waited for workspace %s's lock: %v", held, err) + } + if err := q.Tx().Rollback(); err != nil { + t.Fatalf("release the workspace lock: %v", err) + } + if err := <-rotation; err != nil { + t.Errorf("the rotation returned %v", err) + } +} diff --git a/internal/integrations/fedwiki/workflows/purge_failures_db_test.go b/internal/integrations/fedwiki/workflows/purge_failures_db_test.go index 9bf0c9d8..09a55836 100644 --- a/internal/integrations/fedwiki/workflows/purge_failures_db_test.go +++ b/internal/integrations/fedwiki/workflows/purge_failures_db_test.go @@ -14,7 +14,7 @@ import ( "testing" "time" - "git.coopcloud.tech/wiki-cafe/member-console/internal/integration" + "git.coopcloud.tech/wiki-cafe/member-console/internal/db" fwmod "git.coopcloud.tech/wiki-cafe/member-console/internal/integrations/fedwiki/store" "git.coopcloud.tech/wiki-cafe/member-console/internal/testkit/dbtest" ) @@ -78,7 +78,7 @@ func retentionCancelledWait(env *reqEnv, site fwmod.Site, row purgeFailureCase) env.t.Helper() farm := observePurgeFarm(env, nil) side := env.beginSide(env.lockWaitContext()) - if err := integration.New(side.tx).LockLifecycleInstance(side.ctx, site.SiteID); err != nil { + if err := db.Lock(side.ctx, side.tx, db.Lifecycle, site.SiteID); err != nil { env.t.Fatal(err) } connection, pid := purgeConnection(env.t, env.db) diff --git a/internal/integrations/fedwiki/workflows/reconcile.go b/internal/integrations/fedwiki/workflows/reconcile.go index 341ad0bd..abcd5065 100644 --- a/internal/integrations/fedwiki/workflows/reconcile.go +++ b/internal/integrations/fedwiki/workflows/reconcile.go @@ -13,7 +13,6 @@ import ( "git.coopcloud.tech/wiki-cafe/member-console/internal/db" "git.coopcloud.tech/wiki-cafe/member-console/internal/entitlements" - "git.coopcloud.tech/wiki-cafe/member-console/internal/integration" fwmod "git.coopcloud.tech/wiki-cafe/member-console/internal/integrations/fedwiki/store" ) @@ -148,7 +147,7 @@ var errRetentionFarmDelete = errors.New("retention farm delete") func (a *Activities) purgeArchivedSite(ctx context.Context, database db.Beginner, candidate fwmod.Site, cutoff time.Time) (bool, error) { deleted := false err := db.InTx(ctx, database, db.ReadCommitted, func(tx *sql.Tx) error { - if err := integration.New(tx).LockLifecycleInstance(ctx, candidate.SiteID); err != nil { + if err := db.Lock(ctx, tx, db.Lifecycle, candidate.SiteID); err != nil { return err } siteQ := fwmod.New(tx) diff --git a/internal/integrations/fedwiki/workflows/request_activities.go b/internal/integrations/fedwiki/workflows/request_activities.go index 72d11a26..70f0c308 100644 --- a/internal/integrations/fedwiki/workflows/request_activities.go +++ b/internal/integrations/fedwiki/workflows/request_activities.go @@ -247,7 +247,7 @@ func finishAllWith(ctx context.Context, database db.Beginner, outs []integration func lockOutcomes(ctx context.Context, tx *sql.Tx, outs []integration.LifecycleOutcome) (held []integration.LifecycleRequest, finished int, err error) { q := integration.New(tx) for _, out := range outs { - if err := q.LockLifecycleInstance(ctx, out.InstanceID); err != nil { + if err := db.Lock(ctx, tx, db.Lifecycle, out.InstanceID); err != nil { return nil, 0, err } row, err := q.GetLifecycleRequestByAddress(ctx, integration.GetLifecycleRequestByAddressParams{ diff --git a/internal/integrations/stripe/catalogsync/catalogsync.go b/internal/integrations/stripe/catalogsync/catalogsync.go index 5720c8ac..851c4bb4 100644 --- a/internal/integrations/stripe/catalogsync/catalogsync.go +++ b/internal/integrations/stripe/catalogsync/catalogsync.go @@ -172,20 +172,19 @@ func noEnd() {} // outbox row, permanently "sync in progress" and unretryable, since the // dead-letter retry only matches outbox rows that were created; (2) two // concurrent syncs both cleared the check-then-act guards in Sync and -// enqueued duplicate create_stripe_product entries. The per-product -// advisory lock (the same idiom as fulfillment's reconcile) serializes -// them, and re-reading the mappings under the lock makes the loser a no-op -// instead of a duplicate. Only an enqueue commits. From the begin on, -// enqueue returns end, which rolls the transaction back; it does not roll -// back itself, so a no-op or a failure keeps the lock until the caller -// calls end (Sync). +// enqueued duplicate create_stripe_product entries. The stripe-product-sync +// family's lock (internal/db/locks.go) serializes them, and re-reading the +// mappings under the lock makes the loser a no-op instead of a duplicate. +// Only an enqueue commits. From the begin on, enqueue returns end, which rolls +// the transaction back; it does not roll back itself, so a no-op or a failure +// keeps the lock until the caller calls end (Sync). func enqueue(ctx context.Context, d DB, productID string, product billing.Product, price billing.Price, keyMode string) (Outcome, func(), error) { tx, err := db.Begin(ctx, d, db.Plain) if err != nil { return 0, noEnd, &Error{Stage: StageBegin, Err: err} } end := func() { tx.Rollback() } - if _, err := tx.ExecContext(ctx, `SELECT pg_advisory_xact_lock(hashtext($1)::bigint)`, productID); err != nil { + if err := db.Lock(ctx, tx, db.ProductSync, productID); err != nil { return 0, end, &Error{Stage: StageLock, Err: err} } outcome, err := enqueueLocked(ctx, tx, productID, product, price, keyMode) diff --git a/internal/integrations/stripe/workflows/invoice_reconcile.go b/internal/integrations/stripe/workflows/invoice_reconcile.go index 32071b97..abbff802 100644 --- a/internal/integrations/stripe/workflows/invoice_reconcile.go +++ b/internal/integrations/stripe/workflows/invoice_reconcile.go @@ -415,10 +415,10 @@ func (r *invoiceReconcile) converge(ctx context.Context, customerID string) (Inv } defer tx.Rollback() - // The form reconcile.go uses for subscriptions. A concurrent reconcile - // of the same invoice (two events, or an event and the sweep) waits - // here and then finds what this one wrote. - if _, err := tx.ExecContext(ctx, `SELECT pg_advisory_xact_lock(hashtext($1)::bigint)`, r.id); err != nil { + // The stripe-invoice family's lock (internal/db/locks.go). A concurrent + // reconcile of the same invoice (two events, or an event and the sweep) + // waits here and then finds what this one wrote. + if err := db.Lock(ctx, tx, db.Invoice, r.id); err != nil { return "", fmt.Errorf("stripe invoice %s (%s): advisory lock: %w", r.id, r.trig.Ref, err) } diff --git a/internal/integrations/stripe/workflows/testdata/invoice_payments/a-charge-without-a-currency.golden b/internal/integrations/stripe/workflows/testdata/invoice_payments/a-charge-without-a-currency.golden index 912b98ee..33fa4d34 100644 --- a/internal/integrations/stripe/workflows/testdata/invoice_payments/a-charge-without-a-currency.golden +++ b/internal/integrations/stripe/workflows/testdata/invoice_payments/a-charge-without-a-currency.golden @@ -1,7 +1,7 @@ == statements GetCustomerMappingByStripeCustomerID begin -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockInvoice GetInvoiceMappingByStripeID GetCustomerMappingByStripeCustomerID AssignNextInvoiceNumber diff --git a/internal/integrations/stripe/workflows/testdata/invoice_payments/a-failure-later-reported-succeeded.golden b/internal/integrations/stripe/workflows/testdata/invoice_payments/a-failure-later-reported-succeeded.golden index 93586265..39842407 100644 --- a/internal/integrations/stripe/workflows/testdata/invoice_payments/a-failure-later-reported-succeeded.golden +++ b/internal/integrations/stripe/workflows/testdata/invoice_payments/a-failure-later-reported-succeeded.golden @@ -1,7 +1,7 @@ == statements GetCustomerMappingByStripeCustomerID begin -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockInvoice GetInvoiceMappingByStripeID ConvergeInvoice ConvergeInvoiceMapping diff --git a/internal/integrations/stripe/workflows/testdata/invoice_payments/a-method-that-appears-later.golden b/internal/integrations/stripe/workflows/testdata/invoice_payments/a-method-that-appears-later.golden index 8674e563..c593aa5e 100644 --- a/internal/integrations/stripe/workflows/testdata/invoice_payments/a-method-that-appears-later.golden +++ b/internal/integrations/stripe/workflows/testdata/invoice_payments/a-method-that-appears-later.golden @@ -1,7 +1,7 @@ == statements GetCustomerMappingByStripeCustomerID begin -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockInvoice GetInvoiceMappingByStripeID ConvergeInvoice ConvergeInvoiceMapping diff --git a/internal/integrations/stripe/workflows/testdata/invoice_payments/earlier-payments-kept-beside-the-charges.golden b/internal/integrations/stripe/workflows/testdata/invoice_payments/earlier-payments-kept-beside-the-charges.golden index b739278c..3cbf7efb 100644 --- a/internal/integrations/stripe/workflows/testdata/invoice_payments/earlier-payments-kept-beside-the-charges.golden +++ b/internal/integrations/stripe/workflows/testdata/invoice_payments/earlier-payments-kept-beside-the-charges.golden @@ -1,7 +1,7 @@ == statements GetCustomerMappingByStripeCustomerID begin -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockInvoice GetInvoiceMappingByStripeID ConvergeInvoice ConvergeInvoiceMapping diff --git a/internal/integrations/stripe/workflows/testdata/invoice_payments/paid-without-a-charge-read-again.golden b/internal/integrations/stripe/workflows/testdata/invoice_payments/paid-without-a-charge-read-again.golden index c0198b7a..fc3e904c 100644 --- a/internal/integrations/stripe/workflows/testdata/invoice_payments/paid-without-a-charge-read-again.golden +++ b/internal/integrations/stripe/workflows/testdata/invoice_payments/paid-without-a-charge-read-again.golden @@ -1,7 +1,7 @@ == statements GetCustomerMappingByStripeCustomerID begin -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockInvoice GetInvoiceMappingByStripeID ConvergeInvoice ConvergeInvoiceMapping diff --git a/internal/integrations/stripe/workflows/testdata/invoice_payments/paid-without-a-charge.golden b/internal/integrations/stripe/workflows/testdata/invoice_payments/paid-without-a-charge.golden index 02629534..de64750c 100644 --- a/internal/integrations/stripe/workflows/testdata/invoice_payments/paid-without-a-charge.golden +++ b/internal/integrations/stripe/workflows/testdata/invoice_payments/paid-without-a-charge.golden @@ -1,7 +1,7 @@ == statements GetCustomerMappingByStripeCustomerID begin -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockInvoice GetInvoiceMappingByStripeID GetCustomerMappingByStripeCustomerID AssignNextInvoiceNumber diff --git a/internal/integrations/stripe/workflows/testdata/invoice_payments/the-same-read-again.golden b/internal/integrations/stripe/workflows/testdata/invoice_payments/the-same-read-again.golden index 7a82d67f..a1ce1905 100644 --- a/internal/integrations/stripe/workflows/testdata/invoice_payments/the-same-read-again.golden +++ b/internal/integrations/stripe/workflows/testdata/invoice_payments/the-same-read-again.golden @@ -1,7 +1,7 @@ == statements GetCustomerMappingByStripeCustomerID begin -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockInvoice GetInvoiceMappingByStripeID ConvergeInvoice ConvergeInvoiceMapping diff --git a/internal/integrations/stripe/workflows/testdata/invoice_payments/two-charges-in-the-same-second.golden b/internal/integrations/stripe/workflows/testdata/invoice_payments/two-charges-in-the-same-second.golden index 85f1349c..529ed3ef 100644 --- a/internal/integrations/stripe/workflows/testdata/invoice_payments/two-charges-in-the-same-second.golden +++ b/internal/integrations/stripe/workflows/testdata/invoice_payments/two-charges-in-the-same-second.golden @@ -1,7 +1,7 @@ == statements GetCustomerMappingByStripeCustomerID begin -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockInvoice GetInvoiceMappingByStripeID GetCustomerMappingByStripeCustomerID AssignNextInvoiceNumber diff --git a/internal/integrations/stripe/workflows/testdata/invoice_payments/two-declines-then-a-success.golden b/internal/integrations/stripe/workflows/testdata/invoice_payments/two-declines-then-a-success.golden index 4c195e2f..1ae16e2a 100644 --- a/internal/integrations/stripe/workflows/testdata/invoice_payments/two-declines-then-a-success.golden +++ b/internal/integrations/stripe/workflows/testdata/invoice_payments/two-declines-then-a-success.golden @@ -1,7 +1,7 @@ == statements GetCustomerMappingByStripeCustomerID begin -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockInvoice GetInvoiceMappingByStripeID GetCustomerMappingByStripeCustomerID AssignNextInvoiceNumber diff --git a/internal/integrations/stripe/workflows/testdata/invoice_sweep/every-kind-of-candidate.golden b/internal/integrations/stripe/workflows/testdata/invoice_sweep/every-kind-of-candidate.golden index f5abd705..2a995f50 100644 --- a/internal/integrations/stripe/workflows/testdata/invoice_sweep/every-kind-of-candidate.golden +++ b/internal/integrations/stripe/workflows/testdata/invoice_sweep/every-kind-of-candidate.golden @@ -7,7 +7,7 @@ stripe GET /v1/invoices/in_ stripe GET /v1/charges limit=100&payment_intent=pi_A GetCustomerMappingByStripeCustomerID begin -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockInvoice GetSubscriptionMappingByStripeID GetInvoiceMappingByStripeID ConvergeInvoice @@ -22,7 +22,7 @@ MarkInvoiceMappingReadBack stripe GET /v1/invoices/in_ GetCustomerMappingByStripeCustomerID begin -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockInvoice GetInvoiceMappingByStripeID ConvergeInvoice ConvergeInvoiceMapping @@ -31,7 +31,7 @@ commit stripe GET /v1/invoices/in_ GetCustomerMappingByStripeCustomerID begin -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockInvoice GetInvoiceMappingByStripeID ConvergeInvoice ConvergeInvoiceMapping diff --git a/internal/lint/lint.go b/internal/lint/lint.go index 1769138e..6a111254 100644 --- a/internal/lint/lint.go +++ b/internal/lint/lint.go @@ -59,6 +59,9 @@ type Config struct { // spec-purpose-placeholder rule; "-" disables it. // Default: openspec/specs. SpecsDir string + // LockRoot is the repository root the lock rules walk (ruleLockSites, + // ruleLockFamilies); "-" disables them. Default: the current directory. + LockRoot string } // Violation is a single rule failure with source location. @@ -99,6 +102,9 @@ func Run(cfg Config) (*Result, error) { if cfg.SpecsDir == "" { cfg.SpecsDir = "openspec/specs" } + if cfg.LockRoot == "" { + cfg.LockRoot = "." + } routes, err := extractRoutes(cfg.ServerDir) if err != nil { @@ -140,6 +146,10 @@ func Run(cfg Config) (*Result, error) { res.Violations = append(res.Violations, ruleSpecPurposePlaceholder(cfg.SpecsDir)...) } res.Violations = append(res.Violations, ruleHTMX4Scripts(filepath.Join(filepath.Dir(cfg.TemplateDir), "static"))...) + if cfg.LockRoot != "-" { + res.Violations = append(res.Violations, ruleLockSites(cfg.LockRoot)...) + res.Violations = append(res.Violations, ruleLockFamilies(cfg.LockRoot)...) + } sort.Slice(res.Violations, func(i, j int) bool { if res.Violations[i].File != res.Violations[j].File { diff --git a/internal/lint/locks.go b/internal/lint/locks.go new file mode 100644 index 00000000..ef69fe93 --- /dev/null +++ b/internal/lint/locks.go @@ -0,0 +1,437 @@ +// SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Commercial +// SPDX-FileCopyrightText: 2025-2026 Christian Galo + +package lint + +import ( + "errors" + "fmt" + "go/ast" + "go/parser" + "go/token" + "io/fs" + "os" + "path/filepath" + "regexp" + "slices" + "strconv" + "strings" + + "git.coopcloud.tech/wiki-cafe/member-console/internal/db" +) + +// The lock helpers (docs/database-locks.md): every advisory lock is taken in +// advisoryHelper, every pool row lock through the two queries poolHelper +// calls. +const ( + advisoryHelper = "internal/db/locks.go" + poolHelper = "internal/entitlements/tx.go" + locksDoc = "docs/database-locks.md" +) + +// advisoryCallRe is an advisory lock function named in SQL text, in any case: +// SQL function names are not case-sensitive. +var advisoryCallRe = regexp.MustCompile(`(?i)\bpg_(?:try_)?advisory_`) + +// rowLockRe is a row-locking clause. +var rowLockRe = regexp.MustCompile(`(?i)\bFOR\s+(?:UPDATE|NO\s+KEY\s+UPDATE|SHARE|KEY\s+SHARE)\b`) + +// poolLockQueries names, by query file, the one block of each that may lock +// a core.resource_pools row. +var poolLockQueries = map[string]string{ + "internal/entitlements/queries/resource_pools.sql": "LockResourcePool", + "internal/entitlements/queries/grants.sql": "LockGrantPool", +} + +// lockSeeds are the seeds that take the materialization rendezvous. +var lockSeeds = map[string]bool{ + "test/mockshot-seed.sql": true, + "test/seed/discourse/seed-discourse.sh": true, +} + +// advisoryMigrations are the two migrations that name the rendezvous, by file: +// a new migration's wrapper function would otherwise let any query lock +// outside the helpers. +var advisoryMigrations = map[string]bool{ + "internal/db/migrations/00018_entitlement_set_changes.sql": true, + "internal/db/migrations/00040_rendezvous_family_key.sql": true, +} + +// advisoryAllowed reports whether the file at rel may name an advisory lock +// function: the helper; the test kit's probe and its own wait tests, which +// lock integers no family builds; this rule and its tests, which name the +// rendezvous statement a seed may run and write fixtures that take locks; +// the two migrations of advisoryMigrations; and internal/db's tests, which +// run migrations at older versions against the key each version checks, +// every rendezvous use of which core.assert_rendezvous checks. +func advisoryAllowed(rel string) bool { + switch rel { + case advisoryHelper, "internal/testkit/dbtest/locks.go", "internal/testkit/dbtest/wait_test.go", + "internal/lint/locks.go", "internal/lint/locks_test.go": + return true + } + return advisoryMigrations[rel] || + (strings.HasPrefix(rel, "internal/db/") && strings.HasSuffix(rel, "_test.go")) +} + +// outsidePoolRule reports whether rel is outside the pool row rule: a test, +// the test kit, generated query code, a migration or a shell script. +func outsidePoolRule(rel string) bool { + return strings.HasSuffix(rel, "_test.go") || strings.HasPrefix(rel, "internal/testkit/") || + strings.HasSuffix(rel, ".sql.go") || strings.Contains(rel, "/migrations/") || strings.HasSuffix(rel, ".sh") +} + +// relPath is path relative to root, with forward slashes. +func relPath(root, path string) string { + rel, err := filepath.Rel(root, path) + if err != nil { + return filepath.ToSlash(path) + } + return filepath.ToSlash(rel) +} + +// lockFiles reads every .go, .sql and .sh file under cmd/, scripts/, +// internal/ and test/ of root, and the .go files of root itself. It does not +// enter a testdata directory, which holds fixtures and, under test/, the +// stack's own data, some of it owned by another user, so readTree, which +// enters every directory, cannot serve here. +func lockFiles(root string) ([]sourceFile, []Violation) { + var files []sourceFile + read := func(path string) error { + switch filepath.Ext(path) { + case ".go", ".sql", ".sh": + default: + return nil + } + src, err := os.ReadFile(path) + if errors.Is(err, fs.ErrNotExist) { + return nil + } + if err != nil { + return err + } + files = append(files, sourceFile{path: path, src: src}) + return nil + } + if entries, err := os.ReadDir(root); err == nil { + for _, e := range entries { + if !e.IsDir() && filepath.Ext(e.Name()) == ".go" { + if err := read(filepath.Join(root, e.Name())); err != nil { + return files, []Violation{unreadableFile(root, err)} + } + } + } + } + for _, dir := range []string{"cmd", "scripts", "internal", "test"} { + base := filepath.Join(root, dir) + if _, err := os.Stat(base); errors.Is(err, fs.ErrNotExist) { + continue + } + if err := walkLockDir(base, read); err != nil { + return files, []Violation{unreadableFile(base, err)} + } + } + return files, nil +} + +// walkLockDir calls read with every file under base and does not enter a +// testdata directory or a directory whose name begins with a dot, such as the +// agent runner's disposable copy of the tree, scripts/agent-runner/.code. +func walkLockDir(base string, read func(path string) error) error { + return filepath.WalkDir(base, func(path string, d fs.DirEntry, err error) error { + if err != nil { + return err + } + if d.IsDir() { + if d.Name() == "testdata" || strings.HasPrefix(d.Name(), ".") { + return filepath.SkipDir + } + return nil + } + return read(path) + }) +} + +// ruleLockSites reports every lock under the directories lockFiles reads +// taken outside its helper. +func ruleLockSites(root string) []Violation { + files, v := lockFiles(root) + key, _ := db.Materialization.Key() // a family with no id parts always builds its key + rendezvous := "pg_advisory_xact_lock(hashtextextended('" + key + "', 0))" + for _, f := range files { + rel := relPath(root, f.path) + v = append(v, advisorySites(f, rel, rendezvous)...) + if !outsidePoolRule(rel) { + v = append(v, poolSites(f, rel)...) + } + } + return v +} + +// advisorySites reports each Go string literal of f that names an advisory +// lock function where rel may not, and each line of the code of a .sql or +// .sh file that does (its comments removed, so a comment never matches); in +// a seed, each one but the rendezvous's, and the rendezvous anywhere but as +// the first statement after `BEGIN;`. A chain of literals joined by + is +// one text, so a function name in a literal of its own is found; Go +// comments are not strings and never match. A Go file the parser refuses is +// reported unread here and not again by poolSites. +func advisorySites(f sourceFile, rel, rendezvous string) []Violation { + if advisoryAllowed(rel) { + return nil + } + const message = "an advisory lock outside internal/db/locks.go: take it with db.Lock or db.LockShared and its family (docs/database-locks.md)" + if strings.HasSuffix(rel, ".go") { + strs, v := literalChains(f, "advisory locks") + for _, s := range strs { + if advisoryCallRe.MatchString(s.text) { + v = append(v, Violation{File: f.path, Line: s.line, Rule: "advisory-lock-site", Message: message}) + } + } + return v + } + shell := strings.HasSuffix(rel, ".sh") + var v []Violation + prev := "" // the last line of code before this one, trimmed + heredoc := "" // the delimiter of the shell heredoc the line is in, if any + for i, line := range strings.Split(string(f.src), "\n") { + inDoc := heredoc != "" + code := "" + if inDoc && strings.TrimSpace(line) == heredoc { + heredoc = "" + } else { + code = stripComment(line, shell && !inDoc) + } + if shell && !inDoc { + if m := heredocRe.FindStringSubmatch(line); m != nil { + heredoc = m[1] + } + } + if strings.HasPrefix(strings.TrimSpace(code), "--") { + code = "" // a SQL comment line, in a shell file's heredoc too + } + calls := advisoryCallRe.FindAllString(code, -1) + switch { + case len(calls) == 0: + case lockSeeds[rel] && len(calls) == 1 && strings.Contains(code, rendezvous) && strings.EqualFold(prev, "BEGIN;"): + case lockSeeds[rel] && len(calls) == 1 && strings.Contains(code, rendezvous): + v = append(v, Violation{File: f.path, Line: i + 1, Rule: "advisory-lock-site", + Message: "a seed takes the materialization rendezvous other than as the first statement after BEGIN (docs/database-locks.md)"}) + case lockSeeds[rel]: + v = append(v, Violation{File: f.path, Line: i + 1, Rule: "advisory-lock-site", + Message: "a seed takes an advisory lock other than the materialization rendezvous, " + rendezvous + " (internal/db/locks.go)"}) + default: + v = append(v, Violation{File: f.path, Line: i + 1, Rule: "advisory-lock-site", Message: message}) + } + if t := strings.TrimSpace(code); t != "" { + prev = t + } + } + return v +} + +// heredocRe is the opening of a shell heredoc, whose body is SQL text. +var heredocRe = regexp.MustCompile(`<<-?\s*['"]?([A-Za-z_]\w*)`) + +// stripComment returns line without its comment: in shell code, from a # +// that starts the line or follows a space or tab; in SQL text, from a --. +// A marker inside quotes is not a comment. +func stripComment(line string, shell bool) string { + var quote byte + for i := 0; i < len(line); i++ { + c := line[i] + switch { + case quote != 0: + if c == quote { + quote = 0 + } + case c == '\'' || (shell && c == '"'): + quote = c + case !shell && strings.HasPrefix(line[i:], "--"): + return line[:i] + case shell && c == '#' && (i == 0 || line[i-1] == ' ' || line[i-1] == '\t'): + return line[:i] + } + } + return line +} + +// poolSites reports, in a query file, each block that locks a pool row and +// is not the file's allowed one; in a Go file, each string literal, or chain +// of literals joined by +, that locks a pool row and, outside poolHelper, +// each call or method value of a pool lock query. +func poolSites(f sourceFile, rel string) []Violation { + if strings.HasSuffix(rel, ".sql") { + if !strings.Contains(rel, "/queries/") { + return nil + } + return poolQueryBlocks(f, rel) + } + v := poolStrings(f) + if rel == poolHelper { + return v + } + return append(v, poolCalls(f)...) +} + +// poolCalls reports each selector of LockResourcePool or LockGrantPool in a +// Go file, called or taken as a method value; a comment or a string naming +// one never matches. +func poolCalls(f sourceFile) []Violation { + fset := token.NewFileSet() + file, err := parser.ParseFile(fset, f.path, f.src, parser.SkipObjectResolution) + if err != nil { + return nil // advisorySites reports a file the parser refuses + } + var v []Violation + ast.Inspect(file, func(n ast.Node) bool { + if s, ok := n.(*ast.SelectorExpr); ok && (s.Sel.Name == "LockResourcePool" || s.Sel.Name == "LockGrantPool") { + v = append(v, Violation{File: f.path, Line: fset.Position(s.Sel.Pos()).Line, Rule: "pool-lock-site", + Message: "LockResourcePool or LockGrantPool called outside internal/entitlements/tx.go: lock pools through LockPools or LockPoolOfGrant (docs/go-conventions.md)"}) + } + return true + }) + return v +} + +// locksPool reports whether SQL text, its -- comments removed, names +// resource_pools and takes a row lock. +func locksPool(text string) bool { + var b strings.Builder + for _, line := range strings.Split(text, "\n") { + code, _, _ := strings.Cut(line, "--") + b.WriteString(code + "\n") + } + s := b.String() + return strings.Contains(s, "resource_pools") && rowLockRe.MatchString(s) +} + +// poolQueryBlocks reports each "-- name:" block of a query file that locks +// a pool row, unless it is the file's allowed block (poolLockQueries); text +// before the first block is a block with no name. +func poolQueryBlocks(f sourceFile, rel string) []Violation { + var v []Violation + name, start := "", 1 + var block strings.Builder + flush := func() { + if locksPool(block.String()) && (name == "" || name != poolLockQueries[rel]) { + v = append(v, Violation{File: f.path, Line: start, Rule: "pool-lock-site", + Message: fmt.Sprintf("query %s locks a core.resource_pools row: lock pools through LockPools or LockPoolOfGrant (docs/go-conventions.md)", name)}) + } + block.Reset() + } + for i, line := range strings.Split(string(f.src), "\n") { + if rest, ok := strings.CutPrefix(strings.TrimSpace(line), "-- name: "); ok { + flush() + name, _, _ = strings.Cut(rest, " ") + start = i + 1 + continue + } + block.WriteString(line + "\n") + } + flush() + return v +} + +// goString is a string literal of a Go file, or a chain of literals joined +// by +, with the line of its first literal. +type goString struct { + line int + text string +} + +// literalChains returns every goString of f. A file the parser refuses comes back +// as a violation naming what has not been checked. +func literalChains(f sourceFile, checked string) ([]goString, []Violation) { + fset := token.NewFileSet() + file, err := parser.ParseFile(fset, f.path, f.src, parser.SkipObjectResolution) + if err != nil { + return nil, []Violation{{File: f.path, Rule: "unreadable-file", Message: "the lint cannot parse this file, so it has not checked its " + checked + ": " + err.Error()}} + } + var out []goString + ast.Inspect(file, func(n ast.Node) bool { + e, ok := n.(ast.Expr) + if !ok { + return true + } + text, ok := literalChain(e) + if !ok { + return true + } + out = append(out, goString{line: fset.Position(e.Pos()).Line, text: text}) + return false + }) + return out, nil +} + +// literalChain is the text of a string literal, or of a chain of string +// literals joined by +; it is false when any leaf of the expression is not a +// string literal. +func literalChain(e ast.Expr) (string, bool) { + switch e := e.(type) { + case *ast.BasicLit: + if e.Kind != token.STRING { + return "", false + } + s, err := strconv.Unquote(e.Value) + return s, err == nil + case *ast.BinaryExpr: + if e.Op != token.ADD { + return "", false + } + l, ok := literalChain(e.X) + if !ok { + return "", false + } + r, ok := literalChain(e.Y) + return l + r, ok + } + return "", false +} + +// poolStrings reports each goString of a Go file that locks a pool row. +func poolStrings(f sourceFile) []Violation { + strs, _ := literalChains(f, "pool row locks") // advisorySites reports a file the parser refuses + var v []Violation + for _, s := range strs { + if locksPool(s.text) { + v = append(v, Violation{File: f.path, Line: s.line, Rule: "pool-lock-site", + Message: "a core.resource_pools row lock outside LockResourcePool and LockGrantPool: lock pools through LockPools or LockPoolOfGrant (docs/go-conventions.md)"}) + } + } + return v +} + +// ruleLockFamilies reports the family table of docs/database-locks.md when +// its names differ from db.Families, in a name or in order. The table is +// the one under "## Advisory key families"; each row's first cell is a +// family's name in backticks. +func ruleLockFamilies(root string) []Violation { + path := filepath.Join(root, locksDoc) + src, err := os.ReadFile(path) + if err != nil { + return []Violation{unreadableFile(path, err)} + } + var listed []string + heading, in := 0, false + for i, line := range strings.Split(string(src), "\n") { + switch { + case strings.HasPrefix(line, "## Advisory key families"): + in, heading = true, i+1 + case in && strings.HasPrefix(line, "## "): + in = false + case in && strings.HasPrefix(line, "| `"): + name, _, _ := strings.Cut(strings.TrimPrefix(line, "| `"), "`") + listed = append(listed, name) + } + } + want := make([]string, 0, len(db.Families)) + for _, f := range db.Families { + want = append(want, f.Name()) + } + if slices.Equal(listed, want) { + return nil + } + return []Violation{{File: path, Line: heading, Rule: "lock-families", + Message: fmt.Sprintf("the advisory key families table lists %v, want %v, the order of db.Families (internal/db/locks.go)", listed, want)}} +} diff --git a/internal/lint/locks_test.go b/internal/lint/locks_test.go new file mode 100644 index 00000000..7e1caf98 --- /dev/null +++ b/internal/lint/locks_test.go @@ -0,0 +1,190 @@ +// SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Commercial +// SPDX-FileCopyrightText: 2025-2026 Christian Galo + +package lint + +import ( + "os" + "path/filepath" + "slices" + "strings" + "testing" +) + +// writeTree writes each file of files, by path relative to the root, under a +// new temporary directory and returns the root. +func writeTree(t *testing.T, files map[string]string) string { + t.Helper() + root := t.TempDir() + for rel, content := range files { + path := filepath.Join(root, filepath.FromSlash(rel)) + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(path, []byte(content), 0o644); err != nil { + t.Fatal(err) + } + } + return root +} + +// findings is each violation as "rule relpath", in the order reported. +func findings(root string, vs []Violation) []string { + var out []string + for _, v := range vs { + out = append(out, v.Rule+" "+relPath(root, v.File)) + } + return out +} + +// goConst is a Go file holding expr as a top-level string constant. +func goConst(expr string) string { return "package x\n\nconst q = " + expr + "\n" } + +// goBody is a Go file holding stmt inside a function. +func goBody(stmt string) string { return "package x\n\nfunc f() {\n\t" + stmt + "\n}\n" } + +const ( + advisoryCall = "SELECT pg_advisory_xact_lock(1)" + rendezvousLock = "SELECT pg_advisory_xact_lock(hashtextextended('materialization:', 0));" + lockResource = "-- name: LockResourcePool :one\nSELECT pool_id FROM core.resource_pools\nWHERE pool_id = $1\nFOR UPDATE;\n" + lockGrant = "-- name: LockGrantPool :one\nSELECT rp.pool_id\nFROM core.resource_pools rp\nJOIN core.grants g ON g.granted_to_org_id = rp.org_id\nWHERE g.grant_id = $1\nFOR UPDATE OF rp;\n" + advisoryFound = "advisory-lock-site " + poolFound = "pool-lock-site " +) + +// lockSiteRow is one tree and the findings of the lock site rules on it; a +// row with unreadable names a directory, relative to the root, that the test +// makes unreadable before the rules run. +type lockSiteRow struct { + name string + files map[string]string + unreadable string + want []string +} + +// lockSiteRows is every row of TestLockSites. +func lockSiteRows() []lockSiteRow { + oneFile := func(name, path, content string, want ...string) lockSiteRow { + return lockSiteRow{name: name, files: map[string]string{path: content}, want: want} + } + return []lockSiteRow{ + oneFile("an advisory lock in a handler", "internal/server/x.go", goConst(`"SELECT pg_advisory_xact_lock(hashtext($1)::bigint)"`), advisoryFound+"internal/server/x.go"), + oneFile("an advisory lock in a query file", "internal/billing/queries/q.sql", advisoryCall+";\n", advisoryFound+"internal/billing/queries/q.sql"), + oneFile("a try lock in a test", "internal/fulfillment/x_test.go", goConst(`"SELECT pg_try_advisory_xact_lock(hashtext($1)::bigint)"`), advisoryFound+"internal/fulfillment/x_test.go"), + oneFile("the function name in a literal of its own", "internal/x/y.go", goBody(`lockFn := "pg_advisory_xact_lock_shared"`), advisoryFound+"internal/x/y.go"), + oneFile("the function name in capitals", "internal/x/z.go", goConst(`"SELECT PG_ADVISORY_XACT_LOCK(1)"`), advisoryFound+"internal/x/z.go"), + oneFile("the function name split across two literals", "internal/x/w.go", goConst(`"SELECT pg_advisory_" + "xact_lock(1)"`), advisoryFound+"internal/x/w.go"), + oneFile("the helper", "internal/db/locks.go", goConst(`"`+advisoryCall+`"`)), + oneFile("the test kit's probe", "internal/testkit/dbtest/locks.go", goConst(`"`+advisoryCall+`"`)), + oneFile("an internal/db test", "internal/db/x_test.go", goConst(`"`+advisoryCall+`"`)), + oneFile("the rendezvous's migration", "internal/db/migrations/00018_entitlement_set_changes.sql", advisoryCall+";\n"), + oneFile("a testdata file", "internal/x/testdata/a.sql", advisoryCall+";\n"), + oneFile("a migration that is not the rendezvous's", "internal/db/migrations/00041_x.sql", advisoryCall+";\n", advisoryFound+"internal/db/migrations/00041_x.sql"), + {name: "a command outside internal/", files: map[string]string{ + "cmd/x.go": goConst(`"` + advisoryCall + `"`), + "scripts/x.sh": "psql -c '" + advisoryCall + "'\n", + "main.go": goConst(`"` + advisoryCall + `"`), + }, want: []string{advisoryFound + "main.go", advisoryFound + "cmd/x.go", advisoryFound + "scripts/x.sh"}}, + {name: "a comment naming the function", files: map[string]string{ + "internal/x/y.go": "package x\n\n// pg_advisory_xact_lock, released at commit\nfunc f() {}\n", + "internal/x/z.sql": "-- pg_advisory_xact_lock(1)\nSELECT 1;\n", + "internal/x/z.sh": "# pg_advisory_xact_lock 1\n", + }}, + oneFile("a directory that begins with a dot", "scripts/agent-runner/.code/internal/x.go", goConst(`"`+advisoryCall+`"`)), + {name: "a testdata directory the lint cannot read", files: map[string]string{"test/testdata/stack/a.sql": advisoryCall + ";\n"}, unreadable: "test/testdata/stack"}, + oneFile("a seed with the rendezvous", "test/mockshot-seed.sql", "BEGIN;\n-- a comment\n\n"+rendezvousLock+"\n"), + oneFile("a shell seed with the rendezvous", "test/seed/discourse/seed-discourse.sh", "psql <<'SQL'\nBEGIN;\n-- a comment\n"+rendezvousLock+"\nSQL\n"), + oneFile("a seed that takes the rendezvous after an insert", "test/mockshot-seed.sql", + "BEGIN;\nINSERT INTO core.users (oidc_subject) VALUES ('x');\n"+rendezvousLock+"\n", advisoryFound+"test/mockshot-seed.sql"), + oneFile("a seed's rendezvous outside a transaction", "test/mockshot-seed.sql", rendezvousLock+"\n", advisoryFound+"test/mockshot-seed.sql"), + oneFile("a comment marker in a SQL string", "internal/x/q.sql", "SELECT '--', pg_advisory_xact_lock(1);\n", advisoryFound+"internal/x/q.sql"), + oneFile("$# before a lock in a shell line", "scripts/y.sh", "[ $# -eq 1 ] && psql -c 'SELECT pg_advisory_xact_lock(1)'\n", advisoryFound+"scripts/y.sh"), + oneFile("a hash in a heredoc", "scripts/z.sh", "psql <<'SQL'\nSELECT 1 # x, pg_advisory_xact_lock(1);\nSQL\n", advisoryFound+"scripts/z.sh"), + oneFile("a SQL comment in a shell file", "test/x.sh", "psql <<'SQL'\n-- takes pg_advisory_xact_lock first\nSELECT 1;\nSQL\n"), + oneFile("a seed with the old key", "test/mockshot-seed.sql", "SELECT pg_advisory_xact_lock(hashtextextended('entitlement_materialization', 0));\n", advisoryFound+"test/mockshot-seed.sql"), + } +} + +// poolSiteRows is the rows of TestLockSites about a pool row lock. +func poolSiteRows() []lockSiteRow { + oneFile := func(name, path, content string, want ...string) lockSiteRow { + return lockSiteRow{name: name, files: map[string]string{path: content}, want: want} + } + const resourcePools = "internal/entitlements/queries/resource_pools.sql" + return []lockSiteRow{ + {name: "the pool lock queries", files: map[string]string{resourcePools: lockResource, "internal/entitlements/queries/grants.sql": lockGrant}}, + oneFile("a pool lock query under another name", resourcePools, + lockResource+"\n-- name: LockPoolAgain :one\nSELECT pool_id FROM core.resource_pools WHERE pool_id = $1 FOR UPDATE;\n", poolFound+resourcePools), + oneFile("a key-share pool lock in another package", "internal/billing/queries/x.sql", + "-- name: ReadPool :one\nSELECT pool_id FROM core.resource_pools WHERE pool_id = $1 FOR KEY SHARE;\n", poolFound+"internal/billing/queries/x.sql"), + oneFile("a comment naming the lock", "internal/billing/queries/y.sql", + "-- name: ReadPool :one\n-- core.resource_pools is locked FOR UPDATE by the caller.\nSELECT pool_id FROM core.resource_pools WHERE pool_id = $1;\n"), + oneFile("a pool lock in a Go string", "internal/server/x.go", goConst("`SELECT 1 FROM core.resource_pools WHERE pool_id = $1 FOR UPDATE`"), poolFound+"internal/server/x.go"), + oneFile("a pool lock split across two literals", "internal/server/y.go", + goConst(`"SELECT 1 FROM core.resource_pools WHERE pool_id = $1 " + "FOR UPDATE"`), poolFound+"internal/server/y.go"), + oneFile("a pool lock in a test", "internal/x/y_test.go", goConst("`SELECT 1 FROM core.resource_pools WHERE pool_id = $1 FOR UPDATE`")), + oneFile("a pool lock in the test kit", "internal/testkit/world/x.go", goConst("`SELECT 1 FROM core.resource_pools WHERE pool_id = $1 FOR UPDATE`")), + oneFile("a pool query called outside its helper", "internal/fulfillment/x.go", goBody("q.LockResourcePool(ctx, id)"), poolFound+"internal/fulfillment/x.go"), + oneFile("a method value of the pool query", "internal/fulfillment/y.go", goBody("lock := entQ.LockResourcePool"), poolFound+"internal/fulfillment/y.go"), + oneFile("the pool helper", "internal/entitlements/tx.go", goBody("q.LockResourcePool(ctx, id)")), + oneFile("generated query code", "internal/entitlements/resource_pools.sql.go", goConst("`"+lockResource+"`")), + } +} + +// TestLockSites runs the lock site rules on a tree per row and compares what +// they report with the row's findings. +func TestLockSites(t *testing.T) { + for _, row := range append(lockSiteRows(), poolSiteRows()...) { + t.Run(strings.ReplaceAll(row.name, " ", "_"), func(t *testing.T) { checkLockSites(t, row) }) + } +} + +// checkLockSites is one row of TestLockSites. +func checkLockSites(t *testing.T, row lockSiteRow) { + t.Helper() + root := writeTree(t, row.files) + if row.unreadable != "" { + lock(t, filepath.Join(root, filepath.FromSlash(row.unreadable)), 0o755) + } + got := findings(root, ruleLockSites(root)) + if !slices.Equal(got, row.want) { + t.Errorf("%s: the lint reported %v, want %v", row.name, got, row.want) + } +} + +// familiesDoc is a docs/database-locks.md whose family table names names, in +// order, under the heading the rule reads. +func familiesDoc(names ...string) string { + var b strings.Builder + b.WriteString("# Locks\n\n## Advisory key families\n\n| Family | Id |\n|---|---|\n") + for _, n := range names { + b.WriteString("| `" + n + "` | none |\n") + } + b.WriteString("\n## The order every path keeps\n\n| `not-a-family` | none |\n") + return b.String() +} + +// TestLockFamilies runs the family table rule on a page per row. +func TestLockFamilies(t *testing.T) { + declared := []string{"materialization", "domains-registry", "fedwiki-workspace", "lifecycle", "stripe-subscription", "stripe-invoice", "stripe-product-sync", "desired-state"} + swapped := slices.Clone(declared) + swapped[4], swapped[5] = swapped[5], swapped[4] + const found = "lock-families docs/database-locks.md" + rows := []struct { + name string + doc string + want []string + }{ + {name: "the table as the code declares it", doc: familiesDoc(declared...)}, + {name: "a family missing", doc: familiesDoc(declared[:7]...), want: []string{found}}, + {name: "two families swapped", doc: familiesDoc(swapped...), want: []string{found}}, + {name: "no table", doc: "# Locks\n\nNothing here.\n", want: []string{found}}, + } + for _, row := range rows { + root := writeTree(t, map[string]string{"docs/database-locks.md": row.doc}) + got := findings(root, ruleLockFamilies(root)) + if !slices.Equal(got, row.want) { + t.Errorf("%s: the lint reported %v, want %v", row.name, got, row.want) + } + } +} diff --git a/internal/server/operator_org_type_change_test.go b/internal/server/operator_org_type_change_test.go index 2b03b4d6..aa193349 100644 --- a/internal/server/operator_org_type_change_test.go +++ b/internal/server/operator_org_type_change_test.go @@ -28,6 +28,7 @@ import ( "git.coopcloud.tech/wiki-cafe/member-console/internal/auth" "git.coopcloud.tech/wiki-cafe/member-console/internal/billing" + "git.coopcloud.tech/wiki-cafe/member-console/internal/db" "git.coopcloud.tech/wiki-cafe/member-console/internal/entitlements" "git.coopcloud.tech/wiki-cafe/member-console/internal/identity" "git.coopcloud.tech/wiki-cafe/member-console/internal/logging" @@ -836,7 +837,7 @@ func TestOrgTypeDefaultChange_ConcurrentCommitsSerialize(t *testing.T) { for range 2 { go func() { recs <- h.postRecorded(h.handler.CommitOrgTypeDefaultChange, f.orgType, form) }() } - dbtest.AwaitLockWait(t, database, dbtest.Backend{Query: "pg_advisory_xact_lock(hashtextextended", Waiters: 2}) + dbtest.AwaitLockWait(t, database, dbtest.Backend{Query: db.Materialization.Statement(), Waiters: 2}) if err := side.Rollback(); err != nil { t.Fatalf("release the rendezvous: %v", err) } diff --git a/internal/server/operator_sync_stripe_db_test.go b/internal/server/operator_sync_stripe_db_test.go index a67bd985..e69d6c17 100644 --- a/internal/server/operator_sync_stripe_db_test.go +++ b/internal/server/operator_sync_stripe_db_test.go @@ -31,6 +31,7 @@ import ( "git.coopcloud.tech/wiki-cafe/member-console/internal/auth" "git.coopcloud.tech/wiki-cafe/member-console/internal/billing" + "git.coopcloud.tech/wiki-cafe/member-console/internal/db" "git.coopcloud.tech/wiki-cafe/member-console/internal/entitlements" "git.coopcloud.tech/wiki-cafe/member-console/internal/identity" "git.coopcloud.tech/wiki-cafe/member-console/internal/integration" @@ -316,7 +317,7 @@ func (e *syncEnv) holdProductLock(productID string) (*sql.Tx, int) { e.t.Fatalf("begin the side transaction: %v", err) } e.t.Cleanup(func() { _ = side.Rollback() }) - if _, err := side.ExecContext(context.Background(), `SELECT pg_advisory_xact_lock(hashtext($1)::bigint)`, productID); err != nil { + if err := db.Lock(context.Background(), side, db.ProductSync, productID); err != nil { e.t.Fatalf("hold the product lock: %v", err) } var pid int diff --git a/internal/server/plan_acts_concurrency_db_test.go b/internal/server/plan_acts_concurrency_db_test.go index 833ee858..b386d689 100644 --- a/internal/server/plan_acts_concurrency_db_test.go +++ b/internal/server/plan_acts_concurrency_db_test.go @@ -37,7 +37,7 @@ import ( // The exclusive rendezvous statement as pg_stat_activity shows it, for a wait // on that mode. The text is not part of the shared one. -const exclusiveRendezvous = "pg_advisory_xact_lock(hashtextextended" +var exclusiveRendezvous = db.Materialization.Statement() // sideTx is a transaction on its own connection that holds row locks a test // releases when it chooses. It takes no rendezvous. diff --git a/internal/server/testdata/operator_sync_stripe/default_price_with_nothing_mapped.golden b/internal/server/testdata/operator_sync_stripe/default_price_with_nothing_mapped.golden index 01d8d90a..31a6ce78 100644 --- a/internal/server/testdata/operator_sync_stripe/default_price_with_nothing_mapped.golden +++ b/internal/server/testdata/operator_sync_stripe/default_price_with_nothing_mapped.golden @@ -18,7 +18,7 @@ ListPricesByProduct GetPriceMappingByPriceID SELECT COALESCE(error_message, '') begin -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockProductSync GetPriceMappingByPriceID GetProductMappingByProductID UpsertProductMapping diff --git a/internal/server/testdata/operator_sync_stripe/product_create_in_flight.golden b/internal/server/testdata/operator_sync_stripe/product_create_in_flight.golden index ac61ec70..5355f6d3 100644 --- a/internal/server/testdata/operator_sync_stripe/product_create_in_flight.golden +++ b/internal/server/testdata/operator_sync_stripe/product_create_in_flight.golden @@ -18,7 +18,7 @@ ListPricesByProduct GetPriceMappingByPriceID SELECT COALESCE(error_message, '') begin -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockProductSync GetPriceMappingByPriceID GetProductMappingByProductID GetProductByID diff --git a/internal/server/testdata/operator_sync_stripe/requested_price_of_a_reachable_product.golden b/internal/server/testdata/operator_sync_stripe/requested_price_of_a_reachable_product.golden index 246d706e..48385743 100644 --- a/internal/server/testdata/operator_sync_stripe/requested_price_of_a_reachable_product.golden +++ b/internal/server/testdata/operator_sync_stripe/requested_price_of_a_reachable_product.golden @@ -17,7 +17,7 @@ ListPricesByProduct GetPriceMappingByPriceID SELECT COALESCE(error_message, '') begin -SELECT pg_advisory_xact_lock(hashtext($1)::bigint) +LockProductSync GetPriceMappingByPriceID GetProductMappingByProductID UpsertPriceMapping diff --git a/internal/server/testdata/plan_acts/add-above-the-limit-default-sourced.json b/internal/server/testdata/plan_acts/add-above-the-limit-default-sourced.json index 937c8b72..ef1e7679 100644 --- a/internal/server/testdata/plan_acts/add-above-the-limit-default-sourced.json +++ b/internal/server/testdata/plan_acts/add-above-the-limit-default-sourced.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "rollback", "begin isolation level repeatable read read only", "commit" diff --git a/internal/server/testdata/plan_acts/add-above-the-limit.json b/internal/server/testdata/plan_acts/add-above-the-limit.json index d8abeb06..8fc7db53 100644 --- a/internal/server/testdata/plan_acts/add-above-the-limit.json +++ b/internal/server/testdata/plan_acts/add-above-the-limit.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "rollback", "begin isolation level repeatable read read only", "commit" diff --git a/internal/server/testdata/plan_acts/add-aligns-a-live-provision.json b/internal/server/testdata/plan_acts/add-aligns-a-live-provision.json index e4427219..f6a2e445 100644 --- a/internal/server/testdata/plan_acts/add-aligns-a-live-provision.json +++ b/internal/server/testdata/plan_acts/add-aligns-a-live-provision.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "LockResourcePool", "RecordPlanAct", "CreatePlanLadderTier", diff --git a/internal/server/testdata/plan_acts/add-collides-with-a-held-rung.json b/internal/server/testdata/plan_acts/add-collides-with-a-held-rung.json index b9d7e555..6393d1a9 100644 --- a/internal/server/testdata/plan_acts/add-collides-with-a-held-rung.json +++ b/internal/server/testdata/plan_acts/add-collides-with-a-held-rung.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "LockResourcePool", "RecordPlanAct", "CreatePlanLadderTier", @@ -21,7 +21,7 @@ "ListTiersByLadder", "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "GetProductByID", "CountLiveProductHolders", "GetLivePoolProvisionsByProductID", diff --git a/internal/server/testdata/plan_acts/add-product-held-twice.json b/internal/server/testdata/plan_acts/add-product-held-twice.json index 202b580b..77902f05 100644 --- a/internal/server/testdata/plan_acts/add-product-held-twice.json +++ b/internal/server/testdata/plan_acts/add-product-held-twice.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "LockResourcePool", "RecordPlanAct", "CreatePlanLadderTier", diff --git a/internal/server/testdata/plan_acts/add-product-unread.json b/internal/server/testdata/plan_acts/add-product-unread.json index b048340d..ceb955f9 100644 --- a/internal/server/testdata/plan_acts/add-product-unread.json +++ b/internal/server/testdata/plan_acts/add-product-unread.json @@ -11,7 +11,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "rollback", "begin isolation level repeatable read read only", "commit" diff --git a/internal/server/testdata/plan_acts/default-already-at.json b/internal/server/testdata/plan_acts/default-already-at.json index 2f868b33..a1f48790 100644 --- a/internal/server/testdata/plan_acts/default-already-at.json +++ b/internal/server/testdata/plan_acts/default-already-at.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "LockOrgType", "UpdateOrgTypeDefaultPlanLadder", "RecordPlanAct", @@ -16,13 +16,13 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "WritePlanActObligations", "commit", "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "LockResourcePool", "LockPlanActObligation", "SettlePlanActObligation", diff --git a/internal/server/testdata/plan_acts/default-gate-true-preview-empty.json b/internal/server/testdata/plan_acts/default-gate-true-preview-empty.json index 18fc772e..ab082ce0 100644 --- a/internal/server/testdata/plan_acts/default-gate-true-preview-empty.json +++ b/internal/server/testdata/plan_acts/default-gate-true-preview-empty.json @@ -11,7 +11,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "LockOrgType", "UpdateOrgTypeDefaultPlanLadder", "RecordPlanAct", diff --git a/internal/server/testdata/plan_acts/default-grandfather.json b/internal/server/testdata/plan_acts/default-grandfather.json index 6e583923..1e612134 100644 --- a/internal/server/testdata/plan_acts/default-grandfather.json +++ b/internal/server/testdata/plan_acts/default-grandfather.json @@ -10,7 +10,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "LockOrgType", "UpdateOrgTypeDefaultPlanLadder", "RecordPlanAct", @@ -18,13 +18,13 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "WritePlanActObligations", "commit", "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "LockResourcePool", "LockPlanActObligation", "CreateGrant", @@ -38,7 +38,7 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "LockResourcePool", "LockPlanActObligation", "CreateGrant", @@ -53,7 +53,7 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "LockResourcePool", "LockPlanActObligation", "SettlePlanActObligation", @@ -62,7 +62,7 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "commit", "begin isolation level repeatable read read only", "commit" diff --git a/internal/server/testdata/plan_acts/default-initiate-fails.json b/internal/server/testdata/plan_acts/default-initiate-fails.json index 691d5133..9b5d6298 100644 --- a/internal/server/testdata/plan_acts/default-initiate-fails.json +++ b/internal/server/testdata/plan_acts/default-initiate-fails.json @@ -12,7 +12,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "LockOrgType", "UpdateOrgTypeDefaultPlanLadder", "RecordPlanAct", @@ -20,13 +20,13 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "WritePlanActObligations", "commit", "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "LockResourcePool", "LockPlanActObligation", "CreateGrant", diff --git a/internal/server/testdata/plan_acts/default-listing-fails.json b/internal/server/testdata/plan_acts/default-listing-fails.json index 355f0a5c..f22106d7 100644 --- a/internal/server/testdata/plan_acts/default-listing-fails.json +++ b/internal/server/testdata/plan_acts/default-listing-fails.json @@ -12,7 +12,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "LockOrgType", "UpdateOrgTypeDefaultPlanLadder", "RecordPlanAct", @@ -20,7 +20,7 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "rollback", "begin isolation level repeatable read read only", "commit" diff --git a/internal/server/testdata/plan_acts/default-migrate-restore-fails.json b/internal/server/testdata/plan_acts/default-migrate-restore-fails.json index 67c79141..3911f625 100644 --- a/internal/server/testdata/plan_acts/default-migrate-restore-fails.json +++ b/internal/server/testdata/plan_acts/default-migrate-restore-fails.json @@ -12,7 +12,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "LockOrgType", "UpdateOrgTypeDefaultPlanLadder", "RecordPlanAct", @@ -20,13 +20,13 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "WritePlanActObligations", "commit", "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "LockResourcePool", "LockPlanActObligation", "SELECT core.end_conferral($1, $2, $3, $4, $5, $6, $7, $8)", diff --git a/internal/server/testdata/plan_acts/default-migrate.json b/internal/server/testdata/plan_acts/default-migrate.json index 24ac7b22..0398d35c 100644 --- a/internal/server/testdata/plan_acts/default-migrate.json +++ b/internal/server/testdata/plan_acts/default-migrate.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "LockOrgType", "UpdateOrgTypeDefaultPlanLadder", "RecordPlanAct", @@ -16,13 +16,13 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "WritePlanActObligations", "commit", "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "LockResourcePool", "LockPlanActObligation", "SELECT core.end_conferral($1, $2, $3, $4, $5, $6, $7, $8)", @@ -42,7 +42,7 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "commit", "begin isolation level repeatable read read only", "commit" diff --git a/internal/server/testdata/plan_acts/default-needs-disposition.json b/internal/server/testdata/plan_acts/default-needs-disposition.json index dbf4a373..b462a83e 100644 --- a/internal/server/testdata/plan_acts/default-needs-disposition.json +++ b/internal/server/testdata/plan_acts/default-needs-disposition.json @@ -9,7 +9,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "LockOrgType", "UpdateOrgTypeDefaultPlanLadder", "RecordPlanAct", diff --git a/internal/server/testdata/plan_acts/default-none-migrate.json b/internal/server/testdata/plan_acts/default-none-migrate.json index 35168a92..2be7bc55 100644 --- a/internal/server/testdata/plan_acts/default-none-migrate.json +++ b/internal/server/testdata/plan_acts/default-none-migrate.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "LockOrgType", "UpdateOrgTypeDefaultPlanLadder", "RecordPlanAct", @@ -16,13 +16,13 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "WritePlanActObligations", "commit", "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "LockResourcePool", "LockPlanActObligation", "SELECT core.end_conferral($1, $2, $3, $4, $5, $6, $7, $8)", @@ -34,7 +34,7 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "LockResourcePool", "LockPlanActObligation", "SettlePlanActObligation", @@ -43,7 +43,7 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "commit", "begin isolation level repeatable read read only", "commit" diff --git a/internal/server/testdata/plan_acts/default-preview-capped.json b/internal/server/testdata/plan_acts/default-preview-capped.json index 23b932e8..4139be72 100644 --- a/internal/server/testdata/plan_acts/default-preview-capped.json +++ b/internal/server/testdata/plan_acts/default-preview-capped.json @@ -11,7 +11,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "LockOrgType", "UpdateOrgTypeDefaultPlanLadder", "RecordPlanAct", diff --git a/internal/server/testdata/plan_acts/default-request-cap.json b/internal/server/testdata/plan_acts/default-request-cap.json index e574d2b2..58564ca2 100644 --- a/internal/server/testdata/plan_acts/default-request-cap.json +++ b/internal/server/testdata/plan_acts/default-request-cap.json @@ -11,19 +11,19 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "LockOrgType", "UpdateOrgTypeDefaultPlanLadder", "RecordPlanAct", "commit", "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "WritePlanActObligations", "commit", "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "LockResourcePool", "LockPlanActObligation", "SELECT core.end_conferral($1, $2, $3, $4, $5, $6, $7, $8)", diff --git a/internal/server/testdata/plan_acts/default-retry-nothing.json b/internal/server/testdata/plan_acts/default-retry-nothing.json index 849fc955..1bd0b562 100644 --- a/internal/server/testdata/plan_acts/default-retry-nothing.json +++ b/internal/server/testdata/plan_acts/default-retry-nothing.json @@ -7,7 +7,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "RecordPlanAct", "RequeuePlanActObligations", "rollback", diff --git a/internal/server/testdata/plan_acts/default-retry.json b/internal/server/testdata/plan_acts/default-retry.json index f34de9d3..fbe91f08 100644 --- a/internal/server/testdata/plan_acts/default-retry.json +++ b/internal/server/testdata/plan_acts/default-retry.json @@ -7,14 +7,14 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "RecordPlanAct", "RequeuePlanActObligations", "commit", "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "LockResourcePool", "LockPlanActObligation", "SELECT core.end_conferral($1, $2, $3, $4, $5, $6, $7, $8)", @@ -34,7 +34,7 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "commit", "begin isolation level repeatable read read only", "commit" diff --git a/internal/server/testdata/plan_acts/default-save-fails.json b/internal/server/testdata/plan_acts/default-save-fails.json index 20ef01d4..714bc6dd 100644 --- a/internal/server/testdata/plan_acts/default-save-fails.json +++ b/internal/server/testdata/plan_acts/default-save-fails.json @@ -13,7 +13,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "LockOrgType", "rollback", "begin isolation level repeatable read read only", diff --git a/internal/server/testdata/plan_acts/default-superseded.json b/internal/server/testdata/plan_acts/default-superseded.json index 8662fa05..5509123d 100644 --- a/internal/server/testdata/plan_acts/default-superseded.json +++ b/internal/server/testdata/plan_acts/default-superseded.json @@ -10,7 +10,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "LockOrgType", "UpdateOrgTypeDefaultPlanLadder", "RecordPlanAct", @@ -18,13 +18,13 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "WritePlanActObligations", "commit", "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "LockResourcePool", "LockPlanActObligation", "SELECT core.end_conferral($1, $2, $3, $4, $5, $6, $7, $8)", @@ -43,7 +43,7 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "LockResourcePool", "LockPlanActObligation", "SELECT core.end_conferral($1, $2, $3, $4, $5, $6, $7, $8)", @@ -63,7 +63,7 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "commit", "begin isolation level repeatable read read only", "commit" diff --git a/internal/server/testdata/plan_acts/default-untouched-outgoing.json b/internal/server/testdata/plan_acts/default-untouched-outgoing.json index c03bcfb5..a0048fb0 100644 --- a/internal/server/testdata/plan_acts/default-untouched-outgoing.json +++ b/internal/server/testdata/plan_acts/default-untouched-outgoing.json @@ -10,7 +10,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "LockOrgType", "UpdateOrgTypeDefaultPlanLadder", "RecordPlanAct", @@ -18,13 +18,13 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "WritePlanActObligations", "commit", "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "LockResourcePool", "LockPlanActObligation", "SettlePlanActObligation", diff --git a/internal/server/testdata/plan_acts/delete-bottom-tier.json b/internal/server/testdata/plan_acts/delete-bottom-tier.json index 96ded920..c58277d7 100644 --- a/internal/server/testdata/plan_acts/delete-bottom-tier.json +++ b/internal/server/testdata/plan_acts/delete-bottom-tier.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "RecordPlanAct", "DeleteTier", "UpdateTierRank", diff --git a/internal/server/testdata/plan_acts/delete-held-tier.json b/internal/server/testdata/plan_acts/delete-held-tier.json index 28823d99..53f9a36c 100644 --- a/internal/server/testdata/plan_acts/delete-held-tier.json +++ b/internal/server/testdata/plan_acts/delete-held-tier.json @@ -9,7 +9,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "rollback", "begin isolation level repeatable read read only", "commit" diff --git a/internal/server/testdata/plan_acts/delete-last-tier-of-a-default.json b/internal/server/testdata/plan_acts/delete-last-tier-of-a-default.json index fda7cbb3..cb774f48 100644 --- a/internal/server/testdata/plan_acts/delete-last-tier-of-a-default.json +++ b/internal/server/testdata/plan_acts/delete-last-tier-of-a-default.json @@ -9,7 +9,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "rollback", "begin isolation level repeatable read read only", "commit" diff --git a/internal/server/testdata/plan_acts/delete-org-types-unread.json b/internal/server/testdata/plan_acts/delete-org-types-unread.json index 61af754a..1eaed54e 100644 --- a/internal/server/testdata/plan_acts/delete-org-types-unread.json +++ b/internal/server/testdata/plan_acts/delete-org-types-unread.json @@ -11,7 +11,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "rollback", "begin isolation level repeatable read read only", "commit" diff --git a/internal/server/testdata/plan_acts/delete-suspended-holder.json b/internal/server/testdata/plan_acts/delete-suspended-holder.json index 3610d179..582cec5f 100644 --- a/internal/server/testdata/plan_acts/delete-suspended-holder.json +++ b/internal/server/testdata/plan_acts/delete-suspended-holder.json @@ -9,7 +9,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "rollback", "begin isolation level repeatable read read only", "commit" diff --git a/internal/server/testdata/plan_acts/delete-top-tier.json b/internal/server/testdata/plan_acts/delete-top-tier.json index d217016b..5c00ac8d 100644 --- a/internal/server/testdata/plan_acts/delete-top-tier.json +++ b/internal/server/testdata/plan_acts/delete-top-tier.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "RecordPlanAct", "DeleteTier", "UpdateTierRank", diff --git a/internal/server/testdata/plan_acts/remove-above-the-limit.json b/internal/server/testdata/plan_acts/remove-above-the-limit.json index 53ab8ec3..6d6c5066 100644 --- a/internal/server/testdata/plan_acts/remove-above-the-limit.json +++ b/internal/server/testdata/plan_acts/remove-above-the-limit.json @@ -9,7 +9,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "rollback", "begin isolation level repeatable read read only", "commit" diff --git a/internal/server/testdata/plan_acts/remove-keep-rank-0.json b/internal/server/testdata/plan_acts/remove-keep-rank-0.json index 394d62f8..674862e5 100644 --- a/internal/server/testdata/plan_acts/remove-keep-rank-0.json +++ b/internal/server/testdata/plan_acts/remove-keep-rank-0.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "LockResourcePool", "RecordPlanAct", "DeleteTier", diff --git a/internal/server/testdata/plan_acts/remove-last-tier-keep.json b/internal/server/testdata/plan_acts/remove-last-tier-keep.json index 72ff62a1..6e9f89d5 100644 --- a/internal/server/testdata/plan_acts/remove-last-tier-keep.json +++ b/internal/server/testdata/plan_acts/remove-last-tier-keep.json @@ -9,7 +9,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "rollback", "begin isolation level repeatable read read only", "commit" diff --git a/internal/server/testdata/plan_acts/remove-last-tier-migrate.json b/internal/server/testdata/plan_acts/remove-last-tier-migrate.json index 72ff62a1..6e9f89d5 100644 --- a/internal/server/testdata/plan_acts/remove-last-tier-migrate.json +++ b/internal/server/testdata/plan_acts/remove-last-tier-migrate.json @@ -9,7 +9,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "rollback", "begin isolation level repeatable read read only", "commit" diff --git a/internal/server/testdata/plan_acts/remove-migrate-pool-held-elsewhere.json b/internal/server/testdata/plan_acts/remove-migrate-pool-held-elsewhere.json index a0391b3f..e371bd56 100644 --- a/internal/server/testdata/plan_acts/remove-migrate-pool-held-elsewhere.json +++ b/internal/server/testdata/plan_acts/remove-migrate-pool-held-elsewhere.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "LockResourcePool", "RecordPlanAct", "DeleteTier", diff --git a/internal/server/testdata/plan_acts/remove-migrate-rank-0.json b/internal/server/testdata/plan_acts/remove-migrate-rank-0.json index eb8934b5..81142816 100644 --- a/internal/server/testdata/plan_acts/remove-migrate-rank-0.json +++ b/internal/server/testdata/plan_acts/remove-migrate-rank-0.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "LockResourcePool", "RecordPlanAct", "DeleteTier", diff --git a/internal/server/testdata/plan_acts/remove-needs-disposition.json b/internal/server/testdata/plan_acts/remove-needs-disposition.json index fdd5d957..8b827944 100644 --- a/internal/server/testdata/plan_acts/remove-needs-disposition.json +++ b/internal/server/testdata/plan_acts/remove-needs-disposition.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "LockResourcePool", "rollback", "begin isolation level repeatable read read only", diff --git a/internal/server/testdata/plan_acts/remove-no-holders-left.json b/internal/server/testdata/plan_acts/remove-no-holders-left.json index 0268418a..4dd4410c 100644 --- a/internal/server/testdata/plan_acts/remove-no-holders-left.json +++ b/internal/server/testdata/plan_acts/remove-no-holders-left.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "RecordPlanAct", "DeleteTier", "UpdateTierRank", diff --git a/internal/server/testdata/plan_acts/remove-org-types-unread.json b/internal/server/testdata/plan_acts/remove-org-types-unread.json index fc366177..815ee98d 100644 --- a/internal/server/testdata/plan_acts/remove-org-types-unread.json +++ b/internal/server/testdata/plan_acts/remove-org-types-unread.json @@ -11,7 +11,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "rollback", "begin isolation level repeatable read read only", "commit" diff --git a/internal/server/testdata/plan_acts/remove-other-source.json b/internal/server/testdata/plan_acts/remove-other-source.json index 2c90e1a9..b16fe578 100644 --- a/internal/server/testdata/plan_acts/remove-other-source.json +++ b/internal/server/testdata/plan_acts/remove-other-source.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "LockResourcePool", "RecordPlanAct", "DeleteTier", diff --git a/internal/server/testdata/plan_acts/remove-product-name-unread.json b/internal/server/testdata/plan_acts/remove-product-name-unread.json index 9a282c7a..a3c25c72 100644 --- a/internal/server/testdata/plan_acts/remove-product-name-unread.json +++ b/internal/server/testdata/plan_acts/remove-product-name-unread.json @@ -11,7 +11,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "rollback", "begin isolation level repeatable read read only", "commit" diff --git a/internal/server/testdata/plan_acts/remove-second-pool-fails.json b/internal/server/testdata/plan_acts/remove-second-pool-fails.json index 25285094..b0f7dd78 100644 --- a/internal/server/testdata/plan_acts/remove-second-pool-fails.json +++ b/internal/server/testdata/plan_acts/remove-second-pool-fails.json @@ -11,7 +11,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "LockResourcePool", "LockResourcePool", "RecordPlanAct", diff --git a/internal/server/testdata/plan_acts/remove-two-pools.json b/internal/server/testdata/plan_acts/remove-two-pools.json index d6916c2c..3d760cd9 100644 --- a/internal/server/testdata/plan_acts/remove-two-pools.json +++ b/internal/server/testdata/plan_acts/remove-two-pools.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "LockResourcePool", "LockResourcePool", "RecordPlanAct", diff --git a/internal/server/testdata/plan_acts/reorder-an-organization-fails.json b/internal/server/testdata/plan_acts/reorder-an-organization-fails.json index 98e04a8f..66a9cd26 100644 --- a/internal/server/testdata/plan_acts/reorder-an-organization-fails.json +++ b/internal/server/testdata/plan_acts/reorder-an-organization-fails.json @@ -12,7 +12,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "UpdateTierRank", "UpdateTierRank", "UpdateTierRank", @@ -22,13 +22,13 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "WritePlanActObligations", "commit", "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "LockResourcePool", "LockPlanActObligation", "CreateGrant", @@ -42,7 +42,7 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "LockResourcePool", "LockPlanActObligation", "CreateGrant", diff --git a/internal/server/testdata/plan_acts/reorder-cosmetic.json b/internal/server/testdata/plan_acts/reorder-cosmetic.json index 70a2c6e5..a4a5014e 100644 --- a/internal/server/testdata/plan_acts/reorder-cosmetic.json +++ b/internal/server/testdata/plan_acts/reorder-cosmetic.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "UpdateTierRank", "UpdateTierRank", "UpdateTierRank", diff --git a/internal/server/testdata/plan_acts/reorder-grandfather.json b/internal/server/testdata/plan_acts/reorder-grandfather.json index 91dc5c5b..3b81e25f 100644 --- a/internal/server/testdata/plan_acts/reorder-grandfather.json +++ b/internal/server/testdata/plan_acts/reorder-grandfather.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "UpdateTierRank", "UpdateTierRank", "UpdateTierRank", @@ -18,13 +18,13 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "WritePlanActObligations", "commit", "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "LockResourcePool", "LockPlanActObligation", "SettlePlanActObligation", @@ -32,7 +32,7 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "LockResourcePool", "LockPlanActObligation", "CreateGrant", @@ -46,7 +46,7 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "LockResourcePool", "LockPlanActObligation", "CreateGrant", @@ -62,7 +62,7 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "commit", "begin isolation level repeatable read read only", "commit" diff --git a/internal/server/testdata/plan_acts/reorder-listing-fails.json b/internal/server/testdata/plan_acts/reorder-listing-fails.json index ed83f2eb..4590b338 100644 --- a/internal/server/testdata/plan_acts/reorder-listing-fails.json +++ b/internal/server/testdata/plan_acts/reorder-listing-fails.json @@ -12,7 +12,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "UpdateTierRank", "UpdateTierRank", "UpdateTierRank", @@ -22,7 +22,7 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "rollback", "begin isolation level repeatable read read only", "commit" diff --git a/internal/server/testdata/plan_acts/reorder-migrate.json b/internal/server/testdata/plan_acts/reorder-migrate.json index c31a64a9..5dd801f8 100644 --- a/internal/server/testdata/plan_acts/reorder-migrate.json +++ b/internal/server/testdata/plan_acts/reorder-migrate.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "UpdateTierRank", "UpdateTierRank", "UpdateTierRank", @@ -18,13 +18,13 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "WritePlanActObligations", "commit", "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "LockResourcePool", "LockPlanActObligation", "SELECT core.end_conferral($1, $2, $3, $4, $5, $6, $7, $8)", @@ -44,7 +44,7 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "commit", "begin isolation level repeatable read read only", "commit" diff --git a/internal/server/testdata/plan_acts/reorder-needs-disposition.json b/internal/server/testdata/plan_acts/reorder-needs-disposition.json index 61150676..c30fa9cc 100644 --- a/internal/server/testdata/plan_acts/reorder-needs-disposition.json +++ b/internal/server/testdata/plan_acts/reorder-needs-disposition.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "UpdateTierRank", "UpdateTierRank", "UpdateTierRank", diff --git a/internal/server/testdata/plan_acts/reorder-rank-0-no-default.json b/internal/server/testdata/plan_acts/reorder-rank-0-no-default.json index 7e1c6229..a7689ba2 100644 --- a/internal/server/testdata/plan_acts/reorder-rank-0-no-default.json +++ b/internal/server/testdata/plan_acts/reorder-rank-0-no-default.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "UpdateTierRank", "UpdateTierRank", "UpdateTierRank", diff --git a/internal/server/testdata/plan_acts/reorder-record-fails.json b/internal/server/testdata/plan_acts/reorder-record-fails.json index e4681218..163fd3f0 100644 --- a/internal/server/testdata/plan_acts/reorder-record-fails.json +++ b/internal/server/testdata/plan_acts/reorder-record-fails.json @@ -11,7 +11,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "UpdateTierRank", "UpdateTierRank", "UpdateTierRank", diff --git a/internal/server/testdata/plan_acts/reorder-redrop.json b/internal/server/testdata/plan_acts/reorder-redrop.json index 08106a7f..a35a7997 100644 --- a/internal/server/testdata/plan_acts/reorder-redrop.json +++ b/internal/server/testdata/plan_acts/reorder-redrop.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "UpdateTierRank", "UpdateTierRank", "UpdateTierRank", diff --git a/internal/server/testdata/plan_acts/reorder-retry.json b/internal/server/testdata/plan_acts/reorder-retry.json index 65b66952..c69f93bb 100644 --- a/internal/server/testdata/plan_acts/reorder-retry.json +++ b/internal/server/testdata/plan_acts/reorder-retry.json @@ -7,14 +7,14 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "RecordPlanAct", "RequeuePlanActObligations", "commit", "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "LockResourcePool", "LockPlanActObligation", "SELECT core.end_conferral($1, $2, $3, $4, $5, $6, $7, $8)", @@ -34,7 +34,7 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "commit", "begin isolation level repeatable read read only", "commit" diff --git a/internal/server/testdata/plan_acts/reorder-stale.json b/internal/server/testdata/plan_acts/reorder-stale.json index baa97a78..5748b551 100644 --- a/internal/server/testdata/plan_acts/reorder-stale.json +++ b/internal/server/testdata/plan_acts/reorder-stale.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "UpdateTierRank", "UpdateTierRank", "UpdateTierRank", diff --git a/internal/server/testdata/plan_acts/reorder-two-types.json b/internal/server/testdata/plan_acts/reorder-two-types.json index adbe7904..1ebf7e94 100644 --- a/internal/server/testdata/plan_acts/reorder-two-types.json +++ b/internal/server/testdata/plan_acts/reorder-two-types.json @@ -8,7 +8,7 @@ "statements": [ "begin", "SET LOCAL lock_timeout = '5s'", - "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))", + "LockMaterialization", "UpdateTierRank", "UpdateTierRank", "UpdateTierRank", @@ -18,13 +18,13 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "WritePlanActObligations", "commit", "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "LockResourcePool", "LockPlanActObligation", "CreateGrant", @@ -38,7 +38,7 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "LockResourcePool", "LockPlanActObligation", "CreateGrant", @@ -53,7 +53,7 @@ "begin", "SET LOCAL transaction_timeout = '2s'", "SET LOCAL lock_timeout = '500ms'", - "SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))", + "LockMaterializationShared", "commit", "begin isolation level repeatable read read only", "commit" diff --git a/internal/testkit/dbtest/locks.go b/internal/testkit/dbtest/locks.go new file mode 100644 index 00000000..a95461dd --- /dev/null +++ b/internal/testkit/dbtest/locks.go @@ -0,0 +1,25 @@ +// SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Commercial +// SPDX-FileCopyrightText: 2025-2026 Christian Galo + +package dbtest + +import ( + "context" + "database/sql" + + "git.coopcloud.tech/wiki-cafe/member-console/internal/db" +) + +// TryLock takes f's exclusive lock on the id on tx without waiting and +// reports whether it was free; a free lock stays with tx until tx ends. A +// test probes with it whether the code under test holds a family's lock, +// under the key the code builds. +func TryLock(ctx context.Context, tx *sql.Tx, f db.Family, id ...string) (bool, error) { + key, err := f.Key(id...) + if err != nil { + return false, err + } + var free bool + err = tx.QueryRowContext(ctx, `SELECT pg_try_advisory_xact_lock(hashtextextended($1, 0))`, key).Scan(&free) + return free, err +} diff --git a/internal/testkit/dbtest/locks_test.go b/internal/testkit/dbtest/locks_test.go new file mode 100644 index 00000000..bcbc8294 --- /dev/null +++ b/internal/testkit/dbtest/locks_test.go @@ -0,0 +1,38 @@ +// SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Commercial +// SPDX-FileCopyrightText: 2025-2026 Christian Galo + +package dbtest_test + +import ( + "context" + "testing" + + "github.com/google/uuid" + + "git.coopcloud.tech/wiki-cafe/member-console/internal/db" + "git.coopcloud.tech/wiki-cafe/member-console/internal/testkit/dbtest" +) + +// TestTryLock is the kit's own test of TryLock: a key nobody holds is free, +// a key another transaction holds through db.Lock is not, and an id the +// family refuses is an error. +func TestTryLock(t *testing.T) { + database := dbtest.Open(t) + ctx := context.Background() + id := "sub_" + uuid.NewString() + + probe := dbtest.Tx(t, database).Tx() + if free, err := dbtest.TryLock(ctx, probe, db.Subscription, "sub_"+uuid.NewString()); err != nil || !free { + t.Errorf("TryLock found a fresh key held (%v)", err) + } + holder := dbtest.Tx(t, database).Tx() + if err := db.Lock(ctx, holder, db.Subscription, id); err != nil { + t.Fatalf("hold the key: %v", err) + } + if free, err := dbtest.TryLock(ctx, probe, db.Subscription, id); err != nil || free { + t.Errorf("TryLock found a held key free (%v)", err) + } + if _, err := dbtest.TryLock(ctx, probe, db.Subscription, ""); err == nil { + t.Errorf("TryLock took an empty id") + } +} diff --git a/internal/workflows/desiredstate/export_test.go b/internal/workflows/desiredstate/export_test.go index 594b04c8..b7885c2d 100644 --- a/internal/workflows/desiredstate/export_test.go +++ b/internal/workflows/desiredstate/export_test.go @@ -9,6 +9,7 @@ package desiredstate import ( "context" + "database/sql" "git.coopcloud.tech/wiki-cafe/member-console/internal/integration" ) @@ -34,3 +35,10 @@ func SetExpandHook(fn func(kind, subject string) error) { expandHook = fn } func (a *Activities) WritePushAnswerForTest(ctx context.Context, addr Address, recordID string, pushed int64, hash []byte, result integration.ApplyResult) (PushRound, error) { return a.writePushAnswer(ctx, addr, recordID, pushed, hash, result) } + +// LockAddress runs lockAddress for a on tx, as a recompute's transaction does +// before it reads anything. +func LockAddress(ctx context.Context, tx *sql.Tx, a Address) error { + _, err := lockAddress(ctx, tx, integration.New(tx), a) + return err +} diff --git a/internal/workflows/desiredstate/push.go b/internal/workflows/desiredstate/push.go index 7684184b..2537c1fe 100644 --- a/internal/workflows/desiredstate/push.go +++ b/internal/workflows/desiredstate/push.go @@ -147,7 +147,7 @@ func (a *Activities) writePushAnswer(ctx context.Context, addr address, recordID } defer tx.Rollback() q := integration.New(tx) - rec, err := lockAddress(ctx, q, addr) + rec, err := lockAddress(ctx, tx, q, addr) if err != nil { return PushRound{}, err } diff --git a/internal/workflows/desiredstate/readback.go b/internal/workflows/desiredstate/readback.go index 729f070e..76b71221 100644 --- a/internal/workflows/desiredstate/readback.go +++ b/internal/workflows/desiredstate/readback.go @@ -176,7 +176,7 @@ func (a *Activities) readBackItem(ctx context.Context, addr address, item integr return res, nil } - rec, err := lockAddress(ctx, q, addr) + rec, err := lockAddress(ctx, tx, q, addr) if err != nil { return res, err } diff --git a/internal/workflows/desiredstate/record.go b/internal/workflows/desiredstate/record.go index fd1df350..74a50f54 100644 --- a/internal/workflows/desiredstate/record.go +++ b/internal/workflows/desiredstate/record.go @@ -72,10 +72,8 @@ var afterAddressLock func(address) // every write to a record's content or generation and to its projection // (design D9, D14), then locks the record row when one exists. The returned // record is nil when the address has none. -func lockAddress(ctx context.Context, q *integration.Queries, a address) (*integration.DesiredStateRecord, error) { - if err := q.LockDesiredStateAddress(ctx, integration.LockDesiredStateAddressParams{ - ConnectionID: a.ConnectionID, RecipientKind: string(a.Kind), RecipientID: a.RecipientID, - }); err != nil { +func lockAddress(ctx context.Context, tx *sql.Tx, q *integration.Queries, a address) (*integration.DesiredStateRecord, error) { + if err := db.Lock(ctx, tx, db.DesiredState, a.ConnectionID, string(a.Kind), a.RecipientID); err != nil { return nil, fmt.Errorf("lock address: %w", err) } if afterAddressLock != nil { @@ -158,7 +156,7 @@ func recomputeAddress(ctx context.Context, db *sql.DB, a address) (recomputed, e } defer tx.Rollback() q := integration.New(tx) - rec, err := lockAddress(ctx, q, a) + rec, err := lockAddress(ctx, tx, q, a) if err != nil { return recomputed{}, err } diff --git a/internal/workflows/desiredstate/record_lock_db_test.go b/internal/workflows/desiredstate/record_lock_db_test.go new file mode 100644 index 00000000..f05d28e4 --- /dev/null +++ b/internal/workflows/desiredstate/record_lock_db_test.go @@ -0,0 +1,40 @@ +// SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Commercial +// SPDX-FileCopyrightText: 2025-2026 Christian Galo + +package desiredstate_test + +import ( + "context" + "testing" + + "github.com/google/uuid" + + "git.coopcloud.tech/wiki-cafe/member-console/internal/db" + "git.coopcloud.tech/wiki-cafe/member-console/internal/integration" + "git.coopcloud.tech/wiki-cafe/member-console/internal/testkit/dbtest" + "git.coopcloud.tech/wiki-cafe/member-console/internal/workflows/desiredstate" +) + +// TestLockAddressTakesTheAddressKey holds that lockAddress takes the +// desired-state family's lock on the address's connection, kind and +// recipient: while a transaction has run it for an address with no record, +// another transaction cannot take the family's lock on that address. +func TestLockAddressTakesTheAddressKey(t *testing.T) { + database := dbtest.Open(t) + ctx := context.Background() + conn, recipient := uuid.NewString(), uuid.NewString() + addr := desiredstate.Address{ConnectionID: conn, Kind: integration.RecipientPerson, RecipientID: recipient} + + holder := dbtest.Tx(t, database).Tx() + if err := desiredstate.LockAddress(ctx, holder, addr); err != nil { + t.Fatalf("lock the address: %v", err) + } + side := dbtest.Tx(t, database).Tx() + free, err := dbtest.TryLock(ctx, side, db.DesiredState, conn, "person", recipient) + if err != nil { + t.Fatalf("probe the address: %v", err) + } + if free { + t.Errorf("lockAddress left the address %v free", addr) + } +} diff --git a/openspec/changes/advisory-keys-name-their-family/.openspec.yaml b/openspec/changes/advisory-keys-name-their-family/.openspec.yaml new file mode 100644 index 00000000..4bfa6709 --- /dev/null +++ b/openspec/changes/advisory-keys-name-their-family/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-10-30 diff --git a/openspec/changes/advisory-keys-name-their-family/design.md b/openspec/changes/advisory-keys-name-their-family/design.md new file mode 100644 index 00000000..6a25a99e --- /dev/null +++ b/openspec/changes/advisory-keys-name-their-family/design.md @@ -0,0 +1,233 @@ +## Context + +Every advisory lock the console takes is transaction-scoped (`pg_advisory_xact_lock` or its `_shared` form), so Postgres releases it at commit or rollback. Postgres takes one 64-bit key for such a lock; `pg_locks` shows it split across `classid` (high 32 bits) and `objid` (low 32 bits) with `objsubid = 1`. The key is built from the object's identity by `hashtext` (32 bits, widened) or `hashtextextended` (64 bits). + +This change is the second of two for #186. The first, `transactions-open-through-the-runner`, opens the remaining hand-opened transactions through `db.InTx` and `db.Begin`, adds the lint that refuses `BeginTx`, and fixes the ladder reorder's order; this change assumes it has landed, including the registry's `registryKind`. + +### The advisory locks at HEAD + +| Lock | Key today | Taken by | Before and after it | Transaction kind | +|---|---|---|---|---| +| Materialization rendezvous | `hashtextextended('entitlement_materialization', 0)`, from `rendezvousName` in `internal/entitlements/tx.go`; `core.assert_rendezvous` (migration 00018) checks the same literal | `entitlements.Materializing` and `MaterializingWithin` (shared), `entitlements.RuleChange` (exclusive), as the first statement after `SET LOCAL`; `test/mockshot-seed.sql` and `test/seed/discourse/seed-discourse.sh` (exclusive, as the first statement after their `BEGIN`, before any insert and before `core.commit_rule_change`) | first; then the subscription lock (the reconcile), the set and rule rows (a rule change), the org type row (T0), pool rows | the two rendezvous kinds; the seeds' `psql` transaction | +| Domains registry | the constant `registryAllocationLock`, `0x646f6d61696e73` | `domains.Registry.WithLock`, through `registryKind` after the first change | first; then claim and placement rows, and one site `FOR SHARE` for FedWiki's creators | `registryKind` (server default options) | +| FedWiki workspace | `hashtext()::bigint` | `withWorkspaceTx` in `internal/integrations/fedwiki/web/site_actions.go` | first; then the lifecycle instance locks (activated site, then parked site), the swap policy row | `db.ReadCommitted` | +| Lifecycle instance | `hashtextextended('lifecycle:' \|\| ::uuid::text, 0)`, the `LockLifecycleInstance` query | `integration.lockLatest` (both recording functions); FedWiki's `recordSiteRequestInTx` and `CheckSwapRequestSites` (`store/request_intake.go`); `usage.Hold`'s `lockRequest`; `finishAllWith`'s `lockOutcomes`; `purgeArchivedSite` | after the workspace lock when the handler holds it; before site rows, the usage row and request rows; several in the order the requests were recorded | the caller's (`db.Plain` in `inOwnTx`; `db.ReadCommitted` in intake, the gate, outcomes and the purge) | +| Subscription | `hashtext()::bigint` | `reconcileInTx` and `supersedeReleasedDowngrades` (`internal/fulfillment/reconcile.go`) | after the shared rendezvous, before pool rows (the reconcile); alone (supersede) | `entitlements.Materializing`; `db.Plain` | +| Invoice | `hashtext()::bigint` | `invoiceReconcile.converge` | first; then invoice, line and payment rows | `db.Plain` through `db.Begin` | +| Product sync | `hashtext()::bigint` | `catalogsync.enqueue` | first; then mapping rows and the outbox inserts | `db.Plain` through `db.Begin` | +| Desired-state address | `hashtextextended('desired-state:' \|\| ::uuid::text \|\| ':' \|\| \|\| ':' \|\| ::uuid::text, 0)`, the `LockDesiredStateAddress` query | `lockAddress` (`internal/workflows/desiredstate/record.go`), from `recomputeAddress`, `writePushAnswer` and `readBackItem` | first; then the record row | `db.ReadCommitted` through `db.Begin` | + +No path takes two of these families except the rendezvous before the subscription, the workspace before lifecycle instances, and several lifecycle instances. + +### The row locks at HEAD + +`docs/database-locks.md` lists every row lock and its place in the order. The ones this change touches or orders are the pool row, `FOR UPDATE`, taken by `LockPools` and `LockPoolOfGrant` in `internal/entitlements/tx.go` (the only Go callers of the `LockResourcePool` and `LockGrantPool` queries) and, in SQL, by `core.confer`, `core.end_conferral`, `core.align_conferral_shape`, `core.update_conferral_bounds` and `core.settle_obligation`; and the pool row `FOR KEY SHARE` in `core.write_plan_act_obligations`. D4's third rule orders the rest after them: the set and rule rows of a rule change before pool rows, then a grant's lineage head, obligation rows and a tier's product row. The page's lifecycle row names four takers; the three FedWiki intake and gate takers are missing from it, and so are the two seeds. + +## Goals / Non-Goals + +**Goals:** +- One function builds every advisory key, from a family and an id, and one pair takes every advisory lock; nothing else names an advisory lock function. +- The families and the order every path keeps are written once, in `docs/database-locks.md`, and the lint holds the page's family table to the code. +- The pool row lock stays in its helpers, and the lint holds it there. +- A lock helper takes a transaction, so a connection or a database handle does not compile; recording a lifecycle request on a connection opens a transaction on it. +- The deploy of the changed keys follows the upgrade rule of D8. + +**Non-Goals:** +- `BeginTx` and the ladder reorder (the first change). +- Row locks other than the pool's. +- Checking the order at run time. + +## Decisions + +### D1. The families and the key + +Every key is `hashtextextended( || ':' || , 0)`, where `` is the id's parts joined by `:`. A family with one lock for the whole database has an id of no parts, so its key text is the family's name and a colon. A UUID part is parsed and written in canonical form, as Postgres writes `uuid::text`, so two spellings of one id take one key; a text part is used as given and may be neither empty nor contain `:`, so no part can run into the next. + +| Order | Family | Id | Statement name | Key today | Changes | +|---|---|---|---|---|---| +| 1 | `materialization` | none | `LockMaterialization`, `LockMaterializationShared` | `hashtextextended('entitlement_materialization', 0)` | yes | +| 2 | `domains-registry` | none | `LockDomainsRegistry` | the constant `0x646f6d61696e73` | yes | +| 3 | `fedwiki-workspace` | workspace id (UUID) | `LockFedWikiWorkspace` | `hashtext(id)::bigint` | yes | +| 4 | `lifecycle` | instance id (UUID) | `LockLifecycleInstance` | `hashtextextended('lifecycle:' \|\| id::uuid::text, 0)` | no | +| 5 | `stripe-subscription` | Stripe subscription id | `LockSubscription` | `hashtext(id)::bigint` | yes | +| 6 | `stripe-invoice` | Stripe invoice id | `LockInvoice` | `hashtext(id)::bigint` | yes | +| 7 | `stripe-product-sync` | product id (UUID) | `LockProductSync` | `hashtext(id)::bigint` | yes | +| 8 | `desired-state` | connection id (UUID), recipient kind, recipient id (UUID) | `LockDesiredStateAddress` | `hashtextextended('desired-state:' \|\| conn::uuid::text \|\| ':' \|\| kind \|\| ':' \|\| recipient::uuid::text, 0)` | no | + +The prefixed `hashtextextended` form is the one two families already use; extending it keeps their keys, byte for byte, and gives every family a 64-bit key whose text cannot equal another family's. A family is named for what one lock covers, in lower case with hyphens, as `desired-state` already is; the three that lock a Stripe id or a Stripe sync carry the owner's prefix, as `fedwiki-workspace` does, so a rename later does not cost another key change and another stop-first deploy. A statement's name is the sqlc query it replaced (`LockLifecycleInstance`, `LockDesiredStateAddress`), or `Lock` and the family in CamelCase (`LockDomainsRegistry`, `LockFedWikiWorkspace`), the three Stripe families without their `stripe-` prefix (`LockSubscription`, `LockInvoice`, `LockProductSync`); the Go identifiers (`db.Subscription`, `db.ProductSync`) follow the statement names. These names are decided; they are the vocabulary of `docs/database-locks.md` and of every statement trace. + +Each family has one statement per mode, and the statement's first line is a sqlc-style name: `-- name: LockSubscription :exec\nSELECT pg_advisory_xact_lock(hashtextextended($1, 0))`. Once every family runs the same SQL, the name is what tells them apart: `pg_stat_activity.query` shows it for a backend waiting on the lock, and `dbtest.Counted` names a statement by such a line, so the statement traces and goldens read `LockSubscription` where they read `SELECT pg_advisory_xact_lock(hashtext($1)::bigint)`. The two unchanged families keep the names of the queries they replace, so the goldens that record them do not change. + +**The order** of the table is the order a transaction takes families: the two whole-database locks first, the rendezvous before the registry, then the per-object families, the coarse before the fine (a workspace before its sites' instances). Only three pairs are taken together today (rendezvous then subscription, workspace then lifecycle, lifecycle then lifecycle), and all three already follow it; the rest of the order is fixed so that a path that later combines two families has one answer. + +**Which keys change, and what a rolling deploy would do.** Six families change their key. Two processes of different releases, one taking the old key and one the new for the same object, hold no common lock: + +- `materialization`: the migration of D5 changes the key `core.assert_rendezvous` checks, so the side whose key the database no longer checks refuses every materialization and rule change (`materialization_rendezvous_missing`): requests fail, nothing is written from a stale rule. +- `domains-registry`: two allocations, one per process, run at once; the subtree-disjointness checks span rows no constraint compares, so two overlapping claims can both commit. +- `fedwiki-workspace`: two tabs' rotations of one workspace interleave the cooldown check and stamp, or both pick a site to park. +- `stripe-subscription`: two reconciles of one subscription race the exclusion constraint on its periods; one fails and is retried. +- `stripe-invoice`: two reconciles of one invoice can both create it, refused by a unique index and retried, or write its lines twice. +- `stripe-product-sync`: two syncs of one product can both enqueue the creates, which Stripe then runs as two products under two outbox keys. + +**The deploy rule** is D8's. No code takes both keys for a transition: that would be compatibility code for one deployment's history. + +**Rejected alternatives.** +- *Two 32-bit keys, `pg_advisory_xact_lock(, hashtext())`.* `pg_locks.classid` would show the family, and families could never collide. But every key would change, the lifecycle and desired-state keys included, and ids would hash to 32 bits. The statement's name line gives `pg_stat_activity` the family at no cost to the key. +- *Keeping the rendezvous's literal as a special case.* It would leave two shapes of key and a name with no family. The migration it needs is one function (D5). +- *Checking the order at run time.* With one 64-bit key, a lock's family cannot be read back from `pg_locks`; a check would need the two-key form or per-transaction state the runner does not keep, and in tests an extra statement per lock that every trace golden would record and production would not run. The order is held by the written table, the lint that ties it to the code, and the path tests that wait on each pair. + +### D2. Where the helpers live and what they take + +**Advisory locks** live in one file, `internal/db/locks.go`, beside the transaction runner every owner already imports (`internal/db` imports no other package of the console). It declares `Family`, a struct whose fields are unexported, so only this file declares one, the eight families and `Families` in order, and: + +- `(Family) Key(id ...string) (string, error)`: the key text; it refuses a family it does not know, a wrong number of parts, an empty text part, a text part that holds a `:` and a UUID part that does not parse. +- `Lock(ctx, tx *sql.Tx, f Family, id ...string) error` and `LockShared(...)`: the key, then the statement on `tx`. +- `(Family) Statement()` and `SharedStatement()`: the statement text, which a test waits on (`dbtest.Backend{Query: db.Materialization.Statement()}`). +- `(Family) Name()`: for the lint's table check. + +Both take a `*sql.Tx`: a transaction-scoped lock taken on a `*sql.DB` or a `*sql.Conn` outside a transaction ends with its statement, so passing either does not compile, as `usage.Recount` and `usage.Hold` already require (`docs/go-conventions.md`). The id is variadic: a family's arity is checked when the key is built, before any statement, and a unit test covers each family's arity. + +**The pool row lock** stays in `internal/entitlements/tx.go`: `LockPools` and `LockPoolOfGrant`, methods of `*Queries`, are already the only Go callers of `LockResourcePool` and `LockGrantPool`. The queries are entitlements' sqlc queries against entitlements' table, and two tests fail one pool's lock by wrapping the querier the plan act runner holds (`tier_add_db_test.go`, `operator_plan_ladder_tier_add_test.go`); moving them to `internal/db` as raw SQL would break both. They are the one exception to the rule that a lock helper takes a `*sql.Tx`, which the spec states for advisory locks. Their receiver stays `*Queries`, because of the plan act runner's querier seam above and because every caller builds it from a transaction (the grant acts receive a `*sql.Tx`, settlement and the reconcile branches build theirs from the one they opened), and changing it to `*sql.Tx` would rewrite `settlePool`'s, `lockManagedGrant`'s and the three reconcile branches' signatures for a refusal the lint and those callers already give. Every pool lock outside the two helpers is SQL inside the five conferral and settlement functions, which assert the rendezvous. + +So the lock functions live in two files, one per kind of lock, and the lint (D3) holds each kind to its file. + +**The lifecycle recording** (#186, third comment). `inOwnTx` type-switches on `*sql.DB` and uses any other handle as it is, a `*sql.Conn` included. It goes. Each recording function splits into two forms: + +- `RecordLifecycleRequest(ctx, b db.Beginner, spec)` and `RecordFirstLifecycleRequest(...)`: open their own transaction on `b` with `db.InTx` and `db.Plain`, a database or a connection a test pins alike, and commit it when they answer; the answer comes back with a failed commit's error, as today. +- `RecordLifecycleRequestInTx(ctx, tx *sql.Tx, spec)` and `RecordFirstLifecycleRequestInTx(...)`: in the caller's transaction, which the caller commits. A connection does not compile here. + +`lockLatest` takes the `*sql.Tx` and calls `db.Lock(ctx, tx, db.Lifecycle, instanceID)`. `lockRequest` takes the `*sql.Tx`. FedWiki's `recordSiteRequestInTx` and `usage.hold` take the `*sql.Tx` for the lock and the recording and a `DBTX` for every other statement, because their tests pass a handle that fails one read (`RecordSiteRequestOn`, `HoldOn`), which a `*sql.Tx` cannot be. `lockAddress` takes the record transaction besides its querier. The `-InTx` suffix is the one FedWiki's intake already uses (`RecordSiteRequestInTx`). + +### D3. The lint + +One new set of rules in the project lint (`internal/lint`, run by `go run . lint`), because they read SQL text in Go strings, query files, migrations and seeds, which forbidigo cannot. The rule walks `cmd/`, `scripts/`, `internal/`, `test/` and the root's `*.go` files under a root (`Config.LockRoot`, default the repository root; `-` disables it), skipping `testdata` directories and directories whose name begins with a dot, as the go tool does, and reports: + +| Rule id | Refuses | Allowed | +|---|---|---| +| `advisory-lock-site` | `pg_advisory_*` or `pg_try_advisory_*`, in any case, in a Go string literal (a chain of literals joined by `+` is one text, so a name in a literal of its own is found), or in the code of a `.sql` or `.sh` file (its `--` or `#` comments removed) | `internal/db/locks.go`; `internal/testkit/dbtest/locks.go` (the probe); `internal/testkit/dbtest/wait_test.go` (the kit's own tests, which lock integers no family builds); `internal/lint/locks.go` and its tests (the rule names the rendezvous statement a seed may run, and its tests write fixtures that take locks); the two migrations that name the rendezvous (`internal/db/migrations/00018_entitlement_set_changes.sql`, `00040_rendezvous_family_key.sql`), by file, so a new migration's wrapper function cannot take a lock outside the helpers; `internal/db/*_test.go`, which run migrations at older versions against the key each version checks, every rendezvous use of which `core.assert_rendezvous` checks, so a stale one fails loudly | +| `advisory-lock-site` (seeds) | in `test/mockshot-seed.sql` and `test/seed/discourse/seed-discourse.sh`, any advisory call but `pg_advisory_xact_lock(hashtextextended('', 0))`, and that statement anywhere but as the first statement after `BEGIN;`, comment lines aside | that one statement, first after `BEGIN;` | +| `pool-lock-site` | in a query file (a `.sql` file under a `queries/` directory), a `-- name:` block whose SQL, comments removed, names `resource_pools` and a row-locking clause; in a non-test Go file outside the test kit and generated `*.sql.go`, a string literal, or a chain of literals joined by `+`, that does | the blocks `LockResourcePool` in `internal/entitlements/queries/resource_pools.sql` and `LockGrantPool` in `internal/entitlements/queries/grants.sql`; migrations (the five functions); tests and the test kit | +| `pool-lock-site` (calls) | `.LockResourcePool` or `.LockGrantPool`, called, or named as a method value or a method expression, in a non-test Go file outside the test kit and `*.sql.go` | `internal/entitlements/tx.go` | +| `lock-families` | the table under "## Advisory key families" in `docs/database-locks.md` naming other families than `db.Families`, or the same in another order | none | + +**Tests and the test kit.** A test that takes an advisory lock goes through `db.Lock`, so it holds the key the code takes; the five that repeat `hashtext($1)::bigint` today would otherwise hold a key nobody else takes after this change, and the three that assert a path does not wait would pass vacuously. A test that probes whether a lock is free uses `dbtest.TryLock`, the kit's one allowed file. A test that waits on a lock's statement uses the family's `Statement()`. Pool row locks in tests stay as they are: a test holds a pool row to make the code wait, and a row has no key to drift, so tests and the test kit are outside `pool-lock-site`. + +**Proof.** The rule's own tests (`internal/lint/locks_test.go`) build small trees in a temporary directory, one row per allowance and per refusal. Command breaks run the real lint on the real tree (tasks 7.1), each adding code, not a comment: an advisory call in a Go string of a non-helper file, a seed's old key, a pool lock query added under another name, a pool lock in a Go string, a method expression of `LockResourcePool` outside `tx.go`, a seed's rendezvous after another statement, and a family missing from the doc's table. A function name split before `advisory_` through an identifier stays out of the rule's reach. + +### D4. The order, written once + +`docs/database-locks.md` gains "## Advisory key families", the table of D1 (family, id, statement name, who takes it), whose row order is the order, and "## The order every path keeps": + +1. A transaction that takes the materialization rendezvous takes it as its first statement, through its kind or, in the two seeds, as the first statement after their `BEGIN`. +2. The families the transaction needs, in the table's order; several locks of one family in the order the page states for it (lifecycle instances in the order their requests were recorded). +3. Rows, by the page's rules: the set and rule rows of a rule change before pool rows; pool rows in ascending `pool_id` before a grant's lineage head, obligation rows and a tier's product row; FedWiki's site rows, then placement rows through the trigger, the usage row and request rows; a desired-state record after its address. + +No advisory lock is taken after a row lock. The rule list's first rule says no other advisory lock precedes the rendezvous, and the page's "Where the paths differ" loses its bullet on the four key conventions. + +### D5. The rendezvous's key in the database + +`core.assert_rendezvous` (migration 00018) holds the key's literal. The migration (`internal/db/migrations/00040_rendezvous_family_key.sql`; the number is settled at landing) replaces the function with `CREATE OR REPLACE`, keeping its owner, grants, `SECURITY DEFINER` and `search_path`, and changes one line: `v_key BIGINT := hashtextextended('materialization:', 0);`. Down restores `'entitlement_materialization'`. The function's comment says the key is the materialization family's, built by `internal/db/locks.go`. + +Rolling back: stop every process of this release; as the migration user, in one transaction, run the Down section of `00040_rendezvous_family_key.sql` and `DELETE FROM goose_db_version_core WHERE version_id = 40`; then start the previous release. `member-console migrate down` cannot do it: it rolls back the last stream in source order that has a migration, an integration's, before core (`RollbackMigration`, `internal/db/migrations.go`). The previous image started alone on the migrated schema refuses every materialization, the fail-closed posture `docs/database-management.md` describes. + +### D6. The tests that take a lock + +| File | Today | After | +|---|---|---| +| `internal/entitlements/subscription_delivery_db_test.go:499` | holds `hashtext(stripeID)` to show delivery takes no subscription lock | `db.Lock(ctx, subLock, db.Subscription, fx.stripeID)` | +| `internal/entitlements/subscription_resume_db_test.go:344` | the same for Resume plan | `db.Lock(..., db.Subscription, "sub_"+f.sub.ID)` | +| `internal/fulfillment/release_schedule_after_commit_test.go:250` | `pg_try_advisory_xact_lock(hashtext(...))` | `dbtest.TryLock(ctx, side, db.Subscription, stripeSubID)` | +| `internal/fulfillment/trace_db_test.go:232` | the same probe | `dbtest.TryLock(e.ctx, tx, db.Subscription, e.subID)` | +| `internal/server/operator_sync_stripe_db_test.go:319` | holds `hashtext(productID)` | `db.Lock(..., db.ProductSync, productID)` | +| `internal/entitlements/plan_act_queries_db_test.go:51`, `:369` | the rendezvous's literal | `db.Lock(ctx, q.Tx(), db.Materialization)`, `db.Lock(ctx, retrying, db.Materialization)` | +| `internal/entitlements/plan_act_drain_db_test.go:839`, `internal/server/operator_org_type_change_test.go:839`, `internal/server/plan_acts_concurrency_db_test.go:40` | wait on `"pg_advisory_xact_lock(hashtextextended"` | wait on `db.Materialization.Statement()` | +| `internal/integration/delivery_queries_db_test.go:471`, `:668`; `internal/integrations/fedwiki/store/request_intake_db_test.go:91`, `:195`; `internal/integrations/fedwiki/web/request_identity_db_test.go:273`; `internal/integrations/fedwiki/workflows/purge_failures_db_test.go:81` | the two deleted queries | `db.Lock` on the transaction each holds | +| `internal/db/acts_test.go:34-35` | the rendezvous's literal | the new key's literal (allowed in `internal/db` tests) | +| `internal/db/settlement_pool_first_test.go` | `sharedRendezvous` from `acts_test.go`, at core versions 37 and 38 | its own constant with the old key, which those versions check | +| `internal/domains/registry_db_test.go` (`TestRegistryLockIsTheFirstStatement`, first change) | expects `SELECT pg_advisory_xact_lock($1)` | expects `LockDomainsRegistry` | +| `internal/integrations/fedwiki/usage/usage_db_test.go:358` (`TestHoldReturnsAFailedLimitRead`), `internal/integrations/fedwiki/store/request_intake_db_test.go:157` (`checkProtectedReadError`, the `protected read error` row of `TestIntakeAndPurgeOrder_DB`) | pass a wrapped handle to `HoldOn` and `RecordSiteRequestOn` | pass the transaction for the lock and the wrapped handle for the reads | + +The callers of the recording functions that pass a transaction (`lifecycle_db_test.go`, `request_intake_db_test.go:219`, and FedWiki's `site_actions.go:618` and `:1033`) call the `-InTx` forms. + +### D7. The goldens + +The statement traces record a lock statement by its first line. They are edited by hand before the run, as `docs/testing.md` asks of a golden a change should move, so the run passes without `-update`, which shows the change was intended: + +| Recorded today | Records | Where | Lines | +|---|---|---|---| +| `SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))` | `LockMaterializationShared` | every `testdata` | 93 in 60 files | +| `SELECT pg_advisory_xact_lock(hashtextextended($1, 0))` | `LockMaterialization` | every `testdata` | 53 in 52 files | +| `SELECT pg_advisory_xact_lock(hashtext($1)::bigint)` | `LockSubscription` | `internal/fulfillment/testdata` | 41 in 40 files | +| the same | `LockInvoice` | `internal/integrations/stripe/workflows/testdata` | 12 in 10 files | +| the same | `LockProductSync` | `internal/server/testdata` | 3 in 3 files | +| `LockLifecycleInstance`, `LockDesiredStateAddress` | unchanged | `internal/integration/testdata`, `internal/workflows/desiredstate/testdata` | | + +In every `testdata` directory today, `hashtextextended($1, 0)` with a parameter is only the rendezvous, and `hashtext($1)::bigint` is only the one family named for that directory, so each edit is a fixed substitution. + +### D8. The documents + +- `docs/database-locks.md`: the opening's sentence on #186 goes; the advisory table's Key and Taken by columns read from the families (the lifecycle row gains its intake and gate takers; the rendezvous row the two seeds); "How advisory keys are built" says one way; the two new sections of D4; "Lifecycle intake" names the two forms; "Where the paths differ" loses the bullet on keys; the Registry section names the family in place of the constant; a paragraph on changing a key, which points at the production guide for the upgrade rule. +- `docs/go-conventions.md`, "Transactions and pool locks": an advisory lock goes through `db.Lock` or `db.LockShared` with its family; the project lint refuses an advisory lock function anywhere else, and a pool row lock outside `LockPools` and `LockPoolOfGrant`; `RecordLifecycleRequestInTx` joins the list of helpers that take a `*sql.Tx`. +- `docs/testing.md`: the dbtest list gains `dbtest.TryLock` and the rule that a test takes a family's lock with `db.Lock` and waits on `Statement()`; the project lint's rule list gains `advisory-lock-site`, `pool-lock-site` and `lock-families`. +- `docs/production-deployment.md`, section 5, states the upgrade rule: **every process of the old release stops before any process of the new one starts**, and migrations that run as a separate step run between the stop and the start, with the reasons. It holds for every upgrade, because a release that changes a key in future meets the same race, and a migration that changes what an older binary writes already assumes it ("The previous binary does not write through the migrated schema", `docs/database-management.md`). Section 5's sentence that runs `migrate up` "before each deploy" is rewritten to match, so the guide states one order. The landing note (tasks 9.3) gives the rule for this release. +- `docs/building-an-integration.md:592`: `RecordLifecycleRequestInTx` in the same transaction as the integration's rows. + +### D9. The integration contract + +The integration contract (#171 to #174) will change the lock sites of the integrations: #171 adds a transport to `internal/integration` and `internal/workflows/desiredstate`, and #174 moves FedWiki and Discourse out of the console, leaving the domains registry in core. Files both touch: + +- `internal/integration/lifecycle.go`, `internal/integration/queries/lifecycle_requests.sql`, `internal/integration/queries/desired_state.sql` and their generated code; +- `internal/workflows/desiredstate/record.go`, `push.go`, `readback.go`; +- `internal/integrations/fedwiki/web/site_actions.go`, `web/api.go` (no edit here; it calls the recording functions on its `*sql.DB`), `store/request_intake.go`, `usage/usage.go`, `workflows/reconcile.go`, `workflows/request_activities.go`; +- `internal/domains/registry.go`. + +What keeps them changeable: a lock site becomes one call, `db.Lock(ctx, tx, db., id...)`, on a transaction the site already holds, so the contract's work moves or deletes a line, not a helper. The lifecycle and desired-state families stay core and keep their keys, so nothing about them changes for a connector. When FedWiki leaves the console, `fedwiki-workspace` loses its last caller; that change deletes the family's row from `Families` and from the doc's table together, and `lock-families` fails if it deletes one without the other. Landing this change before the contract's work begins in those files spares one side a rebase; either order rebases in one-line edits. + +### D10. Scope + +Two changes, in order: + +1. `transactions-open-through-the-runner`: the remaining `BeginTx` calls (five converted, three pinned to #221), the forbidigo rule, the ladder reorder's order. It settles #186's comments on hand-opened transactions and the reorder. +2. This change: the families, the helpers, the key changes and their deploy, the order written once, the lint for advisory and pool row locks, the lifecycle recording's two forms. It meets every point of #186's "Done when" and closes #186. + +## Failure table + +| # | Path or state | Today | After | Test | +|---|---|---|---|---| +| F1 | A materializing transaction or a rule change takes the rendezvous | `hashtextextended('entitlement_materialization', 0)`, shared or exclusive | `hashtextextended('materialization:', 0)`; the same waits | every materializing DB test, through the assertion; `TestLockTakesOneKeyPerId` | +| F2 | A transaction holding the old rendezvous key materializes after the migration (an old process, an old seed) | (n/a) | the assertion refuses: `materialization_rendezvous_missing` | `TestRendezvousFamilyKeyMigration`; break 7.1 (a seed's old key) | +| F3 | The migration is rolled back | (n/a) | the assertion checks the old key again | `TestRendezvousFamilyKeyMigration` | +| F4 | Two reconciles of one subscription | serialize on `hashtext(id)` | serialize on the family key | `TestReconcileWaitsForTheSubscriptionLock`, `TestSupersedeAfterAReleaseTakesTheSubscriptionLock`, now holding the family key | +| F5 | Delivery or Resume plan beside a reconcile | take no subscription lock | the same, now tested against the key the reconcile takes | `TestDeliverDisplaced_WaitsForThePoolRow`, `TestResumeSubscription_AnEndingActFirst`, now holding the family key | +| F6 | Two reconciles of one invoice | serialize | serialize on the family key | `TestInvoiceConcurrentEventsMakeOneInvoice`; the invoice goldens record `LockInvoice` | +| F7 | Two syncs of one product | serialize | serialize on the family key | `TestSyncProductToStripeRechecksUnderTheLock`, now holding the family key | +| F8 | Two rotations of one workspace | serialize | serialize on the family key | `TestWorkspaceTxTakesTheWorkspaceLock` | +| F9 | Two registry allocations | serialize | serialize on the family key | `TestRegistryLockIsTheFirstStatement` (statement name); `TestLockTakesOneKeyPerId` | +| F10 | A lifecycle instance's two recordings | serialize | serialize on the same key as today | `TestRecordLifecycleRequestConcurrentOlderTab_DB`, `TestUnchangedFamiliesKeepTheirKeys` | +| F11 | One address's two recomputes | serialize | serialize on the same key as today | `TestRelayRecomputeReadsAfterTheLock`, `TestLockAddressTakesTheAddressKey`, `TestUnchangedFamiliesKeepTheirKeys` | +| F12 | An instance id or a product id spelled in capitals | a different key from the lower-case spelling (`hashtext` of the raw text for the product) | one key | `TestFamilyKeys/an_instance_id_in_capitals` | +| F13 | A caller passes an empty Stripe id, a malformed UUID, a wrong number of parts or a zero family | a key hashed from whatever text it was given; for a malformed UUID in the two SQL queries, a cast error | an error before any statement; the caller's transaction rolls back through its own error path | `TestFamilyKeys` rows | +| F14 | A recording handed a `*sql.Conn` | each statement commits on its own; the instance lock ends at once | a transaction opened on the connection holds the lock from before the latest-generation read to after the insert | `TestARecordingOnAConnectionHoldsTheInstanceLock` | +| F15 | A recording in the caller's transaction | `RecordLifecycleRequest(ctx, tx, ...)` | `RecordLifecycleRequestInTx(ctx, tx, ...)`; a connection does not compile | the existing lifecycle tests through the `-InTx` forms; the build | +| F16 | A waiter on any family lock | `pg_stat_activity.query` shows a bare `SELECT pg_advisory_xact_lock(...)` | shows the family's name line | `TestLockTakesOneKeyPerId` (reads the waiter's query) | +| F17 | A new advisory lock outside the helpers, a seed's other key, a new pool row lock, a pool query called elsewhere, a family missing from the doc | passes the lint | fails `make lint` | `internal/lint/locks_test.go`; breaks 7.1 | +| F18 | A rolling deploy runs the old and the new release at once | (n/a) | the six changed families share no lock across the two (D1) | none: the deploy rule prevents it (tasks 9.3) | + +## Locks and transactions + +No path takes a lock it did not take, gives one up, or takes two in a new order; each advisory lock keeps its mode and its place in its transaction, and each transaction keeps its kind. What changes is the key of six families and the statement's text. The order of D4 is the order the paths already keep, written once. + +## What is unverified + +- That `pg_stat_activity.query` keeps the statement's leading `-- name:` line. Postgres shows the text the client sent, and sqlc's own statements begin the same way, but nothing in the repository reads it today; `TestLockTakesOneKeyPerId` does, and a failure there means the name moves to a trailing comment, with the goldens edited to match. +- That every `hashtext($1)::bigint` line under a `testdata` directory belongs to the one family named for it in D7: read with `git grep` at HEAD, not run; the edited goldens' run confirms it, since a line in the wrong family fails its golden. + +## Risks / Trade-offs + +- **A deployment that runs two replicas, or an orchestrator that starts the new task before stopping the old one**, would run both releases at once. The landing note and the production guide (D8) say to stop first; nothing in the code detects a mixed deploy apart from the rendezvous assertion. +- **The goldens' edit is large** (202 lines in 109 files) but each line is a fixed substitution per directory, read in the diff before the commit. +- **`Families` names FedWiki's workspace in core.** It is one row until #174 removes it; the alternative, families registered by each owner at init, would let a family exist that the doc's table does not list. + +## Open questions + +None. diff --git a/openspec/changes/advisory-keys-name-their-family/proposal.md b/openspec/changes/advisory-keys-name-their-family/proposal.md new file mode 100644 index 00000000..bcf214de --- /dev/null +++ b/openspec/changes/advisory-keys-name-their-family/proposal.md @@ -0,0 +1,55 @@ +## Why + +The console takes eight kinds of transaction-scoped advisory lock, and each caller builds its key and runs its statement itself (`docs/database-locks.md`, "How advisory keys are built"). The keys follow four conventions: `hashtextextended` of a fixed name (the materialization rendezvous); `hashtextextended` of a prefixed string (a lifecycle instance, a desired-state address); `hashtext` of a bare id widened to a bigint (a Stripe subscription, a Stripe invoice, a product's sync, a FedWiki workspace), four families sharing one 32-bit space with no prefix between them; and a literal constant (the domains registry). The statements live in `internal/entitlements/tx.go`, `internal/fulfillment/reconcile.go` (twice), `internal/integrations/stripe/workflows/invoice_reconcile.go`, `internal/integrations/stripe/catalogsync/catalogsync.go`, `internal/integrations/fedwiki/web/site_actions.go`, `internal/domains/registry.go`, two sqlc queries in `internal/integration/queries/`, `core.assert_rendezvous` and two seeds. Nothing stops a ninth from inventing a fifth convention, and nothing ties a test that holds a lock to the key the code takes: five tests hold or probe the subscription's or the product's key by repeating `hashtext($1)::bigint`, so a change to the code's key would leave them holding a lock nobody else takes, and the ones that assert a path does not wait would pass without testing anything. + +`docs/database-locks.md` writes the order each path takes its locks in, but not one order every path keeps, and it misses the two seeds that take the rendezvous (`test/mockshot-seed.sql`, `test/seed/discourse/seed-discourse.sh`). The pool row lock already goes through `LockPools` and `LockPoolOfGrant` alone, but nothing refuses a new `FOR UPDATE` on `core.resource_pools` elsewhere. + +`integration.RecordLifecycleRequest` and `RecordFirstLifecycleRequest` take sqlc's `DBTX` and open a transaction only when handed a `*sql.DB`. Handed a `*sql.Conn`, which `DBTX` also accepts, they run each statement on its own, so the instance lock ends with the statement that took it and the latest-generation read and the insert are no longer serialized: an older tab's request could be recorded after a newer one. No caller passes a connection today (#186, third comment). + +## What Changes + +- **One way to build a key.** `internal/db/locks.go` declares the families and builds every key as `hashtextextended(':', 0)`, the id's parts joined by `:`, a UUID part in canonical form. `db.Lock` and `db.LockShared` take a `*sql.Tx`, a family and the id's parts, and run one statement per mode whose first line names the family (`-- name: LockSubscription :exec`), so a waiter in `pg_stat_activity` and the statement traces show which family it waits on. Every advisory lock in Go goes through them; the two sqlc lock queries are deleted. +- **The families, in order:** `materialization` (the rendezvous), `domains-registry`, `fedwiki-workspace`, `lifecycle`, `stripe-subscription`, `stripe-invoice`, `stripe-product-sync`, `desired-state`. The keys of `lifecycle` and `desired-state` are unchanged, byte for byte. The other six change. +- **The rendezvous's key changes in the database too.** A migration replaces `core.assert_rendezvous` so it checks `hashtextextended('materialization:', 0)`; the two seeds take that key. +- **The order is written once.** `docs/database-locks.md` gains the family table, in the order a transaction takes them, and one order every path keeps: the rendezvous first, then the families in the table's order, then rows by the page's rules. The project lint fails when the table and `db.Families` disagree. +- **The project lint refuses lock sites outside the helpers** (a new rule set in `internal/lint`): an advisory lock function named, in any case, in a Go string or in SQL or shell code anywhere but `internal/db/locks.go`, the test kit's probe and its own wait tests, the lint rule's own file and tests, the two migrations that name the rendezvous and `internal/db`'s own tests, over `cmd/`, `scripts/`, `internal/`, `test/` and the root's Go files; a seed that takes any advisory key but the rendezvous's, or takes the rendezvous anywhere but first after its `BEGIN;`; a row lock on `core.resource_pools` in any query but `LockResourcePool` and `LockGrantPool`, or in a Go string outside tests and the test kit; and a call, method value or method expression of those two queries outside `internal/entitlements/tx.go`. +- **A connection cannot record a lifecycle request outside a transaction.** `RecordLifecycleRequest` and `RecordFirstLifecycleRequest` take a `db.Beginner` and always open their own transaction, on a `*sql.DB` or on a connection; `RecordLifecycleRequestInTx` and `RecordFirstLifecycleRequestInTx` take the caller's `*sql.Tx`. Every helper that takes a lifecycle or desired-state lock takes a `*sql.Tx`. +- **Tests take a family's lock through the helpers**, so they hold the key the code takes; a test that probes whether a lock is free uses `dbtest.TryLock`. +- **The deploy runs no old process beside a new one.** `docs/production-deployment.md` states that an upgrade stops every process of the old release before the new release starts; the landing note says why this release needs it. +- **Docs:** `docs/database-locks.md` (keys, families, order, the lifecycle intake's two forms), `docs/go-conventions.md`, `docs/testing.md`, `docs/production-deployment.md`, `docs/building-an-integration.md`. + +## Out of scope + +- **Transactions opened with `BeginTx`, the lint that refuses them, and the ladder reorder's order.** The first change of #186 (`transactions-open-through-the-runner`) lands before this one. This change edits the lock statement inside the registry's kind that change adds. +- **Row locks other than the pool's.** Each is one named sqlc query in the package that owns its table, run by one or two functions (`docs/database-locks.md`, "Row locks"); the issue asks for the pool's. +- **Pool lock helpers taking a `*sql.Tx`.** `LockPools` and `LockPoolOfGrant` stay methods of `*entitlements.Queries` (design D2). +- **A check of the order at run time** (design D1, rejected alternatives). + +## Capabilities + +### New Capabilities + +- `database-locks`: how an advisory key is built, what a lock helper takes, and where the families and their order are written. + +### Modified Capabilities + +- `entitlements`: "A transaction that materializes a pool is ordered against a rule change" names the rendezvous by its family and requires that no other advisory lock precede it. +- `entitlement-set-management`: "A rule change commit reaches every carrying pool" and "Outstanding recomputations are settled without an operator" name the rendezvous by its family. +- `go-static-analysis`: a new requirement, "Only the lock helpers take an advisory lock or a pool row lock". +- `deployment-docs`: a new requirement, "The production guide states that an upgrade stops the old release first". + +## Impact + +- **Code:** `internal/db/locks.go` (new); `internal/entitlements/tx.go`; `internal/fulfillment/reconcile.go`; `internal/integrations/stripe/workflows/invoice_reconcile.go`; `internal/integrations/stripe/catalogsync/catalogsync.go`; `internal/integrations/fedwiki/web/site_actions.go`; `internal/domains/registry.go`; `internal/integration/lifecycle.go` and its two lock queries' deletion (`queries/lifecycle_requests.sql`, `queries/desired_state.sql`, generated code); `internal/integrations/fedwiki/store/request_intake.go`, `usage/usage.go`, `workflows/reconcile.go`, `workflows/request_activities.go`; `internal/workflows/desiredstate/record.go`, `push.go`, `readback.go`; `internal/testkit/dbtest/locks.go` (new); `internal/lint` and `cmd/lint.go`. +- **Schema:** one core migration replaces `core.assert_rendezvous`; no table or row changes. +- **Seeds:** `test/mockshot-seed.sql`, `test/seed/discourse/seed-discourse.sh`. +- **Behaviour a deployment sees:** none once deployed. Six advisory keys change, so a process of the previous release running beside one of this release would share no lock with it for those six (design D1); the deploy stops the old one first. +- **Tests:** the goldens that record a lock statement now record the family's name (146 rendezvous lines, and 56 for the subscription, the invoice and the product sync: 202 lines in 109 files); ten test files take or probe a lock through the helpers, three wait on a family's statement by its text, and two (`HoldOn`, `RecordSiteRequestOn`) pass the transaction for the lock and a wrapped handle for the reads; new tests in `internal/db`, `internal/lint`, `internal/integration` and `internal/integrations/fedwiki/web`. +- **Docs:** as above. + +## Related issues + +- **#186** closes with this change; the first change of #186 settled the transactions still opened by hand and the reorder's order. +- **#171** and **#174**, the integration contract, touch the lifecycle, desired-state and FedWiki lock sites this change edits (design D9). +- **#196** made rule-change settlement lock the pool before the obligation; the order this change writes keeps it. +- **#180** the plan-ending paths take the pool lock at different points; the order this change writes describes them as they are. diff --git a/openspec/changes/advisory-keys-name-their-family/specs/database-locks/spec.md b/openspec/changes/advisory-keys-name-their-family/specs/database-locks/spec.md new file mode 100644 index 00000000..8d0b7b74 --- /dev/null +++ b/openspec/changes/advisory-keys-name-their-family/specs/database-locks/spec.md @@ -0,0 +1,69 @@ +## ADDED Requirements + +### Requirement: Every advisory key is built from its family and an id + +Go code outside the test kit's lock probe, the test kit's own tests of its lock waits and `internal/db`'s own tests SHALL take every transaction-scoped advisory lock through one pair of helpers, one for the exclusive mode and one for the shared mode, and SHALL build every key in one way: `hashtextextended( || ':' || , 0)`, where the family is one the helpers declare and the id is the id's parts joined by `:`. A family with one lock for the whole database SHALL have an id of no parts. A part that is a UUID SHALL be written in canonical form, as Postgres writes `uuid::text`, so two spellings of one id take one key; any other part SHALL be used as given and SHALL NOT be empty or contain `:`. A helper SHALL refuse, before it sends any statement, a family it does not declare, a number of parts other than the family's, an empty part, a text part that contains `:` and a part that does not parse as the UUID it should be. Each family's statement SHALL begin with a line naming the family, so a backend waiting on the lock in `pg_stat_activity` and a statement trace show which family it is. The database's rendezvous assertion (`core.assert_rendezvous`) and every seed that takes the materialization rendezvous SHALL use the key the helpers build for the `materialization` family. The keys of the `lifecycle` and `desired-state` families SHALL stay the keys the console took before the families were declared, byte for byte. + +#### Scenario: A subscription's key names its family + +- **WHEN** the subscription reconcile locks Stripe subscription `sub_123` +- **THEN** the lock's key is `hashtextextended('stripe-subscription:sub_123', 0)` + +#### Scenario: Two spellings of one instance take one lock + +- **WHEN** one transaction locks a lifecycle instance by its id in capitals and another locks the same id in lower case +- **THEN** both take `hashtextextended('lifecycle:', 0)` and the second waits for the first + +#### Scenario: The lifecycle key is unchanged + +- **WHEN** the helpers build the key of a lifecycle instance +- **THEN** it equals `hashtextextended('lifecycle:' || ::uuid::text, 0)`, the key the console took before + +#### Scenario: An empty id is refused before any statement + +- **WHEN** a caller asks for the lock of a Stripe subscription whose id is empty +- **THEN** the helper returns an error and sends no statement, and the caller's transaction rolls back through its own error path + +#### Scenario: The rendezvous assertion checks the family's key + +- **WHEN** a transaction takes the `materialization` family's lock in shared mode and then materializes a pool +- **THEN** the assertion passes +- **AND** a transaction that holds `hashtextextended('entitlement_materialization', 0)` instead is refused with `materialization_rendezvous_missing` + +#### Scenario: A waiter shows its family + +- **WHEN** a backend waits on a Stripe subscription's lock +- **THEN** its `pg_stat_activity.query` begins with the line that names `LockSubscription` + +### Requirement: A lock helper takes a transaction + +Every advisory lock helper, and every function that takes an advisory lock and leaves its caller to run statements under it, SHALL take a `*sql.Tx`, so that a database handle or a connection outside a transaction does not compile, because on either the lock would end with the statement that took it. A function that opens its own transaction to take such a lock SHALL take what opens one (`db.Beginner`: a `*sql.DB`, or a `*sql.Conn` a test pins) and SHALL always open a transaction on it. Recording a lifecycle request SHALL offer both forms: one that opens its own transaction on a database or a connection and commits it when it answers, and one that runs in the caller's transaction, which the caller commits. The pool row lock helpers, `LockPools` and `LockPoolOfGrant`, SHALL stay methods of the entitlements querier, which every caller builds on a transaction. + +#### Scenario: A recording on a connection holds the instance lock + +- **WHEN** a test records a lifecycle request on a `*sql.Conn` it pinned +- **THEN** the recording opens a transaction on that connection, and the instance's lock is held from before the latest-generation read until after the insert + +#### Scenario: A connection is not a transaction + +- **WHEN** code passes a `*sql.Conn` or a `*sql.DB` to the in-transaction recording form or to an advisory lock helper +- **THEN** the build fails + +### Requirement: The advisory families and the lock order are written once + +`docs/database-locks.md` SHALL list every advisory family in one table, with its id and the name its statement carries, in the order a transaction takes them, and the project lint SHALL fail when that table and the helpers' list of families differ in a name or in order. A transaction that takes the materialization rendezvous SHALL take it before any other lock; every transaction SHALL take the advisory families it needs in the table's order and before any row lock, and rows in the row order the same page states. The page SHALL name every taker of each family, the seeds included. + +#### Scenario: A family the page does not list fails the lint + +- **WHEN** a contributor declares a new advisory family and does not add its row to the page's table +- **THEN** `make lint` fails, naming the families the table lists and the ones the code declares + +#### Scenario: Two families listed in another order fail the lint + +- **WHEN** the page's table lists `stripe-invoice` before `stripe-subscription` while the code declares the opposite order +- **THEN** `make lint` fails + +#### Scenario: The reconcile keeps the order + +- **WHEN** the subscription reconcile opens its transaction +- **THEN** it takes the shared rendezvous, then the subscription's lock, then pool rows, which is the table's order followed by rows diff --git a/openspec/changes/advisory-keys-name-their-family/specs/deployment-docs/spec.md b/openspec/changes/advisory-keys-name-their-family/specs/deployment-docs/spec.md new file mode 100644 index 00000000..3f5fce0e --- /dev/null +++ b/openspec/changes/advisory-keys-name-their-family/specs/deployment-docs/spec.md @@ -0,0 +1,15 @@ +## ADDED Requirements + +### Requirement: The production guide states that an upgrade stops the old release first + +The production deployment guide SHALL state that an upgrade stops every process of the old release before any process of the new release starts, and, when migrations run as a separate step, runs them between the stop and the start. It SHALL give the reasons: two releases running at once can build different keys for one advisory lock and then hold no common lock, so writes the lock serializes can interleave; and a migration can change what the old release's writes need. It SHALL say what that means for an orchestrator: a rolling update that starts the new task before stopping the old one, or two replicas, are stopped or scaled to zero first. + +#### Scenario: The guide orders an upgrade + +- **WHEN** a deployer follows the guide to upgrade a running deployment +- **THEN** it tells them to stop every process of the old release, run the migration when it is a separate step, and then start the new release + +#### Scenario: The guide covers an orchestrator's rolling update + +- **WHEN** a deployer's orchestrator starts a new task before it stops the old one +- **THEN** the guide tells them to stop or scale the service to zero before the upgrade, and says why diff --git a/openspec/changes/advisory-keys-name-their-family/specs/entitlement-set-management/spec.md b/openspec/changes/advisory-keys-name-their-family/specs/entitlement-set-management/spec.md new file mode 100644 index 00000000..01ecf473 --- /dev/null +++ b/openspec/changes/advisory-keys-name-their-family/specs/entitlement-set-management/spec.md @@ -0,0 +1,89 @@ +## MODIFIED Requirements + +### Requirement: A rule change commit reaches every carrying pool + +`Apply changes` SHALL apply the staged batch as one commit. The commit SHALL re-derive everything it acts on: each named rule's current state, the population of pools whose provisions carry the set with `status = 'active'`, and every limit, inside its own transaction. No client-supplied copy of any preview SHALL be read back, and no staleness token SHALL be carried. A delta naming a rule another operator deactivated since it was staged SHALL refuse the whole batch with "That rule is no longer active." on that delta's line and write nothing; a delta whose fields have come to equal the stored ones SHALL refuse the whole batch with "Already applied." on its line and write nothing, because the batch is one transaction and a partial apply would leave a ruleset nobody staged. + +The commit's transaction SHALL take the materialization rendezvous (the advisory lock of the `materialization` family) in exclusive mode first, once for the batch, which is what serializes two rule commits, and SHALL then take the entitlement set's row `FOR UPDATE` as a guard against a writer that changes the set row without that lock. A commit that waits past its lock timeout SHALL refuse with "Another change collided with this one; try again." and write nothing, whichever of the two locks it was waiting on. + +The rule write SHALL be enclosed. One database function SHALL be the only path that writes a row of `entitlement_set_rules`, and direct insert, update and delete on that table SHALL be revoked from the application's role, so a rule cannot be written without its act row and its obligations by any code path, whether or not the code remembers to write them. That function SHALL, as its first statement, assert that its own transaction holds the materialization rendezvous in exclusive mode and raise a named error if it does not. It SHALL then, inside the commit's transaction, lock the set row, validate the proposed kind against the stored rule and raise a named mismatch when they disagree, apply the rule write, snapshot the rule before and after, enumerate the carrying population once, decide the path against the cap passed to it, write the act row with the path it took, and write one pending obligation row per enumerated pool. The commit SHALL call that function once per delta, in resource-key order, so a batch of N deltas files N act rows, one per rule, in one transaction, with the ledger's grain and the set's History unchanged; every act of one batch SHALL carry the same request identifier. The function SHALL NOT materialize; the caller materializes and settles each obligation through a second enclosed function that writes that pool's effect rows and marks its obligation settled. The cap SHALL be passed to the function rather than read inside it. + +At or below the stated cap on carrying pools, the commit SHALL be atomic: after every delta's rule write it SHALL lock every carrying pool `FOR UPDATE` in ascending pool id order, materialize each pool once against the whole new ruleset, and settle that pool's obligation under each act of the batch from that one recomputation, routing each resource key's effect to the act that changed that key and a movement on a key no delta names to the batch's first act; all in one transaction with the rule writes, the act rows and the obligation rows, and with a savepoint per pool so a failure names the organization that caused it. Any error before the commit SHALL leave the database exactly as it was, every delta included. + +Above the cap the commit SHALL take no pool lock at all: the enclosed function writes each rule change, its act row and one durable obligation row per enumerated pool, and they commit together, so that from the instant the rules are live every pool they owe is named in the work list under every act. It SHALL then drain what the request can before returning and leave the rest to be settled without an operator, and the drain SHALL settle a pool's obligations that share one request identifier from one recomputation ("Outstanding recomputations are settled without an operator"). + +A deactivation of a rule that is already inactive SHALL be reported as "That rule is no longer active." + +Both paths SHALL report their outcome in one toast composed only of non-zero counts: how many rules changed, how many pools were recomputed and how many ended at or over their usage, and on the deferred path how many of the owed pools were recomputed before the request ended. + +#### Scenario: A small population commits a batch atomically + +- **WHEN** an operator applies a batch of three deltas on a set carried by fewer pools than the cap +- **THEN** three rule writes, three act rows, their obligation rows and every pool's single materialization commit in one transaction +- **AND** nothing is left to drain + +#### Scenario: A batch recomputes each pool once and records each key under its act + +- **WHEN** a batch edits the rules for keys A and B on a set whose pools also move on key C through stacking +- **THEN** each carrying pool is materialized once, its effect rows for A hang off A's act, for B off B's act, and for C off the batch's first act +- **AND** no effect row is duplicated across the batch's acts + +#### Scenario: A large population commits the rules with their obligations + +- **WHEN** an operator applies a batch on a set carried by more pools than the cap +- **THEN** the transaction commits every rule change, every act row and one pending obligation row per carrying pool under each act, all sharing one request identifier +- **AND** every pool the batch owes is named in the work list from the instant the rules are live + +#### Scenario: A pool that cannot be materialized refuses the whole atomic commit + +- **WHEN** materializing one pool fails during an atomic commit of a batch +- **THEN** the transaction rolls back, leaving every rule unchanged and no act, effect or obligation row written +- **AND** the tray's form-level line names the organization whose pool failed + +#### Scenario: The commit re-derives a rule the batch no longer describes + +- **WHEN** an operator applies a batch containing an edit to a rule another operator deactivated after it was staged +- **THEN** the commit refuses with "That rule is no longer active." on that delta's line and writes nothing, the other deltas included + +#### Scenario: A rule cannot be written outside the enclosed function + +- **WHEN** any console code path attempts to insert, update or delete a row of `entitlement_set_rules` directly +- **THEN** the database refuses it, because the application's role holds no such privilege +- **AND** the only accepted rule write carries its act row and its obligation rows in the same transaction + +#### Scenario: Two rule commits on one set serialize + +- **WHEN** two operators apply batches on the same set at the same time +- **THEN** one commits and the other waits for the exclusive materialization lock before it reaches any row +- **AND** a wait that times out refuses with "Another change collided with this one; try again." and writes nothing + +### Requirement: Outstanding recomputations are settled without an operator + +Obligations a commit did not settle SHALL be drained automatically until the change owes nothing, by a poller workflow on the console's transactional-outbox idiom: a drain execution per change started as soon as the commit lands, and a long-running poller that selects changes with unsettled pools and dispatches their drains, so a commit whose dispatch failed and a process that died mid-drain are the same case. No operator control SHALL finish a drain in the ordinary course; the only control SHALL be the one on the failure path stated below. There SHALL be no operator control that recomputes a set's pools without a rule change, and no scheduled sweep. + +Each drained pool SHALL be settled in its own short transaction that takes the shared materialization rendezvous and that pool's row, materializes it, writes its effect rows and marks its obligation, all three or none, so a retry can never find effect rows written against an unsettled obligation. When a pool holds unsettled obligations under several acts that share one request identifier, the acts of one applied batch, the drain SHALL settle them together from one recomputation, routing each resource key's effect to the act that changed that key and a movement on a key no act of the batch names to the batch's first act, so a batch costs one recomputation per pool on the deferred path as on the atomic one and no act's effects are written twice. A pool that fails SHALL keep its obligation rows with the error and the attempt count, and SHALL be marked failed once the attempts are exhausted. + +While a change owes pools the set's Rules section SHALL carry a transient line stating how many pools are not recomputed, in the ordinary tone, because work still draining is a normal state. Only exhausted attempts SHALL take the attention tone: the section SHALL then state how many pools failed to recompute and SHALL offer a Retry control beside it, rendered on the repo's existing retry-after-failure precedent as a positive non-commit rather than a destructive one, which returns those obligations to pending for redispatch. That count line and that button SHALL carry the fact between them, with no explanatory line beneath either. Each failed pool SHALL be logged with its change, its pool and its error. + +#### Scenario: A deferred change finishes with nobody watching + +- **WHEN** a commit above the cap returns having settled part of its obligations and the operator closes the page +- **THEN** the drain continues until the change owes no pool +- **AND** no operator action is required to finish it + +#### Scenario: A deferred batch is drained once per pool + +- **WHEN** a batch of two acts above the cap owes a pool under both acts +- **THEN** the drain materializes that pool once and settles both obligations from it, each act receiving the effect rows for the key it changed + +#### Scenario: A draining change reads as normal + +- **WHEN** a set has 3 pools still owed +- **THEN** its Rules section states that 3 pools are not recomputed, in the ordinary tone, with no control beside it + +#### Scenario: An exhausted pool surfaces with Retry + +- **WHEN** a pool's recomputation attempts are exhausted +- **THEN** its obligation is marked failed with its error, the set's Rules section states in the attention tone how many pools failed to recompute, and a Retry control sits beside that line +- **AND** activating Retry returns those obligations to pending and they are redispatched + diff --git a/openspec/changes/advisory-keys-name-their-family/specs/entitlements/spec.md b/openspec/changes/advisory-keys-name-their-family/specs/entitlements/spec.md new file mode 100644 index 00000000..bda3610d --- /dev/null +++ b/openspec/changes/advisory-keys-name-their-family/specs/entitlements/spec.md @@ -0,0 +1,84 @@ +## MODIFIED Requirements + +### Requirement: A transaction that materializes a pool is ordered against a rule change + +Every transaction that materializes a pool SHALL take the materialization rendezvous, the advisory lock of the `materialization` family (`database-locks`), as its first acquisition, before it takes any row: in shared mode, or in exclusive mode when the same transaction changes a rule, carries an operator's tier change or commits an org type's default change. Every transaction that adds, changes, deactivates or reactivates an entitlement set rule, every transaction of an operator act that changes a plan ladder's tiers (the tier add, the tier delete, the tier removal and the reorder's commit transaction), and the default change's commit transaction, which saves an org type's default ladder, SHALL take the same lock in exclusive mode as its first acquisition, before it takes any row. A plan act's listing batch, the transaction that settles one plan act obligation and a plan act's Retry SHALL take it in shared mode as their first acquisition. The rule-change transaction SHALL then take the entitlement set's row before any pool row. Any transaction that takes more than one pool row SHALL take them in ascending pool id order. Because the advisory lock is the first acquisition on both sides, no holder waits for something a would-be holder holds, so a rule change, a tier change or a default change and a recomputation are strictly ordered whichever begins first and neither can hold a row the other waits for. A conferral reads a ladder's ranks and a product's conferral shape in more than one statement, so this ordering is what makes it classify each tier change against one order of the ladder; a plan act's move reads an org type's default, that ladder's rank 0 and which act owns the type in more than one statement, so the same ordering is what makes it resolve one catalog and one owner. + +Shared holders SHALL NOT block each other, so recomputations continue to run concurrently with one another. Each such transaction SHALL set a lock timeout, so a wait surfaces as a refusal an operator can read and retry rather than as a hang; a deadlock reported by the database SHALL surface as the same retryable refusal rather than as a server error. A rule change (`entitlement-set-management`) and every plan-wide act (the tier add, the tier delete, the tier removal, the reorder, the default change, and a plan act's Retry) SHALL answer a lock wait past the timeout or a deadlock at any of its steps with "Another change collided with this one; try again.", because the holder it waited for may be a rule change, a tier change, a default change, a move or a conferral. A plan act's work inside a request MAY use a shorter lock timeout than its drain and a transaction timeout, both set before the materialization lock is taken, and SHALL then record no attempt for a wait or a transaction past either. A transaction past its transaction timeout ends its database session (SQLSTATE 25P04), so its connection is discarded. + +Possession SHALL be enforced rather than trusted. Before it writes anything, the materializer's apply step SHALL assert that its own transaction holds the rendezvous, by calling the one shared database assertion that queries the lock state for the calling backend, and SHALL refuse with a named error identifying the pool when it does not. Every transaction that adds, changes, deactivates or reactivates a rule SHALL call the same assertion in its exclusive form as its own first statement; so SHALL every enclosed function that records a plan act's catalog change. Every enclosed function that inserts, supersedes, settles or requeues a plan act's obligations, and the one that records a Retry, SHALL call it in either form. A caller that forgets the rendezvous therefore refuses a conferral, an activation, a grant or a plan act at once, instead of writing limits derived from a rule state another transaction has already replaced. Lock order is not asserted: a transaction that takes a pool row first surfaces its mistake as a lock timeout or a deadlock, which is a refused request rather than a wrong limit. No other advisory lock SHALL be taken before the rendezvous in such a transaction: every other family follows it, in the order `docs/database-locks.md` gives. + +Membership is decided by what a transaction does, not by which primitive it calls. The transactions bound by this requirement today are the rule-change commit and its drain, the tier add, the tier delete, the tier removal, the reorder's commit transaction, the default change's commit transaction, a plan act's listing batch, the transaction that settles one plan act obligation (in the request and in the plan act drain), a plan act's Retry, grant issuance and grant revocation, the demonstration seed's materialization, and the subscription reconcile's suspend and resume paths. The rule-change commit, the four tier changes and the default change's commit transaction SHALL open through the exclusive opener and every other one through the shared opener, each covered by a test that drives the caller and asserts it does not refuse. The system SHALL provide one transaction opener for each side, the shared one taking its lock timeout and its transaction timeout as parameters and setting both before the lock (a request uses a shorter lock timeout than the drain, and a transaction timeout the drain does not set), so that a transaction which materializes a pool without opening through them is a defect that can be found by reading the code as well as by running it. + +#### Scenario: A conferral that begins first finishes before the rule change + +- **WHEN** a conferral transaction has taken the shared lock and a rule commit arrives +- **THEN** the rule commit waits for the exclusive lock until that conferral's transaction ends +- **AND** the conferral materializes against the rules that were live when it began + +#### Scenario: A conferral that begins during a rule change waits for it + +- **WHEN** a rule commit holds the exclusive lock and a conferral for a pool that is about to carry the set begins +- **THEN** the conferral waits before taking any row +- **AND** it materializes against the committed rule state, so no pool is left materialized from a rule state the commit replaced + +#### Scenario: Two recomputations do not block each other + +- **WHEN** two conferrals on different pools run at the same time with no rule change in flight +- **THEN** both hold the lock in shared mode and neither waits for the other + +#### Scenario: A rule change's wait that exceeds the timeout is a readable refusal + +- **WHEN** a rule change waits longer than the configured lock timeout for the materialization lock +- **THEN** it rolls back and the refusal reads "Another change collided with this one; try again." +- **AND** no partial write survives + +#### Scenario: The reconcile's resume path is bound by the invariant + +- **WHEN** a subscription reconcile resumes a suspended provision and materializes its pool +- **THEN** its transaction takes the shared materialization lock before any row +- **AND** it cannot write limits derived from a rule state a rule commit has already replaced + +#### Scenario: A materializer that holds no rendezvous refuses + +- **WHEN** a transaction that did not take the materialization lock calls the materializer's apply step +- **THEN** it refuses with a named error identifying the pool and writes nothing + +#### Scenario: The reconcile's per-subscription lock follows the rendezvous + +- **WHEN** the subscription reconcile takes the shared materialization lock and then its per-subscription advisory lock +- **THEN** the assertion passes and the materialization proceeds + +#### Scenario: A tier change waits for a conferral in flight + +- **WHEN** a conferral transaction holds the shared lock and has read a ladder's ranks, and a reorder, a tier delete, a tier removal or a tier add on that ladder arrives +- **THEN** the tier change waits for the exclusive lock until that conferral's transaction ends +- **AND** the conferral classifies its transitions against the ladder as it read it, so a handoff of one product between sources is recorded as a `transfer` + +#### Scenario: A conferral that begins during a tier change waits for it + +- **WHEN** a reorder, a tier delete, a tier removal or a tier add holds the exclusive lock and a conferral on that ladder begins +- **THEN** the conferral waits before taking any row +- **AND** it reads the ladder's ranks and the product's shape as the tier change committed them + +#### Scenario: A default change's commit is ordered against a plan act's move + +- **WHEN** one organization's move holds the shared lock and a default change for that organization's type arrives +- **THEN** the default change's commit transaction waits for the exclusive lock until the move ends +- **AND** a move that begins while the commit holds the lock waits, then resolves the default the commit saved and finds the new act owning the type + +#### Scenario: A plan act's catalog record refuses without the exclusive lock + +- **WHEN** a transaction that holds only the shared lock asks the database to record a default change or a reorder +- **THEN** the function refuses with the rendezvous assertion's named error and writes nothing + +#### Scenario: A plan act's listing refuses without the lock + +- **WHEN** a transaction that holds no materialization lock asks the database to write a plan act's obligations +- **THEN** the function refuses with the rendezvous assertion's named error and writes nothing + +#### Scenario: Every plan act answers a lock wait with the refusal + +- **WHEN** a tier add, a tier delete, a tier removal, a reorder or a default change waits longer than the lock timeout because another transaction holds the exclusive lock +- **THEN** it answers "Another change collided with this one; try again." + diff --git a/openspec/changes/advisory-keys-name-their-family/specs/go-static-analysis/spec.md b/openspec/changes/advisory-keys-name-their-family/specs/go-static-analysis/spec.md new file mode 100644 index 00000000..28f163b8 --- /dev/null +++ b/openspec/changes/advisory-keys-name-their-family/specs/go-static-analysis/spec.md @@ -0,0 +1,50 @@ +## ADDED Requirements + +### Requirement: Only the lock helpers take an advisory lock or a pool row lock + +The project lint SHALL walk `cmd/`, `scripts/`, `internal/`, `test/` and the Go files of the repository root, skipping `testdata` directories and directories whose name begins with a dot, as the go tool does, and SHALL fail when an advisory lock function (`pg_advisory_*` or `pg_try_advisory_*`, in any case) is named in a Go string literal (a chain of literals joined by `+` counting as one) or in the code of a SQL or shell file, comments aside, other than in the advisory lock helpers (`internal/db/locks.go`), the test kit's lock probe, the test kit's own tests of its lock waits, the rule's own file and tests, the two migrations that name the rendezvous, and `internal/db`'s own tests, which run the migrations at older versions against the key each version checks. A seed that takes the materialization rendezvous SHALL take no advisory lock but the rendezvous under the key the helpers build, as the first statement after its `BEGIN;`, and the lint SHALL fail on any other. The lint SHALL fail when a sqlc query other than `LockResourcePool` and `LockGrantPool` (in `internal/entitlements/queries/`), or a string literal, or a chain of literals joined by `+`, in a Go file outside tests, the test kit and generated query code (`*.sql.go`), takes a row lock on `core.resource_pools`, and when a Go file outside tests, the test kit, generated query code (`*.sql.go`) and `internal/entitlements/tx.go` calls either of those two queries or names one as a method value or a method expression; the pool row lock in the database's own functions is outside the rule. Tests SHALL take an advisory lock through the helpers and probe one through the test kit, so that a test holds the key the code takes. Each finding SHALL name the helper to use. + +#### Scenario: An advisory lock in a handler fails the lint + +- **WHEN** a contributor adds `SELECT pg_advisory_xact_lock(hashtext($1)::bigint)` to a file under `internal/server` +- **THEN** `go run . lint` fails with an `advisory-lock-site` finding that names `db.Lock` + +#### Scenario: The function's name in a literal of its own fails the lint + +- **WHEN** a Go file outside the helpers holds `lockFn := "pg_advisory_xact_lock_shared"` or the function name in capitals +- **THEN** `go run . lint` fails with an `advisory-lock-site` finding + +#### Scenario: A migration that is not the rendezvous's fails the lint + +- **WHEN** a new migration names an advisory lock function +- **THEN** `go run . lint` fails with an `advisory-lock-site` finding + +#### Scenario: A seed with another key fails the lint + +- **WHEN** `test/mockshot-seed.sql` takes `hashtextextended('entitlement_materialization', 0)` +- **THEN** `go run . lint` fails with an `advisory-lock-site` finding that names the rendezvous's key + +#### Scenario: A seed that takes the rendezvous after an insert fails the lint + +- **WHEN** `test/mockshot-seed.sql` inserts a row between its `BEGIN;` and the rendezvous +- **THEN** `go run . lint` fails with an `advisory-lock-site` finding + +#### Scenario: A second pool lock query fails the lint + +- **WHEN** a contributor adds a query that selects from `core.resource_pools` `FOR UPDATE` under another name +- **THEN** `go run . lint` fails with a `pool-lock-site` finding that names `LockPools` and `LockPoolOfGrant` + +#### Scenario: A pool lock split across two literals fails the lint + +- **WHEN** a Go file outside tests joins `"SELECT 1 FROM core.resource_pools WHERE pool_id = $1 "` and `"FOR UPDATE"` with `+` +- **THEN** `go run . lint` fails with a `pool-lock-site` finding + +#### Scenario: A test holding a pool row passes + +- **WHEN** a test locks a pool row with its own `FOR UPDATE` statement to make the code under test wait +- **THEN** the lint reports nothing for it + +#### Scenario: A migration test keeps an older key + +- **WHEN** a test in `internal/db` takes the rendezvous under the key an older migration's assertion checks +- **THEN** the lint reports nothing for it diff --git a/openspec/changes/advisory-keys-name-their-family/tasks.md b/openspec/changes/advisory-keys-name-their-family/tasks.md new file mode 100644 index 00000000..7a938b54 --- /dev/null +++ b/openspec/changes/advisory-keys-name-their-family/tasks.md @@ -0,0 +1,947 @@ +Tests follow `docs/testing.md`: database tests open the database through `dbtest`, build their rows through the test kit or the package's fixtures, filter to the rows their own fixture made, and pass twice in a row without a reset. A test that holds one transaction open while a second blocks waits for the block with `dbtest.AwaitLockWait`, never a sleep, and releases its side transaction in cleanup. A test that takes an advisory lock takes it with `db.Lock` or `db.LockShared`, probes one with `dbtest.TryLock`, and waits on one by its family's `Statement()`; only `internal/db`'s own tests and the test kit's wait tests write an advisory statement by hand (design D3). + +Sections 1 to 8 are the parts of one commit, made after `transactions-open-through-the-runner` has landed; section 9 is not a commit. The commit passes `make test` with the test stack up, which runs `make lint` first and then every package one at a time (`-p 1`). The goldens of item 5.3 are edited by hand before the run, as `docs/testing.md` asks of a golden a change should move, and the run then passes without `-update`; no other golden, route table or existing test result changes, and any other change is a stop. `make sqlc-generate` removes `LockLifecycleInstance` and `LockDesiredStateAddress` from `internal/integration`'s generated code, moves their comments onto `LatestLifecycleGeneration` and `LockDesiredStateRecord`, adds the blank line sqlc leaves after `LockDesiredStateRecord`'s name line in its query text where the section comment stood, rewords `OpenLifecycleGeneration`'s comment (item 2.11) and copies the rewrapped comments of `LatestLifecycleGeneration` and the desired-state section, and changes nothing else; any other change is a stop. A script's usage is learned from its header or source, never by running it. + +**Breaks.** The list of item 7.1 is written to a file outside the repository and run with `go run ./scripts/mutation-check -list -side new` on the commit, from the repository root, with `test/.env` loaded as `make test` loads it (without it the database tests skip and every record reports SKIPPED), after its green control run; the tests are new or rewritten, so there is no old side. A break of Go code, SQL a test reads or a migration is a test record (`name`, `file`, `old`, `new`, `pkg`, `run_new`, `want_new`); a break the project lint sees is a command record (`name`, `file`, `old`, `new`, `cmd`, `exit`, `want_new`), per the header of `scripts/mutation-check/main.go`. A break caught by two tests is two records with the same `file`, `old` and `new`. Every record's `want_new` is a text the failing test or command prints only when it fails, never a test or subtest name alone. Every record must be CAUGHT. An `old` text below is the code as the section's items write it, with `\n` and `\t` standing for the line break and the tab gofmt writes; when the committed code words it differently, the record is corrected to the committed text before the run, never dropped, and its `new` text keeps the package compiling. A break that no test of the commit sees is named with the reason, and the commit message says so. The commit message lists every break with its result. + +## 1. The helpers + +- [x] 1.1 A new `internal/db/locks.go`, with the repository's SPDX header (design D1, D2): + + ```go + package db + + import ( + "context" + "database/sql" + "errors" + "fmt" + "strings" + + "github.com/google/uuid" + ) + + // idPart is the shape of one part of a family's id. + type idPart int + + const ( + // textPart is used as given and may be neither empty nor hold ':', which + // would run into the next part: a Stripe id, a kind. + textPart idPart = iota + // uuidPart is parsed and written in canonical form, as Postgres writes + // uuid::text, so two spellings of one id take one key. + uuidPart + ) + + // Family is one advisory lock family. Its key for an id is + // hashtextextended(name || ':' || id, 0), the id's parts joined by ':'; a + // family with one lock for the whole database has an id of no parts. Only + // this file declares one. docs/database-locks.md lists the families in the + // order a transaction takes them, and the project lint holds its table to + // Families. + type Family struct { + name string + label string + parts []idPart + } + + var ( + // Materialization is the rendezvous: every transaction that materializes + // a pool holds it shared, every rule change and operator tier change + // exclusive (entitlements.Materializing, entitlements.RuleChange). + // core.assert_rendezvous checks its key. + Materialization = Family{name: "materialization", label: "LockMaterialization"} + // DomainsRegistry serializes every mutating allocation of a name + // (domains.Registry.WithLock). + DomainsRegistry = Family{name: "domains-registry", label: "LockDomainsRegistry"} + // FedWikiWorkspace serializes one workspace's rotations of its active + // site. + FedWikiWorkspace = Family{name: "fedwiki-workspace", label: "LockFedWikiWorkspace", parts: []idPart{uuidPart}} + // Lifecycle serializes one instance's lifecycle requests. + Lifecycle = Family{name: "lifecycle", label: "LockLifecycleInstance", parts: []idPart{uuidPart}} + // Subscription serializes two reconciles of one Stripe subscription. + Subscription = Family{name: "stripe-subscription", label: "LockSubscription", parts: []idPart{textPart}} + // Invoice serializes two reconciles of one Stripe invoice. + Invoice = Family{name: "stripe-invoice", label: "LockInvoice", parts: []idPart{textPart}} + // ProductSync serializes two Stripe syncs of one product. + ProductSync = Family{name: "stripe-product-sync", label: "LockProductSync", parts: []idPart{uuidPart}} + // DesiredState serializes every write at one address: a connection, a + // recipient kind and a recipient. + DesiredState = Family{name: "desired-state", label: "LockDesiredStateAddress", parts: []idPart{uuidPart, textPart, uuidPart}} + ) + + // Families is every family, in the order a transaction takes them. + var Families = []Family{Materialization, DomainsRegistry, FedWikiWorkspace, Lifecycle, Subscription, Invoice, ProductSync, DesiredState} + + // Name is the family's name, the prefix of its keys. + func (f Family) Name() string { return f.name } + + // Key is the text whose hashtextextended is f's key for the id's parts. It + // refuses a family this file did not declare, a number of parts other than + // the family's, an empty text part, a text part that holds ':' and a uuid + // part that does not parse. + func (f Family) Key(id ...string) (string, error) { + if f.name == "" { + return "", errors.New("advisory lock: no family") + } + if len(id) != len(f.parts) { + return "", fmt.Errorf("advisory lock %s: %d id parts, want %d", f.name, len(id), len(f.parts)) + } + parts := make([]string, len(id)) + for i, p := range id { + switch f.parts[i] { + case uuidPart: + u, err := uuid.Parse(p) + if err != nil { + return "", fmt.Errorf("advisory lock %s: id part %d: %w", f.name, i+1, err) + } + parts[i] = u.String() + default: + if p == "" { + return "", fmt.Errorf("advisory lock %s: id part %d is empty", f.name, i+1) + } + if strings.Contains(p, ":") { + return "", fmt.Errorf("advisory lock %s: id part %d holds ':'", f.name, i+1) + } + parts[i] = p + } + } + return f.name + ":" + strings.Join(parts, ":"), nil + } + + // Statement is the statement Lock runs for f, the key as $1. Its first line + // names the family, which pg_stat_activity shows for a waiter and the + // statement traces record; a test waits on it by its text. + func (f Family) Statement() string { + return "-- name: " + f.label + " :exec\nSELECT pg_advisory_xact_lock(hashtextextended($1, 0))" + } + + // SharedStatement is the statement LockShared runs for f. + func (f Family) SharedStatement() string { + return "-- name: " + f.label + "Shared :exec\nSELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))" + } + + // Lock takes f's exclusive transaction-scoped advisory lock on the id, + // released at commit or rollback. It takes a *sql.Tx because on a database + // or a connection outside a transaction the lock would end with the + // statement that took it. An id Key refuses sends no statement; the + // driver's error comes back as it is. + func Lock(ctx context.Context, tx *sql.Tx, f Family, id ...string) error { + return take(ctx, tx, f.Statement(), f, id) + } + + // LockShared takes f's shared lock on the id, as Lock takes the exclusive + // one. + func LockShared(ctx context.Context, tx *sql.Tx, f Family, id ...string) error { + return take(ctx, tx, f.SharedStatement(), f, id) + } + + // take builds f's key for id and runs stmt with it on tx. + func take(ctx context.Context, tx *sql.Tx, stmt string, f Family, id []string) error { + key, err := f.Key(id...) + if err != nil { + return err + } + _, err = tx.ExecContext(ctx, stmt, key) + return err + } + ``` + +- [x] 1.2 A new `internal/testkit/dbtest/locks.go`, opening with the repository's SPDX header, `package dbtest` and imports of `context`, `database/sql` and `internal/db`: + + ```go + // TryLock takes f's exclusive lock on the id on tx without waiting and + // reports whether it was free; a free lock stays with tx until tx ends. A + // test probes with it whether the code under test holds a family's lock, + // under the key the code builds. + func TryLock(ctx context.Context, tx *sql.Tx, f db.Family, id ...string) (bool, error) { + key, err := f.Key(id...) + if err != nil { + return false, err + } + var free bool + err = tx.QueryRowContext(ctx, `SELECT pg_try_advisory_xact_lock(hashtextextended($1, 0))`, key).Scan(&free) + return free, err + } + ``` + +## 2. The call sites + +- [x] 2.1 `internal/entitlements/tx.go`. `rendezvousName` and its sentence "The key is hashtextextended of this name." go; the comment's first sentence reads "The materialization rendezvous (Decision 144): one transaction-scoped advisory lock, the materialization family's (internal/db/locks.go), that every transaction materializing a pool holds in shared mode and every rule change and operator tier change holds in exclusive mode, taken as the transaction's first statement so no holder ever waits on something a would-be holder already holds." `MaterializingWithin` returns `db.FirstRuns(rendezvous(db.LockShared, lockTimeout, txTimeout))`; `RuleChange` is `db.FirstRuns(rendezvous(db.Lock, rendezvousLockTimeout, 0))`; `rendezvous` reads: + + ```go + // rendezvous is the first statements of a transaction that takes the + // materialization rendezvous with lock, db.Lock or db.LockShared: the lock + // timeout, the transaction timeout when there is one, then the lock. + func rendezvous(lock func(context.Context, *sql.Tx, db.Family, ...string) error, lockTimeout, txTimeout time.Duration) func(context.Context, *sql.Tx) error { + return func(ctx context.Context, tx *sql.Tx) error { + if txTimeout > 0 { + if _, err := tx.ExecContext(ctx, "SET LOCAL transaction_timeout = '"+milliseconds(txTimeout)+"'"); err != nil { + return fmt.Errorf("set transaction_timeout: %w", err) + } + } + if _, err := tx.ExecContext(ctx, "SET LOCAL lock_timeout = '"+milliseconds(lockTimeout)+"'"); err != nil { + return fmt.Errorf("set lock_timeout: %w", err) + } + if err := lock(ctx, tx, db.Materialization); err != nil { + return fmt.Errorf("take materialization rendezvous: %w", err) + } + return nil + } + } + ``` + +- [x] 2.2 `internal/domains/registry.go`. `registryAllocationLock` and its comment go. `registryKind` (from `transactions-open-through-the-runner`) reads: + + ```go + // registryKind opens a registry allocation's transaction: the registry's + // advisory lock, the domains-registry family's (internal/db/locks.go), is + // its first statement. Allocation is human-paced, so one lock for the whole + // namespace buys the subtree-disjointness invariants, which span rows the + // database cannot constrain against each other, at no practical cost + // (design D3). + // + // LOCK ORDERING INVARIANT: a transaction that touches both the registry and + // entitlement-usage rows MUST take this lock FIRST, as docs/database-locks.md + // orders every advisory family before rows: the usage row's lock + // (LockNumericUsage, taken by a recount) is always second. + var registryKind = db.FirstRuns(func(ctx context.Context, tx *sql.Tx) error { + if err := db.Lock(ctx, tx, db.DomainsRegistry); err != nil { + return fmt.Errorf("domains: registry advisory lock: %w", err) + } + return nil + }) + ``` + + `WithLock`'s comment reads "See registryKind for the lock-ordering invariant against entitlement usage rows." + +- [x] 2.3 `internal/integrations/fedwiki/web/site_actions.go`, `withWorkspaceTx`: its comment's "(pg_advisory_xact_lock, released on commit or rollback)" reads "(the fedwiki-workspace family's, released on commit or rollback)"; inside `db.InTx`, the three-line comment on `hashtext` goes and the lock reads: + + ```go + if err := db.Lock(ctx, tx, db.FedWikiWorkspace, workspaceID); err != nil { + return err + } + ``` + +- [x] 2.4 `internal/fulfillment/reconcile.go`. `reconcileInTx`'s parameter `db *sql.DB` is renamed `database` (its one use, `entitlements.BeginMaterializing(ctx, database)`), so the package name is in scope; its lock reads: + + ```go + if err := db.Lock(ctx, tx, db.Subscription, stripeSubscriptionID); err != nil { + return plan, fmt.Errorf("reconcile: advisory lock: %w", err) + } + ``` + + `supersedeReleasedDowngrades`' lock reads: + + ```go + if err := db.Lock(ctx, tx, db.Subscription, stripeSubscriptionID); err != nil { + return fmt.Errorf("reconcile: advisory lock after releasing the schedule: %w", err) + } + ``` + +- [x] 2.5 `internal/integrations/stripe/workflows/invoice_reconcile.go`, `converge`: the comment "The form reconcile.go uses for subscriptions." reads "The stripe-invoice family's lock (internal/db/locks.go)."; the lock reads `if err := db.Lock(ctx, tx, db.Invoice, r.id); err != nil {`, its error unchanged. + +- [x] 2.6 `internal/integrations/stripe/catalogsync/catalogsync.go`, `enqueue`: "The per-product advisory lock (the same idiom as fulfillment's reconcile)" in its comment reads "The stripe-product-sync family's lock (internal/db/locks.go)"; the lock reads: + + ```go + if err := db.Lock(ctx, tx, db.ProductSync, productID); err != nil { + return 0, end, &Error{Stage: StageLock, Err: err} + } + ``` + +- [x] 2.7 `internal/integration/lifecycle.go` (design D2). `inOwnTx` reads: + + ```go + // inOwnTx runs fn in a transaction of its own on b, committed when fn + // succeeds. The answer comes back with a failed commit's error, as each + // caller returned it before. + func inOwnTx(ctx context.Context, b db.Beginner, fn func(*sql.Tx) (LifecycleAnswer, error)) (LifecycleAnswer, error) { + var answer LifecycleAnswer + err := db.InTx(ctx, b, db.Plain, func(tx *sql.Tx) error { + var err error + answer, err = fn(tx) + return err + }) + return answer, err + } + ``` + + `RecordLifecycleRequest` reads: + + ```go + // RecordLifecycleRequest records spec, or answers from what holds its + // address, in a transaction of its own on b (a *sql.DB, or a *sql.Conn a + // test pins), committed when it answers. RecordLifecycleRequestInTx records + // in the caller's transaction. + func RecordLifecycleRequest(ctx context.Context, b db.Beginner, spec LifecycleRequestSpec) (LifecycleAnswer, error) { + return inOwnTx(ctx, b, func(tx *sql.Tx) (LifecycleAnswer, error) { + return RecordLifecycleRequestInTx(ctx, tx, spec) + }) + } + ``` + + and `RecordLifecycleRequestInTx(ctx context.Context, tx *sql.Tx, spec LifecycleRequestSpec) (LifecycleAnswer, error)` carries today's comment, whose first sentence reads "RecordLifecycleRequestInTx records spec or answers from what holds its address (design D26), in tx, which the caller commits; the lifecycle family's lock serializes it per instance until then.", and today's body with `q := New(tx)` and `lockLatest(ctx, tx, q, spec.InstanceID)`. `RecordFirstLifecycleRequest` and `RecordFirstLifecycleRequestInTx` split the same way, the first's comment pointing at the second. `lockLatest` reads: + + ```go + // afterInstanceLock, when set by a test, runs inside a recording right after + // the instance's lock is granted and before its latest generation is read. + var afterInstanceLock func(instanceID string) + + // lockLatest takes the instance's lock on tx and reads its latest + // generation. + func lockLatest(ctx context.Context, tx *sql.Tx, q *Queries, instanceID string) (int64, error) { + if err := db.Lock(ctx, tx, db.Lifecycle, instanceID); err != nil { + return 0, fmt.Errorf("lock lifecycle instance: %w", err) + } + if afterInstanceLock != nil { + afterInstanceLock(instanceID) + } + latest, err := q.LatestLifecycleGeneration(ctx, instanceID) + if err != nil { + return 0, fmt.Errorf("latest lifecycle generation: %w", err) + } + return latest, nil + } + ``` + + `internal/integration/export_test.go` gains `func SetAfterInstanceLock(fn func(string)) { afterInstanceLock = fn }`. + +- [x] 2.8 The callers that record in their own transaction call the `-InTx` forms: `internal/integrations/fedwiki/web/site_actions.go:618` and `:1033`, and `internal/integrations/fedwiki/store/request_intake.go:53`. `recordSiteRequestInTx(ctx, tx *sql.Tx, q DBTX, spec, domain)` takes the instance lock and records (`integration.RecordLifecycleRequestInTx(ctx, tx, spec)`) on `tx`, and runs `LookupLifecycleRequest` and `checkRequestSite` on `q`; its `queries` variable goes; `RecordSiteRequestInTx` calls it with `(ctx, tx, tx, spec, domain)`. `request_intake_db_test.go:157` calls `fwmod.RecordSiteRequestOn(t.Context(), tx, failedSiteRead{tx}, spec, domain)`. The callers that pass `h.Database` (`web/api.go:469`, `:604`, `site_actions.go:192`, `:681`) are unchanged. + +- [x] 2.9 Every other lifecycle lock: `.LockLifecycleInstance(ctx, )` reads `db.Lock(ctx, tx, db.Lifecycle, )` on the transaction the function holds, its error handling unchanged, in `internal/integrations/fedwiki/store/request_intake.go` (`recordSiteRequestInTx`, `CheckSwapRequestSites`), `internal/integrations/fedwiki/usage/usage.go` (`lockRequest(ctx, tx *sql.Tx, req)`; `hold(ctx, tx *sql.Tx, q entitlements.DBTX, req)` calls `lockRequest(ctx, tx, req)` and runs every other statement on `q`: `fwmod.New(q)`, `recountHeld`, `primaryPool`, `Limit(ctx, entitlements.New(q), poolID)`, `recount`, `lockAndCount` and `slots.write`; `Hold` calls `hold(ctx, tx, tx, req)`; `usage_db_test.go:358` calls `usage.HoldOn(f.ctx, tx, failingLimitRead{tx}, usage.HoldRequest{…})`), `internal/integrations/fedwiki/workflows/reconcile.go` (`purgeArchivedSite`) and `internal/integrations/fedwiki/workflows/request_activities.go` (`lockOutcomes`). `internal/integrations/fedwiki/workflows/reconcile.go` drops its `integration` import, which the edit leaves unused. `var HoldOn = hold` and `var RecordSiteRequestOn = recordSiteRequestInTx` stay. + +- [x] 2.10 `internal/workflows/desiredstate/record.go`, `lockAddress(ctx context.Context, tx *sql.Tx, q *integration.Queries, a address)`; its lock reads: + + ```go + if err := db.Lock(ctx, tx, db.DesiredState, a.ConnectionID, string(a.Kind), a.RecipientID); err != nil { + return nil, fmt.Errorf("lock address: %w", err) + } + ``` + + Its three callers (`recomputeAddress`, `writePushAnswer` in `push.go`, `readBackItem` in `readback.go`) pass the transaction they opened through `beginRecordTx`. `export_test.go` gains `func LockAddress(ctx context.Context, tx *sql.Tx, a Address) error { _, err := lockAddress(ctx, tx, integration.New(tx), a); return err }`. + +- [x] 2.11 The two lock queries go. `internal/integration/queries/lifecycle_requests.sql`: the `LockLifecycleInstance` block goes, and its comment's sequence moves to `LatestLifecycleGeneration`'s, reading "Lifecycle requests (design D24 to D28). Recording a request runs, in one transaction: the lifecycle family's lock (db.Lock), LatestLifecycleGeneration, then for a generation above the latest GenerationCeiling, OpenLifecycleGeneration and InsertLifecycleRequest, and otherwise GetLifecycleRequestByAddress, so the latest-generation check and the insert are one step per instance." `internal/integration/queries/desired_state.sql`: the `LockDesiredStateAddress` block goes, and the section comment's "LockDesiredStateAddress" reads "the desired-state family's lock (db.Lock)". In the first file, `OpenLifecycleGeneration`'s comment reads "The generation of the instance's accepted request, 0 when none is open. Read under the lifecycle family's lock (db.Lock), so the one-open rule is answered before the insert instead of by a unique violation that would abort the caller's transaction."; both comments are wrapped at 78 columns. `make sqlc-generate`. + +## 3. The rendezvous in the database and the seeds + +- [x] 3.1 `internal/db/migrations/00040_rendezvous_family_key.sql` (the number is settled at landing, with its entry in `advisoryMigrations`; design D5), with the repository's SPDX header: under `-- +goose Up` and `-- +goose StatementBegin`, a comment "core.assert_rendezvous checks the materialization family's key, hashtextextended('materialization:', 0), which internal/db/locks.go builds for every rendezvous. A backend holding the key this function checked before, a process of the previous release, is refused. A later replacement keeps `v_key` equal to the key `db.Materialization` builds." and `CREATE OR REPLACE FUNCTION core.assert_rendezvous(p_exclusive BOOLEAN)`, the body of migration 00018 (`:219-247`) with its one line changed to `v_key BIGINT := hashtextextended('materialization:', 0);`, the same `LANGUAGE plpgsql`, `SECURITY DEFINER` and `SET search_path = core, pg_temp`; after it, as migration 00018 (`:555-563`) and 00038 (`:93-94`) do, the `ALTER FUNCTION ... OWNER TO core_owner`, the `REVOKE ... FROM PUBLIC` and the `GRANT EXECUTE ... TO core_writer`, which `CREATE OR REPLACE` would keep anyway, so the stream reads one way. Under `-- +goose Down`, the same with `hashtextextended('entitlement_materialization', 0)`. + +- [x] 3.2 `test/mockshot-seed.sql` and `test/seed/discourse/seed-discourse.sh` take the rendezvous as the first statement after `BEGIN`, before any insert: each seed's lock (`:195` and `:67`) moves to the line after `BEGIN;` (`:47` and `:30`) under the comment below and reads `SELECT pg_advisory_xact_lock(hashtextextended('materialization:', 0));`; the comment above `core.commit_rule_change` keeps its sentences about that call and loses the one on taking the advisory lock first. The moved comment reads "This transaction writes a rule through core.commit_rule_change, so it takes the exclusive materialization rendezvous first, as every rule change does; the key is the materialization family's (internal/db/locks.go), which the function checks." Each insert before the move held its new row and a `FOR KEY SHARE` on every row it references, so a lock taken after them would break the order D4 writes. + +## 4. The lint + +- [x] 4.1 `internal/lint/lint.go`: `Config` gains `LockRoot string` ("the repository root the lock rules walk; '-' disables them"), defaulting to `.` in `Run`, which appends, before sorting: + + ```go + if cfg.LockRoot != "-" { + res.Violations = append(res.Violations, ruleLockSites(cfg.LockRoot)...) + res.Violations = append(res.Violations, ruleLockFamilies(cfg.LockRoot)...) + } + ``` + + `cmd/lint.go` gains the flag `lock-root` (default `.`, `"-" disables the lock rules`), passes it to `lint.Config` as `LockRoot`, and gains in its rule list, " - lock sites: an advisory or pool row lock taken outside its helper, a seed's other key, or a family table that disagrees with internal/db"; the command's description, "lint walks the Go source under internal/", gains "(and, for the lock rules, the Go, SQL and shell files under cmd/, scripts/, internal/ and test/ and the root's Go files)"; the rule-list line is wrapped at 78 columns with the others. + +- [x] 4.2 A new `internal/lint/locks.go` (design D3), opening with the repository's SPDX header, `package lint` and imports of `errors`, `fmt`, `go/ast`, `go/parser`, `go/token`, `io/fs`, `os`, `path/filepath`, `regexp`, `slices`, `strconv`, `strings` and `internal/db`: + + ```go + // The lock helpers (docs/database-locks.md): every advisory lock is taken in + // advisoryHelper, every pool row lock through the two queries poolHelper + // calls. + const ( + advisoryHelper = "internal/db/locks.go" + poolHelper = "internal/entitlements/tx.go" + locksDoc = "docs/database-locks.md" + ) + + // advisoryCallRe is an advisory lock function named in SQL text, in any case: + // SQL function names are not case-sensitive. + var advisoryCallRe = regexp.MustCompile(`(?i)\bpg_(?:try_)?advisory_`) + + // rowLockRe is a row-locking clause. + var rowLockRe = regexp.MustCompile(`(?i)\bFOR\s+(?:UPDATE|NO\s+KEY\s+UPDATE|SHARE|KEY\s+SHARE)\b`) + + // poolLockQueries names, by query file, the one block of each that may lock + // a core.resource_pools row. + var poolLockQueries = map[string]string{ + "internal/entitlements/queries/resource_pools.sql": "LockResourcePool", + "internal/entitlements/queries/grants.sql": "LockGrantPool", + } + + // lockSeeds are the seeds that take the materialization rendezvous. + var lockSeeds = map[string]bool{ + "test/mockshot-seed.sql": true, + "test/seed/discourse/seed-discourse.sh": true, + } + + // advisoryMigrations are the two migrations that name the rendezvous, by file: + // a new migration's wrapper function would otherwise let any query lock + // outside the helpers. + var advisoryMigrations = map[string]bool{ + "internal/db/migrations/00018_entitlement_set_changes.sql": true, + "internal/db/migrations/00040_rendezvous_family_key.sql": true, + } + + // advisoryAllowed reports whether the file at rel may name an advisory lock + // function: the helper; the test kit's probe and its own wait tests, which + // lock integers no family builds; this rule and its tests, which name the + // rendezvous statement a seed may run and write fixtures that take locks; + // the two migrations of advisoryMigrations; and internal/db's tests, which + // run migrations at older versions against the key each version checks, + // every rendezvous use of which core.assert_rendezvous checks. + func advisoryAllowed(rel string) bool { + switch rel { + case advisoryHelper, "internal/testkit/dbtest/locks.go", "internal/testkit/dbtest/wait_test.go", + "internal/lint/locks.go", "internal/lint/locks_test.go": + return true + } + return advisoryMigrations[rel] || + (strings.HasPrefix(rel, "internal/db/") && strings.HasSuffix(rel, "_test.go")) + } + + // outsidePoolRule reports whether rel is outside the pool row rule: a test, + // the test kit, generated query code, a migration or a shell script. + func outsidePoolRule(rel string) bool { + return strings.HasSuffix(rel, "_test.go") || strings.HasPrefix(rel, "internal/testkit/") || + strings.HasSuffix(rel, ".sql.go") || strings.Contains(rel, "/migrations/") || strings.HasSuffix(rel, ".sh") + } + + // relPath is path relative to root, with forward slashes. + func relPath(root, path string) string { + rel, err := filepath.Rel(root, path) + if err != nil { + return filepath.ToSlash(path) + } + return filepath.ToSlash(rel) + } + + // lockFiles reads every .go, .sql and .sh file under cmd/, scripts/, + // internal/ and test/ of root, and the .go files of root itself. It does not + // enter a testdata directory, which holds fixtures and, under test/, the + // stack's own data, some of it owned by another user, or a directory whose + // name begins with a dot, so readTree, which enters every directory, cannot + // serve here. + func lockFiles(root string) ([]sourceFile, []Violation) { + var files []sourceFile + read := func(path string) error { + switch filepath.Ext(path) { + case ".go", ".sql", ".sh": + default: + return nil + } + src, err := os.ReadFile(path) + if errors.Is(err, fs.ErrNotExist) { + return nil + } + if err != nil { + return err + } + files = append(files, sourceFile{path: path, src: src}) + return nil + } + if entries, err := os.ReadDir(root); err == nil { + for _, e := range entries { + if !e.IsDir() && filepath.Ext(e.Name()) == ".go" { + if err := read(filepath.Join(root, e.Name())); err != nil { + return files, []Violation{unreadableFile(root, err)} + } + } + } + } + for _, dir := range []string{"cmd", "scripts", "internal", "test"} { + base := filepath.Join(root, dir) + if _, err := os.Stat(base); errors.Is(err, fs.ErrNotExist) { + continue + } + if err := walkLockDir(base, read); err != nil { + return files, []Violation{unreadableFile(base, err)} + } + } + return files, nil + } + + // walkLockDir calls read with every file under base and does not enter a + // testdata directory or a directory whose name begins with a dot, both of + // which the go tool ignores too (go help packages), such as the agent + // runner's disposable copy of the tree, scripts/agent-runner/.code. + func walkLockDir(base string, read func(path string) error) error { + return filepath.WalkDir(base, func(path string, d fs.DirEntry, err error) error { + if err != nil { + return err + } + if d.IsDir() { + if d.Name() == "testdata" || strings.HasPrefix(d.Name(), ".") { + return filepath.SkipDir + } + return nil + } + return read(path) + }) + } + + // ruleLockSites reports every lock under the directories lockFiles reads + // taken outside its helper. + func ruleLockSites(root string) []Violation { + files, v := lockFiles(root) + key, _ := db.Materialization.Key() // a family with no id parts always builds its key + rendezvous := "pg_advisory_xact_lock(hashtextextended('" + key + "', 0))" + for _, f := range files { + rel := relPath(root, f.path) + v = append(v, advisorySites(f, rel, rendezvous)...) + if !outsidePoolRule(rel) { + v = append(v, poolSites(f, rel)...) + } + } + return v + } + + // advisorySites reports each Go string literal of f that names an advisory + // lock function where rel may not, and each line of the code of a .sql or + // .sh file that does (its comments removed, so a comment never matches); in + // a seed, each one but the rendezvous's, and the rendezvous anywhere but as + // the first statement after `BEGIN;`. A chain of literals joined by + is + // one text, so a function name in a literal of its own is found; Go + // comments are not strings and never match. A Go file the parser refuses is + // reported unread here and not again by poolSites. + func advisorySites(f sourceFile, rel, rendezvous string) []Violation { + if advisoryAllowed(rel) { + return nil + } + const message = "an advisory lock outside internal/db/locks.go: take it with db.Lock or db.LockShared and its family (docs/database-locks.md)" + if strings.HasSuffix(rel, ".go") { + strs, v := literalChains(f, "advisory locks") + for _, s := range strs { + if advisoryCallRe.MatchString(s.text) { + v = append(v, Violation{File: f.path, Line: s.line, Rule: "advisory-lock-site", Message: message}) + } + } + return v + } + shell := strings.HasSuffix(rel, ".sh") + var v []Violation + prev := "" // the last line of code before this one, trimmed + heredoc := "" // the delimiter of the shell heredoc the line is in, if any + for i, line := range strings.Split(string(f.src), "\n") { + inDoc := heredoc != "" + code := "" + if inDoc && strings.TrimSpace(line) == heredoc { + heredoc = "" + } else { + code = stripComment(line, shell && !inDoc) + } + if shell && !inDoc { + if m := heredocRe.FindStringSubmatch(line); m != nil { + heredoc = m[1] + } + } + if strings.HasPrefix(strings.TrimSpace(code), "--") { + code = "" // a SQL comment line, in a shell file's heredoc too + } + calls := advisoryCallRe.FindAllString(code, -1) + switch { + case len(calls) == 0: + case lockSeeds[rel] && len(calls) == 1 && strings.Contains(code, rendezvous) && strings.EqualFold(prev, "BEGIN;"): + case lockSeeds[rel] && len(calls) == 1 && strings.Contains(code, rendezvous): + v = append(v, Violation{File: f.path, Line: i + 1, Rule: "advisory-lock-site", + Message: "a seed takes the materialization rendezvous other than as the first statement after BEGIN (docs/database-locks.md)"}) + case lockSeeds[rel]: + v = append(v, Violation{File: f.path, Line: i + 1, Rule: "advisory-lock-site", + Message: "a seed takes an advisory lock other than the materialization rendezvous, " + rendezvous + " (internal/db/locks.go)"}) + default: + v = append(v, Violation{File: f.path, Line: i + 1, Rule: "advisory-lock-site", Message: message}) + } + if t := strings.TrimSpace(code); t != "" { + prev = t + } + } + return v + } + + // heredocRe is the opening of a shell heredoc, whose body is SQL text. + var heredocRe = regexp.MustCompile(`<<-?\s*['"]?([A-Za-z_]\w*)`) + + // stripComment returns line without its comment: in shell code, from a # + // that starts the line or follows a space or tab; in SQL text, from a --. + // A marker inside quotes is not a comment. + func stripComment(line string, shell bool) string { + var quote byte + for i := 0; i < len(line); i++ { + c := line[i] + switch { + case quote != 0: + if c == quote { + quote = 0 + } + case c == '\'' || (shell && c == '"'): + quote = c + case !shell && strings.HasPrefix(line[i:], "--"): + return line[:i] + case shell && c == '#' && (i == 0 || line[i-1] == ' ' || line[i-1] == '\t'): + return line[:i] + } + } + return line + } + + // poolSites reports, in a query file, each block that locks a pool row and + // is not the file's allowed one; in a Go file, each string literal, or chain + // of literals joined by +, that locks a pool row and, outside poolHelper, + // each call or method value of a pool lock query. + func poolSites(f sourceFile, rel string) []Violation { + if strings.HasSuffix(rel, ".sql") { + if !strings.Contains(rel, "/queries/") { + return nil + } + return poolQueryBlocks(f, rel) + } + v := poolStrings(f) + if rel == poolHelper { + return v + } + return append(v, poolCalls(f)...) + } + + // poolCalls reports each selector of LockResourcePool or LockGrantPool in a + // Go file, called or taken as a method value; a comment or a string naming + // one never matches. + func poolCalls(f sourceFile) []Violation { + fset := token.NewFileSet() + file, err := parser.ParseFile(fset, f.path, f.src, parser.SkipObjectResolution) + if err != nil { + return nil // advisorySites reports a file the parser refuses + } + var v []Violation + ast.Inspect(file, func(n ast.Node) bool { + if s, ok := n.(*ast.SelectorExpr); ok && (s.Sel.Name == "LockResourcePool" || s.Sel.Name == "LockGrantPool") { + v = append(v, Violation{File: f.path, Line: fset.Position(s.Sel.Pos()).Line, Rule: "pool-lock-site", + Message: "LockResourcePool or LockGrantPool called outside internal/entitlements/tx.go: lock pools through LockPools or LockPoolOfGrant (docs/go-conventions.md)"}) + } + return true + }) + return v + } + + // locksPool reports whether SQL text, its -- comments removed, names + // resource_pools and takes a row lock. + func locksPool(text string) bool { + var b strings.Builder + for _, line := range strings.Split(text, "\n") { + code, _, _ := strings.Cut(line, "--") + b.WriteString(code + "\n") + } + s := b.String() + return strings.Contains(s, "resource_pools") && rowLockRe.MatchString(s) + } + + // poolQueryBlocks reports each "-- name:" block of a query file that locks + // a pool row, unless it is the file's allowed block (poolLockQueries); text + // before the first block is a block with no name. + func poolQueryBlocks(f sourceFile, rel string) []Violation { + var v []Violation + name, start := "", 1 + var block strings.Builder + flush := func() { + if locksPool(block.String()) && (name == "" || name != poolLockQueries[rel]) { + v = append(v, Violation{File: f.path, Line: start, Rule: "pool-lock-site", + Message: fmt.Sprintf("query %s locks a core.resource_pools row: lock pools through LockPools or LockPoolOfGrant (docs/go-conventions.md)", name)}) + } + block.Reset() + } + for i, line := range strings.Split(string(f.src), "\n") { + if rest, ok := strings.CutPrefix(strings.TrimSpace(line), "-- name: "); ok { + flush() + name, _, _ = strings.Cut(rest, " ") + start = i + 1 + continue + } + block.WriteString(line + "\n") + } + flush() + return v + } + + // goString is a string literal of a Go file, or a chain of literals joined + // by +, with the line of its first literal. + type goString struct { + line int + text string + } + + // literalChains returns every goString of f. A file the parser refuses comes back + // as a violation naming what has not been checked. + func literalChains(f sourceFile, checked string) ([]goString, []Violation) { + fset := token.NewFileSet() + file, err := parser.ParseFile(fset, f.path, f.src, parser.SkipObjectResolution) + if err != nil { + return nil, []Violation{{File: f.path, Rule: "unreadable-file", Message: "the lint cannot parse this file, so it has not checked its " + checked + ": " + err.Error()}} + } + var out []goString + ast.Inspect(file, func(n ast.Node) bool { + e, ok := n.(ast.Expr) + if !ok { + return true + } + text, ok := literalChain(e) + if !ok { + return true + } + out = append(out, goString{line: fset.Position(e.Pos()).Line, text: text}) + return false + }) + return out, nil + } + + // literalChain is the text of a string literal, or of a chain of string + // literals joined by +; it is false when any leaf of the expression is not a + // string literal. + func literalChain(e ast.Expr) (string, bool) { + switch e := e.(type) { + case *ast.BasicLit: + if e.Kind != token.STRING { + return "", false + } + s, err := strconv.Unquote(e.Value) + return s, err == nil + case *ast.BinaryExpr: + if e.Op != token.ADD { + return "", false + } + l, ok := literalChain(e.X) + if !ok { + return "", false + } + r, ok := literalChain(e.Y) + return l + r, ok + } + return "", false + } + + // poolStrings reports each goString of a Go file that locks a pool row. + func poolStrings(f sourceFile) []Violation { + strs, _ := literalChains(f, "pool row locks") // advisorySites reports a file the parser refuses + var v []Violation + for _, s := range strs { + if locksPool(s.text) { + v = append(v, Violation{File: f.path, Line: s.line, Rule: "pool-lock-site", + Message: "a core.resource_pools row lock outside LockResourcePool and LockGrantPool: lock pools through LockPools or LockPoolOfGrant (docs/go-conventions.md)"}) + } + } + return v + } + + // ruleLockFamilies reports the family table of docs/database-locks.md when + // its names differ from db.Families, in a name or in order. The table is + // the one under "## Advisory key families"; each row's first cell is a + // family's name in backticks. + func ruleLockFamilies(root string) []Violation { + path := filepath.Join(root, locksDoc) + src, err := os.ReadFile(path) + if err != nil { + return []Violation{unreadableFile(path, err)} + } + var listed []string + heading, in := 0, false + for i, line := range strings.Split(string(src), "\n") { + switch { + case strings.HasPrefix(line, "## Advisory key families"): + in, heading = true, i+1 + case in && strings.HasPrefix(line, "## "): + in = false + case in && strings.HasPrefix(line, "| `"): + name, _, _ := strings.Cut(strings.TrimPrefix(line, "| `"), "`") + listed = append(listed, name) + } + } + want := make([]string, 0, len(db.Families)) + for _, f := range db.Families { + want = append(want, f.Name()) + } + if slices.Equal(listed, want) { + return nil + } + return []Violation{{File: path, Line: heading, Rule: "lock-families", + Message: fmt.Sprintf("the advisory key families table lists %v, want %v, the order of db.Families (internal/db/locks.go)", listed, want)}} + } + ``` + + `go run . lint` reports nothing on the commit. A finding is a lock site an item of section 2, 3 or 5 missed, or a comment that still quotes an advisory call, which is reworded; any other is a stop. + +## 5. The existing tests and the goldens + +- [x] 5.1 The tests that take, probe or wait on an advisory lock (design D6): + - `internal/entitlements/subscription_delivery_db_test.go:499`: `db.Lock(ctx, subLock, db.Subscription, fx.stripeID)`; `subscription_resume_db_test.go:344`: `db.Lock(f.w.Ctx, holder, db.Subscription, "sub_"+f.sub.ID)`; `internal/server/operator_sync_stripe_db_test.go:319`: `db.Lock(context.Background(), side, db.ProductSync, productID)`; each keeps its failure text. + - `internal/fulfillment/release_schedule_after_commit_test.go`, `holdSubscriptionLock`: `free, err := dbtest.TryLock(ctx, side, db.Subscription, stripeSubID)`, its two returns unchanged; `trace_db_test.go`, `probeSubscriptionLock`: `free, err := dbtest.TryLock(e.ctx, tx, db.Subscription, e.subID)`, the failure text unchanged. + - `internal/entitlements/plan_act_queries_db_test.go:51`: `if err := db.Lock(context.Background(), q.Tx(), db.Materialization); err != nil { t.Fatalf("take the rendezvous: %v", err) }`; `:369`: the same on `retrying`. + - `internal/entitlements/plan_act_drain_db_test.go:839` and `internal/server/operator_org_type_change_test.go:839`: `Query: db.Materialization.Statement()`; `internal/server/plan_acts_concurrency_db_test.go:40`: `exclusiveRendezvous` becomes a variable, `db.Materialization.Statement()`. + - `internal/integration/delivery_queries_db_test.go:471`: `db.Lock(d.ctx, , db.DesiredState, r.conn, "organization", org)`; `:668`: `db.Lock(l.ctx, , db.Lifecycle, l.instance)`; `internal/integrations/fedwiki/store/request_intake_db_test.go:91`, `:195`, `internal/integrations/fedwiki/web/request_identity_db_test.go:273` and `internal/integrations/fedwiki/workflows/purge_failures_db_test.go:81`: `db.Lock(, , db.Lifecycle, )`. A fixture that keeps only a querier gains its transaction. `purge_failures_db_test.go` drops its `integration` import; `plan_act_queries_db_test.go` imports `context`. + - `internal/db/acts_test.go:34-35`: the two constants read `hashtextextended('materialization:', 0)`. `internal/db/settlement_pool_first_test.go` gains its own `sharedRendezvousBeforeFamilies`, `SELECT pg_advisory_xact_lock_shared(hashtextextended('entitlement_materialization', 0))`, with the comment "the key core.assert_rendezvous checks at core versions before the materialization family", and uses it in place of `sharedRendezvous`. + - `internal/domains/registry_db_test.go`: `registryLockStatement` reads `LockDomainsRegistry`. + +- [x] 5.2 The recording callers that pass a transaction: `internal/integration/lifecycle_db_test.go` (`d.tx`, `newer`, `tx`) and `internal/integrations/fedwiki/store/request_intake_db_test.go:219` call the `-InTx` forms; `internal/integration/lifecycle_answers_golden_db_test.go:148-150` reads `record := integration.RecordLifecycleRequestInTx` and `record = integration.RecordFirstLifecycleRequestInTx` (the answers golden does not move: the statement keeps the name `LockLifecycleInstance`); `TestRecordLifecycleRequestOwnTransaction_DB`'s map holds `func(context.Context, db.Beginner, integration.LifecycleRequestSpec) (integration.LifecycleAnswer, error)`. + +- [x] 5.3 The goldens (design D7), edited by hand, one fixed substitution per line: + - in every file under a `testdata` directory, `SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))` reads `LockMaterializationShared` (93 lines in 60 files) and `SELECT pg_advisory_xact_lock(hashtextextended($1, 0))` reads `LockMaterialization` (53 lines in 52 files), the JSON goldens keeping their quotes and commas; + - `SELECT pg_advisory_xact_lock(hashtext($1)::bigint)` reads `LockSubscription` under `internal/fulfillment/testdata` (41 lines in 40 files), `LockInvoice` under `internal/integrations/stripe/workflows/testdata` (12 in 10) and `LockProductSync` under `internal/server/testdata` (3 in 3). + + Afterwards `git grep -n -E 'pg_advisory|hashtext' -- '*/testdata/*'` finds nothing, and the diff of the testdata directories holds those 202 lines and no other. + +## 6. The new tests + +- [x] 6.1 `internal/db/locks_test.go`, package `db_test`, no database: + - `TestFamilyKeys`, rows through `checkKey`, each a family, its id parts and the key or an error substring; `W` is `a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11` and `C` the same in capitals. + - `the rendezvous`: `Materialization`, none; `materialization:`. + - `the registry`: `DomainsRegistry`, none; `domains-registry:`. + - `a workspace`: `FedWikiWorkspace`, `W`; `fedwiki-workspace:W`. + - `an instance id in capitals`: `Lifecycle`, `C`; `lifecycle:W`. + - `a subscription`: `Subscription`, `sub_123`; `stripe-subscription:sub_123`. + - `an invoice`: `Invoice`, `in_123`; `stripe-invoice:in_123`. + - `a product`: `ProductSync`, `C`; `stripe-product-sync:W`. + - `an address`: `DesiredState`, `C`, `person`, `W`; `desired-state:W:person:W`. + - `an empty subscription id`: `Subscription`, `""`; error `id part 1 is empty`. + - `a malformed product id`: `ProductSync`, `p1`; error `stripe-product-sync: id part 1`. + - `a colon in a subscription id`: `Subscription`, `sub:1`; error `id part 1 holds ':'`. + - `two parts of an address`: `DesiredState`, `W`, `person`; error `2 id parts, want 3`. + - `an id given to the rendezvous`: `Materialization`, `W`; error `1 id parts, want 0`. + - `no family`: `db.Family{}`, `W`; error `no family`. + + Failure texts: `%s: the key is %q, want %q`; `%s: the key is %q with no error, want an error containing %q`; `%s: the error is %v, want one containing %q`. + - `TestStatementsNameTheirFamily`: for each of `Families`, `Statement()` begins `-- name: ` and ends `SELECT pg_advisory_xact_lock(hashtextextended($1, 0))`, and `SharedStatement()` ends `SELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))` (failure text `%s's statements read %q and %q, want a name line and the exclusive and shared locks`); no two families share a first line (failure text `%s and %s share the name line %q`); `Families` holds eight families with eight distinct names (failure text `Families holds %v, want eight distinct names`). + +- [x] 6.2 `internal/db/locks_db_test.go`, package `db_test`, on `dbtest.Open(t)`: + - `TestLockTakesOneKeyPerId`, rows named by the family's name (`stripe-subscription`) with ids as in 6.1. A holder transaction takes `db.Lock(f, id...)`; a second, on a goroutine, takes the same; `dbtest.AwaitLockWait(t, d, dbtest.Backend{BlockedBy: holderPID})`. While it waits, the waiter's `query` from `pg_stat_activity WHERE $1 = ANY(pg_blocking_pids(pid))` begins with the family's `Statement()` first line (failure text `%s: the waiter's statement reads %q, want it to begin with its family's name line`). The holder's lock is granted under `hashtextextended(, 0)` in `pg_locks` (`classid` and `objid` from the key's halves, `objsubid = 1`; failure text `%s: no granted lock on hashtextextended(%q, 0)`). For a family with an id, `dbtest.TryLock` of another id on a third transaction is true (failure text `%s: the lock of another id was held`). The holder rolls back and the waiter returns nil (failure text `%s: the waiter returned %v`). + - `TestSharedHoldersDoNotWait`: two transactions take `db.LockShared(Materialization)` one after the other, the second under `SET LOCAL lock_timeout = '1s'`; the second returns nil (failure text `two shared holders of materialization waited for each other: %v`). + - `TestUnchangedFamiliesKeepTheirKeys`: `SELECT hashtextextended('lifecycle:' || $1::uuid::text, 0) = hashtextextended($2, 0)` with `C` and `Lifecycle.Key(C)`, and `SELECT hashtextextended('desired-state:' || $1::uuid::text || ':' || $2 || ':' || $3::uuid::text, 0) = hashtextextended($4, 0)` with `C`, `organization`, `C` and `DesiredState.Key(C, "organization", C)`; each true (failure text `%s: the family's key differs from the key the console took before`). + +- [x] 6.3 `internal/db/rendezvous_family_key_test.go`, `TestRendezvousFamilyKeyMigration`, built as `settlement_pool_first_test.go` builds its scratch database: `dbtest.UpTo` the version before the migration of item 3.1, then `dbtest.ApplyVersion` up, then down. At each of the three steps, in a transaction per key, the old key's literal and `db.LockShared(ctx, tx, db.Materialization)` each followed by `SELECT core.assert_rendezvous(false)`: before and after down, the old key passes and the new one refuses with `materialization_rendezvous_missing`; after up, the reverse. Failure text `%s, the assertion %s the %s key, want it to %s` (the step: `before the migration`, `after the migration`, `after its down`; `passed` or `refused`; `old` or `new`). Before the migration it reads `pg_get_functiondef('core.assert_rendezvous(boolean)'::regprocedure)`, `pg_get_userbyid(proowner)` and `proacl::text`; after the migration and after its down the owner and ACL are the same (failure text `%s, core.assert_rendezvous's owner or grants read %s, want %s`), and after its down the definition is the one read before (failure text `after its down, core.assert_rendezvous reads %q, want the definition before the migration`). + +- [x] 6.4 `internal/integration/lifecycle_db_test.go`, `TestARecordingOnAConnectionHoldsTheInstanceLock`: a connection pinned with `database.Conn(ctx)`; `integration.SetAfterInstanceLock` set to a function that opens a side transaction on `database`, runs `dbtest.TryLock(ctx, side, db.Lifecycle, id)` and rolls back, recording whether the lock was free, and reset in cleanup. `integration.RecordLifecycleRequest(ctx, conn, spec)` for a committed fixture's new instance answers accepted (failure text `the recording on a pinned connection answered %v, %v, want accepted`), and the side found the lock held (failure text `a recording on a pinned connection left the instance lock free after taking it`). + +- [x] 6.5 `internal/integrations/fedwiki/web/workspace_lock_db_test.go`, package `web`, `TestWorkspaceTxTakesTheWorkspaceLock`: `h := &FedWikiPartialsHandler{Database: dbtest.Open(t)}`; two workspace ids, `uuid.NewString()`. A side transaction takes `db.Lock(ctx, side, db.FedWikiWorkspace, ws)`. On a goroutine, `h.withWorkspaceTx(ctx, ws, func(...) error { return nil })`; `dbtest.AwaitLockWait(t, h.Database, dbtest.Backend{BlockedBy: sidePID})`. Meanwhile `h.withWorkspaceTx` under a five-second context for the other workspace returns nil (failure text `a rotation of another workspace waited for workspace %s's lock: %v`). The side rolls back and the first returns nil (failure text `the rotation returned %v`). + +- [x] 6.6 `internal/workflows/desiredstate/record_lock_db_test.go`, package `desiredstate_test`, `TestLockAddressTakesTheAddressKey`: a transaction runs `desiredstate.LockAddress` for an address of new UUIDs and kind `person` (no record, so the record lock finds none); a side transaction's `dbtest.TryLock(ctx, side, db.DesiredState, conn, "person", recipient)` is false (failure text `lockAddress left the address %v free`). + +- [x] 6.7 `internal/testkit/dbtest/locks_test.go`, `TestTryLock`, the kit's own test of item 1.2: free on a fresh key (failure text `TryLock found a fresh key held`), held while another transaction holds `db.Lock` of the same id (failure text `TryLock found a held key free`), and an error for `db.Subscription` with an empty id (failure text `TryLock took an empty id`). + +- [x] 6.8 `internal/lint/locks_test.go`, package `lint`. Each Go file of a row is a whole file: `package x`, a string as a top-level `const`, and a call or assignment inside `func f() { ... }`. `writeTree(t, files map[string]string) string` writes each relative path under `t.TempDir()` and returns the root. `TestLockSites` runs `ruleLockSites(root)` per row and compares the findings, as `rule relpath` pairs in order, with the row's (failure text `%s: the lint reported %v, want %v`): + - `an advisory lock in a handler`: `internal/server/x.go` holding `const q = "SELECT pg_advisory_xact_lock(hashtext($1)::bigint)"`; `advisory-lock-site internal/server/x.go`. + - `an advisory lock in a query file`: `internal/billing/queries/q.sql` holding `SELECT pg_advisory_xact_lock(1);`; one finding. + - `a try lock in a test`: `internal/fulfillment/x_test.go` holding `pg_try_advisory_xact_lock(hashtext($1)::bigint)`; one finding. + - `the function name in a literal of its own`: `internal/x/y.go` holding `lockFn := "pg_advisory_xact_lock_shared"`; one finding. `in capitals`: `internal/x/z.go` holding `const q = "SELECT PG_ADVISORY_XACT_LOCK(1)"`; one finding. `split across two literals`: `internal/x/w.go` holding `const q = "SELECT pg_advisory_" + "xact_lock(1)"`; one finding. + - `the helper`, `the test kit's probe`, `an internal/db test`, `the rendezvous's migration`, `a testdata file` (`internal/db/locks.go`, `internal/testkit/dbtest/locks.go`, `internal/db/x_test.go`, `internal/db/migrations/00018_entitlement_set_changes.sql`, `internal/x/testdata/a.sql`, each holding `pg_advisory_xact_lock(1)`; the Go files as a string): none. `a migration that is not the rendezvous's`: `internal/db/migrations/00041_x.sql` holding `SELECT pg_advisory_xact_lock(1);`; one finding. + - `a command outside internal/`: `cmd/x.go` holding `const q = "SELECT pg_advisory_xact_lock(1)"`, `scripts/x.sh` holding `psql -c 'SELECT pg_advisory_xact_lock(1)'` and `main.go` holding the Go string; one finding each, listed in the order `main.go`, `cmd/x.go`, `scripts/x.sh`. + - `a comment naming the function`: `internal/x/y.go` holding `// pg_advisory_xact_lock, released at commit`, `internal/x/z.sql` holding `-- pg_advisory_xact_lock(1)` and `internal/x/z.sh` holding `# pg_advisory_xact_lock 1`; none. + - `a testdata directory the lint cannot read`: `test/testdata/stack/a.sql` holding `pg_advisory_xact_lock(1)`, its directory then made unreadable (`os.Chmod(dir, 0)`, restored in cleanup), as the stack's own data under `test/testdata` is; none, and no `unreadable-file`. + - `a directory that begins with a dot`: `scripts/agent-runner/.code/internal/x.go` holding `const q = "SELECT pg_advisory_xact_lock(1)"`; none. + - `a comment marker in a SQL string`: `internal/x/q.sql` holding `SELECT '--', pg_advisory_xact_lock(1);`; one finding. `$# before a lock in a shell line`: `scripts/y.sh` holding `[ $# -eq 1 ] && psql -c 'SELECT pg_advisory_xact_lock(1)'`; one finding. `a hash in a heredoc`: `scripts/z.sh` holding `psql <<'SQL'\nSELECT 1 # x, pg_advisory_xact_lock(1);\nSQL`; one finding. + - `a seed with the rendezvous`: `test/mockshot-seed.sql` holding `BEGIN;\n-- a comment\n\nSELECT pg_advisory_xact_lock(hashtextextended('materialization:', 0));`; none. `a shell seed with the rendezvous`: `test/seed/discourse/seed-discourse.sh` holding `psql <<'SQL'\nBEGIN;\n-- a comment\nSELECT pg_advisory_xact_lock(hashtextextended('materialization:', 0));\nSQL`; none. `a seed that takes the rendezvous after an insert`: `test/mockshot-seed.sql` holding `BEGIN;\nINSERT INTO core.users (oidc_subject) VALUES ('x');\nSELECT pg_advisory_xact_lock(hashtextextended('materialization:', 0));`; one finding. `a seed's rendezvous outside a transaction`: the lock line alone; one finding. `a SQL comment in a shell file`: `test/x.sh` holding `psql <<'SQL'\n-- takes pg_advisory_xact_lock first\nSELECT 1;\nSQL`; none. + - `a seed with the old key`: `test/mockshot-seed.sql` holding `SELECT pg_advisory_xact_lock(hashtextextended('entitlement_materialization', 0));`; one finding. + - `the pool lock queries`: `internal/entitlements/queries/resource_pools.sql` holding `LockResourcePool`'s block and `grants.sql` holding `LockGrantPool`'s, as at HEAD; none. + - `a pool lock query under another name`: `resource_pools.sql` holding also `-- name: LockPoolAgain :one` with `SELECT pool_id FROM core.resource_pools WHERE pool_id = $1 FOR UPDATE;`; `pool-lock-site` at that block. + - `a key-share pool lock in another package`: `internal/billing/queries/x.sql` holding a block that selects from `core.resource_pools` `FOR KEY SHARE`; one finding. + - `a comment naming the lock`: a block whose `--` comment says `locked FOR UPDATE` and names `core.resource_pools`, and whose SQL reads `core.resource_pools` without a locking clause; none. + - `a pool lock in a Go string`: `internal/server/x.go` holding a raw string `SELECT 1 FROM core.resource_pools WHERE pool_id = $1 FOR UPDATE`; one finding. `a pool lock split across two literals`: `"SELECT 1 FROM core.resource_pools WHERE pool_id = $1 " + "FOR UPDATE"`; one finding. + - `a pool lock in a test` and `a pool lock in the test kit` (`internal/x/y_test.go`, `internal/testkit/world/x.go`): none. + - `a pool query called outside its helper`: `internal/fulfillment/x.go` holding `q.LockResourcePool(ctx, id)`; one finding. `a method value of the pool query`: `lock := entQ.LockResourcePool`; one finding. `the pool helper`: the same call in `internal/entitlements/tx.go`; none. `generated query code`: `internal/entitlements/resource_pools.sql.go` holding the query's text; none. + + `TestLockFamilies` runs `ruleLockFamilies(root)` on a `docs/database-locks.md` per row (failure text `%s: the lint reported %v, want %v`): `the table as the code declares it` (the eight names in order), none; `a family missing` (without `desired-state`), one `lock-families` finding; `two families swapped` (`stripe-invoice` before `stripe-subscription`), one; `no table`, one. + +## 7. Breaks + +- [x] 7.1 Package `./internal/db` and file `internal/db/locks.go` unless named; a test record's `run_new` is the test and, for a row, its subtest. + - the key without its separator (`old` `return f.name + ":" + strings.Join(parts, ":"), nil`, `new` `return f.name + strings.Join(parts, ":"), nil`): caught by `TestFamilyKeys/the_rendezvous`, `want_new` `the rendezvous: the key is`, and by `TestUnchangedFamiliesKeepTheirKeys`, `want_new` `the family's key differs`; + - a UUID kept as written (`old` `parts[i] = u.String()`, `new` `parts[i], _ = p, u`): caught by `TestFamilyKeys/an_instance_id_in_capitals`, `want_new` `an instance id in capitals: the key is`; + - an empty id accepted (`old` `if p == "" {`, `new` `if p == "-" {`): caught by `TestFamilyKeys/an_empty_subscription_id`, `want_new` `an empty subscription id: the key is`; + - too few parts accepted (`old` `if len(id) != len(f.parts) {`, `new` `if len(id) > len(f.parts) {`): caught by `TestFamilyKeys/two_parts_of_an_address`, `want_new` `two parts of an address: the key is`; + - too many parts accepted (the same `old`, `new` `if len(id) < len(f.parts) {`): caught by `TestFamilyKeys/an_id_given_to_the_rendezvous`, `want_new` `index out of range`; + - a colon accepted in a text part (`old` `if strings.Contains(p, ":") {`, `new` `if strings.Contains(p, "::") {`): caught by `TestFamilyKeys/a_colon_in_a_subscription_id`, `want_new` `a colon in a subscription id: the key is`; + - the statement without its name line (here and in the next record, `\n` is the escape inside the Go string, written `\\n` in the list's JSON; `old` `return "-- name: " + f.label + " :exec\nSELECT pg_advisory_xact_lock(hashtextextended($1, 0))"`, `new` `return "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))"`): caught by `TestStatementsNameTheirFamily`, `want_new` `want a name line and the exclusive and shared locks`; + - the shared statement exclusive (`old` `Shared :exec\nSELECT pg_advisory_xact_lock_shared(hashtextextended($1, 0))"`, `new` `Shared :exec\nSELECT pg_advisory_xact_lock(hashtextextended($1, 0))"`): caught by `TestSharedHoldersDoNotWait`, `want_new` `two shared holders of materialization waited`; + - the migration's old key (`internal/db/migrations/00040_rendezvous_family_key.sql`, `old` `v_key BIGINT := hashtextextended('materialization:', 0);`, `new` `v_key BIGINT := hashtextextended('entitlement_materialization', 0);`): caught by `TestRendezvousFamilyKeyMigration`, `want_new` `after the migration, the assertion`; + - the migration's down not exact (`internal/db/migrations/00040_rendezvous_family_key.sql`, `old` `v_key BIGINT := hashtextextended('entitlement_materialization', 0);`, `new` `v_key BIGINT := hashtextextended('entitlement_materialization', 0); -- changed`): caught by `TestRendezvousFamilyKeyMigration`, `want_new` `after its down, core.assert_rendezvous reads`; + - the rendezvous under another family (`internal/entitlements/tx.go`, `pkg` `./internal/entitlements`, `old` `if err := lock(ctx, tx, db.Materialization); err != nil {`, `new` `if err := lock(ctx, tx, db.DomainsRegistry); err != nil {`): caught by `TestRuleChangeCommitGolden/atomic`, `want_new` `materialization_rendezvous_missing`; + - the registry under another family (`internal/domains/registry.go`, `pkg` `./internal/domains`, `old` `if err := db.Lock(ctx, tx, db.DomainsRegistry); err != nil {`, `new` `if err := db.Lock(ctx, tx, db.Invoice, "registry"); err != nil {`): caught by `TestRegistryLockIsTheFirstStatement`, `want_new` `want begin and then the registry lock`; + - the workspace under another family (`internal/integrations/fedwiki/web/site_actions.go`, `pkg` `./internal/integrations/fedwiki/web`, `old` `if err := db.Lock(ctx, tx, db.FedWikiWorkspace, workspaceID); err != nil {`, `new` `if err := db.Lock(ctx, tx, db.Lifecycle, workspaceID); err != nil {`): caught by `TestWorkspaceTxTakesTheWorkspaceLock`, `want_new` `no backend waited on a lock held by backend`; + - the reconcile under another family (`internal/fulfillment/reconcile.go`, `pkg` `./internal/fulfillment`, `old` `if err := db.Lock(ctx, tx, db.Subscription, stripeSubscriptionID); err != nil {\n\t\treturn plan,`, `new` the same with `db.Invoice`): caught by `TestReconcileWaitsForTheSubscriptionLock`, `want_new` `top tier attach while the lock is held = 1, want 0`, and by `TestReconcileTrace/replay`, `want_new` `replay.golden differs from what the code produced`; + - supersede under another family (the same file and package, `old` `if err := db.Lock(ctx, tx, db.Subscription, stripeSubscriptionID); err != nil {\n\t\treturn fmt.Errorf("reconcile: advisory lock after releasing the schedule`, `new` the same with `db.Invoice`): caught by `TestSupersedeAfterAReleaseTakesTheSubscriptionLock`, `want_new` `without waiting for the other transaction`; + - the invoice under another family (`internal/integrations/stripe/workflows/invoice_reconcile.go`, `pkg` `./internal/integrations/stripe/workflows`, `old` `if err := db.Lock(ctx, tx, db.Invoice, r.id); err != nil {`, `new` the same with `db.Subscription`): caught by `TestInvoicePaymentConvergence/the_same_read_again`, `want_new` `the-same-read-again.golden differs from what the code produced`; + - the product sync under another family (`internal/integrations/stripe/catalogsync/catalogsync.go`, `pkg` `./internal/server`, `old` `if err := db.Lock(ctx, tx, db.ProductSync, productID); err != nil {`, `new` the same with `db.Invoice`, which takes any text): caught by `TestSyncProductToStripeRechecksUnderTheLock`, `want_new` `the sync answered without waiting for the product lock`; + - the recording's lock on another instance (`internal/integration/lifecycle.go`, `pkg` `./internal/integration`, `old` `if err := db.Lock(ctx, tx, db.Lifecycle, instanceID); err != nil {`, `new` `if err := db.Lock(ctx, tx, db.Lifecycle, uuid.NewString()); err != nil {`): caught by `TestARecordingOnAConnectionHoldsTheInstanceLock`, `want_new` `left the instance lock free`; + - FedWiki's intake on another instance (`internal/integrations/fedwiki/store/request_intake.go`, `pkg` `./internal/integrations/fedwiki/store`, `old` `if err := db.Lock(ctx, tx, db.Lifecycle, spec.InstanceID); err != nil {`, `new` `if err := db.Lock(ctx, tx, db.Lifecycle, spec.WorkspaceID); err != nil {`): caught by `TestIntakeAndPurgeOrder_DB`, `run_new` `^TestIntakeAndPurgeOrder_DB$/^purge_first$`, `want_new` `intake crossed purge without refusing`; + - the address under another kind (`internal/workflows/desiredstate/record.go`, `pkg` `./internal/workflows/desiredstate`, `old` `db.Lock(ctx, tx, db.DesiredState, a.ConnectionID, string(a.Kind), a.RecipientID)`, `new` `db.Lock(ctx, tx, db.DesiredState, a.ConnectionID, "x", a.RecipientID)`): caught by `TestLockAddressTakesTheAddressKey`, `want_new` `lockAddress left the address`; + - an advisory call outside the helpers (command record; `internal/integrations/stripe/workflows/invoice_reconcile.go`, `old` `func (r *invoiceReconcile) converge(`, `new` `const _ = "SELECT pg_advisory_xact_lock(1)"\n\nfunc (r *invoiceReconcile) converge(`, `cmd` `go run . lint`, `exit` 1): `want_new` `an advisory lock outside internal/db/locks.go`; + - a seed's old key (command record; `test/mockshot-seed.sql`, `old` `SELECT pg_advisory_xact_lock(hashtextextended('materialization:', 0));`, `new` `SELECT pg_advisory_xact_lock(hashtextextended('entitlement_materialization', 0));`, the same `cmd` and `exit`): `want_new` `a seed takes an advisory lock other than the materialization rendezvous`; + - a second pool lock query (command record; `internal/billing/queries/plan_ladders.sql`, `old` `-- name: LockPlanLadderOrder :many`, `new` `-- name: LockPoolAgain :one\nSELECT pool_id FROM core.resource_pools WHERE pool_id = $1 FOR UPDATE;\n\n-- name: LockPlanLadderOrder :many`, the same `cmd` and `exit`): `want_new` `query LockPoolAgain locks a core.resource_pools row`; + - a pool lock in a Go string (command record; `internal/fulfillment/reconcile.go`, `old` `func reconcileItems(`, `new` `const _ = "SELECT 1 FROM core.resource_pools WHERE pool_id = $1 FOR UPDATE"\n\nfunc reconcileItems(`, the same `cmd` and `exit`): `want_new` `a core.resource_pools row lock outside LockResourcePool and LockGrantPool`; + - a pool query taken outside its helper (command record; the same file, `old` `func reconcileItems(`, `new` `var _ = (*entitlements.Queries).LockResourcePool\n\nfunc reconcileItems(`, the same `cmd` and `exit`): `want_new` `LockResourcePool or LockGrantPool called outside internal/entitlements/tx.go`; + - a family missing from the page (command record; `docs/database-locks.md`, `old` ``| `desired-state` |``, `new` `| desired-state |`, the same `cmd` and `exit`): `want_new` `the advisory key families table lists`; + - a seed's rendezvous after another statement (command record; `test/mockshot-seed.sql`, `old` `BEGIN;\n-- This transaction writes a rule`, `new` `BEGIN;\nSELECT 1;\n-- This transaction writes a rule`, the same `cmd` and `exit`): `want_new` `a seed takes the materialization rendezvous other than as the first statement after BEGIN`; + - the check that the rendezvous follows `BEGIN;` (`internal/lint/locks.go`, `pkg` `./internal/lint`, `old` `strings.EqualFold(prev, "BEGIN;")`, `new` `strings.EqualFold(prev, prev)`, `run_new` `^TestLockSites$/^a_seed_that_takes_the_rendezvous_after_an_insert$`): `want_new` `a seed that takes the rendezvous after an insert: the lint reported`; + - testdata entered (`internal/lint/locks.go`, `pkg` `./internal/lint`, `old` `if d.Name() == "testdata" ||`, `new` `if d.Name() == "testdata-never" ||`): caught by `TestLockSites/a_testdata_directory_the_lint_cannot_read`, `want_new` `a testdata directory the lint cannot read: the lint reported`; + - a dot directory entered (`internal/lint/locks.go`, `pkg` `./internal/lint`, `old` `strings.HasPrefix(d.Name(), ".")`, `new` `strings.HasPrefix(d.Name(), ".never")`, `run_new` `^TestLockSites$/^a_directory_that_begins_with_a_dot$`): caught by `TestLockSites/a_directory_that_begins_with_a_dot`, `want_new` `a directory that begins with a dot: the lint reported`; + - a comment marker cut inside quotes (`internal/lint/locks.go`, `pkg` `./internal/lint`, `old` `case c == '\'' || (shell && c == '"'):`, `new` `case c == 0:`, `run_new` `^TestLockSites$/^a_comment_marker_in_a_SQL_string$`): caught by `TestLockSites/a_comment_marker_in_a_SQL_string`, `want_new` `a comment marker in a SQL string: the lint reported`; + - a hash taken as a comment anywhere (`old` `(i == 0 || line[i-1] == ' ' || line[i-1] == '\t')`, `new` `true`, `run_new` `^TestLockSites$/^\$#_before_a_lock_in_a_shell_line$`; the file, package and failure text as the record above): caught by `TestLockSites/$#_before_a_lock_in_a_shell_line`, `want_new` `$# before a lock in a shell line: the lint reported`; + - a heredoc read as shell (`old` `code = stripComment(line, shell && !inDoc)`, `new` `code = stripComment(line, shell)`, `run_new` `^TestLockSites$/^a_hash_in_a_heredoc$`; the file and package as the record above): caught by `TestLockSites/a_hash_in_a_heredoc`, `want_new` `a hash in a heredoc: the lint reported`; + + Named without a break, with the reason: the `*sql.Tx` parameters of `db.Lock`, `db.LockShared`, the `-InTx` recording forms, `lockLatest`, `lockAddress`, `hold` and `lockRequest`, because a break that passes a connection or a database does not compile, and the build is the check; the comments of section 2 and the seeds' comments. + +## 8. The documents + +- [x] 8.1 `docs/database-locks.md` (design D8): + - The opening's sentences "One piece of planned work changes the page: #186 gives every advisory key one family prefix and writes one lock order. Whoever lands it edits this page in the same change." go. + - The advisory table: each row's Key cell reads its family's key, `materialization:`, `domains-registry:`, `fedwiki-workspace:`, `lifecycle:`, `stripe-subscription:`, `stripe-invoice:`, `stripe-product-sync:`, `desired-state:::`, each under `hashtextextended(..., 0)`; the rendezvous row's Taken by adds the two seeds; the lifecycle row's names `RecordLifecycleRequestInTx`, `RecordFirstLifecycleRequestInTx`, FedWiki's `RecordSiteRequestInTx` and `CheckSwapRequestSites`, `usage.Hold`, `finishAllWith` and `purgeArchivedSite`; the desired-state row names `lockAddress`; the registry row names `registryKind`. + - "How advisory keys are built" is replaced by: + + > ## Advisory key families + > + > Every advisory key is `hashtextextended(':', 0)`: the family's name, a colon and the id, whose parts are joined by colons. `db.Lock` and `db.LockShared` (`internal/db/locks.go`) build it and take it, on a transaction, and nothing else names an advisory lock function (the project lint's `advisory-lock-site`). A UUID part is written in canonical form, as `uuid::text` writes it, and a text part holds no colon; a family with one lock for the whole database has no id. Each family's statement begins with a name line (`-- name: LockSubscription :exec`), which `pg_stat_activity` shows for a waiter and the statement traces record. The table lists the families in the order a transaction takes them; the lint fails when it and `db.Families` differ (`lock-families`). + > + > | Family | Id | Statement | Taken by | + > |---|---|---|---| + > | `materialization` | none | `LockMaterialization`, `LockMaterializationShared` | the `entitlements.Materializing` and `entitlements.RuleChange` kinds; `test/mockshot-seed.sql` and `test/seed/discourse/seed-discourse.sh`, as their first statement; `core.assert_rendezvous` checks its key | + > | `domains-registry` | none | `LockDomainsRegistry` | `registryKind`, the kind of `domains.Registry.WithLock` | + > | `fedwiki-workspace` | workspace id | `LockFedWikiWorkspace` | `withWorkspaceTx` | + > | `lifecycle` | instance id | `LockLifecycleInstance` | the lifecycle recording, FedWiki's intake, the slot gate, the outcome steps and the retention purge | + > | `stripe-subscription` | Stripe subscription id | `LockSubscription` | `fulfillment.ReconcileSubscription`, `supersedeReleasedDowngrades` | + > | `stripe-invoice` | Stripe invoice id | `LockInvoice` | `invoiceReconcile.converge` | + > | `stripe-product-sync` | product id | `LockProductSync` | `catalogsync.Sync` | + > | `desired-state` | connection id, recipient kind, recipient id | `LockDesiredStateAddress` | `lockAddress` in `internal/workflows/desiredstate` | + > + > A change to a family's name or to the shape of its id changes its keys. Two processes that build different keys for one object hold no common lock, so a release that changes a key is deployed as [production-deployment.md](production-deployment.md), section 5, orders an upgrade. + > + > ## The order every path keeps + > + > 1. A transaction that takes the materialization rendezvous takes it as its first statement. No other advisory lock comes before it. + > 2. The advisory families the transaction needs, in the order of the table above. Several locks of one family follow the order this page gives for that family: lifecycle instances in the order their requests were recorded. + > 3. Rows, by the rules below: a rule change's set and rule rows before pool rows; pool rows in ascending `pool_id`, before a grant's lineage head, obligation rows and a tier's product row; FedWiki's site rows, then their placement rows through the trigger, then the usage row, then request rows; a desired-state record after its address. + > + > No path waits on an advisory lock after it takes a row lock: a recording that takes an instance lock its transaction already holds, as the swap's second recording does, takes it without waiting. The sections below list each path's locks in this order. + + - "Lifecycle intake": "Called with a `*sql.DB`, each opens its own transaction; called with a transaction, each uses the caller's, which is how the handler records under the workspace lock." reads "`RecordLifecycleRequest` and `RecordFirstLifecycleRequest` open their own transaction on a database or a pinned connection; `RecordLifecycleRequestInTx` and `RecordFirstLifecycleRequestInTx` run in the caller's, which is how the handler records under the workspace lock." + - Registry: "The comment on `registryAllocationLock` states the rule" reads "The comment on `registryKind` states the rule". + - Rule 1 of "Rules every path keeps" gains "No other advisory lock precedes it." + - "Where the paths differ" loses its bullet "Advisory keys are built four ways (#186)". +- [x] 8.2 `docs/go-conventions.md`, "Transactions and pool locks": after "nothing reports the loss.", "An advisory lock goes through `db.Lock` or `db.LockShared` (`internal/db/locks.go`) with its family and id; the project lint refuses an advisory lock function anywhere else, and a pool row lock outside `LockPools` and `LockPoolOfGrant`." The list of helpers that take a `*sql.Tx` gains `integration.RecordLifecycleRequestInTx`, and the rule that a lock helper takes a `*sql.Tx` is stated for advisory locks: `LockPools` and `LockPoolOfGrant` take a pool row lock and stay methods of `*entitlements.Queries`, because the plan act runner's querier seam wraps them in tests and every caller builds `Queries` on a transaction. +- [x] 8.3 `docs/testing.md`: the `dbtest` list gains "`dbtest.TryLock` probes whether a family's lock is free, under the key the code builds. A test that holds a family's lock takes it with `db.Lock`, and one that waits on it gives `dbtest.Backend` the family's `Statement()`, so the test meets the key and the statement the code uses."; the table's `testkit/dbtest` row gains `TryLock`; the project lint's rules gain "locks: `advisory-lock-site`, `pool-lock-site`, `lock-families`". +- [x] 8.4 `docs/production-deployment.md`, section 5 (`:161-165`): "run `member-console migrate up` with it set as a one-shot step before each deploy" reads "run `member-console migrate up` with it set as a one-shot step after the old release stops and before the new one starts (below)", so the guide states one order. After "Check configuration before restarting.": "**Stop the old release before starting the new one.** Stop every process of the running release, run `member-console migrate up` if you run migrations as a separate step, then start the new release. Two releases running at once can build different keys for one advisory lock and then hold no common lock, so writes the lock serializes can interleave (`database-locks.md`); and a migration can change what the old release's writes need. An orchestrator's rolling update that starts the new task before it stops the old one, or a service with more than one replica, is stopped or scaled to zero first." +- [x] 8.4a `docs/building-an-integration.md:592`: "`RecordLifecycleRequest` in the same transaction as your own rows" reads "`RecordLifecycleRequestInTx` in the same transaction as your own rows". +- [x] 8.5 No other document changes; `design/` is not edited. + +## 9. Landing + +- [x] 9.1 With the test stack up, `make test` passes (it runs `make lint` first), and `openspec validate advisory-keys-name-their-family --strict` passes. +- [x] 9.2 The break list of item 7.1 reads CAUGHT in the commit message, with the breaks named without a record and why. +- [x] 9.3 The deploy runs no old process beside a new one (design D8). The step: stop every process of the running release; run `member-console migrate up` if migrations run as a separate step; start the new release. An orchestrator that starts the new task before it stops the old one, or a service with more than one replica, is scaled to zero first. Rolling back: stop every process of this release; as the migration user, in one transaction, run the Down section of `00040_rendezvous_family_key.sql` and `DELETE FROM goose_db_version_core WHERE version_id = 40`; then start the previous release (`member-console migrate down` cannot do it: design D5). The deploy changes no row; a deployment that uses the `console_reader` role re-runs `scripts/db-reader-role.sql` as the schema owner, as after every deploy that migrates. +- [ ] 9.4 After the push, post on #186 under `CONTRIBUTING.md`'s rules for forge text (item 1: it reads for someone who has never opened the repository, explains the console's terms once, and names no decision number, change name, date as narrative, deployment or count), and close it. Draft: + + > An advisory lock is a Postgres lock on a number of the application's choosing rather than on a row; the console takes them per transaction to serialize work on one object, such as two reconciles of one Stripe subscription. Each kind of object now has a family, and every key is built one way: `hashtextextended(':', 0)`, a UUID written in canonical form, so two spellings of one id take one lock. `db.Lock` and `db.LockShared` in `internal/db/locks.go` build the key and take the lock, on a `*sql.Tx` only, and every advisory lock in the code goes through them. Each family's statement begins with a name line such as `-- name: LockSubscription :exec`, so `pg_stat_activity` shows which family a waiting backend waits on, and the statement traces in the tests record that name. + > + > The families, in the order a transaction takes them: `materialization` (the shared or exclusive lock every transaction that recomputes an organization's limits, or changes a rule, takes first), `domains-registry`, `fedwiki-workspace`, `lifecycle`, `stripe-subscription`, `stripe-invoice`, `stripe-product-sync`, `desired-state`. `docs/database-locks.md` lists them in a table and writes one order every path keeps: that lock first, then the families in the table's order, then rows. The project lint fails when the table and the code's list disagree. + > + > The keys of `lifecycle` and `desired-state` are unchanged. The other six changed, the materialization lock's included, so a migration changes the key the database's own check of that lock looks for, and the two seeds that take it take the new one. Two processes that build different keys for one object share no lock, so this release was deployed with every process of the previous one stopped first; the production guide now states that order for every upgrade. + > + > The project lint now refuses an advisory lock function named, in any case, in a Go string or in SQL or shell code outside the helpers, the test kit's probe and its own wait tests, the lint rule's own file and tests, the two migrations that name the rendezvous and `internal/db`'s own tests; a seed that takes any advisory key but the materialization lock's, or takes that one anywhere but first after its `BEGIN;`; a row lock on `core.resource_pools` in any query but `LockResourcePool` and `LockGrantPool`, or in a Go string outside the tests; and a call or method value of those two queries outside `internal/entitlements/tx.go`, where `LockPools` and `LockPoolOfGrant` call them. Tests take a family's lock through the same helpers and probe one with `dbtest.TryLock`, so a test holds the key the code takes. + > + > Recording a lifecycle request now comes in two forms: `RecordLifecycleRequest` opens its own transaction on a database or a pinned connection, and `RecordLifecycleRequestInTx` runs in the caller's `*sql.Tx`. A connection handed to the first is no longer used outside a transaction, which would have released the instance lock as soon as it was taken; handed to the second, it does not compile. The same holds for `RecordFirstLifecycleRequest`. + > + > Commit: … + +- [ ] 9.5 The archive waits for the maintainer's go. It creates `openspec/specs/database-locks/spec.md`, whose Purpose the archive leaves as a placeholder the lint's `spec-purpose-placeholder` refuses, so the archive commit writes it: "Defines how the console takes advisory locks: one key built from a family and an id, helpers that take a transaction, and the families and the lock order written once in `docs/database-locks.md`." diff --git a/test/mockshot-seed.sql b/test/mockshot-seed.sql index ec7218bb..14d0cf2c 100644 --- a/test/mockshot-seed.sql +++ b/test/mockshot-seed.sql @@ -44,6 +44,11 @@ -- grants →19 with ≈2,669 signup defaults in the caption, Catalog -- products 10→15. BEGIN; +-- This transaction writes a rule through core.commit_rule_change, so it takes +-- the exclusive materialization rendezvous first, as every rule change does; +-- the key is the materialization family's (internal/db/locks.go), which the +-- function checks. +SELECT pg_advisory_xact_lock(hashtextextended('materialization:', 0)); -- 2,842 users, joined over ~19 months (i*5h + jitter). The subject carries -- the organization key the member's personal org gets below, which is how @@ -189,11 +194,8 @@ ON CONFLICT (key) DO NOTHING; -- The rule goes in through core.commit_rule_change, the only write path to -- core.entitlement_set_rules since migration 00018 revoked the DML. That --- function asserts the exclusive materialization rendezvous, so this --- transaction takes the advisory lock first. No pool carries the set yet, so --- the act row it files enumerates nothing. -SELECT pg_advisory_xact_lock(hashtextextended('entitlement_materialization', 0)); - +-- function asserts the exclusive materialization rendezvous. No pool carries +-- the set yet, so the act row it files enumerates nothing. SELECT core.commit_rule_change( es.set_id, 'rule_added', NULL, jsonb_build_object( diff --git a/test/seed/discourse/seed-discourse.sh b/test/seed/discourse/seed-discourse.sh index 7b827e87..3c255032 100755 --- a/test/seed/discourse/seed-discourse.sh +++ b/test/seed/discourse/seed-discourse.sh @@ -27,6 +27,11 @@ DSN="${MC_DB_DSN:?set MC_DB_DSN (see test/.env)}" psql "$DSN" -v ON_ERROR_STOP=1 <<'SQL' BEGIN; +-- This transaction writes a rule through core.commit_rule_change, so it takes +-- the exclusive materialization rendezvous first, as every rule change does; +-- the key is the materialization family's (internal/db/locks.go), which the +-- function checks. +SELECT pg_advisory_xact_lock(hashtextextended('materialization:', 0)); -- Seed person: the OIDC subject is its external reference, so a fake forum -- user can be created with a matching external id. @@ -61,11 +66,8 @@ ON CONFLICT (key) DO NOTHING; -- Seeds stay raw SQL by convention; the rule goes in through -- core.commit_rule_change, the only write path to core.entitlement_set_rules -- since migration 00018 revoked the DML. The function asserts the exclusive --- materialization rendezvous, so this transaction takes the advisory lock --- first. A boolean rule carries no resource_value and no stacking_policy: --- chk_entitlement_set_rules_type requires both NULL. -SELECT pg_advisory_xact_lock(hashtextextended('entitlement_materialization', 0)); - +-- materialization rendezvous. A boolean rule carries no resource_value and no +-- stacking_policy: chk_entitlement_set_rules_type requires both NULL. SELECT core.commit_rule_change( s.set_id, 'rule_added', NULL, jsonb_build_object(