member-console/docs/database-locks.md

title, audience, summary

title	audience	summary
Database Locks	developer	Every advisory lock and serializing row lock the console takes, how each key is built, and the order each path takes them in.

Database Locks

This page describes the locks as the code takes them today. It records what each lock serializes, who takes it, and the order each path takes several. It does not describe a target design.

The code names functions and sqlc queries; so does this page. Line numbers are left out because they move.

Every advisory lock is transaction-scoped (pg_advisory_xact_lock), so Postgres releases it at commit or rollback. No code takes a session-level advisory lock or LOCK TABLE. A plan act's listing batch takes FOR KEY SHARE on its pool rows; a tier insert locks its product FOR SHARE; FedWiki placement creators lock exactly one site FOR SHARE (see Registry below).

The locks

Advisory locks

Lock	Key	Mode	Taken by	Serializes
Materialization rendezvous	hashtextextended('materialization:', 0)	shared for a transaction that materializes a pool; exclusive for a rule change, for a change to a ladder's tiers (tier add, tier delete, tier removal, the reorder's renumbering) and for a default change's T0	the entitlements.Materializing and entitlements.RuleChange transaction kinds (BeginMaterializing, BeginRuleChange), as the transaction's first statement; test/mockshot-seed.sql and test/seed/discourse/seed-discourse.sh, as their first statement after BEGIN	A rule change or a tier change against every materializing transaction: each one finishes before the change or starts after it, so a conferral reads one set of rules and one order of each ladder. Materializing transactions do not wait on each other here.
Subscription	hashtextextended('stripe-subscription:<Stripe subscription id>', 0)	exclusive	fulfillment.ReconcileSubscription	Two reconciles of one subscription (the checkout return and a webhook), so they converge instead of racing the exclusion constraint on subscription periods.
Invoice	hashtextextended('stripe-invoice:<Stripe invoice id>', 0)	exclusive	invoiceReconcile.converge in internal/integrations/stripe/workflows	Two reconciles of one invoice (two events, or an event and the sweep), so the second finds what the first wrote.
Product sync	hashtextextended('stripe-product-sync:<product id>', 0)	exclusive	catalogsync.Sync in internal/integrations/stripe/catalogsync	Two sync clicks for one product, so the mapping writes and the outbox enqueue happen once.
FedWiki workspace	hashtextextended('fedwiki-workspace:<workspace id>', 0)	exclusive	withWorkspaceTx in internal/integrations/fedwiki/web, for the status-change and swap handlers	One workspace's rotations of its active site, so two tabs cannot interleave the cooldown check and stamp or the choice of the site to park.
Lifecycle instance	hashtextextended('lifecycle:<instance id>', 0)	exclusive	db.Lock on the lifecycle family, run by integration.RecordLifecycleRequestInTx and RecordFirstLifecycleRequestInTx, FedWiki's RecordSiteRequestInTx and CheckSwapRequestSites, usage.Hold, and the activity helpers finishAllWith and purgeArchivedSite	One instance's requests: the latest-generation check and the insert are one step, and an outcome is recorded against a request that is still accepted; and retention's check that no request is open, held through its farm delete and local deletion.
Desired-state address	hashtextextended('desired-state:<connection id>:<recipient kind>:<recipient id>', 0)	exclusive	lockAddress in internal/workflows/desiredstate	Every write to the content or generation of the record at one (connection, recipient) address, and to its projection, including an address that has no record yet.
Domains registry	hashtextextended('domains-registry:', 0)	exclusive	registryKind, the kind of domains.Registry.WithLock	Every mutating allocation in the registry, one lock for the whole namespace, because the disjointness rules span rows no database constraint can compare.

Row locks

FedWiki placement creation takes its site's FOR SHARE after the registry advisory lock and skip checks. A site write takes matching placement row locks through its trigger after the site row, before usage and request rows. These flag-only writes change neither claims nor allocation counts.

Lock	Row	Mode	Taken by	Serializes
Pool row	core.resource_pools row of one pool	FOR UPDATE; FOR KEY SHARE in a plan act's listing batch	the functions core.confer, core.end_conferral, core.align_conferral_shape, core.update_conferral_bounds and core.settle_obligation; LockPools (through the LockResourcePool query), called by RevokeGrantTx, ManageGrantTx, ResumeReplacedGrantsTx, ResumeSubscription, settlePool, reconcileItems, reconcileSuspended and reconcileEnded (the subscription reconcile's active, suspended and ended branches), runPlanAct (AddTier, RemoveTier and EnactObligation, a default change's move), and runPlanAct for a reorder's move (EnactObligation); LockPoolOfGrant (through the LockGrantPool query), called by ExpireGrantTx and ExtendWaitingGrantTx; and core.write_plan_act_obligations, which takes the pool rows of one listing batch FOR KEY SHARE	Everything that changes a pool's positions or entitlements: conferral, endings, a subscription's suspension and resume, grant acts, tier removal, a default change's move, rule-change settlement, grant resumption and a member's Resume plan. A listing batch's key-share lock conflicts with FOR UPDATE, so a batch and a move or a conferral on one of its pools wait for each other, and two batches of one act share the rows.
Entitlement set row and rule row	core.entitlement_sets row and core.entitlement_set_rules row	FOR UPDATE	core.commit_rule_change (the set row also by the LockEntitlementSet query, once per batch)	A rule change against a writer that changes the set or the rule without holding the rendezvous.
Change obligation row	core.entitlement_set_change_obligations row of one (change, pool)	FOR UPDATE	core.settle_obligation	Two passes settling one owed pool, so each effect is written once.
Plan act obligation row	core.plan_act_obligations row of one (act, pool)	FOR UPDATE by a move, a listing batch's supersession and Retry's requeue; UPDATE by the failure recorder	the LockPlanActObligation query (a move, after the pool row), which calls core.lock_plan_act_obligation; the function takes the row FOR UPDATE, core.write_plan_act_obligations (the open obligations of earlier acts on the batch's pools), core.requeue_plan_act_obligations (the retried act's failed obligations of the types it still owns) and core.record_plan_act_obligation_failure	Two passes settling one obligation, and a settlement against a supersession or a requeue. A batch and a Retry take several rows in ascending pool_id order and touch disjoint rows: a batch supersedes the open rows of earlier acts on the types its act owns, a Retry requeues the failed rows of the types its act still owns, and both hold the shared rendezvous, so no default change can move the ownership while either runs.
Product row	core.products row of one product	FOR SHARE by the tier trigger; FOR NO KEY UPDATE by the product edit's UPDATE	core.refuse_tier_product_without_set, the trigger on every insert into core.plan_ladder_tiers and every change of a tier's product; UpdateProduct	A tier insert against a clear of the product's set, so the two cannot both commit at READ COMMITTED, the isolation both writers run at (product-catalog invariant 13). The writer holds UPDATE on the table, so the trigger takes the lock with the caller's rights. The tier add takes the row after its pool rows; the product edit takes it alone.
Org type row	core.org_types row of one type	FOR NO KEY UPDATE	the LockOrgType query, run by the default change's T0	The T0 against the demonstration seed and an edit of the type's other columns, which write the row without the rendezvous. Every other writer of the default already waits on the exclusive rendezvous. FOR NO KEY UPDATE does not conflict with the FOR KEY SHARE an organization insert takes on its type's row.
Plan ladder rows	every core.plan_ladders row	FOR NO KEY UPDATE, ascending plan_ladder_id	the LockPlanLadderOrder query, run by the topology reorder (ReorderPlanLadders) first in its transaction	Two reorders, so the later one waits and writes against the order the earlier committed. It does not conflict with the FOR KEY SHARE a tier, a position or an org type's default takes through its foreign key.
Grant lineage head	the core.grants row no other grant extends	FOR UPDATE OF g	the GetGrantLineageHeadForUpdate query, run by RevokeGrantTx, ExtendWaitingGrantTx and ManageGrantTx	Acts on one lineage, so an extension that committed while another act waited is visible to it.
Usage row	core.numeric_entitlement_usage row of one (pool, resource key)	FOR UPDATE	the LockNumericUsage query, run by usage.Recount and by the slot gate, after the site rows the transaction writes and before the request rows it finishes (the hold rows have no place in the order, see rule 5); the count is a statement of its own after the lock	Two recounts of one pool, so each count reads every change committed before its lock was granted; two slot gates for one pool, so the second counts the first's hold and two creates cannot both take the last slot.
FedWiki site row	fedwiki.sites row of one domain or one site id	FOR UPDATE; FOR SHARE by GetSiteForPlacement	the LockSiteStatusByDomain query, run by sweepLocally and the sync's status write (writeStatus), and the LockSiteStatusByID query, run by the slot gate and by RecordSiteStatusActivity and RecordSwapActivity (which check the recorded workspace and domain under it), and LockArchivedSiteForPurge, run by retention; GetSiteForPlacement, run by reconcileSite and syncPlacement inside Registry.WithLock immediately before the first claim or placement write	Paths that move one site (a member's request, the downgrade sweep and the farm sync), so each writes the status it finds under the lock and the recount that follows counts the row as it ended; the sync's reading of the site's requests, which therefore reflects every request that committed before the lock was granted; the gate's reading of an activation whose row the sweep already made active; retention's eligibility check through deletion; and a status write or deletion against creation of that site's placement, so the writer's trigger sees the placement.
FedWiki placement row	domains.placements rows with provider fedwiki and the site's id	Row lock of the trigger's UPDATE, only where the flag changes	fedwiki.set_site_placement_servability, after the site row and before usage	Nothing beyond the site row; flag-only, outside the registry lock.
Desired-state record	core.desired_state_records row	FOR UPDATE	the LockDesiredStateRecord query (after the address lock); StampDesiredStateRecordsDispatched, which locks the records in identifier order	The record's content and generation within one recompute; the dispatch stamp against another stamp.
Desired-state notice	core.desired_state_outbox rows	FOR UPDATE SKIP LOCKED	the ClaimDesiredStateOutbox query	Two relay passes, so each claims different rows; a pass skips rows another still holds. Triggers write the rows through core.insert_desired_state_notice inside the transaction that changed the source, and they take no lock beyond the insert.
Resource-key row	core.resource_keys row of one key	the update lock of SetResourceKeyProvider	integration.RegisterProviders at boot, stamping keys in byte order	A stamp that changes a key's provider against an open entitlement insert of that key, which holds the row's FOR KEY SHARE through the foreign key. The materializer writes a pool's keys in byte order too, so the two wait in one direction.

Advisory key families

Every advisory key is hashtextextended('<family>:<id>', 0): the family's name, a colon and the id, whose parts are joined by colons. db.Lock and db.LockShared (internal/db/locks.go) build it and take it, on a transaction, and nothing else names an advisory lock function (the project lint's advisory-lock-site). A UUID part is written in canonical form, as uuid::text writes it, and a text part holds no colon; a family with one lock for the whole database has no id. Each family's statement begins with a name line (-- name: LockSubscription :exec), which pg_stat_activity shows for a waiter and the statement traces record. The table lists the families in the order a transaction takes them; the lint fails when it and db.Families differ (lock-families).

Family	Id	Statement	Taken by
materialization	none	LockMaterialization, LockMaterializationShared	the entitlements.Materializing and entitlements.RuleChange kinds; test/mockshot-seed.sql and test/seed/discourse/seed-discourse.sh, as their first statement; core.assert_rendezvous checks its key
domains-registry	none	LockDomainsRegistry	registryKind, the kind of domains.Registry.WithLock
fedwiki-workspace	workspace id	LockFedWikiWorkspace	withWorkspaceTx
lifecycle	instance id	LockLifecycleInstance	the lifecycle recording, FedWiki's intake, the slot gate, the outcome steps and the retention purge
stripe-subscription	Stripe subscription id	LockSubscription	fulfillment.ReconcileSubscription, supersedeReleasedDowngrades
stripe-invoice	Stripe invoice id	LockInvoice	invoiceReconcile.converge
stripe-product-sync	product id	LockProductSync	catalogsync.Sync
desired-state	connection id, recipient kind, recipient id	LockDesiredStateAddress	lockAddress in internal/workflows/desiredstate

A change to a family's name or to the shape of its id changes its keys. Two processes that build different keys for one object hold no common lock, so a release that changes a key is deployed as production-deployment.md, section 5, orders an upgrade.

The order every path keeps

1. A transaction that takes the materialization rendezvous takes it as its first statement. No other advisory lock comes before it.
2. The advisory families the transaction needs, in the order of the table above. Several locks of one family follow the order this page gives for that family: lifecycle instances in the order their requests were recorded.
3. Rows, by the rules below: a rule change's set and rule rows before pool rows; pool rows in ascending pool_id, before a grant's lineage head, obligation rows and a tier's product row; FedWiki's site rows, then their placement rows through the trigger, then the usage row, then request rows; a desired-state record after its address.

No path waits on an advisory lock after it takes a row lock: a recording that takes an instance lock its transaction already holds, as the swap's second recording does, takes it without waiting. The sections below list each path's locks in this order.

The order each path takes locks

Each path below lists its locks in the order it takes them. "Rendezvous" means the shared lock a transaction of the entitlements.Materializing kind (BeginMaterializing) takes as its first statement; the entitlements.RuleChange kind takes the exclusive one. Those two kinds are what set SET LOCAL lock_timeout = '5s', which bounds every later lock wait in the transaction. A plain transaction sets no timeout, so its lock waits last until its context ends. A wait past the bound surfaces as SQLSTATE 55P03. The rule change, tier add, tier delete, tier removal, tier reorder and default change render it as the refusal for a change in progress; the other paths show their own failure copy. Materializing transactions run at READ COMMITTED.

Rule change and drain

• Commit (RuleChangeBatchCommit): the exclusive rendezvous (BeginRuleChange), then the set row (LockEntitlementSet, once for the batch), then, for each delta in resource-key order, the set row again (the same lock) and the rule row inside core.commit_rule_change. At or below the sync cap the same transaction then takes each carrying pool in ascending pool_id order (ListPoolsCarryingSet): settlePool locks the pool row through LockPools, one pool per call, then the pool row again and the obligation row inside core.settle_obligation, once per act.
• Drain (DrainChange, called by the commit's own drain above the cap and by the poller's activity): one short transaction per owed pool, opened through BeginMaterializing. It takes the rendezvous, the pool row (settlePool, through LockPools), then the obligation row of each owed act in the batch's order. A failed pool rolls back, and recordObligationFailure records the attempt in a second transaction, also opened as Materializing: the rendezvous, then, through MarkObligationFailed and core.settle_obligation, the pool row before the obligation row of each owed act in batch order. Later calls reuse the held pool lock.

Grant acts

All of these run in a transaction from BeginMaterializing: the operator handlers, ExpireGrantActivity and the demo seed.

• Issue or confer a grant (ConferGrantTx, which ConferGrant, IssueGrant, provisioning and ReapplyDefaultsForPool call): the rendezvous, the grant insert, then the pool row inside core.confer, then materialization.
• Revoke (RevokeGrantTx): the rendezvous, the organization's default pool row (LockPools), the lineage head row, then core.end_conferral (its pool locks are already held), resumption (the pool row again), the delivery step when the revocation ended a provision (under the pool row the act holds, core.confer for each subscription it delivers, no subscription advisory lock), the floor restoration (core.confer) and materialization.
• Expire (ExpireGrantTx): the rendezvous, then the pool row through LockPoolOfGrant, then the grant update. It does not take the lineage head row. After the grant update and core.end_conferral it runs resumption and, when the expiry ended a provision, the delivery step, both under the pool row, then the floor and materialization. A grant that names no organization has no pool, so the lock, resumption, delivery and materialization are skipped.
• Extend a waiting grant (ExtendWaitingGrantTx): the rendezvous, the pool row (LockPoolOfGrant), the lineage head row.
• Manage (ManageGrantTx): the rendezvous, the pool row (LockPools), the lineage head row, then either ExtendGrantTx (inside it core.confer and core.end_conferral, whose pool lock is already held) or ExtendWaitingGrantTx (the same pool row again).
• The reconcile requests. Issue (IssueGrant), Manage's extension, Revoke, Expire, a default change's grandfathered legacy grant, a tier removal and a subscription reconcile that ended one of its own provisions each write one reconcile_subscription row to core.outbox per live subscription of the organization (RequestPoolSubscriptionReconciles), after the act's materialization (the reconcile's active branch: before it). An outbox request is an insert in the act's transaction and takes no lock. ConferGrantTx requests nothing.
• Deliver displaced subscriptions (DeliverDisplacedSubscriptionsTx): called after resumption and before the floor in Revoke, Expire and the reconcile's ended branch, and after a surplus end in its active branch, only when the act ended a provision. It runs under the pool row the caller holds and takes no other lock: it reads the organization's recorded subscription products and each subscription's live provisions, shapes and displacements, and calls core.confer (and core.sync_source_status('suspended') for a past-due, unpaid or paused subscription). It takes no subscription advisory lock, because every position write is under the pool row and the subscription lock protects the subscription's own rows, which the step only reads. No path takes a pool row and then a subscription's advisory lock.
• Resume replaced grants (ResumeReplacedGrantsTx): called inside the transactions above and in the subscription reconcile. It takes the pool row first and reads the resumable grants only after the lock is held, so an expiry, revocation, mark change or extension is either wholly visible to it or waits for it. Each resumed grant calls core.confer.

Subscription reconcile

ReconcileSubscription reads the subscription from Stripe before it opens a transaction. The transaction then takes the rendezvous, then the subscription advisory lock, then the rest by branch.

• Active: reconcileItems takes the pool row (LockPools) after the subscription lock and before core.sync_source_status('active'). It then mirrors the items, reads the live provisions, then the shapes of the desired products and the subscription's displacements, all under the pool row, and confers a missing product unless another plan took one of its ladders from the subscription and still holds it (core.confer), changes a quantity (core.update_conferral_bounds) and ends a surplus provision (core.end_conferral); each of those functions takes the row the transaction already holds. Materialization follows. convergeScheduledChanges then writes the scheduled-change ledger. A spent Stripe subscription schedule it finds is released after the commit (releaseSpentSchedule), with no lock held. When the release succeeds and the schedule held a downgrade row still scheduled, supersedeReleasedDowngrades opens a second transaction, takes the subscription advisory lock and supersedes those rows; a released schedule that held none opens no second transaction. It takes no rendezvous and no pool row: it writes only scheduled-change rows. When a surplus provision ended, the delivery step (under the pool row the transaction holds) and then the reconcile requests (inserts, no lock) follow convergeConferrals, before the materialization.

• Ended: core.end_conferral by subscription (the pool row of each provision it ends), then, only when something ended, resumption (the pool row) and the delivery step (under that row), then the pool row again (LockPools, which is the first take when nothing ended), then the floor restoration (ReapplyDefaultsIfVacant, which confers through core.confer), then materialization, then the reconcile requests (inserts, no lock) when something ended.

• Suspended: reconcileSuspended takes the pool row (LockPools) before core.sync_source_status('suspended'), then materializes when it suspended a provision. For a past_due or unpaid subscription it then makes the pause decision's reads under the row it holds (collectionInputs: the items' prices and products, the live provisions, the shapes and the displacements); a paused subscription makes none.

• The collection pause: a disagreement between the subscription's pause and the rule writes one reconcile_subscription row to core.outbox in the reconcile's transaction, an insert that takes no lock. The outbox's own reconcile (ApplySubscriptionCollection) makes the Stripe update after the commit, with no lock held.

• The paused switch: a member's switch of a subscription carrying the console's pause (upgradeNow) writes its backstop entry, a reconcile_subscription row due four minutes later, as one insert on the database handle that commits on its own, before any Stripe write, and holds no lock. Its restore after a switch that did not apply is an ordinary ApplySubscriptionCollection reconcile, so it takes the locks above in the order above.

• Resume plan: a member's Resume plan (ResumeSubscription) takes the materialization rendezvous (shared), then the pool row, then reads what its refusals need, confers the subscription's own tier over the grant, writes the reconcile requests (inserts) and materializes the pool, in one transaction. It takes no subscription advisory lock and makes no Stripe call: every position write is under the pool row, the subscription lock protects the subscription's own rows, which the act only reads, and the outbox entry it wrote clears the pause after the commit. A reconcile of the subscription waiting on the pool row finds the conferred provision live and writes nothing.

No path takes a pool row and then a subscription's advisory lock, so the order cannot wait in a cycle with a grant act, which takes the pool row and never a subscription's lock.

Tier add, tier removal, default change and reorder

The tier add, the tier removal, the tier delete and the reorder's renumbering run in transactions from BeginRuleChange, which takes the exclusive rendezvous: each changes a ladder's tiers, and core.confer reads a ladder's ranks and a product's shape in separate statements, so no conferral may run while one of them commits (#213). The default change's T0 takes the exclusive rendezvous too: a move reads the type's default and its rank 0, and a default saved between those reads would mix two catalogs. What follows T0 runs in transactions that take the shared rendezvous: Retry's own and the drain's listing, moves and last probe come from BeginMaterializing; the listing, the moves and the last probe inside the request that made the act come from MaterializingWithin, which sets the request's shorter lock timeout and its transaction timeout before the rendezvous statement.

• Tier add (entitlements.AddTier): the exclusive rendezvous, then the reads that decide the add, inside the transaction: the product (a product no row names, or one not published, refuses the add) and the count of the organizations holding a live provision of it (above planActHolderCap the add is refused), then the pool row of every pool holding a live provision of the product through LockPools, which sorts and deduplicates them, so they lock in ascending pool_id order. The provisions are read again under the locks, then the ladder's name, the act row and its detail row (core.record_plan_act), the tier insert (its trigger takes the product row FOR SHARE and holds it to the commit), and the collision read (ListTierAddCollisions), which refuses a pool that already holds a live position on the ladder or holds two live provisions of the product before any alignment. Then, for each pool in the same order, core.align_conferral_shape for each of its provisions in creation order and one materialization. It runs through runPlanAct.

• Tier removal (entitlements.RemoveTier): the exclusive rendezvous, then the reads that decide the removal, all inside the transaction and before any pool row: the count of the organizations holding a live position on the tier (CountLiveTierHolders), the ladder's tiers and the org types (ListOrgTypes), from which TierRemovalRefusal decides (above planActHolderCap the removal is refused; the only tier of a ladder an org type defaults to is refused under either disposition), the names of the ladder and the product, and the tier's holders. A failed read refuses. Then the pool row of every holder through LockPools, which sorts and deduplicates them, so they lock in ascending pool_id order, and the holders are read again under the locks. Then the act row and its detail row (core.record_plan_act), the tier delete and the renumbering. Then, for each pool in the same order, the incumbents end (core.end_conferral), the remaining positions align (core.align_conferral_shape), the floor is restored when the disposition is migrate, with the act's id on the grants it creates, and the pool materializes, then the reconcile requests (inserts, no lock). It runs through runPlanAct.

• Tier delete without holders (entitlements.DeleteTier): the exclusive rendezvous, then the reads that decide the delete, all inside the transaction: the count of the tier's live holders (positions that are active or suspended), the ladder's tiers and the org types (ListOrgTypes), from which LastTierRefusal decides. A failed read refuses the delete. Then the names of the ladder and the product, the act row and its detail row (core.record_plan_act), the tier delete and the renumbering. It takes no pool lock: a tier with no live holder has no pool to reconcile, and the guard holds because the exclusive rendezvous keeps any conferral from placing a position on the tier between the count and the delete. The delete runs through runPlanAct.

• Default change, T0 (entitlements.ChangeOrgTypeDefault, through runPlanAct): the exclusive rendezvous, then the org type's row (LockOrgType, FOR NO KEY UPDATE), the reads (the candidate's rank 0 and the names), the type's update (the row it already holds), core.record_plan_act (the act, its detail row and the reached type, with one index probe for the type's newest organization) and, when the operator chose no disposition, the gate (core.outgoing_default_exists, index probes). It takes no pool row and no obligation row, so its hold does not grow with the number of organizations. The request's deadline covers the two waits it can make, the rendezvous and the type's row.

• Reorder (entitlements.ReorderTiers, through runPlanAct): T0 takes the exclusive rendezvous and no pool or obligation row. Inside it: the ladder's tiers are listed and the submitted order reconciled against them, the names read, the ranks renumbered (RenumberTiers), the act recorded (core.record_plan_act, which reads the org types that default to the ladder and records them as the types the act reaches, only when rank 0 changed), and, when no disposition was chosen and the act reaches a type, the gate (core.outgoing_default_exists), which refuses the whole T0. Then the listing batches and the moves run as for a default change: the shared rendezvous, core.write_plan_act_obligations, and EnactObligation for each organization, which reads the type's current default and its rank 0 when it runs, not the rank 0 the reorder saved. A reorder that keeps rank 0 reaches no type and writes no obligation.

• The listing (ListPlanActBatch, after T0): the shared rendezvous with the lock timeout and transaction timeout the caller gives (the request's shorter ones, or the drain's 5 s), then core.write_plan_act_obligations: the batch's pool rows FOR KEY SHARE in ascending pool_id order, the open obligations of earlier acts on those pools FOR UPDATE in the same order and their update to superseded, and the inserts in (org type, organization id) order. A deadlock between two batches of one act is benign: the next pass lists the batch again.

• One organization's move (EnactObligation, through runPlanAct): the shared rendezvous, the pool row (LockPools), then the obligation row (FOR UPDATE, taken by core.lock_plan_act_obligation, which the LockPlanActObligation query calls; the function reads the type's owner in the same statement; a move skips an obligation that is not pending or whose act no longer owns the type), then the reads (the type's default and the pool's positions) and the bucket's work (ReapplyDefaultsIfVacant, ConferGrantTx for a grandfathered incumbent, or core.end_conferral, ReapplyDefaultsIfVacant and materialization for a migration), which take the same pool row again, then, after a grandfathered incumbent's legacy grant, the reconcile requests (inserts, no lock), and core.settle_plan_act_obligation. A failure on one organization leaves the others alone.

• Recording a failed move (core.record_plan_act_obligation_failure): a plain transaction that updates the one obligation row, and only while it is still pending. It takes no rendezvous and no pool row, so it cannot wait in a cycle with a move.

• The act's last probe (DrainPlanAct, once the act is done and settled any obligation as migrated or grandfathered): the shared rendezvous, then core.outgoing_default_exists, so the walk over the index entries the moves ended happens outside the next T0's exclusive hold.

• Retry (entitlements.RetryPlanAct): the shared rendezvous, core.record_plan_act (the Retry act and its detail row), then core.requeue_plan_act_obligations, which locks the retried act's failed obligations of the types it still owns FOR UPDATE in ascending pool_id order and updates them. A Retry that requeues nothing rolls back.

  No two of these wait on each other in a cycle. T0 holds the exclusive rendezvous and takes no pool or obligation row, so it waits only for the rendezvous and the type's row. Every path that takes both a pool row and an obligation row takes the pool row first, and a listing batch takes its pool rows FOR KEY SHARE, which conflicts with a move's FOR UPDATE, before it touches any obligation. The two writers of several obligation rows, a batch's supersession and a Retry's requeue, take them in ascending order and touch disjoint rows.

The ladder reorder on the topology page (ReorderPlanLadders) is not a plan act: it changes only the ladders' display order, which no conferral reads. A READ COMMITTED transaction takes every ladder row (LockPlanLadderOrder), then updates the sort_order of each ladder whose position changed. It opens at READ COMMITTED whatever the server's default, because the lock read that waited returns the order the earlier reorder committed, where REPEATABLE READ would fail it with a serialization error.

Invoice reconcile

invoiceReconcile.converge opens a plain transaction, with no rendezvous, and takes the invoice advisory lock. It writes invoice, line and payment rows under that lock and touches no pool.

Product sync

catalogsync.Sync opens a plain transaction, takes the product advisory lock, re-reads the price and product mappings, writes the pending mappings and enqueues the outbox rows. It takes no pool lock and no rendezvous. A sync that commits nothing returns with its transaction open, and SyncProductToStripe rolls it back after it has rendered its answer, so the lock is held until then.

FedWiki at the handler

withWorkspaceTx takes the workspace advisory lock, then runs the handler's statements. A swap (swapActiveSite) and a rotation with a cooldown (changeStatus) then record a lifecycle request, which takes the lifecycle instance lock of the activated site. A swap records a second request for the parked site, which takes that instance's lock after the first. The swap policy row is upserted last. The handler takes no usage-row lock and no site-row lock. A status change without a cooldown skips the workspace lock and records its request in a transaction of its own, taking only the instance lock. Both intake transaction openers use READ COMMITTED. Status intake reads the original site by ID after acquiring its instance lock, so it sees a purge that committed before that lock was granted. A swap takes both instance locks before validating either site and computes the park generation under its lock. These reads take no site-row lock.

At the slot gate

HoldSiteSlotActivity runs usage.Hold in one transaction at READ COMMITTED. It takes the request's lifecycle instance lock, then, for an activation, the site row by id (LockSiteStatusByID), then reads the pool and its limit without a lock, then the usage row (LockNumericUsage), then counts in a statement of its own (CountSlotsByPool), then inserts the request's hold, then recounts. A hold row is inserted only by its request's own workflow, and deleted by that workflow, by a status reconcile that settles the request, or by settle's finish, each under the request's instance lock, so no two transactions wait on one hold row. The gate takes no workspace advisory lock.

When recording an outcome

The workflow activities that finish a lifecycle request, and settle, run finishAllWith, one transaction at READ COMMITTED: the lifecycle instance locks, in the order the handler recorded the requests (the activated site first, then the parked site); then the activity's own site-row writes (status and swap records lock and validate the original IDs, park then activate for a swap); then the requests' holds are deleted (DeleteSlotHolds); then the usage row (usage.Recount: LockNumericUsage, the count, the write); then the swap's guard; then the request rows (FinishLifecycleRequest).

• RecordCreatedSiteActivity: the site row insert.
• RecordSiteStatusActivity: the site row by id (LockSiteStatusByID), checked against the recorded workspace and domain.
• RecordSwapActivity: the site row of the parked site, then the site row of the activated site, both by id (LockSiteStatusByID) and checked against the recorded workspace and domain; the guard reads the pool's limit after the recount.
• RecordSiteDeletedActivity and FinishLifecycleRequestsActivity (every declined or failed outcome): no site row.
• DeleteFedWikiSiteActivity deletes the site row and recounts in a transaction of its own, before the record step, which recounts again.

A pool with no usage row takes no recount lock and writes no count; the transaction commits its other writes.

So a transaction takes its site rows first, the usage row after them, and the request rows last. Its hold rows fall anywhere in that order: only the holder of their request's instance lock writes them, so no other transaction waits on one. The slot gate inserts its hold after the usage row's lock and count.

In the downgrade sweep and at boot

One FedWiki maintenance schedule awaits optional farm projection, then retention, then quota reconciliation. Scheduled runs do not overlap; member requests still use the locks below. Quota and Keep active rank by recorded modification time, falling back to creation time for an unobserved site.

• Park (parkSite): the farm call first, with no lock held, then one transaction (sweepLocally, at READ COMMITTED) that takes the site row, writes the status and the mark, and recounts the pool.
• Reactivate (reactivateSite): the farm call first, with no slot taken, then the same transaction: the site row, the status, the recount, and the pool's limit read in the same transaction; a figure above the limit rolls the transaction back.
• Recount at boot (usage.RecountAll, run by the domains reconciliation, and at the end of each sync pass): one transaction per pool that takes the usage row with LockNumericUsage and counts, then writes the figure, with no other lock.
• Settle (SettleOpenRequests): a closed activation's hold is read (SlotHoldExists) with no lock. A request that holds a slot is handed to the status reconcile, which settle starts without finishing it, so settle takes no lock and leaves the hold in place. Every other request it finishes goes through finishAllWith, as above.
• Status reconcile (ReconcileFedWikiSiteStatusWorkflow): reads the request, the site row and the farm with no lock held. Its outcome ends in finishAllWith, as above: RecordSiteStatusActivity when the farm holds the site in the requested status, FinishLifecycleRequestsActivity otherwise. When the site row is gone it ends through FinishLifecycleRequestsActivity without reading the farm. Each removes the hold and recounts.
• Provider registration (RegisterProviders): the resource-key rows in byte order.
• Domains reconciliation at boot (reconcile_domains.go in the FedWiki package): one Registry.WithLock transaction per orphaned placement or adopted site.

In the retention purge

purgeArchivedSite opens one READ COMMITTED transaction per candidate, takes its lifecycle instance lock, then locks its still-expired archived row by ID with LockArchivedSiteForPurge. It checks for an accepted request after both locks. READ COMMITTED sees requests committed before the instance lock was granted and skips a row whose eligibility changed during the row-lock wait. The farm DELETE and local row deletion run under both locks, through commit. No usage lock or recount is needed: the row is archived and has no open request. Placement release uses a separate registry transaction after commit.

The farm call can hold these locks for 30 seconds. A member request that waits on these locks gets its answer when it finishes within the console's 32-second request timeout; one still running at that timeout is answered 503, and a transaction it began on its request's context rolls back when that context ends. A rollback cannot undo a remote delete, and cancellation does not prove the farm stopped executing it.

In the farm sync

SyncSitesToDBActivity reads its sets with no lock held (ListAllSites, ListDomainsWithOpenRequests, and ListDomainsWithRequestsUpdatedSince when the input carries the moment the farm list was taken), then writes one site at a time. No transaction holds more than one site row, and none takes a lifecycle instance lock or a workspace lock.

• A farm site with no row (insert): one transaction at READ COMMITTED that inserts the row (InsertSiteFromFarm, ON CONFLICT DO NOTHING, so a row a create recorded after the pass read its rows is left alone) and then recounts the pool.
• A listed status equal to the row's: one READ COMMITTED transaction (SetSiteObservedByDomain) that writes the observed storage and last-modified time. It holds the row lock through commit, writes no status and takes no usage row; the trigger also corrects a wrong placement flag.
• A listed status different from the row's (writeStatus): one transaction at READ COMMITTED that takes the site row (LockSiteStatusByDomain), then reads whether a request on the site is still accepted (SiteHasOpenRequest) and whether one was updated at or after the moment the entry was read (SiteHasRequestUpdatedSince), and commits without a write when either answers yes. Otherwise it writes the status and recounts the pool. A record step that writes a status takes the same site row, so a request either committed before the sync's lock, and the two reads see it, or takes the row after the sync's commit and writes its own status last.
• A farm-absent row of the System workspace: one transaction at READ COMMITTED that deletes the row and recounts the pool. The name is released after the commit.
• The end of the pass (usage.RecountAll): one transaction per pool, as at boot.

A pool with no usage row, the System tenant's, takes no recount lock and writes no count; the insert, the status write and the delete commit.

Lifecycle intake

RecordLifecycleRequest takes the instance lock first, then reads the latest generation, then inserts. RecordFirstLifecycleRequest takes the same lock once and reads the latest generation once: it refuses a later generation of an instance that already holds a request, then decides as RecordLifecycleRequest does. RecordLifecycleRequest and RecordFirstLifecycleRequest open their own transaction on a database or a pinned connection; RecordLifecycleRequestInTx and RecordFirstLifecycleRequestInTx run in the caller's, which is how the handler records under the workspace lock. DeleteUnstartedLifecycleRequest, which removes a request whose workflow never started, takes no instance lock, so a repeat that races it may be answered from either side of the delete.

Desired-state relay

• Claim (ProcessDesiredStateOutbox): ClaimDesiredStateOutbox is one statement that locks the pending rows SKIP LOCKED and leases them. The relay never takes the rendezvous and never locks a source row.
• One recompute (recomputeAddress, and the same sequence in readBackItem and writePushAnswer): one READ COMMITTED transaction that takes the address advisory lock, then the record row (LockDesiredStateRecord), then computes the content in a new statement, then updates or inserts the record. Every transaction that writes a record's content or generation, or a projection, takes the address lock first.
• Dispatch stamp (StampDesiredStateRecordsDispatched): one statement that locks the records in identifier order before it writes. It takes no address lock.

Registry

FedWiki flag-only corrections bypass the registry advisory lock: its site trigger locks the site row, then matching placement rows whose flag changes. All site row writes use READ COMMITTED (or PostgreSQL's equivalent READ UNCOMMITTED); the trigger rejects transaction-snapshot isolation. The swap locks both sites before either write. No path holds a placement lock and then takes a site lock. Member deletion releases its name in a preceding transaction; retention and farm-absent sync release it after the site deletion commits false.

Boot and sync creators classify the listed immutable domain and workspace and perform all skip checks first. Immediately before the first claim or placement mutation, they read exactly one site FOR SHARE, using GetSiteForPlacement in the registry transaction. A missing site creates neither claim nor placement; a failed read rolls back. Each creator transaction must lock exactly one site, otherwise it could deadlock with a multi-site swap. FOR SHARE conflicts with both explicit FOR UPDATE and bare status UPDATE; FOR KEY SHARE does not. An eligible creator waits for that site while holding the registry lock. The retention purge may hold it through its farm call for 30 seconds, delaying other registry operations; skipped sites do not incur this wait.

domains.Registry.WithLock opens its transaction through the runner with registryKind, whose first statement is the registry lock. Every composite operation, including the FedWiki create, release and sync projections, runs on the *Tx it passes, so the lock is taken once. The comment on registryKind states the rule for a transaction that would touch both the registry and a usage row: registry lock first. No transaction does both today: the slot gate and the recounts are workflow activities that run their own transactions, apart from the transaction that allocates or releases the name.

Rules every path keeps

1. A transaction that materializes a pool takes the shared rendezvous as its first statement, through BeginMaterializing or, inside the request that made a plan act, MaterializingWithin; a plan act's listing batch, a move, the last probe and a Retry are such transactions. A rule change, an operator's tier change (the tier add, the tier delete, the tier removal, the reorder's renumbering) and a default change's T0 take the exclusive one through BeginRuleChange, and may materialize under it; the demonstration seed adds its ladder's tiers on the plain handle, outside the rendezvous. core.assert_rendezvous raises materialization_rendezvous_missing when a function that materializes or settles runs without it. No other advisory lock precedes it.
2. The pool row is locked before positions or entitlements change. The five conferral and settlement functions lock the row themselves, so a caller that goes straight to one of them still holds it.
3. A transaction that locks several pools takes them in ascending pool_id order, through LockPools. The rule change, the tier add and the tier removal do. A plan act's listing batch locks its pools for key-share in the same order, in SQL (ORDER BY pool_id FOR KEY SHARE). The deferred drain and a plan act's move lock one pool per transaction.
4. The grant lineage head row is locked after the pool row, never before.
5. A FedWiki transaction takes its locks in this order, skipping those it does not need: the lifecycle instance locks, all site rows, their placement rows (through the site trigger), the usage row, then the lifecycle request rows. Hold rows have no place in the order, because only the holder of their request's instance lock writes them. The count under the usage row is a statement of its own after the lock, so it reads every commit that preceded the lock's grant.
6. Lifecycle instance locks of one transaction are taken in the order the requests were recorded.
7. A desired-state write takes the address lock, then the record row, then computes content in a new statement.
8. Every advisory lock belongs to its transaction, so a commit or a rollback releases it, and an error never leaves one behind.
9. A row lock on a table where the writer holds UPDATE on no column is taken inside a SECURITY DEFINER function owned by core_owner, because Postgres requires UPDATE on at least one column for FOR UPDATE, FOR NO KEY UPDATE, FOR SHARE and FOR KEY SHARE, and the runtime user holds only what console_runtime holds. A move takes its lock on core.plan_act_obligations this way, through core.lock_plan_act_obligation.

Where the paths differ

• The plan-ending paths take the pool lock at different points (#180). Revocation and expiry lock the pool before they write. The subscription reconcile's ended branch ends positions first, so the pool row is taken inside core.end_conferral, and takes it again for resumption and before the floor restoration. Each path then runs resumption, the floor restoration and materialization in the same order.