Time the plan acts' commit, listing and preview before their schema
A default change or a tier reorder writes one row per organization of the types it reaches while holding the exclusive entitlement lock, so the time every signup, webhook and grant waits grows with the largest org type, which is one organization per member (#187). The change proposed here saves the act in a short exclusive transaction, checks for outgoing-default positions through an index, and lists the organizations it owes in batches of 500 afterwards under the shared lock. TestTimedPlanActCaps times prototypes of those shapes in a scratch database before any migration is written, and times today's moves, removals and adds per holder. It runs only when PLAN_ACT_CAP_TIMING is set, so make test skips it. Measured on this commit against the test stack's PostgreSQL 18.1, medians of ten, two runs: - The exclusive transaction: 1.5 to 1.6 ms at 256,000 organizations, 1.5 ms at 1,000. - The outgoing-default check at 256,000 organizations after 51,250 positions end, autovacuum's default threshold at that size, after a checkpoint: 27 to 36 ms on the first probe, under 0.4 ms on the next. While an older snapshot is held: 23 to 26 ms, then about 4 ms. After the act's own last probe under the shared lock, the exclusive transaction's probe is 0.21 to 0.24 ms against 0.14 to 0.16 ms before the endings. - A listing batch of 500: 13 to 26 ms at every size from 1,000 to 256,000 organizations and at the start, middle and end of the type. Its read finds each organization's default pool with one index probe under a generic plan; an empty batch costs about 0.3 ms. - The preview's read at 256,000 organizations: 0.90 to 0.92 s. - Today's code, 50 to 3,200 holders: one organization's move 1.9 to 3.4 ms; a tier removal with migrate 2.0 to 2.4 ms per holder (7.6 s at 3,200); a tier add 0.64 to 0.78 ms per holder (2.1 s at 3,200). Every bound the test states passes. The prototype leaves out the persons, grant and actor foreign keys and the exclusion constraint on positions, none of which grows with the type.
This commit is contained in:
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-10-18
|
||||
@@ -0,0 +1,866 @@
|
||||
## Context
|
||||
|
||||
### The five plan-wide acts today
|
||||
|
||||
Five operator acts change what many organizations hold at once. All five already run as functions in `internal/entitlements` (`AddTier`, `DeleteTier` and `RemoveTier` in `ladder_acts.go`; `ChangeOrgTypeDefault` and `EnactDefault` in `default_change.go`; `ReorderTiers` in `ladder_acts.go`), with the transaction shapes they had in the handlers.
|
||||
|
||||
- **Tier add**: one transaction under the exclusive materialization lock (the rendezvous, `BeginRuleChange`). It inserts the tier, then aligns every live provision of the product with `core.align_conferral_shape` in creation order, so the pools lock in creation order (#194), and materializes the provision's pool once per provision. The handler reads the product and checks that it is published before the transaction, and answers "Product not found." for any error of that read.
|
||||
- **Tier delete** (a tier no organization holds): one exclusive transaction that deletes the tier and renumbers the rest. The live-holder count and the guard against emptying a ladder an org type defaults to are read by the handler before the transaction, and the guard's read of the org types returns nothing when it fails, so a failed read lets the delete through.
|
||||
- **Tier removal** (a tier with holders): one exclusive transaction that locks every holder's pool in ascending order, reads the holders again under the locks, deletes the tier, renumbers, and reconciles each pool. Any failure rolls everything back. The guard the plain delete runs is skipped, so a removal under `keep` can leave the default ladder with no tier.
|
||||
- **Default change**: the new default is saved on its own, outside any lock, and then each organization of the type is moved in its own transaction under the shared lock, against a rank-0 product the handler fixed before the save.
|
||||
- **Reorder**: one exclusive transaction renumbers the ladder; when rank 0 changed on a ladder an org type defaults to, each organization of those types is then moved as a default change moves it, against the rank-0 product the reorder saved.
|
||||
|
||||
Facts every design below has to respect:
|
||||
|
||||
- A materializing transaction takes the rendezvous in shared mode as its first statement, with `lock_timeout = '5s'`; a rule change and a tier change take it in exclusive mode (`internal/entitlements/tx.go`). Every conferral, grant act, subscription reconcile, signup and floor restoration on the instance waits behind an exclusive holder and refuses after 5 s. Postgres queues a shared request behind a waiting exclusive one, so a signup that arrives while a commit waits for the exclusive lock runs after that commit.
|
||||
- Pool locks are row locks (`SELECT ... FOR UPDATE` on `core.resource_pools` through `LockPools`), so the number of pools one transaction locks is not bounded by Postgres's shared lock table; each is a tuple write.
|
||||
- An HTTP request has 8 s to start its answer (the server's `WriteTimeout`, `internal/server/server.go`) and 32 s to run (`middleware.Timeout`). One organization's move is about 20 to 25 round trips and one commit; the estimate, not measured, is 5 to 15 ms. The only measured number nearby is a rule change's materialization, 1.8 to 2.0 ms per pool inside one transaction (`TestTimedSyncCap`). A default change on an org type of a few thousand organizations therefore runs past the deadline today, and every organization after it is reported as having no default pool.
|
||||
- The personal org type has one organization per member, so its size is the instance's membership. Any step whose time grows with the number of organizations must run outside the exclusive hold.
|
||||
- An organization has at most one default pool (`uq_resource_pools_one_default_per_org`), and a grant delivers to its organization's default pool (`LockGrantPool`). Organizations are created only by `AutoProvision`, which opens `BeginMaterializing` first.
|
||||
- No plan act calls Stripe, Temporal or another service; the only side effect is the desired-state outbox rows the entitlement triggers insert in the same transaction.
|
||||
|
||||
### What fails today and cannot be re-run
|
||||
|
||||
| Act | What happens | Why it cannot be re-run |
|
||||
|---|---|---|
|
||||
| Default change | One organization's move fails, or the listing fails after the save, or a rule change holds the exclusive lock (every organization fails) | The form refuses the current default as a candidate, so "Re-run the commit" cannot be followed |
|
||||
| Reorder | Same, after the renumbering committed | Dropping the committed order again finds rank 0 unchanged and moves nobody |
|
||||
| Both | A renumbering commits between the act's save and an organization's move | The move classifies against a product that is no longer rank 0, grandfathering an organization already on the current rank 0 |
|
||||
| Both | The request passes 8 s or 32 s | The browser gets nothing or a 503; every later organization is reported as "no default pool" |
|
||||
|
||||
The failures exist only in the answer to the request. Nothing records them.
|
||||
|
||||
### The precedent
|
||||
|
||||
The entitlement set rule change already has a similar shape: one commit transaction under the exclusive lock writes the rule, its record and one obligation per pool (`core.entitlement_set_change_obligations`); a request settles up to 50 pools in 2 s; a Temporal drain workflow per change, started after the commit and backed by a poller every 5 s, settles the rest; a pool that fails five times is `failed`; and a Retry control requeues the failed rows (`internal/entitlements/rule_change.go`, `internal/workflows/entitlements/recompute_drain_*.go`, `internal/server/operator_entitlement_recompute.go`). Two of its traits are not copied. It writes one obligation per pool inside the exclusive hold, which is bounded there by the rule change's population cap; a default change has no such cap, so this change lists its organizations after the exclusive hold instead. And its record writes one row per rule and none for the batch, so the first rule's row stands in for the batch (#181).
|
||||
|
||||
### The act record convention
|
||||
|
||||
An action that writes several rows has no record of its own today, so readers rebuild it from reason prefixes, a borrowed request id or the shape of a workflow id. The convention this change sets, and is the first to adopt, is one immutable act row per action, carrying who acted, when, why from a closed list, and an optional note, with every row the act writes pointing at it, in the way a journal entry's header owns its lines in bookkeeping. #184 (a swap), #179 (tier changes), #181 (a rule-change batch), #131 (subscription changes) and #171 (integration acts) follow it later, each adding its causes and its own detail table.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
|
||||
1. A default change and a reorder never leave an organization unmoved without a record of it, a reason, and a Retry that moves it onto the current default without committing the catalog again.
|
||||
2. No move classifies against a stale rank-0 product, whatever tier change or default change commits in between, and no move of an earlier act acts on a pool a newer act reaches.
|
||||
3. A default change or a reorder of any org type size commits with an exclusive hold whose length does not grow with the number of organizations, apart from the gate probe's bounded walk over index entries vacuum has not yet removed (D4); its preview reads a fixed number of names whatever the type's size; every request answers inside its write timeout; the work the request could not finish completes without the operator.
|
||||
4. The disposition gate inside the commit and the preview apply one rule, so the gate cannot pass an act the preview would have asked a disposition for.
|
||||
5. Tier add and tier removal stay all-or-nothing, and refuse with a stated limit instead of stalling the instance on a population no transaction should hold.
|
||||
6. One act record convention, with the plan acts as its first adopter, that the followers can take without reworking it.
|
||||
7. One place that orders the plan acts' locks, and handlers that only parse, call and render.
|
||||
8. An operator who cannot read the server's logs can hand the person who can a reference that finds every log line of a failure.
|
||||
|
||||
**Non-Goals:**
|
||||
|
||||
- Retiring a tier and draining its holders above the limit (#237).
|
||||
- A note field on the forms (#238), per-organization dispositions (#82), an attention overview (#78), a page per act.
|
||||
- A pointer from transitions to their act (#179); a record of a rule-change batch (#181).
|
||||
- Changing `core.confer` or any conferral function.
|
||||
- The console's other "Details are in the server logs" sentences (#239).
|
||||
|
||||
## Decisions
|
||||
|
||||
### D1. A short commit, then a listing, then moves
|
||||
|
||||
Each plan-wide act that moves organizations one at a time (the default change and the reorder) runs in three phases:
|
||||
|
||||
1. **The commit (T0)**, one transaction under the exclusive rendezvous. It writes the catalog change, one act row, the act's detail row and one row per org type the act reaches (`core.plan_act_reached_org_types`), and runs the disposition gate (D4). Every statement in it is a single-row write or an index probe. The gate's probe is the one statement whose cost can grow: its first read after many positions ended walks the index entries vacuum has not yet removed, about 20 ms per 100,000 measured, after which those entries are marked dead and the probe is back to its usual cost (D4). The act whose moves ended the positions runs that probe once under the shared lock when its moves are done, so the walk is normally paid outside the exclusive hold. No other statement's cost depends on how many organizations the reached types hold.
|
||||
2. **The listing**, short batches under the shared rendezvous, each its own transaction (D10). A batch writes one obligation per organization with an active default pool, or a record of an organization without one, and marks an earlier act's open obligation on the same pool superseded.
|
||||
3. **The moves**, one short transaction per obligation under the shared rendezvous (D2).
|
||||
|
||||
The request runs T0, then one listing batch and as many moves as its budget allows; a Temporal drain continues the listing and the moves (D14).
|
||||
|
||||
Alternatives considered:
|
||||
|
||||
- **A: the whole act in one transaction.** No partial state, and the target is resolved inside. But the act must hold every pool of the type: the 32 s deadline cancels it at a few thousand organizations, so a default change on a large type could never complete; one broken organization blocks the change for every other; and a reorder holds the exclusive rendezvous for the whole act, so every signup, webhook and grant act on the instance refuses after 5 s. Rejected.
|
||||
- **B: save first, a ladder lock across the per-organization transactions, a record of failures, an explicit re-run.** It needs this change's record and re-run anyway. Its own part, a lock held across many transactions, needs either a session advisory lock (the code takes none) or a durable "busy" flag that a dead process leaves set, blocking every tier change until someone clears it. The lock is unnecessary once each move reads its target inside its own transaction (D2). Rejected as dominated.
|
||||
- **C: every obligation written inside T0.** One transaction would hold the whole work list, with no listing phase. But T0's exclusive hold then grows with the type: a personal type of 100,000 members holds it for several seconds, and every conferral, signup and reconcile on the instance refuses for that window. A population limit would make large types impossible to change. Rejected.
|
||||
- **Derived state, no table.** A page compares each organization with its type's current default and offers to apply it. It loses the reason a move failed and the act's intent (a `keep` removal deliberately leaves default-sourced positions off the ladder, which a derived check would flag), and does nothing for scale. Rejected; it remains a possible consistency check between obligations and positions.
|
||||
|
||||
### D2. Each move resolves the current default inside its own transaction, and checks that its act still owns the pool's type
|
||||
|
||||
An obligation names an act and a pool, not a target product. The transaction that settles it locks the pool, then the obligation row, and in the statement that locks the obligation reads which act owns the pool's org type (D9). When that act is not the obligation's act, a newer act reaches the pool, and the move writes nothing. Otherwise it reads the organization's type, the type's current default ladder and that ladder's current rank-0 tier, and classifies the pool against that product. Every writer of either fact (a default save, a renumbering, a tier add or delete) holds the exclusive rendezvous, and the move holds the shared one, so the move reads one consistent catalog and one owner. The floor restoration the move calls reads the same facts again and gets the same answer.
|
||||
|
||||
Alternative considered: store the target product on the obligation. It reintroduces the stale-target defect the day a tier change commits after the act. Rejected.
|
||||
|
||||
### D3. The default change's commit takes the exclusive rendezvous
|
||||
|
||||
Today the default is saved outside any lock. Under D2 that is no longer enough: a move that read the default before the save and the rank 0 after it would mix two catalogs. The default change's T0 therefore opens with `BeginRuleChange`, like the four tier changes, so a default save and every move are strictly ordered. T0 locks the org type's row `FOR NO KEY UPDATE`: under the exclusive rendezvous every other writer of `default_plan_ladder_id` already waits, so the row lock matters only against the demonstration seed, which writes on the plain handle, and against an edit of the type's other columns. `FOR NO KEY UPDATE` does not conflict with the `FOR KEY SHARE` an organization insert takes on its type's row.
|
||||
|
||||
### D4. One rule decides whether a position is outgoing and which class a pool falls in; the gate reads it through an index
|
||||
|
||||
Today the preview reads every live position of the type (`ListLivePlanPositionsByOrgType`) and classifies each organization in Go (`ClassifyPoolAttachments`). That fails at size in two ways. The gate inside T0 has to answer one question in a time that does not grow with the type: does any organization of the reached types hold a live default-sourced position that does not deliver the target's rank-0 product? A signup between the preview and T0 lands on the outgoing default, and only a check under the exclusive hold sees it, so the gate cannot be left to the preview. And the preview itself carries every position of the type into Go on every render, which on a personal type is the membership on every radio click (D27). A second count written for the gate could disagree with the classification on a pool that holds both an at-target and an outgoing default position, or on a None target, and the move would then meet an outgoing position under an act that asked no disposition.
|
||||
|
||||
The rules are therefore written once, in SQL, and the preview, the gate and the move all read them:
|
||||
|
||||
- **Two copies on the position row.** `core.pool_provision_ladders` gains `org_type` (the type of the pool's organization) and `default_sourced` (the provision is sourced by a grant whose `grant_reason` is `default`, decided by `core.provision_is_default_sourced`). A `BEFORE INSERT` trigger fills both, and so does a `BEFORE UPDATE OF pool_id, provision_id` trigger that fires only `WHEN (OLD.pool_id IS DISTINCT FROM NEW.pool_id OR OLD.provision_id IS DISTINCT FROM NEW.provision_id)`; the migration backfills existing rows. The update trigger needs its `WHEN` clause because `core.sync_pool_provision_ladders` assigns `pool_id` on every status, activation or end change of a provision (`internal/db/migrations/00004_doc41_schema_and_data.sql:378-392`), so without it every status sync would read the pool, the organization and the grant again, although the guard below keeps any update from changing the copies. The copies rest on four facts: a pool's organization, an organization's type, a provision's grant and a grant's reason. No runtime statement changes any of them, and guard triggers in the same migration refuse any update that does (below). The position table is already denormalized from its provision the same way (`pool_id`, `status`, `activated_at`, `ended_at`).
|
||||
- **One position predicate.** `core.plan_position_is_outgoing(default_sourced, product_id, target)` is true when the position is default-sourced and the target is None or a different product. It is the only statement of "outgoing" in the code.
|
||||
- **One pool rule.** `core.plan_pool_bucket(has_pool, any_outgoing, any_default_sourced, any_position)` maps what a pool's live positions show into the pool's class: `would_fail` when the organization has no active default pool; `outgoing_default` when any position is outgoing by the predicate; `already_at_default` when none is outgoing and one is default-sourced; `other_source` when none is default-sourced and one exists; `would_initiate` when none exists. Both functions are `IMMUTABLE` SQL with schema-qualified names and no `SET` clause, as `core.transition_change_kind` is (`internal/db/migrations/00025_superseded_transitions.sql`), so PostgreSQL inlines them into the statement that calls them rather than calling a function once per row. The Go class constants mirror the function's five results, and a test calls it with every combination of its four arguments and compares what it returns with them. `ClassifyPoolAttachments` is deleted: no Go code decides a pool's class.
|
||||
- **The preview reads exact counts and at most 50 names per class.** `ListPlanPoolBucketsByOrgType(org_type, target, per_class_limit)` is one statement over the type: each organization with its active default pool, that pool's live positions folded with `bool_or` and `count` into the four arguments of `core.plan_pool_bucket` (each `bool_or` inside `COALESCE(..., false)`, because it is NULL for a pool with no live position and for an organization with no pool), and, for each class, its exact count and its first `per_class_limit` organizations by name and then id. It returns at most 250 rows whatever the type's size, and no row for a class no organization falls in. Go does three things with those rows and nothing else: it groups them into the preview's lists, takes each class's count from the rows' exact count, never from the length of a list, and for the at most 50 other-source organizations reads their positions' sources in one statement (`ListLivePlanSourcesByPools`) to name what holds each one. The preview endpoint, each radio re-post and the 422 rebuild run the same reads. A probe of this statement's shape on temporary tables of 256,000 organizations took 1.4 s, both of its sorts spilling to disk under the default `work_mem`; task 1 times the real shape.
|
||||
- **The move reads its one pool's class through the same rule.** `GetPlanPoolBucket(pool_id, target)` folds the pool's live positions the same way and returns one row, the class. For an outgoing pool, `ListOutgoingPlanPositionsByPool(pool_id, target)` returns the distinct provisions whose positions the predicate marks outgoing, with their grant and product: the incumbents a grandfather or a migrate acts on. Go switches on the class and passes those provisions on.
|
||||
- **The gate reads two index entries per type.** `core.outgoing_default_exists(org_types, target)` reads, for each type, the lowest and the highest `product_id` among the type's live default-sourced positions, through the partial index `idx_pool_provision_ladders_live_default (org_type, product_id) WHERE default_sourced AND status IN ('active', 'suspended')`, and applies `core.plan_position_is_outgoing` to each. If every product equals the target, both extremes equal it and neither is outgoing; if any product differs, one extreme differs; with a None target, any position is outgoing. So the gate is true exactly when some position the preview folds is outgoing, and some pool therefore falls in `outgoing_default`. Each probe reads one live entry, plus the dead entries it passes on the way (below).
|
||||
- **The copies' sources cannot change.** One trigger function, `core.refuse_position_source_change()`, is attached as five triggers, one per watched column, each `BEFORE UPDATE OF` its column, `FOR EACH ROW`, `WHEN (OLD.col IS DISTINCT FROM NEW.col)`, with the column's name as the trigger's argument: `core.organizations (org_type)`, `core.resource_pools (org_id)`, `core.pool_provisions (grant_id)`, `core.pool_provisions (pool_id)` and `core.grants (grant_reason)`. It raises `restrict_violation` naming the column its trigger passes (`TG_ARGV[0]`), so each refusal names the one column that changed. The `WHEN` clause is needed because `UPDATE OF` fires on any statement that assigns the column, and `internal/db/integration_delivery_triggers_test.go:205` assigns `org_type = org_type` to the same value. This is the enforcement of the precondition the gate's copies rely on: the gate, the preview, the obligations' ownership and the act's record of a type all assume that an organization stays in its type, a pool in its organization, a provision on its grant and pool, and a grant at its reason. Without it, a future writer of one of them would leave the old type's gate true while its preview shows nothing outgoing, and the type's default could not be changed. A feature that must change one of them decides what the copies and the obligations do and replaces the guard in its own migration. The only statements that ever rewrote these columns, the migrations of `00004_doc41_schema_and_data.sql`, ran before the guard exists.
|
||||
- **One test pins the agreement.** `TestDispositionGateAgreesWithThePreview` builds a mixed pool (one default-sourced position at the target, one on another ladder not at it), a None target, an all-at-target type, an other-source type, a plan-less type, a pool-less organization and an empty type, and asserts, for each, that the preview's read finds an outgoing organization exactly when `core.outgoing_default_exists` is true, and that `GetPlanPoolBucket` gives every pool the class the preview's read gave it. The preview, the gate and the move therefore cannot disagree on the mixed pool or the None target.
|
||||
|
||||
The gate's population equals the preview's on the facts above: a default-sourced position sits on the organization's default pool, which is unique, and no statement changes a pool's status or type. T0 runs the gate after recording the act, with the reach and target the act's record names (D9), and when the operator chose no disposition. When the gate is true, the act returns `DispositionRequiredError`, the transaction rolls back, and the handler rebuilds the preview for its 422 answer after the rollback, as the removal does today. If the rebuilt preview's outgoing count is zero although the gate was true, the handler logs an Error with the reached types and the target the gate read and answers the form's generic failure, not a request to choose a disposition the card has no radio for. With the guard in place, this happens only when a move settled the last outgoing organization between the rollback and the rebuild, and resubmitting then succeeds.
|
||||
|
||||
**The walk after a mass ending.** An ended position's entry stays in the partial index until vacuum removes it. An index scan that reaches such an entry visits its heap row, and when no open snapshot can still see that row as live, the scan marks the entry dead, so later scans skip it without visiting the heap. The gate's probe is therefore an index probe whose first read after a mass ending walks the entries vacuum has not yet removed, after which those entries are marked dead and the probe is back to its usual cost. Measured against temporary tables in the test stack's PostgreSQL 18.1: with 100,001 live positions the probe took 0.105 ms; the first probe after 100,000 of them ended took 20.4 ms and visited 100,001 heap rows; the second took 0.097 ms. With 51,250 of 256,001 ended: 0.040 ms, 5.0 ms, 0.034 ms. The slower of the two runs is about 20 ms per 100,000 dead entries.
|
||||
|
||||
Two rules keep the walk out of the exclusive hold and bound it when it lands there:
|
||||
|
||||
- **The act that ended the positions pays for the walk.** When a pass, in the request or the drain, finds its act's listing complete and nothing pending, and the act settled any obligation `migrated` or `grandfathered` (the two outcomes that end default-sourced positions), it runs `core.outgoing_default_exists` once for the act's owned reached types and its target, in its own transaction under the shared rendezvous, before the act is done. Every move of the act has committed by then, so when no older snapshot is open that probe marks the ended entries dead, and the next T0's probe reads at its usual cost under the exclusive hold.
|
||||
- **Autovacuum bounds what is left.** Positions also end for other reasons (a subscription replacing a default, a removal), and the act's own probe can meet an older snapshot. Whatever ended them, autovacuum removes the dead entries once they pass its threshold for the position table: with PostgreSQL's defaults (`autovacuum_vacuum_threshold` 50, `autovacuum_vacuum_scale_factor` 0.2), 50 plus a fifth of the table's rows, which is 51,250 when the table holds one position for each of 256,000 organizations. Task 1 bounds the first probe after that many endings.
|
||||
|
||||
While a snapshot older than the endings stays open, in any session on the database, no scan can mark those entries dead, so every probe repeats the walk until that snapshot closes. For the read-only role, `scripts/db-reader-role.sql:177-178` bounds the common cases: `statement_timeout = '15s'` ends a statement, and with it a read-committed snapshot, after 15 s, and `idle_in_transaction_session_timeout = '1min'` ends a transaction left idle for a minute. A repeatable-read transaction that issues a statement at least once a minute keeps its snapshot without a limit, in that role or any other; while it does, each probe costs the walk again, and task 1 bounds each such probe the same way.
|
||||
|
||||
Alternatives considered: run the preview's read inside T0 (one rule, but linear in the type, which the exclusive hold cannot afford); a counter row per org type and product maintained by triggers (every signup of the personal type would update one row, serializing signups on it); live counters per (type, product) sharded across several rows by pool id and kept by triggers (no serialization, but a second record of the positions beside the positions themselves, which can drift from them, and the reason the gate reads the positions' own index instead). Rejected.
|
||||
|
||||
### D5. A generic act header with one detail table per kind
|
||||
|
||||
`core.acts` is the supertype: cause, actor, note, request id, times. What a plan act did goes in `core.plan_act_details`, keyed by the act and tied to it by a composite foreign key on `(act_id, cause)`. Each later kind adds its own detail table, and a provider-specific one sits in the provider's schema with a foreign key to `core.acts`.
|
||||
|
||||
Alternatives considered: a JSONB payload on the act (the repository's JSONB policy forbids relational references in JSONB, and the ladder, product and org type ids are exactly that); one plan-act table with the actor columns on it (repeats #181's flaw of an act record per kind with no common header, so the followers could not share readers); an exclusive arc of nullable foreign keys on the act (grows a column per kind and cannot carry a kind's several columns). Rejected.
|
||||
|
||||
### D6. Plan-specific obligations, a shared convention
|
||||
|
||||
`core.plan_act_obligations` is keyed by act and pool and holds only what a plan act owes, plus the organization's id and type as the listing found them, which order the listing and the work list. What later work lists share is the convention, not the table: a `<kind>_obligations` table keyed by the act and the target, with foreign keys to its act's detail row and to the target; statuses `pending`, `settled`, `failed` and, where a newer act can take over, `superseded`; `attempts`, `last_attempt_at`, `last_error_step`, `last_error`, `settled_at`; a partial index on open rows; a failure recorded on the obligation alone (D13); and Retry by act.
|
||||
|
||||
Alternative considered: one generic table keyed by (act, target kind, target id). It gives up the foreign key to the pool, and the design allows a bare polymorphic reference only on append-only tables; a work list is mutable. A pool-grain table shared with the rule change would need kind-scoped unique indexes and columns that belong to one kind. Rejected.
|
||||
|
||||
### D7. History and decree rows point at their act; catalog rows do not
|
||||
|
||||
Every row an act inserts that records what happened or what was decided carries `act_id`: the detail row, the reached org types, the records of organizations without a default pool, the obligations, and the grants the act creates (`core.grants.act_id`, nullable, because no other writer sets it yet). A grant the default floor creates stays system-authored (`granted_by_person_id` NULL) while its `act_id` names the operator's act: the act and the authorship of the decree are two facts. Catalog rows (`plan_ladder_tiers`, `org_types.default_plan_ladder_id`) do not point: a deleted tier cannot point at anything, and the detail row records the catalog change instead. Current-state rows (provisions, positions, entitlements) and the outbox do not point. Transitions get `act_id` when #179 replaces the conferral functions, which are the only writers of `pool_provision_transitions`; until then the reason text from one builder (D19) names the act.
|
||||
|
||||
### D8. The act record is enclosed
|
||||
|
||||
The five act tables (`core.acts`, `core.plan_act_details`, `core.plan_act_reached_org_types`, `core.plan_act_orgs_without_pool`, `core.plan_act_obligations`) are enclosed in the writer-role sense the per-schema roles already define: `core_writer` holds SELECT and no INSERT, UPDATE, DELETE or TRUNCATE on any of them. Five `SECURITY DEFINER` functions owned by `core_owner`, with `search_path = core, pg_temp`, are the only writers:
|
||||
|
||||
- `core.record_plan_act` inserts the act row, its detail row and, for a default change or a reorder that changed a default ladder's rank 0, the reached org types it derives from the detail (D9); for a reorder it reads the ladder's rank-0 product from `core.plan_ladder_tiers` and refuses a detail whose new rank-0 product disagrees, so the caller cannot misstate which product the reorder made rank 0 (the previous product is still the caller's word, as only the caller saw the ranks before the renumbering). For the four catalog causes it first asserts the exclusive rendezvous (`core.assert_rendezvous(TRUE)`); for a Retry it asserts the rendezvous in either mode, and refuses a retried act whose cause is neither a default change nor a reorder.
|
||||
- `core.write_plan_act_obligations` runs one listing batch (D10). It asserts the rendezvous in either mode.
|
||||
- `core.settle_plan_act_obligation` sets one pending obligation `settled` with its outcome. It asserts the rendezvous in either mode.
|
||||
- `core.record_plan_act_obligation_failure` records one failed attempt on a pending obligation (D13). It asserts nothing: it runs in a plain transaction.
|
||||
- `core.requeue_plan_act_obligations` returns a retried act's failed obligations to `pending` and points them at the Retry act (D11). It asserts the rendezvous in either mode.
|
||||
|
||||
Every write therefore goes through a function, so a status transition the design does not name (a failed row returned to pending without a Retry act, a settled row reopened, a superseded row revived) is not a privilege `core_writer` holds. This is the rule-change ledger's posture (`core.commit_rule_change`).
|
||||
|
||||
`core_reader` reads every column of the five tables except `core.acts.note`, through column grants on `core.acts`; every integration's writer role holds `core_reader`, and the note is the one column that holds a person's own words. `console_reader` withholds the same column. `core_writer` reads the note, because the console will show it (#238).
|
||||
|
||||
### D9. Reach, ownership, and supersession
|
||||
|
||||
T0 records which org types the act reaches, one `core.plan_act_reached_org_types` row per type: the default change's type, or the types that default to the reordered ladder when its rank 0 changed. `core.record_plan_act` derives them from the detail row inside the function, so the record and the work list cannot disagree with the caller about the reach. Each row carries the type's name, the greatest organization id of the type when T0 read it (`last_org_id`, NULL for a type with no organization), and `reached_seq`, an identity value: T0s are serialized by the exclusive rendezvous, so a later T0 always draws a larger value, whatever the wall clock does. That holds while the identity sequence keeps `CACHE 1`, its default: with a larger cache another backend could draw a smaller value later. The column's comment says so, and a test reads `pg_sequences.cache_size` for the sequence and expects 1.
|
||||
|
||||
**The owner of an org type** is the act whose reached-types row for it has the greatest `reached_seq`. At most one act owns a type at any moment, and ownership changes only when a T0 commits under the exclusive rendezvous. One SQL function, `core.plan_act_owns_org_type(act_id, org_type)`, states the rule once (STABLE, invoker rights, one probe of `idx_plan_act_reached_org_types_owner`), and every reader of ownership calls it: the obligation lock, the pending list, the listing-done read, the list of acts owing work, the open-acts reads, the unmoved list and the toast's counts, and both database functions that supersede and requeue (`core.write_plan_act_obligations`, `core.requeue_plan_act_obligations`). A reader that forgot the rule would show an older act's rows as still waiting after a newer act took the type, so a test checks that the text of each of those queries and the definition of each of those functions contains a call to it, in the style of the mirror tests. Every reader of ownership (a move, a listing batch, Retry, the drain's work list, the places' lines) holds the shared rendezvous or reads committed rows only, so it sees a stable owner for its transaction.
|
||||
|
||||
**From the moment a newer act's T0 commits, an earlier act never acts on a pool the newer act reaches**, even before the newer act's listing has reached that pool:
|
||||
|
||||
- A move of the earlier act reads the owner in the statement that locks its obligation, under the shared rendezvous; it finds the newer act and writes nothing (D2).
|
||||
- The earlier act's drain lists only obligations of types it still owns, and its listing batches list only those types, so it stops spending passes on them.
|
||||
- Its Retry requeues only failed obligations of types it still owns.
|
||||
- Its lines on the page count only those types, so its open obligations on a type it lost are not shown as not yet reached or not moved.
|
||||
|
||||
**The newer act's listing marks the earlier rows.** When a batch of the newer act lists a pool, it first marks any open (pending or failed) obligation of an earlier act on that pool `superseded`, with `superseded_by_act_id` naming the newer act, and then inserts its own. The newer act lists every organization the earlier one listed, because its bound is at least as large, so every earlier open row on the type is superseded once the newer listing completes. A partial unique index allows one open obligation per pool, so an insert that skipped the supersession fails rather than leaving two acts owing one pool. That holds only while the insert names its conflict arbiter: an `ON CONFLICT DO NOTHING` with no target also swallows a violation of the partial index and writes nothing, which would leave the organization with no obligation of the newer act. The inserts therefore name their primary keys (D10). The newer act's preview already counted those organizations, so its disposition is the one the operator chose for them. A superseded obligation keeps its attempts and last error, is not a failure, is not listed as unmoved, and stays in the record.
|
||||
|
||||
Alternative considered: refuse the second act until the first finishes. It blocks the operator on work they have just decided to replace, and on a failed obligation it blocks them until someone presses Retry on a change they no longer want. Rejected.
|
||||
|
||||
### D10. The listing writes obligations in short batches after the commit
|
||||
|
||||
A listing batch (`core.write_plan_act_obligations(act_id, batch_size)`) runs in its own transaction under the shared rendezvous:
|
||||
|
||||
1. It reads the types the act still owns and each type's `last_org_id`.
|
||||
2. It derives its cursor from the rows the act has already written: the greatest (org type, organization id) among the act's obligations and its records of organizations without a default pool, each one index probe. No column on the act or its detail changes; both stay immutable.
|
||||
3. It reads the next `batch_size` organizations of the owned types after the cursor, in (org type, organization id) order, through a new index on `core.organizations (org_type, org_id)`, and finds each one's active default pool with its own probe of the pools' organization index, a `LATERAL` subquery that ends in `LIMIT 1` (exact, because the unique index on an organization's default pool allows one). `LIMIT 1` is what makes the probe per organization: a `LATERAL` subquery without it is pulled up into the join tree, and the planner may then merge or hash the batch's organizations with every pool of the type. Written as a plain join, a middle batch at 256,000 organizations walked every pool before the cursor and took 61 ms against 18 ms at the start (an earlier revision of the timed run). The committed run times the read inside a plpgsql function under `plan_cache_mode = force_generic_plan`, and its logged generic plan is a nested loop with one index probe of the pools for each organization; its batches of 500 took 12.9 to 16.5 ms at every size and position in one run and 13.4 to 15.6 ms in the other, apart from one 25.6 ms median at 16,000 organizations, and an empty batch took 0.3 ms. An organization whose id is greater than its type's `last_org_id` is outside the bound and is not listed.
|
||||
4. It takes the batch's pool rows `FOR KEY SHARE` in ascending `pool_id` order, then the open obligations of earlier acts on those pools `FOR UPDATE` in the same order, marks them `superseded`, and inserts the new obligations in (org type, organization id) order. Two batches of one act with nothing to supersede both go straight to their inserts, and waits on the unique index follow insertion order, so a fixed order keeps them from deadlocking; a deadlock that happens anyway is benign, because the next pass retries the batch. The supersession and the insert are two statements: a data-modifying CTE would run both against one snapshot, and the insert's unique check could meet the row not yet superseded.
|
||||
5. For an organization with no active default pool it inserts a `core.plan_act_orgs_without_pool` row instead of an obligation. Two batches of one act can disagree on whether an organization's pool exists, because workspace creation can commit a default pool between them. The obligation insert therefore skips an organization that already has a without-pool record for the act, and the record insert skips one that already has an obligation; where both batches wrote before either committed, the count of organizations without a pool counts only those records whose organization has no obligation of the act.
|
||||
6. It returns the rows it inserted. The obligation insert is `ON CONFLICT (act_id, pool_id) DO NOTHING` and the without-pool insert is `ON CONFLICT (act_id, org_type, org_id) DO NOTHING`, each naming its primary key, so two batches of one act that run at once (the request's and a drain the poller started) write each organization once, while a violation of the one-open-obligation index (an older open row the supersession missed) raises and rolls the batch back instead of being skipped. `ON CONFLICT` checks only its arbiter before it inserts, so two batches of one act can still collide on the other unique indexes: when batch B checks the primary key before batch A's row for the same pool exists, B's insert then waits on that row in `uq_plan_act_obligations_listing` or `uq_plan_act_obligations_one_open` and raises 23505 once A commits. That is benign, because B rolls back and the next pass lists the organizations A did not, but B's failure is logged with the same Error line as a missed supersession, so that line alone does not prove a supersession was missed.
|
||||
|
||||
Whether the listing is complete is derived the same way (`GetPlanActListingDone`): no organization of an owned type lies after the cursor within its bound. A crash between batches loses nothing: the next batch starts from the last committed row.
|
||||
|
||||
**Organizations created after T0** get the current default at signup, which is the act's target, and need no obligation. The bound excludes them: every organization that existed at T0 has an id no greater than the `last_org_id` T0 read (no signup runs during the exclusive hold), and an organization created afterwards draws its id from `uuidv7()` later. When a clock step gives a later organization a smaller id, the listing writes an obligation for it and its move settles it like any other organization's: `already_at` when it was created on the new default, `untouched` when it subscribed first, `left` under a None target. None of these changes a position the organization should not have.
|
||||
|
||||
### D11. Retry is an act; it requeues one act's failed obligations
|
||||
|
||||
Each act's lines on its place carry their own Retry. Pressing it runs one transaction that opens under the shared rendezvous (`Materializing`) first, records one act row with cause `plan_act_retried`, the operator as actor, and a detail row naming the act it retried (`retried_act_id`), then requeues that act's failed obligations of the types it still owns: each returns to `pending` with its attempts reset to zero and its last error cleared, and `requeued_by_act_id` names the Retry act. When nothing was requeued (a second press, or a newer act took the type over), the transaction rolls back and writes nothing. The request then settles what it can within its budget and starts the drain. A later Retry overwrites `requeued_by_act_id`; each Retry's own act row stays.
|
||||
|
||||
One Retry acts on one act, so it commits or fails as one transaction; there is no partial Retry across acts.
|
||||
|
||||
Alternative considered: write new obligations under the Retry act and supersede the old ones. It splits one organization's history across acts and overloads `superseded`, which means a newer change took over. Rejected.
|
||||
|
||||
### D12. The actor and act of each move
|
||||
|
||||
A move records the actor of the act that put its obligation in motion: the act's actor, or the Retry act's actor when `requeued_by_act_id` is set. The grants it creates carry that act's id. Its transitions carry that actor and the reason text of the original act, built from the original act's detail row. A move the drain settles minutes later is still the operator's act, so it records the operator, as today's in-request moves do.
|
||||
|
||||
### D13. A failure is recorded on the obligation alone, in a plain transaction
|
||||
|
||||
A failed move rolls back. A second, plain transaction (no rendezvous, no pool row) calls `core.record_plan_act_obligation_failure`, which updates only that obligation row, and only while it is still `pending`: attempts plus one, the time, the failing step's code and the error text, and `failed` once attempts reach the maximum. The maximum is a constant inside the function (5, equal to the rule change's `recomputeMaxAttempts`), not an argument, so the database enforces the dead-letter threshold; a test compares it with `planActMaxAttempts`. The function returns the obligation's status after the update, or NULL when the row was no longer pending and nothing was updated; the Go caller treats NULL as a recorded no-op and not as a failure to record. The rule change's recorder takes the obligation row before the pool row, the reverse of its drain (#196); this recorder takes no pool row at all, so it cannot form a cycle.
|
||||
|
||||
Two errors are never recorded as an attempt:
|
||||
|
||||
- a context error (the client left, the request's deadline passed, or the worker is stopping), because it says nothing about the organization;
|
||||
- a lock wait past the request's shorter lock timeout inside a request (D14), because the drain retries it with the full timeout.
|
||||
|
||||
When the recording itself fails, the obligation stays pending with its attempt uncounted, both errors are logged, and the next pass retries it, which is the safe way to lose an attempt.
|
||||
|
||||
### D14. The request runs a bounded share; a drain runs the rest
|
||||
|
||||
After T0 commits, the request:
|
||||
|
||||
1. runs one listing batch (`planActListBatch` organizations) when less than `planActRequestBudget` has passed since the request began;
|
||||
2. settles pending obligations of the types the act owns, in (org type, organization id) order, the order the listing wrote them, until `planActRequestCap` obligations are settled or `planActRequestBudget` has passed since the request began;
|
||||
3. stops at the first lock wait past `planActRequestLockTimeout`, at the first transaction the database ends for passing `planActRequestTxTimeout`, at the first context error (the request's deadline included), and at the cap or the budget, recording no attempt for any of these;
|
||||
4. starts the act's drain workflow (`PlanActDrainWorkflow`, id `plan-act-<act id>`, conflict policy use-existing) when the listing is incomplete or an obligation is still pending.
|
||||
|
||||
The listing batch and every move in the request open through the shared opener with `planActRequestLockTimeout` in place of the 5 s timeout. The shared opener takes its lock timeout and its transaction timeout as parameters, one opener with two settings (`MaterializingWithin(lockTimeout, txTimeout)`, of which `Materializing` is the case of 5 s and no transaction timeout), so the "one opener for each side" requirement holds. `lock_timeout` bounds one wait, not their sum, and a listing batch takes up to `planActListBatch` key-share locks, so each in-request transaction also sets `transaction_timeout` to `planActRequestTxTimeout`. The opener sets both with `SET LOCAL` before its rendezvous statement, because PostgreSQL starts the transaction timer at the `SET`, not at `BEGIN`, and a timer set after the rendezvous would leave the rendezvous wait outside it. `transaction_timeout` exists from PostgreSQL 17; the repository requires 18 (`docs/database-management.md:9`, for `uuidv7()`), so every supported database has it.
|
||||
|
||||
When a transaction passes `transaction_timeout`, PostgreSQL ends the session, not only the transaction: it sends FATAL 25P04 ("terminating connection due to transaction timeout"), writes one FATAL line to the server log each time, and closes the connection. pgx closes a connection on any FATAL error (its default `OnPgError`), and `database/sql` then replaces it in the pool. When the timer fires while the client is between statements, the client meets the error on its next statement, either as 25P04 or as a closed connection. The request's deadline can end a connection too: pgx's default context watcher sets a past deadline on the socket when the context is cancelled, so the statement in flight fails and the connection is discarded. The request therefore records no attempt for SQLSTATE 25P04, and none for an error that reports a closed or broken connection once the transaction has run for `planActRequestTxTimeout` or the request's deadline has passed; one function beside `db.IsLockContention` states that rule, and a test drives both shapes (the timer firing during a statement and between two) against the test database. Any other connection error is a failed step and is recorded as one.
|
||||
|
||||
The act's work in the request is bounded by a context deadline, `planActRequestDeadline` after the request began, which covers T0 as well. The deadline's context is derived for the act's own calls only: T0, the listing batch and the moves. The handler renders its answer, reads the card's or the ladder page's lines, rebuilds the 422 preview and starts the drain on the request's own context, which the deadline does not end, so a T0 the deadline ended still renders its refusal with the card's lines, and a move the deadline cut still starts the drain; the server's 8 s write timeout still bounds when that answer can be written. T0 can wait twice, 5 s on the rendezvous and 5 s on the org type's row (held by the demonstration seed or an edit of the type), which together pass the 8 s write timeout; they share the deadline, and a T0 that the deadline ends rolls back and answers the contention refusal. The 422 path's rebuilt preview (D4) is one statement over the type, outside any lock hold, that names at most 50 organizations per list (D27); the deadline is set so that the deadline plus that statement's time at 256,000 organizations stays under the write timeout (task 1). T0 keeps the 5 s lock timeout for each wait.
|
||||
|
||||
A poller (`PollPlanActs`, every 5 s, continue-as-new) dispatches the drain of every act that owns an org type and still has work there (an incomplete listing or a pending obligation), so a failed start, a missing Temporal client or a dead process loses nothing. Only owners can have work, and at most one act owns each type, so the poller reads one row per org type, not one per act ever recorded. Temporal retries the poller's read activity five times with backoff from 1 s; if it still fails, for example across a database restart longer than that, the poller logs the failure and sleeps to its next tick instead of returning the error, so it does not end until the process does. The recompute poller has the same flaw (`PollEntitlementRecompute` returns the activity's error after the retries, and nothing restarts it), so this change gives it the same shape as a ride-along.
|
||||
|
||||
The drain workflow runs passes; each pass is one activity attempt that runs one listing batch when the listing is incomplete and then settles up to `planActRequestCap` pending obligations within `planActRequestBudget`, with the full 5 s lock timeout. A pass starts after the last obligation, in (org type, organization id) order, that the previous pass tried, and starts over when none lies after it, so an obligation whose attempt cannot be recorded does not keep the obligations behind it from being tried. A recorded attempt counts as progress, like a listed batch or a settled obligation, so a type whose every move fails reaches its dead letters without waiting between passes; a pass that listed nothing, settled nothing and recorded nothing makes the workflow sleep until the next poll, so a failure that repeats on every attempt, or a recording that fails deterministically, cannot spin it. The workflow ends when the listing is complete and nothing is pending; the pass that finds this, like a request that finds it, first runs the gate's probe once under the shared lock when the act settled any obligation `migrated` or `grandfathered` (D4), and a failure of that probe is logged with the act id and does not keep the workflow running, because the next T0 still answers, at the cost of the walk. The workflow continues as new after a fixed number of passes, carrying its cursor. A pool's own failure is not an activity failure. The workflow, its activity and the poller are modelled on the recompute drain and its poller.
|
||||
|
||||
The constants are set from the timed run of task 1 (`TestTimedPlanActCaps`, environment-gated like `TestTimedSyncCap`), and each lands in `internal/entitlements/constants.go` in the task of its first caller, with the measurement and its rule in the comment. Until the run, the placeholders are: 50 obligations and 2 s for the request's cap and budget (the rule change's values), 500 organizations per listing batch, 500 ms for the request's lock timeout, 2 s for the request's transaction timeout, 6 s for the request's deadline, and 250 holders for the holder limit (the rule change's synchronous cap). The rules: the request's values keep the request well inside its 8 s write timeout, and the deadline plus the preview read's time at 256,000 organizations stays under it; a listing batch holds its pool key-shares for under 100 ms on the measuring host; the holder limit keeps a removal's or an add's exclusive hold near half a second on the measuring host, the criterion `rematerializeSyncCap` uses.
|
||||
|
||||
### D15. Tier add and removal stay atomic, under a holder limit
|
||||
|
||||
A live position left on a deleted tier makes `core.confer` read no rank for it and record `initiate` where an upgrade, downgrade or transfer belongs, so a removal's alignment cannot be split from the deletion, and an add's from the insert. Both stay one transaction, write their act row and detail row, and write no obligations. Inside the transaction, before locking any pool, each counts its holders: the organizations holding a live position on the tier, or a live provision of the product being added, and how many of them hold it through an org type's default on a ladder (the position's `default_sourced` column). Above `planActHolderCap` it refuses, naming the count and the limit, and, when more than half of the holders hold it through an org type's default on a ladder, names the way through. The way through is a default change with Migrate, which ends only default-sourced positions on a ladder. A default-sourced provision that a `keep` removal left off every ladder (`ladder_acts.go:266-268`) counts toward the limit, because the add aligns it, but not toward the second sentence: a default change classifies its pool as would-initiate, floors it, and leaves the off-ladder provision live, so the holder count would not fall and the add would be refused again.
|
||||
|
||||
The rules are one pure function each in `internal/entitlements`, which read no row and which both the preview and the act call: `HolderLimitRefusal(holders, defaultSourced, cap, rankZeroOfTheirDefault)` returns the sentences or nothing, and `LastTierRefusal(tiers, productID, defaultingTypes)` returns the last-tier sentence or nothing. The removal's preview reads the same counts and the same org types and renders "Apply change" disabled with the sentence as its reason, as the destructive-controls requirement asks of an action certain to be refused. #237 replaces the limit with retire-then-remove.
|
||||
|
||||
The tier add reads the product inside its transaction and refuses a product that is not published there, so the check and the insert are one transaction and a failed read is a failed add. No runtime statement writes a product's lifecycle status today (`UpdateProduct` does not set it; only test kits do), so the read does not guard against a concurrent change; if a writer is added, the read takes the product row `FOR SHARE`, which conflicts with a non-key update, where the tier insert's foreign key takes only `FOR KEY SHARE`. A missing product answers "Product not found."; any other error of that read answers the generic failure, because a failed read is not a missing product.
|
||||
|
||||
**A held rung is found before any alignment.** `core.align_conferral_shape` raises CF003 when another live provision of the same pool already holds a junction row on the ladder (`internal/db/migrations/00005_doc41_conferral_functions.sql:491-504`). Inside one add, that blocker can be a row the same add has just written: a pool that holds two live provisions of the product (an operator grant and a subscription, say) has the first aligned onto the ladder and the second refused, and after the rollback no position on the ladder is left to name. The add therefore reads the collision itself, under its pool locks, after the tier insert and before the first alignment. `ListTierAddCollisions(ladder_id, product_id, pool_ids)` returns, for the locked pools, each pool that already holds a live position on the ladder, with its organization's name and the product that position delivers, and each pool that holds more than one live provision of the product, with its organization's name. The first such pool in ascending pool id order refuses the add with a `*TierAddCollisionError` carrying those names, all read inside the transaction, so the answer never depends on a read after the rollback; a pool that matches both cases is named for the held rung. The exclusive rendezvous and the pool locks keep any conferral from changing those positions between the read and the alignment, so a CF003 that still reaches the add is a defect: it is logged with the driver text and answered with the add's generic failure. The read follows the tier insert so that a product already on the ladder is refused as "already a tier" first.
|
||||
|
||||
Alternative considered: obligations for the removal, settled inside its own transaction. It records the same reach at the cost of a write per holder inside the longest exclusive hold the console takes, and gives #82 nothing it cannot get later. Rejected for now.
|
||||
|
||||
### D16. A tier delete is a removal with no holders
|
||||
|
||||
The plain delete records cause `tier_removed` with no disposition. Its live-holder count, the ladder's tier list and the org types that default to the ladder are read inside its exclusive transaction, and a failed read refuses the delete. The count covers live positions, `active` and `suspended`, the population the removal reconciles; counting only `active` ones let a tier held only by a past-due subscription's suspended position be deleted under it, leaving a junction row on a ladder that no longer has the tier, which makes the next conferral record `initiate` where a move belongs. The tier page's choice between the confirmation modal and Preview removal uses the same live count (a ride-along). The removal with holders runs the same last-tier guard inside its transaction under both dispositions.
|
||||
|
||||
### D17. One runner
|
||||
|
||||
`runPlanAct` in `internal/entitlements` runs every plan act's transaction: it opens the given transaction kind, calls the act's `find` to name the pools, locks them through `LockPools` (ascending, each once), calls `reread` under the locks, then `act`, then commits, and returns a `*PlanActError` naming the step. The tier add, the tier delete, the tier removal, the two T0s and one obligation's move all go through it. Grant acts stay out: their preambles take the lineage head after the pool in an order of their own. The listing batch, the failure recorder and Retry are single-function transactions outside it.
|
||||
|
||||
The runner builds the act's queriers through constructors its input carries (the entitlements, billing and organization `Querier` interfaces sqlc generates). Production passes the generated constructors. The golden and concurrency tests pass constructors that wrap them and refuse named statements, so a scenario can make a read or a write inside the act's own transaction fail; a handler's querier no longer reaches the act. The listing batch, the failure recorder, Retry and the drain's activity run outside the runner and take the same constructors, so a golden can make `WritePlanActObligations` or the recorder's statement fail too.
|
||||
|
||||
The same input carries the request's limits (`PlanActLimits`: the lock timeout, the transaction timeout, the cap, the budget, the listing batch and the deadline of D14). Production passes the constants; the handler takes the constructors and the limits from `OperatorPartialsConfig.PlanActs`, whose zero value means production's, and the worker passes production's to the drain. The handler-driven concurrency tests set the lock timeout to the full 5 s and lift the transaction timeout and the deadline above their own waits, so the lock queue, and not a timer, decides which transaction runs first; `default-request-cap.json` sets the cap to one and the budget above the run, so the cap and not a timer stops the request.
|
||||
|
||||
### D18. Step codes, and sentences rendered from them
|
||||
|
||||
The obligation's `last_error_step` is a closed list (`CHECK`): `in_progress` (any lock wait past the timeout, deadlock or serialization failure), `begin`, `lock_pool`, `read_obligation`, `read_default`, `resolve_default`, `read_positions`, `initiate`, `grandfather`, `grandfather_outcome`, `end_incumbents`, `restore`, `materialize`, `settle`, `commit`. `read_default` is any error other than no rows on the read of the type's default ladder and its rank-0 tier; `resolve_default` is that read finding no rank-0 tier, which only damaged data produces now: a default ladder can no longer lose its last tier (D16), and a default change refuses a candidate with no rank 0. The page renders one sentence per code (section "Operator-facing copy"), naming only what failed. `last_error` keeps the driver text for the logs and the read-only role; no page renders it. A test compares the Go codes and outcomes with `chk_plan_act_obligations_error_step` and `chk_plan_act_obligations_outcome`, so a code the constraint does not list cannot make every recording fail.
|
||||
|
||||
**An outgoing position under an act with no disposition** is settled `untouched`, and one Warn line names the act and the organization. The gate (D4) proves no organization of the reached types held an outgoing position at T0, so such a position appeared later, for example when a tier add inducts a default-sourced provision that a `keep` removal left off the ladder. Leaving it is the floor rule: an act with no disposition only floors vacant pools, as `ReapplyDefaultsIfVacant` touches no occupied pool. A newer default change or reorder with a disposition moves it.
|
||||
|
||||
### D19. One reason builder
|
||||
|
||||
`PlanActReason(detail)` in `internal/entitlements` builds every plan act's transition reason from the detail row's snapshots: `tier added to ladder <ladder name>`, `tier removal (<product> from <ladder>)`, `plan-ladder tier reorder (<ladder>)`, `org-type default change (<org type key>)`. `MigratedReason` and `GrandfatheredReason` add the existing suffixes, and the legacy grant's description is `Grandfathered: <reason>`. The prefixes are unchanged, so the Tier changes list's humanizer needs no edit. Existing rows are not rewritten.
|
||||
|
||||
### D20. A default change to what is already the default refuses
|
||||
|
||||
The form refuses the current default. When two operators commit the same new default, the second T0 waits on the exclusive rendezvous, then reads the type under its row lock and finds the candidate already saved. It refuses with "Already the default.", the preview's existing line, and writes nothing.
|
||||
|
||||
### D21. Organizations without a default pool are recorded, logged and counted, not owed
|
||||
|
||||
An obligation is keyed by a pool, so an organization of a reached type with no active default pool cannot hold one. For a non-reserved type that is damaged data or a test fixture: `AutoProvision` creates the default pool in the transaction that creates the organization. The listing records each such organization once in `core.plan_act_orgs_without_pool` (act, organization, type, time) and, after its batch commits, writes one Error line naming the act and the organization, on the provisioning alarm's precedent (`provisioning.go`, "AUTO-PROVISIONING ALARM"). The count is derived from those rows, so a later page per act can show it. The toast's "N without a default pool" counts the organizations whose record the act holds when the request answers, an organization that also holds an obligation of the act counting once, as an obligation. An organization the request did not list is counted as not yet reached until a batch reaches it. The preview already names them under "would fail". They are not listed as unmoved, because Retry cannot move them.
|
||||
|
||||
### D22. One query lists an act's unmoved organizations; the places show each act's lines
|
||||
|
||||
`ListUnmovedOrganizations(act_ids, per_act_limit)` returns each given act's failed obligations on the types it owns, with the organization's id and current name and the step code, ordered by organization name and id, at most `per_act_limit` per act. Acts and obligations are never deleted, settled and superseded rows included. #78's overview line and a page per act can call it with one act id.
|
||||
|
||||
The org type's card shows the act that owns the type when it is a default change with open work; the ladder's page shows each reorder of the ladder that owns at least one type with open work, newest first. Open work is a pending or failed obligation on an owned type, or an organization of an owned type the listing has not reached. Each act gets its own lines, so one Reference and one Retry belong to one act. Each place reads its lines in two statements whatever the number of cards: the open acts with their counts (`ListOpenPlanActsByOrgTypes` or `ListOpenPlanActsByLadder`), then their unmoved organizations. The two run in one `REPEATABLE READ`, read-only transaction, so a T0 that commits between them cannot make the count come from one owner and the list from another. Every query that reads obligations through ownership calls `core.plan_act_owns_org_type` (D9).
|
||||
|
||||
### D23. Log lines carry the act and the organization; pages carry a reference
|
||||
|
||||
An operator who is not a server administrator cannot read the logs, so a page that only says the details are there gives them nothing to hand on. Instead:
|
||||
|
||||
- Every log line the listing, a move, the recorder and the drain write carries `act_id` and, for one organization, `org_id`; a line about a whole batch carries `act_id`. The obligation's stored driver text stays readable by the read-only role, keyed by the act id.
|
||||
- Each act's unmoved list ends with one line, "Reference: {act id}", with the full id, which finds every log line and row of the act.
|
||||
- An error a request answers in place of an answer (Retry's failure, the places' read failure, the removal preview's guard read failure) ends with "Reference: {request id}", and the handler logs the error through the request-scoped logger, `logging.FromContext(r.Context())`, which carries `request_id` (`middleware.Logging` stores it after `middleware.RequestID`, `internal/server/server.go`), so the line carries the same id.
|
||||
|
||||
### D24. The request id is stored only when it fits
|
||||
|
||||
The console takes `X-Request-ID` as the client sends it, with no length bound. The act stores it when it is at most 100 characters and stores NULL otherwise. It is never a key.
|
||||
|
||||
### D25. The act goldens record the act rows
|
||||
|
||||
`TestPlanActsGolden`'s fixture dump gains the act, detail, reached-type, without-pool and obligation rows (ids named by the normalizer, times dropped), and its Grants dump gains `act_id`, so every one of the 55 files under `internal/server/testdata/plan_acts/` changes on purpose and pins what each act recorded. The act and detail sections land with the act record (task 4); the other three sections and the grant column land with the obligations (task 6).
|
||||
|
||||
### D26. The cause list lives in `internal/entitlements` for now
|
||||
|
||||
The Go mirror of `chk_acts_cause` (`ActCause` constants) lives beside the plan acts, with a test that compares it with `pg_get_constraintdef`. It moves to a leaf package when the first act outside `internal/entitlements` adopts the record.
|
||||
|
||||
### D27. The act's lists of organizations are capped at 50
|
||||
|
||||
The default change's and the reorder's preview names every organization in the outgoing-default and other-source classes, and each change of the disposition radios posts the whole card again. On a personal type that is the membership's names on every click, which the act's any-size rule (D1) does not allow. The act's preview, its radio re-post and the 422 rebuild therefore name at most 50 organizations per list, ordered by name and then id, and end a longer list with "And N more.", as the unmoved list does. The same cap holds for the would-fail list, which only damaged data fills but which nothing else bounds. Every count the preview shows comes from the read's exact count for its class (D4), never from the length of a list the cap shortened: the counts above each list, the outgoing summary ("N organizations hold the outgoing default"), the disclosure label, the card's 422 sentence and the reorder's field error. The disclosure label "show all N" stays when the list holds every name and becomes "show 50 of N" when it does not (copy 49). The removal's preview is unchanged.
|
||||
|
||||
### D28. Defects in the code this change touches that ride along
|
||||
|
||||
Each is listed in the proposal with its reason and lands in the task that touches the code, with its own test.
|
||||
|
||||
- **The plain delete counts only `active` positions** (D16): a tier held only by suspended positions was deleted under them.
|
||||
- **The tier add's CF003 refusal shows driver text** (`refuseTierAdd`, `internal/server/operator_plan_ladders.go:456-459`): the add finds the collision under its pool locks before aligning (D15) and names the organization, and the product it holds on the ladder or the product it holds more than once, in console terms; a CF003 the read did not foresee is logged with the driver text and answered with the generic failure.
|
||||
- **`orgTypesDefaultingToLadder` hides a failed read** (`internal/server/operator_plan_ladders.go:1405-1431`): on a failed `ListOrgTypes` it logs and returns nothing, so the reorder preview says "display order only", the default-ladder badges vanish and the removal preview's promoted-default note drops its types. Its callers show copy 45 instead.
|
||||
- **The rule change's in-progress refusal names a cause it cannot know** (`internal/entitlements/rule_change.go:93`): it is shown for any exclusive wait that times out, behind a conferral, a move or a tier change as much as behind another rule change. It uses the plan acts' sentence (copy 23, pending approval), and the requirements that mandate the old sentence change with it.
|
||||
- **The entitlement-set-history requirement says the drain selects `failed` rows** (`openspec/specs/entitlement-set-history/spec.md:183`): `ListPendingObligations` selects `status = 'pending'` only and the dead letter depends on it. The requirement is corrected to the dead-letter behaviour.
|
||||
- **The recompute poller dies on a long database outage** (D14): it returns the read activity's error after Temporal's five retries.
|
||||
- **The default select's label builds a malformed refusal** (copy 47): the form library writes "Choose " plus the label with its article plus " from the list.", which reads "Choose a change default to from the list." for the label "Change default to".
|
||||
|
||||
## Schema
|
||||
|
||||
Three migrations in the core stream (00025 is the latest), each landing with its first caller. A sketch, not DDL to paste.
|
||||
|
||||
### `00026_outgoing_default_rule.sql`
|
||||
|
||||
```sql
|
||||
ALTER TABLE core.pool_provision_ladders
|
||||
ADD COLUMN org_type VARCHAR(20), -- the pool's organization's type, copied by trigger
|
||||
ADD COLUMN default_sourced BOOLEAN; -- the provision's grant has grant_reason 'default'
|
||||
|
||||
-- core.provision_is_default_sourced(p_grant_id uuid) RETURNS boolean: STABLE, invoker,
|
||||
-- SET search_path = core, pg_temp; false for a NULL grant or a grant of another reason.
|
||||
-- core.copy_position_sources() trigger function, SET search_path = core, pg_temp; sets NEW.org_type
|
||||
-- from the pool's organization and NEW.default_sourced from the provision's grant. Two triggers
|
||||
-- on core.pool_provision_ladders: BEFORE INSERT, and BEFORE UPDATE OF pool_id, provision_id
|
||||
-- WHEN (OLD.pool_id IS DISTINCT FROM NEW.pool_id OR OLD.provision_id IS DISTINCT FROM NEW.provision_id),
|
||||
-- because core.sync_pool_provision_ladders assigns pool_id on every status sync.
|
||||
-- Backfill: UPDATE every existing row with trigger_pool_provision_ladders_updated_at disabled for
|
||||
-- the statement, so updated_at keeps its value; then SET NOT NULL on both columns.
|
||||
|
||||
CREATE INDEX idx_pool_provision_ladders_live_default ON core.pool_provision_ladders (org_type, product_id)
|
||||
WHERE default_sourced AND status IN ('active', 'suspended');
|
||||
|
||||
-- core.plan_position_is_outgoing(p_default_sourced boolean, p_product_id uuid, p_target uuid)
|
||||
-- RETURNS boolean: IMMUTABLE SQL, no SET clause (inlined), p_default_sourced AND (p_target IS NULL OR p_product_id <> p_target).
|
||||
-- core.plan_pool_bucket(p_has_pool boolean, p_any_outgoing boolean, p_any_default_sourced boolean,
|
||||
-- p_any_position boolean) RETURNS varchar: IMMUTABLE SQL, no SET clause (inlined); a CASE in this order:
|
||||
-- 'would_fail' when NOT p_has_pool, 'outgoing_default', 'already_at_default', 'other_source', else 'would_initiate'.
|
||||
-- core.outgoing_default_exists(p_org_types varchar[], p_target uuid) RETURNS boolean: STABLE SQL;
|
||||
-- per type, the lowest and the highest product_id through idx_pool_provision_ladders_live_default
|
||||
-- (ORDER BY product_id LIMIT 1, and DESC), each tested with core.plan_position_is_outgoing.
|
||||
-- All five run with invoker rights; EXECUTE is granted to core_owner, core_writer and core_reader.
|
||||
--
|
||||
-- core.refuse_position_source_change() RETURNS trigger: raises restrict_violation naming TG_ARGV[0], the column.
|
||||
-- Five triggers, one per watched column, each passing its column's name:
|
||||
-- BEFORE UPDATE OF org_type ON core.organizations; OF org_id ON core.resource_pools;
|
||||
-- OF grant_id ON core.pool_provisions; OF pool_id ON core.pool_provisions; OF grant_reason ON core.grants;
|
||||
-- each FOR EACH ROW WHEN (OLD.<col> IS DISTINCT FROM NEW.<col>).
|
||||
-- Down drops the five triggers, then the function.
|
||||
```
|
||||
|
||||
### `00027_acts.sql`
|
||||
|
||||
```sql
|
||||
CREATE TABLE core.acts (
|
||||
act_id UUID NOT NULL DEFAULT uuidv7(),
|
||||
cause VARCHAR(50) NOT NULL,
|
||||
actor_type VARCHAR(20) NOT NULL,
|
||||
actor_person_id UUID,
|
||||
actor_service_account_id UUID, -- no FK: no such table yet
|
||||
note TEXT, -- a person's free text; withheld from core_reader and console_reader
|
||||
request_id VARCHAR(100), -- log correlation only; never a key
|
||||
effective_at TIMESTAMPTZ NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
CONSTRAINT pk_acts PRIMARY KEY (act_id),
|
||||
CONSTRAINT uq_acts_act_id_cause UNIQUE (act_id, cause), -- target of the detail tables' composite FK
|
||||
CONSTRAINT fk_acts_actor_person_id FOREIGN KEY (actor_person_id) REFERENCES core.persons (person_id),
|
||||
CONSTRAINT chk_acts_cause CHECK (cause IN (
|
||||
'tier_added', 'tier_removed', 'tiers_reordered', 'org_type_default_changed', 'plan_act_retried')),
|
||||
CONSTRAINT chk_acts_actor_type CHECK (actor_type IN ('person', 'service_account', 'system')),
|
||||
CONSTRAINT act_actor_coherent CHECK (
|
||||
(actor_type = 'person' AND actor_person_id IS NOT NULL AND actor_service_account_id IS NULL)
|
||||
OR (actor_type = 'service_account' AND actor_service_account_id IS NOT NULL AND actor_person_id IS NULL)
|
||||
OR (actor_type = 'system' AND actor_person_id IS NULL AND actor_service_account_id IS NULL)),
|
||||
CONSTRAINT act_note_by_a_person CHECK (note IS NULL OR actor_type = 'person')
|
||||
);
|
||||
|
||||
CREATE TABLE core.plan_act_details (
|
||||
act_id UUID NOT NULL,
|
||||
cause VARCHAR(50) NOT NULL,
|
||||
plan_ladder_id UUID, -- no FK: history outlives a deleted ladder; default change: the new default
|
||||
plan_ladder_name VARCHAR(255), -- snapshot at the act's commit
|
||||
product_id UUID, -- tier add and removal: the tier; reorder: the new rank 0; default change: the target's rank 0
|
||||
product_name VARCHAR(255),
|
||||
previous_product_id UUID, -- reorder: the rank 0 before
|
||||
previous_product_name VARCHAR(255),
|
||||
org_type VARCHAR(20), -- default change
|
||||
org_type_name VARCHAR(255),
|
||||
previous_plan_ladder_id UUID, -- default change: the outgoing default
|
||||
previous_plan_ladder_name VARCHAR(255),
|
||||
disposition VARCHAR(20), -- grandfather | migrate | keep; NULL when none was asked for
|
||||
retried_act_id UUID, -- plan_act_retried only
|
||||
CONSTRAINT pk_plan_act_details PRIMARY KEY (act_id),
|
||||
CONSTRAINT plan_act_detail_is_its_act FOREIGN KEY (act_id, cause) REFERENCES core.acts (act_id, cause),
|
||||
CONSTRAINT fk_plan_act_details_retried_act_id FOREIGN KEY (retried_act_id) REFERENCES core.plan_act_details (act_id),
|
||||
CONSTRAINT chk_plan_act_details_cause CHECK (cause IN (
|
||||
'tier_added', 'tier_removed', 'tiers_reordered', 'org_type_default_changed', 'plan_act_retried')),
|
||||
CONSTRAINT chk_plan_act_details_disposition CHECK (disposition IS NULL OR disposition IN ('grandfather', 'migrate', 'keep')),
|
||||
CONSTRAINT plan_act_detail_names_what_it_ids CHECK (
|
||||
(plan_ladder_id IS NULL) = (plan_ladder_name IS NULL)
|
||||
AND (product_id IS NULL) = (product_name IS NULL)
|
||||
AND (previous_product_id IS NULL) = (previous_product_name IS NULL)
|
||||
AND (org_type IS NULL) = (org_type_name IS NULL)
|
||||
AND (previous_plan_ladder_id IS NULL) = (previous_plan_ladder_name IS NULL)),
|
||||
CONSTRAINT plan_act_detail_shape CHECK (
|
||||
(cause = 'tier_added' AND plan_ladder_id IS NOT NULL AND product_id IS NOT NULL
|
||||
AND org_type IS NULL AND disposition IS NULL AND retried_act_id IS NULL)
|
||||
OR (cause = 'tier_removed' AND plan_ladder_id IS NOT NULL AND product_id IS NOT NULL
|
||||
AND org_type IS NULL AND (disposition IS NULL OR disposition IN ('keep', 'migrate')) AND retried_act_id IS NULL)
|
||||
OR (cause = 'tiers_reordered' AND plan_ladder_id IS NOT NULL AND org_type IS NULL
|
||||
AND (product_id IS NULL) = (previous_product_id IS NULL)
|
||||
AND (disposition IS NULL OR disposition IN ('grandfather', 'migrate')) AND retried_act_id IS NULL)
|
||||
OR (cause = 'org_type_default_changed' AND org_type IS NOT NULL
|
||||
AND (plan_ladder_id IS NULL) = (product_id IS NULL)
|
||||
AND (disposition IS NULL OR disposition IN ('grandfather', 'migrate')) AND retried_act_id IS NULL)
|
||||
OR (cause = 'plan_act_retried' AND retried_act_id IS NOT NULL AND plan_ladder_id IS NULL
|
||||
AND product_id IS NULL AND org_type IS NULL AND disposition IS NULL))
|
||||
);
|
||||
CREATE INDEX idx_plan_act_details_org_type ON core.plan_act_details (org_type, act_id DESC) WHERE org_type IS NOT NULL;
|
||||
CREATE INDEX idx_plan_act_details_plan_ladder_id ON core.plan_act_details (plan_ladder_id, act_id DESC) WHERE plan_ladder_id IS NOT NULL;
|
||||
|
||||
-- core.record_plan_act(cause, actor triple, note, request_id, effective_at, the detail's columns) RETURNS uuid
|
||||
-- (00028 replaces it to write the reached org types and to check a Retry's retried act)
|
||||
|
||||
GRANT ALL ON core.acts, core.plan_act_details TO core_owner;
|
||||
GRANT SELECT ON core.acts, core.plan_act_details TO core_writer;
|
||||
GRANT SELECT (act_id, cause, actor_type, actor_person_id, actor_service_account_id, request_id, effective_at, created_at)
|
||||
ON core.acts TO core_reader; -- every column but note
|
||||
GRANT SELECT ON core.plan_act_details TO core_reader;
|
||||
```
|
||||
|
||||
`scripts/db-reader-role.sql` gains `'^core\.acts\.note$'` on its `personal` list.
|
||||
|
||||
### `00028_plan_act_obligations.sql`
|
||||
|
||||
```sql
|
||||
CREATE TABLE core.plan_act_reached_org_types (
|
||||
act_id UUID NOT NULL,
|
||||
org_type VARCHAR(20) NOT NULL,
|
||||
org_type_name VARCHAR(255) NOT NULL, -- snapshot at the act's commit
|
||||
last_org_id UUID, -- the greatest organization id of the type at T0; NULL for none
|
||||
reached_seq BIGINT NOT NULL GENERATED ALWAYS AS IDENTITY, -- comment: the sequence must keep CACHE 1; ownership reads it
|
||||
CONSTRAINT pk_plan_act_reached_org_types PRIMARY KEY (act_id, org_type),
|
||||
CONSTRAINT uq_plan_act_reached_org_types_seq UNIQUE (reached_seq),
|
||||
CONSTRAINT fk_plan_act_reached_org_types_act_id FOREIGN KEY (act_id) REFERENCES core.plan_act_details (act_id)
|
||||
);
|
||||
CREATE INDEX idx_plan_act_reached_org_types_owner ON core.plan_act_reached_org_types (org_type, reached_seq DESC);
|
||||
|
||||
CREATE TABLE core.plan_act_orgs_without_pool (
|
||||
act_id UUID NOT NULL,
|
||||
org_type VARCHAR(20) NOT NULL,
|
||||
org_id UUID NOT NULL, -- no FK: a record of what the listing found
|
||||
listed_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
CONSTRAINT pk_plan_act_orgs_without_pool PRIMARY KEY (act_id, org_type, org_id),
|
||||
CONSTRAINT fk_plan_act_orgs_without_pool_reach FOREIGN KEY (act_id, org_type)
|
||||
REFERENCES core.plan_act_reached_org_types (act_id, org_type)
|
||||
);
|
||||
|
||||
CREATE TABLE core.plan_act_obligations (
|
||||
act_id UUID NOT NULL,
|
||||
pool_id UUID NOT NULL, -- the organization's default pool
|
||||
org_type VARCHAR(20) NOT NULL, -- as the listing found them; order the listing and the work list
|
||||
org_id UUID NOT NULL,
|
||||
status VARCHAR(20) NOT NULL DEFAULT 'pending',
|
||||
outcome VARCHAR(20),
|
||||
superseded_by_act_id UUID,
|
||||
requeued_by_act_id UUID,
|
||||
attempts INTEGER NOT NULL DEFAULT 0,
|
||||
last_attempt_at TIMESTAMPTZ,
|
||||
last_error_step VARCHAR(30), -- what the page renders
|
||||
last_error TEXT, -- driver text: logs and the read-only role, never a page
|
||||
settled_at TIMESTAMPTZ,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
CONSTRAINT pk_plan_act_obligations PRIMARY KEY (act_id, pool_id),
|
||||
CONSTRAINT uq_plan_act_obligations_listing UNIQUE (act_id, org_type, org_id),
|
||||
CONSTRAINT fk_plan_act_obligations_reach FOREIGN KEY (act_id, org_type)
|
||||
REFERENCES core.plan_act_reached_org_types (act_id, org_type),
|
||||
CONSTRAINT fk_plan_act_obligations_pool_id FOREIGN KEY (pool_id) REFERENCES core.resource_pools (pool_id),
|
||||
CONSTRAINT fk_plan_act_obligations_superseded_by FOREIGN KEY (superseded_by_act_id) REFERENCES core.plan_act_details (act_id),
|
||||
CONSTRAINT fk_plan_act_obligations_requeued_by FOREIGN KEY (requeued_by_act_id) REFERENCES core.plan_act_details (act_id),
|
||||
CONSTRAINT chk_plan_act_obligations_status CHECK (status IN ('pending', 'settled', 'failed', 'superseded')),
|
||||
CONSTRAINT chk_plan_act_obligations_outcome CHECK (outcome IS NULL OR outcome IN (
|
||||
'initiated', 'already_at', 'grandfathered', 'migrated', 'left', 'untouched')),
|
||||
CONSTRAINT chk_plan_act_obligations_error_step CHECK (last_error_step IS NULL OR last_error_step IN (
|
||||
'in_progress', 'begin', 'lock_pool', 'read_obligation', 'read_default', 'resolve_default', 'read_positions',
|
||||
'initiate', 'grandfather', 'grandfather_outcome', 'end_incumbents', 'restore', 'materialize', 'settle', 'commit')),
|
||||
CONSTRAINT plan_act_obligation_settled_shape CHECK (
|
||||
(status = 'settled') = (settled_at IS NOT NULL) AND (status = 'settled') = (outcome IS NOT NULL)),
|
||||
CONSTRAINT plan_act_obligation_superseded_shape CHECK ((status = 'superseded') = (superseded_by_act_id IS NOT NULL)),
|
||||
CONSTRAINT plan_act_obligation_points_elsewhere CHECK (
|
||||
superseded_by_act_id IS DISTINCT FROM act_id AND requeued_by_act_id IS DISTINCT FROM act_id)
|
||||
);
|
||||
CREATE UNIQUE INDEX uq_plan_act_obligations_one_open ON core.plan_act_obligations (pool_id)
|
||||
WHERE status IN ('pending', 'failed');
|
||||
CREATE INDEX idx_plan_act_obligations_open ON core.plan_act_obligations (act_id, org_type, status, org_id)
|
||||
WHERE status IN ('pending', 'failed');
|
||||
|
||||
CREATE INDEX idx_organizations_org_type_org_id ON core.organizations (org_type, org_id);
|
||||
|
||||
ALTER TABLE core.grants ADD COLUMN act_id UUID;
|
||||
ALTER TABLE core.grants ADD CONSTRAINT fk_grants_act_id FOREIGN KEY (act_id) REFERENCES core.acts (act_id);
|
||||
|
||||
-- core.plan_act_owns_org_type(act_id, org_type) RETURNS boolean: STABLE SQL, invoker rights; true when the act's
|
||||
-- reached-types row for the type has the greatest reached_seq. The one statement of the ownership rule.
|
||||
-- core.record_plan_act: replaced; also writes the reached org types it derives from the detail,
|
||||
-- reads the ladder's rank-0 product from core.plan_ladder_tiers for a reorder and refuses a detail whose
|
||||
-- product_id disagrees, and refuses a retried act whose cause is not a default change or a reorder
|
||||
-- core.write_plan_act_obligations(act_id, batch_size) RETURNS TABLE (org_type, org_id, pool_id, superseded_act_id)
|
||||
-- (ON CONFLICT (act_id, pool_id) and ON CONFLICT (act_id, org_type, org_id), inserts ordered by org_type, org_id)
|
||||
-- core.settle_plan_act_obligation(act_id, pool_id, outcome) RETURNS boolean
|
||||
-- core.record_plan_act_obligation_failure(act_id, pool_id, step, error) RETURNS varchar (the status after the
|
||||
-- update; NULL when the row was no longer pending). The maximum attempts, 5, is a constant in the body.
|
||||
-- core.requeue_plan_act_obligations(retry_act_id) RETURNS int
|
||||
-- The five writers: SECURITY DEFINER, owned by core_owner, SET search_path = core, pg_temp.
|
||||
|
||||
GRANT ALL ON core.plan_act_reached_org_types, core.plan_act_orgs_without_pool, core.plan_act_obligations TO core_owner;
|
||||
GRANT SELECT ON core.plan_act_reached_org_types, core.plan_act_orgs_without_pool, core.plan_act_obligations
|
||||
TO core_writer, core_reader;
|
||||
REVOKE EXECUTE ON FUNCTION <the five> FROM PUBLIC;
|
||||
GRANT EXECUTE ON FUNCTION <the five> TO core_writer;
|
||||
GRANT EXECUTE ON FUNCTION core.plan_act_owns_org_type TO core_owner, core_writer, core_reader;
|
||||
```
|
||||
|
||||
Notes on the shape:
|
||||
|
||||
- The cause is a `CHECK` list, as every closed list in the repository is; a follower adds its causes by replacing `chk_acts_cause` in its own core migration.
|
||||
- No index on `core.acts` beyond its keys: nothing reads acts by cause or actor yet, and the places reach acts through the detail and reached-types indexes.
|
||||
- `core.grants.act_id` is nullable: grant issuance, signup provisioning and the reconcile write no act yet. A `NOT VALID` check would refuse the update that revokes an old grant, so none is added.
|
||||
- An obligation's and a without-pool record's foreign key to the reached-types row means a row can only exist for a type its act reaches.
|
||||
- **The read-only role.** `console_reader` withholds `core.acts.note` and reads every other new column: ids, statuses, catalog names, times, error text and the two position columns. No column holds an organization's or a person's name: an obligation reaches the organization's current name through its pool.
|
||||
|
||||
## Lock order
|
||||
|
||||
T0 takes no pool row and no obligation row. The act insert takes `FOR KEY SHARE` on the actor's `core.persons` row, which conflicts only with `FOR UPDATE` on that row, so its position in any sequence is free provided it precedes the rows that reference it.
|
||||
|
||||
| Path | Locks, in order |
|
||||
|---|---|
|
||||
| Default change, T0 | lock timeout (5 s); exclusive rendezvous; the `org_types` row (`FOR NO KEY UPDATE`); reads (candidate's rank 0); the `org_types` update (its own row, already held); `core.record_plan_act` (act, detail, reached type, with one index probe for the type's newest organization); the gate (index probes); commit |
|
||||
| Reorder, T0 | lock timeout (5 s); exclusive rendezvous; reads (tiers); the tier rows (renumbering); `core.record_plan_act` (act, detail, and, when a default ladder's rank 0 changed, the reached types it reads from `org_types`); the gate; commit |
|
||||
| Listing batch | lock timeout (5 s in the drain; in a request, `planActRequestLockTimeout` and `planActRequestTxTimeout`, both set before the rendezvous); shared rendezvous; `core.write_plan_act_obligations`: the batch's pool rows `FOR KEY SHARE` ascending; the open obligation rows of earlier acts on those pools `FOR UPDATE` ascending, then their update; the inserts in (org type, organization id) order; commit |
|
||||
| One obligation's move | lock timeout (5 s, or the request's with its transaction timeout, both set before the rendezvous); shared rendezvous; the pool row (`LockPools`); the obligation row (`FOR UPDATE`, with the owner read in the same statement, skipped unless `pending` and owned); reads (`GetPoolDefault`, positions); the bucket's work (the conferral functions take the same pool row again; a grandfather inserts a grant first); `core.settle_plan_act_obligation` (its own row, already held); commit |
|
||||
| Recording a failed move | plain transaction: `core.record_plan_act_obligation_failure` (its `UPDATE ... WHERE status = 'pending'`) and nothing else |
|
||||
| The act's last probe (D4) | lock timeout (5 s, or the request's); shared rendezvous; `core.outgoing_default_exists` (index probes); commit |
|
||||
| Retry | lock timeout (5 s); shared rendezvous; `core.record_plan_act` (act, detail); `core.requeue_plan_act_obligations` (the retried act's failed obligation rows of owned types, `FOR UPDATE` ascending, then their update); commit |
|
||||
| Tier removal | lock timeout; exclusive rendezvous; the holder count and the guard's reads; the holders' pool rows ascending (`LockPools`); the holders again; the act and detail; the tier rows (delete, renumber); per pool ascending, the conferral functions (the same pool rows again) and materialization; commit |
|
||||
| Tier add | lock timeout; exclusive rendezvous; the product read; the holder count; the provisions' pool rows ascending (`LockPools`); the provisions again; the act and detail; the tier insert; the collision read (`ListTierAddCollisions`); per pool ascending, `core.align_conferral_shape` for each of its provisions, then materialization once; commit |
|
||||
| Tier delete | lock timeout; exclusive rendezvous; reads (holder count, tiers, org types); the act and detail; the tier rows; commit |
|
||||
|
||||
Why no two of these wait on each other in a cycle:
|
||||
|
||||
- T0 holds the exclusive rendezvous and takes no pool or obligation row, so it waits only for the rendezvous and, at most, the org type's row.
|
||||
- Every path that takes a pool row and an obligation row takes the pool row first. A listing batch takes its pools `FOR KEY SHARE`, which conflicts with a move's `FOR UPDATE` on the same pool, before it touches any obligation; so a move holding a pool never waits for an obligation row a batch holds.
|
||||
- The two writers of several obligation rows, a listing batch's supersession and Retry's requeue, take them in ascending `pool_id` order, so two of them (two Retries of one act, two batches of one act) cannot deadlock. They also touch disjoint rows: a batch of act B supersedes only rows of earlier acts on types B owns, and a Retry of act A requeues only rows of A on types A owns, and both hold the shared rendezvous, so no T0 can change ownership while either runs.
|
||||
- The recorder holds one obligation row and waits on nothing else.
|
||||
- A pool row's `FOR UPDATE` takers lock one pool, or several in ascending order (rule 3 of `docs/database-locks.md`), and a listing batch takes its pools in ascending order, so pool-row waits form no cycle.
|
||||
|
||||
**For `docs/database-locks.md`:** the Row locks table gains "Plan act obligation row" (`core.plan_act_obligations` of one act and pool; `FOR UPDATE` by a move, a listing batch's supersession and Retry's requeue, `UPDATE` by the recorder; serializes two passes settling one obligation, and a settlement against a supersession or a requeue) and "Org type row" (`core.org_types` of one type; `FOR NO KEY UPDATE` by the default change's T0; serializes it against the demonstration seed and an edit of the type). The Pool row entry names the runner's move instead of `EnactDefault`, and adds the listing batch's `FOR KEY SHARE`. The opening sentence that no code takes an explicit `FOR SHARE` names the listing's explicit `FOR KEY SHARE` as the one exception. The section "Tier add, tier removal, default change and reorder" is rewritten from the table above. Rule 1 names the two T0s as exclusive holders and the listing and Retry as shared ones. Rule 3 drops "tier add does not". "Where the paths differ" loses the #194 entry and states that the plan acts' recorder takes no pool row, unlike the rule change's (#196).
|
||||
|
||||
## Flows
|
||||
|
||||
### The commit (T0)
|
||||
|
||||
1. The handler parses the form and calls `ChangeOrgTypeDefault` or `ReorderTiers` with the operator, the request id and the choices.
|
||||
2. The runner opens `BeginRuleChange`.
|
||||
3. The act reads what it changes under its locks: the org type `FOR NO KEY UPDATE` (reserved refused; the candidate already the default refused, D20; the candidate's rank 0 resolved, a new ladder without one refused), or the ladder's tiers (the submitted order reconciled against them; staleness noted).
|
||||
4. It writes the catalog change (the default, or the ranks) and records the act and its detail with the names it read; `core.record_plan_act` derives and writes the reached types.
|
||||
5. When no disposition was chosen and the act reaches a type, it runs the gate (D4) against the reached types and the detail's target; a true gate rolls back with `DispositionRequiredError`.
|
||||
6. It commits. The answer so far: the act id, the reached types, and for a reorder the stale flag and the new rank-0 name.
|
||||
|
||||
A reorder that does not change rank 0, or changes it on a ladder no type defaults to, records its act and reaches no type.
|
||||
|
||||
### One listing batch
|
||||
|
||||
`ListPlanActBatch(ctx, b, actID, kind)` opens the given transaction kind (shared; the request's or the drain's lock timeout), calls `core.write_plan_act_obligations(act_id, planActListBatch)`, commits, and then writes one Error line (act id, organization id) for each organization the batch recorded without a default pool, and one Info line for each earlier obligation it superseded (act id, organization id, the superseded act). A failed batch rolls back and is logged with the act id.
|
||||
|
||||
### Enacting one obligation
|
||||
|
||||
`EnactObligation` runs in the runner under the shared kind: lock the pool; lock the obligation with its type's owner, skip unless pending and owned by its act; read the type's current default and its rank 0 (`read_default` on an error, `resolve_default` when the default ladder has no rank-0 tier; a None default is the empty target); read the pool's class against that target (`GetPlanPoolBucket`, the rule of D4) and, for an outgoing pool, its outgoing provisions (`ListOutgoingPlanPositionsByPool`), both under step `read_positions`; then by class: already at the target or held by another source, settle `already_at` or `untouched`; plan-less, the floor restoration and `initiated` or `left`; outgoing, settle `untouched` with a Warn line when the act chose no disposition (D18), grandfather (legacy grants extending each incumbent, each conferral `created`) or migrate (end the incumbents, restore the floor, materialize). Every grant it creates carries the act id of D12, every transition its actor and the original act's reason. It settles the obligation through `core.settle_plan_act_obligation` with its outcome and commits. A failure is recorded per D13 and logged with the act id, the organization id, the pool id, the step and the error.
|
||||
|
||||
### The request
|
||||
|
||||
`DrainPlanAct(ctx, db, actID, opts)` runs D14's steps 1 to 3 with the request's kind, the request's start time, its cap and the deadline's context, and runs D4's last probe when it finds the act done; the handler then, on the request's own context, starts the drain (`startPlanActDrain`, modelled on `startEntitlementRecomputeDrain`) and renders a success toast: the outcome counts of the moves it settled, "N not yet reached" (pending obligations plus organizations not yet listed), and "N without a default pool" when the act has such records.
|
||||
|
||||
### The drain
|
||||
|
||||
`PlanActDrainWorkflow(actID)` runs `DrainPlanActActivity` passes as D14 describes; the pass that finds the act done runs D4's last probe first. `PollPlanActs` dispatches each owner act with work as a child with the abandon policy, treats an already-started child as success, sleeps the poll interval and continues as new.
|
||||
|
||||
### Retry
|
||||
|
||||
`RetryPlanAct(actID)` runs D11's transaction under the shared rendezvous, then `DrainPlanAct` within the request's budget, and the handler starts the drain and re-renders the card or the ladder page.
|
||||
|
||||
### The places
|
||||
|
||||
Each place renders, after its form on the card and below the tiers table on the ladder page, for each act D22 selects:
|
||||
|
||||
- "N organizations not yet reached." when the act has pending obligations or unlisted organizations on its owned types, with no tone.
|
||||
- "N organizations not moved." in the danger tone, with Retry beside it, when it has failed obligations; below it, each organization by name, linked to its page, with its reason, at most 50, then "And N more." when there are more, then "Reference: {act id}".
|
||||
|
||||
These are lines, as the entitlement set's failed-recompute line is, not the page-anatomy `readout` part, which is reserved for headline counts.
|
||||
|
||||
## Failure paths
|
||||
|
||||
Each table lists what is committed, what the operator sees and how the act is run again. Sentences in quotes are the ones in "Operator-facing copy". "The contention refusal" is "Another change collided with this one; try again." (copy 23).
|
||||
|
||||
### Tier add
|
||||
|
||||
| Failure path | What is committed | What the operator sees | How it is re-run |
|
||||
|---|---|---|---|
|
||||
| No session | nothing | "Unauthorized" | sign in |
|
||||
| Product not offered (form) | nothing | the existing field error | correct and resubmit |
|
||||
| Begin: lock wait past 5 s, or deadlock | nothing | the contention refusal | resubmit |
|
||||
| Begin: other error | nothing | "Failed to add tier. Reference: {request id}."; logged | resubmit |
|
||||
| The product read finds no row | nothing | "Product not found." under the product field | choose another |
|
||||
| The product read fails | nothing | "Failed to add tier. Reference: {request id}."; logged | resubmit |
|
||||
| The product is not published | nothing | the existing "Only published products can be tiers. ..." | publish it first |
|
||||
| Holder count read fails | nothing | "Failed to add tier. Reference: {request id}."; logged | resubmit |
|
||||
| Holders above the limit | nothing | field error: the add's limit sentence, plus its second sentence when most holders are default-sourced | move the default-sourced holders first; otherwise #237 |
|
||||
| Pool locks: wait past 5 s or deadlock | nothing | the contention refusal | resubmit |
|
||||
| Pool locks or second read: other error | nothing | "Failed to add tier. Reference: {request id}."; logged | resubmit |
|
||||
| Recording the act fails | nothing | "Failed to add tier. Reference: {request id}."; logged | resubmit |
|
||||
| Tier insert: product already a tier | nothing | "This product is already a tier in this ladder." | n/a |
|
||||
| Tier insert: rank collision | nothing | the contention refusal, as a field error (unreachable between operators under the exclusive lock) | resubmit |
|
||||
| The collision read fails | nothing | "Failed to add tier. Reference: {request id}."; logged | resubmit |
|
||||
| A pool already holds a live position on the ladder | nothing | "Cannot add this tier; {Organization} already holds {Product} on this ladder.", with the names read inside the transaction | after the colliding position changes |
|
||||
| A pool holds more than one live provision of the product | nothing | "Cannot add this tier; {Organization} holds {Product} more than once, and an organization holds one position per ladder." | after one of those provisions ends |
|
||||
| Alignment: CF003 that the collision read did not find (a defect) | nothing | "Failed to add tier. Reference: {request id}."; the driver text is logged | resubmit; the log line leads to the defect |
|
||||
| Alignment, materialization or commit: lock wait or deadlock | nothing | the contention refusal (#216) | resubmit |
|
||||
| Alignment, materialization or commit: other error | nothing | "Failed to add tier. Reference: {request id}."; logged | resubmit |
|
||||
| Commit reached the server, acknowledgement lost | everything | "Failed to add tier. Reference: {request id}." | a resubmit answers "already a tier"; the tier is on the page |
|
||||
|
||||
### Tier delete (no holders)
|
||||
|
||||
| Failure path | What is committed | What the operator sees | How it is re-run |
|
||||
|---|---|---|---|
|
||||
| Begin: lock wait past 5 s, or deadlock | nothing | the contention refusal | resubmit |
|
||||
| Begin: other error | nothing | "Failed to remove tier. Reference: {request id}."; logged | resubmit |
|
||||
| Holder count read fails | nothing | "Failed to check active positions" (unchanged); logged | resubmit |
|
||||
| Live holders (a stale page) | nothing | the existing "This tier has N live holder(s); ..." | use Preview removal |
|
||||
| Tier list read fails | nothing | "Failed to check the ladder's tiers" (unchanged); logged | resubmit |
|
||||
| Org types read fails | nothing (the guard fails closed) | "Failed to remove tier. Reference: {request id}."; logged | resubmit |
|
||||
| Last tier of a ladder an org type defaults to | nothing | the existing last-tier refusal | add another tier first |
|
||||
| Act, delete, renumber or commit: lock wait or deadlock | nothing | the contention refusal | resubmit |
|
||||
| Act, delete, renumber or commit: other error | nothing | "Failed to remove tier. Reference: {request id}."; logged | resubmit |
|
||||
|
||||
### Tier removal (holders)
|
||||
|
||||
| Failure path | What is committed | What the operator sees | How it is re-run |
|
||||
|---|---|---|---|
|
||||
| The preview's org types read, for the guard, fails | nothing | "Failed to preview the removal. Reference: {request id}."; logged with the request id | reload |
|
||||
| Disposition outside keep and migrate | nothing | 422 preview with the field error | choose |
|
||||
| Begin: lock wait past 5 s, or deadlock | nothing | the contention refusal | resubmit |
|
||||
| Begin: other error | nothing | "Failed to remove the tier; nothing was changed."; logged | resubmit |
|
||||
| Holder count, product name or ladder name read fails | nothing | "Failed to remove the tier; nothing was changed."; logged | resubmit |
|
||||
| Holders above the limit | nothing | the removal's limit sentence, plus its second sentence when most are default-sourced; the preview already shows it with "Apply change" disabled | move the default-sourced holders first; otherwise #237 |
|
||||
| Last tier of a ladder an org type defaults to, either disposition | nothing | the existing last-tier refusal; the preview shows it with "Apply change" disabled | add another tier first |
|
||||
| Holder read, pool locks, second read | nothing | the contention refusal or "Failed to remove the tier; nothing was changed."; now logged | resubmit |
|
||||
| Default-sourced holders, no disposition | nothing | 422: "Choose whether to keep or migrate the N organization(s) ..." | choose |
|
||||
| Recording the act, delete, renumber | nothing | "Failed to remove the tier; nothing was changed."; logged | resubmit |
|
||||
| A pool's end, alignment, restoration or materialization | nothing | "Removal failed and nothing was changed: could not ... Reference: {request id}." for the step (copies 53 to 56), or the contention refusal | once the cause is fixed; one broken organization blocks the whole removal |
|
||||
| Commit | nothing | the contention refusal or "Failed to remove the tier; nothing was changed."; logged | resubmit |
|
||||
| The holders left between the preview and the commit | the deletion and its act | "{Product} removed from the ladder." | n/a |
|
||||
|
||||
### Default change, T0
|
||||
|
||||
| Failure path | What is committed | What the operator sees | How it is re-run |
|
||||
|---|---|---|---|
|
||||
| No session; a reserved type | nothing | the existing page error | n/a |
|
||||
| Form refusal, the current default included | nothing | "Choose a new default from the list." (the select's label becomes "New default"; see the copy) | choose |
|
||||
| Begin: lock wait past 5 s, or deadlock | nothing | card: the contention refusal | resubmit |
|
||||
| T0's two waits pass the request's deadline | nothing | card: the contention refusal, rendered on the request's own context with the card's lines | resubmit |
|
||||
| Begin: other error | nothing | card: "Failed to update the default plan. Reference: {request id}."; logged | resubmit |
|
||||
| The type's row, the candidate's rank 0, the act's record or the gate cannot be read or written | nothing | the same card error, or the contention refusal; logged | resubmit |
|
||||
| The candidate ladder has no rank-0 tier | nothing | the existing "That ladder has no rank-0 tier, ..." | add a tier to it |
|
||||
| The candidate is already the default (a concurrent commit landed first) | nothing | card: "Already the default." | n/a |
|
||||
| Outgoing-default organizations, no disposition | nothing | the card's preview with "N organization(s) hold the outgoing default. Choose ..." | choose |
|
||||
| The gate was true and the rebuilt preview finds nothing outgoing (a move settled the last one between the rollback and the rebuild) | nothing | "Failed to update the default plan. Reference: {request id}."; an Error line with the reached types and the target | resubmit |
|
||||
| The 422 preview cannot be rebuilt after the rollback | nothing | "Failed to classify existing organizations. Reference: {request id}." | resubmit |
|
||||
| Commit | nothing | the contention refusal or the generic card error; logged | resubmit |
|
||||
| Commit reached the server, acknowledgement lost | the default, the act, its detail and reach | the generic card error, on a card rendered from the type read before T0, so it shows the old default | nothing to resubmit; reload shows the new default and, while work is open, its lines; the poller starts the drain |
|
||||
| After the commit | see "The listing" and "One obligation's move" | | |
|
||||
|
||||
### Reorder, T0
|
||||
|
||||
| Failure path | What is committed | What the operator sees | How it is re-run |
|
||||
|---|---|---|---|
|
||||
| No session | nothing | "Unauthorized" | sign in |
|
||||
| Empty or wholly stale submission | nothing | the page re-rendered | drop again |
|
||||
| Begin: lock wait past 5 s, or deadlock | nothing | the contention refusal | resubmit |
|
||||
| Begin: other error | nothing | "Failed to reorder tiers."; logged | resubmit |
|
||||
| Tiers cannot be read | nothing (fails closed) | "Failed to reorder tiers."; logged | resubmit |
|
||||
| Renumbering collides (the page was out of date) | nothing | "This page was out of date; showing the current tier order. Try again.", or the contention refusal on a lock wait | drop again |
|
||||
| Recording the act fails, including its read of the org types | nothing | "Failed to reorder tiers." or the contention refusal; logged | resubmit |
|
||||
| Outgoing-default organizations, no disposition | nothing | 422 preview with "Choose whether to grandfather or migrate the N organization(s) on the outgoing default." | choose |
|
||||
| The gate was true and the rebuilt preview finds nothing outgoing | nothing | "Failed to reorder tiers."; an Error line with the reached types and the target | resubmit |
|
||||
| Commit reached the server, acknowledgement lost | the ranks, the act, its detail and reach | "Failed to reorder tiers." on a page rendered from the order before T0 | reload shows the new order and, while work is open, the act's lines; the poller starts the drain |
|
||||
| The 422 preview cannot be rebuilt | nothing | "Failed to classify affected organizations. Reference: {request id}." | resubmit |
|
||||
| Commit | nothing | "Failed to reorder tiers." or the contention refusal; logged | resubmit |
|
||||
| After the commit | see "The listing" and "One obligation's move" | | |
|
||||
|
||||
### The listing (default change and reorder)
|
||||
|
||||
| Failure path | What is committed | What the operator sees | How it is re-run |
|
||||
|---|---|---|---|
|
||||
| In the request, the request's budget has passed after T0 | nothing listed | the toast counts every organization of the reached types as not yet reached; the place shows "N organizations not yet reached." | the drain lists them |
|
||||
| In the request, the batch waits past the request's lock timeout, or the context ends | nothing from this batch; no attempt anywhere | the same | the drain |
|
||||
| A batch fails with any other error, in the request or the drain | nothing from this batch | the same; one Error line with the act id | the drain's next pass; a pass that lists and settles nothing sleeps until the next poll |
|
||||
| A batch fails on every pass | nothing | "N organizations not yet reached." stops falling; one Error line per pass with the act id | a defect; no Retry applies, because no obligation exists yet; the act id in the log leads to it |
|
||||
| An organization of a reached type has no active default pool | a record of it, no obligation | "N without a default pool" in the toast, counting the organizations whose record the act holds when the request answers; nothing on the place | n/a; one Error line (act id, organization id) |
|
||||
| An organization was created after T0 | nothing (outside the bound), or an obligation the move settles as already at the default | nothing | n/a |
|
||||
| A newer act now owns a type | nothing for that type; the newer act's batches supersede this act's open rows there | this act stops counting that type; the newer one counts it | n/a |
|
||||
| Two batches of one act run at once (the request's and a drain the poller started) | each organization once | nothing | n/a |
|
||||
| The process dies mid-batch | nothing from this batch | "not yet reached" | the poller |
|
||||
| After T0, the request's read of whether the listing is done, or of the pending obligations, fails | nothing more | the toast is not rendered as a finished answer: the request stops settling, starts the drain, and answers the commit's success toast without counts, followed by the places' lines or their read-failure sentence | the drain |
|
||||
| After T0, the request's read of the counts for the toast fails | nothing more | the success toast without "not yet reached" and "without a default pool" parts, and the place's own lines (copy 13 when they cannot be read) | reload |
|
||||
|
||||
### One obligation's move (default change, reorder, Retry)
|
||||
|
||||
| Failure path | What is committed | What the operator sees | How it is re-run |
|
||||
|---|---|---|---|
|
||||
| A move fails in the request or the drain | this obligation pending, attempts plus one, its step | the toast counts it "not yet reached"; the place shows "N organizations not yet reached." | the drain retries it |
|
||||
| The same move fails the maximum number of times | the obligation `failed` | "N organizations not moved." with Retry; the organization's name and the sentence for its step; "Reference: {act id}" | Retry, once the cause is fixed |
|
||||
| In the drain, a lock wait passes 5 s, or a deadlock | an attempt with step `in_progress` | "The last attempt collided with another change." when it dead-letters | the drain retries; Retry |
|
||||
| In the request, a lock wait passes the request's lock timeout | nothing; no attempt; the request stops settling | "not yet reached" | the drain, with the full timeout |
|
||||
| In the request, a transaction passes the request's transaction timeout | nothing; no attempt; the request stops settling. The database ends the session (FATAL 25P04, one line in the server log) and the pool replaces the connection; the request sees 25P04 or a closed connection | "not yet reached" | the drain, with no transaction timeout |
|
||||
| A context error (client gone, deadline, worker stopping) | nothing; no attempt | "not yet reached" | the drain |
|
||||
| The type's default or its rank 0 cannot be read | an attempt with `read_default` | "The move could not be completed." when it dead-letters | the drain; Retry |
|
||||
| The type's default ladder has no rank-0 tier (damaged data only) | an attempt with `resolve_default` | "Its org type's default ladder has no rank-0 tier." | add a tier, then Retry |
|
||||
| An outgoing-default position under an act with no disposition | the obligation settled `untouched`; one Warn line (act id, organization id) | nothing | n/a: a newer default change or reorder with a disposition moves it |
|
||||
| Recording the attempt fails | nothing; the obligation stays pending, its attempt uncounted | both errors in the logs, with the act id and the organization id | the next pass, which starts after this obligation, tries the ones behind it; this one is tried again after the pass wraps |
|
||||
| The recorder finds the obligation no longer pending | nothing (the function returns NULL) | nothing | n/a: another pass settled it or a newer act superseded it |
|
||||
| The request's cap or budget is reached | the rest stay pending | "N not yet reached" in the toast and the place | the drain; no action |
|
||||
| The drain's start fails, or no Temporal client is configured | pending obligations | "not yet reached" persists | the poller within 5 s; with no worker running, when one runs |
|
||||
| The poller's read fails after its retries (a database restart) | nothing | "not yet reached" persists | the poller logs the failure and tries again at its next tick |
|
||||
| The act's last probe (D4) fails, or meets an older snapshot | nothing | nothing | n/a: a failure is logged with the act id; the next commit's gate probe walks the dead entries once, within task 1's bound |
|
||||
| The process dies during the request | committed obligations; the move in flight rolled back | "not yet reached" | the poller |
|
||||
| The obligation is no longer pending when its row is locked | nothing | nothing | n/a: another pass settled it, or a newer act superseded it |
|
||||
| A newer act owns the pool's type | nothing | the older act stops counting it; the newer one counts it | n/a: the newer act's listing supersedes the row |
|
||||
| A tier change or default change commits between the act and the move | nothing extra | nothing | n/a: the move reads the current default and rank 0 under the shared lock |
|
||||
|
||||
### Retry
|
||||
|
||||
| Failure path | What is committed | What the operator sees | How it is re-run |
|
||||
|---|---|---|---|
|
||||
| No session | nothing | "Unauthorized" | sign in |
|
||||
| The act does not belong to the place, or nothing failed on its owned types (a second press, or a newer act took over) | nothing | the place re-rendered with its current lines | n/a |
|
||||
| Begin, the record or the requeue: lock wait past 5 s, or deadlock | nothing | the contention refusal | press Retry again |
|
||||
| The act's place cannot be read, or begin, the record or the requeue fails with any other error | nothing | "Failed to retry. Reference: {request id}."; logged with the request id | press Retry again |
|
||||
| Moves after the requeue | as "One obligation's move" | | |
|
||||
| The drain's start fails | the requeued obligations, pending | "not yet reached" | the poller |
|
||||
|
||||
### The places (org type card, ladder page)
|
||||
|
||||
| Failure path | What is committed | What the operator sees | How it is re-run |
|
||||
|---|---|---|---|
|
||||
| The open acts or their unmoved organizations cannot be read | nothing | "Pending plan changes could not be read. Reference: {request id}." in place of the lines; logged with the request id | reload |
|
||||
| The org types that default to a ladder cannot be read (the tier card, the reorder preview) | nothing | "Failed to read the org types that default to this ladder. Reference: {request id}." in place of the badges and of the preview's consequence panel; logged with the request id | reload |
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- **The gate depends on two copied columns.** `org_type` and `default_sourced` on `core.pool_provision_ladders` copy a pool's organization, an organization's type, a provision's grant and a grant's reason. Guard triggers in `00026` refuse any update that changes one of those four values, so the copies cannot go stale through the database; a feature that must change one of them has to replace the guard and decide what the copies and the obligations do. The agreement test and the trigger's own test cover the copies and the guard in their fixtures.
|
||||
- **The gate's probes read dead index entries after a mass ending.** Positions at a type's lowest or highest product that ended since the last vacuum make the next probe walk their entries, about 20 ms per 100,000 measured, and every probe repeats the walk while a snapshot older than the endings stays open (D4). The act that ended them runs the probe once under the shared lock when its moves are done, so the walk normally lands outside the exclusive hold; autovacuum's threshold bounds what can land inside it, and task 1's stop rule bounds that case. The threshold is a fifth of the position table, which keeps every ended position, so it grows with the table's history as well as with the membership: a table of a million positions lets about 200,000 dead entries gather, about 40 ms at the measured rate.
|
||||
- **The place counts organizations the act has not reached, not only organizations it will move.** An organization settles `untouched`, `already_at` or `left` as often as it moves anything, and an organization without a default pool is counted until a batch reaches it, so "N organizations not yet reached." can fall to nothing with fewer moves than N. A member's plan is unchanged until their organization's move commits, as today.
|
||||
- **A listing batch that fails on every pass has no dead letter.** No obligation exists to mark failed, so the place keeps saying "not yet reached". The Error line on every pass names the act; a deterministic batch failure is a defect, not an operator's problem.
|
||||
- **The holder limit makes some removals and adds impossible** until #237: a tier that many subscribers hold cannot be removed, and moving the default first clears only the default-sourced holders. The refusal names the count, the limit and, where it applies, the way through.
|
||||
- **An act with no disposition leaves a later outgoing position in place.** The Warn line names it; the place does not list it, because nothing failed.
|
||||
- **A later Retry overwrites `requeued_by_act_id`, and a Retry erases the failure history of the rows it requeues.** Each Retry's act row and the act it retried stay, but which rows an earlier Retry requeued is not kept once a later one requeues them again. A Retry resets attempts and clears the last error, so a later per-act page cannot show that an organization failed five times before a Retry moved it; the logs keep each attempt's line with the act id.
|
||||
- **Transitions name their act only through reason text** until #179. The record links grants and obligations to the act now; transitions follow when their writers are replaced.
|
||||
- **More rows per act.** An act on a type of N organizations writes N obligation rows, kept forever, and the obligations' foreign key to pools and the grants' foreign key to acts keep a pool, and so its organization, from being deleted while the record keeps them. Nothing deletes either today.
|
||||
- **55 golden files change twice.** Once when the act sections land (task 4) and once when the obligation sections and the grant column land (task 6); each other change is a behaviour the task names.
|
||||
|
||||
## Migration plan
|
||||
|
||||
1. `00026_outgoing_default_rule.sql`, `00027_acts.sql` and `00028_plan_act_obligations.sql` in the core stream, each with a Down section that drops what it added in reverse order. 00026 backfills the two position columns and adds the five guard triggers on the source columns; no other backfill: rows written before the migrations have no act.
|
||||
2. `scripts/db-reader-role.sql` gains `'^core\.acts\.note$'`; a deployment re-runs the script after migrating, as for every migration.
|
||||
3. Rollback: each Down migration drops its tables, columns and functions. A rollback of the code alone is safe while the migrations stay, because no other writer reads the new tables, and the position columns are filled by the trigger whatever the code does.
|
||||
4. The deploy that ships this starts the plan-act poller at boot, next to the recompute poller.
|
||||
|
||||
## Operator-facing copy
|
||||
|
||||
Every sentence a person reads that this change adds, changes, reuses or removes. Counts render with thousands separators. "N organizations" takes its singular form, "1 organization", in every line below. The last column marks what is new or changed since the previous draft; an empty cell means the entry is as it was. The wording of every entry was approved.
|
||||
|
||||
### Added
|
||||
|
||||
| # | Sentence | Where | Since the previous draft |
|
||||
|---|---|---|---|
|
||||
| 1 | "N organizations not yet reached." | the org type's card, after the form; the ladder's page, below the tiers table; per act, while it has pending obligations or organizations not yet listed. An organization whose move failed is counted here until its attempts reach the maximum, and only then listed as not moved | "Where" states that a failed move is counted here until it is failed |
|
||||
| 2 | "N organizations not moved." | the same places, per act, danger tone, when it has failed obligations, with the Retry button (existing label "Retry") beside it | |
|
||||
| 3 | "And N more." | below an act's list of unmoved organizations, when more than 50 | |
|
||||
| 4 | "Reference: {act id}" | the last line of an act's list of unmoved organizations | |
|
||||
| 5 | "The last attempt collided with another change." | an unmoved organization's reason (`in_progress`) | |
|
||||
| 6 | "Its org type's default ladder has no rank-0 tier." | an unmoved organization's reason (`resolve_default`) | |
|
||||
| 7 | "The default plan could not be conferred." | an unmoved organization's reason (`initiate`, `restore`) | |
|
||||
| 8 | "The legacy grant could not be issued." | an unmoved organization's reason (`grandfather`, `grandfather_outcome`) | |
|
||||
| 9 | "The outgoing default could not be ended." | an unmoved organization's reason (`end_incumbents`) | |
|
||||
| 10 | "Its entitlements could not be recomputed." | an unmoved organization's reason (`materialize`) | |
|
||||
| 11 | "The move could not be completed." | an unmoved organization's reason (`begin`, `lock_pool`, `read_obligation`, `read_default`, `read_positions`, `settle`, `commit`) | |
|
||||
| 12 | "Failed to retry. Reference: {request id}." | the card or the ladder page, as an error, when Retry fails for a reason other than lock contention | |
|
||||
| 13 | "Pending plan changes could not be read. Reference: {request id}." | the card or the ladder page, in place of the lines, when they cannot be read | narrowed: a failed read of the toast's counts leaves those counts out of the toast and does not show this sentence |
|
||||
| 14 | "Failed to preview the removal. Reference: {request id}." | the tier card, in place of the removal preview, when the org types its last-tier guard needs cannot be read | |
|
||||
| 15 | "N organizations hold this tier; a removal is refused above C." | the removal's commit refusal; the removal preview's disabled "Apply change" reason | |
|
||||
| 16 | "M of them hold it through their org type's default; changing that default with Migrate moves them in the background." When the tier is rank 0 of a ladder their type defaults to: "M of them hold it through their org type's default; changing that default, or reordering this ladder, with Migrate moves them in the background." | after 15, when more than half of the holders are default-sourced | |
|
||||
| 17 | "N organizations hold this product; adding it as a tier is refused above C." | the tier-add form's product field error | |
|
||||
| 18 | "M of them hold it through their org type's default; changing that default with Migrate moves them in the background." | after 17, when more than half of the holders hold the product through a default-sourced position on a ladder; M counts only those holders, not default-sourced provisions a removal left off every ladder | |
|
||||
| 19 | summary part "N not yet reached" | the default change's toast ("Default change committed for {type}: ...") and the reorder's ("{Product} is now the default plan for new {types} organizations: ..."); counts pending obligations and organizations not yet listed, including an organization whose move the request has just tried and failed | "Where" states that it counts an organization the request tried and failed |
|
||||
| 20 | summary part "N without a default pool" | the same toasts; counts the organizations whose record the act holds when the request answers, an organization that also holds an obligation of the act counting once, as an obligation | "Where" reworded to the specs' count |
|
||||
| 43 | "Cannot add this tier; {Organization} already holds {Product} on this ladder." | the tier-add form's product field error when a pool among the product's holders already holds a live position on the ladder; the names are read under the add's pool locks, before any alignment | found before the alignment, inside the add's transaction, instead of read after the rollback |
|
||||
| 44 | withdrawn | was the fallback for a colliding position that could not be read after the rollback | withdrawn: the collision is read inside the add's transaction, so its names are always read with it |
|
||||
| 45 | "Failed to read the org types that default to this ladder. Reference: {request id}." | the tier card in place of the default-ladder badges, and the reorder preview in place of its consequence panel, when `ListOrgTypes` fails; the logged line carries the request id | |
|
||||
| 48 | "Cannot add this tier; {Organization} holds {Product} more than once, and an organization holds one position per ladder." | the tier-add form's product field error when a pool holds more than one live provision of the product being added, so the add would have to place both on the ladder | new |
|
||||
|
||||
### Reused in a new place
|
||||
|
||||
| # | Sentence | New place | Since the previous draft |
|
||||
|---|---|---|---|
|
||||
| 21 | "This ladder's only remaining tier; {types} defaults new signups to it. Deleting it would leave those signups without a starting plan, so the removal is refused." | the removal's commit refusal, and the removal preview's disabled "Apply change" reason, under either disposition | |
|
||||
| 22 | "Already the default." | the default change's card error when a concurrent commit saved the same default first | |
|
||||
| 23 | "Another change collided with this one; try again." | tier add, tier delete, tier removal, the reorder, the default change's card and Retry, and the rule change's Apply changes, on a lock wait past the timeout or a deadlock (#216); today the tier add's field error for a rank collision | |
|
||||
| 24 | withdrawn | was the tier add's "Failed to add tier. Reference: {request id}." on its new failure paths | withdrawn: the tier add answers copy 50 on every failure path |
|
||||
| 46 | "And N more." | the default change's and the reorder's preview lists of outgoing-default, other-source and would-fail organizations, their radio re-post and the 422 rebuild, when a list has more than 50 names | extended to the would-fail list |
|
||||
|
||||
### Changed (old → new)
|
||||
|
||||
| # | Old | New | Where | Since the previous draft |
|
||||
|---|---|---|---|---|
|
||||
| 25 | "Another entitlement set change is in progress. Try again." | "Another change collided with this one; try again." | the default change's card and the tier removal, on a lock wait or deadlock; the tier add, tier delete and reorder answered "see server logs" there before (#216); and the rule change's Apply changes (`MessageInProgress`) | |
|
||||
| 26 | "{Product} removed from the ladder: ." | "{Product} removed from the ladder." | the removal's success toast when no holder was left to reconcile | |
|
||||
| 27 | grant description "Grandfathered on org-type default change: {reason}" | "Grandfathered: {reason}" | the description of a legacy grant a grandfather disposition issues, shown in the organization's grants ledger | |
|
||||
| 28 | stored reason "tier added to ladder {ladder id}" | "tier added to ladder {ladder name}" | the transition rows of a tier add; the Tier changes list still shows "Tier added to the ladder" | |
|
||||
| 29 | summary part "N failed" | removed (failures are "not yet reached", then listed as not moved) | both toasts | |
|
||||
| 47 | select label "Change default to"; its help popover's title "Change default to" and help button label "Help: Change default to"; the form library's refusal "Choose a change default to from the list." | label "New default"; popover title "New default"; help button label "Help: New default"; refusal "Choose a new default from the list." | the org type card's default select | the popover's title and the help button's label are added; the help text itself is unchanged |
|
||||
| 49 | disclosure label "show all {N}" | "show all {N}" while the list holds every name; "show 50 of {N}" when the cap of 50 shortens it; N is the class's exact count | the default change's and the reorder's preview lists of outgoing-default and other-source organizations | new: with the cap, "show all {N}" would be false |
|
||||
| 50 | "Failed to add tier; see server logs." | "Failed to add tier. Reference: {request id}." | the tier add on every failure path that is not a refusal: begin, the product read (other than no rows; today "Product not found."), the holder count, the pool locks, the act, the collision read, a collision the collision read did not find, the alignment, materialization and commit | approved |
|
||||
| 51 | "Failed to remove tier; see server logs." | "Failed to remove tier. Reference: {request id}." | the tier delete, on every failure path that is not a refusal, including the read of the org types | approved |
|
||||
| 52 | "Failed to load the tier's holders. Details are in the server logs." | "Failed to load the tier's holders. Reference: {request id}." | the removal preview, in place of the preview, when the holders cannot be listed | approved |
|
||||
| 53 | "Removal failed and nothing was changed: could not end a default-sourced position. Details are in the server logs." | "Removal failed and nothing was changed: could not end a default-sourced position. Reference: {request id}." | the removal, when ending a default-sourced position fails | approved |
|
||||
| 54 | "Removal failed and nothing was changed: could not reconcile a holder's position. Details are in the server logs." | "Removal failed and nothing was changed: could not reconcile a holder's position. Reference: {request id}." | the removal, when aligning a holder fails | approved |
|
||||
| 55 | "Removal failed and nothing was changed: could not apply the current default. Details are in the server logs." | "Removal failed and nothing was changed: could not apply the current default. Reference: {request id}." | the removal, when restoring the current default fails | approved |
|
||||
| 56 | "Removal failed and nothing was changed: could not materialize entitlements. Details are in the server logs." | "Removal failed and nothing was changed: could not materialize entitlements. Reference: {request id}." | the removal, when materializing fails | approved |
|
||||
| 57 | "Failed to classify affected organizations. Details are in the server logs." | "Failed to classify affected organizations. Reference: {request id}." | the reorder's preview and its 422 rebuild, when the classification cannot be read | approved |
|
||||
| 58 | "Failed to update the default plan. Details are in the server logs." | "Failed to update the default plan. Reference: {request id}." | the default change's card, on every failure path that is not a refusal, including a gate that was true while the rebuilt preview found nothing outgoing | approved |
|
||||
| 59 | "Failed to classify existing organizations. Details are in the server logs." | "Failed to classify existing organizations. Reference: {request id}." | the default change's preview and its 422 rebuild, when the classification cannot be read | approved |
|
||||
| 60 | "Failed to verify the plan ladder. Details are in the server logs." | "Failed to verify the plan ladder. Reference: {request id}." | the default change's preview and commit, when the candidate's rank-0 read fails | approved |
|
||||
| 61 | "Failed to load the organization type. Details are in the server logs." | "Failed to load the organization type. Reference: {request id}." | the default change's preview and commit, when the type cannot be read | approved |
|
||||
|
||||
### Removed
|
||||
|
||||
| # | Sentence | Where it was | Since the previous draft |
|
||||
|---|---|---|---|
|
||||
| 30 | "Default change committed: {summary}." | the card's failure alert | |
|
||||
| 31 | "Failed organizations were skipped; fix the cause and commit again." | the card's failure alert | |
|
||||
| 32 | " Failed organizations were skipped; fix the cause, re-drop the same order, and commit again: {org}: {reason}; ..." | the reorder's error toast | |
|
||||
| 33 | "Tier order saved, but affected organizations could not be listed for enactment. Re-drop the same order and commit again." | the reorder's error toast | |
|
||||
| 34 | "Default saved, but existing organizations could not be listed for enactment. Re-run the commit." | the card's error | |
|
||||
| 35 | "no default pool" | an enactment failure's reason (the preview's "would fail" reason is unchanged) | |
|
||||
| 36 | "could not begin transaction", "could not lock the pool", "could not read the pool's plan position", "commit failed" | enactment failure reasons, replaced by 11 | |
|
||||
| 37 | "could not initiate the default: {error}", "could not apply the new default: {error}" | replaced by 7 | |
|
||||
| 38 | "could not grandfather: {error}", "grandfather conferral reported {outcome}, expected the lineage supersession" | replaced by 8 | |
|
||||
| 39 | "could not end the outgoing default position: {error}" | replaced by 9 | |
|
||||
| 40 | "could not materialize entitlements: {error}" | replaced by 10 | |
|
||||
|
||||
The default change's and the reorder's toasts become success toasts whenever the commit landed: what is still owed is on the card or the ladder page, not in the toast. Every other sentence on these surfaces is unchanged. The sentences of 50 to 61 are converted here because these handlers are rewritten (proposal, ride-alongs); #239 keeps the console's other "Details are in the server logs" sentences.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Sentences 1 and 19 count an organization whose move failed as not yet reached until its attempts reach the maximum, so a first failure shows only in the logs and on the obligation row until the obligation is failed and listed as not moved.
|
||||
- The gate costs two columns, two triggers and an index on `core.pool_provision_ladders`, and a guard on five columns of four tables, and its probe walks the dead entries of a mass ending once (D4). The alternatives (a linear gate under the exclusive hold, a counter row every signup updates, or sharded counters kept beside the positions) are rejected in D4; the cost needs approval with the copy.
|
||||
- "Reference: {act id}" shows only with an act's unmoved list. A listing that fails on every pass shows only "not yet reached", with no reference on the page; showing the reference with "not yet reached" too would cover that defect at the price of a line on every act in progress.
|
||||
@@ -0,0 +1,82 @@
|
||||
## Why
|
||||
|
||||
Two of the five plan-wide acts can leave organizations behind with no way to move them. An org type's default change saves the new default first and then moves each organization in its own transaction; a ladder reorder renumbers first and then does the same. When one organization's move fails, the act tells the operator to re-run it, and neither re-run exists: the default-change form refuses the current default, and dropping the committed order again finds rank 0 unchanged and moves nobody. The failure is recorded only in the answer to that request, so it is lost as soon as the page is reloaded. Both acts also fix the rank-0 product they move organizations onto before a concurrent tier change can renumber the ladder, so an organization can be grandfathered onto a legacy grant of a product that is no longer rank 0 (`TestReorderEnactsAgainstTheRankZeroItSaved` pins what that records today). A large org type makes it worse: the request runs past its 8 s write timeout and its 32 s deadline, and every organization after the deadline is reported as having no default pool. The personal org type has one organization per member, so the act has to work at the size of the membership.
|
||||
|
||||
The other three acts (tier add, tier delete, tier removal) run in one transaction each, but tier add locks pools in creation order (#194), tier delete reads its guards outside its transaction, and the removal can empty the ladder an org type defaults to. None of the five records that it happened, so nothing can list what an act still owes.
|
||||
|
||||
## What Changes
|
||||
|
||||
- **An act record.** One new table, `core.acts`, holds one row per act: a cause from a closed list (a `CHECK`, not an enum), the actor as the audit triple (person, service account or system), a nullable note, the request id for log correlation only, and the act's time. A row is never updated or deleted. What an act did, by kind, lives in a side table: `core.plan_act_details` holds the ladder, product and org type the act changed, by id with their names as they read when the act committed, and no foreign key to the catalog. Every history or decree row an act writes points at its act: the detail row, the org types it reaches, its records of organizations without a default pool, the obligations, and the grants the act creates (`core.grants.act_id`, nullable). This change is the first adopter of the convention; later acts add their own causes and detail tables. Neither read-only role reads the note.
|
||||
- **A short commit at org-type grain.** A default change and a reorder that changes a default ladder's rank 0 commit in one transaction under the exclusive materialization lock that writes only the catalog change, the act row, its detail row and one row per org type the act reaches, and runs the disposition gate. Every statement in it is a single-row write or an index probe. The gate's probe is an index probe whose first read after a mass ending walks the entries vacuum has not yet removed, about 20 ms per 100,000 measured, after which those entries are marked dead and the probe is back to its usual cost. The act whose moves ended the positions runs that probe once under the shared lock when its moves are done, so the walk is normally paid outside the exclusive hold; autovacuum's threshold bounds what is left. No other statement's cost grows with the number of organizations.
|
||||
- **One rule for an outgoing default and for a pool's class.** Whether a position is outgoing is decided by one database function, and which class a pool falls in (would initiate, already at the target, outgoing default, other source, would fail) by another, built on the first. The preview applies them to every pool of the type in one statement that returns each class's exact count and at most 50 names per class, so its cost on the application side does not grow with the type; a move applies them to its one pool. The gate applies the position rule to two index entries per reached type, through two columns the position table gains (the organization's type and whether a default grant sources the position, both copied by trigger) and a partial index, so it cannot disagree with the preview or the move on a pool that holds both an at-target and an outgoing position, or on a None target. Guard triggers refuse any update of the four values the copies rest on (an organization's type, a pool's organization, a provision's grant and pool, a grant's reason), so the copies cannot go stale.
|
||||
- **A listing after the commit.** Short batches, each its own transaction under the shared lock, write one obligation per organization with an active default pool (`core.plan_act_obligations`), in organization order, starting after the rows the act has already written, so a crash between batches loses nothing. Organizations created after the commit are outside the listing's bound and get the new default at signup. Organizations without a default pool are recorded, logged and counted, not owed.
|
||||
- **Enactment against the current default.** Each obligation is settled in its own short transaction under the shared lock: the pool row, then the obligation row, skipped unless still pending and still owned by its act, then the org type's current default and its current rank 0, resolved inside that transaction. A tier change that commits between the act and an organization's move can no longer make the move classify against a stale product.
|
||||
- **A bounded request, then a drain.** The request runs the commit, one listing batch and as many moves as a measured count and time budget allow, with a shorter lock timeout, a transaction timeout and one deadline for the act's work, and answers, on its own context, well inside its write timeout. The act's preview, its radio re-post and its 422 rebuild name at most 50 organizations per list, then "And N more.", with every count exact. A Temporal drain workflow, started after the commit and backed by a poller, continues the listing and the moves.
|
||||
- **Failures recorded and retried.** A failed move rolls back, and a plain transaction records the attempt, the failing step and the error on the obligation row alone. After the maximum attempts the obligation is `failed`. The org type's card (for a default change) and the ladder's page (for a reorder) list each act's organizations not moved, each with a sentence naming what failed, a reference to the act, and a Retry beside them. Retry is its own act row (who pressed it, when, which act it retried); it requeues the act's failed obligations, which then point at it, and moves them onto the current default. The catalog is never committed again.
|
||||
- **A newer act takes over.** A second default change or reorder owns the org types it reaches from the moment it commits: an earlier act no longer moves, lists, requeues or shows their organizations, and the newer act's listing marks the earlier open obligations `superseded`, pointing at the newer act. Superseded is not a failure.
|
||||
- **Tier add and tier removal stay atomic.** A live position left on a deleted tier would make the next conferral record `initiate`, so their alignment commits with the tier change. Each writes its act row and detail row with no obligations. Each refuses, before changing anything, when more organizations hold the tier or product than a measured limit allows, naming the count and the limit, and names the way through when most of them hold it through an org type's default.
|
||||
- **One runner.** One function in `internal/entitlements` runs every catalog act's transaction and each obligation's move: it opens the rendezvous kind, finds the pools, locks them in ascending order, reads again under the locks and runs the act. Grant acts stay out. Handlers parse, call the act and render.
|
||||
- **References an operator can hand on.** Every log line of the listing, the moves and the drain carries the act's id and the organization's id; each act's list of unmoved organizations ends with "Reference: {act id}"; Retry's failure and the new read-failure sentences end with the request's id, and their handlers log through the request's logger.
|
||||
- **A timed run first.** A timed test, in the shape of `TestTimedSyncCap`, times a prototype of the new commit, of the gate after a mass ending, of one listing batch and of the preview's read at up to 256,000 organizations before any migration, and must show the commit's hold flat, the gate's walk bounded and the preview inside the write timeout; it also times one organization's move and one holder's reconciliation, which set the request's count and time budget and the holder limit.
|
||||
|
||||
No column for per-organization dispositions: #82 adds it to the obligation row when it is built.
|
||||
|
||||
## Ride-along fixes
|
||||
|
||||
Each is a defect in code this change rewrites; each lands in the commit that rewrites that code, with its own test.
|
||||
|
||||
- **`ListOrganizationsByType` orders organizations by name only** (reported on #187): two organizations with one name come out in no fixed order. The query gains `org_id` as its tie-break. The acts stop calling it; its other callers keep the fix.
|
||||
- **One reason builder for every plan act**: tier add records the ladder's id instead of its name (reported on #187), and a reorder's legacy grant is described as "Grandfathered on org-type default change" (no issue). The names are read inside the act's transaction.
|
||||
- **Failed steps leave no log line** (no issue): a failed begin of a default change's or a reorder's move logs nothing, and neither do three failed steps of a tier removal. Every failed move is now logged with its act, organization, pool, step and error, in the request and in the drain, and so is every failed step of a tier removal.
|
||||
- **A tier removal with `keep` can empty a default ladder** (no issue): removing the last tier of a ladder an org type defaults to is refused inside the removal's transaction, under `keep` and `migrate` alike, as the plain delete refuses it.
|
||||
- **The plain delete reads its guards outside its transaction and swallows a failed read** (no issue): the live-holder count and the last-tier guard move inside the delete's transaction, and a failed read of the org types refuses the delete instead of allowing it.
|
||||
- **The tier add checks the product outside its transaction and answers "Product not found." for any read error** (no issue): the check moves inside, and a failed read answers the generic failure.
|
||||
- **A lock wait on tier add, tier delete or the reorder answers "see server logs"** (#216): every plan act answers a lock wait or a deadlock with "Another change collided with this one; try again." (see design.md "Operator-facing copy").
|
||||
- **The rewritten handlers answer a failure with a reference** (no issue): the tier add, the tier delete, the tier removal with its preview, the reorder with its preview and the default change with its preview and its 422 rebuild end every failure sentence that said "see server logs" or "Details are in the server logs." with "Reference: {request id}." and log through the request's logger (copies 50 to 61). These handlers are rewritten here, and leaving their sentences would add new paths ending in the sentence #239 removes.
|
||||
- **Tier add locks pools in creation order** (#194): the runner locks the holders' pools in ascending `pool_id` before any alignment, and tier add materializes each pool once rather than once per provision.
|
||||
- **A reorder that cannot read the org types says no organization was affected** (no issue): the reorder reads the types that default to its ladder inside its own transaction and refuses when the read fails.
|
||||
- **Per-organization failure reasons show database driver text** (no issue): the card and the ladder page render one sentence per failing step; the driver text goes to the logs and the obligation row only.
|
||||
- **A removal whose holders all left says "removed from the ladder: ."** (no issue): it says "{Product} removed from the ladder."
|
||||
- **The plan-ladder-management spec relies on a periodic sweep that does not exist** (no issue): the requirement says instead that the exclusive lock keeps any holder from escaping the reconciliation.
|
||||
- **The plain delete counts only `active` positions** (no issue): a tier held only by suspended positions was deleted under them and the next conferral recorded `initiate`. The delete and the tier page count `active` and `suspended` positions, as the removal does.
|
||||
- **The tier add shows database text when a rung is already held** (no issue): the add finds the collision under its pool locks, before any alignment, and its field error names the organization and the product it holds on the ladder, or the product it holds more than once; the database text of a collision the read did not find goes to the log.
|
||||
- **The tier page and the reorder preview treat a failed read of the org types as "no type defaults to this ladder"** (no issue): they show "Failed to read the org types that default to this ladder. Reference: {request id}." in place of the badges and the consequence panel.
|
||||
- **The rule change's in-progress refusal names a cause it cannot know** (no issue): it is shown behind a conferral, a move or a tier change as well as behind another rule change, so it uses the plan acts' sentence, and the requirements that mandate the old sentence change with it. For approval; see design.md "Operator-facing copy".
|
||||
- **The entitlement-set-history requirement says the drain selects failed rows** (no issue): the drain selects only pending rows, and the dead letter depends on it; the requirement is corrected, with a test.
|
||||
- **The recompute poller ends on a database outage longer than its retries** (no issue): it returns the read activity's error after five attempts and nothing restarts it. It logs the failure and reads again at its next tick, the shape the new plan-act poller has.
|
||||
- **The default select's refusal reads "Choose a change default to from the list."** (no issue): the form library builds the sentence from the label, so the label becomes "New default". For approval with the copy.
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
|
||||
- `act-records`: the act record convention: one immutable row per act with a closed cause list, the actor triple, a note and a correlation id; per-kind detail tables; which rows point at their act; the grants pointer; who may write and read the record.
|
||||
- `plan-acts`: the plan-wide acts' record and work list, and the guard on the facts a position's copies rest on: the detail row, the commit at org-type grain, the reach and its ownership, the listing in batches, obligations and their statuses, organizations without a default pool, enactment against the current default, the request budget and the drain, failure recording, Retry, supersession, the holder limit on tier add and removal, the list of organizations an act did not move, and the log lines' identifiers.
|
||||
|
||||
### Modified Capabilities
|
||||
|
||||
- `org-types`: "Default-change preview classifies existing organizations" (one database rule decides whether a position is outgoing and another a pool's class; the preview reads exact counts and at most 50 names per class) and "Default-change commit enacts per-bucket dispositions" (the commit writes the act and its reach in a hold that does not grow with the type, a listing writes the obligations, each move resolves the current default, an outgoing position under an act with no disposition is left untouched, failures are retried rather than re-run, and the card lists the organizations not moved).
|
||||
- `plan-ladder-management`: "Operator tier management within a ladder" (the reorder's enactment through the listing and obligations; the removal's and the add's holder limit; the removal's last-tier guard; the add's product check inside its transaction; the contention sentence), "Tier membership changes reconcile existing live holders via align_conferral_shape" (no sweep; pools in ascending order), "The last tier of a live default ladder cannot be deleted" (the removal with holders is refused too, inside the transaction).
|
||||
- `entitlements`: "A transaction that materializes a pool is ordered against a rule change": the two commits take the exclusive lock; the listing, each move and Retry take the shared one with a lock timeout the caller sets; the rule change and the plan acts answer contention with one sentence, "Another change collided with this one; try again."
|
||||
- `entitlement-set-management`: "A rule change commit reaches every carrying pool" (the contention sentence).
|
||||
- `entitlement-set-history`: "A commit that defers its recomputation states the obligations it takes on" (the drain selects pending rows, not failed ones).
|
||||
- `per-schema-roles`: a new requirement encloses the act record's five tables behind five `SECURITY DEFINER` functions; "Writer role grants read-write data access" lists them as enclosed; "Reader role grants read-only access" admits a withheld column, `core.acts.note`.
|
||||
|
||||
## Impact
|
||||
|
||||
- **Schema:** three migrations in the core stream, each with its first caller: `00026_outgoing_default_rule.sql` (two columns, two copy triggers and a partial index on `core.pool_provision_ladders`, the outgoing and pool-class rules' functions, and five guard triggers on `core.organizations.org_type`, `core.resource_pools.org_id`, `core.pool_provisions.grant_id` and `pool_id`, and `core.grants.grant_reason`), `00027_acts.sql` (`core.acts`, `core.plan_act_details`, `core.record_plan_act`) and `00028_plan_act_obligations.sql` (`core.plan_act_reached_org_types`, `core.plan_act_orgs_without_pool`, `core.plan_act_obligations`, `core.grants.act_id`, an index on organizations by type, and the listing, settlement, failure and requeue functions). `scripts/db-reader-role.sql` withholds `core.acts.note`; a deployment re-runs the script after the migrations.
|
||||
- **Code:** `internal/entitlements` (the runner, the five acts rewritten on it, the listing, enactment of one obligation, the drain, Retry, the reason builder, the rules the previews share, the caps, about twenty sqlc queries); `internal/workflows/entitlements` (a drain workflow and a poller with their activities, and the recompute poller's fix); `internal/workflows/worker.go` and `cmd/start.go` (registration and the poller's start); `internal/server` (the five handlers reduced to parse, call and render; two Retry routes; the card's and the ladder page's lines; the removal preview's disabled commit); `internal/organization/queries`.
|
||||
- **Tests:** every file under `internal/server/testdata/plan_acts/` (55 files) changes on purpose, in the tasks that add the act sections and the obligation sections; the files whose behaviour changes are named in the task that changes them. `TestTierRemovalAgainstDefaultChange`, `TestReorderEnactsAgainstTheRankZeroItSaved`, `TestDefaultChangeTwiceSerializesOnThePool` and `TestPlanActsRefuseWhileARuleChangeHoldsTheRendezvous` change on purpose. New tests cover the agreement of the preview, the gate and the move on the outgoing and pool-class rules, the listing, ownership and supersession, failure recording, Retry, the drain, the holder limit, the enclosure and the timed run.
|
||||
- **Docs:** `docs/database-locks.md`, `docs/plan-management.md`, `docs/models/plan-ladders-transitions.md`, `docs/models/entitlements.md`, `docs/go-conventions.md`, a new card `docs/models/acts.md` listed in `docs/models/README.md`, and `status/design-feedback.md` (the act record and the position copies are not in the design's data model).
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **A removal or add above the holder limit** (#237): retiring a tier and draining its holders replaces the limit later.
|
||||
- **A note on the default-change and reorder forms** (#238): `core.acts.note` exists and stays empty from these forms.
|
||||
- **#78's attention registry and its overview line.** The list of an act's unmoved organizations is one query the overview can reuse.
|
||||
- **A page for each act**, listing every organization it reached and how each one ended. The records this change keeps, including the organizations without a default pool, are enough to build it.
|
||||
- **Per-organization dispositions** (#82), including the obligation column they need.
|
||||
- **A pointer from each transition to its act** (#179). The conferral functions write the transitions and #179 replaces them; until then the reason text names the act.
|
||||
- **A record of a rule-change batch and the actor of its Retry** (#181).
|
||||
- **A swap recorded as one act** (#184).
|
||||
- **The console's other "Details are in the server logs" sentences** (#239), which this change does not touch.
|
||||
@@ -0,0 +1,110 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: An act is recorded once, in one row that is never changed
|
||||
|
||||
An action that writes several rows SHALL record itself in one row of `core.acts`, written in the same transaction as the first row it causes and before any row that refers to it. The row SHALL carry the act's identifier (a UUIDv7), its cause, its actor, an optional note, the identifier of the request that made it when there is one, the time the act takes effect (`effective_at`) and the time the row was written (`created_at`). No process SHALL update or delete an act row, and the application's database role SHALL hold no privilege that would let it: progress, counts and outcomes SHALL be derived from the rows that point at the act, never stored on it.
|
||||
|
||||
#### Scenario: A committed act writes one row
|
||||
|
||||
- **WHEN** an operator commits an org type's default change
|
||||
- **THEN** exactly one `core.acts` row is written for it, in the commit's transaction
|
||||
- **AND** its cause, actor, effective time and write time are set
|
||||
|
||||
#### Scenario: An act row is never rewritten
|
||||
|
||||
- **WHEN** the organizations an act reached are settled minutes after it committed
|
||||
- **THEN** no column of its `core.acts` row changes
|
||||
- **AND** an attempt by the application's role to update or delete the row is refused by the database
|
||||
|
||||
### Requirement: An act's cause comes from a closed list
|
||||
|
||||
`core.acts.cause` SHALL be a `VARCHAR` constrained by a `CHECK` listing every cause, not an enumerated type and not a lookup table. The causes SHALL be exactly `tier_added`, `tier_removed`, `tiers_reordered`, `org_type_default_changed` and `plan_act_retried` until a later change adds its own by replacing the constraint in its own core migration. Causes SHALL name what happened in the past tense and SHALL name no provider. The Go code SHALL mirror the list as typed constants, and a test SHALL compare the constants with the constraint's definition in a migrated database.
|
||||
|
||||
#### Scenario: An unknown cause is refused
|
||||
|
||||
- **WHEN** a row is inserted into `core.acts` with a cause the constraint does not list
|
||||
- **THEN** the database refuses the insert
|
||||
|
||||
#### Scenario: The Go list and the database list agree
|
||||
|
||||
- **WHEN** the test reads `chk_acts_cause` from a migrated database
|
||||
- **THEN** its values are exactly the Go cause constants
|
||||
|
||||
### Requirement: An act's actor is the audit triple
|
||||
|
||||
An act's actor SHALL be recorded as an actor type of `person`, `service_account` or `system`, with `actor_person_id` (a foreign key to `core.persons`) set if and only if the type is `person`, and `actor_service_account_id` set if and only if the type is `service_account`, enforced by a coherence `CHECK` in the form the entitlement set change ledger uses. Which surface acted (an operator's page, a member's page, a sweep) SHALL be read from the cause, not from the actor type.
|
||||
|
||||
#### Scenario: An operator's act names the person
|
||||
|
||||
- **WHEN** an operator commits a tier removal
|
||||
- **THEN** its act row has `actor_type = 'person'` and that operator's person id
|
||||
|
||||
#### Scenario: An incoherent actor is refused
|
||||
|
||||
- **WHEN** a row is inserted with `actor_type = 'system'` and a person id
|
||||
- **THEN** the database refuses the insert
|
||||
|
||||
### Requirement: The note is optional and the request id is for correlation only
|
||||
|
||||
`core.acts.note` SHALL be nullable free text, SHALL be written only for an act whose actor is a person (a `CHECK`), and SHALL be the one place an act keeps a person's own words; no act SHALL require one. `core.acts.request_id` SHALL hold the identifier of the request that made the act, for correlating the act with log lines, when that identifier is at most 100 characters, and NULL otherwise. It SHALL NOT be unique, and no code SHALL use it to find, group or deduplicate acts.
|
||||
|
||||
#### Scenario: An act without a note is recorded
|
||||
|
||||
- **WHEN** an operator commits a reorder from a form that offers no note
|
||||
- **THEN** the act row is written with a NULL note
|
||||
|
||||
#### Scenario: An over-long request id is not stored
|
||||
|
||||
- **WHEN** a request carrying a 300-character `X-Request-ID` commits an act
|
||||
- **THEN** the act is recorded with a NULL `request_id`
|
||||
- **AND** the act commits as it would with a short one
|
||||
|
||||
### Requirement: What an act did is recorded in a detail table for its kind
|
||||
|
||||
What an act did, beyond its header, SHALL be recorded in one detail table per kind of act, keyed by the act's id and tied to its act row by a composite foreign key on `(act_id, cause)`, so a detail row can belong only to an act of its own kind. Detail SHALL NOT be stored as JSONB. A detail row SHALL refer to catalog rows (ladders, products, org types) by id with no foreign key, beside the name each had when the act committed, so the record outlives a deleted or renamed catalog row and reads as it read then. A provider-specific detail table SHALL live in its provider's schema with a foreign key to `core.acts`.
|
||||
|
||||
#### Scenario: A detail row cannot belong to another kind's act
|
||||
|
||||
- **WHEN** a plan act detail row is inserted with a cause other than its act row's cause
|
||||
- **THEN** the database refuses the insert
|
||||
|
||||
#### Scenario: A deleted ladder leaves its acts readable
|
||||
|
||||
- **WHEN** a tier was added to a ladder and the ladder is later deleted
|
||||
- **THEN** the tier add's detail row still names the ladder by the name it had when the tier was added
|
||||
- **AND** the ladder's deletion is not refused because of the record
|
||||
|
||||
### Requirement: The rows an act writes point at it
|
||||
|
||||
Every row an act inserts to record what happened or what was decided SHALL carry the act's id: its detail row, the org types it reaches, its records of organizations without a default pool, its obligations, and every grant it creates (`core.grants.act_id`, a nullable foreign key to `core.acts`, nullable because other writers of grants do not record an act yet). A grant that the default floor creates inside an act SHALL stay system-authored, with no granting person, while its `act_id` names the act. Catalog rows, rows of current state (provisions, positions, materialized entitlements) and the desired-state outbox SHALL NOT point at an act; the act's detail row records the catalog change. Transition rows SHALL point at their act once the functions that write them are replaced to do so; until then a transition SHALL name its act through its reason text.
|
||||
|
||||
#### Scenario: A default grant points at the act that conferred it
|
||||
|
||||
- **WHEN** an org type's default change initiates a plan-less organization onto the new default
|
||||
- **THEN** the default grant it creates carries that act's id
|
||||
- **AND** its `granted_by_person_id` is NULL
|
||||
|
||||
#### Scenario: A tier row does not point at its act
|
||||
|
||||
- **WHEN** a tier is added to a ladder
|
||||
- **THEN** the `plan_ladder_tiers` row carries no act id, and the act's detail row names the ladder and the product
|
||||
|
||||
### Requirement: The act record withholds personal text from the read-only roles
|
||||
|
||||
No column of `core.acts`, `core.plan_act_details`, `core.plan_act_reached_org_types`, `core.plan_act_orgs_without_pool` or `core.plan_act_obligations` other than `core.acts.note` SHALL hold a personal name, an email address or a person's free text. Neither read-only role SHALL read `core.acts.note`: `core_reader`, which every integration's writer role holds, SHALL hold SELECT on every other column of `core.acts` through column grants and none on the note; and `core.acts.note` SHALL be on `scripts/db-reader-role.sql`'s list of personal columns, with `internal/migrate/testdata/console-reader-columns.txt` listing it as `withhold` and every other column of the five tables, `core.grants.act_id`, `core.pool_provision_ladders.org_type` and `core.pool_provision_ladders.default_sourced` as `read`. An organization's name SHALL NOT be copied onto any of the five tables; a reader reaches it through the organization's id or the pool.
|
||||
|
||||
#### Scenario: The column list covers the act record
|
||||
|
||||
- **WHEN** the console-reader column test runs against a migrated database
|
||||
- **THEN** it passes with `core.acts.note` withheld and every other column of the five tables readable
|
||||
|
||||
#### Scenario: The read-only role cannot read a note
|
||||
|
||||
- **WHEN** `scripts/db-reader-role.sql` has run and a `console_reader` session selects `core.acts.note`
|
||||
- **THEN** the query is refused
|
||||
|
||||
#### Scenario: An integration's writer role cannot read a note
|
||||
|
||||
- **WHEN** a session holding only `core_reader` selects `core.acts.note`
|
||||
- **THEN** PostgreSQL refuses the query on privilege grounds
|
||||
- **AND** the same session can select every other column of `core.acts`
|
||||
@@ -0,0 +1,29 @@
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: A commit that defers its recomputation states the obligations it takes on
|
||||
|
||||
A commit whose population is above the synchronous cap SHALL state before it is taken that recomputation continues after the change is applied, so which path a commit takes is operator-visible and not only internal. Replay of a deferred recomputation SHALL be idempotent: re-settling a pool SHALL NOT write a duplicate effect row for the same change, pool and resource key, and settling an obligation that is already settled SHALL return without writing. A pool that cannot be recomputed SHALL be retained as failed with its error rather than dropped. How many pools a set still owes and how many have failed SHALL be visible on the set's page, and the failed count SHALL be visible on the sets list without opening the set.
|
||||
|
||||
The drain SHALL select its work from the work list rather than from any other enumeration, and its selection SHALL include rows that are `pending` and SHALL NOT include rows that are `failed`, which are the dead letter, so a row that has failed the maximum number of times is not picked up again until the Retry control returns it to `pending` and it is then picked up without a second code path. The one operator-caused transition SHALL be the Retry control, which SHALL return `failed` rows to `pending` and SHALL act on no other status, so retrying cannot disturb a pool that is still draining or one that has already settled.
|
||||
|
||||
#### Scenario: The path is stated before it is taken
|
||||
|
||||
- **WHEN** an operator stages a rule change whose population is above the cap
|
||||
- **THEN** the tray states, before the change is applied, that recomputation continues after it is applied
|
||||
|
||||
#### Scenario: A replayed settlement writes no duplicate
|
||||
|
||||
- **WHEN** a pool's obligation is drained twice for the same change and resource key
|
||||
- **THEN** exactly one effect row exists for that change, pool and resource key
|
||||
|
||||
#### Scenario: Retry touches only failed rows
|
||||
|
||||
- **WHEN** a change owes 3 pending pools and holds 2 failed ones and an operator activates Retry
|
||||
- **THEN** the 2 failed rows return to `pending` and are redispatched
|
||||
- **AND** the 3 already-pending rows and every settled row are untouched
|
||||
|
||||
#### Scenario: The drain does not select a dead letter
|
||||
|
||||
- **WHEN** a pool's obligation has failed the maximum number of times and the drain selects its work
|
||||
- **THEN** the failed row is not selected
|
||||
- **AND** it is selected again only after the Retry control has returned it to `pending`
|
||||
@@ -0,0 +1,58 @@
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: A rule change commit reaches every carrying pool
|
||||
|
||||
`Apply changes` SHALL apply the staged batch as one commit. The commit SHALL re-derive everything it acts on: each named rule's current state, the population of pools whose provisions carry the set with `status = 'active'`, and every limit, inside its own transaction. No client-supplied copy of any preview SHALL be read back, and no staleness token SHALL be carried. A delta naming a rule another operator deactivated since it was staged SHALL refuse the whole batch with "That rule is no longer active." on that delta's line and write nothing; a delta whose fields have come to equal the stored ones SHALL refuse the whole batch with "Already applied." on its line and write nothing, because the batch is one transaction and a partial apply would leave a ruleset nobody staged.
|
||||
|
||||
The commit's transaction SHALL take the `entitlement_materialization` advisory lock in exclusive mode first, once for the batch, which is what serializes two rule commits, and SHALL then take the entitlement set's row `FOR UPDATE` as a guard against a writer that changes the set row without that lock. A commit that waits past its lock timeout SHALL refuse with "Another change collided with this one; try again." and write nothing, whichever of the two locks it was waiting on.
|
||||
|
||||
The rule write SHALL be enclosed. One database function SHALL be the only path that writes a row of `entitlement_set_rules`, and direct insert, update and delete on that table SHALL be revoked from the application's role, so a rule cannot be written without its act row and its obligations by any code path, whether or not the code remembers to write them. That function SHALL, as its first statement, assert that its own transaction holds the materialization rendezvous in exclusive mode and raise a named error if it does not. It SHALL then, inside the commit's transaction, lock the set row, validate the proposed kind against the stored rule and raise a named mismatch when they disagree, apply the rule write, snapshot the rule before and after, enumerate the carrying population once, decide the path against the cap passed to it, write the act row with the path it took, and write one pending obligation row per enumerated pool. The commit SHALL call that function once per delta, in resource-key order, so a batch of N deltas files N act rows, one per rule, in one transaction, with the ledger's grain and the set's History unchanged; every act of one batch SHALL carry the same request identifier. The function SHALL NOT materialize; the caller materializes and settles each obligation through a second enclosed function that writes that pool's effect rows and marks its obligation settled. The cap SHALL be passed to the function rather than read inside it.
|
||||
|
||||
At or below the stated cap on carrying pools, the commit SHALL be atomic: after every delta's rule write it SHALL lock every carrying pool `FOR UPDATE` in ascending pool id order, materialize each pool once against the whole new ruleset, and settle that pool's obligation under each act of the batch from that one recomputation, routing each resource key's effect to the act that changed that key and a movement on a key no delta names to the batch's first act; all in one transaction with the rule writes, the act rows and the obligation rows, and with a savepoint per pool so a failure names the organization that caused it. Any error before the commit SHALL leave the database exactly as it was, every delta included.
|
||||
|
||||
Above the cap the commit SHALL take no pool lock at all: the enclosed function writes each rule change, its act row and one durable obligation row per enumerated pool, and they commit together, so that from the instant the rules are live every pool they owe is named in the work list under every act. It SHALL then drain what the request can before returning and leave the rest to be settled without an operator, and the drain SHALL settle a pool's obligations that share one request identifier from one recomputation ("Outstanding recomputations are settled without an operator").
|
||||
|
||||
A deactivation of a rule that is already inactive SHALL be reported as "That rule is no longer active."
|
||||
|
||||
Both paths SHALL report their outcome in one toast composed only of non-zero counts: how many rules changed, how many pools were recomputed and how many ended at or over their usage, and on the deferred path how many of the owed pools were recomputed before the request ended.
|
||||
|
||||
#### Scenario: A small population commits a batch atomically
|
||||
|
||||
- **WHEN** an operator applies a batch of three deltas on a set carried by fewer pools than the cap
|
||||
- **THEN** three rule writes, three act rows, their obligation rows and every pool's single materialization commit in one transaction
|
||||
- **AND** nothing is left to drain
|
||||
|
||||
#### Scenario: A batch recomputes each pool once and records each key under its act
|
||||
|
||||
- **WHEN** a batch edits the rules for keys A and B on a set whose pools also move on key C through stacking
|
||||
- **THEN** each carrying pool is materialized once, its effect rows for A hang off A's act, for B off B's act, and for C off the batch's first act
|
||||
- **AND** no effect row is duplicated across the batch's acts
|
||||
|
||||
#### Scenario: A large population commits the rules with their obligations
|
||||
|
||||
- **WHEN** an operator applies a batch on a set carried by more pools than the cap
|
||||
- **THEN** the transaction commits every rule change, every act row and one pending obligation row per carrying pool under each act, all sharing one request identifier
|
||||
- **AND** every pool the batch owes is named in the work list from the instant the rules are live
|
||||
|
||||
#### Scenario: A pool that cannot be materialized refuses the whole atomic commit
|
||||
|
||||
- **WHEN** materializing one pool fails during an atomic commit of a batch
|
||||
- **THEN** the transaction rolls back, leaving every rule unchanged and no act, effect or obligation row written
|
||||
- **AND** the tray's form-level line names the organization whose pool failed
|
||||
|
||||
#### Scenario: The commit re-derives a rule the batch no longer describes
|
||||
|
||||
- **WHEN** an operator applies a batch containing an edit to a rule another operator deactivated after it was staged
|
||||
- **THEN** the commit refuses with "That rule is no longer active." on that delta's line and writes nothing, the other deltas included
|
||||
|
||||
#### Scenario: A rule cannot be written outside the enclosed function
|
||||
|
||||
- **WHEN** any console code path attempts to insert, update or delete a row of `entitlement_set_rules` directly
|
||||
- **THEN** the database refuses it, because the application's role holds no such privilege
|
||||
- **AND** the only accepted rule write carries its act row and its obligation rows in the same transaction
|
||||
|
||||
#### Scenario: Two rule commits on one set serialize
|
||||
|
||||
- **WHEN** two operators apply batches on the same set at the same time
|
||||
- **THEN** one commits and the other waits for the exclusive materialization lock before it reaches any row
|
||||
- **AND** a wait that times out refuses with "Another change collided with this one; try again." and writes nothing
|
||||
@@ -0,0 +1,83 @@
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: A transaction that materializes a pool is ordered against a rule change
|
||||
|
||||
Every transaction that materializes a pool SHALL take the `entitlement_materialization` advisory lock as its first acquisition, before it takes any row: in shared mode, or in exclusive mode when the same transaction changes a rule, carries an operator's tier change or commits an org type's default change. Every transaction that adds, changes, deactivates or reactivates an entitlement set rule, every transaction of an operator act that changes a plan ladder's tiers (the tier add, the tier delete, the tier removal and the reorder's commit transaction), and the default change's commit transaction, which saves an org type's default ladder, SHALL take the same lock in exclusive mode as its first acquisition, before it takes any row. A plan act's listing batch, the transaction that settles one plan act obligation and a plan act's Retry SHALL take it in shared mode as their first acquisition. The rule-change transaction SHALL then take the entitlement set's row before any pool row. Any transaction that takes more than one pool row SHALL take them in ascending pool id order. Because the advisory lock is the first acquisition on both sides, no holder waits for something a would-be holder holds, so a rule change, a tier change or a default change and a recomputation are strictly ordered whichever begins first and neither can hold a row the other waits for. A conferral reads a ladder's ranks and a product's conferral shape in more than one statement, so this ordering is what makes it classify each tier change against one order of the ladder; a plan act's move reads an org type's default, that ladder's rank 0 and which act owns the type in more than one statement, so the same ordering is what makes it resolve one catalog and one owner.
|
||||
|
||||
Shared holders SHALL NOT block each other, so recomputations continue to run concurrently with one another. Each such transaction SHALL set a lock timeout, so a wait surfaces as a refusal an operator can read and retry rather than as a hang; a deadlock reported by the database SHALL surface as the same retryable refusal rather than as a server error. A rule change (`entitlement-set-management`) and every plan-wide act (the tier add, the tier delete, the tier removal, the reorder, the default change, and a plan act's Retry) SHALL answer a lock wait past the timeout or a deadlock at any of its steps with "Another change collided with this one; try again.", because the holder it waited for may be a rule change, a tier change, a default change, a move or a conferral. A plan act's work inside a request MAY use a shorter lock timeout than its drain and a transaction timeout, both set before the materialization lock is taken, and SHALL then record no attempt for a wait or a transaction past either. A transaction past its transaction timeout ends its database session (SQLSTATE 25P04), so its connection is discarded.
|
||||
|
||||
Possession SHALL be enforced rather than trusted. Before it writes anything, the materializer's apply step SHALL assert that its own transaction holds the rendezvous, by calling the one shared database assertion that queries the lock state for the calling backend, and SHALL refuse with a named error identifying the pool when it does not. Every transaction that adds, changes, deactivates or reactivates a rule SHALL call the same assertion in its exclusive form as its own first statement; so SHALL every enclosed function that records a plan act's catalog change. Every enclosed function that inserts, supersedes, settles or requeues a plan act's obligations, and the one that records a Retry, SHALL call it in either form. A caller that forgets the rendezvous therefore refuses a conferral, an activation, a grant or a plan act at once, instead of writing limits derived from a rule state another transaction has already replaced. Lock order is not asserted: a transaction that takes a pool row first surfaces its mistake as a lock timeout or a deadlock, which is a refused request rather than a wrong limit. Other advisory locks MAY be taken before the rendezvous, which stays safe because no rule-change, tier-change or default-change transaction takes any of them.
|
||||
|
||||
Membership is decided by what a transaction does, not by which primitive it calls. The transactions bound by this requirement today are the rule-change commit and its drain, the tier add, the tier delete, the tier removal, the reorder's commit transaction, the default change's commit transaction, a plan act's listing batch, the transaction that settles one plan act obligation (in the request and in the plan act drain), a plan act's Retry, grant issuance and grant revocation, the demonstration seed's materialization, and the subscription reconcile's suspend and resume paths. The rule-change commit, the four tier changes and the default change's commit transaction SHALL open through the exclusive opener and every other one through the shared opener, each covered by a test that drives the caller and asserts it does not refuse. The system SHALL provide one transaction opener for each side, the shared one taking its lock timeout and its transaction timeout as parameters and setting both before the lock (a request uses a shorter lock timeout than the drain, and a transaction timeout the drain does not set), so that a transaction which materializes a pool without opening through them is a defect that can be found by reading the code as well as by running it.
|
||||
|
||||
#### Scenario: A conferral that begins first finishes before the rule change
|
||||
|
||||
- **WHEN** a conferral transaction has taken the shared lock and a rule commit arrives
|
||||
- **THEN** the rule commit waits for the exclusive lock until that conferral's transaction ends
|
||||
- **AND** the conferral materializes against the rules that were live when it began
|
||||
|
||||
#### Scenario: A conferral that begins during a rule change waits for it
|
||||
|
||||
- **WHEN** a rule commit holds the exclusive lock and a conferral for a pool that is about to carry the set begins
|
||||
- **THEN** the conferral waits before taking any row
|
||||
- **AND** it materializes against the committed rule state, so no pool is left materialized from a rule state the commit replaced
|
||||
|
||||
#### Scenario: Two recomputations do not block each other
|
||||
|
||||
- **WHEN** two conferrals on different pools run at the same time with no rule change in flight
|
||||
- **THEN** both hold the lock in shared mode and neither waits for the other
|
||||
|
||||
#### Scenario: A rule change's wait that exceeds the timeout is a readable refusal
|
||||
|
||||
- **WHEN** a rule change waits longer than the configured lock timeout for the materialization lock
|
||||
- **THEN** it rolls back and the refusal reads "Another change collided with this one; try again."
|
||||
- **AND** no partial write survives
|
||||
|
||||
#### Scenario: The reconcile's resume path is bound by the invariant
|
||||
|
||||
- **WHEN** a subscription reconcile resumes a suspended provision and materializes its pool
|
||||
- **THEN** its transaction takes the shared materialization lock before any row
|
||||
- **AND** it cannot write limits derived from a rule state a rule commit has already replaced
|
||||
|
||||
#### Scenario: A materializer that holds no rendezvous refuses
|
||||
|
||||
- **WHEN** a transaction that did not take the materialization lock calls the materializer's apply step
|
||||
- **THEN** it refuses with a named error identifying the pool and writes nothing
|
||||
|
||||
#### Scenario: The reconcile's per-subscription lock follows the rendezvous
|
||||
|
||||
- **WHEN** the subscription reconcile takes the shared materialization lock and then its per-subscription advisory lock
|
||||
- **THEN** the assertion passes and the materialization proceeds
|
||||
|
||||
#### Scenario: A tier change waits for a conferral in flight
|
||||
|
||||
- **WHEN** a conferral transaction holds the shared lock and has read a ladder's ranks, and a reorder, a tier delete, a tier removal or a tier add on that ladder arrives
|
||||
- **THEN** the tier change waits for the exclusive lock until that conferral's transaction ends
|
||||
- **AND** the conferral classifies its transitions against the ladder as it read it, so a handoff of one product between sources is recorded as a `transfer`
|
||||
|
||||
#### Scenario: A conferral that begins during a tier change waits for it
|
||||
|
||||
- **WHEN** a reorder, a tier delete, a tier removal or a tier add holds the exclusive lock and a conferral on that ladder begins
|
||||
- **THEN** the conferral waits before taking any row
|
||||
- **AND** it reads the ladder's ranks and the product's shape as the tier change committed them
|
||||
|
||||
#### Scenario: A default change's commit is ordered against a plan act's move
|
||||
|
||||
- **WHEN** one organization's move holds the shared lock and a default change for that organization's type arrives
|
||||
- **THEN** the default change's commit transaction waits for the exclusive lock until the move ends
|
||||
- **AND** a move that begins while the commit holds the lock waits, then resolves the default the commit saved and finds the new act owning the type
|
||||
|
||||
#### Scenario: A plan act's catalog record refuses without the exclusive lock
|
||||
|
||||
- **WHEN** a transaction that holds only the shared lock asks the database to record a default change or a reorder
|
||||
- **THEN** the function refuses with the rendezvous assertion's named error and writes nothing
|
||||
|
||||
#### Scenario: A plan act's listing refuses without the lock
|
||||
|
||||
- **WHEN** a transaction that holds no materialization lock asks the database to write a plan act's obligations
|
||||
- **THEN** the function refuses with the rendezvous assertion's named error and writes nothing
|
||||
|
||||
#### Scenario: Every plan act answers a lock wait with the refusal
|
||||
|
||||
- **WHEN** a tier add, a tier delete, a tier removal, a reorder or a default change waits longer than the lock timeout because another transaction holds the exclusive lock
|
||||
- **THEN** it answers "Another change collided with this one; try again."
|
||||
@@ -0,0 +1,124 @@
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Default-change preview classifies existing organizations
|
||||
|
||||
The preview SHALL classify every existing organization of the org type into exactly one of five classes, computed read-only (no writes, no rolled-back writes), and SHALL render the classification scoped to the org type's card so its attribution is unambiguous when multiple types are listed:
|
||||
|
||||
1. **Would initiate** — the org's default pool has no live (`active` or `suspended`) plan-ladder attachment; committing would initiate it onto the candidate default (when one is set).
|
||||
2. **Already at the target** — the pool's default-sourced position already delivers the candidate ladder's rank-0 product. These SHALL be reported and SHALL never be touched by the commit; this class is what makes a move idempotent: an organization that a Retry, a repeated drain pass or a newer change reaches after it was moved lands here, not back in the outgoing bucket.
|
||||
3. **Outgoing default** — the pool has a live attachment whose provision is grant-sourced with `grant_reason = 'default'` and does NOT deliver the candidate's rank-0 product. Membership SHALL otherwise be determined by this source class (provenance), not by which ladder the position occupies, so default-sourced positions stranded on a former ladder are still surfaced for disposition. With a None candidate there is no target, so every default-sourced position is outgoing. Whether one position is outgoing SHALL be decided by one database function, `core.plan_position_is_outgoing`, and which class a pool falls in by another, `core.plan_pool_bucket`, which folds the pool's live positions through the first: one outgoing position makes the pool outgoing. The preview SHALL apply them to every pool of the type in one statement, a move SHALL apply them to its one pool, and the commit's disposition gate SHALL apply the position function to the positions it reads, so the preview, the gate and the move cannot disagree. No application code SHALL decide a pool's class.
|
||||
4. **Other source** — the pool has a live attachment from any other source (operator or legacy grant, subscription). These SHALL be reported with the holding source named per org (so "why doesn't the default reach this org?" answers itself) and SHALL never be touched by the commit.
|
||||
5. **Would fail** — the org cannot be classified (e.g. no default pool); these SHALL be listed per-org with reasons.
|
||||
|
||||
The preview SHALL name the organizations in the outgoing-default, other-source and would-fail classes, not merely count them, at most 50 names per class ordered by name and then identifier, ending a longer list with "And N more."; it SHALL read each class's exact count and at most 50 names per class from the database, never every organization of the type, and every count it renders SHALL be that exact count, never the length of a shortened list. A list's disclosure label SHALL read "show all N" while the list holds every name and "show 50 of N" when it does not. The same reads and the same limit SHALL serve the preview, each change of the disposition that re-renders the card, and the commit's 422 answer that rebuilds the preview. Preview output SHALL state that nothing has been changed; the commit's reported results are authoritative.
|
||||
|
||||
#### Scenario: Classification over a mixed population
|
||||
|
||||
- **WHEN** an operator selects candidate ladder B for a type whose orgs comprise two with no plan attachment, three on the outgoing default-sourced position, one on an operator-grant position, and one org lacking a default pool
|
||||
- **THEN** the preview SHALL report 2 would-initiate, 3 outgoing-default (named), 1 other-source, and 1 would-fail with its reason
|
||||
- **AND** no grant, provision, attachment, or transition row SHALL be written by the preview
|
||||
|
||||
#### Scenario: Outgoing default is classified by provenance, not ladder
|
||||
|
||||
- **WHEN** an org's default pool holds a live position sourced by a `grant_reason = 'default'` grant on a ladder that is no longer the org type's configured default, and that position does not deliver the candidate's rank-0 product
|
||||
- **THEN** the preview SHALL place that org in the outgoing-default bucket
|
||||
|
||||
#### Scenario: A pool with an at-target and an outgoing default position is outgoing
|
||||
|
||||
- **WHEN** an org's default pool holds one default-sourced position delivering the candidate's rank-0 product and another, on a former ladder, that does not
|
||||
- **THEN** the preview SHALL place that org in the outgoing-default bucket
|
||||
- **AND** a commit without a disposition SHALL be rejected by the gate inside its transaction
|
||||
|
||||
#### Scenario: Default-sourced position at the candidate is already at the target
|
||||
|
||||
- **WHEN** an org's default pool holds a default-sourced position delivering the candidate ladder's rank-0 product (e.g. after a prior commit initiated or migrated it there)
|
||||
- **THEN** the preview SHALL classify that org as already at the target
|
||||
- **AND** a subsequent commit SHALL NOT require a disposition for it and SHALL NOT touch its position
|
||||
|
||||
#### Scenario: A long list of organizations is capped
|
||||
|
||||
- **WHEN** a type has 120 organizations on the outgoing default and the operator selects a candidate
|
||||
- **THEN** the preview SHALL report 120 outgoing-default organizations, label the list "show 50 of 120", name 50 of them, and end the list with "And 70 more."
|
||||
|
||||
#### Scenario: The preview, the gate and the move agree on a pool's class
|
||||
|
||||
- **WHEN** a pool of the type holds two default-sourced positions, one delivering the candidate's rank-0 product and one not
|
||||
- **THEN** the preview, the commit's gate and a move of that pool SHALL each find the pool outgoing
|
||||
- **AND** they SHALL find it outgoing again when the candidate is None
|
||||
|
||||
#### Scenario: Preview is attributed to its org type
|
||||
|
||||
- **WHEN** previews have been produced for two different org types on the same page
|
||||
- **THEN** each preview SHALL render inside its own org type's card
|
||||
- **AND** neither SHALL render as a page-global result
|
||||
|
||||
|
||||
### Requirement: Default-change commit enacts per-bucket dispositions
|
||||
|
||||
The commit SHALL accept the candidate default (a plan ladder or None) together with a disposition for the outgoing-default bucket: exactly one of `grandfather` or `migrate` — the commit SHALL be rejected with a validation error, writing nothing, when the outgoing-default bucket is non-empty and no disposition for it was supplied. The would-initiate bucket SHALL carry no disposition: when a candidate is set it is always initiated (the default is a floor; a skipped pool would be re-floored by its next restoration event anyway, so an opt-out would be a soft skip, not a policy), and when the candidate is None it stays plan-less. When the candidate is None, `migrate` SHALL mean end-to-baseline: the default-source position is ended with no replacement and the pool legitimately goes off-ladder.
|
||||
|
||||
Enactment SHALL first commit, in one transaction that holds the materialization lock in exclusive mode (`entitlements`), the type's new `org_types.default_plan_ladder_id`, the act's record and the type as the org type the act reaches (`plan-acts`). That transaction SHALL write no per-organization row, so its length does not grow with the type. The disposition gate SHALL be evaluated inside that transaction, so the bucket it reads is the one at the commit, not the preview's: it SHALL ask whether any organization of the type holds a position the outgoing rule marks as outgoing, through an index probe whose first read after a mass ending walks the entries vacuum has not yet removed and otherwise reads a fixed number of entries (`plan-acts`). A candidate that is already the type's default when that transaction reads the type, including one a concurrent commit saved first, SHALL be refused with "Already the default.", writing nothing. After the commit, a listing SHALL write one obligation per organization of the type that has an active default pool, in short batches (`plan-acts`). Each obligation SHALL then be settled in its own short per-pool transaction, which derives the pool's bucket in its own transaction against the type's current default ladder and that ladder's current rank-0 product, never a product fixed before the commit, and applies that bucket's disposition. Would-initiate pools SHALL be enacted via the floor-guarded restoration (`ReapplyDefaultsIfVacant`), whose guard doubles as the in-tx re-derivation. Outgoing-default pools SHALL be enacted per "Grandfathering reissues the outgoing default as legacy grants" or, under `migrate`, by `end_conferral` resolved by the incumbent default grant's source followed by the floor-guarded restoration (the two-call model; `end` + `initiate` transitions, cross-ref `plan-downgrade` — the guard, rather than an unconditional re-apply, ensures a pool that also carries another source's plan or pre-floor union debris is healed, never re-layered). Already-at-target and other-source pools SHALL NOT be touched; their obligations are settled as such. An outgoing-default pool found under an act that carries no disposition SHALL NOT be touched either: the gate proved no such pool existed at the commit, so the position appeared afterwards, and the move SHALL settle it `untouched` and log it with the act's id and the organization's id (`plan-acts`). Every transition recorded by operator-disposition enactment SHALL carry `actor_type = 'operator'`, `actor_id` set to the person whose act put the obligation in motion (the committing operator, or the operator who pressed Retry), and a reason naming the default change.
|
||||
|
||||
Per-pool failures SHALL NOT abort the run and SHALL NOT be lost: each is recorded on its obligation with its attempts and its failing step, and after the maximum attempts the obligation is failed. The answer SHALL report the actual per-bucket counts of the obligations settled within the request, how many organizations are not yet reached (pending obligations and organizations the listing has not reached), and how many organizations have a record of the act as having no default pool when the request answers. The org type's card SHALL list the organizations its default change did not move, each with its reason, beside a Retry that moves them onto the type's current default without committing the default again (`plan-acts`). A re-chosen current default is not a re-run: the form refuses it, and Retry is the way to move what a commit left behind.
|
||||
|
||||
#### Scenario: A true gate with nothing outgoing in the rebuilt preview is a failure, not a request
|
||||
|
||||
- **WHEN** the gate inside the commit's transaction found an outgoing position and the preview rebuilt after the rollback finds none
|
||||
- **THEN** the commit SHALL answer the form's generic failure and log an Error with the reached types and the target, and SHALL NOT ask the operator to choose a disposition
|
||||
|
||||
#### Scenario: Commit without a required disposition is rejected
|
||||
|
||||
- **WHEN** an operator commits a candidate default while the outgoing-default bucket is non-empty and no grandfather/migrate disposition was supplied
|
||||
- **THEN** the commit SHALL be rejected with a validation error
|
||||
- **AND** neither the `org_types` row nor any act, obligation, grant, provision, attachment, or transition SHALL be written
|
||||
|
||||
#### Scenario: A signup between the preview and the commit is seen by the gate
|
||||
|
||||
- **WHEN** the preview found no outgoing-default organization, the operator commits with no disposition, and an organization of the type signed up onto the outgoing default after the preview
|
||||
- **THEN** the gate inside the commit's transaction SHALL find that organization's position outgoing and the commit SHALL be rejected, writing nothing
|
||||
|
||||
#### Scenario: Migrate moves the outgoing-default population via end + initiate
|
||||
|
||||
- **WHEN** an operator commits a change from ladder A to ladder B with disposition `migrate` for the 3 outgoing-default orgs
|
||||
- **THEN** for each of the 3 pools, in its own transaction, the incumbent default-source position SHALL be ended (`end` transition) and a fresh system-authored default SHALL be conferred at B's rank-0 (`initiate` transition)
|
||||
- **AND** the disposition-driven transitions SHALL carry `actor_type = 'operator'` with the invoking operator's `actor_id`
|
||||
|
||||
#### Scenario: Clearing the default with end-to-baseline
|
||||
|
||||
- **WHEN** an operator commits candidate None with disposition `migrate` for the outgoing-default population
|
||||
- **THEN** each outgoing-default position SHALL be ended with no replacement conferral, and only `end` transitions SHALL be recorded
|
||||
- **AND** `default_plan_ladder_id` SHALL be NULL after the commit
|
||||
|
||||
#### Scenario: The default and the act commit before any organization is listed
|
||||
|
||||
- **WHEN** an operator commits a default change for a type of three organizations with default pools
|
||||
- **THEN** the new default, one act and its reach SHALL be committed in one transaction before any obligation is written
|
||||
- **AND** three pending obligations SHALL then be written by the listing before any organization is moved
|
||||
|
||||
#### Scenario: An outgoing position that appeared after the commit is left in place
|
||||
|
||||
- **WHEN** an act committed with no disposition and a move then finds an outgoing-default position on its pool
|
||||
- **THEN** the move SHALL settle the obligation `untouched`, change nothing on the pool, and log the act's id and the organization's id
|
||||
|
||||
#### Scenario: A failed organization is retried, not re-run
|
||||
|
||||
- **WHEN** one organization's move fails until its obligation is failed, the cause is fixed, and an operator presses Retry on the type's card
|
||||
- **THEN** that organization SHALL be moved onto the type's current default
|
||||
- **AND** the type's `default_plan_ladder_id` SHALL NOT be written again
|
||||
|
||||
#### Scenario: The card lists the organizations not moved
|
||||
|
||||
- **WHEN** a default change's obligation for one organization is failed
|
||||
- **THEN** the type's card SHALL read "1 organization not moved." with Retry beside it, that organization's name and reason below, and "Reference: " followed by the act's id
|
||||
|
||||
#### Scenario: Concurrent commits are ordered by the exclusive lock
|
||||
|
||||
- **WHEN** two operators commit the same new default for one type at the same time
|
||||
- **THEN** the second commit SHALL wait for the first to commit and SHALL then be refused with "Already the default.", writing nothing
|
||||
- **AND** no pool SHALL hold two live positions from the enactments
|
||||
|
||||
#### Scenario: A newer commit takes over the remaining organizations
|
||||
|
||||
- **WHEN** a second default change for the same type commits while the first still owes organizations
|
||||
- **THEN** from that commit on, the first act SHALL move none of the type's organizations
|
||||
- **AND** the first's open obligations SHALL be superseded by the second as its listing reaches them, and the second's disposition SHALL govern those organizations
|
||||
@@ -0,0 +1,112 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Act record enclosure
|
||||
|
||||
`core.acts`, `core.plan_act_details`, `core.plan_act_reached_org_types`, `core.plan_act_orgs_without_pool` and `core.plan_act_obligations` SHALL be enclosed: `core_writer` SHALL hold SELECT on each and no INSERT, UPDATE, DELETE or TRUNCATE on any. Five functions SHALL be the only writers of the five tables: `core.record_plan_act` (an act row, its detail row and the org types it reaches), `core.write_plan_act_obligations` (one listing batch: the supersession of older open obligations, the insert of new ones and the records of organizations without a default pool), `core.settle_plan_act_obligation` (one obligation settled with its outcome), `core.record_plan_act_obligation_failure` (one failed attempt on a pending obligation) and `core.requeue_plan_act_obligations` (a Retry's requeue). Each SHALL be created `SECURITY DEFINER`, owned by `core_owner`, and SHALL declare `SET search_path = core, pg_temp`, with `pg_temp` pinned last; EXECUTE on each SHALL be revoked from `PUBLIC` and granted to `core_writer`. `core.record_plan_act`, for every cause but a Retry, SHALL first assert that the calling transaction holds the materialization rendezvous in exclusive mode; `core.record_plan_act` for a Retry, `core.write_plan_act_obligations`, `core.settle_plan_act_obligation` and `core.requeue_plan_act_obligations` SHALL first assert that it holds the rendezvous in either mode. `core_reader` SHALL hold SELECT on `core.plan_act_details`, `core.plan_act_reached_org_types`, `core.plan_act_orgs_without_pool` and `core.plan_act_obligations`, and SELECT on every column of `core.acts` except `note`, through column grants. `core_owner` SHALL hold all privileges.
|
||||
|
||||
#### Scenario: The writer cannot write an act directly
|
||||
|
||||
- **WHEN** a session holding only `core_writer` inserts, updates or deletes a row of `core.acts`, `core.plan_act_details` or `core.plan_act_reached_org_types`
|
||||
- **THEN** PostgreSQL SHALL reject the statement on privilege grounds
|
||||
|
||||
#### Scenario: The writer cannot reopen or repoint an obligation
|
||||
|
||||
- **WHEN** a session holding only `core_writer` updates any column of `core.plan_act_obligations`, such as returning a failed row to `pending` or changing `superseded_by_act_id`
|
||||
- **THEN** PostgreSQL SHALL reject the statement on privilege grounds
|
||||
|
||||
#### Scenario: The writer settles and records failures through the functions
|
||||
|
||||
- **WHEN** a session holding only `core_writer` calls `core.settle_plan_act_obligation` inside a transaction that holds the rendezvous, and `core.record_plan_act_obligation_failure` in a plain one
|
||||
- **THEN** each call SHALL succeed and change only the named obligation
|
||||
|
||||
#### Scenario: PUBLIC cannot call the act functions
|
||||
|
||||
- **WHEN** a login role that is not a member of `core_writer`, and holds no other explicit grant, calls any of the five functions
|
||||
- **THEN** the call SHALL be rejected
|
||||
|
||||
#### Scenario: Pinned search_path resists temp-table shadowing
|
||||
|
||||
- **WHEN** a calling session has created a temporary table named `acts` or `plan_act_obligations` and then calls `core.record_plan_act` or `core.write_plan_act_obligations`
|
||||
- **THEN** the function SHALL write the real `core` table
|
||||
- **AND** the caller's temporary table SHALL NOT be written
|
||||
|
||||
#### Scenario: The core reader cannot read a note
|
||||
|
||||
- **WHEN** a session holding only `core_reader` selects `core.acts.note`
|
||||
- **THEN** PostgreSQL SHALL reject the statement on privilege grounds
|
||||
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Writer role grants read-write data access
|
||||
|
||||
The `{schema}_writer` role SHALL have `SELECT`, `INSERT`, `UPDATE`, `DELETE`
|
||||
privileges on all tables in its schema, EXCEPT for `INSERT`, `UPDATE`, and
|
||||
`DELETE` on any table that a capability spec explicitly designates as enclosed
|
||||
behind SECURITY DEFINER functions, and EXCEPT for `INSERT` and `UPDATE` on any
|
||||
table that a capability spec explicitly designates as an insert enclosure.
|
||||
For each such enclosed table,
|
||||
`{schema}_writer`'s direct INSERT, UPDATE, and DELETE privileges SHALL be
|
||||
revoked, and `{schema}_writer` SHALL instead be granted EXECUTE on the
|
||||
designated functions as its sole write path; SELECT on the enclosed table
|
||||
SHALL be unaffected. For each insert-enclosed table, `{schema}_writer` SHALL
|
||||
hold no INSERT, SHALL hold UPDATE only on the columns the designating spec
|
||||
names, and SHALL keep SELECT and DELETE; rows SHALL be inserted only by the
|
||||
SECURITY DEFINER triggers and functions that spec names. The `entitlements`
|
||||
schema's `pool_provisions`,
|
||||
`pool_provision_ladders`, and `pool_provision_transitions` tables are the
|
||||
first such enclosure — see the "Entitlements position-machinery enclosure"
|
||||
requirement in this same capability for the full grant/revoke posture and the
|
||||
five conferral functions; it is not repeated here. `core.person_cut_offs` is
|
||||
enclosed the same way, and so are the act record's five tables
|
||||
(`core.acts`, `core.plan_act_details`, `core.plan_act_reached_org_types`,
|
||||
`core.plan_act_orgs_without_pool` and `core.plan_act_obligations`). The
|
||||
obligations are a mutable work list, and they are enclosed rather than given
|
||||
column UPDATE grants so that every status change of an obligation goes
|
||||
through a function, and a transition the plan acts do not name is not a
|
||||
privilege the writer holds. `core.desired_state_outbox` is the first insert
|
||||
enclosure — see the "Person cut-off enclosure", "Act record enclosure" and
|
||||
"Desired-state outbox insert enclosure" requirements in this same capability.
|
||||
It SHALL be used by the application at runtime.
|
||||
|
||||
#### Scenario: Writer role can perform CRUD operations
|
||||
|
||||
- **WHEN** the application connects using the `organization_writer` role
|
||||
- **THEN** it SHALL be able to `SELECT`, `INSERT`, `UPDATE`, and `DELETE` rows in all `organization` schema tables
|
||||
- **AND** it SHALL NOT be able to `ALTER TABLE` or `DROP TABLE`
|
||||
|
||||
#### Scenario: Enclosed tables are excepted from the unqualified writer grant
|
||||
|
||||
- **WHEN** a capability spec designates one or more tables in a schema as
|
||||
enclosed behind SECURITY DEFINER functions (for example `entitlements`'s
|
||||
three position tables)
|
||||
- **THEN** `{schema}_writer` SHALL NOT hold direct INSERT, UPDATE, or DELETE
|
||||
privileges on those tables
|
||||
- **AND** `{schema}_writer` SHALL instead hold EXECUTE on the designated
|
||||
functions as its only path to writing them
|
||||
- **AND** `{schema}_writer`'s privileges on every other, non-enclosed table
|
||||
in the schema SHALL remain the unqualified SELECT/INSERT/UPDATE/DELETE
|
||||
grant
|
||||
|
||||
#### Scenario: Insert-enclosed tables refuse the writer's inserts
|
||||
|
||||
- **WHEN** a capability spec designates a table as an insert enclosure (for
|
||||
example `core.desired_state_outbox`)
|
||||
- **THEN** `{schema}_writer` SHALL NOT hold INSERT on that table
|
||||
- **AND** `{schema}_writer` SHALL hold UPDATE only on the columns that spec
|
||||
names, and SELECT and DELETE on the table
|
||||
|
||||
### Requirement: Reader role grants read-only access
|
||||
|
||||
The `{schema}_reader` role SHALL have `SELECT` privileges on all tables in its schema, except on a column a capability spec explicitly withholds from it, where it SHALL hold SELECT on every other column of that table through column grants. `core.acts.note`, an operator's own words, is the one such column: every integration's writer role holds `core_reader`, so a whole-table grant would show every integration an operator's note ("Act record enclosure"). It SHALL be used for cross-module read access and reporting.
|
||||
|
||||
#### Scenario: Reader role can only select
|
||||
|
||||
- **WHEN** a connection uses the `billing_reader` role
|
||||
- **THEN** it SHALL be able to `SELECT` from all `billing` schema tables
|
||||
- **AND** it SHALL NOT be able to `INSERT`, `UPDATE`, `DELETE`, or perform DDL operations
|
||||
|
||||
#### Scenario: A withheld column is not readable by the reader role
|
||||
|
||||
- **WHEN** a connection uses the `core_reader` role and selects `core.acts.note`
|
||||
- **THEN** the statement SHALL be refused
|
||||
- **AND** a select naming every other column of `core.acts` SHALL succeed
|
||||
@@ -0,0 +1,322 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Every plan-wide act records its act and its detail
|
||||
|
||||
Each of the five plan-wide acts SHALL record one act (`act-records`) with its cause: a tier add `tier_added`; a tier delete and a tier removal `tier_removed` (the detail's disposition is NULL when no organization held the tier); a reorder `tiers_reordered`, whether or not rank 0 changed; an org type's default change `org_type_default_changed`. A Retry SHALL record `plan_act_retried`. Each act SHALL write one `core.plan_act_details` row in the same transaction, holding, by id and by the name each had when the act committed: the ladder and the product of a tier add or removal; the ladder, the new rank-0 product and the previous one of a reorder that changed rank 0; the org type, the new default ladder, the previous default ladder and the target's rank-0 product of a default change (each NULL for None); the disposition the operator chose, when one was asked for; and for a Retry, the act it retried. The names SHALL be read inside the act's own transaction, and an act whose names cannot be read SHALL be refused, writing nothing. The database function that records a reorder SHALL read the ladder's rank-0 product itself and refuse a detail whose new rank-0 product is a different one.
|
||||
|
||||
#### Scenario: A cosmetic reorder is still recorded
|
||||
|
||||
- **WHEN** an operator commits a reorder that keeps the same rank-0 product
|
||||
- **THEN** one act with cause `tiers_reordered` and its detail row naming the ladder are written
|
||||
- **AND** the act reaches no org type and no obligation is written
|
||||
|
||||
#### Scenario: A tier delete is a removal with no disposition
|
||||
|
||||
- **WHEN** an operator deletes a tier no organization holds
|
||||
- **THEN** its act has cause `tier_removed` and its detail's disposition is NULL
|
||||
|
||||
### Requirement: A default change or reorder commits its change, its act and its reach in a hold that does not grow with the organizations
|
||||
|
||||
An org type's default change, and a reorder that changes the rank-0 product of a ladder one or more org types default to, SHALL commit in one transaction that holds the materialization lock in exclusive mode (`entitlements`): the catalog change (the type's `default_plan_ladder_id`, or the ladder's ranks), the act row, its detail row, and one `core.plan_act_reached_org_types` row per org type the act reaches. The database function that records the act SHALL derive the reached types from the detail row (the default change's type, or the types whose default ladder is the reordered ladder), SHALL record each type's name, the greatest organization id the type held when the commit read it, and a sequence value that a later commit always exceeds. The disposition gate SHALL be evaluated inside that transaction against the reach and the target the record names (`org-types`); a commit that needs a disposition and carries none SHALL be rejected, writing nothing. The transaction SHALL write no obligation and SHALL take no pool row, and every statement in it SHALL be a single-row write or an index probe. The gate's probe is an index probe whose first read after a mass ending walks the entries vacuum has not yet removed, after which those entries are marked dead and the probe is back to its usual cost; no other statement's cost SHALL depend on how many organizations the reached types hold. So that the walk is normally paid outside the exclusive hold, the request or drain pass that finds an act's listing complete and nothing pending, when the act settled any obligation as migrated or grandfathered, SHALL run the gate's probe once for the act's owned reached types, in its own transaction under the shared lock, before it treats the act as done; a failure of that probe SHALL be logged with the act's id and SHALL NOT keep the act's drain running. A failure anywhere in the commit transaction SHALL roll back all of it.
|
||||
|
||||
#### Scenario: The commit writes the act and its reach, not the organizations
|
||||
|
||||
- **WHEN** an operator commits a default change for a type of ten thousand organizations
|
||||
- **THEN** the default, one act, one detail row and one reached-type row are committed together
|
||||
- **AND** no obligation exists when the commit's transaction ends
|
||||
|
||||
#### Scenario: A failed commit saves nothing
|
||||
|
||||
- **WHEN** the commit transaction fails after saving the default and before it commits
|
||||
- **THEN** the type's default is unchanged and no act, detail or reached-type row exists
|
||||
|
||||
#### Scenario: The exclusive hold does not grow with the type
|
||||
|
||||
- **WHEN** the timed run commits a default change at one thousand and at two hundred fifty-six thousand organizations of one type
|
||||
- **THEN** the commit's exclusive hold at the larger size is within twice its hold at the smaller size and under 20 milliseconds
|
||||
|
||||
#### Scenario: The act that ended positions pays for the walk
|
||||
|
||||
- **WHEN** a default change with Migrate settles its last obligation, its moves having ended default-sourced positions of the type
|
||||
- **THEN** the pass that finds the act done runs the gate's probe for the type once, in its own transaction under the shared lock, before it treats the act as done
|
||||
|
||||
#### Scenario: The walk the exclusive hold can meet is bounded
|
||||
|
||||
- **WHEN** the timed run ends as many positions at the type's lowest product as autovacuum's default threshold allows for its largest size, and the next commit's gate probes the type
|
||||
- **THEN** that probe takes under 50 milliseconds, with no older snapshot open and with one held
|
||||
|
||||
### Requirement: The listing writes obligations in short batches after the commit
|
||||
|
||||
After the commit, a listing SHALL write one obligation in `core.plan_act_obligations` per organization of each reached type that has an active default pool, keyed by the act and that pool, in batches of a measured size. Each batch SHALL be its own transaction holding the materialization lock in shared mode. A batch SHALL list only the types its act owns, in (org type, organization id) order, only organizations whose id is at most the type's greatest organization id at the commit, and SHALL start after the greatest (org type, organization id) its act has already written, read from the rows written, not from a column of the act. A batch SHALL take the pool rows it lists in ascending pool id order before it marks any older obligation superseded or inserts any new one. A batch SHALL insert its rows in (org type, organization id) order, and each insert SHALL name its primary key as its conflict target, so two batches of one act that run at once write each organization once while a violation of the one-open-obligation index raises and rolls the batch back instead of being skipped. A crash between batches SHALL lose nothing: the next batch starts after the last committed row. An organization created after the commit SHALL NOT need an obligation: it is outside the bound, and one inside the bound by a clock step SHALL be settled by its move like any other organization.
|
||||
|
||||
#### Scenario: A large type is listed in batches
|
||||
|
||||
- **WHEN** a default change reaches a type of ten thousand organizations and the batch size is five hundred
|
||||
- **THEN** the obligations are written by twenty batches, each in its own transaction under the shared lock
|
||||
|
||||
#### Scenario: A listing resumes after a crash
|
||||
|
||||
- **WHEN** the process dies after three batches of an act have committed
|
||||
- **THEN** the next batch writes the organizations after the last one the third batch wrote, and none twice
|
||||
|
||||
#### Scenario: An organization created after the commit is not owed
|
||||
|
||||
- **WHEN** an organization of a reached type is created after the act's commit
|
||||
- **THEN** the listing writes no obligation for it, because its id is greater than the type's greatest id at the commit
|
||||
|
||||
### Requirement: Organizations without a default pool are recorded, logged and counted
|
||||
|
||||
An organization of a reached type that has no active default pool SHALL hold no obligation. The listing SHALL record it once in `core.plan_act_orgs_without_pool` with its act, its type and its id, and after the batch that recorded it commits SHALL write one Error log line naming the act and the organization. How many organizations an act found without a default pool SHALL be derived from those rows. The request's answer SHALL count the organizations whose record its act holds when the request answers, an organization that also holds an obligation of the act counting once, as an obligation. These organizations SHALL NOT be listed as unmoved.
|
||||
|
||||
#### Scenario: An organization without a pool is logged once
|
||||
|
||||
- **WHEN** a default change reaches a type of five organizations, four with a default pool and one without, and the request lists all five
|
||||
- **THEN** four pending obligations and one record of the organization without a pool are written
|
||||
- **AND** one Error line names the act and that organization, and the answer reports one organization without a default pool
|
||||
|
||||
### Requirement: Each obligation is settled in its own transaction against the current default
|
||||
|
||||
Each obligation SHALL be settled in its own short transaction that holds the materialization lock in shared mode and takes the pool's row, then the obligation's row, and SHALL do nothing when the obligation is no longer `pending` or when its act no longer owns the organization's type. Inside that transaction it SHALL resolve the organization's type, the type's current default ladder and that ladder's current rank-0 product, and classify the pool against that product; it SHALL NOT use a product fixed when the act committed. It SHALL apply the act's disposition to the bucket it derives in its own transaction, as the org-type default-change commit defines it (`org-types`), mark the obligation `settled` with its outcome (`initiated`, `already_at`, `grandfathered`, `migrated`, `left` or `untouched`) and commit. A pool already at the target or held by another source SHALL be settled without being touched. A move SHALL record the actor of the act that put its obligation in motion (the act's, or the Retry's that requeued it), every grant it creates SHALL carry that act's id, and its transitions SHALL carry the original act's reason.
|
||||
|
||||
#### Scenario: A reorder between the act and a move cannot make it classify against a stale rank 0
|
||||
|
||||
- **WHEN** a reorder makes Plus rank 0 of a default ladder, and before one organization's obligation is settled an operator deletes the Plus tier, which no organization holds yet, so Basic is rank 0 again
|
||||
- **THEN** that organization's move classifies it against Basic
|
||||
- **AND** an organization already on Basic is settled as already at the default, with no legacy grant
|
||||
|
||||
#### Scenario: A superseded obligation is skipped
|
||||
|
||||
- **WHEN** a pass reaches an obligation a newer act has superseded
|
||||
- **THEN** it changes nothing and writes nothing for that pool
|
||||
|
||||
### Requirement: A failed move is recorded on its obligation alone
|
||||
|
||||
A move that fails SHALL roll back, and a second, plain transaction, taking no materialization lock and no pool row, SHALL update only that obligation, and only while it is still `pending`: its attempts plus one, the time, the failing step from a closed list, and the error text. Once attempts reach the maximum the obligation SHALL be `failed`; the database function SHALL hold the maximum itself, and SHALL report that nothing was updated when the obligation was no longer pending, which the caller SHALL NOT treat as a failure to record. The failing step SHALL be recorded as `in_progress` whenever the error is a lock wait past the timeout, a deadlock or a serialization failure, and as `read_default` when the type's default or its rank-0 tier cannot be read for any reason other than its absence. A context error SHALL NOT be recorded as an attempt, and neither SHALL, inside a request, a lock wait past the request's shorter lock timeout, a transaction the database ended for passing its transaction timeout (SQLSTATE 25P04), or an error reporting a closed or broken connection once the transaction has run for its transaction timeout or the request's deadline has passed; any other connection error SHALL be recorded as a failed step. The error text SHALL go to the logs and the obligation row and SHALL NOT be rendered on any page. Every failed move SHALL be logged with its act, its organization, its pool, its step and its error, whether the request or the drain ran it. When the recording itself fails, the obligation SHALL stay pending and both errors SHALL be logged. A test SHALL compare the step codes and the outcomes in the code with the obligation table's constraints.
|
||||
|
||||
#### Scenario: A failing organization costs the others nothing
|
||||
|
||||
- **WHEN** one organization's move fails in a default change reaching three organizations
|
||||
- **THEN** the other two are settled
|
||||
- **AND** the failing one's obligation is pending with one attempt and its step recorded
|
||||
|
||||
#### Scenario: The fifth failure is the dead letter
|
||||
|
||||
- **WHEN** an obligation's move fails for the fifth time
|
||||
- **THEN** its status is `failed` and no pass picks it up again until a Retry
|
||||
|
||||
#### Scenario: Recording a failure takes no pool row
|
||||
|
||||
- **WHEN** another transaction holds the failing organization's pool row
|
||||
- **THEN** the failure is still recorded without waiting for that row
|
||||
|
||||
#### Scenario: A client that leaves costs no attempt
|
||||
|
||||
- **WHEN** the request's context ends while a move is in flight
|
||||
- **THEN** the move rolls back and the obligation's attempts are unchanged
|
||||
|
||||
### Requirement: An act with no disposition leaves a later outgoing position untouched
|
||||
|
||||
When a move under an act that carries no disposition finds a live default-sourced position that does not deliver the target's rank-0 product, it SHALL settle the obligation `untouched`, change nothing on the pool, and write one Warn log line naming the act and the organization. The disposition gate proves no such position existed at the commit, so it appeared later; a newer default change or reorder with a disposition moves it.
|
||||
|
||||
#### Scenario: A position inducted after the commit is left in place
|
||||
|
||||
- **WHEN** a default change with no disposition commits, and before an organization's move a tier add inducts that organization's default-sourced provision onto a ladder
|
||||
- **THEN** the move settles the organization `untouched` and one Warn line names the act and the organization
|
||||
- **AND** the organization is not listed as unmoved
|
||||
|
||||
### Requirement: The request settles a measured share and a drain settles the rest
|
||||
|
||||
After its commit, a default change, a reorder or a Retry SHALL, while less than a measured time budget has passed since the request began: run one listing batch, then settle pending obligations of the types its act owns in (org type, organization id) order, up to a measured count. Each of these transactions SHALL use a measured lock timeout shorter than the 5 s one and a measured transaction timeout, both set before the transaction takes the materialization lock, and the request SHALL stop settling at the first lock wait past the first, the first transaction past the second, and the first context error. A transaction past its transaction timeout ends its database session, so its connection SHALL be discarded rather than reused. The act's work in the request, its commit included, SHALL be bounded by a measured context deadline, so a commit that waits twice for a lock cannot pass the server's write timeout; the request's answer, the reads it renders and the start of the drain SHALL run on the request's own context, which that deadline does not end. The values SHALL be set from a timed run, recorded beside the constants with their measurement, so the request answers well inside its write timeout. A read the request makes after the commit and cannot complete SHALL NOT be shown as an answer: the request stops settling, starts the drain, and answers the commit's success without the counts it could not read. The listing and the obligations left pending SHALL be completed by a drain workflow for the act, started after the commit with one workflow id per act so a second start lands on the running drain, and by a poller that dispatches the drain of every act that owns an org type with work left on it, so a failed start, an absent workflow client or a dead process loses nothing. The poller SHALL log a failed read and sleep to its next tick, not end. A drain pass SHALL start after the last obligation the previous pass tried and start over when none lies after it, so an obligation whose attempt cannot be recorded does not hold back the ones behind it. A recorded attempt SHALL count as progress, and a pass that listed nothing, settled nothing and recorded nothing SHALL wait until the next poll before the next pass. Settling an obligation twice SHALL be impossible: a pass that finds it settled skips it.
|
||||
|
||||
#### Scenario: A large type answers in time
|
||||
|
||||
- **WHEN** an operator commits a default change reaching one hundred thousand organizations
|
||||
- **THEN** the request answers after one listing batch and at most the measured count of moves, within the measured budget
|
||||
- **AND** the listing and the rest of the moves are completed by the drain without the operator
|
||||
|
||||
#### Scenario: A failed workflow start loses nothing
|
||||
|
||||
- **WHEN** the drain's start fails after the commit
|
||||
- **THEN** the poller dispatches the act's drain on its next tick, and the drain completes the listing and the moves
|
||||
|
||||
#### Scenario: A read that fails after the commit is not shown as a count
|
||||
|
||||
- **WHEN** a default change commits and the request's read of the counts for its answer fails
|
||||
- **THEN** the answer reports the commit without the counts, the drain is started, and no count of zero is shown
|
||||
|
||||
#### Scenario: A pass with no progress does not spin
|
||||
|
||||
- **WHEN** a pass lists nothing, settles nothing and records no attempt, and the act's listing is complete
|
||||
- **THEN** the drain waits until the next poll before its next pass
|
||||
|
||||
#### Scenario: Failures that are recorded are progress
|
||||
|
||||
- **WHEN** every pending obligation of an act fails on a pass and each failure is recorded
|
||||
- **THEN** the drain starts its next pass without waiting, so the obligations reach their dead letters
|
||||
|
||||
#### Scenario: An obligation that cannot be recorded does not starve the rest
|
||||
|
||||
- **WHEN** recording the failure of the first obligation in order fails on every pass
|
||||
- **THEN** the next pass starts after that obligation and tries the ones behind it
|
||||
|
||||
#### Scenario: A poller outlives a failed read
|
||||
|
||||
- **WHEN** the poller's read of the acts owing work fails after its retries
|
||||
- **THEN** the poller logs the failure and reads again at its next tick
|
||||
|
||||
#### Scenario: A commit that waits too long answers the refusal
|
||||
|
||||
- **WHEN** a default change's commit waits for the rendezvous and then for the org type's row until the request's deadline passes
|
||||
- **THEN** it rolls back and answers "Another change collided with this one; try again."
|
||||
- **AND** the card renders with its lines, read on the request's own context
|
||||
|
||||
#### Scenario: A transaction timeout costs no attempt
|
||||
|
||||
- **WHEN** a move inside a request runs past the request's transaction timeout, while a statement runs or between two statements
|
||||
- **THEN** the obligation's attempts are unchanged and the request stops settling
|
||||
|
||||
### Requirement: A newer plan act takes over an older one's open obligations
|
||||
|
||||
The act that owns an org type SHALL be the act whose reached-type row for it carries the greatest sequence value. From the moment a newer default change's or reorder's commit reaches a type, an earlier act SHALL NOT move, list, requeue or show as moving or unmoved any organization of that type. When a listing batch of the newer act lists a pool on which an earlier act still owes an open (pending or failed) obligation, it SHALL first mark that obligation `superseded`, naming the newer act, and then write its own. A pool SHALL hold at most one open obligation across all plan acts, enforced by a partial unique index. One database function SHALL state which act owns a type, and every query that reads obligations through ownership and every function that supersedes or requeues them SHALL call it. The sequence that orders the reach SHALL keep a cache size of one. A superseded obligation SHALL keep its attempts and last error, SHALL NOT count as a failure, SHALL NOT be listed as unmoved, and SHALL stay in the record.
|
||||
|
||||
#### Scenario: The second change takes over the first's remaining organizations
|
||||
|
||||
- **WHEN** a default change still owes forty organizations and a second default change for the same type commits
|
||||
- **THEN** the first act no longer moves or shows those organizations from that moment
|
||||
- **AND** once the second act's listing reaches them, the forty obligations of the first are superseded, naming the second, and the second owes them with its own disposition
|
||||
|
||||
#### Scenario: An older move does not act on a pool the newer act has not listed yet
|
||||
|
||||
- **WHEN** a newer default change has committed and has not listed pool P, and a pass of the older act reaches its pending obligation on P
|
||||
- **THEN** the pass writes nothing for P
|
||||
|
||||
#### Scenario: Two open obligations for one pool are impossible
|
||||
|
||||
- **WHEN** an obligation is inserted for a pool that already holds an open one
|
||||
- **THEN** the database refuses the insert
|
||||
|
||||
### Requirement: Retry is an act that requeues one act's failed obligations
|
||||
|
||||
Each act's lines on its place SHALL offer one Retry beside its organizations not moved. Pressing it SHALL run one transaction that takes the materialization lock in shared mode first, records one act with cause `plan_act_retried`, the person who pressed it as actor and a detail row naming the act it retried, and returns that act's failed obligations on the types it still owns, and no others, to `pending` with their attempts reset and their last error cleared, pointing them at the Retry act. It SHALL then settle within the request's budget and start the drain. Each requeued obligation SHALL move the organization onto its type's current default; the catalog SHALL NOT be committed again. A Retry that finds no failed obligation SHALL write nothing. A Retry that fails SHALL answer "Failed to retry. Reference: {request id}." and log the error through the request's logger, so the line carries the same request id, except that a lock wait past the timeout or a deadlock SHALL answer "Another change collided with this one; try again."
|
||||
|
||||
#### Scenario: Retry touches only failed rows and records who pressed it
|
||||
|
||||
- **WHEN** an act owes three pending organizations and two failed ones and an operator presses Retry
|
||||
- **THEN** a Retry act naming that operator and the act is written
|
||||
- **AND** the two failed obligations return to pending and point at it, and the three pending ones are untouched
|
||||
|
||||
#### Scenario: A Retry with nothing to do writes nothing
|
||||
|
||||
- **WHEN** an operator presses Retry after the failed organizations' type was reached by a newer change
|
||||
- **THEN** no act is written and the place re-renders with its current lines
|
||||
|
||||
### Requirement: The place where an act was made lists the organizations it did not move
|
||||
|
||||
The org type's card SHALL show the default change that owns the type, and the ladder's page each reorder of that ladder that owns a type, when the act has open work: a pending or failed obligation on a type it owns, or an organization of such a type its listing has not reached. Each act SHALL get its own lines. While it has pending obligations or organizations not yet listed, the place SHALL read "N organizations not yet reached.", counting both; the count is of organizations the act has not reached, not of organizations it will move. While it has failed obligations it SHALL read "N organizations not moved." in the danger tone with Retry beside it, and below it each organization by name, linked to its page, with one sentence naming what failed for its failing step, at most 50 by name and then "And N more.", then "Reference: {act id}" with the act's full id. One query SHALL list an act's unmoved organizations for any set of acts, so the overview and a page per act can reuse it, and each place SHALL read its lines in a fixed number of statements, run in one read-only transaction at a repeatable-read isolation level, whatever the number of cards it renders. When the lines cannot be read, the place SHALL show "Pending plan changes could not be read. Reference: {request id}." in their place, never an empty readout. Acts and obligations SHALL never be deleted.
|
||||
|
||||
#### Scenario: The card names an organization that was not moved
|
||||
|
||||
- **WHEN** a default change's obligation for Acme fails five times because the default plan could not be conferred
|
||||
- **THEN** the type's card reads "1 organization not moved." with Retry beside it
|
||||
- **AND** below it Acme, linked to its page, with "The default plan could not be conferred.", then "Reference: " and the act's id
|
||||
|
||||
#### Scenario: Organizations not yet listed count as not yet reached
|
||||
|
||||
- **WHEN** a default change reaching ten thousand organizations has listed five hundred and settled fifty
|
||||
- **THEN** the type's card reads "9,950 organizations not yet reached."
|
||||
|
||||
#### Scenario: The org-types page reads a fixed number of times
|
||||
|
||||
- **WHEN** the org-types page renders three cards, two with open work
|
||||
- **THEN** it reads the open acts and their unmoved organizations in two statements, not one per card
|
||||
|
||||
#### Scenario: A failed read is not an empty readout
|
||||
|
||||
- **WHEN** the open acts of the org-types page cannot be read
|
||||
- **THEN** each card shows "Pending plan changes could not be read. Reference: " and the request's id in place of its lines
|
||||
|
||||
### Requirement: Log lines of a plan act name the act and the organization
|
||||
|
||||
Every log line written by a listing batch, a move, the failure recorder and the drain SHALL carry the act's id, and every one about one organization SHALL also carry the organization's id. The obligation's stored error text SHALL stay readable by the read-only role, keyed by the act's id.
|
||||
|
||||
#### Scenario: A failed move's log line can be found from the page
|
||||
|
||||
- **WHEN** an obligation is listed as not moved under an act whose reference reads R
|
||||
- **THEN** every log line of that obligation's attempts carries act id R and the organization's id
|
||||
|
||||
### Requirement: Tier add and tier removal refuse above a measured holder limit
|
||||
|
||||
A tier add and a tier removal SHALL each stay one transaction that reconciles every holder with the tier change. Inside that transaction, before taking any pool row, each SHALL count its holders, the organizations holding a live (active or suspended) position on the tier or a live provision of the product being added, and how many of them hold it through a live default-sourced position on a ladder, and SHALL refuse, writing nothing, when the count exceeds a limit set from a timed run of their per-holder cost, recorded beside the constant with its measurement. The refusal SHALL name the count and the limit, and when more than half of the holders hold it through an org type's default it SHALL add that changing that default with Migrate moves them in the background, and, for a removal of the rank-0 tier of a ladder their type defaults to, that reordering this ladder with Migrate does too. The comparison with the limit, the more-than-half rule and the last-tier guard SHALL each be one function that reads no row, called by the preview and by the act. The removal's preview SHALL read the same counts and render its commit disabled with the same sentence as its reason.
|
||||
|
||||
#### Scenario: A removal above the limit is refused before anything changes
|
||||
|
||||
- **WHEN** an operator commits the removal of a tier more organizations hold than the limit
|
||||
- **THEN** the answer names the number of holders and the limit, and the tier, its ranks and every position are unchanged
|
||||
- **AND** no act is recorded
|
||||
|
||||
#### Scenario: The way through is named when the default put them there
|
||||
|
||||
- **WHEN** most of a tier's holders hold it through an org type's default and the removal is refused for the limit
|
||||
- **THEN** the refusal adds that changing that default with Migrate moves them in the background
|
||||
|
||||
#### Scenario: A default-sourced provision off every ladder is not a way through
|
||||
|
||||
- **WHEN** most holders of a product hold it through default-sourced provisions that a removal under `keep` left off every ladder, and the add is refused for the limit
|
||||
- **THEN** the refusal names the count and the limit and does not say that changing a default moves them
|
||||
|
||||
#### Scenario: The preview states the limit
|
||||
|
||||
- **WHEN** an operator previews the removal of a tier more organizations hold than the limit
|
||||
- **THEN** "Apply change" renders disabled with the limit sentence as its reason
|
||||
|
||||
### Requirement: Plan acts take their locks in one order
|
||||
|
||||
Every plan act transaction except the failure recorder, which takes no materialization lock and no pool row, SHALL take the materialization lock first: in exclusive mode for the tier add, the tier delete, the tier removal and the commit transactions of the default change and the reorder; in shared mode for a listing batch, for settling one obligation and for a Retry. A transaction that takes pool rows SHALL take every one it needs in ascending pool id order, each once, before it reconciles any, and read again under those locks what the first read named. An obligation's row SHALL be taken after its pool's row. A transaction that updates several obligation rows SHALL take them in ascending pool id order. The commit transactions of the default change and the reorder SHALL take no pool row and no obligation row. One function SHALL open the catalog acts' transactions and an obligation's move and take their locks; grant acts SHALL NOT use it.
|
||||
|
||||
#### Scenario: Tier add locks its holders' pools in ascending order
|
||||
|
||||
- **WHEN** a tier add reaches two pools whose provisions were created in descending pool id order
|
||||
- **THEN** it locks the lower pool id first
|
||||
|
||||
#### Scenario: A move takes the pool before the obligation
|
||||
|
||||
- **WHEN** an obligation's move runs
|
||||
- **THEN** it takes the pool row before the obligation row
|
||||
|
||||
#### Scenario: A Retry and a superseding listing do not deadlock
|
||||
|
||||
- **WHEN** an operator presses Retry on an act's failed obligations while a newer act's listing batch runs on the same type
|
||||
- **THEN** both commit, and the Retry requeues nothing on the type the newer act owns
|
||||
|
||||
### Requirement: Every plan act names itself in its transitions in one form
|
||||
|
||||
One function SHALL build every plan act's transition reason from the act's detail row: "tier added to ladder <ladder name>", "tier removal (<product name> from <ladder name>)", "plan-ladder tier reorder (<ladder name>)" and "org-type default change (<org type key>)", with ": migrated off the outgoing default" and ": grandfathered as legacy" appended where a migration or a grandfather writes the row. A legacy grant a grandfather disposition issues SHALL be described as "Grandfathered: " followed by the reason. Rows written before this rule are not rewritten.
|
||||
|
||||
#### Scenario: A tier add records the ladder's name
|
||||
|
||||
- **WHEN** a tier is added to the ladder named Main and aligns a live provision
|
||||
- **THEN** the transition's reason reads "tier added to ladder Main"
|
||||
|
||||
#### Scenario: A reorder's legacy grant names the reorder
|
||||
|
||||
- **WHEN** a reorder with the Grandfather disposition reissues an organization's position as a legacy grant
|
||||
- **THEN** the grant's description reads "Grandfathered: plan-ladder tier reorder (Main)"
|
||||
|
||||
### Requirement: The facts a position's copies rest on cannot be changed
|
||||
|
||||
Each live position SHALL carry a copy of its pool's organization's type and of whether its provision is sourced by a grant with the reason `default`, and the disposition gate, the preview, the obligations and each act's record of a type SHALL rest on four facts staying fixed: an organization's type, a pool's organization, a provision's grant and pool, and a grant's reason. The database SHALL refuse any `UPDATE` that changes the value of `core.organizations.org_type`, `core.resource_pools.org_id`, `core.pool_provisions.grant_id`, `core.pool_provisions.pool_id` or `core.grants.grant_reason`, with a `restrict_violation` error naming the column, and SHALL accept an `UPDATE` that assigns a column its own value. Inserts SHALL be unaffected.
|
||||
|
||||
#### Scenario: Changing an organization's type is refused
|
||||
|
||||
- **WHEN** a statement updates an organization's `org_type` to a different value
|
||||
- **THEN** the database refuses it with a `restrict_violation` error naming `org_type`
|
||||
|
||||
#### Scenario: Assigning a column its own value passes
|
||||
|
||||
- **WHEN** a statement sets `org_type = org_type` on an organization
|
||||
- **THEN** it succeeds and changes nothing
|
||||
|
||||
#### Scenario: Changing a grant's reason is refused
|
||||
|
||||
- **WHEN** a statement updates a grant's `grant_reason` to a different value
|
||||
- **THEN** the database refuses it with a `restrict_violation` error naming `grant_reason`
|
||||
@@ -0,0 +1,400 @@
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Operator tier management within a ladder
|
||||
|
||||
The system SHALL provide an operator UI for adding, removing, and reordering tiers
|
||||
within a ladder. The tier view SHALL display each tier's product name, display
|
||||
category, rank, and whether any active provisions are attached at that tier. Adding
|
||||
a tier SHALL permit selecting **any published product**
|
||||
(`core.products.lifecycle_status = 'published'`); the system SHALL NOT reject a
|
||||
candidate product on the basis of `display_category` or any other type/kind label —
|
||||
the "non-plan products cannot be tiers" rejection dissolves, because no structural
|
||||
kind label remains for a tier-eligibility check to consult (Doc 41 §6.7, Decision
|
||||
138). A product becomes **position-bearing** for a ladder the instant it gains a
|
||||
tier row on that ladder; a product carrying live provisions before it was inducted
|
||||
onto the ladder is reconciled by the induction backfill described under "Tier
|
||||
membership changes reconcile existing live holders via align_conferral_shape"
|
||||
below. A product with `lifecycle_status = 'draft'` SHALL be rejected as a tier
|
||||
candidate, since draft products are never conferrable. A newly added tier SHALL be
|
||||
**appended at the end of the ladder** — the system self-assigns the next rank in
|
||||
the INSERT (no operator-supplied rank) — and the operator positions it by
|
||||
drag-and-drop. The product's publication SHALL be checked inside the tier add's
|
||||
own transaction, so the check and the insert are one transaction, and a failed
|
||||
read of the product SHALL answer the tier add's generic failure, not "Product
|
||||
not found.", which is for a product that does not exist. A tier add SHALL find,
|
||||
under its pool locks and before it aligns any provision, each holder's pool that
|
||||
already holds a live position on the ladder and each that holds more than one
|
||||
live provision of the product, and SHALL refuse, writing nothing, with a field
|
||||
error naming the organization and the product it holds on the ladder, or the
|
||||
organization holding the product more than once, all read inside the add's
|
||||
transaction. A collision the alignment raises anyway SHALL be answered with the
|
||||
tier add's generic failure, and its database text SHALL be logged, not shown.
|
||||
Adding a tier SHALL be refused, writing nothing, when more
|
||||
organizations hold a live provision of the product than the measured holder limit
|
||||
(`plan-acts`), with the refusal under the product field naming the count and the
|
||||
limit.
|
||||
|
||||
Reordering tiers SHALL be a two-step flow: dropping a dragged tier row SHALL write
|
||||
nothing — it SHALL render a server-side preview of the pending order inside the
|
||||
tier card (pending ranks visible, the rank-0 marker following the pending top
|
||||
tier), together with a consequence panel and Commit / Discard actions, so the
|
||||
pending order is server-rendered state that no swap can silently discard. When the
|
||||
pending order does not change which product holds rank 0, or the ladder is not any
|
||||
org type's default plan ladder, the panel SHALL state that the reorder affects
|
||||
display order only. When the pending order changes rank 0 of a ladder that is one
|
||||
or more org types' default, the panel SHALL classify each affected org type's
|
||||
population per the org-type default-change classification (cross-ref `org-types`, including its limit of 50 named organizations per class),
|
||||
with the candidate being this same ladder's pending rank-0 product, and the commit
|
||||
SHALL require exactly one disposition — `grandfather` or `migrate` — applied to
|
||||
the outgoing-default bucket across all affected types; a commit whose affected
|
||||
outgoing-default population is non-empty and whose disposition is absent SHALL be
|
||||
rejected with a validation error, writing nothing, including ranks. Committing
|
||||
SHALL, in one transaction that holds the materialization lock in exclusive mode,
|
||||
reconcile the submitted order against the ladder's current tiers (products no
|
||||
longer in the ladder dropped, absent tiers appended in current relative order),
|
||||
assign each tier a contiguous `rank` by its pending position (`0..N-1`, preserving
|
||||
the `(plan_ladder_id, rank)` UNIQUE constraint), record the act, and — when rank
|
||||
0 changed on a default ladder — record, inside that transaction, the org types
|
||||
that default to the ladder as the types the act reaches, and evaluate the
|
||||
disposition gate against the order it commits and those types (`plan-acts`). That
|
||||
transaction SHALL write no per-organization row. A reorder that cannot read which
|
||||
org types default to the ladder SHALL be refused, writing nothing, rather than
|
||||
treated as affecting no organization. A reorder preview, and the tier card's
|
||||
default-ladder badges, that cannot read which org types default to the ladder
|
||||
SHALL show "Failed to read the org types that default to this ladder. Reference:
|
||||
{request id}." in their place, the preview offering no commit, and SHALL NOT
|
||||
treat the failed read as no org type defaulting to the ladder. After the commit, a listing SHALL write one
|
||||
obligation per organization of the reached types that has an active default pool,
|
||||
in short batches (`plan-acts`). Each obligation SHALL then be settled with the
|
||||
same semantics, per-organization transactions, actor attribution, failure
|
||||
recording and Retry as the org-type default-change commit (cross-ref
|
||||
`org-types`), against the affected type's current default ladder's current
|
||||
rank-0 product, would-initiate pools always initiated onto it. The ladder's page
|
||||
SHALL list, per reorder, the organizations it did not move, beside a Retry
|
||||
(`plan-acts`). Discard SHALL restore the persisted order without writing.
|
||||
Ladder display reordering on the topology overview
|
||||
(`core.plan_ladders.sort_order`) is display-only and SHALL remain a single-step
|
||||
write.
|
||||
|
||||
Removing a tier SHALL renumber the remaining tiers to contiguous ranks (`0..N-1`)
|
||||
in the same transaction, so ranks never gap and a ladder with tiers always has a
|
||||
rank-0 tier. Removing a tier with **no** live holders SHALL use a confirmation
|
||||
modal, whose copy SHALL state that deleting the top tier promotes the next tier to
|
||||
the new-org provisioning default. A holder is live when its position is `active`
|
||||
or `suspended`, so a tier held only by suspended positions SHALL be removed
|
||||
through the preview and SHALL be refused by the plain delete. Removing a tier that has one or more live
|
||||
(`active` or `suspended`) `pool_provision_ladders` rows at that tier SHALL be
|
||||
**permitted, not blocked**, as a two-step flow: the Remove action SHALL render a
|
||||
server-side read-only preview inside the tier card, naming the affected
|
||||
organizations classified by the source of their position, together with Commit /
|
||||
Discard actions — nothing SHALL be written by the preview. Positions held by
|
||||
another source (subscription, operator or legacy grant, purchase) SHALL be
|
||||
reconciled on commit via `align_conferral_shape` (cross-ref
|
||||
`entitlement-conferral`), ending the junction row on the removed ladder while
|
||||
delivery of the product continues, and SHALL never be force-ended. Positions held
|
||||
through the org type default (`grant_reason = 'default'`) SHALL require exactly
|
||||
one disposition: `keep` — align-shrink identically, with the preview stating
|
||||
plainly that the organization keeps the product's entitlements AND will still
|
||||
receive the current type default at its next floor restoration (an off-ladder
|
||||
product cannot block the floor) — or `migrate` — the default-sourced position is
|
||||
ended and the floor-guarded restoration confers the current default immediately
|
||||
(resolving rank 0 after renumbering, so removing a default ladder's top tier
|
||||
migrates holders onto the promoted tier; a None default ends to baseline). A
|
||||
commit whose default-sourced population is non-empty and whose disposition is
|
||||
absent SHALL be rejected with a validation error, writing nothing. The commit
|
||||
SHALL be atomic: tier deletion, renumbering, and every holder reconciliation run
|
||||
in one transaction with affected pools locked in deterministic order, so a
|
||||
failure rolls back everything including the deletion and no provision is left
|
||||
mismatched with its product's shape. A removal whose holders number more than the
|
||||
measured holder limit SHALL be refused before anything changes, and the removal
|
||||
preview SHALL render its commit disabled with the refusal as its reason
|
||||
(`plan-acts`). A removal, with or without holders, of the last tier of a ladder an
|
||||
org type defaults to SHALL be refused under either disposition (see "The last tier
|
||||
of a live default ladder cannot be deleted"), and the removal preview SHALL render
|
||||
its commit disabled with that refusal as its reason. A removal preview that cannot
|
||||
read the org types its last-tier guard needs SHALL show "Failed to preview the
|
||||
removal. Reference: {request id}." in place of the preview, never an enabled
|
||||
commit. When the removed tier is
|
||||
rank 0 of a ladder that is one or more org types' default, the preview SHALL state
|
||||
which product is promoted to the new-org provisioning default and for which org
|
||||
types. Constraint violations on tier operations (for example re-adding a product
|
||||
already in the ladder) SHALL render as field-level validation errors — raw
|
||||
database driver text SHALL NOT reach the UI. A lock wait past the timeout or a
|
||||
deadlock in a tier add, a tier delete, a tier removal or a reorder SHALL be
|
||||
answered with "Another change collided with this one; try again."
|
||||
(cross-ref `entitlements`). A tier's `rank` is its display order within the ladder
|
||||
(rank 0 is the default product for new-org provisioning); active member
|
||||
provisions bind to a tier by `product_id`, so rank changes SHALL NOT move an
|
||||
existing member between plans.
|
||||
|
||||
#### Scenario: Operator adds a tier — it appends at the end
|
||||
|
||||
- **WHEN** an operator adds product `standard-tier` to a ladder that has tiers at
|
||||
ranks 0 and 1
|
||||
- **THEN** a new `core.plan_ladder_tiers` row SHALL be inserted with `rank = 2`
|
||||
(self-assigned), without the operator supplying a rank
|
||||
|
||||
#### Scenario: A rung held by another product is named in console terms
|
||||
|
||||
- **WHEN** an operator adds a product held by an organization that already holds another product on the ladder
|
||||
- **THEN** the field error SHALL read "Cannot add this tier; {Organization} already holds {Product} on this ladder."
|
||||
- **AND** no database text SHALL appear in it, and no tier row SHALL be inserted
|
||||
|
||||
#### Scenario: A product held twice by one organization is named
|
||||
|
||||
- **WHEN** an operator adds a product that one organization holds through two live provisions, neither on the ladder
|
||||
- **THEN** the field error SHALL read "Cannot add this tier; {Organization} holds {Product} more than once, and an organization holds one position per ladder."
|
||||
- **AND** no tier row SHALL be inserted
|
||||
|
||||
#### Scenario: A failed read of the org types is not shown as none
|
||||
|
||||
- **WHEN** an operator drops a tier on a ladder page while the org types cannot be read
|
||||
- **THEN** the preview SHALL show "Failed to read the org types that default to this ladder. Reference: " and the request's identifier, and no Commit action
|
||||
|
||||
#### Scenario: A tier held only by suspended positions is not deleted plainly
|
||||
|
||||
- **WHEN** an operator deletes a tier whose only holders are suspended positions
|
||||
- **THEN** the delete SHALL be refused and Remove SHALL lead to the removal preview
|
||||
|
||||
#### Scenario: Re-adding a product renders a field error, not driver text
|
||||
|
||||
- **WHEN** an operator adds a product that is already a tier in the ladder
|
||||
- **THEN** the form SHALL re-render with a field-level error stating the product is
|
||||
already a tier, and no raw constraint/SQLSTATE text SHALL appear
|
||||
|
||||
#### Scenario: Dropping a dragged tier row renders a preview, writing nothing
|
||||
|
||||
- **WHEN** an operator drags a tier row to a new position and drops it
|
||||
- **THEN** the tier card SHALL re-render showing the pending order and its
|
||||
consequence panel with Commit and Discard actions
|
||||
- **AND** no `core.plan_ladder_tiers` row SHALL be written by the drop
|
||||
|
||||
#### Scenario: Committing a display-order-only reorder
|
||||
|
||||
- **WHEN** an operator commits a pending order that keeps the same rank-0 product
|
||||
(or reorders a ladder that is not any org type's default)
|
||||
- **THEN** the system SHALL assign each tier a contiguous `rank` by its pending
|
||||
position (`0..N-1`), preserving the `(plan_ladder_id, rank)` UNIQUE constraint
|
||||
- **AND** existing member provisions SHALL remain attached to the same product
|
||||
- **AND** no dispositions SHALL be required or enacted, and no obligation SHALL be
|
||||
written
|
||||
|
||||
#### Scenario: Reorder that changes a default ladder's rank 0 requires a disposition
|
||||
|
||||
- **WHEN** an operator's pending order puts a different product at rank 0 of a
|
||||
ladder that is an org type's default, and organizations of that type hold the
|
||||
outgoing rank-0 product through default-sourced positions
|
||||
- **THEN** the preview SHALL classify the affected type's population with the
|
||||
outgoing-default bucket named, and the commit SHALL require `grandfather` or
|
||||
`migrate` for that bucket
|
||||
- **AND** a commit without the required disposition SHALL be rejected with a
|
||||
validation error, writing nothing — the persisted ranks SHALL be unchanged
|
||||
|
||||
#### Scenario: Committed rank-0 change enacts dispositions like a default change
|
||||
|
||||
- **WHEN** an operator commits a rank-0-changing reorder of a default ladder with
|
||||
the `grandfather` disposition
|
||||
- **THEN** the ranks, the act and the org types it reaches SHALL be committed
|
||||
together first, a listing SHALL then write one obligation per affected
|
||||
organization, and each affected org's outgoing-default position
|
||||
SHALL then be enacted with the same semantics as the org-type default-change
|
||||
commit (cross-ref `org-types`): legacy-grant reissue under `grandfather`,
|
||||
per-pool transactions, in-transaction bucket re-derivation against the current
|
||||
rank 0, per-org failure recording, and Retry
|
||||
- **AND** would-initiate pools of the affected types SHALL be initiated onto the
|
||||
new rank-0 product
|
||||
|
||||
#### Scenario: A reorder that cannot read the org types is refused
|
||||
|
||||
- **WHEN** an operator commits a rank-0-changing reorder and the org types cannot
|
||||
be read inside its transaction
|
||||
- **THEN** the commit SHALL be refused and the ranks SHALL be unchanged
|
||||
|
||||
#### Scenario: Discard restores the persisted order
|
||||
|
||||
- **WHEN** an operator discards a pending reorder preview
|
||||
- **THEN** the tier card SHALL re-render the persisted tier order and ranks,
|
||||
without writing
|
||||
|
||||
#### Scenario: Removing a tier renumbers the remainder
|
||||
|
||||
- **WHEN** an operator removes the rank-1 tier of a three-tier ladder (ranks 0,1,2)
|
||||
- **THEN** the remaining tiers SHALL hold ranks 0 and 1 after the removal (no gap)
|
||||
|
||||
#### Scenario: Removing the top tier promotes the next tier to default
|
||||
|
||||
- **WHEN** an operator removes the rank-0 tier of a ladder with more than one tier
|
||||
- **THEN** the confirmation (modal for a holder-less tier, preview panel
|
||||
otherwise) SHALL state the next tier becomes the new-org default, and after
|
||||
removal the former rank-1 tier SHALL hold rank 0
|
||||
|
||||
#### Scenario: Removing a tier with holders renders a preview, writing nothing
|
||||
|
||||
- **WHEN** an operator clicks Remove on a tier that has live holders
|
||||
- **THEN** the tier card SHALL re-render with a removal preview naming the
|
||||
affected organizations classified by position source, with Commit and Discard
|
||||
actions
|
||||
- **AND** no tier row, junction, provision, or grant SHALL be written by the
|
||||
preview
|
||||
|
||||
#### Scenario: Removal with default-sourced holders requires a disposition
|
||||
|
||||
- **WHEN** an operator commits the removal of a tier whose holders include
|
||||
default-sourced positions and supplies no disposition
|
||||
- **THEN** the commit SHALL be rejected with a validation error and nothing SHALL
|
||||
be written — the tier row and all junctions SHALL be unchanged
|
||||
|
||||
#### Scenario: Committed removal align-shrinks other-source holders
|
||||
|
||||
- **WHEN** an operator commits the removal of a tier held by a
|
||||
subscription-sourced position
|
||||
- **THEN** the tier row SHALL be deleted, remaining tiers renumbered, and the
|
||||
holder's junction row on this ladder ended via `align_conferral_shape`
|
||||
- **AND** the provision SHALL continue delivering the product (no grant,
|
||||
purchase, or subscription item is created or ended)
|
||||
|
||||
#### Scenario: Committed removal migrates default-sourced holders on request
|
||||
|
||||
- **WHEN** an operator commits the removal of a tier held by default-sourced
|
||||
positions with the `migrate` disposition
|
||||
- **THEN** each default-sourced position SHALL be ended and the floor-guarded
|
||||
restoration SHALL confer the org type's current default resolved after
|
||||
renumbering, in the same transaction as the tier deletion
|
||||
- **AND** a failure in any holder's reconciliation SHALL roll back the entire
|
||||
commit, including the tier deletion
|
||||
|
||||
#### Scenario: A removal above the holder limit is refused
|
||||
|
||||
- **WHEN** an operator previews and then commits the removal of a tier that more
|
||||
organizations hold than the holder limit
|
||||
- **THEN** the preview SHALL render "Apply change" disabled with the limit as its
|
||||
reason, and the commit SHALL be refused with the tier, its ranks and every
|
||||
position unchanged
|
||||
|
||||
#### Scenario: A tier add above the holder limit is refused
|
||||
|
||||
- **WHEN** an operator adds a product that more organizations hold than the holder
|
||||
limit
|
||||
- **THEN** the form SHALL re-render with the refusal under the product field and
|
||||
no tier row SHALL be inserted
|
||||
|
||||
#### Scenario: A tier change that waits too long is a readable refusal
|
||||
|
||||
- **WHEN** a tier add, a tier delete, a tier removal or a reorder waits past the
|
||||
lock timeout for the materialization lock
|
||||
- **THEN** the answer SHALL read "Another change collided with this one; try
|
||||
again." and nothing SHALL be written
|
||||
|
||||
#### Scenario: Any published product may be added as a tier
|
||||
|
||||
- **WHEN** an operator adds a published product carrying `display_category =
|
||||
'addon'` (or any other display category, or none) to a ladder
|
||||
- **THEN** the system SHALL insert the `core.plan_ladder_tiers` row without
|
||||
rejecting on the product's `display_category` or any other type label
|
||||
- **AND** the product SHALL become position-bearing for that ladder from the
|
||||
moment the tier row exists
|
||||
|
||||
#### Scenario: Draft product rejected as tier
|
||||
|
||||
- **WHEN** an operator attempts to add a product with `lifecycle_status = 'draft'`
|
||||
as a ladder tier
|
||||
- **THEN** the UI SHALL reject the operation with an explanatory error and SHALL
|
||||
NOT issue the INSERT
|
||||
|
||||
### Requirement: Tier membership changes reconcile existing live holders via align_conferral_shape
|
||||
|
||||
The system SHALL reconcile every live provision of a product, via
|
||||
`align_conferral_shape` (cross-ref `entitlement-conferral`), whenever the
|
||||
product's ladder-tier membership changes — a tier row is added for the product on
|
||||
a ladder, or an existing tier row for the product is removed from a ladder —
|
||||
invoking the primitive once per affected live provision rather than fabricating a
|
||||
new commercial event (a grant, purchase, or subscription item). Adding a tier
|
||||
row SHALL run an induction backfill: for each live provision of the product not
|
||||
already occupying the newly-added ladder, `align_conferral_shape` SHALL attach a
|
||||
new `pool_provision_ladders` junction row and record an `initiate` transition.
|
||||
Removing a tier row SHALL run a retirement reconciliation: for each live provision
|
||||
of the product still occupying the removed ladder, `align_conferral_shape` SHALL
|
||||
logically end that junction row and record an `end` transition; the provision
|
||||
itself SHALL remain live if it still occupies at least one other ladder or is
|
||||
otherwise valid off-ladder. Both reconciliations SHALL run in the transaction that
|
||||
changes the tier row, which holds the materialization lock in exclusive mode from
|
||||
its first statement, so no conferral runs between the read of the holders and
|
||||
their reconciliation and no holder escapes either pass. Each SHALL lock the
|
||||
holders' pools in ascending pool id order before it reconciles any of them, and
|
||||
SHALL materialize each pool once.
|
||||
|
||||
Induction SHALL carry no check for a rule-less entitlement set and SHALL warn
|
||||
nowhere: the catalog's checkpoint for that state is publication readiness and the
|
||||
operator previews that hand a rank-0 product to a population, and a rule added to
|
||||
the set afterwards now re-materializes every pool attached to it, so a pool that
|
||||
attaches to a product whose set is empty is unfinished rather than stranded.
|
||||
|
||||
#### Scenario: Adding a tier attaches newly-required rungs to existing holders
|
||||
|
||||
- **WHEN** an operator adds product `p` as a tier of ladder `L`, and pool `X`
|
||||
already holds a live provision of product `p` with no existing junction row on
|
||||
ladder `L`
|
||||
- **THEN** the system SHALL call `align_conferral_shape` for pool `X`'s provision
|
||||
of `p`
|
||||
- **AND** a new `pool_provision_ladders` row SHALL be attached for
|
||||
`(provision, L)` with an `initiate` transition recorded
|
||||
- **AND** no new grant, purchase, or subscription item SHALL be created to enact
|
||||
the attachment
|
||||
|
||||
#### Scenario: Removing a tier ends the corresponding rung for existing holders
|
||||
|
||||
- **WHEN** an operator removes product `p`'s tier row from ladder `L`, and pool
|
||||
`X` holds a live provision of product `p` currently occupying ladder `L`
|
||||
- **THEN** the system SHALL call `align_conferral_shape` for pool `X`'s provision
|
||||
of `p`
|
||||
- **AND** the existing `(provision, L)` junction row SHALL be logically ended with
|
||||
an `end` transition recorded
|
||||
- **AND** the provision itself SHALL remain live if it still occupies at least one
|
||||
other ladder or is otherwise valid off-ladder
|
||||
|
||||
#### Scenario: Adding a tier locks its holders' pools in ascending order
|
||||
|
||||
- **WHEN** an operator adds product `p` as a tier and two pools hold live
|
||||
provisions of `p`, the one with the higher pool id created first
|
||||
- **THEN** the add SHALL lock the pool with the lower pool id first, before
|
||||
aligning either provision
|
||||
|
||||
#### Scenario: Reconciliation raises on conflicting incumbency rather than silently overwriting
|
||||
|
||||
- **WHEN** `align_conferral_shape` attempts to attach a newly-required ladder for
|
||||
a provision, and another provision already holds a non-ended (active or
|
||||
suspended) junction row on that same ladder for the same pool
|
||||
- **THEN** the system SHALL surface the named error `conferral_shape_collision` to
|
||||
the operator rather than inserting a colliding row
|
||||
|
||||
#### Scenario: Induction against a rule-less set proceeds unremarked
|
||||
|
||||
- **WHEN** induction attaches live provisions of a product whose entitlement set
|
||||
carries no active rule
|
||||
- **THEN** the backfill SHALL proceed and SHALL raise no warning or refusal
|
||||
- **AND** a rule added to that set afterwards SHALL reach every pool the backfill
|
||||
attached
|
||||
|
||||
### Requirement: The last tier of a live default ladder cannot be deleted
|
||||
|
||||
Deleting a plan ladder tier, whether by the plain delete of a tier no organization holds or by the removal of a tier with holders under either disposition, SHALL be refused when it is the ladder's only remaining tier AND any org type's default plan ladder references that ladder, with copy explaining that new signups land on this ladder's bottom tier. The guard, and the plain delete's count of live holders, SHALL be read inside the transaction that would delete the tier, and a failed read of the org types SHALL refuse the deletion rather than allow it. Deleting the last tier of a ladder no org type defaults to SHALL remain allowed, because emptying a non-default ladder to rebuild it is a legitimate workflow.
|
||||
|
||||
#### Scenario: Last tier of the signup default is protected
|
||||
|
||||
- **WHEN** an operator attempts to delete the only tier of a ladder that is some org type's default plan ladder
|
||||
- **THEN** the deletion is refused with an explanation naming the org type dependency
|
||||
|
||||
#### Scenario: A removal with holders cannot empty a default ladder
|
||||
|
||||
- **WHEN** an operator commits the removal, with `keep` or with `migrate`, of the only tier of a ladder an org type defaults to while organizations hold it
|
||||
- **THEN** the removal is refused with the same explanation, and the tier and every position are unchanged
|
||||
|
||||
#### Scenario: A failed read of the org types refuses the delete
|
||||
|
||||
- **WHEN** an operator deletes the only tier of a ladder and the org types cannot be read inside the delete's transaction
|
||||
- **THEN** the deletion is refused and the tier remains
|
||||
|
||||
#### Scenario: Last tier of a non-default ladder still deletable
|
||||
|
||||
- **WHEN** an operator deletes the only tier of a ladder no org type defaults to
|
||||
- **THEN** the deletion proceeds as before
|
||||
@@ -0,0 +1,111 @@
|
||||
Tests follow `docs/testing.md`: database tests build their rows with the test kit (`world`, `dbtest`), goldens compare through `golden` and change only in a commit that names the change, concurrency is made deterministic with a side transaction and `dbtest.AwaitLockWait`, and tests meet the same complexity limit as production code.
|
||||
|
||||
Each numbered section below is one commit, and its items are that commit's parts. Every commit passes `make test` with the test stack up, which runs `make lint` first: `TestConsoleReaderColumns`, `scripts/sqlc-unused`, `scripts/deadcode.sh`, `walkthrough-coverage`, `filler-copy` and every golden. So no query, table, function, constant or workflow lands before the task of its first production caller, except a closed list's Go mirror, which lands whole with the test that compares it with its constraint; code is deleted only in the task after which nothing calls it; `scripts/deadcode-testonly.txt` gains no entry for this change; a test seam lands before the first golden that needs it; a route that mutates state is claimed by a `// covers:` line in the header of its page's walkthrough under `test/e2e/operator-walkthroughs/`, the only files `walkthrough-coverage` reads (`internal/lint/lint.go:727-759`), with the handler test that covers it named in that header's prose, as `entitlement_sets_test.go` claims the recompute Retry; a test that makes one transaction wait on another gives the waiting side the full lock timeout, so the lock queue and not a timer decides who runs first; each task edits the rows of `docs/database-locks.md` and `docs/plan-management.md` that its change makes false; and each task names the goldens and tests it changes. A golden, screen or test result that changes outside the task that names it is a stop.
|
||||
|
||||
## 1. The timed run, before any schema
|
||||
|
||||
- [ ] 1.1 Add `TestTimedPlanActCaps` to `internal/entitlements/cap_timing_test.go` beside `TestTimedSyncCap`, skipped unless `PLAN_ACT_CAP_TIMING` is set. Part A times prototypes of the new commit (T0), of the gate, of one listing batch and of the preview's read, in a scratch schema the test creates and drops in a scratch database from `dbtest`: organizations of one type, each with one default pool and one live default-sourced position (one organization per member, the personal type's shape); a position table carrying `org_type`, `default_sourced` and the partial index of design D4; the reached-type, without-pool and obligation tables of design "Schema" with their indexes, and the index on organizations by type and id. At 1,000, 4,000, 16,000, 64,000 and 256,000 organizations it times, as the median of ten runs:
|
||||
- the prototype T0: the lock timeout, the exclusive advisory lock, the org type's row `FOR NO KEY UPDATE` and its update, the act, detail and reached-type inserts with the probe for the type's newest organization, and the gate's two probes against a target every position matches, which is the case where the gate has to prove that nothing is outgoing;
|
||||
- the same gate after N of the type's positions at its lowest product have ended since the last vacuum, with N of 1,000, 10,000 and autovacuum's default threshold for the position table at that size (50 plus a fifth of its rows, 51,250 at 256,000): the first probe after the endings and the second; after a fresh set of endings, one probe in its own shared-lock transaction, as the act's last probe (design D4), followed by T0's probe; and, with a second session holding a snapshot taken before the endings, three probes in a row (the scratch tables are ordinary tables, because PostgreSQL judges a temporary table's dead rows by its own session alone, so another session's snapshot would not hold them back);
|
||||
- one listing batch of 500 organizations at the start, the middle and the end of the type: the shared advisory lock, the batch's pools `FOR KEY SHARE` in ascending order, the supersession of 500 open rows of an earlier act, and the 500 inserts;
|
||||
- the preview's read (design D4: each class's exact count and at most 50 names per class, in one statement), with a tenth of the positions at a product other than the target, so outgoing, and another tenth not default-sourced, so held by another source, under the server's default `work_mem`.
|
||||
|
||||
Part B times today's code at 50 to 3,200 holders: one organization's move under each disposition (`EnactDefault` with grandfather, migrate and initiate), a tier removal with migrate per holder (`RemoveTier`), and a tier add per holder (`AddTier`).
|
||||
- [ ] 1.2 Write the stop rule in the test's comment and log its verdict:
|
||||
- T0's median at 256,000 organizations is within twice its median at 1,000 and under 20 ms;
|
||||
- at 256,000 organizations, the gate's first probe after as many endings as autovacuum's default threshold is under 50 ms, one hundredth of the 5 s lock timeout, and so is each of the three probes while the older snapshot is held: each probe repeats the walk for as long as that snapshot stays open, which for the read-only role is at most its 15 s statement timeout in a read-committed statement or its one-minute idle limit in an idle transaction (`scripts/db-reader-role.sql:177-178`, design D4);
|
||||
- T0's probe after the act's last probe is within twice its probe before the endings, which shows that the last probe moves the walk out of the exclusive hold;
|
||||
- a listing batch's median is within twice its smallest across sizes and positions;
|
||||
- the preview read's median at 256,000 organizations, added to the request deadline's placeholder (6 s, design D14), is under the 8 s write timeout, so a 422 rebuilt after a commit that waited until its deadline still answers in time.
|
||||
|
||||
If any fails, stop before task 3 and bring the measurement back: a failed T0, gate or listing bound means the exclusive hold or the listing grows with the type, and a failed preview bound means the preview's read needs another shape or the deadline a smaller value. The run's numbers go into this commit's message; each constant cites them in the task that adds it (5.1, 6.2).
|
||||
- [ ] 1.3 No golden or suite result changes: the test skips in `make test`.
|
||||
|
||||
## 2. Ride-along fixes that stand alone
|
||||
|
||||
- [ ] 2.1 End `ListOrganizationsByType` (`internal/organization/queries/organizations.sql`) with `ORDER BY name, org_id`, regenerate sqlc, and add a test with two organizations of one name that pins their order. No golden moves (the fixtures' names are distinct); the commit says so after running the suite.
|
||||
- [ ] 2.2 `PollEntitlementRecompute` (`internal/workflows/entitlements/recompute_drain_workflow.go`) logs a failed `ListChangesOwingRecomputeActivity` and goes on to its sleep and continue-as-new instead of returning the error, because Temporal's five retries end after about fifteen seconds and nothing restarts a failed poller. Test with the Temporal test suite: the list activity fails on every attempt, and the poller ends with a continue-as-new error, not the activity's.
|
||||
- [ ] 2.3 The `entitlement-set-history` delta corrects the drain-selection sentence to the dead-letter behaviour. Add a database test beside `TestSettleObligationOnSettledObligationWritesNothing` that a `failed` obligation is not returned by `ListPendingObligations` and is returned after `RequeueFailedObligations`.
|
||||
- [ ] 2.4 Change `MessageInProgress` (`internal/entitlements/rule_change.go:93`) to "Another change collided with this one; try again." and the literal in `internal/entitlements/rule_change_test.go:490`; the other tests compare the constant. The `entitlements` and `entitlement-set-management` deltas carry the sentence. No golden moves: the plan-act goldens' copy of the old sentence comes from the plan-act handlers, which change in tasks 4 to 8.
|
||||
|
||||
## 3. One rule for an outgoing default and a pool's class
|
||||
|
||||
- [ ] 3.1 Add `internal/db/migrations/00026_outgoing_default_rule.sql` as design.md "Schema" sketches it: `org_type` and `default_sourced` on `core.pool_provision_ladders`, `core.provision_is_default_sourced`, the copy trigger function with its two triggers (`BEFORE INSERT`, and `BEFORE UPDATE OF pool_id, provision_id` with its `WHEN` clause, design D4), the backfill with `trigger_pool_provision_ladders_updated_at` disabled for its statement, `NOT NULL` on both columns, `idx_pool_provision_ladders_live_default`, `core.plan_position_is_outgoing` and `core.plan_pool_bucket` (`IMMUTABLE` SQL with no `SET` clause, so they inline), `core.outgoing_default_exists`, `core.refuse_position_source_change` with its five `BEFORE UPDATE` triggers, one per watched column, each passing the column's name (`core.organizations.org_type`, `core.resource_pools.org_id`, `core.pool_provisions.grant_id`, `core.pool_provisions.pool_id`, `core.grants.grant_reason`, each `WHEN (OLD.col IS DISTINCT FROM NEW.col)`), their grants, and a Down section that drops the five triggers before the function.
|
||||
- [ ] 3.2 List the two columns as `read` in `internal/migrate/testdata/console-reader-columns.txt`; `TestConsoleReaderColumns` passes.
|
||||
- [ ] 3.3 Queries, regenerated (design D4): `ListPlanPoolBucketsByOrgType(org_type, target, per_class_limit)` replaces `ListLivePlanPositionsByOrgType`; `ListLivePlanSourcesByPools(pool_ids)` reads what holds each listed other-source pool's positions; `GetPlanPoolBucket(pool_id, target)` and `ListOutgoingPlanPositionsByPool(pool_id, target)` replace `GetLivePlanAttachmentsWithSourceByPool`; `ListLiveTierHoldersByLadderProduct` selects `default_sourced`; `GetOutgoingDefaultExists(org_types, target)` calls `core.outgoing_default_exists` (named `Get...` so the plan-act goldens treat it as a read). The two classifying queries fold a pool's live positions into `core.plan_pool_bucket`'s four arguments through `core.plan_position_is_outgoing`, each `bool_or` inside `COALESCE(..., false)`. The two replaced queries are deleted in this commit with their last callers, so `scripts/sqlc-unused` passes.
|
||||
- [ ] 3.4 Delete `ClassifyPoolAttachments`; the Go class constants gain `BucketWouldFail` and mirror `core.plan_pool_bucket`'s five results. `buildOrgTypeChangePreview` reads `ListPlanPoolBucketsByOrgType` with a per-class limit of 50 and `ListLivePlanSourcesByPools` for the listed other-source pools, which `otherSourceLabel` labels, and does nothing else with the type (design D4). `OrgTypeChangePreview` gains each class's exact count, and its lists hold at most 50 names; `Empty`, `OutgoingDefaultSummary`, `HasOutgoingDefault`, the default change's 422 sentence and the reorder's field error read the counts, never a list's length (design D27). `operator_org_type_preview.html` and `operator_plan_ladder_tier_reorder_message.html` render the counts, "And N more." under a shortened outgoing, other-source or would-fail list (copy 46) and the disclosure label of copy 49. Every preview render (the preview endpoint, the radio re-post, the default change's and the reorder's 422) goes through this read. `enactDefault` reads `GetPlanPoolBucket` and, for an outgoing pool, `ListOutgoingPlanPositionsByPool`; `lockTierHolders` and the removal preview read `DefaultSourced`. The default change's and the reorder's commit handlers decide that a disposition is required with `GetOutgoingDefaultExists`, and still build the preview for the 422 answer. When the gate is true and the rebuilt preview's outgoing count is zero, they log an Error with the reached types and the target and answer the form's generic failure ("Failed to update the default plan. Details are in the server logs." for the default change, "Failed to reorder tiers." for the reorder), not a request for a disposition.
|
||||
- [ ] 3.5 Tests: `TestClassifyPoolAttachments_*` (`internal/entitlements/default_change_test.go`) are deleted with their function, and their cases become a database table test of `GetPlanPoolBucket` (plan-less, default at the target, default outgoing, other source, a default and another source in one pool, a None target, two outgoing provisions listed once each by `ListOutgoingPlanPositionsByPool`); `TestPlanPoolBucketMirrorsItsConstants` (`core.plan_pool_bucket` called with every combination of its four arguments returns only the Go constants, and each of them for some combination); `TestPositionCopiesMatchTheirSources` (default, legacy, operator-grant and subscription positions on two org types: every row's two copies equal a recomputation from pools, organizations, provisions and grants, including rows written before the migration and filled by the backfill, and after a provision's status changes); `TestDispositionGateAgreesWithThePreview` (a mixed pool, a None target, an all-at-target type, an other-source-only type, a plan-less type, a pool-less organization and an empty type: the preview's read finds an outgoing organization exactly when `core.outgoing_default_exists` is true, and `GetPlanPoolBucket` gives every pool the class the preview's read gave it); `TestPositionSourcesCannotChange` (an update that changes each of the five columns is refused with `restrict_violation` naming that column; an update that assigns the same value passes; inserts are unaffected); the preview, the radio re-post and the 422 of the default change, and the reorder preview, each with more than 50 outgoing, other-source and would-fail organizations: 50 names per list, "And N more.", "show 50 of N" and exact counts, read in two statements per type whatever the type's size; `operator_anatomy_test.go`, `operator_org_types_render_test.go` and `rule_less_warning_test.go`, which build `OrgTypeChangePreview` values, set the counts; a handler test for the gate-true, preview-empty case (the Error line and the generic failure); the migration's up, down and up again.
|
||||
- [ ] 3.6 Goldens: none move, because the changed and new queries are reads (`Get`, `List`), which the plan-act goldens leave out (`isActRead`, `internal/server/plan_acts_golden_db_test.go:364`), the position dump selects explicit columns, and every fixture's lists are shorter than 50; the commit says so after running the suite.
|
||||
|
||||
## 4. The act record and the tier delete on the runner
|
||||
|
||||
- [ ] 4.1 Add `internal/db/migrations/00027_acts.sql`: `core.acts` and `core.plan_act_details` with every constraint and the two detail indexes, a comment on each table (never updated), `core.record_plan_act` for every cause (the reached types are added by 00028), the grants of design.md "Schema" including `core_reader`'s column grants on `core.acts` without `note`, and a Down section. Add `'^core\.acts\.note$'` to `scripts/db-reader-role.sql`'s `personal` list, and list the two tables in `console-reader-columns.txt` with `core.acts.note` as `withhold` and every other column as `read`.
|
||||
- [ ] 4.2 Add the five `ActCause` constants to `internal/entitlements` and a test that compares them with `pg_get_constraintdef` of `chk_acts_cause` and `chk_plan_act_details_cause`. The list lands whole here because its mirror test needs every cause; four of them have no caller until tasks 5 to 7.
|
||||
- [ ] 4.3 Add `runPlanAct` (`internal/entitlements/plan_act_runner.go`): open the given kind, `find` the pools, lock them through `LockPools`, `reread` under the locks, `act`, commit; every failure comes back as a `*PlanActError` naming its step. Its input carries the constructors of the entitlements, billing and organization queriers (design D17); production passes the generated ones. Add the `RecordPlanAct` query. Add `LastTierRefusal`, the last-tier guard as a function that reads no row.
|
||||
- [ ] 4.4 Rewrite `DeleteTier` on the runner: inside the exclusive transaction, count the live holders (`active` and `suspended` positions, the removal's population), read the tiers and the org types defaulting to the ladder (a failed read refuses), apply `LastTierRefusal`, record the act (`tier_removed`, no disposition, names read inside), delete and renumber. `DeletePlanLadderTier` only parses, calls and renders: its reads before the transaction go; a lock wait or deadlock at any step answers "Another change collided with this one; try again." (#216); every failure is logged with its step. Ride-along (copy 51): its failure sentence "Failed to remove tier; see server logs." (`internal/server/operator_plan_ladders.go:821`) becomes "Failed to remove tier. Reference: {request id}." and the failure is logged through `logging.FromContext(r.Context())`. Ride-along: `CountActiveAttachmentsByTier` becomes `CountLiveAttachmentsByTier` (status `active` or `suspended`), and `CountActiveAttachmentsPerTier` also returns each tier's live count, which the tier page uses to choose between the confirmation modal and Preview removal (the column of active positions stays as it is); a tier held only by suspended positions is routed to Preview removal and refused by the plain delete.
|
||||
- [ ] 4.5 The test seam first: `OperatorPartialsConfig` gains `PlanActs`, an `entitlements.PlanActConfig` carrying the queriers' constructors the runner builds the act's queriers from (design D17), whose zero value means the generated ones; task 6.2 adds the request's limits to it. `TestPlanActsGolden`'s harness (`actFixture.prepare`) builds it with refusing wrappers, so a scenario can make a named statement fail inside the act's own transaction; a handler's querier no longer reaches the act. The fixture dump gains the act and detail sections, with act ids named by the normalizer and times dropped (design D25).
|
||||
- [ ] 4.6 Tests: role tests for per-schema-roles "Act record enclosure" on the two tables (`core_writer` cannot insert, update or delete; `PUBLIC` cannot call `core.record_plan_act`; a temporary table named `acts` is not written; `core_reader` cannot read `core.acts.note` and can read every other column; a catalog cause refuses under the shared rendezvous); constraint tests (an unknown cause, an incoherent actor, a note on a system act, a detail row of another cause, a detail shape that breaks its cause's rule); `TestDeletePlanLadderTier_*` for the guards that moved inside, including the guard refusing when the act's organization querier fails `ListOrgTypes` and the delete refusing a tier held only by a suspended position; a test of the tier page's routing for a tier held only by suspended positions.
|
||||
- [ ] 4.7 Goldens: every file under `internal/server/testdata/plan_acts/` gains the act and detail sections, empty except where an act now records. Behaviour changes on purpose in `delete-begin-fails.json` (the log line names the step and carries `request_id`; the toast and the alert read "Failed to remove tier. Reference: {request id}."), `delete-bottom-tier.json` and `delete-top-tier.json` (the act row), `delete-held-tier.json` (the holder count read inside), `delete-last-tier-of-a-default.json` (the guard read inside), `tier-change-during-conferral/a-tier-delete.json` (the act row) and `contention.json` (the delete answers the new refusal). New: `delete-org-types-unread.json` (refused; the tier remains) and `delete-suspended-holder.json` (refused: a live holder). The commit lists the files whose behaviour changed and says the rest gain only the empty sections.
|
||||
- [ ] 4.8 Docs: the tier delete's reads inside its transaction, in the "Tier add, tier removal, default change and reorder" section of `docs/database-locks.md`, and the delete's guards in `docs/plan-management.md`.
|
||||
|
||||
## 5. The tier add on the runner
|
||||
|
||||
- [ ] 5.1 Add `planActHolderCap` to `internal/entitlements/constants.go`, set from task 1's per-holder cost of the add and the removal, with the measurement and the rule in its comment (an add's or a removal's exclusive hold near half a second on the measuring host). Add `HolderLimitRefusal`, the limit and more-than-half rule as a function that reads no row, returning the sentences of copy 15 to 18 or nothing.
|
||||
- [ ] 5.2 Add `CountLiveProductHolders` (organizations holding a live provision of the product, and how many hold it through a live default-sourced position on some ladder) and the reason builder `PlanActReason` (design D19). Rewrite `AddTier` on the runner: read the product inside and refuse one that is not published (no row answers "Product not found."; any other error the generic failure), count the holders and refuse above `planActHolderCap`, lock the provisions' pools in ascending order before any alignment (#194), read the provisions again, record the act (`tier_added`, names read inside), insert the tier, read `ListTierAddCollisions` for the locked pools and refuse the first collision in ascending pool id order with a `*TierAddCollisionError` carrying the organization's and the product's names (design D15), then align each pool's provisions and materialize each pool once, with the reason naming the ladder.
|
||||
- [ ] 5.3 `CreatePlanLadderTier` only parses, calls and renders: its product read goes; a lock wait or deadlock at any step answers the new refusal (#216); every failure is logged with its step. Ride-along: `refuseTierAdd` answers a `*TierAddCollisionError` under the product field with copy 43 (a held rung) or copy 48 (the product held more than once), and answers a CF003 from the alignment, which the collision read should have prevented, with "Failed to add tier. Reference: {request id}." and logs the driver text through `logging.FromContext(r.Context())`; it no longer shows the driver text. Ride-along (copy 50): every other "Failed to add tier; see server logs." in `CreatePlanLadderTier` (`internal/server/operator_plan_ladders.go:425`) takes the same reference form.
|
||||
- [ ] 5.4 Tests: ascending lock order with provisions created in descending pool order; a table test of `HolderLimitRefusal` (below, at and above the limit; with and without more than half default-sourced; the rank-0 variant); a default-sourced provision a `keep` removal left off every ladder counts toward the limit and not toward the second sentence; the product read failing inside answers the generic failure; the collision read refuses, before any alignment and writing nothing, a pool already holding a live position on the ladder (copy 43, naming the organization and the product it holds) and a pool holding two live provisions of the product, an operator grant and a subscription, neither on any ladder (copy 48); a `refuseTierAdd` test that a CF003 alignment error answers "Failed to add tier. Reference: {request id}.", logs the driver text with the same `request_id` and shows none; a test that each other failure path of the add shows the reference and its log line carries that id; `TestTierChangesWaitForAConferral`'s tier add records the ladder's name.
|
||||
- [ ] 5.5 Goldens: `add-aligns-a-live-provision.json` (the act row; the reason names the ladder; pools lock in ascending order and materialize once), `add-begin-fails.json` (the log line; the toast and the alert read the reference form), `add-collides-with-a-held-rung.json` (refused by the collision read, so no alignment statement runs; the field error naming the organization and the product), `contention.json` (the add answers the new refusal; the tier delete's and the removal's "see server logs" lines in this file take the reference form with tasks 4 and 8), `tier-change-during-conferral/a-tier-add.json` (the act row and the reason). New: `add-above-the-limit.json`, `add-above-the-limit-default-sourced.json`, `add-product-unread.json`, `add-product-held-twice.json` (copy 48; nothing written).
|
||||
- [ ] 5.6 Docs: the tier add's ascending lock order in `docs/database-locks.md` (rule 3 without the tier add exception; "Where the paths differ" without the #194 entry) and the add's holder limit in `docs/plan-management.md`.
|
||||
|
||||
## 6. The default change on obligations, its drain, the card's lines and Retry
|
||||
|
||||
- [ ] 6.1 Add `internal/db/migrations/00028_plan_act_obligations.sql` as design.md "Schema" sketches it: `core.plan_act_reached_org_types` (the sequence keeps `CACHE 1`), `core.plan_act_orgs_without_pool`, `core.plan_act_obligations` with every constraint and index, `idx_organizations_org_type_org_id`, `core.grants.act_id` with `fk_grants_act_id`, `core.plan_act_owns_org_type`, `core.record_plan_act` replaced so it writes the reached types it derives from the detail, reads a reorder's rank-0 product from `core.plan_ladder_tiers` and refuses a detail that disagrees, and refuses a retried act that is not a default change or a reorder, `core.write_plan_act_obligations` (inserts ordered by org type and organization id, `ON CONFLICT (act_id, pool_id)` and `ON CONFLICT (act_id, org_type, org_id)`, the without-pool and obligation inserts each skipping an organization the other holds), `core.settle_plan_act_obligation`, `core.record_plan_act_obligation_failure` (the maximum attempts a constant inside, NULL returned for a row no longer pending), `core.requeue_plan_act_obligations`, their grants, and a Down section. List the new columns in `console-reader-columns.txt` as `read`.
|
||||
- [ ] 6.2 Add `planActRequestCap`, `planActRequestBudget`, `planActListBatch`, `planActRequestLockTimeout`, `planActRequestTxTimeout`, `planActRequestDeadline` and `planActMaxAttempts` to `constants.go`, each with task 1's measurement and its rule (design D14); `planActRequestDeadline` leaves the preview read's median at 256,000 organizations, and a margin, inside the 8 s write timeout. Give the shared opener its lock timeout and its transaction timeout as parameters (`MaterializingWithin(lockTimeout, txTimeout)`; `Materializing` is the case of 5 s and none), setting both with `SET LOCAL` before the rendezvous statement in `internal/entitlements/tx.go`, so there is still one opener for each side and the rendezvous wait is inside the transaction timeout. Add `PlanActLimits` (the request's lock timeout, transaction timeout, cap, budget, listing batch and deadline) to `PlanActConfig` (task 4.5), production's from the constants, so a handler test can set them (design D17). Add, beside `db.IsLockContention` in `internal/db/tx.go`, the one function that reports an error the request records no attempt for: SQLSTATE 25P04, or a closed or broken connection once the transaction has run for its transaction timeout or its context has ended (design D14).
|
||||
- [ ] 6.3 Add `internal/entitlements/queries/plan_acts.sql` and regenerate: `WritePlanActObligations`, `GetPlanActListingDone`, `LockPlanActObligation` (`FOR UPDATE OF` the obligation, with its type's owner in the same statement), `GetPoolDefault`, `SettlePlanActObligation`, `RecordPlanActObligationFailure`, `ListPendingPlanActObligations` (owned types, in org type and organization id order, after a cursor), `ListPlanActReachedOrgTypes`, `CountPlanActOrgsWithoutPool`, `ListPlanActsOwing`, `RequeuePlanActObligations`, `ListOpenPlanActsByOrgTypes` and `ListUnmovedOrganizations` (organization name, then id); and `LockOrgType` (`FOR NO KEY UPDATE`) in `internal/organization/queries`. Every one that reads obligations through ownership calls `core.plan_act_owns_org_type`. Each query that reaches a page or a golden ends its `ORDER BY` on a unique column. `CreateGrant` writes `act_id`; `ConferGrantInput` and `Actor` gain `ActID`, which `ConferGrantTx`, `ReapplyDefaultsForPool` and `ReapplyDefaultsIfVacant` pass through.
|
||||
- [ ] 6.4 Add the obligation step codes of design D18 and the mapping from an error to its code (`in_progress` for every error `db.IsLockContention` reports, `read_default` for any error but `sql.ErrNoRows` on the default's read, `resolve_default` for no rank-0 tier), with a test that compares the codes and the outcomes with `chk_plan_act_obligations_error_step` and `chk_plan_act_obligations_outcome`. Add `MigratedReason`, `GrandfatheredReason` and the legacy description `Grandfathered: <reason>`, with a table test of every cause; `humanizeTransitionReason` still recognizes every output.
|
||||
- [ ] 6.5 Rewrite `ChangeOrgTypeDefault`: T0 on the runner under the exclusive kind and the deadline's context, which is derived for the act's own calls only (design D14) (lock the type, refuse a reserved type and a candidate already the default, resolve the candidate's rank 0 and refuse a new ladder with none, save the default, record the act, and when no disposition was given run `GetOutgoingDefaultExists` over `ListPlanActReachedOrgTypes` and the detail's target, refusing with `DispositionRequiredError`), then `ListPlanActBatch`, `EnactObligation` (with the `untouched` settlement and its Warn line under an act with no disposition), the failure recorder and `DrainPlanAct` in request mode, as design.md "Flows" describes. `EnactObligation` writes its own helpers, or generalizes the old ones, so that `enactDefault` and the helpers only the reorder still reaches keep compiling until task 7.1 deletes them. `DrainPlanAct` records no attempt for an error 6.2's function reports, and runs the act's last probe (design D4) when it finds the act done after a `migrated` or `grandfathered` settlement. Every log line carries the act id, and those about one organization its id. A failed read of the listing-done state, the pending obligations or the toast's counts follows the failure table: the request stops settling, starts the drain, and answers the success toast without the parts it could not read.
|
||||
- [ ] 6.6 Add `PlanActDrainWorkflow` and `DrainPlanActActivity` (`internal/workflows/entitlements/plan_act_drain_workflow.go`, `plan_act_drain_activities.go`) and `PollPlanActs` with `ListPlanActsOwingActivity`, modelled on the recompute drain and its poller (design D14), with the pass cursor and the progress rule the design states, the act's last probe in the pass that finds the act done, and the poller logging a failed read and sleeping to its next tick as task 2.2 makes the recompute poller do; the activity takes production's `PlanActConfig`; register them in `internal/workflows/worker.go`; start the poller in `cmd/start.go` next to the recompute poller, with its own fixed workflow id; add `startPlanActDrain` in `internal/server`, modelled on `startEntitlementRecomputeDrain`.
|
||||
- [ ] 6.7 `CommitOrgTypeDefaultChange` only parses, calls and renders; it renders, rebuilds the 422 preview, reads the card's lines and starts the drain on the request's own context, not the deadline's (design D14): the 422 preview is rebuilt after the rollback through task 3's read, the toast is a success toast with the parts of copy 19 and 20, and the drain is started after the request's work. `renderDefaultChangeFailure`'s listing branch and `OrgTypeDefaults` go; ride-along (copies 58 to 61): the handlers' sentences ending "Details are in the server logs." (`internal/server/operator_org_types.go:352`, `:369`, `:418`, `:440`, `:484`, `:539`) take the reference form and their errors are logged through `logging.FromContext(r.Context())`; the card answers a lock wait or deadlock in the act's error with the new refusal through `db.IsLockContention` directly. The default select's label, its help popover's title and its help button's label become "New default" (copy 47), in `orgTypeDefaultForm` (`internal/server/operator_org_type_forms.go:49-52`), and the comments that quote the old label follow. `OrgLister`, `DefaultPools`, `EnactDefault`, `enactDefault`, `Enactment`, `EnactResult`, `EnactFailure`, `StepUndecided`, `orgTypeChangeResult`, `enactFailureReason`, `dispositionFailure` and `logEnactFailures` stay unchanged: the reorder still reaches them until task 7.
|
||||
- [ ] 6.8 The org type's card: a partial that renders an act's lines (design.md "The places"), included in `operator_org_type_card.html` after the form in place of the failure alert, loaded for every card in two statements inside one read-only `REPEATABLE READ` transaction, and showing "Pending plan changes could not be read. Reference: {request id}." when they cannot be read, with the error logged through `logging.FromContext(r.Context())`. Add the step-to-sentence function (copy 5 to 11) with a test that every step code has a sentence and no sentence carries driver text.
|
||||
- [ ] 6.9 Retry: `RetryPlanAct` in `internal/entitlements` (design D11), and `POST /partials/operator/org-types/{orgType}/plan-acts/{actID}/retry` behind `requireOperatorRole`, which refuses an act whose detail names another type, re-renders the card, answers a lock wait or deadlock with the new refusal and any other failure with "Failed to retry. Reference: {request id}.", logged through the request's logger; declare it in `internal/server/action_triggers.go`, and claim it for `walkthrough-coverage` with a `POST /partials/operator/org-types/{orgType}/plan-acts/{actID}/retry` line in the `// covers:` block of `test/e2e/operator-walkthroughs/org_types_test.go`, whose header prose names the database-backed handler test of 6.12 as the route's runtime coverage, as `entitlement_sets_test.go:4-27` claims the recompute Retry.
|
||||
- [ ] 6.10 `TestPlanActsGolden`'s fixture dump gains the reached-type, without-pool and obligation sections, and its Grants dump gains `act_id` (design D25). The refusing wrappers of 4.5 carry the scenarios that make a statement fail inside the act.
|
||||
- [ ] 6.11 Database and concurrency tests: role tests for the three new tables and five functions (the writer updates no obligation column directly; `PUBLIC` calls none; a temporary table named `plan_act_obligations` is not written; the listing refuses with no rendezvous); constraint tests (a settled obligation without an outcome, a superseded one without its successor, a second open obligation for one pool, an obligation for a type its act does not reach); a listing batch fails, writing nothing, when an older open row for a pool was not superseded; `TestPlanActOwnerRuleIsStatedOnce` (the text of each query and the definition of each function that reads obligations through ownership contains `core.plan_act_owns_org_type`); the `reached_seq` sequence's `cache_size` is 1; the recorder's maximum equals `planActMaxAttempts`, and it returns NULL for a settled row; `core.record_plan_act` refuses a reorder detail whose new rank-0 product is not the ladder's rank 0; `TestTierRemovalAgainstDefaultChange`, whose two queues-first cases are rebuilt so the side transaction holds the shared rendezvous and the pool row, as a conferral does, and whose default change gets, through `actFixture.prepare`'s `PlanActs.Limits` (6.2), the full 5 s lock timeout, no transaction timeout and a deadline above the test's own waits, so the lock queue, not a timer, decides who runs first; `TestDefaultChangeTwiceSerializesOnThePool` becomes `TestDefaultChangeTwiceRefusesTheSecond` (the second waits on the exclusive lock and is refused with "Already the default."); `TestPlanActsRefuseWhileARuleChangeHoldsTheRendezvous` records the default change's refusal at T0 and no saved default; `TestTierChangesWaitForAConferral` (`internal/server/plan_acts_concurrency_db_test.go`) gives its default change the same limits, because its conferral now runs inside the request and the tier change must queue before the conferral is released; new `TestDefaultChangeWaitsForAMoveInFlight`, `TestAnOlderActDoesNotMoveAPoolANewerActReaches` (before the newer act's listing reaches the pool), `TestRetryAndASupersedingListingTouchDisjointRows`, `TestRecordingAFailureTakesNoPoolRow`, `TestTwoListingBatchesWriteEachOrganizationOnce` (including an organization with a without-pool record), `TestListingResumesAfterItsLastRow`, `TestOrganizationCreatedAfterTheCommitIsNotListed`, and `DrainPlanAct` tests (it stops when its context ends and records no attempt for a context error; in request mode a lock wait past the shorter timeout stops it with no attempt; a transaction past the transaction timeout stops it with no attempt both when the timer fires during a statement and when it fires between two, and the test records which error the driver returns in each case and that 6.2's function reports it; the transaction timeout bounds the rendezvous wait; a request whose T0 waits past the deadline answers the contention refusal; after a Migrate act's last move, the pass that finds the act done runs `core.outgoing_default_exists` once under the shared rendezvous, and an act that settled nothing as migrated or grandfathered does not). Workflow tests with the Temporal test suite, modelled on `recompute_drain_test.go`: the workflow ends when the listing is done and nothing is pending, sleeps when a pass listed, settled and recorded nothing, does not sleep after a pass that only recorded attempts, starts each pass after the last obligation the previous pass tried (an obligation whose recording fails does not hold back the ones behind it), and continues as new; the poller treats an already-started child as success and, when its read fails, ends with a continue-as-new error, not the activity's.
|
||||
- [ ] 6.12 Handler and render tests: `TestOrgTypeDefaultChange_GrandfatherEnactmentAndRerun` (a failed organization moved by Retry, the default not written again); `TestOrgTypeDefaultChange_ConcurrentCommitsSerialize` made deterministic with the side-transaction pattern; the 422 rebuilt after the rollback reads through task 3's capped read; render tests for the card's lines (pending only, failed only, both, more than 50, organizations not yet listed, the read failure); the request's failed count read renders the toast without counts; a T0 the deadline ends renders the refusal with the card's lines and a move the deadline cut still starts the drain; the `operator_anatomy_test.go` case that wanted the card's failure alert now wants the new lines, and its want of `Change default to` (`operator_anatomy_test.go:213`) becomes `New default`; `operator_org_types_render_test.go:81` checks for "New default", and its comment at :73 quotes the new label; `TestOperatorPagesReadAFixedNumberOfTimes` covers the org-types page with open acts; the default select's refusal reads "Choose a new default from the list."; each of copies 58 to 61 renders with the request id its log line carries (preview, commit, 422 rebuild, a gate that was true with an empty rebuilt preview); `member-console lint` passes (`filler-copy`, no em dash, `walkthrough-coverage`).
|
||||
- [ ] 6.13 Goldens. The form goldens move with the label: `internal/server/testdata/forms/operator.org-type.default.html` and `internal/server/testdata/forms/per-request/org-type-default-with-disposition.html` and `org-type-default-without-disposition.html` (`forms_golden_test.go:184-185`) render "New default" as the label, the popover's `data-bs-title` and the help button's `aria-label` "Help: New default"; nothing else in them moves. Every file under `internal/server/testdata/plan_acts/` gains the three sections and the grants' `act_id` column. Behaviour changes on purpose in `contention.json` (the default change refuses at T0 with the new refusal and the type keeps its default); `default-already-at.json`, `default-migrate.json` and `default-none-migrate.json` (the act, its reach and settled obligations); `default-needs-disposition.json` (the gate rolls T0 back; the 422 is rebuilt after); `default-begin-fails.json` (nothing saved; a log line); `default-initiate-fails.json` and `default-migrate-restore-fails.json` (a pending obligation with one attempt and its step, the toast's "not yet reached", no driver text); `default-grandfather.json` (a without-pool record and its Error line, "1 without a default pool", the description "Grandfathered: ..."); `default-change-twice.json` (the second commit refused with "Already the default."); `default-current-refused.json` (the statements, and its two alerts read "Choose a new default from the list."); `default-save-fails.json` (the act's organization querier refuses the default's update inside T0; nothing saved; both alerts read "Failed to update the default plan. Reference: {request id}."); the four files under `removal-against-default/` (the default change's T0 queues on the exclusive lock; the queues-first cases follow their rebuilt mechanics; a grandfather case's legacy description); and the four files under `tier-change-during-conferral/` (`a-tier-add.json`, `a-tier-delete.json`, `a-tier-removal.json`, `a-reorder.json`): each runs a default change with Grandfather as the paused conferral, so its Grants rows read "Grandfathered: ..." and its statements follow the new default change. `default-list-fails.json` is retired, because no listing runs after the save outside the act any more, and replaced by `default-listing-fails.json` (the act's querier refuses `WritePlanActObligations`: T0 committed, a success toast counting every organization not yet reached, an Error line with the act id). New: `default-superseded.json`, `default-request-cap.json` (its fixture sets `PlanActs.Limits` to a cap of one and a budget above the run, so the cap and not a timer stops the request), `default-untouched-outgoing.json`, `default-retry.json`, `default-retry-nothing.json`, `default-preview-capped.json` (more than 50 outgoing organizations), `default-gate-true-preview-empty.json`. The commit lists the files whose behaviour changed and says the rest gain only the new sections and column.
|
||||
- [ ] 6.14 Docs: the "Plan act obligation row" and "Org type row" entries in Row locks of `docs/database-locks.md`; the Pool row entry's callers and the listing's `FOR KEY SHARE`; the opening sentence on explicit `FOR SHARE` names the listing as its exception; the section "Tier add, tier removal, default change and reorder" for the default change's T0, the listing, the move and Retry from design.md "Lock order"; rule 1 names the default change's T0 as an exclusive holder and the listing and Retry as shared ones; "Where the paths differ" with the recorder that takes no pool row. `docs/plan-management.md` "Changing the Default (Preview → Commit)": the commit records the act, a listing then writes the work and the drain moves it; failures are listed on the card with Retry instead of "fix the cause and commit again"; "already at the default" makes a Retry or a repeated pass a no-op instead of a re-run. `docs/operator-a11y-baseline.md:241` and `docs/design-system.md:818` quote the select's label as "New default".
|
||||
|
||||
## 7. The reorder on obligations and the ladder page's lines
|
||||
|
||||
- [ ] 7.1 Rewrite `ReorderTiers`: T0 on the runner under the exclusive kind (read the tiers and reconcile the submitted order inside, renumber, record the act, whose function reads the org types defaulting to the ladder when rank 0 changed, and run the gate when it reaches a type and no disposition was given), then the listing, the moves and `DrainPlanAct` as for the default change. `ReorderPlanLadderTiers` only parses, calls and renders: it runs `buildPendingTierReorder` on the commit path only to render a 422; a lock wait or deadlock answers the new refusal (#216); `tierReorderAnswer` loses its failure suffix and builds its summary from the act's result, as the default change's toast does. Now that nothing calls them, these go, and `scripts/deadcode.sh` and `unused` pass after the deletion: in `internal/entitlements`, `EnactDefault`, `enactDefault`, `enactReorder`, `Enactment`, `EnactResult`, `EnactFailure`, `ReorderEnactment`, `OrgLister`, `DefaultPools`, `LadderNames`, `StepUndecided`, `StepFindPool`, `StepListOrganizations`, and every helper of the old enactment that `EnactObligation` does not call (6.5); in `internal/server`, `orgTypeChangeResult`, `enactFailureReason`, `dispositionFailure` (whose only caller is `enactFailureReason`), `logEnactFailures`, the listing branch of `tierReorderFailure`, `OrgTypeChangeResult.Failures` and the "failed" part of `OrgTypeChangeResult.Summary` (copy 29). `OrgTypeChangeFailure` stays: the preview's would-fail list uses it.
|
||||
- [ ] 7.2 The ladder page: add `ListOpenPlanActsByLadder`, include the place partial in `operator_plan_ladder_tiers.html` below the tiers table, one act's lines per reorder that owns a type, newest first, read in two statements in one read-only `REPEATABLE READ` transaction; add `POST /partials/operator/plan-ladders/{ladderID}/plan-acts/{actID}/retry` as 6.9 describes for the card, declare it in `action_triggers.go`, and claim it with a line in the `// covers:` block of `test/e2e/operator-walkthroughs/plan_ladders_test.go`, whose header prose names the handler test of 7.3 as its runtime coverage.
|
||||
- [ ] 7.3 Tests: `TestReorderEnactsAgainstTheRankZeroItSaved` becomes `TestReorderEnactsAgainstTheCurrentRankZero` (the organization already on the current rank 0 is settled as already at the default and no legacy grant is written); `TestTierReorder_GrandfatherCommitAndRerun` (a failed organization moved by Retry, the ranks not written again); `TestTierReorder_CommitRacesDefaultChangeCommit` re-read against the new shape and its comment corrected; a reorder reaching two types followed by a default change of one of them (the reorder keeps the other type; the default change owns its type); the reorder's 422 reads through task 3's capped read; the reorder preview's "Failed to classify affected organizations." (`internal/server/operator_plan_ladders.go:578`) takes the reference form (copy 57), logged through the request's logger, with a test that the shown id is the log line's `request_id`; render tests for the ladder page's lines; `TestOperatorPagesReadAFixedNumberOfTimes` covers the ladder page with open acts.
|
||||
- [ ] 7.4 Ride-along: `orgTypesDefaultingToLadder` returns its error, and the tier card shows copy 45 in place of the default-ladder badges and the reorder preview shows it in place of its consequence panel, instead of treating a failed `ListOrgTypes` as "no org type defaults to this ladder". Its other caller, the removal preview's promoted-default note (`internal/server/operator_plan_ladders.go:907`), keeps its current behaviour in this task, logging the error and leaving the note without types, until task 8.3 rewrites the removal preview's reads; the plain delete stopped calling it in task 4. Tests: with `ListOrgTypes` failing, the tier card and the reorder preview show copy 45 with the request id and the preview offers no commit.
|
||||
- [ ] 7.5 Goldens: `reorder-an-organization-fails.json` (a pending obligation with one attempt, the toast's "not yet reached"), `reorder-begin-fails.json` (the log line), `reorder-cosmetic.json` and `reorder-rank-0-no-default.json` (an act that reaches no type), `reorder-grandfather.json` and `reorder-two-types.json` (the reach, the obligations, the description "Grandfathered: plan-ladder tier reorder (...)"), `reorder-migrate.json`, `reorder-needs-disposition.json` (the gate inside T0), `reorder-redrop.json`, `reorder-stale.json`, `stale-candidate.json` (no legacy grant; already at the default), `tier-change-during-conferral/a-reorder.json` (the reorder's own act, on top of task 6's change) and `contention.json` (the reorder answers the new refusal). Retired: `reorder-list-fails.json`, replaced by `reorder-listing-fails.json` (as `default-listing-fails.json`), and `reorder-org-types-unread.json`, whose read moved into `core.record_plan_act`, replaced by `reorder-record-fails.json` (the act's querier refuses `RecordPlanAct`; the ranks are unchanged). New: `reorder-retry.json`. The commit names which were retired and why.
|
||||
- [ ] 7.6 Docs: the reorder's T0, its listing and moves in the "Tier add, tier removal, default change and reorder" section of `docs/database-locks.md`, and the reorder's commit and Retry in `docs/plan-management.md`.
|
||||
|
||||
## 8. The tier removal on the runner
|
||||
|
||||
- [ ] 8.1 Rewrite `RemoveTier` on the runner: count the holders and refuse above `planActHolderCap` through `HolderLimitRefusal` (with its rank-0 variant when the tier is rank 0 of a ladder the holders' type defaults to); refuse the last tier of a default ladder through `LastTierRefusal` under either disposition, inside the transaction; read the product and ladder names inside, refusing when they cannot be read; record the act (`tier_removed`, the disposition); then the existing reconciliation, with the act id on every grant the migrate floor creates.
|
||||
- [ ] 8.2 `CommitPlanLadderTierRemoval` only parses, calls and renders, logs every failing step through the request's logger, `tierRemovalFailure` and `tierRemovalRefusalOr` take the request, and the four sentences "Removal failed and nothing was changed: could not ... Details are in the server logs." (`internal/server/operator_plan_ladders.go:1098-1107`) end "Reference: {request id}." (copies 53 to 56), answers "{Product} removed from the ladder." when nothing was left to reconcile, and answers a lock wait or deadlock with the new refusal (`tierRemovalRefusalOr`).
|
||||
- [ ] 8.3 The removal preview reads the holder counts and the org types with reads that fail visibly, calls the same two functions, and renders "Apply change" disabled through the disabled-control part with the sentence as its reason; when the org types cannot be read it renders "Failed to preview the removal. Reference: {request id}." in place of the preview, logged through the request's logger; "Failed to load the tier's holders. Details are in the server logs." (`internal/server/operator_plan_ladders.go:864`) takes the reference form (copy 52). Render tests for both reasons and the read failure.
|
||||
- [ ] 8.4 Tests: the last-tier refusal under keep and under migrate; the limit refusal; the toast with no holders left; a migrate floor grant's `act_id`; `TestTierRemoval_*` for the guards that moved inside.
|
||||
- [ ] 8.5 Goldens: `remove-begin-fails.json`, `remove-second-pool-fails.json` (every failing step logged with `request_id`; the alerts read the reference form of copy 55), `remove-keep-rank-0.json`, `remove-migrate-pool-held-elsewhere.json`, `remove-migrate-rank-0.json` (the floor grant's `act_id`), `remove-needs-disposition.json`, `remove-other-source.json`, `remove-refused-disposition.json`, `remove-two-pools.json` (the act row), `remove-last-tier-keep.json` and `remove-last-tier-migrate.json` (refused with the last-tier sentence; its failure alerts read the reference form of copy 55), `remove-no-holders-left.json` ("{Product} removed from the ladder."), `remove-product-name-unread.json` (the act's billing querier refuses the name inside the transaction: refused, nothing changed), the four files under `removal-against-default/` (the removal's side of the rebuilt queues-first cases), `tier-change-during-conferral/a-tier-removal.json` (the removal's act rows, on top of task 6's change), and `contention.json` (the removal answers the new refusal). New: `remove-above-the-limit.json`.
|
||||
- [ ] 8.6 Docs: the removal's holder count, last-tier guard and ascending locks in the "Tier add, tier removal, default change and reorder" section of `docs/database-locks.md`, and the removal's holder limit and last-tier refusal in `docs/plan-management.md`.
|
||||
|
||||
## 9. Docs that describe the new structures
|
||||
|
||||
- [ ] 9.1 `docs/models/plan-ladders-transitions.md`: the map rows for ladder management and org-type default changes name the acts, the listing, the obligations and the drain; invariant 17 states that every plan act's reason comes from one builder and names the ladder; the `core.superseded_ends` row says grants already point at their act and transitions will once #179 replaces their writers; the removal's last-tier refusal is true for both paths.
|
||||
- [ ] 9.2 Add `docs/models/acts.md` (Purpose, Where this lives, Invariants, Dimensions, per `docs/models/README.md`'s anatomy and style contract) for the act record: the header, the causes, the actor triple, the note and who may read it, the request id, the detail tables, the reached types and ownership (one function states it), the rows that point at an act, the obligations and their statuses, the enclosure; and one sentence that the record is never purged, so the obligations' foreign key to pools and the grants' foreign key to acts keep a pool, and so its organization, from being deleted, and that the guard on an organization's type keeps an act's obligations on the type that was reached. List it in `docs/models/README.md`.
|
||||
- [ ] 9.3 `docs/models/entitlements.md`: `core.grants.act_id`, and the position table's `org_type` and `default_sourced` copies with the rule they serve and the guard that keeps their four sources fixed.
|
||||
- [ ] 9.4 `docs/go-conventions.md` "Where a rule and an act live": the plan acts' runner is the pattern for an act that locks several pools, and `core.plan_position_is_outgoing` and `core.plan_act_owns_org_type` are examples of a rule SQL and Go both read.
|
||||
- [ ] 9.5 `status/design-feedback.md`: the act record and its detail tables are not in the design's data model; the transitions' actor convention differs from the audit triple the act uses; the position table carries two copies the data model does not list.
|
||||
|
||||
## 10. Landing
|
||||
|
||||
- [ ] 10.1 With the test stack up, `make test` passes (it runs `make lint` first), and `openspec validate plan-act-obligations --strict` passes.
|
||||
- [ ] 10.2 Re-read this comment against what landed, correct anything that no longer holds, show it before posting, and post it on #78:
|
||||
|
||||
> Changes to an org type's default plan or to a ladder's tier order record every organization they still have to move (#187). The ones that could not be moved are listed on the org type's card or the ladder's page, next to a Retry button. That gives this overview one kind of item: "organizations a plan change could not move", with its line linking to that card or page. A page for each change, listing every organization it reached and how each one ended, would be a better place for that line to link to, and the records #187 keeps are enough to build it.
|
||||
|
||||
- [ ] 10.3 Draft a closing comment on #187 under CONTRIBUTING.md's rules for forge text (self-contained, the console's terms explained once, no internal shorthand), stating how each part of its "Done when" is met and what it leaves to #237, #238, #239, #82 and #179; show the draft before posting, post it, and close #187.
|
||||
- [ ] 10.4 Comment on and close #194 (tier add locks its pools in ascending order) and #216 (every plan act answers a lock wait with the refusal), each naming the commit that fixed it; comment on #82 that the obligation row is where its per-organization disposition goes; comment on #239 that the sentences of the tier add, tier delete, tier removal and its preview, reorder preview and default change already carry the reference (design.md copies 50 to 61), so its list keeps only the console's other sentences. Show each draft before posting.
|
||||
Reference in New Issue
Block a user