2f1803d1e678ebdecd80ce219cc263befa69dfd2
260
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 cgalo5758
|
2f1803d1e6 |
Mark a subscription that delivers nothing with a badge, on the organization page and in the subscriptions list
Section 8 of #212's change, after review: the Billing card's muted line under the Subscriptions row becomes an amber "Not delivering" badge on the row's value line, after the state badge, with a title that names the ladders and the holder ("Billing continues while a grant holds Demo Ladder."; with several subscriptions "Billing continues on <k> of <n> subscriptions while ..."). The operator's subscriptions list marks each displaced subscription's row with the same badge and title, read with one grouped statement per page and turned into titles by the same steps the card uses. A failed read on the list says "Failed to load deliveries." once above the table and marks no row. A paused subscription no longer counts: it generates no invoice, so "Billing continues" would be false; it is marked once it is active again. The grant form's warning now reads "Products marked ⚠ would supersede a tier a subscription pays for; the subscription keeps billing and delivers nothing on that ladder until the grant ends." The billing view it named before can change nothing. A Stripe-paused subscription's tier is still marked, an accepted edge until #269's change, which pauses a displaced subscription's billing and rewords the badge and the warning. Goldens: 16 change, each as item 8.9 lists: the organization page's JSON goldens rename DisplacementNote to DisplacementTitle, the two subscriptions-list goldens gain the title field, three HTML goldens carry the new warning, and grant-over-subscription.html the badge in place of the line. No other golden changed. Breaks (scripts/mutation-check, 31 of 31 CAUGHT): - paused restored - not_delivering removed from badgeMap - badge tone - list badge without title - card title never set - card template without badge - account form never chosen - n read as displaced count - subscription holder as grant - several holders as one - ladders joined with commas - sort removed - ladders not deduplicated - holders not deduplicated - status filter dropped (card) - status filter dropped (list) - list row in account form - list row in account form (list vs card) - list titles never set (the field set to strings.TrimPrefix of the title by itself, an empty string, because removing the line leaves titles unused and does not compile) - delivering row given the displaced title - list template without badge - list pool not default - list reads moved into the row loop (emulated: an inline closure calls the title read once per row; a helper only the break called would fail deadcode) - list DisplacementError assignment and L4 removed - O4 template line removed - O4 line moved under the table - rows dropped on a failed read - badge label - card badge without title - delivering row given an entry - old warning restored Hand capture on the Stripe sandbox (API 2025-02-24.acacia): a member subscribed to a $20/month tier through Checkout; an operator issued a complimentary grant of the tier above; an edit to the subscription at Stripe then reconciled it, and the reconcile logged "reconcile: product not conferred; another plan holds its ladder". The Billing card and the subscription's list row each showed Active and Not delivering at 1280 and at the 390 mobile width, with no sideways page scroll; the badge's title read by script: "Billing continues while a grant holds Demo Ladder." Captures under test/screens/displaced-subscription-waits/. The suite on reset databases passed on this section's tree, 4711 tests, 14 skipped; make screens changed no screen this section owns, and every registered form was captured. The review's last fixes (the list test's row checks now fail when a row is missing, two comment rewraps, task wording) came after it; the list test and its five breaks were rerun on them: pass, 5 of 5 CAUGHT. make lint passes; openspec validate --strict passes. |
||
|
cgalo5758
|
701ab7e5b9 |
Refuse in the database a plan tier whose product has no entitlement set
Second section of #153's change. Migration 00035 adds two trigger functions and two triggers on core: - On a plan ladder tier insert, or an update of its product, the tier's product is read FOR SHARE and the row is refused with chk_plan_ladder_tiers_product_has_set when the product has no entitlement set. - On an update that clears a product's entitlement_set_id (from a set to NULL; a product already without one is left alone), the update is refused with chk_products_tier_keeps_set when any ladder tier names the product. The product row taken FOR SHARE by the tier trigger is a new lock; a tier add and a set clear on one product cannot both commit. That holds under READ COMMITTED with the trigger functions VOLATILE, which the migration header and docs/database-locks.md now state; the product edit's write runs at the server's default isolation today, and item 4.6 opens it explicitly READ COMMITTED in section 4. docs/database-locks.md gains the product row, the tier add's order and the second FOR SHARE exception; docs/models/product-catalog.md invariant 13 and docs/models/identity-organization-workspace.md invariant 7 state the rule, with the set id that names no set (#99) as the case it does not cover; docs/models/plan-ladders-transitions.md names it on the tiers row. Tests (internal/db/tier_products_keep_a_set_test.go): TestATierOfAProductWithNoEntitlementSetIsRefused, TestClearingTheSetOfATierProductIsRefused, TestATierAddAndAClearedSetCannotBothCommit (both orders), TestTierProductsKeepASetMigrationUpDownUp. Race run (2.3): `go test -race -p 1 -count=1 -run '^TestATierAddAndAClearedSetCannotBothCommit$' ./internal/db` after test/reset-test-db.sh: ok (1.126 s). The review's blocker: T2 renamed a product whose set was already empty with SET name alone, which the trigger's UPDATE OF entitlement_set_id never sees, so the break on the trigger's WHEN passed. The statement now names the column too, and the break is caught. Breaks (scripts/mutation-check, command records on the migration, each preceded by a test-database reset; 14 of 14 CAUGHT): - tier check never raises (T1) - tier check never raises (T3) - tier check raises for every product - tier trigger on inserts only - tier check constraint name changed - tier lock removed (set clear returns) - tier lock removed (tier insert returns) - WHEN refuses a replacement - WHEN refuses the state - product check ignores the tiers - product trigger not created (T2) - product trigger not created (T3) - product check constraint name changed - Down leaves the product trigger and function The migration adds no relation or column, so scripts/db-reader-role.sql needs no re-run. make lint passes; openspec validate --strict passes. The suite on reset databases: 4699 pass, 14 skip; two failures, both outside this section: TestEveryRegisteredFormWasCaptured read the capture list from before the previous commit's capture and passes after it, and TestPlanLaddersWalkthrough timed out once on its create toast (#222) and passed twice alone. make screens: no change from this section. |
||
|
cgalo5758
|
710591aad7 |
Give the demo's add-ons an entitlement set
The demo seed's three products on no ladder (demo-addon, demo-usage,
demo-onetime) had no entitlement set. Since the previous commit the Add
tier picker leaves out a product with no set, so the demo ladder had
nothing to add: the screen capture never rendered the tier add form, and
TestEveryRegisteredFormWasCaptured failed once make screens wrote a new
capture list. That commit's suite ran before the capture and did not see
it.
The add-ons now share a second demo set, demo_addons ("demo-addons"),
with the same two rules as demo_baseline (a limit of 5 on demo.widgets,
a boolean on demo.feature-x), created and found by its key like the
first and never written over. An add-on with no set is also #272's
fault in the demo's own data: a member could buy one and receive
nothing.
The seed golden, the demo README's catalog and counts, the identifiers
page and the demo seed spec's catalog requirement (a delta in
tiers-need-an-entitlement-set) say two sets.
make screens: the ladder page's open and invalid captures show the Add
tier form again, with the add-ons in its list; the Entitlement sets list
and the products list show demo-addons; the add-ons' product page reads
ready to grant. TestEveryRegisteredFormWasCaptured passes after the
capture. The suite ran on this tree with the next commit's migration;
TestPlanLaddersWalkthrough timed out once waiting for its create toast
(#222) and passed twice alone.
|
||
|
cgalo5758
|
8fdecf0e0e |
Stop tiering products with no entitlement set in tests, and leave them out of the tier picker
First section of the change that makes a product with no entitlement set unable to be a plan tier (#153). The change's proposal, design, tasks and spec deltas come in with it. Fixtures: seven tests that made a product with no set a ladder tier give it one (productWithSet in internal/db, mkTierProduct in internal/server, tierProduct in the merchant-disclosure sheet). The sheet is a capture make test does not run; its edit was checked by reading and go vet. The test kit's WithoutSet comment says that from migration 00035 on a tier's set cannot be cleared. The failure the plan-act goldens inject moves from a product with no set to a product made draft after it is laddered, which core.confer refuses at the same statement and step. Four goldens under internal/server/testdata/plan_acts change only there: the product reads Drafted instead of NoSet, and the driver text reads "is draft and not conferrable". TestResumption_ARefusedConferralIsRolledBackAlone drafts Plus the same way. The tier picker: tierCandidates keeps a product that is published, has an entitlement set and is not on the ladder, in the order given; availableTierProducts calls it. The Add tier panel's empty state keeps its first sentence and says "A tier needs a published product with an entitlement set that is not on this ladder yet." A posted product the list left out is refused with "Choose a product from the list." make screens: operator-plan-ladder-detail's open and invalid captures change at both widths. The demo's three products on no ladder have no set, so the demo ladder has nothing to add and the open panel shows the empty state. The other changed screens are the known data-only differences and the host's time zone (#274). Read-only counts on the test stack's app database before any reset: no default ladder starts on a product with no set, no plan act waits for one, and the two tiers with no set sat on a ladder the browser walkthroughs had left behind. Breaks (scripts/mutation-check, 11 of 11 CAUGHT): - default-initiate-fails without its draft - default-migrate-restore-fails without its draft - remove-second-pool-fails without its draft - reorder-an-organization-fails without its draft - the draft removed from the refused conferral test - the tier filter without the set - the tier candidates in reverse order - the empty state's old second sentence restored - the tier add accepts a product the list left out - the tier filter without the lifecycle - the tier filter keeps a product on the ladder The seven fixture edits and the WithoutSet comment carry no break: they remove a state section 2's triggers forbid, and section 2's run is their test. make lint passes; openspec validate --strict passes; the suite on reset databases passes, 4694 tests, 15 skipped. |
||
|
cgalo5758
|
8c0fb4ae42 |
Archive the suspended-plans and reconcile pool-lock changes
reconcile-takes-pool-lock (#193) and suspended-plans-stay-held (#148) move to openspec/changes/archive/ with every task ticked, and their deltas are synced into the main specs: - plan-transitions: 1 requirement modified (#193). - member-dashboard, member-entitlement-view, member-offer: 1 modified each; member-plan-upgrade: 1 renamed ("active" to "live" ladder attachments) and 1 modified; member-invoice-history: 2 added; member-product-discovery, stripe-subscription-creation: 1 added each; plan-enrollment-administration: 1 added, 3 modified; stripe-subscription-management: 1 added, 2 modified (#148). What #148's research and sandbox check taught, written into the pages that own each topic: - docs/models/plan-ladders-transitions.md: the member position's readers, all eight, each holding a suspended position as the organization's plan. - docs/models/payments-billing.md: pausing collection is not the paused status, so the plan keeps delivering. - docs/stripe.md: an expired pending upgrade's invoice is voided; a period-end cancellation stops retrying only that subscription's open invoice; only a subscription made outside the console reaches paused; paying on Stripe's page leaves the subscription's own default card in place; an uncollectible invoice's page still takes payment. - docs/testing.md: what a break record's wanted message must be, and how to drive a failed renewal in the Stripe sandbox. - status/design-feedback.md: the data model delivers through a past_due grace period, the console suspends at once (#255). - The archived design gains three reasons: why the immediate-cancel refusal comes first, the https-only redirect, and the paused badge's tone. |
||
|
cgalo5758
|
335aa85738 |
Record the landing checks for displaced subscriptions
make test passes on the tree of the six commits before this one (it runs make lint first; 57 packages ok, none failed), and openspec validate displaced-subscription-waits --strict passes. Race run (4.9): `go test -race -p 1 -count=1 -run '^TestReconcile_AGrantIssuedWhileTheReconcileWaitsIsSeen$' ./internal/fulfillment` after test/reset-test-db.sh: ok. The section's own commit says only that the test also passed under -race; this is the run the item names. Breaks, every section CAUGHT: section 1, 6 of 6; section 2, 2 of 2; section 3, 8 of 8 (the grant_id clause of the pairs view has no break, for the reason item 3.4 gives); section 4, 27 of 27; section 5, 11 of 11; section 6, 14 of 14. The organization page's line for a subscription a grant holds the ladder against was also checked by hand on the Stripe sandbox, a state the screen harness cannot reach: a member subscribed through Checkout, an operator issued a grant over the same tier, and the Billing card read "Delivers nothing on Demo Ladder, which a grant holds" at desktop and mobile widths. Items 7.3 to 7.6 (the read-only count on a deployment, the comments after the push, the candidate issues, the archive) are left open. |
||
|
cgalo5758
|
04862abe9c |
Say on the organization page which ladder a subscription delivers nothing on
An operator could not see that a subscription bills while another plan holds the ladder it pays for. The organization page's Billing card now says it under the subscriptions row, from the same rule the reconcile uses: the page reads the account's displacements, the subscriptions' current items and their rungs once per account, and calls fulfillment.RungsByProduct and DisplacedProducts over them, so a subscription whose item moved off the ladder reads no line. The line, for one subscription: "Delivers nothing on <ladders>, which a grant holds" (or "which another subscription holds", "which another plan holds", "which other plans hold"); for several: "<k> delivers nothing on ..." or "<k> deliver nothing on ...". Ladders are distinct, sorted by name, joined with joinWithAnd. A canceled or incomplete subscription reads no line. A failed read of the displacements, the items or their shapes logs and shows "Failed to load deliveries." in the line's place, styled as the page's other errors, never an absent line. Queries: ListAccountSubscriptionDisplacements (entitlements, one statement per account with ladder names by join) and ListCurrentSubscriptionItemsByBillingAccountID (billing). The page's read count stays fixed per account (a read-count row grows displaced subscriptions), and TestOrgEnrollmentReadFailures fails each new read. The template uses trim markers, so every existing HTML golden is unchanged; the new state grant-over-subscription shows the line, and eight JSON goldens gain the two empty fields. plan-management.md, stripe.md and plan-ladders-transitions.md now say the organization page reads the rule. Tests: TestDisplacementNote, TestDisplacedOnPage, TestSubscriptionMayBeDisplaced, TestDisplacementLineFor (a canceled subscription reads no line), the golden state, the read-count and read-failure rows. Breaks (scripts/mutation-check, 14 of 14 CAUGHT): - DisplacementNote never set - the template without the line - a subscription holder worded as a grant - several holders worded as one - the ladders joined with commas only - the plural form never chosen - the sort removed - the ladders not deduplicated - the holders not deduplicated - canceled admitted - displacedOnPage without the current products - the displacement read in a loop over the subscriptions - the DisplacementError assignment and the L3 log line removed - the status filter at the call site removed make screens: no screen shows the line (the demo seed holds no displaced subscription). Fourteen screens differ from the baseline, none from this change: the host's time zone moved from Eastern to Central, and the captures render pinned times in the host's zone, so every time column reads an hour earlier; the Stripe integration page's wall-clock "last read back" line and the five data-only differences seen on every run tonight are the rest. The suite on reset databases passes, 4686 tests, 15 skipped. make lint passes; openspec validate --strict passes. |
||
|
cgalo5758
|
e3f3ec56b9 |
Confer the higher tier when one subscription pays for two of one ladder
A subscription whose items name two tiers of one ladder (Plan and Plus,
say) conferred whichever its live read left, so the member could keep
the lower tier while paying for both. The item pass now runs
chooseTiers over the desired products, after the displacement rule has
left its products out: it walks them by their highest rank, keeps a
product whose ladders no kept product has taken, and outranks the rest;
a tie goes to the lower product id, and a product on no ladder is always
kept. Each outranked product is logged once at Warn ("reconcile:
product not conferred; a higher tier of the subscription holds its
ladder", with the product, the ladder and the product that took it). The rank compared is
a product's top rank across its ladders, so a bundle at rank 3 on one
ladder outranks a tier at rank 2 on a ladder they share.
reconcile_trace/two_tiers_one_ladder flips on purpose: Plus stays the
ladder's position, no core.confer, the Warn line names Plan. The
productsLeftAlone helper now receives the rungs the item pass read once,
which keeps the statement order of section 4's goldens unchanged.
docs/models/payments-billing.md states the choice in the Reconciler row.
Tests: TestChooseTiers (two tiers keep the higher, tiers of two ladders
are both kept, a bundle outranks a tier on one of its ladders, a higher
tier takes a shared ladder from a lower bundle, a tie, an add-on always
kept), each row in the given and the reversed order; T10
TestReconcile_TwoTiersOfOneLadderConferTheHigher with Plan then Plus and
Plus then Plan.
Breaks (scripts/mutation-check, 11 of 11 CAUGHT):
- the lower tier walked first (TestChooseTiers, T10)
- every product kept (TestChooseTiers, T10)
- the tie broken the other way
- a product on no ladder left out
- the Warn line removed
- top from the last rung
- any held ladder blocks
- a bundle never left out
- the left-out skip dropped (T1; section 4's skip break restated against
this section's filter)
The first run reported four records WRONG-MESSAGE because the test
printed each row's order inside the parentheses; the test now prints
the row's name alone there.
The suite on reset databases passes, 4652 tests, 15 skipped; after the
review's fixes (test output, comments, a rename), ./internal/fulfillment
passes. make lint passes; openspec validate --strict passes.
|
||
|
cgalo5758
|
76ff9d693a |
Leave a ladder alone while another plan holds what it took
#212: when a grant or another source took a ladder from a paying subscription, the subscription's next reconcile conferred its tier again and ended the plan that held the ladder. Now the reconcile reads which ladders the subscription was displaced on and leaves those alone while the displacer still holds them. Migration 00034 adds core.subscription_displacements(subscription, pool): for each ladder, the subscription's latest episode; it is a displacement when that episode ended in a supersession pair (core.supersession_pairs) whose successor is another source, and it is returned while an active or suspended, non-default position of another source holds the ladder. The function returns the holder's provision and kind (grant, subscription, purchase). A default-sourced position never holds against a subscription. fulfillment.DisplacedProducts is the one rule over those rows: it reads no database row, takes the desired products' rungs (ListConferralShapes, fulfillment.RungsByProduct) and the displacements, and returns the products to leave alone whole, any of whose rungs is on a displaced and held ladder. The item pass reads both under the pool row, confers the rest in ascending product id, and logs each left-out product once at Info ("reconcile: product not conferred; another plan holds its ladder", with product, ladder, holder provision and kind). Add-ons and other ladders are conferred; live products only update their quantity. When the holder ends, the subscription's next reconcile confers its tier again. Goldens: 29 trace goldens change. Four flip on purpose to the left-alone answer (grant_over_subscription, grant_over_suspended_recovery, second_subscription_over_first, downgrade_lands_while_displaced): the Info line, no core.confer, no materialization, the holder's provision still active. The other 25 gain only the two reads. The trace kit names provisions by product and source. TestReconcile_SupersededWhileSuspended NotResurrected now wants no live provision after the recovery. Tests: nine TestSubscriptionDisplacements_* on the function (a grant over the subscription, an extension still holds, the default does not hold, a successor with no from side, an episode the subscription ended itself, the subscription's own switch, the latest episode decides, a second subscription, one ladder of a bundle); twelve on the reconcile (a grant over the subscription and one of its own tier keep the ladder, a marked grant is not replaced, the subscription delivers again over the default and on a vacant ladder, a grant issued over the default and an extended grant keep the ladder, a subscription bought over a grant supersedes it, a later subscription keeps the ladder, a grant issued while the reconcile waits is seen (also under -race), a bundle displaced on one ladder is left alone, an item back on a ladder it left supersedes the grant); a downgrade that lands while displaced is applied and not conferred (sweep); TestDisplacedProducts. Docs: plan-ladders-transitions.md invariant 20 and the function's row; payments-billing.md, stripe.md, plan-management.md and database-locks.md say a subscription waits while another plan holds what it took. Breaks (scripts/mutation-check, 27 of 27 CAUGHT): - convergeConferrals confers the left-out products (seven records: T1, T1b, T2, T3, T6b, T12, trace grant_over_subscription) - DisplacedProducts matches the product, not the ladder (T8, unit) - DisplacedProducts reads the first rung only (T12, unit) - reconcileItems without its LockPools call (T11) - the log line removed - the item pass walks the ids in reverse (section 2's break restated against this section's loop, which now ranges over the ordered ids) - holder: a default holds against the subscription (T4, function) - an ended position counts as a holder - displacer's chain: the holder must be the displacement's successor (T6, function) - occupancy alone - grant successors only (T9, function) - the pair condition removed (function, T13) - every episode read, not the latest - the earliest episode read - the pairs view without from_rank The convergeConferrals record is written as `if _, out := left[productID]; false && out {`, since `if false {` leaves `left` unused and does not build. Two mutants no test can see: productsLeftAlone passing the missing products rather than every product (a live product's rungs are its latest episodes, so it can never sit on a displaced ladder), and DisplacedProducts keeping only the first row per ladder (the exclusion constraint allows one live position per pool and ladder, and the function returns one latest episode per ladder). The suite on reset databases passes, 4650 tests, 15 skipped; after the review's wording-only fixes, ./internal/fulfillment and ./internal/entitlements pass. make lint passes; openspec validate --strict passes. |
||
|
cgalo5758
|
8181d5461c |
State a supersession pair once, in core.supersession_pairs
A conferral that supersedes a ladder's incumbent writes two transitions
at one instant by one actor: the incumbent's end and the successor's
typed row. Migration 00033 adds core.supersession_pairs, which joins
those two rows of core.plan_transitions on pool, ladder, instant, actor
type and actor id, with a from-side on the typed row, and names both
provisions. A restoration (from_rank NULL) is not a pair. The view does
not group: a transaction that confers twice on one ladder at one
instant yields every end and typed combination. core.grant_replacements
is re-expressed over the view, its rows unchanged; the Down restores
00023's body before dropping the view. The console reader may select
the view (its column list gains five lines), and
docs/models/plan-ladders-transitions.md lists it.
test/pin-timestamps.sql now walks base tables only. It updated every
column information_schema.columns names, views included, and the demo
snapshot's rebuild failed on the new view ("cannot update view
supersession_pairs", the first suite run of this section stopped at the
app database reset). No existing view carries a pinned column, so the
snapshot's pinned values do not change.
Tests (internal/entitlements): the pair names both provisions; an
expiry's restoration is not a pair; a whole-bundle end pairs only the
contested ladder; a chain of acts pairs each end at its own instant; an
expiry's end is not paired with a subscription's conferral. The
extension filter's ep.grant_id clause has no break: only a grant
extension succeeding a subscription would need it, and no test builds
that case.
Breaks (scripts/mutation-check, command records on 00033, 8 of 8
CAUGHT):
- view without from_rank
- view without same ladder
- grant_replacements without extension filter
- no SELECT grant on the view
- view selects reason
- view swaps the two provisions
- view without same instant
- view without same actor type
The suite on reset databases passes, 4627 tests, 15 skipped. make lint
passes; openspec validate --strict passes.
|
||
|
cgalo5758
|
1b1ab1d849 |
Write the item pass's conferrals in product id order
convergeConferrals ranged over the desired-products map, so a reconcile that confers several products wrote them in Go's random map order. The trace kit sorted conferral statements to hide that. The item pass now ranges over sortedProducts(desired), ascending product id, so two reconciles that touch the same products write them in one order, which the displacement rule's writes rely on (design, D6). The trace kit's sortConferralRuns and conferralWrites are deleted: a scan of every golden found no run of conferral statements longer than one line, so the sort changed nothing they record, and no break can show its removal. Ride-along, listed in the proposal: GetLivePoolProvisionsBySubscriptionID orders by created_at and then provision_id. Provisions conferred in one transaction share created_at, so the reconcile's end loop walked them in no stated order. No golden or test changed. reconcile_trace/three_ladders_first_sight pins the order for three ladders seen for the first time in one reconcile (A, B, C). TestSortedProducts feeds maps built forward, backward and in one fixed shuffle, fifty calls each. Breaks (scripts/mutation-check, 2 of 2 CAUGHT): - sortedProducts returns map order (TestSortedProducts; the golden sees it on about seven runs in eight, since a small map can iterate in order, so it carries no record of its own for this break) - item pass walks the ids in reverse (three_ladders_first_sight) The suite on reset databases passes, 4622 tests, 15 skipped. make lint passes; openspec validate --strict passes. |
||
|
cgalo5758
|
bb90c5fa0b |
Record how the reconcile takes a plan back from a grant today
#212: a grant that holds a ladder displaces the subscription paying for
it, and the subscription's next reconcile confers its tier again and
ends the grant. These goldens record that answer before the change that
fixes it, so its sections flip them on purpose:
- reconcile_trace/grant_over_suspended_recovery: a grant issued while the
subscription is past due; the recovery's item pass confers the plan
and ends the grant (#212 itself).
- reconcile_trace/second_subscription_over_first: a second subscription
of the same customer on the same ladder ends the first's plan (the
two-subscription form).
- reconcile_trace/two_tiers_one_ladder: one subscription paying for two
tiers of one ladder confers whichever its live read leaves (the
two-tier form).
- scheduled_change_trace/downgrade_lands_while_displaced: a scheduled
downgrade landing while a grant holds the ladder.
TestReconcile_SupersededWhileSuspendedNotResurrected now recovers
through ReconcileSubscription and records that the subscription holds
one live provision afterwards; section 4 flips it to none. The trace
kit gains a second subscription (<stripe-subscription-b>) and fails a
test whose setup reconcile fails, so a later -update cannot write a
broken setup in as the answer.
The OpenSpec change displaced-subscription-waits is added with this
commit; it depends on reconcile-takes-pool-lock (
|
||
|
cgalo5758
|
16cc1de651 |
Take the pool row in every branch of the subscription reconcile
ReconcileSubscription changed a provision's status and materialized the pool holding only the shared materialization lock and the subscription's own lock (#193). An operator act on the same pool, such as issuing or revoking a grant, could run at the same time; each transaction computed the pool's entitlements without the other's uncommitted change, and the last commit won, leaving entitlements that missed one of the two changes. The reconcile now takes the pool row through LockPools after the subscription lock and before any write, in all three branches: - active: before the resume's sync_source_status; - suspended: before the suspension's sync_source_status; - ended: before the floor restoration. When a provision ended, end_conferral and resumption already hold the row and the call re-takes it; on a replayed cancellation that ends nothing, it is the first take, so the vacancy read and the materialization run under it. The order matches docs/database-locks.md: no path takes a pool row and then a subscription lock, so no cycle appears. docs/database-locks.md: the pool row's "Taken by" cell, "Subscription reconcile" and rule 2 name the three calls; rule 2 loses its exception and "Where the paths differ" loses the suspended branch's entry. docs/models/resource-pools.md: invariant 11 names the reconcile's own call in all three branches. TestAStatusChangeAndAGrantActOnOnePoolKeepBothChanges runs a suspension and a recovery beside a grant act on one pool, waits on the other side's lock (AwaitLockWait, no sleep), and compares the stored limit with the sum of the active provisions. Thirty-three trace goldens gain one LockResourcePool line each; the eight the tasks list stay unchanged. Race run (2.2): `go test -race -count=1 -p 1 ./internal/fulfillment` after test/reset-test-db.sh: ok (41.3 s). Breaks (scripts/mutation-check, 5 of 5 CAUGHT): - reconcileSuspended without its LockPools - reconcileItems without its LockPools - reconcileItems LockPools after SyncSourceStatus - reconcileSuspended LockPools after SyncSourceStatus - reconcileEnded without its LockPools make lint passes; openspec validate --strict passes; the suite on reset databases passes, 4616 tests, 15 skipped. |
||
|
cgalo5758
|
0db7ce9933 |
Record what the sandbox check found for suspended plans
The landing check ran on the Stripe sandbox with API version 2025-02-24.acacia. S1 to S5 and S8 went through the console's pages with stripe listen forwarding; S9 to S11 ran from the CLI on test-clock customers the console does not map, with a daily price so that a period ended before the retries ran out. - S1: settled. The billing-cycle-anchor update was accepted on a subscription the console's schedule manages; the charge failed and the subscription read past_due at once, its invoice open. The Products page, the entitlements panel, and the organization page's Pools panel and Billing card matched the design. - S2: settled. cancel_at_period_end=true was accepted while past_due; the card read "ends <date>", the Billing card "Past due 1 subscription · ends <date>". - S3: settled. Keep plan cleared the flag; the card read "payment due". - S4: settled. The plan's Pay invoice opened Stripe's page for the open invoice, and the detail's Pay invoice answered a redirect to it. Paying with 4242 returned the subscription to active; the card renewed, the entitlements returned, the Suspended badge went away. - S5: no. The hosted page made the new card the customer's invoice default; the subscription kept its own default, the failed card, which Stripe charges first, and Checkout sets that default. Posted on #256. - S8, pending half: settled. Keep plan released the console's schedule on the past_due subscription. The landed half is not settled and rests on TestKeepRefusesADowngradeThatLanded. - S9: no for other subscriptions. The period-end cancellation stopped collection of the cancelled subscription's own open invoice only; the other subscription's open invoice kept collecting. No issue follows. - S10: settled. Marked uncollectible from the CLI, since the check leaves the shared account's retry settings as they are; Stripe's page took payment and the invoice read paid. Posted on #150. - S11: settled. Stripe voided the abandoned upgrade's invoice when its pending update expired; the webhook routes invoice.voided. No issue. - S6 and S7: not settled. The proposal's card and collection lines and the two risks those answers decide now state the answers. Tasks 6.5, 6.6, 7.1 and 7.3 are ticked; 7.2 and 7.4 wait. make screens: the same five screens every section's run reported (operator-person-detail, operator-billing-invoice-detail at both widths, operator-domains at desktop) differ from the baseline in data only. make lint passes; openspec validate --strict passes; the suite on reset databases passes, 4613 tests, 15 skipped. |
||
|
cgalo5758
|
dc2fd46aa0 |
Show an operator a suspended delivery, and say a suspended plan's end
The organization page's Pools panel listed only active deliveries, so a plan whose payment is due vanished from it and the Issue grant form did not warn that a grant would supersede it. The panel now lists a suspended delivery with a Suspended badge, taken from the delivery's first row (the status trigger copies a provision's status to every rung that has not ended), and subscriptionHeldLadders holds its ladder, so the supersession warning shows. ExcludeSuspended is deleted: nothing reads it any more. The Billing card's subscription summary orders states unpaid, past_due, paused, canceling, active, trialing, so the worst state names the card. A suspending state drops "renews <date>", which names a period nobody has paid for; one set to cancel at the period end keeps "ends <date>", which is true, since Stripe cancels it then whatever its invoice says, and is the card's only sign that the member already cancelled. The copy table's row 14 says so. The subscription's Unpaid badge has its own key, subscription_unpaid, in the danger tone, used by the card's Subscriptions badge and the subscriptions list through one helper; the invoice's and the balance's Unpaid keep the warning tone. Rides along, listed in the proposal's quick fixes: a single incomplete_expired subscription said "renews <date>"; like a canceled one it now says nothing. The design's S10 row now says the hourly invoice sweep re-reads an invoice still open a day after its last read, so an uncollectible one reads open for up to a day, not until its next event. Tests: TestOrgEnrollmentGolden's suspended-subscription changes on purpose (Small2 delivering with its badge, the supersession warning, no renewal note), four JSON goldens gain "Suspended": false; TestSummarizeSubscriptions gains the precedence, no-note, ends and incomplete-expired rows; TestSubscriptionUnpaidBadgeIsDanger pins the label and the tones. Suite 4613 pass, 15 skip, 0 fail after the review's fixes. make screens: only the known data-only differences (regenerated ids, a domain's age). Breaks, all CAUGHT: pools panel loop filters suspended positions subscriptionHeldLadders skips suspended positions delivery Suspended never set precedence without unpaid ahead of past_due paused left out of the precedence subscriptions row unpaid badge through the shared key subscriptions list unpaid badge through the shared key note kept for a suspending state unpaid label through the Suspended word unpaid not a suspending state for the note paused not a suspending state for the note past due ranked below paused shared unpaid key drawn in danger past due drawn as unpaid suspending state says nothing even when canceling incomplete expired renews |
||
|
cgalo5758
|
ddd57bf162 |
Show a suspended plan in the member's entitlements panel with its state
The "Your entitlements" panel listed only active plan positions, so a plan whose payment is due read as no plan at all. It now lists every live position, one line per ladder from the ladder's first row in query order, which is its active row when it has one, and badges a suspended line "Payment due" (danger) or "Paused" (secondary). An active row never carries a badge, whatever its subscription's status says. The badge comes from badgeMap through a new payment_due key, so the map stays the one place a state becomes a label. orgHasEntitlement counts a suspended plan too. Its only caller is the dashboard's empty state, which grants no access, so a member whose payment is due is no longer told they hold nothing. Tests: TestLadderRowsYieldOneLinePerLadder (one line per ladder from its active row, and an active row whose subscription is past_due carries no badge); TestMemberEntitlementsGolden's past-due-plan changes on purpose (the plan's line with Payment due, and Enrolled true, which the template's empty-state check reads) and gains paused-plan; three JSON goldens gain "State": "" and nothing else; TestOrgHasEntitlement's payment-due row now wants true. Suite 4605 pass, 15 skip, 0 fail after the review's fixes. make screens: only the known data-only differences (regenerated ids, a domain's age). Breaks, all CAUGHT: ladderRows keeps ExcludeSuspended State always payment_due the template without the badge orgHasEntitlement keeps ExcludeSuspended ladderRows without the one-line skip subscription status decides an active row |
||
|
cgalo5758
|
a7bea2fadf |
Let a member pay a plan's unpaid invoice on Stripe's page
A plan whose payment is due shows Pay invoice as its card's first
control, and an open invoice's detail shows it too. Each opens a console
route in a new tab: GET /billing/plans/{ladderID}/unpaid-invoice and
GET /billing/invoices/{invoiceID}/hosted. The route checks that the plan
or invoice belongs to the member's organization, the way the invoice
detail does and through the same gate function, reads the invoice from
Stripe when the member opens it, and answers 303 to its hosted invoice
page. The URL is a bearer link: it is never stored, logged or written
into a body, the redirect has no body, only an absolute https URL is
followed, and every answer carries Cache-Control: no-store.
The by-plan route answers /billing without a Stripe call for a plan
that is not suspended, a mapping the key cannot reach or no Stripe
querier, and for a latest invoice that is already paid; the by-invoice
route redirects a paid invoice to its receipt. A failed Stripe read logs
"member hosted invoice: stripe read failed" and answers /billing. Both
routes answer a refusal with the styled error page, reusing "The page
you requested does not exist." and "An unexpected error occurred. Try
again.", which the Copy table now lists as reused; the invoice detail,
a partial, keeps its plain answers. Pay invoice counts as a paid control
for the Products page's billing line.
Tests: TestHostedInvoiceRoutes (18 rows, another organization's plan and
invoice among them), TestInvoiceDetailOffersPayInvoice,
TestPayInvoiceReferencesTheBillingLine, new rows in
TestPlansOfferPaidMoveGate. The test log capture now carries With
attributes and keeps a repeated key's values, so the check that no line
carries the URL cannot be bypassed. Goldens: seven payment-due
member-plans pages gain Pay invoice; routes.golden and
routes-not-ready.golden gain the two routes (anonymous: 302 to /login);
route_pages.golden gains them as styled 404s.
Not caught by a break of their own: the by-plan gate's pool scoping and
its billing-account check guard together, and the other-organization
row catches only their conjunction (no single edit removes the scoping,
and the test kit cannot back one organization's position with another's
subscription); that the detail and the by-invoice route call one gate is
read in review; pageRefused's 500 branch has no row, since no fixture
can fail a read on the real database.
Rides along: the change's proposal and design name #255 to #258, the
issues filed for its out-of-scope items, and the #150 note; the design
gains S11, a sandbox check of an abandoned upgrade's invoice after its
pending update expires.
Suite 4603 pass, 15 skip, 0 fail. make screens: only the known
data-only differences (regenerated ids, a domain's age).
Breaks, all CAUGHT:
by-invoice route without the ownership gate
by-invoice route without the environment check
by-invoice route redirecting only an open invoice
by-invoice route accepting a URL that is not https
by-plan route without its suspended check
by-plan route redirecting a paid latest invoice
by-plan route without its environment check
by-plan read without the expand of latest_invoice
by-plan route answering a missing position 303 to /billing
by-invoice handler without its own Cache-Control
by-plan handler without its own Cache-Control
the hosted URL logged on success
failed invoice read without its log line
failed subscription read without its log line
card without Pay invoice
Pay invoice on a paused plan
Pay invoice drawn without the LadderID check
plansOfferPaidMove not counting Pay invoice
Pay invoice without its aria-describedby
PayPath set for an invoice in the other environment
PayPath set for a paid invoice
by-plan route not registered
refusal of a Pay invoice route as bare text
the hosted URL logged through Logger.With
|
||
|
cgalo5758
|
996d6eee34 |
Hold a plan whose payment is due on the Products page and in the plan moves
Third section of #148, which fixes the issue's two points. A plan whose subscription is past due, unpaid or paused is still the organization's: the Products page shows it as the current plan and offers no Checkout on its ladder, and the Checkout endpoint refuses one, so a second Stripe subscription can no longer start while the first is retried. The current card of such a plan reads "<price>/<interval>, payment due" (or ", paused") in place of its renewal line, with "Entitlements are suspended until the invoice is paid." (or "Entitlements are suspended."). A past-due subscription may be a failed card or a renewal waiting on the bank's 3-D Secure approval, so the words say what is owed, not why. The badge tooltip that promised access until the period end is dropped there, and the cancel confirmation names the end ("Pro ends on {date}.", or "..., then you move to {plan}.") followed by "The unpaid invoice stays due." The plan moves on that subscription: - A switch, in either direction, and its preview are refused with "Pay the unpaid invoice before changing this plan." before any Stripe call, in the decision and in fulfillment, so no second invoice interleaves with the open one. - Cancel plan at period end and Keep plan work as on an active plan. An immediate cancellation is refused: Stripe stops collecting every finalized invoice of the customer when a subscription is cancelled. - Keep plan is refused once the scheduled downgrade has landed at Stripe, which the failed renewal usually means, so it cannot undo a change Stripe already bills; the page then draws no "moves to" line and no Keep plan. A landed downgrade is recognised by the schedule Stripe reports or, once Stripe has released that schedule, by the subscription already carrying the lower price, the test the reconcile uses; the second part comes from this section's review and the change's design says so. - A paused plan offers no move at all and fulfillment refuses each one, with "A paused plan cannot be changed."; no console-made subscription reaches paused today. HasActiveSubscriptionOnLadder becomes SubscriptionBacksLadder for its wider meaning. The test kit gains ConferUnprojectedSuspended for a subscription not yet projected. The member-plans goldens gain seven states for these plans; the two characterization goldens of the previous section now show the corrected page, and every other golden gains only an empty Suspension field. make screens: no changed screen comes from this section (the demo has no suspended plan); the seven changed shots are the known data-only and timing differences. Full suite on reset databases: 4578 pass, 15 skip, none failed. make lint passes. Breaks, each caught: - resolveLadderSubscription keeps ExcludeSuspended (TestCancelASuspendedSubscriptionAtPeriodEnd) - resolveLadderSubscription keeps ExcludeSuspended (TestSubscriptionBacksASuspendedLadder) - SwitchPlan without its suspended refusal - SwitchPlan refuses only after a Stripe read (the refusal after the first Stripe call of the downgrade) - positionStatus never set - the suspended refusal placed in resolveLadderSubscription instead of SwitchPlan - KeepCurrentPlan without its landed-downgrade check - the landed check without its price comparison - the immediate branch without the check - the immediate check placed after the schedule release - SwitchPlan without the paused refusal - KeepCurrentPlan without the paused refusal - CancelSubscription without the paused refusal - ErrSubscriptionSuspended returned for a paused subscription in SwitchPlan - the suspended rule (15) removed (offer table) - the suspended rule (15) removed (the page) - the suspended rule applied before the held rung rule (8) - the suspended rule applied before the free rung rule (14) - the paused rule removed - the paused rule placed after the held rung rule (8) - onLadder without pos.Suspended - positions keeps ExcludeSuspended (TestMemberPlansGolden) - positions keeps ExcludeSuspended (TestCheckoutRefusesALadderWhosePlanIsSuspended) - switchOffer without the suspended case - the suspended case placed after the current-tier case - ErrSubscriptionPaused never returned by switchOffer - the refusal log line without subscription_status - renderPlansAfterMove without its four cases - renderPlansAfterMove without its row 17 case - renderPlansAfterMove without its row 18 case - KeepCurrentPlan without its paused refusal, seen through the page - Suspension always payment_due (the paused branch removed) - the payment-due price line in place of "ends" - setCancelBodies without the appended sentence - the sentence appended on the current card only (the loop narrowed to CanCancel) - Cancel plan drawn on a paused plan - Suspension payment_due only for past_due - onLadder keeping a downgrade whose date is not after now - onLadder dropping every downgrade of a suspended position - setCancelBodies keeping "You keep {current} until {date}" on a payment-due plan - setCancelBodies choosing the form of row 19 when another plan follows - setCancelBodies choosing the form of row 20 when nothing follows - the body of row 24 read as row 21 - the body of row 20 read as row 21 - CancelBadge keeps its Title on a suspended plan - KeepCurrentPlan without rule 1's downgradeLanded test - onLadder keeping a landed downgrade (the date check removed), seen through the refused Keep - ConferUnprojectedSuspended without the suspension - the paused rule removed (the page's move check) |
||
|
cgalo5758
|
10057f8e3a |
Read suspended plan positions, and keep every reader's answer unchanged
Second section of #148. Goldens first pin what the member sees today for a plan whose payment is due: the Products page treats the plan as not held and offers Subscribe on every priced plan, the entitlements panel reads "No active plan", and a plan of boolean entitlements reads as none. Those are the wrong answers the later sections fix. ListLadderPositionsByPool now reads live positions, active and suspended, with the position's and the subscription's status, and puts an active row before a suspended one on the same ladder so a suspended row can never be chosen over an active one. Every reader filters suspended rows out through one helper, entitlements.ExcludeSuspended: the Products page's position loader, the entitlements panel, the dashboard's entitlement check, the plan moves' subscription lookup, and the operator's Pools panel and grant advisory. No existing answer, golden or test result changes; each later section removes the filter from the surfaces it releases. A test proves that a suspended position confers nothing, because entitlements come only from active provisions. The plan moves' lookup filters too, and no test of this section can see it; its break runs with the section that releases those moves. Also regenerated every sqlc package through make sqlc-generate, which restores the license headers a bare sqlc generate drops: BRIDGEFILES generated files that had lost their headers gain them back, with no other change. Full suite on reset databases before the review fixes: 4524 pass, 15 skip, none failed. The review fixes add a test arrangement and change comments and license headers; after them internal/entitlements, plans, fulfillment, server and the test kit pass. make lint passes. Breaks, each caught: - the position query keeps only active rows - the position query without its active-first order - the subscription status is NULL - ExcludeSuspended keeps every row - ExcludeSuspended keeps only active rows - positionLoader.positions without ExcludeSuspended - ladderRows without ExcludeSuspended - orgHasEntitlement without ExcludeSuspended - the Pools panel's loop without ExcludeSuspended - subscriptionHeldLadders without ExcludeSuspended - materialization reads suspended provisions - the position query orders oldest first |
||
|
cgalo5758
|
979e38b985 |
Suspend a test subscription the way the reconcile does
First section of #148. The test kit's Suspend suspended the provisions a subscription confers and left the subscription itself active, a pair the app never writes: the reconcile sets the subscription's Stripe status (past_due, unpaid or paused) and suspends its provisions together. Suspend now takes that status, writes it to the subscription row with the row's own period, cancellation and end fields, and then suspends and settles as before; any other status fails the test. The three callers pass past_due. The organization page's golden for a suspended subscription changes accordingly: its Billing card read Active and now reads Past due. The other two callers read no subscription status and keep their results. This change also adds openspec/changes/suspended-plans-stay-held, the plan for #148: a suspended plan stays the member's on every surface, confers nothing, refuses a second Checkout and a switch, and offers a way to pay the unpaid invoice. Full suite on reset databases: 4517 pass, 15 skip, none failed. make lint passes. Breaks, each caught: - Suspend without the status write - Suspend writes an empty period end |
||
|
cgalo5758
|
715404500a |
Archive the runtime database user and the boot that serves first
serve-before-dependencies (#141) and runtime-database-user (#23) move to the archive, and their deltas are synced into openspec/specs. The two Purpose edits a delta cannot make are made here: temporal-connection's Purpose says start connects in the background and retries for the life of the process, and error-pages' names the 503. What the two investigations learned beyond each decision moves into the docs: - go-conventions.md: a new section on code that reaches a service that answers after the listener opens (holders, per-request reads, the refused write, retryloop, viper writes before the listener). - production-deployment.md: what a liveness probe requests, and why it must not wait on Temporal or the identity provider. - building-an-integration.md: declared paths skip the session layer; a completed workflow id runs again, and the guarded first update. - models/payments-billing.md: invariant 14 states the 503 and Discourse's failed rows; invariant 15 states the guarded update. - operator-ux-conventions.md: an error answer carries no HX- header. - database-management.md: why writers hold only the four DML privileges, identity columns and sequences, the qualified SQL the migration user runs, and sqlc alias lists for set-returning functions. - database-locks.md: rule 9, a row lock on a table the writer cannot update is taken in a SECURITY DEFINER function. - testing.md: privilege tests drop to console_runtime, race-only breaks as command records, the race run's -p 1, synctest for timer loops, and test workers off the console's task queue. Both archived design.md files gain the options weighed and the drafts the approved copy replaced. |
||
|
cgalo5758
|
67e69eed0d |
Name Temporal in the end date's refusal
The end date's field error read "An end date needs the expiration scheduler, which is not connected." It now reads "End dates can't be saved while Temporal is not connected.", which names the service that whoever runs the console configures and searches for. The bounded-grant test compares the message as html/template renders it, apostrophe escaped, so its checks that the message is absent no longer pass on the escaping alone. |
||
|
cgalo5758
|
2d71aa8db0 |
Describe the boot that serves before its dependencies
Seventh section of #141, the docs. docs/production-deployment.md's first boot section lists the steps in the order start runs them, says what serves from the moment the listener opens, and describes the two loops that run beside it: the identity provider's discovery, with sign-in at 503 until a usable document is published and roles not re-derived before a document with the configured issuer is read, and Temporal, retried for the life of the process with its log turning to Error after temporal-connect-timeout. The service list says that Stripe and Discourse need Temporal. README.md and docs/plan-management.md lose their description of a bounded retry before the server comes up. docs/environment-reference.md rewrites the temporal-connect-timeout and temporal-host rows; docs/stripe.md and docs/discourse-setup.md say why temporal-host is required; docs/building-an-integration.md documents NeedsTemporal. docs/fedwiki-setup.md says the console answers the certificate ask as soon as it serves. docs/identity-provider-setup.md and internal/auth/README.md replace "refuses to start" with what the console does, and docs/temporal-authorization-setup.md explains the token URL. Comments in the FedWiki and Stripe integrations and the FedWiki workflows README that still described the old boot are corrected. Every quoted log line is the one the code writes. No code changes. make test passes with the test stack up, and go test -race -p 1 over cmd, auth, server, retryloop, workflows and the three integrations after reset-test-db.sh finds no data race. |
||
|
cgalo5758
|
bfc24e2c74 |
Serve before Temporal connects, and connect it in the background
Sixth section of #141, which closes the issue's defect: start no longer waits on Temporal before it serves. Start derives the Stripe key's mode and fingerprint, sets the SDK's key, opens the database, applies the operator overrides and the boot data state, binds the listener, and only then starts the Temporal connector and the identity provider's discovery beside the server. A bind error returns before any dial or discovery request. While Temporal is unreachable the console serves: sign-in, the domain ask, the static files and every page that needs no workflow answer as usual, and the Temporal-backed actions answer as the previous section made them. The connector runs the retry loop. Each attempt dials once (NewClient no longer retries on its own; dialWithRetry and its budget are gone), builds the worker and runs its precondition. A precondition that refuses keeps the client and retries at the cap, so FedWiki's refusal over open runs of a removed workflow type is read again on the same connection; one that cannot read what it checks, a worker that fails to build or start, closes the client and dials again. The attempt ends when the worker starts: "connected to Temporal server" is logged with the attempts and the time it took, then the holder is published and the integrations' Startup hooks and the schedules run as before. "Temporal not ready yet, retrying" is a Warn until temporal-connect-timeout (90 seconds by default), whose help now says so, and an Error from then on, under the loop's decay. A Temporal OAuth configuration that cannot be built still refuses at start. start exits 1 when serving fails, at any point, and returns only after the connector and discovery have returned. A failure before the listener keeps the message it had (the overrides, the boot data state, the integration connections); a serving failure logs "server failed to start". The Stripe webhook activity marks an event processing only from received, processing or failed, so a run of an event already completed, skipped or dead-lettered returns at once and dispatches nothing. The handlers and drains that found no Temporal client logged that it was not configured; with Temporal still connecting that was false, so they log "Temporal is not connected" (the six plan-act goldens carry the new drain line). Copies 13, 15 and 16 also carry the host and namespace. Checked by hand on the test stack: with Temporal stopped, the console served /login and the ask at once; the connector logged Warn, then Error from the first attempt after 90 seconds; when Temporal came back the same process logged "connected to Temporal server" after 14 attempts. With the issuer refusing, /login answered the styled 503 with Retry-After: 5 and the ask answered. Full suite on reset databases: 4513 pass, 15 skip, none failed. go test -race -p 1 over cmd, auth, server, retryloop, workflows and the three integrations after reset-test-db.sh: every package passes, no data race. make lint passes. Breaks, each caught (section 1's UseKey break among them). Two depart from item 6.10's first wording, which the item now carries: the synchronous-fetch break sits between listen and the server's start (before listen the row would fail waiting for the address, not at its one-second assertions), and the "returns without waiting" break is caught by the fourth boot row, whose fake dial holds for 200 ms of wall-clock time past the context's end, the least that makes an early return observable. - serveConsole without UseKey - UseKey after the bound server's Serve - Serve without go Discover - Serve fetches the discovery document before serving - serveConsole waits for connectTemporal before Serve - connectTemporal started before Bind returns - serveConsole returns without waiting for its goroutines - serveConsole returns without waiting for its goroutines (race) - skipSession without the ask's path - connectTemporal publishes after afterConnect - connectTemporal publishes before worker.Start - connectTemporal returns after the escalation time - the open-runs refusal closes the client - ErrCannotCheck treated as a refusal - worker.Start failure leaves the client open - listRunsError without its Is method - worker.Start failure logged from inside the attempt - ProcessWebhookEvent without its status guard - the guard without failed - the guard without processing - NewClient without the deadline - connectTemporal under serveConsole's own context - the attempt runs the hooks before copy 14 - Run logs copy 21 for a step failure - the overrides failure is a plain error - a Temporal authorization that cannot be built is not refused |
||
|
cgalo5758
|
3bfd0f67f4 |
Read the Temporal connection per request, and answer for its absence
Fifth section of #141. internal/workflows/clientholder holds the Temporal client: start builds an empty holder, hands it to the server and the integrations, and publishes the client once, after the worker starts and before the first Startup hook; a second Publish panics. Every handler and integration that starts a workflow reads the holder when a request arrives, so a client that connects after the listener opens reaches them without a restart; a test scans the handler configurations and fails on any field that still holds a client.Client. While the holder is empty: - POST /webhooks/stripe stores the event as received, then answers 503 with Retry-After: 5 and the body "Temporal is not connected", and logs that it answered 503 so Stripe redelivers it. A redelivery after Temporal connects starts the event's workflow, and the processing guard keeps a finished event from running twice. - A Discourse event is recorded as failed with "start reconcile: Temporal is not connected"; the sweep restores what it missed. - Issuing or managing a grant with an end date is refused under the end-date field with "An end date needs the expiration scheduler, which is not connected.", after the date itself is checked, and writes nothing. The operator page's "no scheduler" warnings, which saved the grant and left it without an expiry, are gone. - FedWiki's site actions and the domain claims answer that they are not available, as they did with no Temporal configured. The operator test harnesses and the plan-management end-to-end harness publish a holder over a fake client whose ExecuteWorkflow succeeds, so every bounded save keeps its result. Full suite on reset databases: 4485 pass, 15 skip, none failed. The console booted on the test stack and connected to Temporal. go test -race -p 1 over clientholder, internal/server and the three integrations after reset-test-db.sh: every package passes, no data race. make lint passes. Breaks, each caught: - 5.10 member domains handler reads the client at construction - 5.10 FedWiki site action handler reads the client at construction - 5.10 Stripe environment-check handler reads the client at construction - 5.10 the holder is a plain field in place of an atomic.Pointer (-race) - 5.10 a client.Client field added back to MemberDomainsConfig - 5.10 the scan ignores import names, so a temporalsdk.Client field would pass - 5.10 Stripe StartProcessing reads the holder at registration - 5.10 the webhook answers 500 for ErrNotConnected - 5.10 the webhook answers 503 before the insert - 5.10 Discourse StartReconcile calls the nil client - 5.10 the issuance refusal after ConferGrant - 5.10 the management refusal removed - 5.10 the issuance refusal applied to a grant without an end date - 5.10 the management refusal applied to a save that clears the end date - 5.10 Publish stores a second client without panicking - 5.10 a temporalsdk.Client field added back to a FedWiki handler configuration - 5.10 Discourse StartReconcile reads the holder at registration - 5.10 the issuance refusal before the parse - 5.10 the management refusal before the parse - 5.10 the issuance refusal is a form error, not under the end date |
||
|
cgalo5758
|
ed613ef952 |
Answer 503 while the session store is down, and stop only on a refused login
Fourth section of #141. At start the console probes the session store once. A store that refuses the console's credentials (WRONGPASS, NOAUTH, NOPERM, or an ERR reply to AUTH other than "max number of clients reached") stops the start, as before; anything else, a refused or silent address or a store that answers LOADING, BUSY or MASTERDOWN, is logged as "session store not ready at startup; requests that need a session answer 503" and the console serves. The probe takes its connection with pool.GetContext, so the dial's error is classified apart from PING's. valkey-addr is read once. Every store error a request meets now answers a styled 503 "Sign-in unavailable" with Retry-After: 5 and a "Try again" action to the request's own path (a GET or HEAD that validReturnTo accepts) or to /, in place of scs's bare 500; for /callback the action is /login, because the authorization code it carries is spent once. The store is wrapped so that its errors are tagged; scs's ErrorFunc answers a tagged error with the 503 and logs "session store request failed", and any other error, a session that cannot be decoded or encoded, with the 500 and "session could not be decoded or encoded". The handlers' own store calls (RenewToken in /login and /register, Destroy in POST /logout) answer a store error through the same path. (*Config).LoadAndSave wraps the writer in a latch: once the ErrorFunc answers, nothing the handler set before reaches the client, since every HX- header, Location, Set-Cookie, Content-Type and Content-Length is deleted and the latched writer drops later writes and headers. A request without a session cookie makes no store call, and the routes that need no session (the static files, the favicon, and the exempt webhook and ask paths) skip the session layer. start now exits 1 when serving fails, after "server failed to start", so an orchestrator that restarts only on failure restarts it. Full suite on reset databases: 4456 pass, 15 skip, none failed. The console booted on the test stack with the session store connected. go test -race -p 1 over internal/auth and internal/server after reset-test-db.sh: every package passes, no data race. make lint passes. Breaks, each caught: - probe fatal for every error, as today - probe fatal for every error reply, LOADING included - probe never fatal - max clients fatal (the exception removed) - ErrorFunc not set (scs default) - store not wrapped (errors returned untagged) - every error classified as a store error - latch not marked - latch not marked (HX-Redirect row) - latched writer still passes a body - ErrorFunc deletes a fixed list of headers without HX-Refresh - 503 without Retry-After - action always / - action built from RequestURI without validReturnTo - skipSession without /static/ - skipSession without /favicon.ico - skipSession without the exempt list - the ErrorFunc keeps Location - the ErrorFunc keeps every HX- header - the probe takes its connection without GetContext - an ERR reply to AUTH not fatal - the PING path classified as the dial - a header set after the 503 follows it under TimeoutHandler - tagStoreError without its nil check - copy 17 at Warn - copy 17 without addr - Retry-After set for both branches - login keeps its 500 on a store error - register keeps its 500 on a store error - logout keeps its 500 on a store error - logout before discovery keeps its 500 on a store error - callback keeps its 500 on a store error - the 503 for a callback repeats the spent callback |
||
|
cgalo5758
|
d1151dd6fe |
Wait for the identity provider's discovery document without blocking start
Third section of #141. start no longer fetches the identity provider's discovery document before it serves. Setup returns a Config whose two discovery values are empty, and Discover runs beside the server from before the listener opens: it fetches the document, publishes the verifier and then the sign-in endpoints, logs "identity provider discovery succeeded; sign-in is available", and returns. Until it has, /login and /callback answer a styled 503 "Sign-in unavailable" and /register "Registration unavailable", each with Retry-After: 5 and a "Try again" action, and a sign-in request wakes the loop's wait. A document whose issuer is not the configured one, or that names no end-session endpoint (or one that does not parse as an absolute URL), publishes nothing that sign-in reads and is retried at the cap. The two values sit behind two atomic.Pointers in auth.Config. The sign-in value holds a pointer to the verifier value it was published with, and is stored only after the verifier, from the same document, so a request never pairs endpoints from one document with a verifier from another. Re-derivation of roles reads the verifier alone, so it runs while sign-in still waits for an end-session endpoint; POST /logout before discovery ends the console's session, lands on /, and revokes nothing. A Config built by hand, as most tests build theirs, reads its exported fields as it did. internal/retryloop is the loop both discovery and, in a later section, the Temporal connection run: waits from 0.5 seconds doubling to a 15-second cap, plus up to half again of jitter; a wake ends a wait no sooner than a second after the last attempt began; a failure marked AtCap makes only the next wait's base the cap, and a wake does not end that wait. Its log decay writes the first failure, a changed error at most once per 30 seconds, otherwise one line per 5 minutes, Warn until the escalation time and Error from it, with the line at the escalation time written at once; every line's attempts is the attempts so far. The htmx 5xx toast reads "Server error. Try again later." Corrections to the change text, made while building: the wait after an AtCap failure is the cap plus the jitter (the design's bound of about 23 seconds after an outage already counted the jitter), and every log line's attempts is the number of attempts so far (one meaning on every line, matching elapsed), where the design said the cap alone and the attempts since the last line. internal/server/testdata/routes-not-ready.golden is new: it differs from routes.golden in the four sign-in rows only. routes.golden and route_pages.golden do not change. Full suite on reset databases: 4422 pass, 15 skip, none failed. The console booted on the test stack and logged the discovery success. go test -race -p 1 over internal/retryloop, internal/auth and internal/server after reset-test-db.sh: every package passes, no data race. make lint passes. Breaks, each caught: - LoginHandler's gate after RenewToken - the gate without the wake - the gate answers through http.Error - the gate answers 500 in place of 503 - the gate without Retry-After - /register without the gate - the sign-in value published from a document without an end-session endpoint - the sign-in value stored outside the issuer check - the verifier published only together with the sign-in value - the verifier published from a document with another issuer - an unusable document returned as a plain error - the attempt on http.DefaultClient - LogoutHandler before publication answers 500 - LogoutHandler before publication redirects without destroying the session - LogoutHandler with the verifier and no end-session endpoint revokes nothing - refreshIdentity ends the session when no verifier is published - Run without the cap - Run without the jitter - a wake that ends an AtCap wait - the wait ignores ctx.Done() - the discovery holder is a plain field in place of an atomic.Pointer - a Failure's level ignored - the decay without its five-minute rule - the escalation never reached - the changed-text rule without its thirty-second floor - a wake that ignores the one-second gap - answer() without the Retry-After line - re-derivation reads the exported fields in place of the holder - a level Error line written at once whatever the last line - an AtCap failure ratchets the doubling to the cap - the discovery log lines without the issuer attribute - the failure lines with a fixed attempts attribute - an end-session endpoint that does not parse or is not absolute is accepted - Run attempts once on a context that has ended - a Failure returned by pointer drops its message and level |
||
|
cgalo5758
|
6b57598df6 |
Refuse to start Stripe or Discourse without temporal-host
Second section of #141. Stripe and Discourse receive webhook events that only a Temporal worker processes. Configured without temporal-host, the console stored each event and never processed it, and with the first section's key set at boot a Stripe checkout or plan move would reach Stripe while the events it causes waited forever. Start now refuses that configuration: Stripe needs Temporal, which processes its events: set temporal-host (or leave stripe-api-key and stripe-webhook-secret empty to disable it) ConfigKey gains NeedsTemporal. Stripe declares it on stripe-api-key and Discourse on discourse-base-url; FedWiki declares nothing, because it receives no events and each of its Temporal-backed actions answers that it is not available. ValidateStart checks every group that has every key set and carries a NeedsTemporal key against a trimmed temporal-host, in first-seen order; a partly set group gets only the partial-group refusal. CheckSpecs refuses NeedsTemporal on a key without a RequiredGroup. validateRequiredGroups and the new check share one collection of the groups, with the same results and order as before. Every configuration the repository ships or boots its tests with sets temporal-host wherever it sets the Stripe or Discourse keys. Full suite on reset databases, before the review fixes: 4382 pass, 15 skip, none failed; the console booted on the test stack. The review fixes change comments and internal/config's tests only; after them internal/config, internal/integrations and the Stripe and Discourse trees pass, and make lint passes. Breaks, each caught: - validateTemporalNeeds not called from ValidateStart - the check reads temporal-namespace in place of temporal-host - the check reads temporal-host untrimmed - the check fires on a partly set group - CheckSpecs without the NeedsTemporal group check - Discourse ConfigSpec without NeedsTemporal - Stripe ConfigSpec without NeedsTemporal - FedWiki ConfigSpec declaring NeedsTemporal |
||
|
cgalo5758
|
29520a8120 |
Set the Stripe key once at boot and read the boot mode everywhere
First section of #141. start assigns the stripe-go key through UseKey,
directly after it derives the key's mode and fingerprint and before any
goroutine starts; the Stripe integration's Startup no longer assigns it,
and nothing else does. UseKey assigns the trimmed key, so the SDK sends
the key whose mode and fingerprint boot derived, and an empty key leaves
the SDK's key as it was.
Every surface that asked a function for the key mode now reads the mode
boot derived: Checkout, the plan moves and the eager reconcile in
internal/server, the fulfillment entry points (SwitchPlan,
KeepCurrentPlan, CancelSubscription, the two reconciles and the
scheduled-change sweep), and the worker's billing activities. The Stripe
workflows' activities still read the SDK's key, which start set before
the worker existed. SDKKeyMode, Config.StripeKeyMode and
WorkerConfig.StripeKeyMode are gone. stripe-api-key is a secret, which
an operator override cannot replace, so the value boot reads is the one
Startup used to assign.
One behaviour changes until the next section lands: with no
temporal-host, the SDK now holds the key, so checkout sessions and plan
moves reach Stripe while the webhooks they cause are stored and never
processed. The next section refuses that configuration; the two are not
deployed apart.
The tests pass the boot mode, stripetest.KeyMode(), where they passed
the function. TestCheckoutThroughTheRouter lost its count of how often
the mode function was asked, since the function no longer exists, and
gained a row that posts a price mapped in live mode under the test key
and wants 422, which fails if the router hands Checkout an empty mode.
The eager reconcile's wiring is covered by no break.
Race baseline (go test -race -p 1 -count=1 over cmd, auth, server,
workflows and the three integrations, on
|
||
|
cgalo5758
|
bdd042bc51 | Tick #23's landing tasks | ||
|
cgalo5758
|
e722e43603 |
Document the two database users and how an existing deployment moves to them
Sixth section of #23, docs only. docs/production-deployment.md explains why the console connects as two users and what the runtime user may not be (a superuser, an owner of the database or of any object, or a member of any role but console_runtime), how a new deployment creates the runtime login with its password set in psql, how to run migrations as a separate step, and the steps for a deployment that has connected as one login so far: create the runtime login, store its string as a new secret (Docker Swarm secrets cannot be changed), point MC_DB_DSN_FILE at it and MC_DB_MIGRATE_DSN_FILE at the existing secret, deploy, verify the sessions, privileges, memberships, database settings and the functions of public, re-run the reader-role script, and how to roll back. The bootstrap superuser of the official image cannot lose SUPERUSER, so it stays as the migration user. docs/database-management.md runs the reader-role script as the migration user. make lint and TestEnvironmentReferenceDocumentsEveryConfigKey pass; the section changes no code, and the previous section's full suite ran with these files already in the tree. |
||
|
cgalo5758
|
dd4a0e2ec0 |
Refuse to start on a runtime user that can get around the grants
Fifth section of #23: the start-up check. Before any migration runs, on both connections, the console refuses when the two strings log in as one user or reach different databases, when the runtime user is a superuser or a member of a superuser role, when it owns or is a member of the owner of the database, when the database is owned by neither the migration user nor a superuser, when the runtime user can create objects in the database, in public or in a console schema, or when it owns an object in those schemas that a name lookup could resolve (relations, functions, operators, types, and operator classes, families and extensions database-wide). After the migrations and the boot grant, on the runtime connection, it refuses when console_runtime is missing, when the runtime user does not inherit it or holds it with SET, when the schema is behind the code, when the runtime user reaches an owner role, owns an object, holds any direct membership but console_runtime, or holds a privilege an enclosure withholds. Each refusal names the statement that fixes it, with names quoted by the database; a check query that fails refuses too. migrate up, down and status refuse a database whose owner is neither user before goose runs. The boot grant is now WITH INHERIT TRUE, SET FALSE: the runtime user needs only to inherit console_runtime, and without SET on that first membership no SET ROLE reaches a role through it. Granting again updates an existing membership in place; this worktree's runtime user held SET from the previous section and lost it at the suite's boot. Every check runs in a read-only REPEATABLE READ transaction that begins with SET LOCAL search_path = pg_catalog, pg_temp, and names every function and catalog relation with pg_catalog., so nothing the runtime user planted (a function such as public.quote_ident(name), or an operator such as public.=(oid, integer), which qualified function names alone did not exclude) runs in the migration user's session or writes a statement the console prints. TestCheckQueriesQualifyEveryName scans the query constants and fails on an unqualified function call or catalog relation. Full suite on reset databases: 4370 pass, 15 skip, none failed. The console booted on the test stack and logged "runtime database user checked" with the runtime and migration users. make lint passes. Break runs that disable an early refusal or a test's cleanup can leave a role or a membership in the shared test cluster; those were removed after the run. Deliberate breaks, scripts/mutation-check -side new, all CAUGHT (76): checks 3a to 3c run before checks 1 to 3 checks 1 to 4 run after the migrations the boot grant runs before the superuser check the database-ownership check removed (TestCheckBeforeMigrating) the database-ownership check removed (TestOpenRuntimeDatabase) check 4 tests datdba = current_user in place of MEMBER check 4 reads the first database in pg_database check 4 without its datname = current_database() filter the migration identity read outside the pinned read-only transaction, with an unqualified quote_ident the runtime identity read outside the pinned read-only transaction, with an unqualified quote_ident the migration pool opened without role=none (owner set its own role) check 12 reads has_schema_privilege in place of the ACL entries check 12's schema CREATE query removed (TestCheckBeforeMigrating) check 12's schema CREATE query removed (TestOpenRuntimeDatabase) check 12's ownership query removed (TestCheckBeforeMigrating) check 12's ownership query removed (TestOpenRuntimeDatabase) check 12 runs after the migrations the boot grant's pg_catalog.format written format check 12's queries read outside the pinned read-only transaction (TestCheckBeforeMigrating) check 12's queries read outside the pinned read-only transaction (TestOpenRuntimeDatabase) CheckRuntimeUser's transaction without the search-path pin the second query of checks 9 and 12 casts the login with regrole CheckMigrationDatabaseOwner removed from dbMigration dbMigration checks before up only CheckMigrationDatabaseOwner removed from dbRuntime the owner half refuses a superuser-owned database copy 3d prints the database name unquoted check 11 without the four-privilege read copy 9 prints the user in place of the grantee check 11's grantee filter dropped check 11's predefined-role query removed copy 3c prints the direct role unquoted copy 6 prints the direct role unquoted check 6 tests MEMBER in place of USAGE check 8 tests USAGE in place of MEMBER check 9 without functions check 9 without public check 9 tests owner = current_user in place of MEMBER the direct-membership check removed (TestCheckRuntimeUser) the direct-membership check removed (TestOpenRuntimeDatabase) the direct-membership check counts console_runtime the direct-membership check runs before check 8 check 11 removed check 3 without its membership half copy 5 quotes the name in Go (%q) an exemption when env is development a failed check query passes the scratch database is dropped after the login that owns it (test-side) an unqualified quote_ident( in the identity query an unqualified pg_roles in the owner check PendingMigrations calls to_regclass without pg_catalog. the query constants renamed so none begins query (test-side) the search-path pin removed from the pinned read-only transaction the search-path pin removed from the savepoint a test runs a check in check 5 removed check 6 does not read the SET option check 6 does not read the SET option (separate-step start) the boot grant leaves the SET option (TestGrantRuntimeRole) the boot grant leaves the SET option (TestOpenRuntimeDatabase) the database CREATE query removed (TestCheckBeforeMigrating) the database CREATE query removed (TestOpenRuntimeDatabase) the database CREATE query reads has_database_privilege in place of the ACL entries check 12's ownership query reads public only check 12's ownership query reads public only (TestOpenRuntimeDatabase) check 9 without operators check 9 without types check 9 without operator classes check 9 without operator families check 9 without extensions start's refusal log lacks the owner refusal the scan folds no case (test-side) the scan reads a quoted call as no call (test-side) the scan reads no line comment (test-side) the scan skips a concatenated constant (test-side) the scan passes a query constant without a value (test-side) copy 5a prints only the GRANT, not the REVOKE by the grantor |
||
|
cgalo5758
|
f9c9b3225e |
Run the console on a runtime user and migrate on a second connection string
Fourth section of #23. db-dsn now names the console's runtime user, and the new db-migrate-dsn (or db-migrate-dsn-file) names the user that runs migrations. start opens both, migrates on the migration connection, grants console_runtime to the runtime user there (WITH INHERIT TRUE, the name quoted by the database), closes it, and serves on the runtime pool alone. With db-migrate-dsn left out, start runs no migration and refuses when the schema is behind its code, which is how a host runs migrations as a separate step; migrate up, down and status use the migration string and refuse without it, with no fallback to db-dsn. Both pools are opened from one pgx.ParseConfig: the migration pool's search_path is public and both pools' role is none, set in the startup parameters after every key of either name is removed case-insensitively, so neither a setting the database or role carries nor a key the string repeats in another case moves a session off its login or its path. PendingMigrations reads each stream's ledger by qualified name inside a transaction pinned to search_path pg_catalog, pg_temp. The test stack runs on two users: postgres-seed creates member_console_runtime in new and existing volumes, bootstrap-stack.sh writes MC_DB_DSN as that user and MC_DB_MIGRATE_DSN as member_console, and the four scripts refuse a .env without the migration string, with the command that regenerates it. Scripts and fixtures that need a superuser on the application database (the reset, the screens' timestamp pinning) use the migration string. This worktree's stack was regenerated with that command and the console booted as member_console_runtime: its sessions were the runtime user's only, no migration-user session stayed open, and the runtime user could not insert into core.pool_provisions. Full suite on reset databases: 4277 pass, 15 skip, none failed. The walkthroughs' names, results and skip messages match the previous section's run. make screens lists three changed screens, all data: regenerated ids on operator-person-detail and operator-billing-invoice-detail (the demo snapshot was rebuilt) and a domain's age on operator-domains (294d to 295d); the baseline was not accepted. make lint passes. Deliberate breaks, scripts/mutation-check -side new, all CAUGHT (36): dbRuntime runs the migrations on the runtime pool dbRuntime skips the boot grant dbRuntime leaves the migration pool open dbRuntime skips the pending check when db-migrate-dsn is unset dbRuntime migrates on db-dsn when db-migrate-dsn is unset GrantRuntimeRole without WITH INHERIT TRUE GrantRuntimeRole with %s in place of %I PendingMigrations takes a missing ledger as current PendingMigrations resolves the ledger without the public. qualifier the migration pool opened without search_path=public the migration pool opened without role=none the runtime pool opened without role=none ConnectMigration keeps the search_path and role the string carries dbMigration falls back to db-dsn ValidateStart accepts a malformed db-migrate-dsn db-migrate-dsn-file is not a database secret pair db-migrate-dsn-file is not a database secret pair (the migrate commands read db-dsn-file only) the bootstrap guard removed reset-test-db.sh guard removed seed-demo.sh guard removed reset-app-db.sh guard after its first psql startup parameters match their keys case-sensitively the migration pool opened without role=none (keys and options in the string) the runtime pool opened without role=none (keys and options in the string) the runtime pool replaces the search_path the string carries the string's spaces are written as plus signs ValidateStart reads db-migrate-dsn as set whatever it holds PendingMigrations lists only the versions the ledger shows when it is missing migrationDSN keeps whitespace migrate up accepts a whitespace-only db-migrate-dsn runtimeDSN accepts a whitespace-only db-dsn openDB runs the migrating boot for an unknown mode bootstrap-stack.sh trusts the exported MC_DB_MIGRATE_DSN reset-test-db.sh trusts the exported MC_DB_MIGRATE_DSN reset-app-db.sh trusts the exported MC_DB_MIGRATE_DSN seed-demo.sh trusts the exported MC_DB_MIGRATE_DSN |
||
|
cgalo5758
|
3cdf7bbc25 |
Create webhook partitions and take the plan-act obligation lock through core_owner
Third section of #23. Two runtime paths worked only because the console's login is a superuser; once it runs as console_runtime, both would fail. Creating a partition requires owning the parent table, so the boot step and the maintenance schedule could not create a webhook_events partition as the runtime user, and inserts would start failing once the lookahead ran out. Migration 00031 gives core.webhook_events and each of its partitions to core_owner (partitions do not follow their parent's owner) and adds core.ensure_webhook_event_partitions, a SECURITY DEFINER function owned by core_owner, with a pinned search_path, EXECUTE only for console_runtime and a window of 0 to 12 months, that computes month bounds in UTC whatever the session's time zone. EnsureWebhookEventPartitions calls it. The core baseline's initial partition block now computes its bounds in UTC too (a quick fix listed in the change): in a non-UTC session it ended the first partitions at local midnight, which the next UTC partition would overlap. The plan-act obligation move locked its row with SELECT ... FOR UPDATE on core.plan_act_obligations, which the enclosure leaves the writer only SELECT on; Postgres requires UPDATE for a row lock, so every move would fail as the runtime user. Migration 00032 adds core.lock_plan_act_obligation, owned by core_owner, which takes the same lock and returns the same row, and the LockPlanActObligation query calls it. The lock lasts until the caller's transaction ends, as before; docs/database-locks.md says where it is taken now. schemaSnapshot compares ACLs as sets, so the ownership move's down-then-up round trip compares equal on a correct Down, which re-grants core_owner. TestPlanActOwnerRuleIsStatedOnce changed: LockPlanActObligation leaves its list of queries that call core.plan_act_owns_org_type, because it now calls core.lock_plan_act_obligation, and that function joins the list of functions whose definitions must call it. TestBaselinePartitionBoundsAreUTC checks the bounds on any day, but it can observe the UTC month computation only within the 14 hours before a UTC month start, because the baseline block reads the real NOW(); a run that crosses a UTC month start between the migration and the test skips. The function's session-zone break is caught by the partition-exists assertion (the gap test's now is 2119-12-31 12:00Z, already 2120 in Pacific/Kiritimati); the last-hour insert catches bounds rendered without AT TIME ZONE 'UTC'. Full suite on reset databases: 4215 pass, 15 skip; one failure, TestEntitlementSetsWalkthrough's missing toast (#222), passed when run alone. make lint passes. Deliberate breaks, scripts/mutation-check -side new, all CAUGHT (33), among them: the month or the bounds in the session's time zone; no EXECUTE grant to console_runtime; SECURITY INVOKER; no SET search_path; EXECUTE left with PUBLIC; the partition ownership loop removed; a window of 13 allowed; monthsAhead - 1 passed; the baseline block in the session zone, or restored whole; the Down giving the parent or the partitions to the wrong owner, or without the core_owner re-grant; ACLs compared in stored order; the direct-DDL test without its SET LOCAL ROLE; the lock query as the bare FOR UPDATE again; the lock function without FOR UPDATE, without the owner rule, as SECURITY INVOKER, or returning a composite row; the owner-rule test's function list without the lock function. |
||
|
cgalo5758
|
40ce2d3d7f |
Read the database's -file key in every command that opens the database
Second section of #23, a quick fix listed in its change. Only start and config validate resolved db-dsn-file, so migrate, config list|set|clear and seed-demo, run in a container that mounts the connection string as a file, found no string and let the driver connect with its defaults. They now resolve the database's secret pair before opening it, and openDB refuses an empty db-dsn ("db-dsn is not set (MC_DB_DSN or MC_DB_DSN_FILE)") before it opens anything. connectSettings returns its error, so a test can call it; the config commands print it as before. TestDatabaseCommandsReadTheFileKeys runs migrate up, down and status, connectSettings and seed-demo with only db-dsn-file set, naming a closed port, and expects the connection failure; with neither key set it expects the refusal. It blanks MC_DB_DSN and MC_DB_DSN_FILE, so the rows, migrate down among them, never reach a real database. Full suite on reset databases: 4207 pass, 15 skip. make lint passes. Deliberate breaks, scripts/mutation-check -side new, all CAUGHT (6): migrate up, migrate down, connectSettings and seed-demo each without resolveDatabaseSecretFiles openDB without the empty db-dsn check resolveDatabaseSecretFiles resolves an empty list |
||
|
cgalo5758
|
028e2ef448 |
Grant every writer to one runtime role, and leave writers only the four writes
First section of #23. The console still connects as before; this section puts in place the role its runtime user will hold. Each stream's new migration grants its writer, and SELECT on its goose ledger, to console_runtime, a NOLOGIN role core creates, and revokes the writer from member_console where an earlier release granted it there. The five baselines no longer grant a writer to any login, so a fresh install needs no role of a particular name. The baselines granted ALL on their tables to the writers, which gave the runtime path TRUNCATE, REFERENCES, TRIGGER and MAINTAIN as well. With TRIGGER, a user holding the writer could put a trigger on an ordinary table; a SECURITY DEFINER function's write to that table then ran the trigger as the function's owner, which wrote the enclosed tables directly. Each stream's writer_dml_only migration revokes the four, and TestRuntimeRolePrivileges fails, naming the relation and the privilege, when console_runtime holds any of them, holds a column grant that reopens a write, or is denied a write it needs. The enclosure tests now drop to console_runtime instead of core_writer or a copied role, and the conferral enclosure test probes TRUNCATE, CREATE TRIGGER and LOCK TABLE on the three position tables. Full suite on reset databases: 4195 pass, 15 skip; one failure, TestIntegrationSettingsWalkthrough's missing toast (#222), passed when run alone. make lint passes. Deliberate breaks, scripts/mutation-check -side new, all CAUGHT (39): core 00029 without GRANT core_writer TO console_runtime fedwiki 00005 grants domains_writer in place of fedwiki_writer stripe 00006 without its revoke from member_console discourse 00004 without its ledger grant (two tests) domains 00003 without GRANT domains_writer TO console_runtime domains 00003's revoke without its guard core and discourse baselines with the login grant restored core 00029's Down revokes core_writer from console_runtime domains 00003's Down revokes domains_writer from console_runtime core 00029's Down drops the role without its dependency handler core 00029's Down grants core_writer back to member_console core 00030 emptied (catalog test, three probes) core 00030 leaves TRIGGER (catalog test, probe) core 00030 revokes from two position tables only the domains, fedwiki, stripe and discourse writer_dml_only emptied GRANT ALL on integration_config_overrides appended after the revoke 00009's grant loses core_writer 00027's SELECT on acts becomes SELECT, INSERT (two tests) 00005's revoke loses INSERT; 00024's SELECT becomes SELECT, INSERT core 00029 also grants INSERT, or TRIGGER, on pool_provisions (four) column UPDATE and REFERENCES grants on pool_provisions, column UPDATE on the outbox's subject_id test side: the membership assertion without its fedwiki_writer exception; a locked entry for a table that does not exist The two converted checks were not run on the old side. |
||
|
cgalo5758
|
fe75c82759 | Archive the Discourse forum-ids change | ||
|
cgalo5758
|
f81b2ca66e | Tick #163's landing tasks | ||
|
cgalo5758
|
625295166a |
Record the data model's contradiction about stored webhook bodies
status/design-feedback.md gains an entry for the upstream design: the data model calls core.webhook_events.payload the "Full raw webhook payload" and also says it "Must not contain PII", which cannot both hold for a provider whose event bodies carry personal data. The entry gives what the console does now (the Discourse receiver keeps forum ids, the Stripe receiver redacts a list of keys and keeps card descriptors) and a wording for the upstream fix. |
||
|
cgalo5758
|
8e1127edd8 |
Name the person by id in a forum-link error, and date billing by Stripe's events
Two fixes ride along with #163, each listed in its change. When the forum reports an account created and the lookup that follows finds nothing, confirmCreated's error named the person by their email address, which reached the logs and the targeted reconcile's activity failure in Temporal. It now names the person by id; the rest of its text is unchanged. The other copies of personal data in Discourse errors are #250 (logs) and #162 (Temporal). The operator billing views are projected from Stripe's webhook events, but their "Last updated" stamp took the newest processed event of any provider, so a processed forum event moved it forward while Stripe's stream could have stopped. The stamp now reads Stripe's events only (latestProcessedStripeEventAt), and the operator-billing-views spec says so. Deliberate breaks, scripts/mutation-check -side new, all CAUGHT: the confirmation error carries the email again the confirmation error drops the person's id the stamp's provider condition removed (both subtests) |
||
|
cgalo5758
|
5219a45a4b |
Keep only the forum ids of a Discourse webhook event
The Discourse receiver stored each verified event body whole in core.webhook_events.payload, so every forum signup's email address, username, name, avatar, profile fields and external id stayed in the database and its backups, members and non-members alike (#163). Nothing reads a stored Discourse payload back: the receiver handles the event from the body it was delivered. The receiver now parses the body once, after it takes the dedupe key, and stores only the ids processing reads: {"user":{"id":...}} for user_created, {"group_user":{"group_id":...,"user_id":...}} for the two group events. A missing id is stored as 0. A body that does not parse, or lacks the object its event names, is stored as {} and recorded failed. Email mode still reads the address from the delivered body. Two answers change. A signed body that is not JSON, or is empty, failed the JSONB insert: 500, no row, and redelivery of the same bytes, which can only fail again. It now gets a failed row and 200, as every recorded event does. A body Go parses but JSONB rejects (a \u0000 escape, invalid UTF-8) answered 500 on every delivery; its ids are now stored and it is processed. docs/models/payments-billing.md says what each receiver stores, and the comments of scripts/db-reader-role.sql stop calling the column raw provider payloads; no reader column changes. The OpenSpec change discourse-events-keep-forum-ids is added with this commit. Ride-along: the test helper newMapping drops a return value nothing read. Deliberate breaks, scripts/mutation-check -side new, 29 records, all CAUGHT: the insert stores the raw body (8 rows of both tests) keep copies the email into the stored user keep drops UserID from the stored group user (both group rows) keep swaps GroupID and UserID (both group rows) keep takes the user object for every event type (3 rows) keep stores a user with id 0 when the user object is nil a parse error stores keep of the partly filled view the parse-error path answers 500 (the three parse-error rows) resolvePerson is called with an empty email an event of another type is not dropped a group event without group_user reconciles from the user object the insert writes the body into provider_environment the processed body is the stored one (NUL escape, invalid UTF-8) the stored group user omits a zero group_id, and a zero user_id |
||
|
cgalo5758
|
ffe983aad0 |
Archive the metered usage recount change
Every task is done and #165 is closed. The change's specs merge into the main specs: entitlements gains the rule that a metered key's usage is an absolute count its owner writes and that usage falls when a counted resource leaves its counted state, modifies the usage check, and loses the decrement and reconciliation requirements; fedwiki-sites gains the slot release, the count under the usage row's lock, the slot holds and the boot refusal, modifies ten requirements, and loses the decrement and convergence requirements. All 90 specs validate strictly. |
||
|
cgalo5758
|
04a7b91ffd |
Tick #165's landing tasks
The suite passed on the final tree, the change validates, #165 and #231 are closed with their comments, and #186, #225, #184, #44 and #192 carry what this change did to them. |
||
|
cgalo5758
|
0d81d9b627 |
Describe usage as a count its owner writes under the row's lock
docs/models/entitlements.md (invariant 13) and docs/models/resource-pools.md (invariant 8) now say that a metered key's usage row holds an absolute count of the pool's units, written by the key's owner under the row's lock with the count as its own statement, and that consumers only read it. The pages no longer describe a counter that consumers increment or an atomic draw against it; the purpose list, the consumers row, the pools introduction, invariants 4, 7 and 9, and the vocabulary lines of the pools and provider-integration pages follow. docs/go-conventions.md: a transaction whose correctness needs each statement's own snapshot opens with db.ReadCommitted, and the doc comment of the function that opens it, or of the function it relies on, says so; the FedWiki sync's insert and farm-absent delete, the permanent delete's row transaction and DecrementUsageActivity now say it. status/design-feedback.md records that the design describes usage as a counter consumers increment and does not name reservations in flight. The deadcode entries for startClaimVerification and claimErrorMessage now name #44, and the complexity exceptions of CreateSite and swapActiveSite name #174, which deletes both functions; their pinned texts are unchanged. |
||
|
cgalo5758
|
446178e0af |
Run every FedWiki site path at once and compare the figure with the count
TestConcurrentLifecycleRunLeavesTheFigureOnTheRows_DB is the test #165's "Done when" asks for. One pool under a force_reduce limit, two sibling workspaces; a create, a permanent delete, an archive, a restore and a swap each run in their own Temporal test environment over the shared activities, and the downgrade sweep is called directly. The fake farm holds the first request of all six until every one has arrived (Farm.HoldUntilAllArrive); the test then lowers the limit, holds the pool's usage row in a side transaction, and releases it only once all six writers are queued on that row's lock, so their recounts contend. After each path returns, and at the end, one read-only REPEATABLE READ transaction finds the stored figure equal to CountSlotsByPool; at the end it equals the active rows the end statuses give, and no hold of the run is left. Deliberate breaks, scripts/mutation-check -side new: CAUGHT a1: the lock-and-count counts before it locks CAUGHT a2: the recount is one UPDATE with a subquery, no lock statement CAUGHT a2b: the same UPDATE, labelled as the lock statement CAUGHT b: finishAllWith skips DeleteSlotHolds CAUGHT c: the sweep's park writes no recount A gate comparing a count read before its lock is not reachable here; the usage package's lock-wait tests catch it. go test -race -count=1 ./internal/integrations/fedwiki/workflows/...: ok git.coopcloud.tech/wiki-cafe/member-console/internal/integrations/fedwiki/workflows 203.521s The new test passed -count=20, and the suite passed. |
||
|
cgalo5758
|
1faf520bf2 |
Show FedWiki's stored site figure on every member surface
The sites card, the create form's disabled commit, Keep active's choice between an activation and a swap, and GET /api/fedwiki/quota now read the pool's stored figure through usage.Read, the same row the member's entitlements panel and the operator's pools panel read, and the figure the slot gate decides on. A create or activation in flight therefore counts: two active sites and a held create at a limit of 3 read "3 of 3 active sites used" with Create disabled. A failed read is no longer answered as "no entitlement": the card shows its allowance notice with the sites listed, the form's commit says the limit could not be checked, Keep active answers that the sites could not be loaded, and the JSON endpoint answers 500. A missing usage row under an entitlement, and a workspace with no primary pool, are failed reads, as the gate treats them. With no entitlement the endpoint reports 0, 0, false instead of counting every row of the workspace. Both copies of getWorkspaceQuota, usage.ActiveSites, CountActiveSitesByPool and CountSitesByWorkspace are deleted. No template or sentence changed; make screens moved nothing but a domain's age. |
||
|
cgalo5758
|
9ff125dd26 |
Write the FedWiki sync's status changes under the site row's lock
The sync no longer writes the farm's earlier status over an outcome that committed after the farm list was taken. ListFedWikiSitesActivity reads the database clock before the farm call and passes it on as ListedAt. A site whose request was updated at or after that moment is read from the farm again. Each status write locks the site row, checks again for an accepted request and for a request updated since the entry was read, writes, and recounts the pool, all at READ COMMITTED. The insert and the farm-absent delete recount in their own transactions; a status that did not change writes only the observed storage and last-modified time. SyncSitesToDBActivity is split into functions under the complexity limit and loses its exception. The site's name is now servable or not by the status the row holds after the write, not by the listed status. A status-unchanged pass writes no status, so following the listed one would make an archived name servable again. Two older gaps in the same code ride along. A site permanently deleted after the list was taken was inserted again as a System row for one tick; both request sets now also match a request by its recorded domain, so the sync reads that domain again. A row a create recorded after the pass read its rows had the farm's status written over it by the insert's upsert; the insert is now InsertSiteFromFarm, ON CONFLICT DO NOTHING, and leaves that domain to the next pass. The stale farm list interleaving now leaves its row archived and its name unservable. The package's Temporal test environment waits 30 seconds instead of 3, because a run can reach a sync pass whose closing recount reads every pool in the shared test database (#192). No history is recorded for the sync workflow; the recorded ones replay unchanged. |
||
|
cgalo5758
|
60981ce396 |
Read a closed FedWiki activation back from the farm before settling it
Settle no longer ends a closed set_status to active as failed while its request still holds a slot. It starts ReconcileFedWikiSiteStatusWorkflow (reconcile-fedwiki-status-<instance>-<generation>), which loads the request, reads the farm's entry with no attempt limit, and records the site active when the farm holds it so, or ends the request failed with retries_exhausted when the farm holds another status or no entry, or when the site row is gone. Either outcome removes the hold and recounts. A farm that cannot answer keeps the request accepted and its slot held. The reconcile's request load retries without an attempt limit too. A reconcile already running counts as handed on. A request the first pass's reconcile settled before the second pass reached it is logged at Info, not as a failed finish. The boot order and settle then sync interleavings now run the reconcile, and each compares the site, the request and the figure before and after the next sync. docs/database-locks.md gives settle's and the reconcile's rows. No history is recorded for the new workflow; the recorded ones replay unchanged. |
||
|
cgalo5758
|
1b2dc773eb |
Count FedWiki's site usage instead of keeping a running total
The fedwiki_sites usage row now holds an absolute figure: a pool's active site rows plus the slots its in-flight creates and activations hold. Every writer recounts under the usage row's own lock, with the lock and the count as separate statements under READ COMMITTED, so each count sees every change committed before the lock was granted. - fedwiki.slot_holds keeps one row per request between its slot gate and its outcome. It has no foreign key to core.lifecycle_requests, because no integration table references a delivery table; the count joins the request row. - The create and status workflows gate on a version marker. Version 1 takes the slot in HoldSiteSlotActivity and drops the compensating decrement; the recorded histories replay through the default branch. A status change whose farm call fails for a reason other than a refusal reads the farm back before releasing its slot. - Each outcome step removes its hold and recounts in the transaction that records the outcome; settle finishes through the same function; the sweep takes no slot before its farm call and reverts a reactivation that finds the pool full, counting held slots in its headroom; the sync ends and boot begins with a recount of every pool, which writes only a figure that changed. - AtomicIncrementUsage, AtomicDecrementUsage, RaiseNumericUsageTo, BoundNumericUsage, ListWorkspaceIDsHoldingUsage, CountSiteSlotsByPool and DeleteSiteReturningStatus are deleted, with the boot repair and the sync's drift arithmetic. Core keeps LockNumericUsage and SetNumericUsage. - docs/database-locks.md gives the usage row's place in the lock order and the gate's, outcome, sweep and boot paths. Tests: the six interleavings now find the figure equal to the rows plus holds, except that the stale farm list still leaves its row active. Four changed a step besides their expectation: the double take restores through HoldSiteSlotActivity at a limit of 2, the boot order and settle then sync hold their slot, the double release's side transaction writes no decrement, and the unlocked raise waits on LockNumericUsage and recounts in its side transaction. Settle then sync starts from a figure of 1. quota_check_db_test.go's failing limit read runs the recorded branch. TestGrantRevocationBlocksSiteCreation is now TestGrantRevocationLeavesTheFigureOverTheLimit. The two server goldens that call World.Use write the same figures from a zero row, and no golden moved; the suite passed. |
||
|
cgalo5758
|
b26a7c1e45 |
Pin the six ways FedWiki's site usage drifts from its rows
One database test per interleaving, each asserting the figure today's code leaves against the active rows read in the same statement: the unlocked raise, the sync's double release, the double take and the stale farm list wait on a lock the test holds; the boot order and settle then sync run their steps in order. Each comment names the section of #165 that changes its expectation, and the tasks name the three tests whose steps change with it. The figure-and-rows read is one fedwikitest helper. The suite passed except TestPlanLaddersWalkthrough, which failed on the toast wait #222 describes and passed when run alone. |