Files
cgalo5758 6f65c744fe Pin how the plan-wide acts interleave with each other, with a rule change and with a conferral
The plan-wide acts (tier removal, default change, reorder, tier add and
delete) are now sent against each other and against a conferral, each
interleaving chosen by a lock a side transaction holds and releases once
the requests wait on it. dbtest.AwaitLockWait proves a request is queued,
and PostgreSQL grants a row lock to its waiters in the order they queued,
so no test sleeps. The kit gains Backend.Waiters, how many backends must
wait at once, with a test of its own; its first caller is these tests.

TestTierRemovalAgainstDefaultChange runs a removal and a default change
on one pool in each order: queued on the pool's row, and one after the
other. Every case ends with no live position on a deleted tier.
TestDefaultChangeTwiceSerializesOnThePool sends one default change twice
and shows one request grandfathering the pool and the other finding the
legacy grant the first wrote. TestPlanActsRefuseWhileARuleChangeHoldsTheRendezvous
holds the exclusive rendezvous while all five acts run, and pins each
act's contention answer: the removal's and the organization's "Another
entitlement set change is in progress. Try again." and the generic
sentence of the reorder, the add and the delete. It waits out the 5 s
lock timeout once. TestRenumberingCommitsDuringAConferral commits a
reorder, a tier delete and a tier removal while a default change's
conferral waits between its rank reads; the files pin the transition as
it is recorded today (upgrade 0 to 1 for the reorder where a transfer
belongs; downgrade 1 to 0 for the delete and the removal), because a
renumbering takes the rendezvous in shared mode only. Pinned with #187:
TestReorderEnactsAgainstTheRankZeroItSaved sends two reorders whose
enactments queue on one pool; the first enacts against the rank 0 it
saved after the second has put the other product back at rank 0, so it
grandfathers the holder onto a legacy grant of the product that is rank 0
again, and the second finds the holder on that grant and leaves it.

No output changed: the commit adds tests, their golden files and one field
of the test kit, and touches no production file. The names and results of
./internal/server and ./internal/testkit/..., run after a database reset at
the parent and at this commit, differ by the 13 new tests and subtests, all
passing; the new tests passed three runs in a row and 20 runs of the four
that do not wait out the timeout. The edit to the plan-wide acts' golden
test is its JSON encoder, split so the new tests encode their files the same
way; its 44 files are unchanged. The kit's wait.go and wait_test.go and the
AwaitLockWait bullet of docs/testing.md change beside it. 14 breaks, each
caught by its new test (new side only): 12 edit production code unchanged
from the parent and 2 edit this commit's wait.go. The section's tests take
about 6 s, 5 s of it the one wait.

Unprobed:
- Interleavings the tests do not choose, such as a removal that arrives
  between the default change's save and its pool lock; the cases cover the
  two queue orders on the pool and the two serial orders.
- A conferral other than the default change's during a renumbering (a
  reconcile, a grant act, the floor); it is the same database function,
  which the reorder case reaches.
- The default change's own stale candidate: a renumbering between its save
  and its enactment. It uses the same fixed product the reorder case
  reaches; no case pauses the default change there.
- The two new failure messages of AwaitLockWait for a count above one;
  they are text, read only when a wait fails.
2026-10-10 16:42:02 -05:00

153 lines
3.9 KiB
Go

// SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Commercial
// SPDX-FileCopyrightText: 2025-2026 Christian Galo
package dbtest
import (
"context"
"database/sql"
"errors"
"strings"
"testing"
"time"
)
// holdAndWait holds an advisory lock on one transaction and parks a second
// connection on it, returning both backends' pids and the statement the
// second one runs. The caller rolls the holder back to release the waiter.
func holdAndWait(t *testing.T) (holderPID, waiterPID int, stmt string, release func()) {
t.Helper()
d := Open(t)
ctx := context.Background()
key := time.Now().UnixNano()
stmt = "SELECT pg_advisory_xact_lock($1) /* dbtest wait test */"
holder, err := d.BeginTx(ctx, nil)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = holder.Rollback() })
if err := holder.QueryRowContext(ctx, `SELECT pg_backend_pid()`).Scan(&holderPID); err != nil {
t.Fatal(err)
}
if _, err := holder.ExecContext(ctx, stmt, key); err != nil {
t.Fatal(err)
}
waiter, err := d.BeginTx(ctx, nil)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = waiter.Rollback() })
if err := waiter.QueryRowContext(ctx, `SELECT pg_backend_pid()`).Scan(&waiterPID); err != nil {
t.Fatal(err)
}
done := make(chan error, 1)
go func() {
_, err := waiter.ExecContext(ctx, stmt, key)
done <- err
}()
return holderPID, waiterPID, stmt, func() {
if err := holder.Rollback(); err != nil {
t.Fatal(err)
}
if err := <-done; err != nil {
t.Fatal(err)
}
}
}
func TestAwaitLockWaitSelectsTheWaitingBackendThreeWays(t *testing.T) {
holderPID, waiterPID, _, release := holdAndWait(t)
defer release()
d := Open(t)
for name, b := range map[string]Backend{
"by pid": {PID: waiterPID},
"by the holder": {BlockedBy: holderPID},
"by statement": {Query: "dbtest wait test"},
} {
t.Run(name, func(t *testing.T) { AwaitLockWait(t, d, b) })
}
}
func TestAwaitLockWaitFailsWithTheAbortProbesText(t *testing.T) {
d := Open(t)
r := &recorder{TB: t}
func() {
defer func() {
if p := recover(); p != nil {
if _, ok := p.(stopped); !ok {
panic(p)
}
}
}()
AwaitLockWait(r, d, Backend{PID: -1, Abort: func() error { return errors.New("the call returned without waiting") }})
}()
if !strings.Contains(r.failure, "the call returned without waiting") {
t.Errorf("Fatalf = %q, want the probe's text", r.failure)
}
}
// TestAwaitLockWaitCountsTheWaiters: with Waiters 2 the wait does not end
// while one backend waits on a held lock; it ends once a second one queues
// behind it. The probe starts the second waiter on its first call, so a wait
// that ended on one waiter never calls it.
func TestAwaitLockWaitCountsTheWaiters(t *testing.T) {
d := Open(t)
ctx := context.Background()
key := time.Now().UnixNano()
const stmt = "SELECT pg_advisory_xact_lock($1) /* dbtest waiters test */"
// The holder is first, so a failing test releases it before it waits for
// the waiters' transactions.
var open []*sql.Tx
t.Cleanup(func() {
for _, tx := range open {
_ = tx.Rollback()
}
})
begin := func() *sql.Tx {
tx, err := d.BeginTx(ctx, nil)
if err != nil {
t.Fatal(err)
}
open = append(open, tx)
return tx
}
holder := begin()
if _, err := holder.ExecContext(ctx, stmt, key); err != nil {
t.Fatal(err)
}
done := make(chan error, 2)
wait := func(tx *sql.Tx) {
_, err := tx.ExecContext(ctx, stmt, key)
if rerr := tx.Rollback(); err == nil {
err = rerr
}
done <- err
}
go wait(begin())
AwaitLockWait(t, d, Backend{Query: "dbtest waiters test"})
second := begin()
probes := 0
AwaitLockWait(t, d, Backend{Query: "dbtest waiters test", Waiters: 2, Abort: func() error {
if probes == 0 {
go wait(second)
}
probes++
return nil
}})
if probes == 0 {
t.Fatal("the wait for two waiters ended while one waited")
}
if err := holder.Rollback(); err != nil {
t.Fatal(err)
}
for range 2 {
if err := <-done; err != nil {
t.Fatal(err)
}
}
}