The plan-wide acts (tier removal, default change, reorder, tier add and delete) are now sent against each other and against a conferral, each interleaving chosen by a lock a side transaction holds and releases once the requests wait on it. dbtest.AwaitLockWait proves a request is queued, and PostgreSQL grants a row lock to its waiters in the order they queued, so no test sleeps. The kit gains Backend.Waiters, how many backends must wait at once, with a test of its own; its first caller is these tests. TestTierRemovalAgainstDefaultChange runs a removal and a default change on one pool in each order: queued on the pool's row, and one after the other. Every case ends with no live position on a deleted tier. TestDefaultChangeTwiceSerializesOnThePool sends one default change twice and shows one request grandfathering the pool and the other finding the legacy grant the first wrote. TestPlanActsRefuseWhileARuleChangeHoldsTheRendezvous holds the exclusive rendezvous while all five acts run, and pins each act's contention answer: the removal's and the organization's "Another entitlement set change is in progress. Try again." and the generic sentence of the reorder, the add and the delete. It waits out the 5 s lock timeout once. TestRenumberingCommitsDuringAConferral commits a reorder, a tier delete and a tier removal while a default change's conferral waits between its rank reads; the files pin the transition as it is recorded today (upgrade 0 to 1 for the reorder where a transfer belongs; downgrade 1 to 0 for the delete and the removal), because a renumbering takes the rendezvous in shared mode only. Pinned with #187: TestReorderEnactsAgainstTheRankZeroItSaved sends two reorders whose enactments queue on one pool; the first enacts against the rank 0 it saved after the second has put the other product back at rank 0, so it grandfathers the holder onto a legacy grant of the product that is rank 0 again, and the second finds the holder on that grant and leaves it. No output changed: the commit adds tests, their golden files and one field of the test kit, and touches no production file. The names and results of ./internal/server and ./internal/testkit/..., run after a database reset at the parent and at this commit, differ by the 13 new tests and subtests, all passing; the new tests passed three runs in a row and 20 runs of the four that do not wait out the timeout. The edit to the plan-wide acts' golden test is its JSON encoder, split so the new tests encode their files the same way; its 44 files are unchanged. The kit's wait.go and wait_test.go and the AwaitLockWait bullet of docs/testing.md change beside it. 14 breaks, each caught by its new test (new side only): 12 edit production code unchanged from the parent and 2 edit this commit's wait.go. The section's tests take about 6 s, 5 s of it the one wait. Unprobed: - Interleavings the tests do not choose, such as a removal that arrives between the default change's save and its pool lock; the cases cover the two queue orders on the pool and the two serial orders. - A conferral other than the default change's during a renumbering (a reconcile, a grant act, the floor); it is the same database function, which the reorder case reaches. - The default change's own stale candidate: a renumbering between its save and its enactment. It uses the same fixed product the reorder case reaches; no case pauses the default change there. - The two new failure messages of AwaitLockWait for a count above one; they are text, read only when a wait fails.
98 lines
3.5 KiB
Go
98 lines
3.5 KiB
Go
// SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Commercial
|
|
// SPDX-FileCopyrightText: 2025-2026 Christian Galo
|
|
|
|
package dbtest
|
|
|
|
import (
|
|
"database/sql"
|
|
"fmt"
|
|
"testing"
|
|
"time"
|
|
)
|
|
|
|
// lockWaitDeadline bounds how long AwaitLockWait polls, well over any wait a
|
|
// test provokes.
|
|
const lockWaitDeadline = 10 * time.Second
|
|
|
|
// Backend selects the server backend a test waits on. Exactly one of PID,
|
|
// BlockedBy and Query is set.
|
|
type Backend struct {
|
|
PID int // pg_backend_pid() of the connection that is expected to wait
|
|
BlockedBy int // pg_backend_pid() of the connection whose lock some backend waits on
|
|
Query string // text the waiting backend's current statement contains, within this database
|
|
// Waiters is how many selected backends must wait at once; zero means
|
|
// one. Two requests queued on one row lock are Waiters 2 with Query;
|
|
// BlockedBy counts only the first of them, because the second waits on
|
|
// the first's tuple lock.
|
|
Waiters int
|
|
// Abort is called between polls. A non-nil error fails the test with its
|
|
// text, so a test whose waiting call can return without ever waiting
|
|
// reports that instead of polling to the deadline.
|
|
Abort func() error
|
|
}
|
|
|
|
// waiting is the statement that counts the backends b selects that wait on a
|
|
// lock, and its argument.
|
|
func (b Backend) waiting() (string, any) {
|
|
switch {
|
|
case b.BlockedBy != 0:
|
|
return `SELECT count(*) FROM pg_stat_activity WHERE $1 = ANY(pg_blocking_pids(pid))`, b.BlockedBy
|
|
case b.Query != "":
|
|
return `SELECT count(*) FROM pg_stat_activity
|
|
WHERE datname = current_database() AND pid <> pg_backend_pid()
|
|
AND wait_event_type = 'Lock' AND position($1 in query) > 0`, b.Query
|
|
default:
|
|
return `SELECT count(*) FROM pg_stat_activity WHERE pid = $1 AND wait_event_type = 'Lock'`, b.PID
|
|
}
|
|
}
|
|
|
|
// AwaitLockWait polls pg_stat_activity until b waits on a lock (b.Waiters
|
|
// backends at once, when it is above one), so a test can assert what the
|
|
// waiting backends hold without sleeping for a fixed time. It fails the test
|
|
// when b has not waited after ten seconds or when b.Abort reports an error.
|
|
func AwaitLockWait(tb testing.TB, d *sql.DB, b Backend) {
|
|
tb.Helper()
|
|
query, arg := b.waiting()
|
|
deadline := time.Now().Add(lockWaitDeadline)
|
|
for {
|
|
var waiting int
|
|
if err := d.QueryRow(query, arg).Scan(&waiting); err != nil {
|
|
tb.Fatalf("read pg_stat_activity: %v", err)
|
|
}
|
|
if waiting >= b.waiters() {
|
|
return
|
|
}
|
|
if b.Abort != nil {
|
|
if err := b.Abort(); err != nil {
|
|
tb.Fatalf("%v", err)
|
|
}
|
|
}
|
|
if time.Now().After(deadline) {
|
|
tb.Fatal(b.neverWaited())
|
|
}
|
|
time.Sleep(5 * time.Millisecond)
|
|
}
|
|
}
|
|
|
|
// waiters is how many selected backends AwaitLockWait waits for.
|
|
func (b Backend) waiters() int {
|
|
return max(b.Waiters, 1)
|
|
}
|
|
|
|
// neverWaited is the failure message for a selection that not enough
|
|
// backends matched before the deadline.
|
|
func (b Backend) neverWaited() string {
|
|
switch {
|
|
case b.Waiters > 1 && b.BlockedBy != 0:
|
|
return fmt.Sprintf("fewer than %d backends waited on a lock held by backend %d", b.Waiters, b.BlockedBy)
|
|
case b.Waiters > 1:
|
|
return fmt.Sprintf("fewer than %d backends running a statement containing %q waited on a lock", b.Waiters, b.Query)
|
|
case b.BlockedBy != 0:
|
|
return fmt.Sprintf("no backend waited on a lock held by backend %d", b.BlockedBy)
|
|
case b.Query != "":
|
|
return fmt.Sprintf("no backend running a statement containing %q waited on a lock", b.Query)
|
|
default:
|
|
return fmt.Sprintf("backend %d never waited on a lock", b.PID)
|
|
}
|
|
}
|