Should pure functions get property-based tests beside their examples? #235

New Issue

2026-10-16T04:59:29Z

cgalo5758 commented

2026-10-16 04:59:29 +00:00

The question

Should the console's pure functions get property-based tests beside their example tests?

An example test picks an input and states the answer. A property-based test states a rule that must hold for every input, and a library generates many inputs to try to break it, then shrinks a failing input to the smallest one that still fails. The inputs are seeded, so a failure reproduces. The technique fits code that reads no database and calls no service: internal/plans, the derive steps of the read pages, the domain check's classifier and the form library's checks.

Many of the console's defects have been about order: an answer that changes when the same input arrives in another order. "Shuffling the input never changes the answer" is one property that covers every order at once, where examples cover only the orders someone wrote down.

What depends on the answer

Whether order and edge cases in pure code are tested by a generator or only by the examples someone thinks of. Whether the repository takes a test-only dependency for it.

A likely first catch, found by reading plans.Across (internal/plans/offer.go) and not yet run: when two ladders both offer a disabled Checkout for different reasons, for example a one-time price on one and no price on the other, Across keeps the reason of whichever ladder comes first, so its answer depends on the order of the ladders. Today that only changes the reason Checkout logs, since the Products page shows one message for every disabled move. TestAcross has no case with two different disabled reasons.

Options known so far

Try it on plans.Offer and plans.Across with a maintained library such as pgregory.net/rapid, after checking that its license fits the project's, as a test-only dependency. Keep it if it finds what the examples miss.
Try it with the standard library's testing/quick: no new dependency, but no shrinking of failing inputs, and the package takes no new features.
Keep example tests only.

Decided when

The project records whether it uses property-based tests, with which library, and for which kind of code; or that it does not, and why.

Related: #228 asks the same question for generated mutants, and #225 covers order in queries.

## The question Should the console's pure functions get property-based tests beside their example tests? An example test picks an input and states the answer. A property-based test states a rule that must hold for every input, and a library generates many inputs to try to break it, then shrinks a failing input to the smallest one that still fails. The inputs are seeded, so a failure reproduces. The technique fits code that reads no database and calls no service: `internal/plans`, the derive steps of the read pages, the domain check's classifier and the form library's checks. Many of the console's defects have been about order: an answer that changes when the same input arrives in another order. "Shuffling the input never changes the answer" is one property that covers every order at once, where examples cover only the orders someone wrote down. ## What depends on the answer Whether order and edge cases in pure code are tested by a generator or only by the examples someone thinks of. Whether the repository takes a test-only dependency for it. A likely first catch, found by reading `plans.Across` (`internal/plans/offer.go`) and not yet run: when two ladders both offer a disabled Checkout for different reasons, for example a one-time price on one and no price on the other, `Across` keeps the reason of whichever ladder comes first, so its answer depends on the order of the ladders. Today that only changes the reason Checkout logs, since the Products page shows one message for every disabled move. `TestAcross` has no case with two different disabled reasons. ## Options known so far - Try it on `plans.Offer` and `plans.Across` with a maintained library such as `pgregory.net/rapid`, after checking that its license fits the project's, as a test-only dependency. Keep it if it finds what the examples miss. - Try it with the standard library's `testing/quick`: no new dependency, but no shrinking of failing inputs, and the package takes no new features. - Keep example tests only. ## Decided when The project records whether it uses property-based tests, with which library, and for which kind of code; or that it does not, and why. Related: #228 asks the same question for generated mutants, and #225 covers order in queries.

cgalo5758 added the

labels 2026-10-16 04:59:29 +00:00

Sign in to join this conversation.