A create or a settings save sometimes lands without its success toast, so two walkthroughs fail now and then #222

Open
opened 2026-10-16 01:52:36 +00:00 by cgalo5758 · 2 comments
Owner

What happens

TestIntegrationSettingsWalkthrough has failed at least four times in full suite runs, and TestPlanLaddersWalkthrough once, each on the same wait:

still false after 20s: () => !!document.querySelector("#successToast.show, #errorToast.show")

Each passed when run again alone; the plan ladders walkthrough passed four runs out of four.

The wait is browsertest.AwaitToast (test/e2e/browsertest/shell.go:41). It polls every 100 ms for 20 seconds (shell.go:43, through WaitTrue in test/e2e/browsertest/wait.go:27) until one of the two shell toasts, #successToast or #errorToast, carries Bootstrap's show class. Then it reads the toast, closes it and waits for it to hide. Every failing call came right after ClickAndLand (wait.go:97), on a page the browser had just navigated to:

  • a settings save (test/e2e/operator-walkthroughs/integration_settings_test.go:114-115 and :163-164), which redirects to the settings page with ?flash=saved (internal/server/operator_integration_settings.go:408);
  • the Clear of an override (integration_settings_test.go:178-179), which redirects with ?flash=cleared (operator_integration_settings.go:453);
  • the ladder create (test/e2e/operator-walkthroughs/plan_ladders_test.go:175, :187), which lands on the new ladder's page with ?flash=created (internal/server/operator_record_page.go:57-58).

No htmx response can carry the toast to such a page. The server writes the message into #successToast and marks the element data-show-on-load (internal/embeds/templates/operator.html:383-384). internal/embeds/static/success-toast.js shows it once: showFlashToastOnLoad (:52) runs from htmx.onLoad (:61-64), removes the marker and shows the toast with a five-second autohide (:57-58). Earlier runs of these walkthroughs missed the toast on about one landing in thirty, and in those misses data-show-on-load stayed on the element, so showFlashToastOnLoad had never run.

A likely cause, not yet confirmed: the operator page loads htmx and success-toast.js as separate deferred scripts, with one other script between them (operator.html:27, :29). A deferred script runs after parsing has ended, when document.readyState is interactive, so htmx 4.0.0 (internal/embeds/static/htmx.min.js) starts itself with setTimeout instead of on DOMContentLoaded. That start processes the body once and fires htmx:after:process, the event htmx.onLoad listens for. The browser may run the timer before it runs success-toast.js, for example while that file is still loading. The listener is then added after the only event it would have caught, and the toast never shows. A busy machine during a full run makes that order more likely, which fits a walkthrough that passes alone.

The wait also cannot see a toast that showed and hid itself before polling began, and its message does not say which of the two happened.

What should happen

Every page that arrives with a flash message shows its toast, whatever order the deferred scripts run in. When the walkthrough wait times out, its message says whether data-show-on-load is still set, so a toast that never showed is told apart from one the wait missed.

Where

  • internal/embeds/static/success-toast.js:52-69 and the script order in internal/embeds/templates/operator.html:25-34.
  • AwaitToast in test/e2e/browsertest/shell.go:41-52.
  • The waits that follow a landing: integration_settings_test.go:115, :164, :179 and plan_ladders_test.go:187, which have failed, and products_test.go:99 and entitlement_sets_test.go:74 in the same directory, which wait the same way after a create.

Steps

  1. Bring up the test stack and start the console, as docs/testing.md describes for the Go browser walkthroughs (type 7, end-to-end tests).
  2. Run make test several times. Now and then TestIntegrationSettingsWalkthrough or TestPlanLaddersWalkthrough fails with the message above.
  3. Run the failing walkthrough alone with go test ./test/e2e/operator-walkthroughs/ -run <name> -count=1. It passes.

Why it matters

An operator who creates a plan ladder, a product or an entitlement set, or saves or clears an integration setting, is sometimes left without the confirmation the page is meant to give. And a walkthrough that fails by chance teaches people to rerun it, so a real failure of the same wait gets rerun too.

## What happens `TestIntegrationSettingsWalkthrough` has failed at least four times in full suite runs, and `TestPlanLaddersWalkthrough` once, each on the same wait: ``` still false after 20s: () => !!document.querySelector("#successToast.show, #errorToast.show") ``` Each passed when run again alone; the plan ladders walkthrough passed four runs out of four. The wait is `browsertest.AwaitToast` (`test/e2e/browsertest/shell.go:41`). It polls every 100 ms for 20 seconds (`shell.go:43`, through `WaitTrue` in `test/e2e/browsertest/wait.go:27`) until one of the two shell toasts, `#successToast` or `#errorToast`, carries Bootstrap's `show` class. Then it reads the toast, closes it and waits for it to hide. Every failing call came right after `ClickAndLand` (`wait.go:97`), on a page the browser had just navigated to: - a settings save (`test/e2e/operator-walkthroughs/integration_settings_test.go:114-115` and `:163-164`), which redirects to the settings page with `?flash=saved` (`internal/server/operator_integration_settings.go:408`); - the Clear of an override (`integration_settings_test.go:178-179`), which redirects with `?flash=cleared` (`operator_integration_settings.go:453`); - the ladder create (`test/e2e/operator-walkthroughs/plan_ladders_test.go:175`, `:187`), which lands on the new ladder's page with `?flash=created` (`internal/server/operator_record_page.go:57-58`). No htmx response can carry the toast to such a page. The server writes the message into `#successToast` and marks the element `data-show-on-load` (`internal/embeds/templates/operator.html:383-384`). `internal/embeds/static/success-toast.js` shows it once: `showFlashToastOnLoad` (`:52`) runs from `htmx.onLoad` (`:61-64`), removes the marker and shows the toast with a five-second autohide (`:57-58`). Earlier runs of these walkthroughs missed the toast on about one landing in thirty, and in those misses `data-show-on-load` stayed on the element, so `showFlashToastOnLoad` had never run. A likely cause, not yet confirmed: the operator page loads htmx and `success-toast.js` as separate deferred scripts, with one other script between them (`operator.html:27`, `:29`). A deferred script runs after parsing has ended, when `document.readyState` is `interactive`, so htmx 4.0.0 (`internal/embeds/static/htmx.min.js`) starts itself with `setTimeout` instead of on `DOMContentLoaded`. That start processes the body once and fires `htmx:after:process`, the event `htmx.onLoad` listens for. The browser may run the timer before it runs `success-toast.js`, for example while that file is still loading. The listener is then added after the only event it would have caught, and the toast never shows. A busy machine during a full run makes that order more likely, which fits a walkthrough that passes alone. The wait also cannot see a toast that showed and hid itself before polling began, and its message does not say which of the two happened. ## What should happen Every page that arrives with a flash message shows its toast, whatever order the deferred scripts run in. When the walkthrough wait times out, its message says whether `data-show-on-load` is still set, so a toast that never showed is told apart from one the wait missed. ## Where - `internal/embeds/static/success-toast.js:52-69` and the script order in `internal/embeds/templates/operator.html:25-34`. - `AwaitToast` in `test/e2e/browsertest/shell.go:41-52`. - The waits that follow a landing: `integration_settings_test.go:115`, `:164`, `:179` and `plan_ladders_test.go:187`, which have failed, and `products_test.go:99` and `entitlement_sets_test.go:74` in the same directory, which wait the same way after a create. ## Steps 1. Bring up the test stack and start the console, as `docs/testing.md` describes for the Go browser walkthroughs (type 7, end-to-end tests). 2. Run `make test` several times. Now and then `TestIntegrationSettingsWalkthrough` or `TestPlanLaddersWalkthrough` fails with the message above. 3. Run the failing walkthrough alone with `go test ./test/e2e/operator-walkthroughs/ -run <name> -count=1`. It passes. ## Why it matters An operator who creates a plan ladder, a product or an entitlement set, or saves or clears an integration setting, is sometimes left without the confirmation the page is meant to give. And a walkthrough that fails by chance teaches people to rerun it, so a real failure of the same wait gets rerun too.
cgalo5758 added the
kind
bug
area/operator-uiarea/testing
priority
medium
labels 2026-10-16 01:52:36 +00:00
Author
Owner

TestPlanLaddersWalkthrough failed again on the same wait in a full suite run, after the ladder create's landing (now plan_ladders_test.go:197), and passed when run alone. The change under test touched only FedWiki tests.

`TestPlanLaddersWalkthrough` failed again on the same wait in a full suite run, after the ladder create's landing (now `plan_ladders_test.go:197`), and passed when run alone. The change under test touched only FedWiki tests.
Author
Owner

The code confirms the order this issue suspected. htmx 4.0.0 starts with setTimeout when document.readyState is not loading, which is the case for a deferred script, and its start fires htmx:after:process once for the body. htmx.onLoad adds a plain listener for that event and replays nothing, so success-toast.js:61-64 misses the page's only event whenever htmx's timer runs before that file does. The marker left on the element fits only this path: showFlashToastOnLoad (:52-58) keeps it only when the toast or Bootstrap is missing, and Bootstrap loads first (operator.html:25).

internal/embeds/static/help-init.js:129-139 has the same shape: with htmx present it sets up the help icons' tooltips and popovers only from htmx.onLoad, so a page can open with dead help icons in the same window. hint-on-choice.js:57-60 and sortable-reorder.js:97-109 avoid it by also running once on the document directly; the toast and the help icons can do the same, guarded so a second run does nothing.

The integration settings walkthrough failed this way again in phase 2c and passed on rerun.

The code confirms the order this issue suspected. htmx 4.0.0 starts with `setTimeout` when `document.readyState` is not `loading`, which is the case for a deferred script, and its start fires `htmx:after:process` once for the body. `htmx.onLoad` adds a plain listener for that event and replays nothing, so `success-toast.js:61-64` misses the page's only event whenever htmx's timer runs before that file does. The marker left on the element fits only this path: `showFlashToastOnLoad` (`:52-58`) keeps it only when the toast or Bootstrap is missing, and Bootstrap loads first (`operator.html:25`). `internal/embeds/static/help-init.js:129-139` has the same shape: with htmx present it sets up the help icons' tooltips and popovers only from `htmx.onLoad`, so a page can open with dead help icons in the same window. `hint-on-choice.js:57-60` and `sortable-reorder.js:97-109` avoid it by also running once on the document directly; the toast and the help icons can do the same, guarded so a second run does nothing. The integration settings walkthrough failed this way again in phase 2c and passed on rerun.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: wiki-cafe/member-console#222