Removing a request whose workflow never started takes no instance lock, so a repeat of it can be answered as a conflict #229

New Issue

2026-10-16T02:59:48Z

cgalo5758 commented

2026-10-16 02:59:48 +00:00

What happens

When a FedWiki handler has recorded a lifecycle request and Temporal then answers that the request's workflow does not exist, startRequestWorkflow deletes the request through integration.DeleteUnstartedLifecycleRequest, so the member's retry records a fresh one. The delete runs on the plain database handle and takes no lock. Recording a request and finishing one both take the instance lock first (LockLifecycleInstance); this delete is the one write to a request's row that does not.

Recording a repeat takes the instance lock, then reads twice: the instance's latest generation, then the request at the posted address. Each read sees what was committed when it ran. If the delete commits between the two, the first read finds the request's generation, the second finds nothing at the address, and the repeat is answered LifecycleConflict. Neither order of the two acts gives that answer: before the delete the repeat is answered as open, and after it the repeat is recorded as new.

The member then reads "This site changed since the list loaded.", and a JSON client gets 409 "Another request holds this instance and generation." while no request holds it.

The swap and the status changes that rotate the active site look for the request under the workspace lock before they record, so they answer a repeat before the two reads. Creates, deletes and the other status changes record without that lock and are exposed.

docs/database-locks.md (lines 269-271) records the missing lock and says a repeat that races the delete "may be answered from either side of the delete". It does not mention this third answer.

What should happen

The delete takes the instance lock in a transaction of its own, as recording does, so a repeat sees the request either before the delete or after it. A swap's two requests are deleted in one transaction that takes their instance locks in the order the requests were recorded. The lock page then drops its sentence about the race.

Where

internal/integration/lifecycle.go:441-455 (DeleteUnstartedLifecycleRequest) and internal/integration/queries/lifecycle_requests.sql:64-72
internal/integrations/fedwiki/web/requests.go:239-244 (startRequestWorkflow, which every FedWiki create, delete and status change calls)
internal/integration/lifecycle.go:138-142 and :292-311 (the two reads)
docs/database-locks.md:269-271

Steps

Found by reading the code; not reproduced.

Submit the same create, delete or status change twice at once, so the second submission looks for the request before the first has committed it, and goes on to record.
Have the first submission's workflow start fail in a way after which Temporal answers that the workflow does not exist.
The first submission deletes its request between the second submission's two reads.
The second submission is answered as a conflict.

The window is two consecutive statements, so this is rare.

Why it matters

When it happens, the member is told that someone else changed the site, and an API client is told that another request holds an address that nothing holds. The rewrite in #165 and the shared request helpers in #174 would copy this delete as it stands.

## What happens When a FedWiki handler has recorded a lifecycle request and Temporal then answers that the request's workflow does not exist, `startRequestWorkflow` deletes the request through `integration.DeleteUnstartedLifecycleRequest`, so the member's retry records a fresh one. The delete runs on the plain database handle and takes no lock. Recording a request and finishing one both take the instance lock first (`LockLifecycleInstance`); this delete is the one write to a request's row that does not. Recording a repeat takes the instance lock, then reads twice: the instance's latest generation, then the request at the posted address. Each read sees what was committed when it ran. If the delete commits between the two, the first read finds the request's generation, the second finds nothing at the address, and the repeat is answered `LifecycleConflict`. Neither order of the two acts gives that answer: before the delete the repeat is answered as open, and after it the repeat is recorded as new. The member then reads "This site changed since the list loaded.", and a JSON client gets 409 "Another request holds this instance and generation." while no request holds it. The swap and the status changes that rotate the active site look for the request under the workspace lock before they record, so they answer a repeat before the two reads. Creates, deletes and the other status changes record without that lock and are exposed. `docs/database-locks.md` (lines 269-271) records the missing lock and says a repeat that races the delete "may be answered from either side of the delete". It does not mention this third answer. ## What should happen The delete takes the instance lock in a transaction of its own, as recording does, so a repeat sees the request either before the delete or after it. A swap's two requests are deleted in one transaction that takes their instance locks in the order the requests were recorded. The lock page then drops its sentence about the race. ## Where - `internal/integration/lifecycle.go:441-455` (`DeleteUnstartedLifecycleRequest`) and `internal/integration/queries/lifecycle_requests.sql:64-72` - `internal/integrations/fedwiki/web/requests.go:239-244` (`startRequestWorkflow`, which every FedWiki create, delete and status change calls) - `internal/integration/lifecycle.go:138-142` and `:292-311` (the two reads) - `docs/database-locks.md:269-271` ## Steps Found by reading the code; not reproduced. 1. Submit the same create, delete or status change twice at once, so the second submission looks for the request before the first has committed it, and goes on to record. 2. Have the first submission's workflow start fail in a way after which Temporal answers that the workflow does not exist. 3. The first submission deletes its request between the second submission's two reads. 4. The second submission is answered as a conflict. The window is two consecutive statements, so this is rare. ## Why it matters When it happens, the member is told that someone else changed the site, and an API client is told that another request holds an address that nothing holds. The rewrite in #165 and the shared request helpers in #174 would copy this delete as it stands.

cgalo5758 added the

labels 2026-10-16 02:59:48 +00:00

cgalo5758 referenced this issue

2026-10-16 03:00:02 +00:00

FedWiki's site usage is a running total that many code paths adjust; derive it instead #165

cgalo5758 referenced this issue

2026-10-22 18:40:11 +00:00