Fix #154: cancellable notify tasks #160

Merged
crueber merged 2 commits from fix/issue-154 into main 2026-09-05 21:12:35 +00:00
Owner

Task goroutines ignore ctx cancel: StartWebhooks used WithoutCancel, drainFanout used Background(), neither tracked in a WaitGroup — a wedged store call hung them forever, immune to drain/shutdown.

Same shape as the #74 import fix: the notify Service owns a drainCtx (cancelled by new Service.Drain, wired into serve.go phase 1 beside importSvc.Drain); both leaders derive store/network work from it and are tracked in Service.wg; new tasks refuse fast once draining (sweep-durable seqs re-drive via redrain). Leader on dead drainCtx force-ends via tasks.end (documented exception to #72 rule).

Tests: TestDrainInterruptsWedgedWebhooks + TestDrainInterruptsWedgedFanout (verified HANGING pre-fix, task stuck running) + TestDrainRefusesNewTasks. Full package suite green under -race; coverage 96.1% (>=95%); gofmt/vet clean. Doc Decisions entry appended (law 12).

Task goroutines ignore ctx cancel: StartWebhooks used WithoutCancel, drainFanout used Background(), neither tracked in a WaitGroup — a wedged store call hung them forever, immune to drain/shutdown. Same shape as the #74 import fix: the notify Service owns a drainCtx (cancelled by new Service.Drain, wired into serve.go phase 1 beside importSvc.Drain); both leaders derive store/network work from it and are tracked in Service.wg; new tasks refuse fast once draining (sweep-durable seqs re-drive via redrain). Leader on dead drainCtx force-ends via tasks.end (documented exception to #72 rule). Tests: TestDrainInterruptsWedgedWebhooks + TestDrainInterruptsWedgedFanout (verified HANGING pre-fix, task stuck running) + TestDrainRefusesNewTasks. Full package suite green under -race; coverage 96.1% (>=95%); gofmt/vet clean. Doc Decisions entry appended (law 12).
tasks.go StartWebhooks used WithoutCancel and drainFanout used
Background(); neither ran in a tracked WaitGroup, so a wedged store
call hung them forever, immune to drain/shutdown.

Same shape as the #74 import fix: the notify Service owns a drainCtx
(cancelled by the new Service.Drain, wired into serve.go phase 1
beside importSvc.Drain); both leaders derive their store/network work
from it and are tracked in Service.wg; new tasks refuse fast once
draining (sweep-durable seqs re-drive on restart). A leader observing
a dead drainCtx force-ends via tasks.end (documented exception to the
#72 endIfQuiescent-only rule).

Regression: TestDrainInterruptsWedgedWebhooks/Fanout (verified hanging
pre-fix) + TestDrainRefusesNewTasks. Coverage 96.1%. Doc Decisions
entry appended (law 12).
Pre-fix the ctx.Done arm could never fire (WithoutCancel); with the
#154 drainCtx it can, and the bare return orphaned in-flight hook
workers past wg.Wait. Break the launch loop but still Wait — workers
observe the dead ctx and fail fast, so the wait stays bounded.
Sign in to join this conversation.
No description provided.