Speech and long-running agent work operate on different timescales, and the runtime has to hold both without letting one cancel the other. Coupling them can silence the assistant during work or make an interruption cancel an unrelated task.
The architecture
Two clocks, deliberately uncoupled
Speech and interruption run in the foreground. Sessions, tools and long-running work run in the background, with results reinserted when relevant. Speech, foreground turns, background tasks and disconnects each get their own cancellation scope, so interrupting speech does not cancel its task.
A deterministic state machine makes turn behavior testable without inferring state from timers.
What counts as heard
Synthesis, delivery, queuing and playback are tracked as separate states, and delivery is only confirmed on receipt. Interrupted or stale playback does not count as heard. Without that distinction the assistant quietly records information it never actually delivered, and then reasons as though you had it.
Task control
Dispatch is versioned, deduplicates retries, rejects conflicting task IDs and acknowledges monotonically. A task is only accepted against a real delegation handle: missing context, or a response without one, fails closed rather than guessing. Reconnects keep session identity and task ordering intact.
Stop targets a task by handle rather than by whatever ran most recently, and server work is serialised to one active task so a single approval cannot cover two ambiguous runs.
Testing
937 tests pass in 39 seconds, with more test code than source, and mypy runs strict across the whole package with no findings. The release check never packages the working tree: it rebuilds the candidate from a clean archive of the commit first, so what gets tested is what would ship.
Ken Faiman · Applied AI, agent & evaluation systems · faiman.com