These games are driven by a two-word parser. You can understand the puzzle and still fail because you did not guess the exact verb its author chose. The knowledge barrier is the parser, not the puzzle, so I put a grounded language layer in front of it.
The architecture
The system has two separated layers. The Bridge contains clean-room AGI and Z-Machine interpreters, GPL licensed, exposing live game state across a process boundary over WebSocket or pipe. The proprietary Brains layer holds the natural language orchestrator, metadata library and voice interface.
Grounding carries the system
Each game has a pre-generated, compacted metadata library. Its JSON logic maps describe that game's actual rules, objects and state transitions. Every model response is resolved against those maps, so the model cannot invent a command the game does not accept.
This is a structural guarantee. Prompt instructions can ask a model to stay within a game's vocabulary, but they cannot make that vocabulary binding. Grounding can. The resolver turns an open-ended model response into an action that the live interpreter recognizes.
Route locally, escalate by confidence
A small local model handles most turns and escalates to a cloud model by confidence. A judged benchmark picks the best practical local model for the user's VRAM tier, so model selection is tied to both observed behavior and available hardware.
Protect the playthrough
Dead-state protection warns before the player enters a state from which the game cannot be won. A progressive hint system can reveal help in stages, while autopilot lets the model play. Voice input and output make the whole game playable hands-free.
Autopilot turns a response into an agent
A grounded response is a translation problem. Autopilot makes it an agent problem: the player can hand over a hard section or watch passively while the same loop runs continuously. It reads live game state, chooses the next action, resolves that proposal against the logic map, executes it through the interpreter, observes the result and repeats. Long-horizon errors compound, so game state remains the only ground truth. Grounding keeps the loop from drifting into invented commands because every action must survive the game's real rules before execution. The dead-state check is inside the cycle, making unattended play acceptable by rejecting moves that would make the game unwinnable. Progressive hints offer graded nudges without taking control. Streamer mode turns the model into a live audience co-pilot, while an automatically generated map and journal preserve the playthrough.
The project ships an MCP server, has CI and a large automated test suite. It never bundles game files. The user points it at their own legal copy. The repository is private.
Ken Faiman · Applied AI, agent & evaluation systems · faiman.com