LLM Fortress: A Living Simulation Where the AI Doesn't Drive

The Situation

This is a personal project, built on my own time and my own hardware. It started with a question about the default architecture of LLM-powered games and agent demos: almost all of them put the model in the driver's seat. The LLM decides what characters do, the harness executes it, and the world is whatever the model says it is. It's the obvious architecture — and it has obvious costs.

The first cost is nondeterminism. When the model decides world state, the world inherits the model's variance: the same situation resolves differently on every run, saves can't be trusted, and bugs can't be reproduced. The second is economics — a token cost on every tick of the simulation, which caps how many agents you can afford and how fast the world can run. The third is latency. A simulation loop running at 4× speed resolves a tick in milliseconds; even fast local inference takes seconds. Put the LLM on that critical path and the world stutters, or the world waits. And the deepest cost is a design one: agents whose every move is generated by a chatbot don't read as inhabitants of a world. They read as puppets with a very expensive puppeteer.

I'd spent my professional work on the other side of this question — building RAG search where bounding the model's context was what earned user trust. LLM Fortress was the personal-lab version of the same instinct, pushed further: what does the LLM actually add to a living world, and what should it never be allowed to touch? Dwarf Fortress and Conway's Game of Life both demonstrate that compelling emergence comes from deterministic rules, not from authored intelligence. So the design question became: can you get the texture LLMs are genuinely good at — voice, personality, narration — without surrendering any authority over the world to them?

The Approach

Simulation first

The organizing principle is simple to state: the simulation is authoritative; the agents reason about it. The world advances on discrete ticks, and every tick executes the same ordered sequence of systems, deterministically. There are no LLM calls anywhere inside the tick loop — that's a hard constraint in the codebase, not a guideline.

Each tick resolves, in order: weather and ground cover (snow and wetness persist and affect movement); scent decay and diffusion (food and presence trails that drive routing); drive decay and mood homeostasis; hunger; perception; the decision phase, where each entity checks combat threats, then critical hunger, then evaluates task priorities against its aspirations; the action phase, where movement, eating, working, and socializing actually execute; behavior-trace sampling (more on that below); the animal ecosystem; visitors; combat resolution; deaths; resource updates — farm production, crafting job queues, food spawning; visitor spawning driven by procedurally generated race relations; and mood-change detection. Pathfinding is A* weighted by scent gradients, momentum, and social pull. Nothing in that list consults a language model. Run the same world with the LLM switched off entirely and the dwarves still forage, fight, craft, and die — they just do it silently.

The social-cognition layer

Where the LLM does enter is a thin layer on top: cognition and voice. Every dwarf carries a schema the simulation maintains — ten personality traits on a 0–1 range, six drives, eight skills, a mood value, relationship history, and a behavior trace: a rolling sample of what the entity actually did over recent ticks, recorded by the tick loop itself. When a social trigger fires — two dwarves meet, a hunger threshold trips, a mood shifts sharply — that schema gets compiled into prompt context, layered with permanent world lore (race relations, geography, history) and a chronicle of recent events.

This is the piece that makes the voices feel grounded rather than generated. The model isn't asked to imagine what a dwarf might say; it's handed what this specific dwarf just did, how it feels about the dwarf it's talking to, and what's happening around them, and asked to put words to it. The simulation supplies the facts; the LLM supplies the phrasing.

Ambient dialogue: nobody prompted this exchange. A social interaction resolved in the tick loop, and the LLM narrated it after the fact — from each dwarf's traits, mood, and relationship history.

Three conversational surfaces

The player can talk to the world in three distinct ways, and each surface gets a deliberately different contract:

• Chat with any character — click a dwarf and talk to it. Responses are roleplay, shaped by personality, mood, relationships, and the behavior trace, capped at six turns with cooldowns so conversation stays a texture of the world rather than the game itself.
• Chat with the game engine — an "Analyst" persona that answers read-only questions about world state, history, and how the simulation works. It has no game commands. You can ask it anything; you can't make it do anything. Same inversion, applied to the player's own tooling.
• Ambient agent-to-agent dialogue — unprompted speech bubbles when dwarves interact, so the world talks whether or not you're listening.

Agents don't just answer — they deflect, in character, based on what the simulation says they were actually doing. Finnley's excuse is grounded in his real behavior trace, not invented by the model.

The Analyst persona: read-only access to world state and mechanics, no game commands. The player gets an explainer, not a cheat console.

Mobile-first for a keyboard-and-desktop genre

Dwarf Fortress's genre has always assumed a keyboard, a large monitor, and a tolerance for arcana. I wanted this to run on a phone in a browser tab, so the interaction model is touch-first: tap an entity to inspect or chat, on-screen play/pause/speed controls instead of keybindings, a camera that follows the action, and speech bubbles and panels sized for a small viewport. The CSS Grid ASCII renderer helps here — it reflows like the web content it is, rather than emulating a fixed terminal.

The Build

No frameworks, no game engine — plain JavaScript ES Modules bundled with Vite, deliberately kept at low abstraction. The architecture separates into: the simulation core (world state, entities, tasks, combat, crafting, hunting, fishing, weather — orchestrated by the tick loop in world.js); movement (A* pathfinding with scent-weighted routing); procedural generation (Simplex-noise biome maps, cellular-automata caves, and a mixed mode, with 27 preset biomes and per-session color palettes); rendering (a CSS Grid ASCII renderer with dirty-checked cell updates and an 8-bit sprite layer); an event bus that keeps systems loosely coupled; and the AI/cognition layer, which is the only part that ever talks to a model.

The LLM integrates at seven specific points — thoughts, speech, name and bio generation, biome naming, the Analyst, event narration, and scenario generation — every one of them async, outside the tick loop, capped by a request queue, and backed by a scripted fallback so the world degrades gracefully to a pure simulation if the model is slow or absent. The flow for giving an agent a voice looks like this:

// Simplified: how a dwarf gets a voice
// 1. Tick loop samples a behavior trace — what the dwarf actually did
// 2. A trigger fires (meeting, hunger threshold, mood shift)
// 3. Context assembled: traits + drives + relationships + behavior trace
//    + L0 lore (world history) + L1 chronicle (recent events)
// 4. Async LLM call — never inside tick(); the sim doesn't wait
// 5. Response renders as thought or speech; on failure, scripted fallback
//
// The LLM never mutates world state. It only puts words to it.

Inference is self-hosted: Ollama running on a dual-3090 home lab, speaking the OpenAI-compatible chat completions API, with a cloud fallback if the local server is unreachable. Local inference is the economically honest version of this architecture. Because conversation is the only thing the model does, every chat, thought, and speech bubble has zero marginal cost; nothing a player types leaves hardware I control; and because the LLM is never on the critical path, the latency of a mid-size local model is fine — a speech bubble arriving two seconds after a meeting reads as natural, where a world-state update arriving two seconds late would read as broken. The architecture is what makes self-hosting viable, and self-hosting is what makes the architecture free to run.

The Embersteppe — Simplex-noise terrain with a per-session color palette, one of 27 preset biomes across biome, cave, and mixed generation modes. The LLM's only contribution is naming the result; the terrain itself is pure math.

Outcomes

• Shipped and publicly playable — the simulation runs in a browser tab, on desktop or phone, backed by inference on my own hardware. play it live →
• Open source with contribution guidelines that encode the thesis — the repo asks contributors to respect the simulation-first philosophy and to "avoid adding features that turn LLMs into authority figures." The architecture argument is written into how the project accepts changes.
• Zero marginal cost per conversation — every agent thought, dialogue, and Analyst answer runs on local inference. There is no per-token bill deciding how alive the world gets to be.
• A foundation, not a finished game — and I want to be honest about which is which. What's proven: the deterministic loop scales to dozens of concurrent agents with the LLM fully off the critical path, and the world runs with the model absent entirely. What's directional: because agent behavior is cheap and world state is deterministic, the same architecture should extend to persistent multi-session worlds and much larger populations without a change in kind — that's the next thing to test, not a claim.

Reflection