Determinism Injection

I’ve been refining a specification-driven development model for LLM software engineering for over a year now. The RDR model started with a simple template, but has evolved into a full human-driven engineering workflow. There are a number of stages in the flow, all derived from process, management, and engineering literature, as well as 30 years of experience writing software and managing software engineering teams.

Every step has an artifact a human is either expected or can review and refine to ensure they are building the correct thing, while also ensuring the whole project maintains its conceptual integrity as it evolves.

The challenge is that even though the flow is well defined and orchestrated by a human, the flow elements are just large LLM prompts that internally make localized decisions and look up facts across related documents.

This is decidedly non-deterministic. Guo et al., in The Deterministic Horizon, identify a deterministic horizon as the step depth past which unaided model accuracy collapses. And name the mechanism state-space decoherence, divergence that accumulates from extended reasoning rather than being resolved by it. This decoherence is aggravated by a prompt performing tasks that can be externalized.

So, instead of writing an application that executes prompts in a deterministic way, I need to inject determinism into the prompts I execute in order to improve their ability to actually solve the problem they are tasked with and not lose accuracy or focus because they had to grep 100 files for a single fact.

I think we should call this what it is: Determinism Injection.

Determinism Injection is the inverse of code orchestrating small prompts, the approach behind frameworks like Alibaba’s Blueprint First, Model Second. The prompt is the orchestrator, and generic or purpose-built tools perform the lookups, state transitions, and writes.

This is a clear play on Dependency Injection, where capabilities are polymorphically supplied to adapt an application’s behavior to a given context. Determinism Injection is a precursor step that isolates a deterministic behavior and externalizes it, which allows contextual binding of tools that will satisfy the expectations.

Intrastate is a tool for resolving state transitions. I call it a stateless state-machine. It has no backing datastore, it is handed context for a given flow it’s intended to execute, it may look up additional facts, then determines the next state. Since an LLM is driving it, if there is any ambiguity or failure, it generally has enough context to recover. But the net effect is that we lower prompt size, token spend, and wall-clock duration. And gain both (more) deterministic results and higher reasoning quality.

Retrofit is another tool for Determinism Injection. Instead of relying on a prompt to generate SQL schemas and migration scripts, the LLM stops at intent. It hands its intent (RenameColumn, DecomposeTable, etc) to retrofit, and when asked, retrofit can render a SQL dump (with or without data), or migration scripts (or their reverse for rollback).

Arcaneum was my first such tool, started over a year and a half ago, to compensate for hallucinated assertions. It is a RAG search tool for both full-text and semantic indexes. Instead of filling my context by grepping all my documents, I use it daily for searching and retrieving literature, documentation, source-code, and LLM harness transcript histories. Every design decision I make is grounded in research. And every assertion has a valid citation.

The kata and roborev projects are also examples of this model. The first is an inline issue tracker for LLM agents, and the second is an asynchronous code reviewer. Work that is actually quite hard for an LLM to manage (remember beads?).