Graff's RLM runtime: how 135k input tokens became 25.3k

A model is the programmer. The harness is the desk around it.

A wasteful desk makes the programmer reread the same file, carry every tool manual into every task, wait for one job to finish before starting the next, and paste bulky tool results back into every future conversation. A better desk keeps useful working state, starts safe work early, and throws away context that no longer earns its cost.

That is the point of Graff's RLM runtime. The headline result was a stable bind-reuse run on Grok 4.6: 135k to 25.3k input tokens, 99.6 to 39.2 seconds, and 11 to 5 model calls. The result is real for that one run. It is not an 81% promise for every coding task.

The announcement thread (opens in a new tab) bundled two consecutive releases into one story. The distinction matters.

v0.0.276 (opens in a new tab) shipped on August 25. It made the RLM loop the default and published the bind-reuse, five-task RLM suite, prompt-cache, and grok-build numbers. v0.0.277 (opens in a new tab) followed on August 26 with MCP-inside-RLM, learned result slimming, late RLM showcase, and hosted X search.

So the 135k to 25.3k result belongs to the v0.0.276 stable-reuse case. v0.0.277 made the same prompt-economics argument apply to connected tools and X search.

Recursive Language Models treat a long prompt as data in an external environment. The model can inspect slices, write code over them, and recursively ask another model about focused fragments. The original RLM paper (opens in a new tab) established the pattern. Graff implements a Zig-native subset around its existing file, shell, code-intelligence, and subagent tools.

Speculative programmatic tool calling adds timing. Once a streamed program contains a closed statement with sufficiently stable inputs, Graff can launch the host call as a future. When the authoritative program reaches the same call, it claims that result or falls back to normal execution. The upstream SpecPTC write-up (opens in a new tab) explains the scheduling idea in detail.

Persistent binds are the less glamorous and more dependable win. A turn can assigns = read_file(path), then use s later without asking the model to rediscover the file and repeat the request path.

Prompt caching rewards repetition. Change the system prompt or reorder the tool catalog and the reusable prefix can disappear. Graff keeps the catalog head byte-stable and puts loaded schemas in the tool result instead of rewriting the definitions on the next call. xAI's cache guidance (opens in a new tab) supports the stable-prefix and routing approach, although no harness can guarantee a hit.

The Twitter thread reported a 99.8% cached-token ratio in one live Grok 4.6 run. We did not publish the raw response log for that exact denominator, so treat it as an observed run, not a standing guarantee.

v0.0.277 also stopped advertising RLM on every tiny task. It appears when forced, when a batch reaches four native tools, when context reaches half the model's compaction threshold, or when explicitly loaded. The schema costs roughly 200 input tokens before the model writes a program, so small turns should not pay for it.

Connected tools often return much more JSON than a task needs. If the harness pastes that object into history, every later call can repay for it. v0.0.277 lets loaded MCP names run as RLM host functions, then infers a value-free return shape made of keys and broad types. That shape is stored in .graff/mcp-shapes.json and attached to a later schema load, never the always-on prompt head.

The implementation can slim issue-shaped arrays to identity fields and comment arrays to a count plus latest author. This is deliberately mechanical. An earlier explicit prompt recipe told Grok how to iterate the data and blew out to 220 seconds, 462k input tokens, and 29 calls. We rejected that path and kept the transformation in Zig.

xAI supports server-side X search (opens in a new tab) on its Responses API. Graff now adds that hosted object to eligible xAI tool turns. It is not a local catalog function, does not sit in the stable prefix, and is already complete when Grok returns the tool-call item.

On one identical prompt asking what people were saying about xAI, the Graff-run scraping path reported 101 seconds, 7 calls, and 64k input tokens. Hosted search reported 37 seconds, 1 call, and 28k input tokens. The capability is independently documented by xAI. The performance delta is our one-prompt measurement.

The thread also compared Graff with grok-build on six DeepSWE-shaped tasks at parallelism six. Both reported 5/6, and both failed label-sort. Graff reported 198s, 106k input, 24 calls, and 8.5 MB peak RSS. grok-build reported 506s, 194k input, 39 calls, and 177 MB peak RSS.

That internal run favored Graff on four deployment signals while preserving the same reported pass rate. It is not a normalized inference leaderboard. Harness prompts, tool policies, cache behavior, and process architecture differ by design.

We wrote “GDM-style” in the announcement thread. The correct acronym is DGM, for Darwin Gödel Machine (opens in a new tab). The project keeps an archive of candidate agent strategies, evaluates variants, and can promote one that passes the scoring contract.

That does not retrain Grok 4.6 or modify model weights. MCP learned slim is narrower still: it stores structural information about a tool result so the next call can carry less data. “Self-improving” only means something when a change beats a frozen baseline on protected checks and survives a rerun.