Large language models usually spend test-time compute by generating more tokens: chain-of-thought, self-consistency samples, search traces, or tool calls. Tiny Recursive Models (TRMs) explore a different route: spend compute by repeatedly refining a latent answer state inside a small neural network. The model is tiny, but it gets multiple opportunities to revise its internal solution.
The paper “Less is More: Recursive Reasoning with Tiny Networks” simplifies the earlier Hierarchical Reasoning Model idea. Instead of two recurrent networks operating at different frequencies, TRM uses a single small two-layer network and recursive updates. With roughly 7M parameters, the paper reports 45% test accuracy on ARC-AGI-1 and 8% on ARC-AGI-2, along with improvements on Sudoku-Extreme and Maze-Hard.
The training-hub lesson is not that TRMs replace general LLMs. They target structured reasoning tasks where inputs and outputs can be represented as discrete grids or sequences and where a compact model can iteratively repair its answer. For AI engineers, TRMs are an important design point: sometimes recurrence and supervision over iterative refinement can beat brute-force scale.
TRM maintains a latent state z and a current answer representation y. Given the embedded input x, the model repeatedly updates z, then uses the updated latent state to revise y. The same small network is reused across steps, so additional reasoning depth comes from recurrence rather than more parameters.
Recursive systems need training signal at intermediate steps. Deep supervision reuses the previous latent features as initialization for the next forward pass and applies losses across improvement steps. This teaches the model to make useful partial progress rather than relying on one final jump from input to answer.
A small recursive model trades width and parameter count for repeated computation. This is attractive when the task has a compact state space and a natural notion of refinement, such as correcting a puzzle grid, propagating constraints, or improving a candidate answer under fixed rules.
TRM spends test-time compute without emitting extra reasoning tokens. The model updates an internal state several times, then returns the final answer. This makes reasoning depth a runtime knob while keeping the external output compact.
The paper describes TRM as starting from an embedded input, an initial embedded answer, and a latent state. For each supervision step, the model performs several latent recursion steps, then updates the answer. A simplified view looks like this:
def trm_forward(x, y0, z0, supervision_steps=16, latent_steps=4):
y = y0
z = z0
predictions = []
for _ in range(supervision_steps):
for _ in range(latent_steps):
z = tiny_network_z(x=x, y=y, z=z)
y = tiny_network_y(x=x, y=y, z=z)
predictions.append(output_head(y))
return predictions[-1], predictionsThe important property is weight sharing. The recursive updates use the same small learned transformation repeatedly. That encourages the network to learn a general “improve the current state” operation rather than memorizing one fixed-depth computation.
During training, intermediate predictions can be supervised so the model receives feedback on each refinement step. This reduces the risk that early recurrence steps drift into useless latent states and gives the optimizer a shorter path from error to corrective update.
def trm_loss(predictions, target):
losses = []
for step, pred in enumerate(predictions):
weight = 1.0 if step == len(predictions) - 1 else 0.25
losses.append(weight * cross_entropy(pred, target))
return sum(losses)Puzzles such as Sudoku, mazes, and ARC-style grids have local constraints that interact globally. A single feed-forward pass may miss a constraint propagation step. Recursion lets the model revise a tentative solution after each latent update, similar to running a learned solver for multiple iterations.
TRMs are naturally suited to tasks where the input and output are compact grids or token arrays. ARC-AGI, Sudoku, and maze solving all require repeated constraint propagation, but do not require broad natural-language knowledge. A small recurrent solver can specialize in the transformation structure.
Many engineering problems involve iteratively improving a state under constraints: scheduling, routing, layout repair, symbolic grid transformations, and configuration validation. TRM-style architectures suggest a way to learn the update operator rather than hand-writing every rule.
A 7M-parameter model is small enough for environments where large LLMs are impractical. If the task is narrow and structured, recurrence may provide useful reasoning depth without requiring a cloud-hosted frontier model.
TRM is not a good fit when the answer is an unconstrained paragraph. It works best when the answer can be embedded, updated, and decoded repeatedly. Design the output representation so partial answers are meaningful and can improve step by step.
More recursion steps cost more compute but may improve accuracy. Keep supervision steps and latent update steps configurable so you can plot accuracy versus latency and choose a deployment point deliberately.
Log intermediate predictions. A healthy recursive model should often improve or stabilize over steps. If accuracy oscillates or degrades late in the loop, the model may need better halting, different loss weighting, or a smaller iteration budget.