BDH-CQ: In-Context Learning with Recurrent Latent Reasoning

The Core Claim

A new reasoning model called BDH-CQ achieves state-of-the-art cost efficiency on the ARC-AGI benchmark—a notoriously difficult test of general problem-solving. With just 150 million parameters, it reaches 29.5% pass@2 (meaning it solves nearly 30% of tasks within two attempts) while costing only $0.0007 per task. This breaks the previous cost-accuracy tradeoff, outperforming larger models on the efficiency frontier.

How It Works

The model combines two key ideas. First, it uses in-context learning: when given examples of a problem type at test time, the model learns from them on the fly rather than relying solely on knowledge baked into its weights during training. Second, it performs recurrent latent reasoning—the model maintains an internal memory that updates as it processes input examples, then solves the actual problem through iterative computation in a hidden, high-dimensional space.

Critically, this reasoning happens silently. Unlike chain-of-thought approaches where language models spell out their steps ("First, I notice the pattern…"), BDH-CQ reasons entirely in latent space without verbalizing intermediate steps. This likely makes computation more efficient and allows the model to explore complex transformations without the bottleneck of generating natural language.

The recurrent memory mechanism is central: as demonstrations are fed in, they continuously reshape the model's internal state. By the time the query arrives, the model has already built a contextual representation tuned to the problem at hand. It then iterates—refining its solution step-by-step in the latent space—until it produces an answer.

Why This Matters

ARC-AGI is a benchmark specifically designed to require abstract reasoning and generalization rather than pattern-matching on common tasks. It remains extremely hard; most language models perform poorly. The significance here is twofold.

Efficiency gains are real. The authors found a sweet spot: a model small enough to run cheaply but sophisticated enough to handle the benchmark's complexity. At $0.0007 per task, this is far cheaper than deploying larger models while maintaining competitive accuracy. For practical applications, this changes the calculus of when reasoning is economically viable.

The method is interpretable at a high level. By evaluating on controlled variants of ARC tasks, the authors tested what the model actually learns from in-context examples—does it genuinely infer transformations, or just memorize? They also identified which conceptual classes remain difficult for the model. This kind of analysis matters for understanding where the approach breaks down and suggests paths for future improvement.

The recurrent latent reasoning design is a departure from standard architectures. Rather than treating reasoning as a language generation problem (which scales poorly), the authors treat it as iterative hidden computation. This is closer to how humans might solve abstract puzzles internally before announcing an answer, and it seems to unlock better cost-efficiency in this domain.

In short: BDH-CQ shows that reasoning-heavy tasks don't necessarily demand billion-parameter models. Careful architecture design—combining in-context learning with silent, iterative computation in latent space—can achieve strong results at a fraction of the cost.