So here's a big research bet that all the labs are making. They think that if we train AIs to accomplish millions of verifiable tasks across thousands of diverse RL environments, then we will have basically built AGI, because this kind of training will have created a kind of problem-solving agent: the kind of thing that can make progress on open-ended tasks for weeks on end in the face of errors and mistakes and ambiguity.

This frames the central thesis of the entire essay — a single empirical bet that defines the current trajectory of frontier AI development.

The people who are optimistic about this vision will say that all these things we talk about as the fundamental deficits in the current training paradigm — for example, the data inefficiency of these models, or the fact that they lack continual learning — can just be steamrolled if we scale training more, in the same way that all the fundamental research problems in natural language processing collapsed when we threw enough compute into LLMs.

This articulates the 'scaling optimist' position and the historical analogy that underlies it, which is the crux of the debate about whether current methods can reach AGI.

Training is this one-time cost that is amortized across billions of sessions that a model will experience. What really matters is how smart and general and sample-efficient the model is during a session, and this has clearly been improving as we've been doing more RL training.

This reframes sample inefficiency during training as economically irrelevant if in-context efficiency is strong enough — a non-obvious but consequential argument.

Because if in-context learning gets so good across longer and longer time horizons, then you don't need to distill everything the model is learning on the job back into the weights. People often say that their employees are not net productive until six months or more on the job. So clearly, online learning is necessary for competence. But what if you could just fit those six months into the context window?

This is a vivid reframing of the continual-learning debate — asking whether infinite context could substitute for weight updates — with a concrete human analogy.

Why has progress on computer use been so much slower than other domains? Computer use is so clearly verifiable. You could ask a question like: did the desired Etsy item I ordered get delivered? Is the venue for an event I'm trying to organize booked? Have my taxes been submitted? So isn't it weird that computer use has been making so much slower progress than coding and math and these other verifiable domains?

This sets up a revealing empirical puzzle that challenges the naive equation of 'verifiability' with 'trainability.'

It is not enough for a domain to be verifiable. It also has to be very grindable, in the sense that you have to be able to run lots of parallel rollouts against a deterministic and replayable simulator, and you have to run those rollouts from the same starting point. If you're trying to make a model better at coding, you can define some container that has a software repo with some missing feature that you have tasked the AIs with creating. And then you have a thousand parallel agents go at the problem, each of which has an identical copy of the container. But this doesn't work with computer use, at least not trivially.

The concept of 'grindability' is a crisp, novel criterion that explains why many real-world domains resist RL-based training even when outcomes are objectively measurable.

For so many other different kinds of skills that we need AIs to have, we simply can't do this. How do we train an AI to get really good at building a business from scratch? How about winning court cases, or having a profitable day of trading in the markets, or helping a candidate win an election? The rollout here requires interacting with the real world, and you can't recreate it from just within a datacenter. The outer-loop verification here may take months or even years of real-world actions to elicit.

This concretely maps the boundary of what current RL training can reach, highlighting a large class of high-value human skills that may be structurally out of reach.

The labs are betting that RLVR will generalize. That is, that if you train on enough containerized, reproducible environments, you will develop a very general agent that can make and execute plans and learn rapidly from new information, and even pick up new skills, all within a single session. If you drop this endlessly RLVR'd AI into Texas politics in 1948, it could give you better advice than LBJ about winning the Senate seat. And if you gave it a hundred million dollars in 2002 and let it cook, it would build SpaceX for you.

These vivid hypotheticals sharply test the limits of the generalization claim — and make the stakes of whether RLVR generalizes feel concrete and enormous.

Dario gave a telling quote during our podcast together, which I think hints that RLVR generalization is not infinitely strong. When he was explaining why model performance tends to degrade at long context, he said: 'There's two things. There's the context length you train at, and there's a context length that you serve at. If you train at a small context length and then try to serve at a long context length, maybe you get these degradations.' Now, maybe I'm reading too much into this, but it seems like he's saying that short-horizon RL training doesn't necessarily generalize to long-horizon RL performance.

Using a candid quote from Anthropic's CEO to infer a structural limitation in the current paradigm is a strong analytical move that grounds the speculation in real evidence.

Around 30 to 50 percent of a lab's compute goes to inference, and that compute is currently not playing any productive role in helping improve the model. This seems like a huge waste. And it's even worse than it sounds, because it is only in deployment that the most valuable bits of information which your model could learn from are actually revealed. What's actually happening in the organizations where I'm being used? What are they using me for? And what kinds of mistakes do I tend to make in the real world?

Framing inference compute as wasted training signal redefines the efficiency problem and motivates the entire case for continual learning.

AIs can't just keep building up a bigger and bigger KV cache as they learn from more and more users. That's just not scalable, and that's also not how humans do it. There's no clean separation in our brain between parameters and activations, and it's not like some part of your skull keeps expanding as you learn more things throughout your lifetime. When we learn stuff, there's clearly some kind of compression, and this aids our generalization and grokking.

The neuroscience analogy reveals a deep architectural mismatch between current AI systems and human memory, pointing toward a necessary design shift.

There are, in fact, some humans who have this autistic-savant-type ability to recall random tables of numbers or nonsense syllables years later — basically the kind of fidelity of information that models have in context. And such sheer volume cripples these humans' ability to understand abstractions and metaphors. Human continual learning is less about having all your observations at the tip of your tongue and more about chiseling the right intuitions and big-picture knowledge back into the weights.

This analogy elegantly explains why perfect recall is not the goal of learning, and why weight compression may be more valuable than context accumulation.

Gradient updates are super sample-inefficient, all of the successfully shipped online-learning models have had to learn the exact same thing across millions of users. For example, the Cursor Tab model online-learns by predicting the same exact objective for over 400 million requests a day. The objective here is which edits actually got accepted by the user. At least so far, we haven't seen models online-learn different kinds of things for different users, because while a single session may generate more than enough data for a human to learn from, it's not enough to train a more capable AI.

The Cursor Tab example grounds the abstract continual-learning problem in a real deployed product, clarifying exactly where current approaches succeed and fail.

Sample efficiency and continual learning are actually deeply connected problems. Relatively little data is available to the model on the job. Now, to learn from this data requires sample efficiency, and models can do that in context, but using the fast weights that are built on the fly by attention, which allow for this sample efficiency, scales very poorly in terms of memory. So we need architectural innovations that allow for some kind of intermediate representation.

This synthesis links two seemingly separate research threads — sample efficiency and continual learning — into a single architectural challenge.

A lot of people are talking about this technique called on-policy self-distillation recently. The idea is that we encourage the base model to make the same predictions when trying to solve some real-world problem as the model with all the context accumulated after a long session would have made. The whole point of this procedure is to distill what the model learned in a session back into the weights themselves. This is better than RLVR for two reasons. One, OPSD doesn't require us to have some outer-loop verifiable reward. We just need a model that can learn the right things within the context window. And two, OPSD provides a much denser supervision signal than naive RL.

On-policy self-distillation is presented as a concrete technical solution to the continual learning problem that avoids the verifiability and sparse-reward bottlenecks.

The way you get better at your job is not by recalling the transcript of every single thing that happened every day with perfect fidelity. Rather, it's by consolidating the handful of insights and pieces of knowledge that are actually relevant to you getting better at your job. RL training doesn't suffer from this failure mode. RL is great at concentrating the update to only what is relevant to getting the outcome right. That's why the updates from RL are incredibly sparse. And this is a very important property for continual learning, because as you're learning on the job, you don't want to overwrite and forget all the other things that the base model knows.

This reframes RL's notorious data inefficiency as a feature rather than a bug for continual learning, a perspective-shift with significant practical implications.

Let's call it dreaming. If the AI can build a good simulation of reality against which to rehearse new skills, or try alternative strategies and reinforce what actually works, then AIs could experience orders of magnitude more simulated samples in the same wall-clock time. A couple years after DeepMind released AlphaZero, a group of researchers trained a model called EfficientZero, and the whole point of this model is to be very efficient with data. So if this model and a human both got two hours to play against a simulator of an Atari game that they hadn't seen before, this model would actually probably beat the novice human.

The 'dreaming' concept — generating synthetic training environments from real-world context — is the most speculative but potentially transformative idea in the essay.

If it works, it would become a fourth axis of scaling alongside pretraining, RL, and inference-time compute. You could call it test-time training or dreaming. The model spends compute writing up RL environments and then training against them, and it's rehearsing all the skills that will actually be used in production for a specific user. So instead of hitting /compact in Codex or Cursor or Claude, which kindles a small amount of compute to write up a summary, and which gives you the simulacrum of continual learning, you hit /dream. And this incinerates huge amounts of compute to build and train against a video-game version of what the model is witnessing in the real world.

Proposing 'dreaming' as a fourth scaling axis is a bold and falsifiable prediction about future AI development that reframes how we think about compute allocation.

All of this RLVR training is producing an agent that can get its bearings when it's thrown at an unfamiliar problem, and it can try different strategies, and it can iterate when it hits a roadblock. This is the crucial thing that RLVR has given you: an AI that is at least competent enough to start getting some real-world experience, if it could learn from it. And once you have that, you send it out into the world to do real work, even on projects that are off the training distribution.

This articulates a bootstrap theory of AI development — RLVR is not the endpoint but the ramp that enables real-world learning to begin.

Just as pretraining created a base intelligence that was smart enough to become a competent agent with enough RLVR on top, so RLVR has created an agent that is competent enough to actually be broadly deployed in the world, and from this broad deployment to learn on the job once the training recipe for continual learning actually arrives. By this point, the main way that AIs get better is not from the training they have received before they are released to the public. Rather, it's from all this experience that they'll be accumulating from being broadly deployed in the economy and engaging in so many different kinds of tasks.

This closing vision — where deployment becomes the primary driver of AI improvement rather than pre-release training — would represent a fundamental shift in how AI progress works.