Agent Lightning v1.0: Towards Harnessed Agentic RL

The Core Innovation

Traditional reinforcement learning for language models treats agents as black boxes—the RL trainer controls everything, managing tool calls and environment interaction directly. Agent Lightning v1.0 flips this architecture: the agent harness (the deployed system managing tools, context, and control flow) stays in charge of the environment loop, while the RL trainer only observes sequences of LLM request-response pairs. This "harnessed agentic RL" paradigm is simpler to deploy—you train the model the way it actually runs—but introduces new technical challenges that previous approaches glossed over.

The authors' key insight is that these challenges matter. When the harness controls interaction, the trainer faces problems like retokenization (how to handle variable-length tool outputs), sample merging (combining trajectories from different harnesses), advantage calculation (estimating value improvements), loss normalization (stabilizing gradients across variable-length sequences), and backend scheduling (efficient batching). Agent Lightning v1.0 is a 3,500-line framework that tackles these concrete problems and provides a reproducible foundation for study.

Why It Works and Why It Matters

The framework is minimal but complete. It handles arbitrary harnesses—instruction-following systems, search engines, code development agents—without baking in assumptions about task structure. The authors test it on three distinct domains and release full reproducible code for coding agents, removing a major barrier to progress in this space.

The results demonstrate both tractability and impact. On SWE-bench Verified (a real coding benchmark), RL training on Qwen3.5-9B jumps from 41.8% to 56.4% success rate—a 14.6-point absolute gain—using only 6,000 training examples and standard compute. This isn't marginal improvement; it's the kind of gain that justifies the infrastructure.

Why does harnessed agentic RL matter more than it initially appears? In production, agents live in harnesses. The agent's behavior emerges not from the model alone but from the model-plus-harness system. Training that system end-to-end, rather than treating the harness as fixed scaffolding, captures real constraints: how the model must format tool calls, how it recovers from errors, how it reasons within actual latency budgets. This alignment between training and deployment reduces sim-to-real gaps.

The paradigm has already influenced practice—the abstract notes that frameworks like verl, Uni-Agent, AReaL 2.0, and others have adopted the disaggregated architecture—but without shared understanding of the technical pitfalls. Agent Lightning v1.0 centralizes that knowledge and provides a testbed. Future work can focus on the hard problems: better advantage estimators for long-horizon tool use, efficient sample merging across heterogeneous harnesses, or handling distribution shift when harness behavior changes.

The release of reproducible code is crucial. Agentic RL remains a young field where many results don't reproduce and infrastructure dominates the barrier to entry. By opening the full pipeline and training scripts, the authors lower that barrier and create a shared reference point for the community.