- Source
- arXiv
- Published
- Runtime
- 0:00
- Snippets
- 5
A conversation between
OpenForgeRL: Train Harness-native Agents in Any Environment
§02
Snippets
-
OpenForgeRL decouples training from inference via a lightweight proxy that records harness model calls as training data for standard RL codebases, enabling end-to-end agent training in any harness.
Researchers can now train agents directly in real deployment harnesses without building custom infrastructure, lowering the barrier to agent research.
-
Different harnesses (ZeroClaw, OpenClaw, Codex) show substantially different learnability; agent behavior and RL improvements vary significantly by harness choice.
Harness selection is not transparent—understanding which harnesses enable better learning could guide deployment and model development.
-
RL improves agentic reliability in self-verification, tool coverage, and multi-step planning, but error recovery remains a critical weak point.
Current RL methods don't adequately address robustness—agents still fail ungracefully, limiting real-world deployment.
-
OpenForgeGUI agents reach 37.7–72.3 on diverse GUI benchmarks using only hundreds to a few thousand tasks, matching or exceeding larger models.
Data efficiency breakthrough suggests open-source agents can be practical without proportionally larger training datasets.
-
A Kubernetes orchestrator runs each RL rollout in its own remote container, enabling distributed training across diverse harnesses and environments at scale.
Containerized orchestration makes large-scale multi-harness RL reproducible and accessible, not just a capability of well-resourced labs.
§03
Synthesis
The Problem: Training Agents Inside Real Harnesses
Modern AI agents don't run in isolation. They rely on complex inference harnesses—think Claude's code interpreter, OpenAI's Codex, or specialized tool-calling frameworks—that manage multi-turn reasoning, tool execution, and external system access. The catch: these harnesses are stateful, multi-process systems that existing open-source reinforcement learning (RL) stacks can't natively handle. Researchers wanting to improve agents must either work with closed APIs or abandon the real harness they'll deploy in. OpenForgeRL solves this by decoupling training from the harness itself, letting teams train agents end-to-end in the exact environments where they'll be used.
How It Works
OpenForgeRL uses two clever pieces of infrastructure. First, a lightweight proxy intercepts the harness's model API calls, records them as training data, and passes them to a standard open RL codebase like veRL for optimization. This proxy acts as a translator—it speaks both the harness's language and the training framework's language. Second, a Kubernetes orchestrator spins up a fresh remote container for each rollout (trial run), ensuring stateful harness inference doesn't block parallel training at scale.
By decoupling this way, researchers train on whatever harness they want—ZeroClaw, OpenClaw, Codex—without rewriting their training pipeline. The harness runs as-is in production; the RL learner sees clean training signals.
Results and Insights
The authors test across two agent types: tool-use and GUI-based. On the tool/claw track, OpenForgeClaw reaches 31.7% pass@3 on ClawEval and 55.9% pass@3 on another benchmark, using only hundreds to a few thousand training tasks. For GUI agents (which navigate websites and desktop environments), OpenForgeGUI achieves 37.7% on OSWorld-Verified, 63.0 on Online-Mind2Web, and 72.3 on WebVoyager. Across nearly all benchmarks, these numbers match or exceed open baselines of comparable model size and sometimes rival proprietary models several times larger.
Beyond raw scores, the paper reveals how harness choice matters. Different harnesses—OpenClaw vs. ZeroClaw vs. Codex—have markedly different learnability; some are substantially harder to optimize than others. RL training noticeably improves reliability: agents get better at self-verification, expanding which tools they use, and completing multi-step plans. However, critical gaps remain—error recovery and handling unexpected failures are still weak points.
Why It Matters
OpenForgeRL removes a major friction point in agent research. Previously, improving agents meant either accepting closed-source constraints or rebuilding everything from scratch. Now, researchers can iterate on agents in their actual deployment harness using standard RL tools. That's not just convenience—it's scientifically cleaner, because you're optimizing for the real system you'll use, not a simplified proxy. The framework is open-source and designed for scale, making harness-native agent training accessible to teams without massive compute budgets.
Mine your own.
Lode is a workbench, not a feed. Paste a YouTube URL. The model proposes a transcript, a set of quote-grounded snippets, a synthesis essay, and the fan-out. You decide what stays.