- Source
- arXiv
- Published
- Runtime
- 0:00
- Snippets
- 5
A conversation between
DCAS: Decoupling CLI Agent Scaffolding to Internalize Planning across Scaffolds
§02
Snippets
-
Models fine-tuned on OpenHands trajectory data score well under OpenHands but degrade substantially under non-training scaffolds, while untrained base models show no such divergence.
This reveals that current training practices embed hidden scaffold assumptions into the model, limiting real-world deployment flexibility.
-
Planning structure—both explicit (pre-execution plans) and implicit (structural conventions in the agent loop)—is the load-bearing scaffold-specific behavior driving the performance gap.
Identifying planning as separable and learnable opens a path to scaffold-agnostic training rather than environment-locked models.
-
DCAS is a backend-substitution interception layer that routes API traffic between any CLI scaffold and any backend model, enabling cross-scaffold evaluation without modifying scaffolds.
This infrastructure lets researchers isolate planning from scaffold quirks and test generalization directly, accelerating progress on portable agents.
-
A controlled intervention that improves planning source produces gains exceeding the cross-scaffold performance degradation observed.
This quantifies planning as a high-leverage lever, justifying focused effort on decoupling it from scaffold conventions.
-
A model fine-tuned on a small set of DCAS-collected planning-aware trajectories from a single scaffold gains consistently across non-training scaffolds.
This demonstrates that planning-aware training transfers, suggesting a practical path to building agents that work across diverse deployment environments.
§03
Synthesis
The Problem: Models Forget How to Plan When You Change Their Tools
Current CLI-based software engineering agents—systems that write code and run terminal commands autonomously—work great in the lab but fail in production. Specifically, models fine-tuned on OpenHands (the dominant training environment) score well under OpenHands but degrade sharply when deployed under different scaffolds (frameworks that structure how the agent interacts with the environment). Untrained base models don't show this problem, which means the issue is learned, not fundamental. The authors' insight: models are learning to rely on scaffold-specific planning conventions rather than developing generalizable planning skills.
What Planning Actually Means Here
The paper distinguishes two types of planning baked into scaffolds:
Explicit planning is a structured document—an actual plan artifact the agent produces before acting. Implicit planning is subtler: the conventions embedded in how the scaffold structures the agent's loop (what information appears when, how errors are presented, what actions are available at each step).
Both leak information about "how to think" that models absorb during training. When you switch scaffolds, both vanish, and the model has no internal planning capability to fall back on.
The Method: DCAS
DCAS is a technical intermediary layer that sits between any CLI scaffold and any backend model. It intercepts API traffic without modifying either the scaffold or the model, essentially letting researchers swap scaffolds on the fly. This solves two problems:
- Cross-scaffold evaluation becomes possible: Test a model trained on Scaffold A under Scaffold B without rewiring code.
- Planning-aware data collection becomes possible: Deliberately collect training trajectories that teach the model to generate explicit plans, even under a scaffold that doesn't naturally produce them.
Think of it as a translator that lets incompatible systems talk while logging everything.
The Evidence
The authors ran a controlled experiment: they intervened on planning quality in trajectories and measured the impact. The gains from teaching models to plan internally exceeded the performance drops they observed when switching scaffolds. This confirms planning quality is genuinely high-leverage.
They then fine-tuned a model on a small set of planning-aware trajectories (collected via DCAS) from a single scaffold. The result: consistent gains across non-training scaffolds. The model had learned to plan, not to depend on scaffold quirks.
Critically, they showed the two planning senses are empirically separable in training data—you can tease apart which parts of the performance gap come from explicit versus implicit planning.
Why It Matters
The CLI agent ecosystem is consolidating on OpenHands, creating a generalization blind spot. Models that work there may fail elsewhere. DCAS breaks this cycle by making it cheap to train on planning-aware data and test across scaffolds. For practitioners, it means better production robustness. For researchers, it's a tool to isolate what agents actually learn versus what they memorize from their training environment. The result is a path to agents that plan, not just imitate.
Mine your own.
Lode is a workbench, not a feed. Paste a YouTube URL. The model proposes a transcript, a set of quote-grounded snippets, a synthesis essay, and the fan-out. You decide what stays.