Self-Evolving Embodied Agents via Skill-Harness Evolution

The Core Insight

Embodied agents built on foundation models (like large language models) can improve without retraining the underlying model. Instead of updating weights through fine-tuning or reinforcement learning, the authors show you can evolve the surrounding system—the skills the agent can call, the context it receives, and the code that executes actions—by rolling out the frozen model in the target environment. This "skill-and-harness evolution" lets a single model adapt to new tasks and environments without touching its parameters.

Why This Matters

Most approaches to embodied agent adaptation require either expensive retraining (supervised fine-tuning, RL) or access to programmable robot APIs that encode domain knowledge. Neither option is always available: training is slow and data-hungry; custom APIs don't exist in fixed-interface settings where you can only send and receive text or predefined action tokens.

SHAPER sidesteps both constraints. The frozen model acts as both planner and optimizer—it can generate new skills, propose refinements to the context and code harness, and evaluate outcomes from environment rollouts, all without parameter updates. This keeps computational cost low and doesn't require task-specific reward signals.

How It Works

SHAPER evolves two things:

Reusable skills: Collections of actions or sub-tasks the agent can compose. Rather than hard-coding these upfront, SHAPER iteratively builds and refines them based on what works in the target environment.

Context-code harness: The wrapper around the model that controls how it receives input (context) and translates its output into executable actions (code). A better harness can bridge the gap between the model's native outputs and the environment's fixed action interface.

Both evolve through target-environment rollouts—the agent attempts tasks, observes outcomes, and the frozen model proposes improvements to its own tooling. The mechanism for proposing and selecting improvements isn't detailed in the abstract, but the framework keeps the model weights constant throughout.

The Evidence

The authors test SHAPER on VLABench and ESI-Bench, which include embodied agents with different low-level action interfaces—varying from abstract action spaces to more constrained, fixed options. They compare against:

• Pure execution: baseline agent with no adaptation
• Supervised fine-tuning: retraining the model on task data
• Test-time-scaling baselines: verifier-free selection and voting, which generate multiple outputs and pick the best without retraining

Their results suggest that skill-and-harness optimization outperforms these baselines, especially when model training is expensive, unavailable, or undesirable (e.g., proprietary models).

The Tradeoff

SHAPER trades retraining cost for optimization-in-the-loop—it requires rolling out the agent many times in the target environment to refine skills and harness. Whether that's cheaper than fine-tuning depends on environment interaction cost versus compute cost, a practical consideration not fully addressed in the abstract.

The approach is most valuable when the foundation model is strong enough to serve as its own optimizer and when environmental rollouts are affordable.