FACET: Preserving Source Intent and Executable State in Terminal Task Synthesis

The Problem with Building Training Tasks for Terminal Agents

Training agents to work in terminal environments (like bash shells) requires thousands of executable, solvable tasks. Each task is actually four linked pieces: an instruction telling the agent what to do, an initialized environment state where that task makes sense, a reference solution showing how to solve it, and a verifier that checks if the agent succeeded. If these pieces don't align—say, the instruction assumes a file exists but the environment doesn't create it, or the verifier checks for the wrong output—the task becomes unsolvable or teaches the agent nothing useful.

Current task synthesis pipelines generate these pieces separately or sequentially, which means each generation step can drift from the original intent. A reference solution written in isolation might rely on assumptions the environment doesn't satisfy. A verifier built later might misunderstand what the instruction actually requires. The authors call this the consistency problem: multi-stage generation orphans procedural constraints, state dependencies, and goals from the source material, leaving broken tasks in its wake.

How FACET Preserves Intent and Consistency

FACET (Fine-grained Agentic Construction of Executable Tasks) inverts the typical pipeline. Instead of generating instruction → environment → solution → verifier in isolation, the framework grounds everything in a shared, executable environment state.

The method works in three phases:

1. Scenario construction: FACET extracts related agent skills from source material (e.g., documentation, examples, or existing task collections) and stitches them into coherent, information-dense scenarios. This step reconstructs the procedural logic and dependencies that would otherwise evaporate.

2. Environment realization and repair: The framework actually builds and runs the initialized environment, not just describing it. Execution-based validation catches silent failures—missing files, misconfigured tools, broken dependencies—early. Targeted repair fixes environment-specific issues without regenerating everything from scratch.

3. Artifact generation with grounding: With a verified container state in hand, FACET generates the instruction, reference solution, and verifier. Because they all reference the same, tested environment snapshot, consistency becomes structural rather than accidental.

Execution-based validation and targeted repair are key: instead of regenerating components wholesale when something fails, the framework identifies which piece broke and fixes only that piece. A solution might work; only the verifier might need adjustment. This keeps valid work intact.

Why This Matters

The authors evaluate FACET on Terminal-Bench 2.1, showing that tasks generated this way consistently improve fine-tuned models across multiple scales. More importantly, the framework produces dense executable checks—thorough, multi-step verifiers grounded in real environment state—and successful trajectories collected from these tasks provide data-efficient supervision (fewer, higher-quality examples teach better than many low-quality ones).

The research establishes two design principles for scalable task synthesis: source-intent preservation (reconstructing all the constraints and dependencies from originals) and shared executable-state grounding (making all four task artifacts reference the same verified environment). These aren't just nice-to-haves; the ablations show alternative schemes (generating environment-free, or verifying after the fact) produce noticeably worse tasks and misaligned solutions.

In short, FACET trades the simplicity of independent generation for the discipline of environment-grounded construction, and that discipline translates directly into usable training signal.