Specification-First Refactoring at Scale: AI Agent Completes "Infeasible" Architectural Change

An AI coding agent successfully dismantled a core architectural invariant across a 717k-line TypeScript codebase—a task the author deemed infeasible for incremental refactoring—by following a strict specification-first protocol with no human code review and no pre-existing test oracle to validate correctness.

The Problem and the Change

The system is a production application spanning 3,648 files. The central invariant being removed: that a UI panel must remain open for the entire duration of an AI request. The target behavior was more flexible: streaming generations should survive panel closure, persist as live streams, and reattach to the same stream when the panel reopens—with zero data loss or duplication.

This is a deep architectural constraint, touching 189 files directly (31 newly created). The full diff across two commits spans 288 files, 34,770 additions, 16,422 deletions. Conventionally, such a change would require a codebase rewrite rather than incremental refactoring.

The Protocol

The agent followed a two-phase audit-heavy workflow:

Specification phase (14 refinement cycles): The agent formalized the target behavior in writing, then audited its specification against the existing source code to ensure it captured the actual constraints and behavior that needed to change.

Implementation phase (17 verification cycles): After atomic implementation, the agent ran a compile/test feedback loop, then re-audited the resulting code against the frozen specification. Across all 31 audit passes, the agent identified and corrected 201 defects before any human ran the program.

The convergence criterion was empirical: two consecutive verification passes with zero findings.

Why This Matters

The case study challenges a widespread assumption about large refactorings: that they require human judgment and incremental testing. Here, formal specification + relentless automated auditing + tight feedback loops replaced human code review. The author executed no tests themselves during development; the first human execution of the refactored code occurred three days later, after the agent had reached convergence.

The results: Across the initial session and roughly thirty subsequent sessions, the software behaved as specified with no observed bugs. Total elapsed time: three days. Total cost: USD 2,430.

The evidence is public: the authors published over 1,500 pages of session logs and the full specification (in French) to allow inspection of the process and consistency checking by language models.

Key Insight

This work suggests that specification-first development with AI agents may unlock a new mode for high-stakes refactoring—one where formal rigor and exhaustive auditing substitute for the human code review and incremental validation that architects have relied on for decades. The absence of an oracle (no pre-existing golden tests) did not prevent convergence; the specification itself became the oracle.