Self-Improvements in Modern Agentic Systems: A Survey

The Core Claim

Modern AI agents can improve themselves through experience without human retraining—by systematically updating either their underlying models or their operational scaffolding (prompts, memory, tools, logic). This survey argues that self-improvement is becoming practical in deployed systems, not just research curiosities, and provides a unifying framework to understand how it works.

What Makes an Agent Self-Improving

The authors define a modern agent as a foundation model (like GPT-4) paired with an operational scaffold—the practical infrastructure that makes the agent functional. This scaffold includes prompts that guide behavior, memory systems that retain context, tools the agent can call, and control logic that orchestrates decisions. Self-improvement happens when the agent updates any of these components based on its own experience.

The key insight is that updates can target two different levels. Model-level updates modify the foundation model's parameters directly—for instance, fine-tuning on successful trajectories the agent discovered itself. Scaffold-level updates adjust prompts, refine memory structures, or swap out tools without touching the underlying model. Many practical systems use scaffold updates because they're faster, cheaper, and easier to debug than retraining a massive model.

How the Survey Organizes the Landscape

The authors structure the field along two dimensions: what gets updated (model parameters versus scaffold components) and what signals drive the update (reward signals, error feedback, or self-generated critique). This two-axis view helps clarify why different approaches succeed in different domains.

For example, an agent might improve its prompts by observing which questions led to correct answers (reward signal) or by analyzing its own mistakes (error feedback). Some systems use both signals in tandem. The survey catalogs these combinations across applications: autonomous coding, scientific discovery, planning, and reasoning tasks.

Why This Matters Now

Self-improving agents shift the bottleneck from labeled data and human feedback to the agent's own ability to learn from deployment. A system deployed in the field can adapt to its specific environment, fix its own mistakes, and accumulate capabilities over time—without waiting for engineers to retrain it centrally. For safety-critical domains (medicine, autonomous systems), this controllable evolution is essential; for commercial applications, it reduces operational overhead.

The survey also acknowledges the hard problems: how to evaluate whether self-improvement is genuine or illusory, how to ensure agents don't overfit to spurious patterns in their own experience, and how to maintain human oversight as agents become more autonomous. It doesn't claim to solve these, but maps where they sit in the research landscape.

The authors provide a GitHub repository tracking technical updates, recognizing that this field moves faster than peer review cycles. This reflects the practical urgency: self-improving agents are no longer theoretical—they're already in use in startups and labs—and practitioners need a shared vocabulary and taxonomy to communicate across projects.