Remember When It Matters: Proactive Memory Agent for Long-Horizon Agents

The Problem: Memory Decay in Long Tasks

Long-horizon tasks—think navigating a terminal, debugging code, or multi-step planning—scatter critical information across trajectories that grow faster than any fixed context window can hold. A task requirement stated early gets buried. An earlier diagnostic attempt becomes irrelevant to the action agent by step 50. Prior subgoal attempts and environmental facts fade from influence. The authors call this "behavioral state decay": the agent forgets what matters when it matters most.

Standard approaches treat memory as passive retrieval—the agent queries a database when it feels like it. The authors flip the model: what if a separate memory agent actively monitors the trajectory and decides when to interrupt the action agent with a timely reminder? Not constant noise. Not silent retrieval. Strategic injection.

How It Works

A memory agent runs in parallel to an unmodified action agent. As the trajectory unfolds, the memory agent:

1. Maintains a structured memory bank updated incrementally from recent trajectory steps.
2. Decides whether to inject a reminder into the action agent's context—or stays silent.

The memory agent's job is selectivity. It learns what information is actionable right now versus background noise. If the action agent is stuck debugging a file and an earlier diagnostic is relevant, the memory agent surfaces it. If the current decision is local and self-contained, it abstains.

The design is intentionally modular: the memory agent wraps existing agents without modifying them, making it compatible with any frontier LLM-based action agent and any existing agent framework (like ReAct or AutoGPT-style loops).

Results and Training

The authors test on two benchmarks designed for long-horizon tasks: Terminal-Bench 2.0 and τ²-Bench (tau-squared), where trajectories span many steps with scattered dependencies.

Pass@1 improvements:

• Terminal-Bench: +8.3 percentage points
• τ²-Bench: +6.8 percentage points

These gains hold for both weaker and stronger base action agents, suggesting the memory layer generalizes.

Ablations confirm the mechanism matters. Selective intervention (the memory agent choosing when to remind) outperforms:

• Passive bank exposure (agent can see memory but doesn't get nudged)
• Always-on injection (constant reminders—noisy)
• Advisor-only guidance (memory talks without grounding)
• Generic retrieval baselines

The authors also train an open-weight memory policy: they fine-tune Qwen 3.5-27B on a dataset called SETA using supervised fine-tuning (SFT) and a reinforcement learning approach (GRPO). The trained policy improves validation reward and partially transfers to Terminal-Bench, suggesting the memory selection strategy can be learned rather than hard-coded.

Why It Matters

Most agent research assumes the context window is sufficient if you just retrieve smartly enough. This work identifies a deeper problem: when to retrieve is as hard as what. The result is practical—the module works with any off-the-shelf LLM agent—and the insight is general. Any long-horizon task where state sprawls and deadlines matter could benefit from an active memory agent that knows when silence is golden.