Addressable Memory for Video World Models

The Core Problem: Memory That Forgets

Video world models—systems that predict future frames by generating them step-by-step—use a memory cache to track what they've seen. Think of it like a notebook where the model jots down visual details from each frame so it can refer back to them later. But there's a catch: these models use a standard positional encoding scheme (Rotary Positional Embeddings, or RoPE) that assumes the model will only look back a fixed distance into the past. Once a video rollout exceeds the training horizon—say, the model was trained on 100-frame sequences but now needs to generate 500 frames—the positional offsets venture into territory the model never encountered during training. The model can no longer reliably find and retrieve the right information from its memory, like trying to navigate a city using a map that only covers familiar neighborhoods.

Compressing the memory to make it manageable doesn't help. Naively squashing old frames together creates a new problem: the positional phases get averaged and corrupted, destroying the structure that lets the model locate memories in the first place.

WorldTrace: Addressable Compressed Memory

The authors propose WorldTrace, a training-free framework that solves this by keeping compressed memory addressable—meaning the model can still find what it needs. The key insight is simple: instead of compressing frames and hoping the positional encoding still works, assign each compressed memory slot a virtual position that falls squarely within the distribution the model saw during training. This keeps everything in-distribution where the model is confident.

Within this framework, the authors introduce two complementary compression strategies:

WorldTrace-Field prioritizes temporal coherence. It compresses recent history into a smooth summary, useful when you want continuous visual flow—like tracking motion or lighting changes over time.

WorldTrace-Landmark prioritizes episodic recall. It stores scene traces (verbatim visual snapshots) at detected transitions—doorways, scene cuts, major changes. This helps the model remember and reconstruct specific places it visited long ago, even after a long detour.

Why It Matters and What They Measure

The problem is practically important: long-horizon video generation is vital for robotics, simulation, and interactive environments, but it demands memory that remains usable far beyond training. WorldTrace requires no retraining—it's a plug-and-play fix.

To evaluate this, the authors introduce LoopBench, a benchmark that directly tests episodic memory: can the model reconstruct a previously visited scene after a long detour? This is a meaningful proxy for whether compressed memory actually works in practice.

Results: WorldTrace-Field improves temporal consistency by 15.5%, and WorldTrace-Landmark improves episodic recall by 19.5% on LoopBench. Both variants extend visually persistent generation without the computational and data cost of retraining.

The contribution is elegant: by respecting the positional encoding distribution the model learned, they make compressed memory work reliably at scales far beyond the training horizon.