Context-Matched Distillation: Teacher Causality for Autoregressive Video Distillation

The Problem: Teacher-Student Mismatch in Video Distillation

Video generation models that produce frames one at a time (autoregressive) need to be fast for real-time use and responsive to user controls like camera movements. The standard approach is distillation: train a small, fast "student" model by learning from a large "teacher" model. But existing distillation methods have a hidden flaw. Teachers evaluate video quality by looking at complete clips—including future frames the student hasn't generated yet. When the student trains on this feedback, it's trying to match a teacher's judgment that depended on information the student won't actually have during real use. This fundamental mismatch degrades performance and breaks user control.

Context-Matched Distillation (CMD)

The authors' core insight is simple: the teacher should only judge each frame using the same causal information (history and controls) available to the student when it was generated.

CMD uses a causal teacher that scores each target frame without seeing future frames or future controls. The same causal teacher also initializes the student, ensuring both operate under identical constraints from training through deployment.

Two refinements tighten this alignment:

Prefix Scoring evaluates each target frame against the actual student-generated rollout that preceded it—the cached sequence of frames the student created up to that point. Rather than comparing against generic context, the teacher sees exactly what the student produced, matching supervision to realized rollout.

Prefix Corruption handles instability during early training, when the student's prefixes are unreliable. The method perturbs these early prefixes slightly during training while preserving the target-context alignment, steadying the learning process.

Why It Matters

The framework naturally handles practical extensions: frame-wise generation (one frame at a time), chunk-wise generation (groups of frames), long-form videos, and camera-conditioned generation where control signals evolve over time.

Experiments on standard benchmarks show CMD achieves state-of-the-art results among autoregressive methods on both short and long videos. More importantly, the model substantially improves adherence to time-varying camera controls—a direct win for interactive applications where users expect the system to respond precisely to their input.

The contribution is architecturally elegant: by enforcing causal consistency, CMD eliminates a source of hidden supervision noise without adding computational overhead. The same causal formulation applies across teacher training, student distillation, and inference, reducing engineering complexity. This unified approach makes the method interpretable and modular, letting practitioners swap components or extend to new generation modes without rethinking the core mechanism.

For practitioners building interactive video systems, CMD addresses a real gap—it ensures that how you train mirrors how you deploy, closing the theory-practice loop that plagues naive distillation.