StreamOPD: A Post-Training Recipe with Spatio-Temporal Cue Gating for Streaming Video Understanding

The Central Finding

Streaming video understanding requires models to answer questions about videos as they unfold—no rewinding, no future frames. Most existing systems add complex inference machinery (memory buffers, retrieval modules, compression). The authors show that a simple sliding-window baseline already matches those systems, so they ask a harder question: how much can post-training alone improve performance, without changing inference?

Their answer is StreamOPD, a post-training recipe that lifts StreamingBench from 77.9% to 83.9%—closing the gap to a 9B teacher model from 6 points to just 0.3. On OVO-Bench (excluding a hallucination-detection subtask), they gain 9.1 points.

Why Existing Approaches Miss the Mark

Reinforcement learning (RL) with verifiable rewards sounds natural for streaming video: reward correct answers, penalize wrong ones. But the authors find RL encourages models to generate long "think-then-answer" sequences—internal reasoning followed by the actual response. This bloat doesn't help streaming tasks.

On-policy distillation (OPD) offers an alternative: during training, a teacher model provides token-level supervision as the student generates its response. The catch is stability. The authors discover OPD only works reliably when both teacher and student operate in "thinking mode"—generating internal reasoning before answering. At inference, they switch to "instruct mode" (no thinking, direct answers), which is faster and cleaner for real deployment.

This mismatch between training (thinking) and deployment (instruct) is unconventional but critical. Mixing modes during training destabilizes the training process.

The Recipe and ST-CueGate Extension

StreamOPD combines three ingredients:

1. Verifiable streaming-video data (questions with ground-truth answers)
2. On-policy distillation in thinking mode during post-training
3. Direct answer generation at inference time

The authors then introduce Spatio-Temporal CueGate (ST-CueGate), a "teacher-privilege" extension available during training. It works by comparing the teacher's likelihood of the correct answer when given visual cues versus when cues are removed, then aggregates these ratios into a reweighting signal for OPD. This cue-aware weighting reaches 71.9% on OVO-Bench (excluding hallucination detection) and 64.9% on Video-MME—the only variant staying above baseline across all four benchmarks tested.

Self-Distillation and Reproducibility

A striking finding: replacing the 9B teacher with a frozen copy of the student's own initial weights—self-distillation—retains most gains and even lifts the hallucination-detection subtask to 57.0%, above both the untrained student and the teacher. This suggests the recipe's effectiveness isn't tied to having a larger teacher; the structure of thinking-mode training plus instruct-mode deployment matters.

The work is positioned as a "transparent and reproducible reference" for open-source streaming-video research, with clear post-training protocols rather than architecture tricks or inference-time hacks.