Beyond Visual CoT: Internalized Visual Thinking for Proactive Video Reasoning

The Core Finding

Visual chain-of-thought (CoT)—where models generate intermediate reasoning images to tackle video understanding tasks—is slow and possibly unnecessary. The authors demonstrate that multimodal language models can learn to simulate visual reasoning internally during training, then skip the expensive image-generation step at inference while maintaining or improving accuracy. Their method, Internalized Visual Thinking (IVT), achieves comparable or better performance than explicit Visual CoT while cutting end-to-end latency by more than 5x.

How IVT Works

The key insight is to decouple visual reasoning from visual generation. Instead of training models to produce intermediate frames during inference, IVT trains them to predict latent representations (embeddings) of future frames alongside the textual answer.

During training, given a partially observed video, the model jointly predicts two things: (1) the target text answer, and (2) vector embeddings that capture what the next unobserved frames will look like. These embeddings encode motion, object transitions, interactions, and latent intent—the spatial-temporal patterns that reasoning requires—but remain hidden inside the model's representation space.

At inference, the model skips image synthesis entirely. It takes the partial video and outputs the answer directly, having already internalized the predictive visual reasoning during training. The latent embedding prediction acts as an inductive bias: the training objective forces the model to develop internal mechanisms for anticipating future visual states, even though those predictions never surface as pixels.

The authors use unlabeled video data for this post-training framework, making it broadly applicable. They experiment extensively with design choices: different target representations, decoder architectures, prediction horizons (how far into the future to predict), data mixtures, and training curricula.

Why This Matters

Proactive video reasoning—answering questions about partially observed videos by anticipating future events—is computationally expensive with Visual CoT. Generating and re-encoding intermediate reasoning frames adds substantial latency, which compounds in real-time applications like robotics or embodied AI.

IVT sidesteps this bottleneck without sacrificing reasoning quality. By making visual prediction an internal training objective rather than an inference requirement, the method achieves a rare win: better accuracy and dramatic efficiency gains. The 5x latency reduction is particularly significant for deployment scenarios where inference speed matters.

The authors' controlled ablation studies—systematically varying representation types, architectures, horizons, and curricula—strengthen the finding that internalized visual thinking is genuinely effective, not an artifact of specific design choices. All six settings showed improvements over direct fine-tuning, and IVT consistently matched or exceeded explicit Visual CoT.

The broader implication challenges a widespread assumption: that multimodal reasoning requires explicit intermediate outputs. The work suggests that with proper training signals, models can learn to reason visually in latent space, keeping computational costs low while preserving or enhancing reasoning fidelity. This opens a path toward more efficient multimodal reasoning systems without sacrificing the spatial-temporal understanding that video tasks demand.