Flex-Forcing: Towards a Unified Autoregressive and Bidirectional Video Diffusion Model

The Core Claim

Video generation today faces a fundamental tradeoff: bidirectional diffusion models (which look at the entire video at once) produce globally coherent, high-quality results but are slow; autoregressive models (which generate frame-by-frame sequentially) are fast and can stream but lose long-range consistency and suffer from compounding errors. The authors show that a single model can do both—and do both better than either alone—through a flexible chunking scheme that operates across both time and denoising steps.

How It Works

The key insight is treating the temporal axis and the denoising process as orthogonal dimensions. Instead of forcing a model into a single generation order, Flex-Forcing lets it operate flexibly within dynamically sized chunks.

Here's the mechanism in practice: during training and inference, the model chunks the video temporally (e.g., into short segments) and also operates across different denoising timesteps. Within each chunk, frames are generated autoregressively—meaning the model conditions on previously generated frames to produce the next one, enabling fast, streaming synthesis. Across chunks, the model can perform bidirectional inference, meaning it can look backward and forward to plan global structure before filling in fine details within each segment.

This design buys three concrete capabilities:

1. Flexible resource allocation: Users can adjust chunk size based on device memory or compute budget. Smaller chunks use less memory; larger chunks improve global coherence.

2. Hybrid generation: The model plans shot composition and long-range dependencies bidirectionally between chunks, then synthesizes frames efficiently within chunks autoregressively.

3. Free generation order: Unlike strict left-to-right autoregressive models, Flex-Forcing can generate frames in any order and at any denoising step without enforcing causal constraints, reducing exposure bias (the training-inference mismatch where errors during generation compound).

The training procedure remains unified—the model learns a single set of weights that work across all these inference modes.

Why It Matters

The results validate a pragmatic middle ground. On standard video generation benchmarks, Flex-Forcing achieves better visual quality and temporal stability than baselines with rigid schedules (either fully bidirectional or fully autoregressive). Inference is faster than pure bidirectional approaches, while maintaining the long-video coherence that pure autoregressive methods struggle with.

The flexibility is operationally valuable: the same model adapts to different hardware constraints and use cases—streaming applications can use small chunks; quality-focused scenarios can use larger chunks or bidirectional passes. No retraining needed.

This addresses a real pain point in diffusion-based video generation: the rigid choice between speed and quality. By decoupling the temporal and denoising axes, the authors show that models don't have to pick one paradigm. The approach is general enough to apply to other sequential generation tasks where similar speed–quality tradeoffs arise.