Sol-Attn: Accelerating Video Generation Inference via On-the-Fly Attention Sparsification

The Problem: Attention is Strangling Video Generation Speed

Video generation models based on diffusion transformers produce stunning results, but they're slow. The culprit is attention—specifically, the mechanism that lets each token in a video attend to every other token. With video, you're dealing with thousands or tens of thousands of tokens (frames, patches, etc.), making the attention computation quadratic in cost. During inference, this becomes the dominant bottleneck.

Existing sparse attention methods try to dodge this by computing only a subset of key-value blocks instead of the full set. But they have two critical flaws: (1) they waste time and memory computing and storing proxy scores (rough guides for which blocks matter), and (2) when they drop unselected blocks entirely, they lose information that hurts final quality, especially under aggressive sparsity.

How Sol-Attn Works: Sparsify Without Waste

Sol-Attn eliminates both problems by doing sparsification during the online softmax computation itself—the single forward pass that normalizes attention weights. Instead of pre-computing all proxy scores upfront, the method compares each block's proxy score against a threshold on the fly. Blocks that score above the threshold get fully computed; blocks below it are skipped but their proxy scores are recycled to approximate their contribution to the final output.

The key insight: reusing proxy scores as approximations for dropped blocks is far cheaper than discarding them entirely, yet preserves enough signal to maintain quality. The threshold adapts dynamically—you don't need to commit to sparsifying a fixed 50% of blocks. This gives you controllable, efficient budgets without materializing a full proxy map.

Think of it as a single-pass gate: as you process each block, you decide on-the-fly whether it's important enough to compute in full, and if not, you use a quick stand-in estimate instead.

Why It Matters and What It Achieves

This matters because video generation is computationally expensive and increasingly deployed in real applications. A 2.1× speedup for video generation and 2.3× for video editing—while preserving visual quality—is substantial. These aren't toy numbers; they translate to meaningful improvements in inference latency.

The authors validate Sol-Attn across image and video generation tasks, showing it pushes the quality-efficiency frontier forward compared to prior training-free sparse attention methods. Crucially, it achieves this without requiring any model retraining or fine-tuning, making it immediately applicable to existing deployed models.

The unification of dynamic routing, sparse computation, and approximation correction into one online-softmax pass is elegant. It eliminates redundant overhead and sidesteps the false choice between fixed-budget rigidity and dynamic-budget waste. For practitioners running diffusion transformers in production, Sol-Attn offers a plug-in speedup that respects quality constraints—something training-free methods rarely achieve well.