PaDoc: Layout-Grounded Parallel Decoding for Document Parsing

The Problem: Parsing Documents Faster Without Losing Context

Document parsing—extracting text, layout, and structure from pages—faces a fundamental tradeoff. End-to-end parsers using large language models (LLMs) process entire pages in one autoregressive pass, preserving full context but forcing the model to decode tokens one-by-one in a single sequence. This creates a bottleneck: if a page has many regions (text blocks, tables, formulas), the decoder must wait for earlier regions to finish before starting later ones, even though those regions are independent. Crop-based two-stage parsers avoid this by parsing regions in parallel, but they sacrifice page-level context and waste time reprocessing visual features for overlapping regions.

The authors propose PaDoc, which decodes layout and content in parallel while keeping the full page visible to the model—achieving speedups of 67–118% over sequential baselines without sacrificing accuracy.

How It Works: Layout as a Branching Tree

The key insight is treating page layout prediction as a tree structure: first predict where regions sit on the page, then fill in each region's content in parallel branches that share a common root (the full-page image).

Mathematically, the authors derive a factorization where the joint probability of layout and content breaks into independent conditional branches. The layout stream predicts bounding boxes sequentially, but once a region's box is known, its content can be decoded concurrently with other regions—they only need the layout prefix as context, not each other's outputs.

To implement this in a single multimodal LLM, they use two techniques. First, packed variable-length ancestor attention lets tokens "see" their position in the layout tree during standard next-token training, preserving causality so the model learns which tokens depend on which layout predictions. Second, masked parallel decoding creates multiple token sequences (branches) that decode simultaneously; the inference backend (vLLM) treats them as concurrent requests and reuses the cached page embeddings across all branches, avoiding redundant visual processing.

Results: Speed and Quality

On OmniDocBench Full, PaDoc achieves a layout F1 of 91.1 and an overall parsing score of 94.24—competitive with or better than existing end-to-end parsers. Critically, it excels at low-error metrics: text edit distance of 0.038 and formula accuracy of 95.59, showing the parallel strategy doesn't compromise precision.

In throughput tests on 384 pages with one A800 GPU, PaDoc is the fastest end-to-end parser, improving valid-page throughput by 67–118% and cutting P95 latency (tail latency at the 95th percentile) by 39–55% versus a sequential baseline with the same backbone model. The gains scale across five different concurrency levels, indicating the speedup is robust.

Why It Matters

This work solves a real constraint in production document systems: end-to-end parsing is convenient but slow for long documents or high-throughput scenarios. By exploiting the structure of the problem—layouts are sparse, regions are mostly independent—PaDoc recovers parallelism without building a separate visual pipeline. It's a neat algorithmic insight wrapped in practical engineering, useful wherever document parsing meets latency or throughput budgets.