Better Retrieval, Worse Robustness:How Multi-hop RAG Amplifies Upstream ASR Errors

The Paradox: Better Methods Make ASR Errors Worse

Multi-hop retrieval—chaining together multiple search steps to answer complex questions—sounds like it should be more robust to speech errors. The authors find the opposite: advanced retrieval-augmented generation (RAG) systems that improve performance on clean text actually amplify ASR errors catastrophically.

Using four synthesized English accents with varying speech error rates, the authors tested four RAG setups on three multi-hop QA benchmarks (HotpotQA, 2WikiMultiHopQA, and MuSiQue). The gap between clean-text and highest-error-rate performance balloons 36–67% wider when using entity-graph linking and iterative reformulation—the "better" methods—compared to simple dense retrieval. On 2WikiMultiHopQA, the F1 score degradation is catastrophic: these sophisticated systems actually perform worse than naive approaches when faced with ASR noise.

Why This Happens: Entity Corruption Cascades

The culprit is straightforward but devastating: 87–96% of failures trace back to corrupted query entities. In multi-hop QA, you need to identify key entities in the question, retrieve documents containing them, then chain searches through intermediate hops. When ASR mangled a name or term, downstream retrieval steps fail to find the right documents, and subsequent hops collapse.

Entity-graph linking—which structures queries as graphs of entities and relations—and iterative reformulation—which refines queries across multiple rounds—both amplify this fragility. They introduce more moving parts that depend on entity accuracy. A single corrupted entity early in the chain breaks everything downstream. Dense retrieval, by contrast, is more forgiving: it searches for fuzzy similarity rather than exact entity matches, so it tolerates misspellings better.

The authors tested two lightweight mitigations based on surface-form matching (likely phonetic similarity or character-level fallbacks), but these barely dent the gap. This reveals that the problem isn't just detecting corrupted entities—it's that the downstream retrieval structure itself is brittle. Even when you patch entity errors locally, the multi-hop architecture still struggles to recover.

Why It Matters

This finding challenges a comfortable assumption: that stronger models on standard benchmarks will simply be stronger everywhere. The paper exposes a real deployment risk for speech-based QA systems. Voice interfaces are growing, but if you deploy a sophisticated multi-hop RAG system, ASR errors—inevitable in real speech—will hurt you disproportionately compared to simpler methods.

The result is a call to rethink architecture for robustness, not just accuracy. Current multi-hop RAG systems are brittle pipelines where one upstream error cascades through multiple retrieval steps. Better solutions likely need to tolerate uncertainty in entities earlier—perhaps by keeping multiple entity hypotheses, performing parallel retrieval paths, or using retrieval methods that gracefully degrade under noise rather than fail sharply.

The authors' release of code and data enables others to benchmark their own methods against this problem, making it a concrete benchmark for speech-to-QA robustness rather than an isolated observation.