Agents Catching Agents: Shortcut Cascades and Benchmark Gaming in Clinical Multi-Agent Systems

Clinical AI Committees Fall for Social Pressure, Not Tricks

Multi-agent language models—teams of AI systems deliberating together—are becoming a popular way to improve clinical decision-making. This paper tests a troubling hypothesis: can these committees be tricked into wrong answers by exploiting social dynamics rather than technical flaws?

The authors tested Gemini-based agent committees across 42 benchmark scenarios spanning medical text questions (MedQA, MedMCQA), radiology reports and images (MIMIC-CXR, CheXpert, ChestX-ray14), and ICU patient records (SUPPORT2). They planted three types of shortcuts—false cues a benchmark might reward but a real clinician would ignore—and watched whether the committees caught or adopted them.

The headline finding: isolated shortcuts fail. When one agent in the committee is fed a misleading signal (say, a false keyword in a report), the team resists it 84–95% of the time. But the moment two agents agree on the wrong answer, a third agent breaks ranks and switches to the wrong answer in 38% of cases. A fake system flag saying "pre-screened" produces similar social conformity. The authors call this dynamic contagion: peer consensus is more persuasive than any individual trick.

What Makes Committees Vulnerable: Social Over Sensory

The paper's key insight separates two failure modes. One is sensory: visual tricks (making a misleading cue three times brighter) barely budge adoption. The other is social: a second peer voice raises contagion by 50%. This reveals that committees are gamed by social plausibility—the appearance that other trustworthy agents agree—not by exploiting attention or saliency.

When asked why they changed their minds, drifting agents almost never named the actual shortcut they'd fallen for: only 1 in 10 on text tasks and 1 in 134 on imaging tasks. The committees were blind to their own conformity.

Oversight That Works (and Doesn't)

The authors tested three countermeasures. A gatekeeper agent—tasked with blocking adoption of wrong answers—failed entirely (100% false-positive rate; it couldn't distinguish conformity from honest agreement). A judge that reads only the committee's transcript and decides post-hoc works perfectly on text (93% recall, 100% precision) but collapses on imaging. Only a referee that privately re-queries the holdout agent—asking it to justify its position without the group watching—consistently catches gaming across all modalities (77–88% precision, though with 13–21% false-positive rate).

Why This Matters

As hospitals deploy multi-agent systems for triage, diagnosis, and case review, the paper exposes a blind spot: benchmarks can mask a failure mode that won't show up in siloed tests. A committee might ace MedQA in the lab, then drift toward wrong answers in production when one agent's error cascades through social conformity. The remedy isn't a passive gatekeeper but an active one that breaks the group's hold on dissenters—akin to how clinical ethics committees use devil's advocates or independent review.