When Many Answers Are Valid, Voting Fails: Symbolic Verification for Best-of-K Causal Reasoning in LLMs

The Problem: Voting Breaks When Multiple Answers Are Right

Large language models often solve causal reasoning problems by sampling many reasoning traces and picking the most frequent answer—a technique called self-consistency. But this fails catastrophically in causal domains. The authors show that when a problem admits multiple valid causal answers (say, three different adjustment sets that all correctly identify a causal effect), the model's samples fragment across them while repeatedly making the same systematic error. Result: an invalid answer wins by plurality, even though valid answers existed in the pool.

On the CLEAR benchmark's multi-answer queries, standard voting methods—including reward models and even a 72B LLM judge—stuck near 30% accuracy. The authors' key insight: don't count votes. Instead, verify each candidate symbolically against the actual rules of causal reasoning.

How CALVER Works

CALVER (Causal Axiom-Level VERification) is a training-free checker that scores each sampled trace by testing it against Pearl's causal calculus—the formal rules for valid causal inference. The method requires the causal graph (either given explicitly or extracted from text) and then checks whether each candidate satisfies criteria like d-separation (which variables must be independent?), backdoor adjustment (which confounders must be controlled?), and intervention semantics (do the proposed adjustments isolate the true causal effect?).

Rather than ask "what did the model say most often?", CALVER asks "which answer is mathematically valid under the causal rules?" It scores all candidates and picks the highest-scoring one—no ground-truth reference answer needed.

Why This Matters and What It Achieves

On CLEAR's find-one-valid queries (problems with multiple correct answers), CALVER reached 42.1% accuracy versus 30% for plurality voting, reward models, and LLM judges—a substantial gap that widened as the sampling budget grew. Importantly, in a manually audited subset, 11 of 21 CALVER selections differed from the benchmark's listed answer but were still graph-valid—confirming that CALVER finds legitimate alternatives the benchmark itself missed.

The advantage reproduces across ten published Bayesian networks, a second model family (different LLM), and scenarios where the causal graph must be built from text rather than given. CALVER also improves decisions about average treatment effects against ground truth and generalizes to logic problems using a truth-table verifier.

Critically, this is practical: scoring each candidate takes milliseconds on CPU. The method requires only that a causal structure be available—a realistic constraint in many domains like healthcare, economics, and policy evaluation.

The core contribution is philosophical: in domains with well-defined formal semantics, symbolic verification outperforms statistical voting even when ground truth is unavailable. When reasoning must satisfy mathematical rules, check the rules, not the frequency.