CURV: Enhancing Chart Understanding Through Curriculum Visual Grounded Reasoning

The Core Problem

Chart question answering sounds simple—look at a chart, answer a question about it—but it requires multimodal large language models (MLLMs) to do two hard things at once: see the chart accurately and reason about it logically. Current models fail because they either guess answers without looking at the right parts of the chart, or they reason in ways disconnected from what's actually visible. Existing fixes like chain-of-thought prompting help, but they're band-aids; they don't teach models to intrinsically ground their reasoning in visual evidence.

How CURV Works

The authors propose CURV, a curriculum learning framework that reframes chart question answering as a step-by-step process where each reasoning step explicitly connects to specific visual regions. Rather than jumping to an answer, the model learns to:

1. Identify what visual elements matter for the current step
2. Use spatial attention to focus on those regions
3. Perform logical reasoning informed by what it sees
4. Move to the next step

This "dynamic visual grounding" forces the model to constantly tie its reasoning back to the chart rather than drifting into abstract logic disconnected from evidence.

To train this approach, the authors created CCQA, a three-level curriculum dataset with synthetically generated charts. The curriculum deliberately progresses from easy to hard: it starts with simple single-operation tasks (e.g., "What is the value at X?"), then moves to multi-operation reasoning (e.g., "Which value is larger?"), and finally tackles complex compositional problems that combine multiple charts. This scaffolding lets models learn visual grounding on simple cases before tackling harder ones.

Why It Works and What Changed

The key insight is that visual grounding and reasoning should happen together, not sequentially. By interleaving them at every step, the model learns to build reasoning chains where each link is anchored to the chart. This is fundamentally different from asking a model to first understand the chart globally and then reason—an approach that often produces plausible-sounding but visually unfounded answers.

The results are substantial. On synthetic benchmarks, CURV achieves up to 20.5% improvement over baselines. More importantly, it generalizes: it improves real-world chart QA benchmarks by up to 12.3% and transfers to other multimodal reasoning tasks with up to 10.2% gains. This suggests the model is learning a genuine capability—visual grounded reasoning—rather than memorizing patterns specific to one dataset.

The curriculum dataset itself is interesting: it's scalable and synthetic, meaning it can cover diverse chart types and reasoning patterns without requiring expensive manual annotation. This makes the approach practical.

Why It Matters

Chart understanding powers real applications—financial analysis, scientific data interpretation, accessibility tools. Current MLLMs often fail on seemingly straightforward charts because they lack internal mechanisms to tie reasoning to evidence. CURV addresses this by making visual grounding a first-class citizen in the learning process, not an afterthought. The generalization results suggest this approach teaches something robust that transfers beyond the training domain, hinting at a more principled way to build multimodal reasoning systems.