Self-Evolving Visual Questioner

## The Core Finding

Vision-language models like GPT-4V are trained to answer questions about images, but asking good questions is a separate and harder skill. The authors show that a VLM can teach itself to ask progressively better visual questions without any human-labeled data—by using itself as both a question generator and a critic to filter out weak questions. This self-improvement loop produces questions that are harder, more visually grounded, and more diverse than those in typical training datasets, and doing so actually makes the model a better answerer too.

## How Self-Evolution Works

The framework operates in cycles. In each iteration, the VLM proposes candidate questions about images, then acts as a filter to rank and select the most valuable ones. The selection criteria are crucial: questions must be non-trivial (not answerable by obvious visual shortcuts), visually grounded (actually about what's in the image rather than generic knowledge), and exploratory (covering different aspects of images to avoid repetition).

The key tension the authors navigate is avoiding "training collapse"—where the model converges to asking the same easy-to-answer questions repeatedly. Their solution maintains diversity by ensuring selected questions span different reasoning types and visual regions. Once filtered, these self-generated questions become the training data: the model trains on them both as a questioner (generating better questions) and as an answerer (understanding what makes a question worth asking).

This differs from existing approaches that either require expensive human annotation or rely on static corpora that bottleneck performance. The authors effectively convert the model's own knowledge into a curriculum that tightens over time.

## Why This Matters

The practical payoff is significant. Under the same computational budget, training on self-evolved questions outperforms training on standard datasets. The authors test this across multiple VLM backbones, showing the approach generalizes. They also introduce an "agentic protocol" to evaluate question quality—assessing questions on three dimensions: perception (does it probe visual understanding?), reasoning (does it require multi-step inference?), and diversity (do the questions cover varied aspects?). This evaluation framework itself is a contribution, since assessing question quality automatically is non-trivial.

A surprise: the self-evolved questioner remains competitive or even superior as an answerer. This suggests that asking hard questions teaches the model to understand images more deeply. The method scales naturally—as the model improves, so does its ability to generate harder questions, creating a virtuous cycle.

The broader implication is that VLMs needn't be passive absorbers of fixed training data. They can bootstrap their own learning through self-critique and iterative refinement, opening a path to reducing dependence on expensive human curation. For applications like embodied AI or interactive systems where a model needs to actively probe its environment, this is especially valuable.