From Generation to Simulation: How Far Are World Models from Being True Simulators?

Generative world models—AI systems trained to predict and generate future video frames—are increasingly marketed as replacements for physics engines, game engines, and reinforcement-learning environments. This paper reveals they're not yet there. The authors systematically map 200 world-model papers against eight core capabilities that traditional simulators provide, and show that despite impressive generation quality, world models fundamentally lack the reliability, transparency, and control needed to truly substitute for purpose-built simulators.

The Assessment Framework

Rather than cherry-pick metrics, the authors define what a simulator actually needs to do. Their eight-capability yardstick captures both technical performance and practical usability: asset construction (representing objects), physics engines (enforcing physical laws), interaction (responding to actions), controllability (taking meaningful input), stability (consistent long-horizon behavior), state feedback (exposing runtime object properties), diversity (varied outcomes), and evaluation metrics (quantifying fidelity).

They then categorized 200 published works into three technical routes: latent dynamics models (learning compressed state representations), video generation models (directly predicting pixels), and joint-embedding prediction (learning shared latent spaces for multiple modalities). This taxonomy lets them ask not just "how good?" but "good at what, and according to whom?"

The Gap Is Real, But Specific

The findings are nuanced. World models have genuinely achieved functional parity with simulators in two areas—interaction (responding to user actions in limited scenarios) and controllability (steering generation toward desired outcomes). For certain video game environments or constrained robotics tasks, they can stand in for simulators.

But they fail catastrophically on others. Traditional simulators guarantee that physical laws hold exactly—energy is conserved, objects don't pass through walls—because rules are hardcoded. World models learn these constraints statistically and will eventually hallucinate violations, especially over long horizons. Worse, they almost never expose state feedback: only 6 of 163 papers that actually built working systems gave users a runtime interface to query "what is the position of that object right now?" or "what is the friction coefficient?" Without this, using a world model as a simulator means you can watch it generate plausible video, but you can't actually measure what's happening inside it.

Six Missing Pieces

The authors propose a research agenda. Systems need: (1) formalized physics—baking physical constraints into the architecture rather than hoping the model learns them; (2) unified action interfaces—standardized ways to control any world model, not bespoke per model; (3) first-class state feedback—treating entity queries as core functionality, not an afterthought; (4) long-horizon stability—tracking and fixing error drift over time; (5) downstream-utility evaluation—testing whether world models actually help downstream tasks like planning or control, not just measuring pixel similarity; and (6) cross-route hybridization—combining video generation's visual quality with latent dynamics' computable state space.

The paper's core insight is that generation quality and simulation fidelity are different problems. A world model can be phenomenally good at predicting what the next frame looks like while being useless at predicting whether that frame satisfies a physical constraint. Until the field stops conflating them, generative world models will remain impressive videos—not simulators.