World models: two meanings of one term

The term does two jobs

"World model" is used for two things that share a name and little else.

The first is the reinforcement-learning sense: a learned model of environment dynamics that an agent uses to predict the consequences of its actions and to plan or train without touching the real environment. The second is the generative spatial sense: a model that produces images, video, or 3D scenes that stay consistent as a camera moves through them. Both predict what comes next; they differ in what the prediction is for.

Sense A: the model an agent plans inside

Ha and Schmidhuber's 2018 paper set the template: a vision component compresses observations to a latent code, a memory component predicts the next latent given the action, and a small controller is trained entirely inside the learned model's "dream." The Dreamer line extended this to harder control problems. Dreamer 4 (Hafner et al., 2025) trains an agent inside a world model learned from Minecraft video and evaluates the model by having humans play inside it.

LeCun's 2022 position paper puts the same component at the center of autonomous intelligence, with a twist: the Joint Embedding Predictive Architecture (JEPA) predicts in abstract representation space and refuses to reconstruct pixels, on the view that pixel-level prediction wastes capacity on unpredictable detail. In this sense, a world model is judged by whether the agent using it acts better.

Sense B: the model that renders a world

Genie 3 (Google DeepMind, 2025) generates interactive environments frame by frame from a text description and user actions, in real time. OpenAI's 2024 Sora report framed video generation as "world simulation." World Labs' AtlasNoteSeedAtlas: World Labs' spatial world modelWorld Labs announced Atlas on 2026-09-01 as a single multimodal model for camera-controlled video, 3D reconstruction, and simulation. It is in early access only; no paper, weights, or pricing have been published. extends this to explicit 3D outputs (point clouds and Gaussian splats) with camera pose as a native input.

Here the model is judged by visual and geometric consistency: does the scene hold together when you turn around? Agents training inside such models is a hope, not the current evaluation.

Why the distinction matters

The two senses fail differently. Dreamer 4's authors note that controllable video models "still struggle to learn the precise physics of object interactions and game mechanics," which limits their use for training agents. A model can look right and be useless for planning; a latent model can support planning while rendering nothing recognizable.

The distinction also predicts corporate behavior. OpenAI shut down the Sora consumer app in 2026 and, per Business Insider, redirected the team toward "world simulation research to advance robotics": a move from sense B toward sense A. World Labs is moving the other way, from rendering toward robot simulation.

Markets are not physical space

A third case sits outside both senses. A "world model" of a market must handle ReflexivityNoteSeedReflexivityA model of a market changes the market. Soros's reflexivity, the Lucas critique, and Goodhart's law are three statements of the same limit, and it is the one way markets are permanently harder than chemistry: molecules do not read the paper.: participants observe the model's outputs and change their behavior, so the thing being modeled reacts to the model. Physical space does not. A room does not rearrange itself because Atlas predicted it. That is why a world model of a market is a different object from a world model of a kitchen, and why techniques built for one do not transfer cleanly.

Open questions

• Will generative spatial models get good enough at contact physics to serve as training environments, closing the gap between the two senses?
• Does JEPA-style latent prediction scale to the diversity pixel-space models handle? No public results answer this yet.

Review log

• 2026-09-08: created; sources checked on this date.