How a robot predicts the next second

A text model guesses the next word. A world model guesses the next frame — mass, speed and gravity included. One slider walks the whole loop, from the camera to the catch.

Stage What it sees
  1. 0
  2. 1
  3. 2
  4. 3
  5. 4

Tap a part of the diagram to open that stage.

The loop, end to end

See a frame, compress it into a state, run that state forward faster than real time, take the action that survives the rollout, then compare what happened with what was imagined. The difference is the lesson.

What's inside

  1. 1 A ball rolling toward the edge — A language model would caption this. A world model has to answer a different question: what happens next?
  2. 2 The picture becomes a handful of numbers — Not a description of the scene — a state you can run forward.
  3. 3 Fifty steps ahead, without moving — The dots are not the real ball. They are the imagination of the model, drawn ahead of it.
  4. 4 The arm takes the trajectory that gets there first — Planning here means simulating your own options, not only the world.
  5. 5 Two curves that almost coincide — Prediction error is what a world model learns from, the way next-word error trains a text model.

Explorable Explanations

Stage What it sees
  1. 0
  2. 1
  3. 2
  4. 3
  5. 4
1 / 5