How a language model learns
Pull one slider and watch a pile of random numbers turn into a prediction over a thousand training steps.
Training steps 0
Tap a part of the diagram to open that step.
The whole loop
Pieces of text → numbers → layers of weights → scores → percentages → error → a correction to every weight. Nothing is memorised: training is millions of tiny adjustments to numbers, repeated until the right continuation becomes the most likely one.
What's inside
- 1 Cutting the text up — The leading space belongs to the piece, so " drinks" and "drinks" are two different rows.
- 2 Numbers instead of words — Move the slider and the jitter stops — the numbers settle on values that mean something.
- 3 Layers and attention — Line colour is the sign of the weight, thickness is its size. Right now every weight is random.
- 4 The prediction — The correct continuation, "Milk", is still in third place.
- 5 Error and learning — The slider drives that cycle: by step 1000 "Milk" is at 92%, "Tractor" at 0.01%, and the error has fallen to 0.08.
Explorable Explanations
- How a drug is smuggled past the blood–brain barrier The brain refuses almost every drug molecule, and it is right to. One slider follows the trick the labs are trying instead: wrap the molecule in a bubble the wall already knows how to carry.
- How a robot draws the map of a room it has never seen A lidar returns a few hundred dots per sweep and no idea where it is. One slider walks from that cloud to a floor plan with the robot standing in the right spot.
- How two people type in one line without losing a letter Two cursors, one line, nobody holding the pen in the middle. One slider takes an index-based merge apart and rebuilds it out of addresses that sit between the letters.
- Why a quantum computer stops at a lattice Shor’s algorithm eats RSA in one move. The same machine gets almost nothing on a grid of points with a little noise added. One slider shows the difference.
- Cooling a building by pointing it at space A coating that stays colder than the air around it, in full sun, drawing no power at all. One slider follows the heat from the sun to the far side of the atmosphere.
- How a robot predicts the next second A text model guesses the next word. A world model guesses the next frame — mass, speed and gravity included. One slider walks the whole loop, from the camera to the catch.
- Proving a secret without revealing it One slider walks through the stages, the other adds rounds. Watch certainty climb to almost total while the secret stays a secret.
- Getting lithium out of brine: ponds versus direct extraction Move the slider through five stages and compare two ways of taking the same metal out of the same salt water.
- How a personalised mRNA cancer vaccine works Move the slider through five stages and watch how a tumour’s own mutations become the instructions that teach the immune system to find it.
- The edge of the app Everything arriving from outside is untrusted. Break one link at a time and watch where the chain stops and what the screen shows.
- How search over your documents works Move the sliders and watch a search pipeline decide what the model is allowed to see — and what happens when it is fed the wrong passage.
- Retries and slow responses The network drops in the middle of an operation. Press the button twice and see what ends up in the database — with an idempotency key and without one.
- How a language model writes The weights are frozen and nothing is remembered. Pull the sliders and watch the model pick the next piece of text, over and over.
- Where electricity comes from Pull one slider and follow the energy from a river all the way to a pump that lifts water back up.