How the Machine Guesses the Next Word
Before we can talk about agents, look at what a large language model actually does. It does exactly one thing: guess the next token. Everything agentic we build later stands on that single, humble mechanic.
Type The capital of France is into a language model and it will answer Paris. It feels like the model knows something. It does not. What actually happened is narrower and stranger. Given those five words, the model computed that, of every possible next token in its vocabulary, the one most likely to come next was Paris. Then it stopped.
That is the whole trick. A large language model is a machine that, given some text, guesses what comes next. Not the answer. The next token.
I ended the last post1 with a promise: that language is becoming something that runs, that words are turning into instructions a machine can carry out. This series is where I pull on that thread. We are going to build up, primitive by primitive, to agents that plan and act on their own. But you cannot understand agentic AI without the thing at the bottom of it. Here is the thesis for the whole series.
A language model is a next-token guesser, and nothing more. Every "agentic" capability we build in later posts is a runtime wrapped around that one mechanic.
Words become tokens
The model never sees your letters. The first thing that happens to your text is that it gets chopped into tokens: chunks of characters drawn from a fixed vocabulary the model learned once and never changes. A token is often a whole word, sometimes a word-piece, sometimes a single character or a space.
"unbelievable" -> ["un", "bel", "iev", "able"]
"The cat sat" -> ["The", " cat", " sat"]
That splitting is done by a scheme like byte-pair encoding, which merges the most common character sequences into single tokens.2 The model's entire world is a sequence of these integer token IDs. It has no words, only numbers standing in for pieces of words. You can paste text into OpenAI's tokenizer and watch your own words split.
Tokens become vectors
A token ID like 1044 is just an index. On its own it carries no meaning, so the next step hands each token a vector, a long list of numbers, pulled from a lookup table called an embedding. Tokens that tend to appear in similar contexts end up with similar vectors, which is how the model gets a usable notion of "these two things are related" without anyone ever defining a single word.
So The capital of France is is now a handful of vectors, sitting in a very high-dimensional space, waiting to be processed.
Vectors go through a transformer, and out comes the next token
Those vectors flow into the transformer. For now, treat it as a box with one very specific job. At the very end it produces one number for every token in the vocabulary, a score for how likely each one is to come next. Run those scores through a softmax and you get a probability distribution over the whole vocabulary. What is actually inside that box, the attention mechanism that lets every word read every other word, is a post of its own later in this series; here we only need what it does, not how.
Paris scores highest. The runtime picks a token from that distribution, and then does the one thing that turns a next-token guesser into a writer of paragraphs. It appends the token it just picked to the input, and runs the whole thing again.
Here is that loop on a fresh sentence. The model starts with The best food in Japan is, guesses one token, appends it, and goes again, a few times over. Watch the last word arrive in two pieces.
version: "2.0"
scene:
type: player
width: 920
height: 596
fps: 30
background: transparent
controls: true
loop: true
autoPlay: true
children: [r0_t0, r0_t1, r0_t2, r0_t3, r0_t4, r0_t5, r0_in, r0_m, r0_out, r0_emit, r1_t0, r1_t1, r1_t2, r1_t3, r1_t4, r1_t5, r1_t6, r1_in, r1_m, r1_out, r1_emit, r2_t0, r2_t1, r2_t2, r2_t3, r2_t4, r2_t5, r2_t6, r2_t7, r2_in, r2_m, r2_out, r2_emit, r3_t0, r3_t1, r3_t2, r3_t3, r3_t4, r3_t5, r3_t6, r3_t7, r3_t8, r3_in, r3_m, r3_out, r3_emit, r4_t0, r4_t1, r4_t2, r4_t3, r4_t4, r4_t5, r4_t6, r4_t7, r4_t8, r4_t9, r4_in, r4_m, r4_out, r4_emit, r5_t0, r5_t1, r5_t2, r5_t3, r5_t4, r5_t5, r5_t6, r5_t7, r5_t8, r5_t9, r5_t10, r5_in, r5_m, r5_out, r5_emit]
elements:
r0_t0:
id: r0_t0
type: text
props: { type: text, x: 44, y: 70, content: "The", fill: "$primary", fontSize: 22, fontWeight: 600, textAnchor: start }
r0_t1:
id: r0_t1
type: text
props: { type: text, x: 96, y: 70, content: "best", fill: "$success", fontSize: 22, fontWeight: 600, textAnchor: start }
r0_t2:
id: r0_t2
type: text
props: { type: text, x: 160, y: 70, content: "food", fill: "$warning", fontSize: 22, fontWeight: 600, textAnchor: start }
r0_t3:
id: r0_t3
type: text
props: { type: text, x: 224, y: 70, content: "in", fill: "$error", fontSize: 22, fontWeight: 600, textAnchor: start }
r0_t4:
id: r0_t4
type: text
props: { type: text, x: 262, y: 70, content: "Japan", fill: "$secondary", fontSize: 22, fontWeight: 600, textAnchor: start }
r0_t5:
id: r0_t5
type: text
props: { type: text, x: 339, y: 70, content: "is", fill: "$primary", fontSize: 22, fontWeight: 600, textAnchor: start }
r0_in:
id: r0_in
type: line
props: { type: line, x1: 376, y1: 63, x2: 682, y2: 63, stroke: "$muted", strokeWidth: 2, endCap: arrow }
r0_m:
id: r0_m
type: rect
props: { type: rect, x: 690, y: 36, width: 54, height: 54, rx: 8, fill: "$surface", stroke: "$foreground", strokeWidth: 2 }
r0_out:
id: r0_out
type: line
props: { type: line, x1: 752, y1: 63, x2: 802, y2: 63, stroke: "$muted", strokeWidth: 2, endCap: arrow }
r0_emit:
id: r0_emit
type: text
props: { type: text, x: 814, y: 70, content: "Sushi", fill: "$success", fontSize: 22, fontWeight: 600, textAnchor: start }
r1_t0:
id: r1_t0
type: text
props: { type: text, x: 44, y: 164, content: "The", fill: "$primary", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r1_t1:
id: r1_t1
type: text
props: { type: text, x: 96, y: 164, content: "best", fill: "$success", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r1_t2:
id: r1_t2
type: text
props: { type: text, x: 160, y: 164, content: "food", fill: "$warning", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r1_t3:
id: r1_t3
type: text
props: { type: text, x: 224, y: 164, content: "in", fill: "$error", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r1_t4:
id: r1_t4
type: text
props: { type: text, x: 262, y: 164, content: "Japan", fill: "$secondary", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r1_t5:
id: r1_t5
type: text
props: { type: text, x: 339, y: 164, content: "is", fill: "$primary", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r1_t6:
id: r1_t6
type: text
props: { type: text, x: 378, y: 164, content: "Sushi", fill: "$success", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r1_in:
id: r1_in
type: line
props: { type: line, x1: 452, y1: 157, x2: 682, y2: 157, stroke: "$muted", strokeWidth: 2, endCap: arrow, opacity: 0 }
r1_m:
id: r1_m
type: rect
props: { type: rect, x: 690, y: 130, width: 54, height: 54, rx: 8, fill: "$surface", stroke: "$foreground", strokeWidth: 2, opacity: 0 }
r1_out:
id: r1_out
type: line
props: { type: line, x1: 752, y1: 157, x2: 802, y2: 157, stroke: "$muted", strokeWidth: 2, endCap: arrow, opacity: 0 }
r1_emit:
id: r1_emit
type: text
props: { type: text, x: 814, y: 164, content: ",", fill: "$warning", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r2_t0:
id: r2_t0
type: text
props: { type: text, x: 44, y: 258, content: "The", fill: "$primary", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r2_t1:
id: r2_t1
type: text
props: { type: text, x: 96, y: 258, content: "best", fill: "$success", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r2_t2:
id: r2_t2
type: text
props: { type: text, x: 160, y: 258, content: "food", fill: "$warning", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r2_t3:
id: r2_t3
type: text
props: { type: text, x: 224, y: 258, content: "in", fill: "$error", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r2_t4:
id: r2_t4
type: text
props: { type: text, x: 262, y: 258, content: "Japan", fill: "$secondary", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r2_t5:
id: r2_t5
type: text
props: { type: text, x: 339, y: 258, content: "is", fill: "$primary", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r2_t6:
id: r2_t6
type: text
props: { type: text, x: 378, y: 258, content: "Sushi", fill: "$success", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r2_t7:
id: r2_t7
type: text
props: { type: text, x: 454, y: 258, content: ",", fill: "$warning", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r2_in:
id: r2_in
type: line
props: { type: line, x1: 479, y1: 251, x2: 682, y2: 251, stroke: "$muted", strokeWidth: 2, endCap: arrow, opacity: 0 }
r2_m:
id: r2_m
type: rect
props: { type: rect, x: 690, y: 224, width: 54, height: 54, rx: 8, fill: "$surface", stroke: "$foreground", strokeWidth: 2, opacity: 0 }
r2_out:
id: r2_out
type: line
props: { type: line, x1: 752, y1: 251, x2: 802, y2: 251, stroke: "$muted", strokeWidth: 2, endCap: arrow, opacity: 0 }
r2_emit:
id: r2_emit
type: text
props: { type: text, x: 814, y: 258, content: "ramen", fill: "$error", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r3_t0:
id: r3_t0
type: text
props: { type: text, x: 44, y: 352, content: "The", fill: "$primary", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r3_t1:
id: r3_t1
type: text
props: { type: text, x: 96, y: 352, content: "best", fill: "$success", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r3_t2:
id: r3_t2
type: text
props: { type: text, x: 160, y: 352, content: "food", fill: "$warning", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r3_t3:
id: r3_t3
type: text
props: { type: text, x: 224, y: 352, content: "in", fill: "$error", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r3_t4:
id: r3_t4
type: text
props: { type: text, x: 262, y: 352, content: "Japan", fill: "$secondary", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r3_t5:
id: r3_t5
type: text
props: { type: text, x: 339, y: 352, content: "is", fill: "$primary", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r3_t6:
id: r3_t6
type: text
props: { type: text, x: 378, y: 352, content: "Sushi", fill: "$success", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r3_t7:
id: r3_t7
type: text
props: { type: text, x: 454, y: 352, content: ",", fill: "$warning", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r3_t8:
id: r3_t8
type: text
props: { type: text, x: 481, y: 352, content: "ramen", fill: "$error", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r3_in:
id: r3_in
type: line
props: { type: line, x1: 556, y1: 345, x2: 682, y2: 345, stroke: "$muted", strokeWidth: 2, endCap: arrow, opacity: 0 }
r3_m:
id: r3_m
type: rect
props: { type: rect, x: 690, y: 318, width: 54, height: 54, rx: 8, fill: "$surface", stroke: "$foreground", strokeWidth: 2, opacity: 0 }
r3_out:
id: r3_out
type: line
props: { type: line, x1: 752, y1: 345, x2: 802, y2: 345, stroke: "$muted", strokeWidth: 2, endCap: arrow, opacity: 0 }
r3_emit:
id: r3_emit
type: text
props: { type: text, x: 814, y: 352, content: ",", fill: "$warning", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r4_t0:
id: r4_t0
type: text
props: { type: text, x: 44, y: 446, content: "The", fill: "$primary", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r4_t1:
id: r4_t1
type: text
props: { type: text, x: 96, y: 446, content: "best", fill: "$success", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r4_t2:
id: r4_t2
type: text
props: { type: text, x: 160, y: 446, content: "food", fill: "$warning", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r4_t3:
id: r4_t3
type: text
props: { type: text, x: 224, y: 446, content: "in", fill: "$error", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r4_t4:
id: r4_t4
type: text
props: { type: text, x: 262, y: 446, content: "Japan", fill: "$secondary", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r4_t5:
id: r4_t5
type: text
props: { type: text, x: 339, y: 446, content: "is", fill: "$primary", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r4_t6:
id: r4_t6
type: text
props: { type: text, x: 378, y: 446, content: "Sushi", fill: "$success", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r4_t7:
id: r4_t7
type: text
props: { type: text, x: 454, y: 446, content: ",", fill: "$warning", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r4_t8:
id: r4_t8
type: text
props: { type: text, x: 481, y: 446, content: "ramen", fill: "$error", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r4_t9:
id: r4_t9
type: text
props: { type: text, x: 558, y: 446, content: ",", fill: "$warning", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r4_in:
id: r4_in
type: line
props: { type: line, x1: 582, y1: 439, x2: 682, y2: 439, stroke: "$muted", strokeWidth: 2, endCap: arrow, opacity: 0 }
r4_m:
id: r4_m
type: rect
props: { type: rect, x: 690, y: 412, width: 54, height: 54, rx: 8, fill: "$surface", stroke: "$foreground", strokeWidth: 2, opacity: 0 }
r4_out:
id: r4_out
type: line
props: { type: line, x1: 752, y1: 439, x2: 802, y2: 439, stroke: "$muted", strokeWidth: 2, endCap: arrow, opacity: 0 }
r4_emit:
id: r4_emit
type: text
props: { type: text, x: 814, y: 446, content: "temp", fill: "$secondary", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r5_t0:
id: r5_t0
type: text
props: { type: text, x: 44, y: 540, content: "The", fill: "$primary", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r5_t1:
id: r5_t1
type: text
props: { type: text, x: 96, y: 540, content: "best", fill: "$success", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r5_t2:
id: r5_t2
type: text
props: { type: text, x: 160, y: 540, content: "food", fill: "$warning", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r5_t3:
id: r5_t3
type: text
props: { type: text, x: 224, y: 540, content: "in", fill: "$error", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r5_t4:
id: r5_t4
type: text
props: { type: text, x: 262, y: 540, content: "Japan", fill: "$secondary", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r5_t5:
id: r5_t5
type: text
props: { type: text, x: 339, y: 540, content: "is", fill: "$primary", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r5_t6:
id: r5_t6
type: text
props: { type: text, x: 378, y: 540, content: "Sushi", fill: "$success", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r5_t7:
id: r5_t7
type: text
props: { type: text, x: 454, y: 540, content: ",", fill: "$warning", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r5_t8:
id: r5_t8
type: text
props: { type: text, x: 481, y: 540, content: "ramen", fill: "$error", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r5_t9:
id: r5_t9
type: text
props: { type: text, x: 558, y: 540, content: ",", fill: "$warning", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r5_t10:
id: r5_t10
type: text
props: { type: text, x: 584, y: 540, content: "temp", fill: "$secondary", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
r5_in:
id: r5_in
type: line
props: { type: line, x1: 646, y1: 533, x2: 682, y2: 533, stroke: "$muted", strokeWidth: 2, endCap: arrow, opacity: 0 }
r5_m:
id: r5_m
type: rect
props: { type: rect, x: 690, y: 506, width: 54, height: 54, rx: 8, fill: "$surface", stroke: "$foreground", strokeWidth: 2, opacity: 0 }
r5_out:
id: r5_out
type: line
props: { type: line, x1: 752, y1: 533, x2: 802, y2: 533, stroke: "$muted", strokeWidth: 2, endCap: arrow, opacity: 0 }
r5_emit:
id: r5_emit
type: text
props: { type: text, x: 814, y: 540, content: "ura", fill: "$primary", fontSize: 22, fontWeight: 600, textAnchor: start, opacity: 0 }
timelines:
intro:
id: intro
duration: 328
tracks:
- target: r1_t0
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 16, value: 0 }, { frame: 28, value: 1, easing: easeOutCubic }]
- target: r1_t1
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 16, value: 0 }, { frame: 28, value: 1, easing: easeOutCubic }]
- target: r1_t2
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 16, value: 0 }, { frame: 28, value: 1, easing: easeOutCubic }]
- target: r1_t3
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 16, value: 0 }, { frame: 28, value: 1, easing: easeOutCubic }]
- target: r1_t4
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 16, value: 0 }, { frame: 28, value: 1, easing: easeOutCubic }]
- target: r1_t5
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 16, value: 0 }, { frame: 28, value: 1, easing: easeOutCubic }]
- target: r1_t6
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 16, value: 0 }, { frame: 28, value: 1, easing: easeOutCubic }]
- target: r1_in
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 16, value: 0 }, { frame: 28, value: 1, easing: easeOutCubic }]
- target: r1_m
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 16, value: 0 }, { frame: 28, value: 1, easing: easeOutCubic }]
- target: r1_out
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 40, value: 0 }, { frame: 52, value: 1, easing: easeOutCubic }]
- target: r1_emit
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 40, value: 0 }, { frame: 52, value: 1, easing: easeOutCubic }]
- target: r2_t0
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 62, value: 0 }, { frame: 74, value: 1, easing: easeOutCubic }]
- target: r2_t1
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 62, value: 0 }, { frame: 74, value: 1, easing: easeOutCubic }]
- target: r2_t2
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 62, value: 0 }, { frame: 74, value: 1, easing: easeOutCubic }]
- target: r2_t3
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 62, value: 0 }, { frame: 74, value: 1, easing: easeOutCubic }]
- target: r2_t4
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 62, value: 0 }, { frame: 74, value: 1, easing: easeOutCubic }]
- target: r2_t5
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 62, value: 0 }, { frame: 74, value: 1, easing: easeOutCubic }]
- target: r2_t6
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 62, value: 0 }, { frame: 74, value: 1, easing: easeOutCubic }]
- target: r2_t7
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 62, value: 0 }, { frame: 74, value: 1, easing: easeOutCubic }]
- target: r2_in
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 62, value: 0 }, { frame: 74, value: 1, easing: easeOutCubic }]
- target: r2_m
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 62, value: 0 }, { frame: 74, value: 1, easing: easeOutCubic }]
- target: r2_out
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 86, value: 0 }, { frame: 98, value: 1, easing: easeOutCubic }]
- target: r2_emit
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 86, value: 0 }, { frame: 98, value: 1, easing: easeOutCubic }]
- target: r3_t0
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 108, value: 0 }, { frame: 120, value: 1, easing: easeOutCubic }]
- target: r3_t1
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 108, value: 0 }, { frame: 120, value: 1, easing: easeOutCubic }]
- target: r3_t2
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 108, value: 0 }, { frame: 120, value: 1, easing: easeOutCubic }]
- target: r3_t3
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 108, value: 0 }, { frame: 120, value: 1, easing: easeOutCubic }]
- target: r3_t4
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 108, value: 0 }, { frame: 120, value: 1, easing: easeOutCubic }]
- target: r3_t5
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 108, value: 0 }, { frame: 120, value: 1, easing: easeOutCubic }]
- target: r3_t6
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 108, value: 0 }, { frame: 120, value: 1, easing: easeOutCubic }]
- target: r3_t7
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 108, value: 0 }, { frame: 120, value: 1, easing: easeOutCubic }]
- target: r3_t8
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 108, value: 0 }, { frame: 120, value: 1, easing: easeOutCubic }]
- target: r3_in
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 108, value: 0 }, { frame: 120, value: 1, easing: easeOutCubic }]
- target: r3_m
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 108, value: 0 }, { frame: 120, value: 1, easing: easeOutCubic }]
- target: r3_out
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 132, value: 0 }, { frame: 144, value: 1, easing: easeOutCubic }]
- target: r3_emit
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 132, value: 0 }, { frame: 144, value: 1, easing: easeOutCubic }]
- target: r4_t0
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 154, value: 0 }, { frame: 166, value: 1, easing: easeOutCubic }]
- target: r4_t1
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 154, value: 0 }, { frame: 166, value: 1, easing: easeOutCubic }]
- target: r4_t2
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 154, value: 0 }, { frame: 166, value: 1, easing: easeOutCubic }]
- target: r4_t3
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 154, value: 0 }, { frame: 166, value: 1, easing: easeOutCubic }]
- target: r4_t4
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 154, value: 0 }, { frame: 166, value: 1, easing: easeOutCubic }]
- target: r4_t5
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 154, value: 0 }, { frame: 166, value: 1, easing: easeOutCubic }]
- target: r4_t6
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 154, value: 0 }, { frame: 166, value: 1, easing: easeOutCubic }]
- target: r4_t7
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 154, value: 0 }, { frame: 166, value: 1, easing: easeOutCubic }]
- target: r4_t8
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 154, value: 0 }, { frame: 166, value: 1, easing: easeOutCubic }]
- target: r4_t9
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 154, value: 0 }, { frame: 166, value: 1, easing: easeOutCubic }]
- target: r4_in
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 154, value: 0 }, { frame: 166, value: 1, easing: easeOutCubic }]
- target: r4_m
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 154, value: 0 }, { frame: 166, value: 1, easing: easeOutCubic }]
- target: r4_out
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 178, value: 0 }, { frame: 190, value: 1, easing: easeOutCubic }]
- target: r4_emit
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 178, value: 0 }, { frame: 190, value: 1, easing: easeOutCubic }]
- target: r5_t0
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 200, value: 0 }, { frame: 212, value: 1, easing: easeOutCubic }]
- target: r5_t1
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 200, value: 0 }, { frame: 212, value: 1, easing: easeOutCubic }]
- target: r5_t2
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 200, value: 0 }, { frame: 212, value: 1, easing: easeOutCubic }]
- target: r5_t3
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 200, value: 0 }, { frame: 212, value: 1, easing: easeOutCubic }]
- target: r5_t4
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 200, value: 0 }, { frame: 212, value: 1, easing: easeOutCubic }]
- target: r5_t5
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 200, value: 0 }, { frame: 212, value: 1, easing: easeOutCubic }]
- target: r5_t6
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 200, value: 0 }, { frame: 212, value: 1, easing: easeOutCubic }]
- target: r5_t7
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 200, value: 0 }, { frame: 212, value: 1, easing: easeOutCubic }]
- target: r5_t8
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 200, value: 0 }, { frame: 212, value: 1, easing: easeOutCubic }]
- target: r5_t9
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 200, value: 0 }, { frame: 212, value: 1, easing: easeOutCubic }]
- target: r5_t10
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 200, value: 0 }, { frame: 212, value: 1, easing: easeOutCubic }]
- target: r5_in
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 200, value: 0 }, { frame: 212, value: 1, easing: easeOutCubic }]
- target: r5_m
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 200, value: 0 }, { frame: 212, value: 1, easing: easeOutCubic }]
- target: r5_out
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 224, value: 0 }, { frame: 236, value: 1, easing: easeOutCubic }]
- target: r5_emit
property: opacity
keyframes: [{ frame: 0, value: 0 }, { frame: 224, value: 0 }, { frame: 236, value: 1, easing: easeOutCubic }]
stateMachines:
main:
id: main
entry: play
states:
play: { timeline: intro }
transitions:
- { id: entry-play, from: entry, to: play, trigger: onStart }
- { id: play-loop, from: play, to: entry, exitTime: 1 }
defaultStateMachine: main
The generation loop. One forward pass produces one token, the token is appended, and the loop runs again. Tempura arrives as two tokens, temp and ura, the same word-piece split from the top of the post.
This is autoregression, and it is the engine under every chatbot you have ever used. The model does not compose a sentence and hand it to you. It guesses one token, glues it on, and guesses again, over and over, until it guesses a special "I'm done" token. A paragraph is just this loop run a few hundred times.
In pseudocode the whole thing is embarrassingly small:
tokens = tokenize(prompt)
while True:
logits = model(tokens) # one score per vocabulary token
probs = softmax(logits) # turn scores into probabilities
next_token = sample(probs) # pick one
if next_token == END:
break
tokens.append(next_token) # glue it on and go again
text = detokenize(tokens)
Hold onto that while loop. When we get to agents, the agentic loop will look suspiciously like this one. That is because it wraps this one. This is the inner loop that produces the tokens; the outer loop watches those tokens for a tool call, runs the tool, feeds the result back in, and lets this loop run again.
How it learned to guess
None of this works unless the guesses are good, and good guesses are the entire product of training. The idea is almost insultingly simple. Take that same next-token machinery, feed it an enormous pile of text from the internet, and at every position hide the real next token, let the model guess, and measure how wrong it was with a loss function. That last part, the nudge, is calculus. The loss is a single number that says how wrong the model is, and it depends on every one of the billions of numbers inside. Calculus hands you the slope of that error with respect to each number, which way is uphill and which way is down. A minimum is a spot where the slope flattens to zero, so you step every number a hair downhill toward it, then measure again. Why does this matter though? Well, if the loss function measures how wrong the model is, then we want to update all of the numbers inside the model so it is basically 0 wrong.3
Do that trillions of times, across a huge slice of everything humans have written, and the guesser becomes something more. A machine whose only objective was "predict the next token, and be a little less wrong each time" ends up with a working model of grammar, facts, styles, and a good deal of reasoning, all as a side effect of getting better at the guess. The intelligence is emergent. Nobody programmed the grammar; it was the cheapest way to lower the loss.
The idea is older than the hardware. Claude Shannon was already measuring how predictable English is in 1951, framing language itself as a guessing game.4 The machinery is almost as old. In 1957 a Cornell psychologist named Frank Rosenblatt described the perceptron, the simplest form of what we now call a neural network. It was a bank of adjustable numbers wired to a grid of light sensors that learned to tell patterns apart by nudging those numbers every time it guessed wrong.5 That is the same move we just walked through, the slow correction toward less error, running on a machine the size of a room.
Everything since has been that one idea in a different structure. Stack those simple units into layers, push each layer's output through a function that lets it bend, and you get a neural network.6 Keep stacking, add a trick called attention that lets each token weigh every other one, and you get the transformer from earlier in this post.7 The numbers are the same, and so is the nudge away from being wrong. Only the arrangement got larger. What finally changed was scale, a network big and complex enough, fed enough text, to play the guessing game well.
So what
Everyone wants to skip to agents, to models that browse, call APIs, write and run code, and pursue goals. But a model that browses is still, underneath, doing nothing but guessing the next token.
The whole art of agentic AI is arranging things around that guesser so that its tokens mean something to a runtime that can act. The first move in that arrangement is to stop letting the model ramble in free-form prose and start making it speak in shapes a program can trust. First plain text, then structured output, then function calls.
That evolution, from text to something that runs, is the next post.
Notes
1
The previous essay in this arc: "Scribbling Machines: GenAI and the New Industrial Revolution," which ends on the idea that language is becoming something that executes.
2
Byte-pair encoding (BPE) as subword tokenization for neural models: Rico Sennrich, Barry Haddow, and Alexandra Birch, "Neural Machine Translation of Rare Words with Subword Units" (2016), arXiv:1508.07909. Modern GPT-style models use a byte-level variant introduced with GPT-2.
3
Language models as next-token predictors trained by minimizing prediction loss: see the GPT line of work, e.g. Tom Brown et al., "Language Models are Few-Shot Learners" (2020), arXiv:2005.14165. The training objective is cross-entropy loss over the next token.
4
Claude E. Shannon, "Prediction and Entropy of Printed English" (1951), Bell System Technical Journal, an early treatment of language as next-symbol prediction. Overview via Wikipedia.
5
Frank Rosenblatt, "The Perceptron: A Perceiving and Recognizing Automaton" (1957), Cornell Aeronautical Laboratory Report 85-460-1, followed by the peer-reviewed "The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain" (1958), Psychological Review 65(6), 386–408. The Mark I Perceptron hardware was built at Cornell soon after. Overview via Wikipedia.
6
Stacking those units into trainable layers is the leap from a single perceptron to a deep network. The practical recipe arrived with David Rumelhart, Geoffrey Hinton, and Ronald Williams, "Learning representations by back-propagating errors" (1986), Nature 323, 533–536, nature.com/articles/323533a0. Backpropagation is how you get the downhill slope from the training section through every layer at once; gradient descent then takes the step.
7
Attention is the mechanism that lets every token in the sequence look at every other one and decide what matters. It is the heart of the transformer, introduced in Ashish Vaswani et al., "Attention Is All You Need" (2017), arXiv:1706.03762. I take it apart slowly later in this series.