Models Don't Have Agency. Systems Do.
A language model only ever emits tokens. Everything we call agentic is a story about shaping those tokens into intent a runtime can act on. One orderly way to see it runs from a word to a structure to a function. The model never gains agency. The system we build around it does.
Ask a language model to book you a flight and it might say, Sure, I can help with that. It sounds like you want to fly to Tokyo next Tuesday, probably in the afternoon. A person reads that and knows exactly what to do. A program reads it and chokes. There is no reliable way to pull a destination, a date, and a time out of a friendly sentence. The words are all there. The shape is not.
That gap, between a sentence a person can act on and tokens a program can act on, is where this post lives. Closing that gap is the move that hands a system agency, and the model never gets any.
The last post1 left the language model as a next word guesser and nothing more. Now we make that guesser useful to software by shaping what it is allowed to say, not by making it smarter.
A model only ever emits tokens. Everything that follows is a story about people learning to shape those tokens into intent a program can act on. One orderly way to see it runs from a word to a structure to a function. The tokens never gain agency. The runtime that acts on them does.
What agency actually means
Agency has a real definition, and it comes from the philosophy of action. An agent is a being with the capacity to act, and an action is a behavior brought about by intention.2 Hold a language model against that definition and it comes up empty. It produces tokens. It originates no intent and it performs no action. What it can do is take a person's intent, put it into a shape, and pass that shape to something that can act.
So the useful question is not how to give the model agency. It is how to shape its output so that a runtime can read the intent more reliably. A clear way to tell that story is in three stages, each asking the model for a little more shape. The stages are an ordering for explanation, not the only path a real system can take.
First, a word
The smallest useful shape is a single word. We used it early on, and it worked. In my group we even had a name for it, intent mapping, sorting a customer message into sales or support so the app could branch on it. The same move works for a flight. The traveler types a request and the assistant has to route it before it can do anything else, so you ask the model a narrow question, Is this a booking, a cancellation, or a question?, and require one word back. It answers BOOK.
On its own that word does nothing. It is a token like any other. Then the runtime does the one thing that matters. It branches. if reply == "BOOK" sends the conversation down the booking path and CANCEL down another. That branch is the whole event. The model expressed a single bit of intent, and the runtime brought about a behavior because of it. A behavior brought about by intention is an action, and that is what makes the system an agent. The branch is what acted, not the model.
version: "2.0"
scene:
type: player
fps: 30
width: 880
height: 244
background: transparent
children: [runtimeBox, glyph, gl1, gl2, gl3, inT1, inT2, inArrow, hex, hexLabel, moArrow, outText, boundLab, diamond, diaLabel, cUp, cDown, bkPill, bkText, cnPill, cnText, inBound]
elements:
glyph:
id: glyph
type: rect
props: { type: rect, x: 60, y: 94, width: 44, height: 36, rx: 6, fill: "$surface", stroke: "$foreground", strokeWidth: 2 }
gl1:
id: gl1
type: line
props: { type: line, x1: 70, y1: 106, x2: 94, y2: 106, stroke: "$foreground", strokeWidth: 2 }
gl2:
id: gl2
type: line
props: { type: line, x1: 70, y1: 114, x2: 98, y2: 114, stroke: "$foreground", strokeWidth: 2 }
gl3:
id: gl3
type: line
props: { type: line, x1: 70, y1: 122, x2: 88, y2: 122, stroke: "$foreground", strokeWidth: 2 }
inT1:
id: inT1
type: text
props: { type: text, x: 82, y: 154, content: "Instructions", fill: "$foreground", fontSize: 12, textAnchor: middle, fontWeight: 600 }
inT2:
id: inT2
type: text
props: { type: text, x: 82, y: 172, content: "Message", fill: "$foreground", fontSize: 12, textAnchor: middle, fontWeight: 600 }
inArrow:
id: inArrow
type: line
props: { type: line, x1: 116, y1: 130, x2: 190, y2: 130, stroke: "$foreground", strokeWidth: 2.5, endCap: arrow }
hex:
id: hex
type: polygon
props:
type: polygon
points: [[198,130],[217,96],[255,96],[274,130],[255,164],[217,164]]
fill: "$surface"
stroke: "$foreground"
strokeWidth: 2.5
hexLabel:
id: hexLabel
type: text
props: { type: text, x: 236, y: 135, content: "model", fill: "$foreground", fontSize: 12, textAnchor: middle, fontWeight: 700 }
moArrow:
id: moArrow
type: line
props: { type: line, x1: 284, y1: 130, x2: 326, y2: 130, stroke: "$foreground", strokeWidth: 2.5, endCap: arrow }
outText:
id: outText
type: text
props: { type: text, x: 373, y: 136, content: "\"BOOK\"", fill: "$foreground", fontSize: 17, textAnchor: middle, fontWeight: 700, fontFamily: "monospace" }
runtimeBox:
id: runtimeBox
type: rect
props: { type: rect, x: 19, y: 32, width: 841, height: 196, rx: 18, fill: "transparent", stroke: "$accent", strokeWidth: 1.5 }
boundLab:
id: boundLab
type: text
props: { type: text, x: 19, y: 22, content: "runtime", fill: "$accent", fontSize: 14, textAnchor: start, fontWeight: 600 }
inBound:
id: inBound
type: line
props: { type: line, x1: 412, y1: 130, x2: 530, y2: 130, stroke: "$foreground", strokeWidth: 2.5, endCap: arrow }
diamond:
id: diamond
type: polygon
props:
type: polygon
points: [[576,98],[616,130],[576,162],[536,130]]
fill: "$surface"
stroke: "$accent"
strokeWidth: 2
diaLabel:
id: diaLabel
type: text
props: { type: text, x: 576, y: 134, content: "reply?", fill: "$foreground", fontSize: 11, textAnchor: middle }
cUp:
id: cUp
type: line
props: { type: line, x1: 616, y1: 118, x2: 656, y2: 86, stroke: "$muted", strokeWidth: 2, endCap: arrow }
cDown:
id: cDown
type: line
props: { type: line, x1: 616, y1: 142, x2: 656, y2: 174, stroke: "$muted", strokeWidth: 2, endCap: arrow }
bkPill:
id: bkPill
type: rect
props: { type: rect, x: 660, y: 64, width: 168, height: 44, rx: 22, fill: "$surface", stroke: "$muted", strokeWidth: 2 }
bkText:
id: bkText
type: text
props: { type: text, x: 744, y: 92, content: "booking path", fill: "$foreground", fontSize: 14, textAnchor: middle, fontWeight: 600 }
cnPill:
id: cnPill
type: rect
props: { type: rect, x: 660, y: 152, width: 168, height: 44, rx: 22, fill: "$surface", stroke: "$muted", strokeWidth: 2 }
cnText:
id: cnText
type: text
props: { type: text, x: 744, y: 180, content: "cancel path", fill: "$foreground", fontSize: 14, textAnchor: middle, fontWeight: 600 }
timelines:
decide:
id: decide
duration: 400
tracks:
- target: cUp
property: stroke
keyframes:
- { frame: 0, value: "$muted" }
- { frame: 24, value: "$muted", easing: linear }
- { frame: 48, value: "$accent", easing: easeOutCubic }
- target: bkPill
property: stroke
keyframes:
- { frame: 0, value: "$muted" }
- { frame: 24, value: "$muted", easing: linear }
- { frame: 48, value: "$accent", easing: easeOutCubic }
- target: bkText
property: fill
keyframes:
- { frame: 0, value: "$foreground" }
- { frame: 24, value: "$foreground", easing: linear }
- { frame: 48, value: "$accent", easing: easeOutCubic }
- target: cnPill
property: opacity
keyframes:
- { frame: 0, value: 1 }
- { frame: 24, value: 1, easing: linear }
- { frame: 48, value: 0.35, easing: easeOutCubic }
- target: cnText
property: opacity
keyframes:
- { frame: 0, value: 1 }
- { frame: 24, value: 1, easing: linear }
- { frame: 48, value: 0.35, easing: easeOutCubic }
- target: cDown
property: opacity
keyframes:
- { frame: 0, value: 1 }
- { frame: 24, value: 1, easing: linear }
- { frame: 48, value: 0.35, easing: easeOutCubic }
stateMachines:
main:
id: main
entry: play
states:
play: { timeline: decide }
transitions:
- { id: entry-play, from: entry, to: play, trigger: onStart }
- { id: play-loop, from: play, to: entry, exitTime: 1 }
defaultStateMachine: main
One word is also thin, and it is brittle. The day the model answers Looks like a booking, though they also want to change a seat, your equality check quietly fails. And one word cannot carry a destination or a date. If you want the runtime to actually book the flight, you have to ask the model for more than a word.
Then, a structure
So you ask for more shape. To book the flight the runtime needs a destination, a date, and a time, so instead of one word you hand the model a schema, a description of the exact structure you want back, and require the answer to fit it.
{ "destination": "Tokyo", "date": "2026-09-08", "time": "15:00" }
Now the runtime is not branching on a single token. It can read fields, check them, combine them, and route on several at once. More intent lives in the tokens, so there is more the system can do with them.
version: "2.0"
scene:
type: player
fps: 30
width: 924
height: 240
background: transparent
children: [boundary, glyph, gl1, gl2, gl3, inT1, inT2, inArrow, hex, hexLabel, moArrow, oj1, oj2, oj3, inBound, boundLab, fld1, fld1t, fld2, fld2t, fld3, fld3t, a1, a2, a3, act, actt]
elements:
glyph:
id: glyph
type: rect
props: { type: rect, x: 60, y: 96, width: 44, height: 36, rx: 6, fill: "$surface", stroke: "$foreground", strokeWidth: 2 }
gl1:
id: gl1
type: line
props: { type: line, x1: 70, y1: 108, x2: 94, y2: 108, stroke: "$foreground", strokeWidth: 2 }
gl2:
id: gl2
type: line
props: { type: line, x1: 70, y1: 116, x2: 98, y2: 116, stroke: "$foreground", strokeWidth: 2 }
gl3:
id: gl3
type: line
props: { type: line, x1: 70, y1: 124, x2: 88, y2: 124, stroke: "$foreground", strokeWidth: 2 }
inT1:
id: inT1
type: text
props: { type: text, x: 82, y: 156, content: "Instructions", fill: "$foreground", fontSize: 12, textAnchor: middle, fontWeight: 600 }
inT2:
id: inT2
type: text
props: { type: text, x: 82, y: 174, content: "Message", fill: "$foreground", fontSize: 12, textAnchor: middle, fontWeight: 600 }
inArrow:
id: inArrow
type: line
props: { type: line, x1: 116, y1: 132, x2: 192, y2: 132, stroke: "$foreground", strokeWidth: 2.5, endCap: arrow }
hex:
id: hex
type: polygon
props:
type: polygon
points: [[198,132],[217,98],[255,98],[274,132],[255,166],[217,166]]
fill: "$surface"
stroke: "$foreground"
strokeWidth: 2.5
hexLabel:
id: hexLabel
type: text
props: { type: text, x: 236, y: 137, content: "model", fill: "$foreground", fontSize: 12, textAnchor: middle, fontWeight: 700 }
moArrow:
id: moArrow
type: line
props: { type: line, x1: 278, y1: 132, x2: 318, y2: 132, stroke: "$foreground", strokeWidth: 2.5, endCap: arrow }
oj1:
id: oj1
type: text
props: { type: text, x: 320, y: 112, content: "{ \"destination\": \"Tokyo\",", fill: "$foreground", fontSize: 11, textAnchor: start, fontFamily: "monospace" }
oj2:
id: oj2
type: text
props: { type: text, x: 334, y: 132, content: "\"date\": \"2026-09-08\",", fill: "$foreground", fontSize: 11, textAnchor: start, fontFamily: "monospace" }
oj3:
id: oj3
type: text
props: { type: text, x: 334, y: 152, content: "\"time\": \"15:00\" }", fill: "$foreground", fontSize: 11, textAnchor: start, fontFamily: "monospace" }
inBound:
id: inBound
type: line
props: { type: line, x1: 504, y1: 132, x2: 554, y2: 132, stroke: "$foreground", strokeWidth: 2.5, endCap: arrow }
boundary:
id: boundary
type: rect
props: { type: rect, x: 19, y: 34, width: 887, height: 190, rx: 18, fill: "transparent", stroke: "$accent", strokeWidth: 1.5 }
boundLab:
id: boundLab
type: text
props: { type: text, x: 19, y: 24, content: "runtime", fill: "$accent", fontSize: 14, textAnchor: start, fontWeight: 600 }
fld1:
id: fld1
type: rect
props: { type: rect, x: 560, y: 66, width: 150, height: 34, rx: 6, fill: "$surface", stroke: "$muted", strokeWidth: 2 }
fld1t:
id: fld1t
type: text
props: { type: text, x: 635, y: 88, content: "destination", fill: "$foreground", fontSize: 12, textAnchor: middle }
fld2:
id: fld2
type: rect
props: { type: rect, x: 560, y: 112, width: 150, height: 34, rx: 6, fill: "$surface", stroke: "$muted", strokeWidth: 2 }
fld2t:
id: fld2t
type: text
props: { type: text, x: 635, y: 134, content: "date", fill: "$foreground", fontSize: 12, textAnchor: middle }
fld3:
id: fld3
type: rect
props: { type: rect, x: 560, y: 158, width: 150, height: 34, rx: 6, fill: "$surface", stroke: "$muted", strokeWidth: 2 }
fld3t:
id: fld3t
type: text
props: { type: text, x: 635, y: 180, content: "time", fill: "$foreground", fontSize: 12, textAnchor: middle }
a1:
id: a1
type: line
props: { type: line, x1: 712, y1: 83, x2: 768, y2: 118, stroke: "$muted", strokeWidth: 2, endCap: arrow }
a2:
id: a2
type: line
props: { type: line, x1: 712, y1: 129, x2: 768, y2: 132, stroke: "$muted", strokeWidth: 2, endCap: arrow }
a3:
id: a3
type: line
props: { type: line, x1: 712, y1: 175, x2: 768, y2: 146, stroke: "$muted", strokeWidth: 2, endCap: arrow }
act:
id: act
type: rect
props: { type: rect, x: 770, y: 104, width: 104, height: 56, rx: 10, fill: "$surface", stroke: "$accent", strokeWidth: 2 }
actt:
id: actt
type: text
props: { type: text, x: 822, y: 137, content: "book(…)", fill: "$foreground", fontSize: 14, textAnchor: middle, fontWeight: 600 }
timelines:
read:
id: read
duration: 440
tracks:
- target: fld1
property: stroke
keyframes:
- { frame: 0, value: "$muted" }
- { frame: 6, value: "$muted", easing: linear }
- { frame: 18, value: "$accent", easing: easeOutCubic }
- target: a1
property: stroke
keyframes:
- { frame: 0, value: "$muted" }
- { frame: 12, value: "$muted", easing: linear }
- { frame: 24, value: "$accent", easing: easeOutCubic }
- target: fld2
property: stroke
keyframes:
- { frame: 0, value: "$muted" }
- { frame: 18, value: "$muted", easing: linear }
- { frame: 30, value: "$accent", easing: easeOutCubic }
- target: a2
property: stroke
keyframes:
- { frame: 0, value: "$muted" }
- { frame: 24, value: "$muted", easing: linear }
- { frame: 36, value: "$accent", easing: easeOutCubic }
- target: fld3
property: stroke
keyframes:
- { frame: 0, value: "$muted" }
- { frame: 30, value: "$muted", easing: linear }
- { frame: 42, value: "$accent", easing: easeOutCubic }
- target: a3
property: stroke
keyframes:
- { frame: 0, value: "$muted" }
- { frame: 36, value: "$muted", easing: linear }
- { frame: 48, value: "$accent", easing: easeOutCubic }
- target: actt
property: fill
keyframes:
- { frame: 0, value: "$foreground" }
- { frame: 48, value: "$foreground", easing: linear }
- { frame: 62, value: "$accent", easing: easeOutCubic }
stateMachines:
main:
id: main
entry: play
states:
play: { timeline: read }
transitions:
- { id: entry-play, from: entry, to: play, trigger: onStart }
- { id: play-loop, from: play, to: entry, exitTime: 1 }
defaultStateMachine: main
At first you got this shape by asking nicely in the prompt and hoping. Then the providers made it a guarantee. OpenAI shipped a feature called Structured Outputs that takes a JSON Schema and holds the model to it, so every response is valid against the schema you handed in.3 It works by constraining generation itself. As the model samples each token, anything that would break the schema is ruled out before it can be chosen,4 so the output cannot drift off the structure even when the model would rather ramble. This is stronger than the older JSON mode, which promised valid JSON but not your particular shape.
The guess is still a guess. The model has fenced it into a structure a program can consume.
Then, a function
Push the shape one step further, though it is really the same shape. A function call is just the structured output from the last section, its fields arranged so they read as intent to run a specific function. Someone decided that if a structure is going to drive an action anyway, it may as well look like the action, a name and a set of arguments. So along with the schema you give the model the name of a tool it may use, book_flight, and what its arguments look like. When the model decides that tool fits the conversation, the shape it emits comes down to this.
{ "name": "book_flight", "arguments": { "destination": "Tokyo", "date": "2026-09-08", "time": "15:00" } }
version: "2.0"
scene:
type: player
fps: 30
width: 852
height: 190
background: transparent
children: [boundary, glyph, gl1, gl2, gl3, inT1, inT2, inArrow, hex, hexLabel, moArrow, fj1, fj2, fj3, fj4, fj5, inBound, boundLab, exec, ex1, ex2]
elements:
glyph:
id: glyph
type: rect
props: { type: rect, x: 60, y: 64, width: 44, height: 36, rx: 6, fill: "$surface", stroke: "$foreground", strokeWidth: 2 }
gl1:
id: gl1
type: line
props: { type: line, x1: 70, y1: 76, x2: 94, y2: 76, stroke: "$foreground", strokeWidth: 2 }
gl2:
id: gl2
type: line
props: { type: line, x1: 70, y1: 84, x2: 98, y2: 84, stroke: "$foreground", strokeWidth: 2 }
gl3:
id: gl3
type: line
props: { type: line, x1: 70, y1: 92, x2: 88, y2: 92, stroke: "$foreground", strokeWidth: 2 }
inT1:
id: inT1
type: text
props: { type: text, x: 82, y: 124, content: "Instructions", fill: "$foreground", fontSize: 12, textAnchor: middle, fontWeight: 600 }
inT2:
id: inT2
type: text
props: { type: text, x: 82, y: 142, content: "Message", fill: "$foreground", fontSize: 12, textAnchor: middle, fontWeight: 600 }
inArrow:
id: inArrow
type: line
props: { type: line, x1: 116, y1: 100, x2: 192, y2: 100, stroke: "$foreground", strokeWidth: 2.5, endCap: arrow }
hex:
id: hex
type: polygon
props:
type: polygon
points: [[198,100],[217,66],[255,66],[274,100],[255,134],[217,134]]
fill: "$surface"
stroke: "$foreground"
strokeWidth: 2.5
hexLabel:
id: hexLabel
type: text
props: { type: text, x: 236, y: 105, content: "model", fill: "$foreground", fontSize: 12, textAnchor: middle, fontWeight: 700 }
moArrow:
id: moArrow
type: line
props: { type: line, x1: 278, y1: 100, x2: 320, y2: 100, stroke: "$foreground", strokeWidth: 2.5, endCap: arrow }
fj1:
id: fj1
type: text
props: { type: text, x: 320, y: 66, content: "{ \"name\": \"book_flight\",", fill: "$foreground", fontSize: 11, textAnchor: start, fontFamily: "monospace" }
fj2:
id: fj2
type: text
props: { type: text, x: 334, y: 84, content: "\"arguments\": {", fill: "$foreground", fontSize: 11, textAnchor: start, fontFamily: "monospace" }
fj3:
id: fj3
type: text
props: { type: text, x: 348, y: 102, content: "\"destination\": \"Tokyo\",", fill: "$foreground", fontSize: 11, textAnchor: start, fontFamily: "monospace" }
fj4:
id: fj4
type: text
props: { type: text, x: 348, y: 120, content: "\"date\": \"2026-09-08\",", fill: "$foreground", fontSize: 11, textAnchor: start, fontFamily: "monospace" }
fj5:
id: fj5
type: text
props: { type: text, x: 348, y: 138, content: "\"time\": \"15:00\" } }", fill: "$foreground", fontSize: 11, textAnchor: start, fontFamily: "monospace" }
inBound:
id: inBound
type: line
props: { type: line, x1: 512, y1: 100, x2: 576, y2: 100, stroke: "$foreground", strokeWidth: 2.5, endCap: arrow }
boundary:
id: boundary
type: rect
props: { type: rect, x: 19, y: 32, width: 817, height: 142, rx: 18, fill: "transparent", stroke: "$accent", strokeWidth: 1.5 }
boundLab:
id: boundLab
type: text
props: { type: text, x: 19, y: 22, content: "runtime", fill: "$accent", fontSize: 14, textAnchor: start, fontWeight: 600 }
exec:
id: exec
type: rect
props: { type: rect, x: 580, y: 70, width: 224, height: 66, rx: 10, fill: "$surface", stroke: "$accent", strokeWidth: 2 }
ex1:
id: ex1
type: text
props: { type: text, x: 692, y: 98, content: "book_flight()", fill: "$foreground", fontSize: 15, textAnchor: middle, fontWeight: 700 }
ex2:
id: ex2
type: text
props: { type: text, x: 692, y: 118, content: "runs the code", fill: "$muted", fontSize: 11, textAnchor: middle }
timelines:
dispatch:
id: dispatch
duration: 480
tracks:
- target: inBound
property: stroke
keyframes:
- { frame: 0, value: "$foreground" }
- { frame: 16, value: "$foreground", easing: linear }
- { frame: 34, value: "$accent", easing: easeOutCubic }
- target: exec
property: stroke
keyframes:
- { frame: 0, value: "$muted" }
- { frame: 30, value: "$muted", easing: linear }
- { frame: 52, value: "$accent", easing: easeOutCubic }
- target: ex1
property: fill
keyframes:
- { frame: 0, value: "$foreground" }
- { frame: 30, value: "$foreground", easing: linear }
- { frame: 52, value: "$accent", easing: easeOutCubic }
stateMachines:
main:
id: main
entry: play
states:
play: { timeline: dispatch }
transitions:
- { id: entry-play, from: entry, to: play, trigger: onStart }
- { id: play-loop, from: play, to: entry, exitTime: 1 }
defaultStateMachine: main
This did not arrive all at once. Research came first. Toolformer showed a model could be trained to decide when an API call would help and to emit the call itself.5 Then the providers turned it into an interface. OpenAI added function calling in June 2023, where you describe a function and the model returns a separate field naming it, with the arguments packed into a JSON string.6 The idea spread fast. Anthropic shipped tool use for Claude the next year, and the envelope is shaped a little differently. The reply comes back as a list of content blocks, and a tool call is one block of type tool_use sitting inline next to the model's own text, its arguments already a structured object rather than a string.7
Here are the two, the same booking wrapped by each provider.
OpenAI
{
"role": "assistant",
"tool_calls": [
{
"id": "call_8f2a",
"type": "function",
"function": {
"name": "book_flight",
"arguments": "{\"destination\":\"Tokyo\",\"date\":\"2026-09-08\",\"time\":\"15:00\"}"
}
}
]
}
Anthropic
{
"role": "assistant",
"content": [
{ "type": "text", "text": "Booking that now." },
{
"type": "tool_use",
"id": "toolu_01A",
"name": "book_flight",
"input": { "destination": "Tokyo", "date": "2026-09-08", "time": "15:00" }
}
]
}
Different envelope, same essence. Both are tokens that name a tool and describe its arguments. Both later grew to allow several calls in one turn, and both tightened how strictly the arguments follow the schema. Each step made the intent sharper and easier for a runtime to read.
This is about as sharp as a single call gets. The intent is now precise enough that the runtime can hand it almost straight to code. And yet the model did not book a flight. It cannot book a flight. It produced tokens that describe a flight it thinks you want booked. The output is a recommendation. Something still has to pick it up and act.
The boundary
That something is the runtime, and between the model emitting a call and the runtime running it, there is a gap.
version: "2.0"
scene:
type: player
width: 766
height: 240
background: transparent
children: [box, seam, inArrow, hex, hexLab, t1, c1, t2, c2, t3, c3, a1, a2, a3, act, aLab, aSub, seamLab, boxLab]
elements:
box:
id: box
type: rect
props: { type: rect, x: 20, y: 36, width: 726, height: 184, rx: 18, fill: "transparent", stroke: "$accent", strokeWidth: 1.5 }
boxLab:
id: boxLab
type: text
props: { type: text, x: 20, y: 26, content: "runtime", fill: "$accent", fontSize: 14, textAnchor: start, fontWeight: 600 }
hex:
id: hex
type: polygon
props:
type: polygon
points: [[52,136],[71,102],[109,102],[128,136],[109,170],[71,170]]
fill: "$surface"
stroke: "$foreground"
strokeWidth: 2.5
hexLab:
id: hexLab
type: text
props: { type: text, x: 90, y: 141, content: "model", fill: "$foreground", fontSize: 12, textAnchor: middle, fontWeight: 700 }
inArrow:
id: inArrow
type: line
props: { type: line, x1: 132, y1: 136, x2: 188, y2: 136, stroke: "$foreground", strokeWidth: 2.5, endCap: arrow }
t1:
id: t1
type: text
props: { type: text, x: 200, y: 112, content: "word", fill: "$muted", fontSize: 11, textAnchor: start }
c1:
id: c1
type: text
props: { type: text, x: 258, y: 112, content: "\"BOOK\"", fill: "$foreground", fontSize: 12, textAnchor: start, fontFamily: "monospace" }
t2:
id: t2
type: text
props: { type: text, x: 200, y: 140, content: "structure", fill: "$muted", fontSize: 11, textAnchor: start }
c2:
id: c2
type: text
props: { type: text, x: 258, y: 140, content: "{ destination, date, time }", fill: "$foreground", fontSize: 12, textAnchor: start, fontFamily: "monospace" }
t3:
id: t3
type: text
props: { type: text, x: 200, y: 168, content: "function", fill: "$muted", fontSize: 11, textAnchor: start }
c3:
id: c3
type: text
props: { type: text, x: 258, y: 168, content: "book_flight({ … })", fill: "$foreground", fontSize: 12, textAnchor: start, fontFamily: "monospace" }
a1:
id: a1
type: line
props: { type: line, x1: 452, y1: 112, x2: 540, y2: 128, stroke: "$muted", strokeWidth: 2, endCap: arrow }
a2:
id: a2
type: line
props: { type: line, x1: 452, y1: 140, x2: 540, y2: 138, stroke: "$muted", strokeWidth: 2, endCap: arrow }
a3:
id: a3
type: line
props: { type: line, x1: 452, y1: 168, x2: 540, y2: 150, stroke: "$muted", strokeWidth: 2, endCap: arrow }
seam:
id: seam
type: line
props: { type: line, x1: 470, y1: 84, x2: 470, y2: 188, stroke: "$accent", strokeWidth: 2, strokeDasharray: "6 6" }
seamLab:
id: seamLab
type: text
props: { type: text, x: 470, y: 76, content: "the boundary", fill: "$muted", fontSize: 11, textAnchor: middle }
act:
id: act
type: rect
props: { type: rect, x: 542, y: 114, width: 172, height: 52, rx: 10, fill: "$surface", stroke: "$accent", strokeWidth: 2 }
aLab:
id: aLab
type: text
props: { type: text, x: 628, y: 136, content: "book_flight()", fill: "$foreground", fontSize: 15, textAnchor: middle, fontWeight: 700 }
aSub:
id: aSub
type: text
props: { type: text, x: 628, y: 154, content: "the action happens here", fill: "$muted", fontSize: 11, textAnchor: middle }
timelines:
cross:
id: cross
duration: 480
tracks:
- target: c1
property: fill
keyframes:
- { frame: 0, value: "$foreground" }
- { frame: 4, value: "$foreground", easing: linear }
- { frame: 14, value: "$accent", easing: easeOutCubic }
- target: a1
property: stroke
keyframes:
- { frame: 0, value: "$muted" }
- { frame: 10, value: "$muted", easing: linear }
- { frame: 20, value: "$accent", easing: easeOutCubic }
- target: c2
property: fill
keyframes:
- { frame: 0, value: "$foreground" }
- { frame: 14, value: "$foreground", easing: linear }
- { frame: 24, value: "$accent", easing: easeOutCubic }
- target: a2
property: stroke
keyframes:
- { frame: 0, value: "$muted" }
- { frame: 20, value: "$muted", easing: linear }
- { frame: 30, value: "$accent", easing: easeOutCubic }
- target: c3
property: fill
keyframes:
- { frame: 0, value: "$foreground" }
- { frame: 24, value: "$foreground", easing: linear }
- { frame: 34, value: "$accent", easing: easeOutCubic }
- target: a3
property: stroke
keyframes:
- { frame: 0, value: "$muted" }
- { frame: 30, value: "$muted", easing: linear }
- { frame: 40, value: "$accent", easing: easeOutCubic }
- target: aLab
property: fill
keyframes:
- { frame: 0, value: "$foreground" }
- { frame: 40, value: "$foreground", easing: linear }
- { frame: 54, value: "$accent", easing: easeOutCubic }
stateMachines:
main:
id: main
entry: play
states:
play: { timeline: cross }
transitions:
- { id: entry-play, from: entry, to: play, trigger: onStart }
- { id: play-loop, from: play, to: entry, exitTime: 1 }
defaultStateMachine: main
Word, structure, function. Whichever one the model emits, it crosses the same boundary, and on the far side the runtime turns a proposal into an action.
Whether the tool should run at all, whether the arguments are sane, whether this user is even allowed, is decided on the runtime side of that gap. The model produced tokens; what happens to them is somebody else's call. This is the seam where safety and permissions live, and it is the same seam where agency lives.
One line, many shapes
Word, structure, function. Three examples out of many. Each one asks a little more of the model, and they are only the first points on a line that keeps going, other schemas, richer tools, loops that call the model again and again. What stays fixed underneath all of them is smaller than any single shape. The model emits tokens that carry intent, and the agency belongs to the runtime that reads them and acts. You choose the shape that fits the job. The boundary is the part that does not move.
What you have built has an old name. In complexity theory an oracle machine is an ordinary machine handed a black box that answers some hard question in a single step, a way to reason about what becomes possible once you assume a given capability for free.8 A model wired into a runtime is that idea made real, with one difference that matters. The classic oracle is always right; this one guesses. You hand it language and it hands back a shape that names an executable path. Naming is all it does. Walking the path is the runtime's call.
That is enough to build on. With nothing more than the shapes you have seen here, you can put a model to work inside your own code today. Shape the tokens, read them on your side of the line, act on what you trust.
In this post we only looked at single steps. The system reads the intent, takes a step, and stops. Feed each step into the next and it seems to act on its own, though the model is never the one acting. That loop, and the thread that runs through it, is the next post.
Notes
1
The previous post in this series: "How the Machine Guesses the Next Word," which builds a language model up from tokens to the next-token guess and its training loop.
4
Constrained or guided decoding forces generated tokens to match a formal schema or grammar: Brandon T. Willard and Rémi Louf, "Efficient Guided Generation for Large Language Models" (2023), arXiv:2307.09702. Commercial APIs expose this as JSON mode or structured outputs.
5
Models learning to emit tool calls: Timo Schick et al., "Toolformer: Language Models Can Teach Themselves to Use Tools" (2023), arXiv:2302.04761. Provider function-calling interfaces formalize the same idea as a named schema the model fills in.
2
The definition of agency is drawn from philosophy of action, not marketing: Markus Schlosser, "Agency," Stanford Encyclopedia of Philosophy (2019), plato.stanford.edu/entries/agency. An agent is a being with the capacity to act, and an action is a behavior brought about by intention. A token stream satisfies neither clause.
3
Structured Outputs was introduced by OpenAI in August 2024 to guarantee that a model response conforms to a supplied JSON Schema: OpenAI, "Introducing Structured Outputs in the API" (2024). It supersedes the earlier JSON mode, which guaranteed only that the output parsed as JSON, not that it matched a specific schema.
6
Function calling was added to the OpenAI API in June 2023, letting a caller describe functions and receive a JSON object that names one and supplies its arguments: OpenAI, "Function calling and other API updates" (2023). It was later generalized to a tools interface with parallel calls, and argument adherence was tightened by Structured Outputs in 2024.
7
Tool use became generally available for the Claude 3 models on May 30, 2024: Anthropic, "Claude can now use tools" (2024). A tool is defined with a name, a description, and a full input_schema (a JSON Schema), and the model returns a tool_use content block whose input is the arguments object, which the caller runs and sends back as a tool_result block.
8
The oracle machine comes from computability and complexity theory: a Turing machine connected to a black-box "oracle" that returns the answer to some problem in a single step, used to study what is computable relative to a given capability (Turing reductions, relativized classes such as P^A and NP^A, the polynomial hierarchy). Alan Turing introduced the idea as the "oracle" (o-machine) in his 1939 paper "Systems of Logic Based on Ordinals"; for a modern overview see "Oracle machine," Wikipedia. The analogy here is deliberately loose: the theoretical oracle is assumed to answer correctly every time, whereas a language model only guesses.