What a language model actually does

What you will learn One layer deeper into "a program that guesses what word comes next" from what AI is. Knowing this lets you predict what it will be good at and what it will not.

It really only does one thing

What a language model does is literally this one thing:

Look at the text so far and pick the next fragment.

And then repeat.

You have probably seen an answer appear one character at a time. That is not a visual effect — it is genuinely being produced that way.

So how does that become summarising and translating?

This is the key part. If it only guesses the next word, how does it summarise?

You are not commanding it to summarise; you are creating a situation where a summary is the only thing that can come next.

In that state, what comes next? In human writing, what follows "Summary of the above:" is a summary. The model has seen that pattern countless times.

Translation is the same:

The natural continuation after "English:" is a translation.

So it is less "it understood the instruction" and more "it produced the natural continuation for that situation." The results are similar, but this distinction explains the limits.

What follows from this structure

1. What comes before is everything

The model picks the next fragment based only on the text it is looking at. So:

• Describe the situation in detail → you get a continuation fitted to it
• Write nothing → you get the most generic continuation

That is why getting the answer you wanted says to give the situation. It is not a trick — it follows directly from how it works.

2. The same question gives different answers

We said it "picks the next fragment," but there are several candidates and it picks probabilistically.

Always picking first place makes writing stilted, so a little randomness is built in. That is why asking the same question twice gives slightly different answers.

This is design, not malfunction. It also means it is unsuited to work requiring identical results (same input must always give the same output).

3. It is bad at arithmetic

Ask for 237 × 481 and it does not calculate — it produces a number that looks like a plausible answer. That is where "right number of digits, wrong value" comes from.

So in practice you make a calculator do the calculating. Rather than the model doing it, you have it call a tool — the subject of the next chapters.

What it is good and bad at

Knowing the structure makes this predictable.

All four in the second table are solved with "tools." Attach a calculator, attach search, attach the internal database. That is tool use.

Common misconceptions

"The model understands my question"

The word "understand" causes confusion. From the results it looks like understanding; in reality it produced the most natural continuation in that context.

This distinction matters in practice because natural and correct are not the same thing. A plausible but non-existent book title is the example — the subject of the next chapter.

"Won't a bigger model solve everything?"

Size increases the richness of patterns. But it does not make the model know facts it never saw. No model, however large, knows your company's revenue. That is not a size problem but a connection problem.

---

Check yourself

1. How does "summarise this" lead to a summary?

Answer

Because the natural continuation after "Summary of the above:" is a summary. The model has seen that pattern countless times in human writing. It is less understanding and executing an instruction, and more producing what naturally comes next in that situation.

2. Why does the same question give slightly different answers each time?

Answer

Because the next fragment is picked probabilistically. Always picking first place makes writing stilted, so some randomness is built in. That is design, not malfunction — but it is unsuited to work that needs identical output for identical input.

3. What is the common fix for the four things models are bad at?

Answer

Attaching tools. Arithmetic to a calculator, current information to search, internal information to a database. Making the model bigger does not fix it.

---

Next, the problem that follows inevitably from this structure → Why it is wrong so plausibly