In short
When you send a prompt, the app wraps it in instructions, and the model splits it into tokens, turns them into numbers, runs them through its layers and picks the next token, again and again until it stops. Every step is something you've already learned in this track.
Over 15 lessons, you've taken an LLM apart: tokens, embeddings, attention, training, fine-tuning, reasoning and tools. Now let's put it back together and follow one prompt all the way, from the moment you press Enter to the last word of the reply.
For the inside of the model, we'll use GPT-2 again (lessons 10 and 12), because it's small enough to run and every number is visible; the big chatbots don't show theirs. Where today's chatbots add extra steps, we'll say so. Our prompt is "Mumbai is the capital of the state of".
The whole journey at a glance
1. You press Enterlesson 15
Your message leaves your device for the company's computers.
2. The app builds the contextlessons 5, 8, 12 and 14
It writes the system message, the chat so far and your message into one long text, marked with special tokens. It may add search results or descriptions of tools.
3. Tokenslesson 5
The text is split into tokens, and each token becomes its ID number.
You get: 9 tokens, for our prompt.
4. Embeddingslessons 7 and 10
Each ID picks out a list of numbers, and the token's position is added.
You get: 9 lists of 768 numbers each, in GPT-2.
5. Transformer blockslessons 9 and 10
In each block, attention lets the tokens share information, and a small neural network works on each one. The block repeats, again and again.
You get: 12 blocks, with 144 attention heads, in GPT-2.
6. Chances for the next tokenlesson 4
The last token's numbers become a chance for every token the model knows.
You get: 50,257 chances, in GPT-2.
7. Pick one, and go round againlessons 4 and 13
One token is picked, guided by the temperature, added to the text, and everything runs again. A reasoning model writes its thinking in this loop before its answer.
8. Stop and showlesson 12
A chatbot stops when it writes its end-of-turn token. The app shows the words as they arrive.
9. You check itlessons 6 and 15
Is it true? Does it leave something out? Is it safe to use?
Before the model sees anything
When you press Enter, your message leaves your device for the company's computers, which is why lesson 15 said to think before you paste.
Then the app builds the text the model will actually read. A model only ever reads one long run of tokens, so the app writes everything into one text, marked with special tokens (lesson 12): a system message with the app's rules, the conversation so far, and your new message. It may add search results (RAG, lesson 8) or descriptions of tools the model can ask for (lesson 14). All of it has to fit in the context window (lesson 5).
GPT-2 is a base model, so it can't chat (lesson 12). We'll give it a plain sentence to continue, which is all it was trained to do.
Tokens
Mumbai is the capital of the state of
- M, ID 44
- umbai, ID 21645
- space then is, ID 318
- space then the, ID 262
- space then capital, ID 3139
- space then of, ID 286
- space then the, ID 262
- space then state, ID 1181
- space then of, ID 286
Common words get a token of their own, like " the" (ID 262) and " of" (ID 286). Each appears twice, with the same ID both times. "Mumbai" is rarer in the text GPT-2's tokenizer learned from, so it's built from two pieces, "M" and "umbai" (lesson 5).
From IDs to numbers
Each ID picks out one row from a big table of embeddings: a list of 768 numbers that GPT-2 learned during training (lesson 7). The token's position is added in (lesson 10), so the two " the" tokens start out almost the same, but not quite. Our 9 tokens are now 9 lists of 768 numbers.
Through the Transformer blocks
The 9 lists now pass through GPT-2's 12 Transformer blocks, one after another (lesson 10). In each block, attention lets every token gather information from the tokens before it, so " state" can take in that this sentence is about "capital" and "Mumbai". Then a small neural network (lesson 9) works on each token's numbers on its own.
By the last block, the numbers for the final token, " of", hold what the model needs to guess what comes next. All this work is done by GPT-2's 124 million parameters, which were set during training (lesson 11).
A chance for every token
The final numbers are turned into a score for each of the 50,257 tokens GPT-2 knows, and then into chances that add up to 100% (lesson 4). Here are GPT-2's real chances:
Mumbai is the capital of the state of ___
- ·Maharashtra64.4%
- ·Gujarat8.7%
- ·Mad8.0%
- ·Karn2.7%
- ·Tamil2.3%
- the other 50,252 tokens14.0%
A dot marks a space at the start of a token.
" Maharashtra" is the clear favourite, at 64.4%. " Mad" and " Karn" are the first pieces of longer names, Madhya Pradesh and Karnataka. The model never sees whole words, only tokens.
Pick one, and go round again
Next, one token is picked, and the temperature decides how adventurous the pick is (lesson 4). Here's what GPT-2's real chances become at the three temperatures from lesson 4:
| Next token | Low (0.5) | Normal (1) | High (1.5) |
|---|---|---|---|
| Maharashtra | 96.3% | 64.4% | 22.8% |
| Gujarat | 1.7% | 8.7% | 6.0% |
| Mad | 1.5% | 8.0% | 5.7% |
| Karn | 0.2% | 2.7% | 2.7% |
| Tamil | 0.1% | 2.3% | 2.5% |
| Everything else | 0.2% | 14.0% | 60.2% |
At a high temperature, 60.2% of the chance goes to tokens outside the top five, spread across more than 50,000 of them.
Let's take the top token every time, like a temperature of 0. GPT-2 adds " Maharashtra" to the text, and the whole thing runs again, through all 12 blocks, now with 10 tokens. Then again, and again:
Mumbai is the capital of the state of Maharashtra, which is home to the largest number of people of…
- ·Maharashtra64.4%
- ,36.1%
- ·which18.7%
- ·is37.9%
- ·home26.1%
- ·to99.6%
- ·the13.8%
- ·largest23.1%
- ·number37.3%
- ·of99.6%
- ·people3.7%
- ·of25.3%
Some picks were nearly certain: after " home", the token " to" had 99.6%. Others were close calls: " people" was the top choice with just 3.7%. That's why the same prompt can get different replies (lesson 4). At a temperature of 1, the chance of getting exactly this reply again is about 1 in 2 million.
A reasoning model runs this same loop to write its thinking first, often hidden, before it writes the answer (lesson 13).
Knowing when to stop
GPT-2 would have carried on writing. At every step, it gave its end-of-text token less than a hundredth of a percent. It's a base model, trained only to continue text. A chatbot is fine-tuned on conversations that end with an end-of-turn token (lesson 12), so once its answer is complete, that token becomes its likely next pick, and the app stops there. Meanwhile, the app shows you each piece as it's picked, which is why replies appear a few words at a time.
The last step is you
The reply starts with "Maharashtra", which is right. But GPT-2 gave Gujarat 8.7% and " Mad" 8.0%, and at a high temperature it would pick something other than Maharashtra 77.2% of the time, writing it just as smoothly. The rest of its reply, about Maharashtra being "home to the largest number of people of", is the model continuing a pattern, not checking a fact. That's lesson 6 in one example: it sounds just as sure either way, so check what matters (lesson 15).
Where the knowledge came from
Every number above came from GPT-2's parameters, and those came from training (lesson 11). GPT-2 learned from about 8 million web pages, roughly 40 GB of text, by predicting the next token over and over. That's how it learned that "Mumbai is the capital of the state of" is usually followed by "Maharashtra". Today's chatbots go through the same kind of pre-training at a vastly larger scale, then fine-tuning and feedback from people (lesson 12) to become assistants.
Where to go from here
You now know how the whole thing works, from your keyboard to the reply. Some ways to keep going:
- Watch it happen. Transformer Explainer (opens in a new tab) runs the same GPT-2 in your browser, and LLM Visualization (opens in a new tab) walks you through a small model in 3D.
- Review the big picture. 3Blue1Brown's short video (opens in a new tab) and Andrej Karpathy's one-hour talk (opens in a new tab) cover the same ground from different angles.
- Start building. Hugging Face's free LLM Course (opens in a new tab) teaches you to use and build with models in Python.
- Keep learning. Our Resources page has courses, books and tools for every level.
Key takeaways
- Before the model sees your prompt, the app wraps it with its rules, the chat so far and sometimes search results or tools, all in one long text.
- Inside the model, the text becomes tokens, then embeddings, then passes through stacked Transformer blocks of attention and small neural networks.
- At the end of each pass, the model gives a chance to every token it knows, picks one and goes round again. A chatbot stops at its end-of-turn token.
- Temperature and close calls are why the same prompt can give different replies.
- The model writes what's likely, not what it has checked. You're the last step: check what matters.
Check your understanding
Sources and further reading
- Language Models are Unsupervised Multitask Learners, paper (opens in a new tab)Paper · The 2019 GPT-2 paper. GPT-2 is the model this lesson follows, trained on about 8 million web pages.
- Transformer Explainer, tool (opens in a new tab)Tool · Runs the same GPT-2 in your browser, so you can follow this lesson's prompt yourself.
- LLM Visualization, tool (opens in a new tab)Tool · A 3D walk through every step inside a small GPT-style model, then GPT-2 and GPT-3.
- Large Language Models explained briefly, video (opens in a new tab)Video · Grant Sanderson's short video on how LLMs predict the next word, how they're trained and how chatbots are fine-tuned.
- [1hr Talk] Intro to Large Language Models, video (opens in a new tab)Video · Andrej Karpathy's one-hour talk on how LLMs are built and used, a good way to review the whole track.
- Hugging Face LLM Course, course (opens in a new tab)Course · A free course on using and building with LLMs in code, for when you're ready. It expects good Python.