Course map · Season 1 of 15
Technical season
S1 How LLMs work
What a large language model does, from tokens and attention to hallucination and reasoning.
- 13 lessons live
- 16 core ideas
Lessons, in order
-
Season 1: How LLMs work
Season introduction, to outline what the upcoming episodes will cover; What large language models (LLMs) actually do, from the first token to reasoning.
-
One token at a time
What a large language model (LLM) actually does, and why every call starts fresh.
Rule Treat every model call as stateless and probabilistic: send the context it needs, and verify what it returns.
-
Tokens
Tokens are the unit every model reads, writes, and bills in, and one sentence can need up to 9.2 times as many tokens in Amharic as in English.
Rule Budget, limit, and price in tokens, measured on your own traffic in every language you serve.
-
Embeddings
Embeddings place text as coordinates on a map of meaning, but similar is not the same as relevant, so treat your embedding model as a versioned dependency.
Rule Treat the embedding model as a versioned dependency: pin it, test it on your own queries, and re-embed when it changes.
-
Attention
Attention lets every token weigh the tokens before it, so meaning comes from context, but every extra token competes for that attention and adds cost.
Rule Send the model what it needs, not everything you have: every extra token competes for attention and adds cost.
-
The transformer
The transformer stacks attention and feed-forward layers over token embeddings, and its shape explains why writing took 22 times as long per token as reading.
Rule Choose the smallest model that passes your own tests, and budget latency on output length.
-
Context windows
The context window is one shared budget for instructions, history, documents, and the answer, and what falls past its edge is often dropped without any error.
Rule Treat the context window as a budget: count what you send, reserve room for the answer, and test at your real size.
-
Field note: lost in the middle
Across three current models and 270 calls, where a fact sat in documents up to 64,000 tokens made no difference to finding it, unlike the 2023 finding.
Rule Stop arranging long prompts by position for simple lookups. Test the harder shapes of your own task before trusting a long context.
-
Sampling
Sampling draws each token from the model's odds, so settings like temperature trade consistency for variety, and a feature must be judged by its rate across many runs.
Rule Set sampling for the task, and judge a model by its rate across many runs, never by one answer.
-
Hallucination
Fluent answers can be confidently wrong, and the only fabrication in this test came from the keyword scoring, not the model, so check error rates by reading answers.
Rule Never ship an answer you cannot trace: ground it in a source, let the model say it does not know, and measure the error rate by reading, not just by keyword.
-
Field note: reasoning versus standard
Across three providers and 720 calls, high reasoning gained about 20 points on multi-step tasks and nothing on lookups or simple sums, mostly from one model.
Rule Turn reasoning on for problems that need several steps held in mind at once. Leave it off for lookups and simple sums, and pilot before you commit.
-
When is a reasoning model worth the cost?
Reasoning pays only on tasks that hold several steps in mind at once, so keep it low by default and prove each increase with a paired test.
Rule Keep reasoning at its lowest setting by default. Turn it up per task, only where a paired test on your own items shows it earns its tokens and seconds.
-
Season 1 recap
Eleven episodes on how LLMs work in one place: the numbers that surprised, the eleven rules worth keeping, and one rule for the whole season.
Rule Send the model what it needs, verify what it returns, and trust a measured rate over any single answer.