A 15-season series on applied AI, structured like a TV series: technical seasons on how AI works in production, and leadership seasons on how to decide, organise, and govern it. https://brendankellyai.github.io/ 2026-10-18T00:00:00+00:00 Brendan Kelly https://brendankellyai.github.io/episodes/s2-e2-first-rag-system/ 2026-10-18T00:00:00+00:00 2026-10-18T00:00:00+00:00

A complete RAG system with the simplest choice at every stage, run on all 520 test questions, shows most wrong answers never reached the model.

Build the simplest complete pipeline and measure it before tuning any stage. https://brendankellyai.github.io/episodes/s2-e1-why-retrieval-exists/ 2026-10-17T00:00:00+00:00 2026-10-17T00:00:00+00:00

Retrieval hands the model the right passage at the moment of the question, which fixes stale and private facts but moves the failure to search.

Use retrieval when the answer lives in your own data. Then log what it retrieved, and measure search and answer separately. https://brendankellyai.github.io/episodes/s2-e0-season-intro/ 2026-10-16T00:00:00+00:00 2026-10-16T00:00:00+00:00

Season introduction: build an assistant that answers from your own documents, then measure every stage until you know why it is right and why it is wrong.

Build the simplest complete system. Measure it. Change one stage at a time, and keep only what clears a bar set in advance. https://brendankellyai.github.io/episodes/s1-e12-recap/ 2026-10-09T00:00:00+00:00 2026-10-09T00:00:00+00:00

Eleven episodes on how LLMs work in one place: the numbers that surprised, the eleven rules worth keeping, and one rule for the whole season.

Send the model what it needs, verify what it returns, and trust a measured rate over any single answer. https://brendankellyai.github.io/episodes/s1-e11-when-is-reasoning-worth-the-cost/ 2026-10-08T00:00:00+00:00 2026-10-08T00:00:00+00:00

Reasoning pays only on tasks that hold several steps in mind at once, so keep it low by default and prove each increase with a paired test.

Keep reasoning at its lowest setting by default. Turn it up per task, only where a paired test on your own items shows it earns its tokens and seconds. https://brendankellyai.github.io/episodes/s1-e10-reasoning-vs-standard/ 2026-10-07T00:00:00+00:00 2026-10-07T00:00:00+00:00

Across three providers and 720 calls, high reasoning gained about 20 points on multi-step tasks and nothing on lookups or simple sums, mostly from one model.

Turn reasoning on for problems that need several steps held in mind at once. Leave it off for lookups and simple sums, and pilot before you commit. https://brendankellyai.github.io/episodes/s1-e9-hallucination/ 2026-10-06T00:00:00+00:00 2026-10-06T00:00:00+00:00

Fluent answers can be confidently wrong, and the only fabrication in this test came from the keyword scoring, not the model, so check error rates by reading answers.

Never ship an answer you cannot trace: ground it in a source, let the model say it does not know, and measure the error rate by reading, not just by keyword. https://brendankellyai.github.io/episodes/s1-e8-sampling/ 2026-10-05T00:00:00+00:00 2026-10-05T00:00:00+00:00

Sampling draws each token from the model's odds, so settings like temperature trade consistency for variety, and a feature must be judged by its rate across many runs.

Set sampling for the task, and judge a model by its rate across many runs, never by one answer. https://brendankellyai.github.io/episodes/s1-e7-lost-in-the-middle/ 2026-10-04T00:00:00+00:00 2026-10-04T00:00:00+00:00

Across three current models and 270 calls, where a fact sat in documents up to 64,000 tokens made no difference to finding it, unlike the 2023 finding.

Stop arranging long prompts by position for simple lookups. Test the harder shapes of your own task before trusting a long context. https://brendankellyai.github.io/episodes/s1-e6-context-windows/ 2026-10-03T00:00:00+00:00 2026-10-03T00:00:00+00:00

The context window is one shared budget for instructions, history, documents, and the answer, and what falls past its edge is often dropped without any error.

Treat the context window as a budget: count what you send, reserve room for the answer, and test at your real size. https://brendankellyai.github.io/episodes/s1-e5-the-transformer/ 2026-10-02T00:00:00+00:00 2026-10-02T00:00:00+00:00

The transformer stacks attention and feed-forward layers over token embeddings, and its shape explains why writing took 22 times as long per token as reading.

Choose the smallest model that passes your own tests, and budget latency on output length. https://brendankellyai.github.io/episodes/s1-e4-attention/ 2026-10-01T00:00:00+00:00 2026-10-01T00:00:00+00:00

Attention lets every token weigh the tokens before it, so meaning comes from context, but every extra token competes for that attention and adds cost.

Send the model what it needs, not everything you have: every extra token competes for attention and adds cost. https://brendankellyai.github.io/episodes/s1-e3-embeddings/ 2026-09-30T00:00:00+00:00 2026-09-30T00:00:00+00:00

Embeddings place text as coordinates on a map of meaning, but similar is not the same as relevant, so treat your embedding model as a versioned dependency.

Treat the embedding model as a versioned dependency: pin it, test it on your own queries, and re-embed when it changes. https://brendankellyai.github.io/episodes/s1-e2-tokens/ 2026-09-29T00:00:00+00:00 2026-09-29T00:00:00+00:00

Tokens are the unit every model reads, writes, and bills in, and one sentence can need up to 9.2 times as many tokens in Amharic as in English.

Budget, limit, and price in tokens, measured on your own traffic in every language you serve. https://brendankellyai.github.io/episodes/s1-e1-one-token-at-a-time/ 2026-09-28T00:00:00+00:00 2026-09-28T00:00:00+00:00

What a large language model (LLM) actually does, and why every call starts fresh.

Treat every model call as stateless and probabilistic: send the context it needs, and verify what it returns. https://brendankellyai.github.io/episodes/s1-e0-season-intro/ 2026-09-27T00:00:00+00:00 2026-09-27T00:00:00+00:00

Season introduction, to outline what the upcoming episodes will cover; What large language models (LLMs) actually do, from the first token to reasoning.