LLM internals

Three self-contained pages explaining how large language models work, from the matrix arithmetic up to why your API bill is mostly cache reads. They were written for someone comfortable with vectors, dot products and optimisation who has never had reason to learn machine-learning vocabulary — so every ML term is defined where it first appears, and no mechanism is introduced before the problem it solves.

Each page has a TL;DR at the top and a glossary at the bottom. They build on each other in order, but any one works alone.

The three pages

transformers.html ~17 min · start here

How does a pile of matrices turn text into a prediction?

  • Text to tokens to integers — and why the integer means nothing on its own
  • The embedding table: how a row number becomes a vector, and why that's useful
  • Why word order has to be added separately, and how
  • The problem attention solves, stated before attention is named
  • Query, key and value built up from an ordinary dictionary lookup
  • Heads, feed-forward blocks, the residual stream, and what depth buys
  • Logits, temperature and top-p — where the randomness actually enters
live: tokeniser · embedding lookup · attention step-through · sampling controls
llms.html ~11 min

How does that become something you can hand a codebase to?

  • Pretraining: one exercise repeated ~1014 times, and why it produces more than it asks for
  • Post-training: supervised fine-tuning, reward models, RLHF and Constitutional AI, all spelled out
  • Why the model is stateless, and what a conversational turn really sends
  • The agent loop — what tool use is and is not
  • Failure modes traced back to their causes rather than listed as quirks
live: training step walkthrough · transcript replay showing cumulative token cost
cache.html ~16 min · the practical one

Why is my usage breakdown mostly cache reads, and what breaks it?

  • The two unrelated things both called "the cache", separated once and for all
  • What a request is actually made of — blocks, prefixes, and why the distinction matters
  • Why an identical opening can be reused: it follows from the causal mask
  • Breakpoints, the 20-block lookback, and the placement mistake nearly everyone makes
  • TTLs, free refreshes, and when the 5-minute cache costs more than no cache
  • Reading the three usage counters, and the one-line test for whether caching engaged at all
live: divergence-point slider · TTL timeline simulator · cost calculator with current prices

Where to start

if you turned up wanting…go to
to understand the thing properly, from the bottomtransformers, then straight on
to know why an API bill looks the way it doescache
to explain to someone why the model "forgot" somethingllms, the turn section
to work out whether the 1-hour TTL is worth buyingcache, the calculator
to know why it can't count letters or do arithmetictransformers, step 1
to know why it sometimes agrees with everything you sayllms, post-training

What is real and what is illustrative

Worth being explicit, since the interactive parts could otherwise be taken for more than they are.

Computed live, and correct: the attention arithmetic (dot products, scaling, masking, softmax, weighted sum) on a hand-picked 4×4 toy; the sinusoidal positional encoding; the sampling softmax under temperature and top-p; every figure in the cache calculator and TTL simulator.

Illustrative, and labelled as such in place: the tokeniser is a heuristic, not a learned merge table; embedding-table values are seeded random numbers; the multi-head attention patterns are hand-drawn to show what specialised heads look like, not extracted from a trained model.

Prices and thresholds in the cache page were checked against Anthropic's documentation in September 2026 and are the sort of thing that moves — the page links its sources so you can re-check before acting on any number.

where to go next