How Large Language Models Actually Work

Large language models feel like magic, but the core idea is surprisingly small: predict the next token.

Tokens, not words

Text is split into tokens (chunks of characters). The model turns each token into a vector and learns which tokens tend to follow which, across trillions of examples.

Attention is the trick

The transformer architecture lets every token look at every other token and decide what matters. That is how a pronoun finds the noun it refers to, even sentences later.

Training in two steps

1. Pre-training on huge amounts of text to learn language and general knowledge.
2. Fine-tuning with human feedback so the model becomes helpful instead of just fluent.

What it is not

It is not a database and it does not "look things up". It generates the most plausible continuation, which is why it can sound confident and still be wrong. Treat it as a very fast collaborator, not an oracle.