What Is an LLM? Explained for Designers
An LLM is a language model trained on huge amounts of text to predict the next token. Here is how it works and what it means for the people who design with it.
What is an LLM, in one sentence?
An LLM is a system that, given a text, calculates the probability of each possible next piece of text and picks one. It then repeats the process with the extended text. It does not look the answer up in a database or an internal encyclopedia: it generates it step by step.
Those pieces are called tokens: sometimes a whole word, sometimes a fragment of a word or a punctuation mark. Models work with numbers, so the text is first converted into tokens (more in tokens).
If this sounds like very advanced autocomplete, you are on the right track. The difference is scale: the model is trained on far more text than a person could read in several lifetimes, and it has billions of parameters, the numerical values adjusted during training. IBM notes that the largest models can reach trillions.
How does an LLM learn?
Training has two main stages.
- Pretraining. The model reads text and learns to predict the next token. Each time it misses, its parameters are nudged. With enough examples, it absorbs grammar, facts about the world, styles and patterns of reasoning. Because nobody has to label the data, this is called self-supervised learning.
- Post-training. A freshly pretrained model is mostly a text predictor. To make it follow instructions and behave like an assistant, it is tuned with techniques such as instruction tuning (learning from examples of instructions with good answers) and reinforcement learning from human feedback (RLHF), where people rank answers and the model learns to prefer the higher-ranked ones.
One fact that helps you avoid obsessing over size: in the paper that introduced InstructGPT (OpenAI, 2022), answers from a 1.3-billion-parameter model tuned with human feedback were preferred over those of GPT-3, with 175 billion. More parameters do not guarantee a better experience; see small vs. large models.
The architecture that made this possible is called the transformer, introduced in 2017 in the paper “Attention Is All You Need”. Its key mechanism, attention, lets the model weigh how much each part of the text affects the interpretation of the others. In Google’s classic example, “The animal didn’t cross the street because it was too tired”, attention helps link “it” to “animal” rather than to “street”. If the sentence ended in “too wide”, the link would shift to “street”.
How does it generate an answer?
Every time the model writes, it does three things:
- It reads all the text in front of it (your message, the product’s instructions, the history).
- It calculates a probability for each possible next token.
- It picks one, appends it to the text and goes back to step 1.
Always picking “the most probable” token would produce repetitive text, so the product can leave some randomness in the choice. The settings that control that randomness are explained in temperature and top-p.
Two details that affect design. First, the model only sees what fits in its context window; whatever falls outside does not exist for it. Second, it remembers nothing by itself. The Claude API documentation puts it this way: the Messages API is stateless, which is why you always send the full conversation history. A chat’s “memory” is the product resending that history on every turn.
What does an LLM do well and what does it do badly?
| Task | What to expect | What to do in the design |
|---|---|---|
| Summarize, rewrite, change tone | Usually goes well | Let people edit the result |
| Classify or extract data from free text | Fine, with doubtful cases | Show the chosen category and let people correct it |
| Draft text and variants | Usually goes well | Present it as a draft |
| Exact data (figures, dates, quotes) | Risk of error | Show the source or connect to real data |
| Calculations and business rules | Risk of error | Use deterministic code |
| Facts after its training | It does not know them | State the knowledge cutoff or supply the data |
When an LLM states something false with confidence, we talk about a hallucination: according to Wikipedia, it asserts claims of fact that do not seem justified by its training data. It is a consequence of how it works, not an isolated bug; see hallucinations.
Why does this matter to a designer?
Because the LLM shapes the experience even when you do not see it. These are the practical consequences:
- It generates, it does not retrieve. An answer can sound confident and still be wrong. Your interface needs a way to show sources or let people verify.
- It works in tokens. What fits in a conversation is measured in tokens. If the content is long, the model can lose parts of it.
- It responds with latency. Because it writes token by token, the perceived speed depends on how you display the text while it is being generated.
- It does not know what happened after training. It has a knowledge cutoff, and it does not see your data unless you provide it.
- It can answer the same question differently. Because of the randomness in the choice, do not expect two identical answers.
When NOT to use an LLM
An LLM is the wrong tool when you need an exact, repeatable result: a calculation, a database query, a form validation or a business rule. For those, use deterministic code and leave the model the parts that involve language, fuzzy classification or synthesis. If you mix both without separating them, errors become hard to detect.
Example
Illustrative example: a product team wants a notes app to summarize meetings. Before designing the “Summarize” button, the designer asks three questions.
- What happens if the transcript is 40 pages long? It may not fit in the context window.
- What happens if the model invents an agreement nobody made? It generates plausible text, not verified text.
- How does the person see where each sentence of the summary came from? A model that predicts text has no internal notion of “this is in the transcript”.
All three follow directly from how an LLM works. The design answer: each sentence of the summary links to the original passage in the transcript, and agreements appear marked “to confirm” until someone accepts them.
Common mistakes
- Treating the LLM as a database. Remember that it generates; it does not always retrieve.
- Assuming a fluent answer is a correct answer.
- Believing more parameters always mean a better answer for your case.
- Thinking the model remembers earlier conversations. It only sees what the product resends to it.
- Using an LLM for tasks that code solves exactly, such as adding up an invoice.
- Confusing the model with the product: what “ChatGPT” or “Claude” does includes tools and rules that do not come from the model alone.
Sources
- Large language model (Wikipedia): definition, pretraining, post-training, tokenization and hallucinations.
- What are large language models? (IBM): parameters, self-supervised learning, RLHF and transformers.
- LLMs: What’s a large language model? (Google, Machine Learning Crash Course): transformers, self-attention and the animal-and-street example.
- Attention Is All You Need (Vaswani et al., 2017): the paper that introduced the transformer.
- Training language models to follow instructions with human feedback (Ouyang et al., 2022): RLHF and the comparison between 1.3 billion and 175 billion parameters.
- Using the Messages API (Claude documentation): the API is stateless and the full history is sent.