# Fine-Tuning vs. Prompting: When to Train and When to Instruct

> Prompting gives a model instructions in each request; fine-tuning adjusts its weights with your own data. When each makes sense and how to decide with evals.

*Tags: ai, fundamentals, prompting, architecture*

---


> [!info] Quick Definition
> **Prompting** means giving the model instructions and examples inside each request, without changing the model. **Fine-tuning** means continuing to train an already pretrained model on your own data, so the behavior is built into its weights.


## What is each one?

A base model already knows a lot about language. With **prompting** you state the task in the message: what role it plays, what format it must return, which examples to follow. All of that travels with every call and takes space in the [[context-window|context window]]. Changing the instruction changes the behavior immediately. Wikipedia describes this as in-context learning: the model learns temporarily from what is in the prompt, and its weights do not change.

With **fine-tuning**, you prepare a set of examples (input pairs and the desired output) and run additional training. The model's weights are adjusted, and the behavior appears without repeating the instruction in every request.

Tuning can be full, updating all parameters, or lighter. LoRA, for example, freezes the original weights and trains a few added matrices. The original LoRA paper reports that, compared with fully fine-tuning GPT-3 175B with Adam, it cuts trainable parameters by 10,000 times and the GPU memory requirement by 3 times, with equal or better quality on the models they tested (RoBERTa, DeBERTa, GPT-2 and GPT-3).

## What kinds of fine-tuning exist?

OpenAI's documentation distinguishes four, which shows that "tuning" is not a single technique:

| Method | How it works | What it is for |
|---|---|---|
| Supervised fine-tuning (SFT) | Trains on examples of correct responses | Classification, nuanced translation, content in a specific format, fixing instruction-following failures |
| Vision fine-tuning | SFT with images as input | Classifying images and fixing failures on complex prompts |
| Direct preference optimization (DPO) | You give a good and a bad response to the same prompt | Summarizing with the right focus, generating messages with the right tone and style |
| Reinforcement fine-tuning (RFT) | An expert grades responses and the better-scored reasoning is reinforced | Complex reasoning in a specialized domain (reasoning models only) |

For a product team, DPO is the closest to design work: it teaches tone and style by showing "this, not that" pairs. RFT requires experts who agree on the ideal answer, which few teams have.

## Why does this matter to a designer?

Because the choice affects the product's iteration cycle:

- **Speed.** Changing a prompt takes minutes. Fine-tuning means preparing data, training, evaluating and deploying. If the design team wants to test different voice tones, prompting lets them iterate much faster.
- **Format consistency.** If the interface always needs the same output format, fine-tuning can reduce errors. But try [[structured-outputs|structured outputs]] first.
- **Changing knowledge.** If information changes every week, training the model on it is a poor idea. That case fits [[rag|RAG]] better.
- **Maintenance.** A fine-tuned model is an asset to re-evaluate every time the base model changes. And providers retire models: OpenAI states that fine-tuned models remain available only until their base model is deprecated.

## When does fine-tuning DO make sense?

OpenAI lists four advantages over prompts alone:

- Training on more examples than fit in one context window.
- Using shorter prompts, which lowers [[tokens|token]] costs and latency at scale.
- Training on proprietary or sensitive data without including it in every request.
- Training a smaller, cheaper, faster model for a specific task.

Notice the pattern: nearly all are reasons of scale, cost or consistency, not "the model cannot do it". At low volume, a good prompt usually wins.

## When should you NOT fine-tune?

When you have not tried a good prompt. Many teams train because it sounds more serious, then discover that a clear instruction with three examples would have solved the problem. OpenAI's own guide suggests starting with about 50 well-crafted examples and, if they make no difference, revisiting the task or the prompt before adding more data. With data, quality matters more than quantity.

And if what you want is for the model to remember facts, fine-tuning is the wrong tool: it is better to hand it the documents at answer time, with [[rag|RAG]]. Also, according to Wikipedia, tuning can weaken a model's performance on data unlike its training data, so do not assume it only improves things.

## How do you decide?

OpenAI proposes a loop that repeats: write evals that set a baseline, try prompts, fine-tune where it makes sense, measure both and adjust. Translated into a practical order:

1. Build a set of real cases and measure quality. Without measurement you cannot tell whether anything improved. See [[what-are-evals|what are evals]].
2. Write the clearest prompt you can, with two or three examples. See [[few-shot-prompting|few-shot prompting]].
3. If the problem is knowledge, try RAG.
4. If the problem is behavior or format that prompting does not achieve consistently, consider fine-tuning.
5. Compare the cost, latency and maintenance of each option, and measure again with the same cases.

## Example

Illustrative example: a support application must always answer in the brand's tone with a three-section format. With a well-written prompt and four examples, the format comes out right in 9 of 10 cases, but the rest break. The team does not train right away: it first adds format validation and reviews the failed cases. If it still fails in one specific category afterwards, it prepares a set of examples for that category only and evaluates whether fine-tuning solves it. The decision is made with data, not intuition.

## Common mistakes

- Fine-tuning on data that changes, which forces retraining every time.
- Training without an evaluation before and after, or without holding out part of the data as a control.
- Using low-quality data because there is a lot of it.
- Treating the prompt as a minor detail. Many improvements come from there.
- Assuming a provider will keep offering the same model or tuning method.

## Sources

- [Fine-tuning (deep learning), Wikipedia](https://en.wikipedia.org/wiki/Fine-tuning_(deep_learning))
- [Prompt engineering, Wikipedia](https://en.wikipedia.org/wiki/Prompt_engineering)
- [LoRA: Low-Rank Adaptation of Large Language Models (arXiv:2106.09685)](https://arxiv.org/abs/2106.09685)
- [OpenAI: Model optimization](https://developers.openai.com/api/docs/guides/model-optimization)
- [OpenAI: Supervised fine-tuning](https://developers.openai.com/api/docs/guides/supervised-fine-tuning)


---

Source: https://www.fernandoux.com/en/wiki/ai/fine-tuning-vs-prompting/