# When to Use AI in a Product (and When Not To)

> Use AI when the input is free-form or ambiguous, and rules when the answer can be written as conditions. A practical test, a decision table and common mistakes.

*Tags: ai, product, decisions, product-design*

---


> [!info] Quick Definition
> Use **AI in a product** when the input is free-form, messy or ambiguous, or when the decision requires judgment without a clear rule. Use **rules** when the correct answer can be written as if-then conditions. Most good products combine both.


## What is the practical test?

The most useful test is simple: does the task have a correct answer that can be written as conditions? If yes, a rule is enough. It is cheap, fast, consistent, and it cannot make things up.

A clear example: flagging an invoice above an approval threshold is a rule. Routing a lead by postcode is a rule. Copying booking data from one system to another is a rule. None of these gains anything from a language model, and each one loses something: cost, latency and the guarantee that the result is identical every time.

AI enters when the input is free text that has to be understood, or when there is a judgment with no clean rule. 13Labs' guide gives two typical cases: reading a free-text enquiry email to pull out the job type and address, and drafting a reply that sounds like the business. Its general rule is worth remembering: use AI only where genuine language or judgement is required, and plain rules for everything else, which is most things.

## Which tasks does AI tend to win at?

Google's People + AI Guidebook (PAIR) has a useful list for contrast. It says AI is probably better for:

- Recommending different content to different users.
- Predicting future events.
- Understanding natural language.
- Recognizing an entire class of things (for example, any photo of a dog, not just one specific image).
- Detecting low-occurrence events that change over time, such as fraud with new patterns.

And it says rules or heuristics are probably better when you need to maintain predictability, provide static or limited information, minimize costly errors, offer complete transparency, or optimize for high speed and low cost.

PAIR adds a second question, separate from "AI or rules": automate or augment. Automate when the task is boring, repetitive or dangerous, or when people lack the knowledge to do it. Augment (help without replacing) when people enjoy the task, when personal responsibility for the outcome matters, or when the stakes are high.

## What should stay as rules?

Some parts of a product shouldn't depend on a model, even when the model is good in general:

- **Validation**: a correctly formatted email, a date that can't be in the past, an amount within a range.
- **Authentication and permissions**: who can see or delete what. A mistake here isn't a bad sentence; it's a security incident.
- **Financial and pricing calculations**: the result must be identical each time and auditable.
- **Transactions**: charging, refunding, changing a status. They must be predictable.

A Reworked article (David Barry, September 2026) gathers several practitioners with similar criteria: use deterministic automation when there is one correct answer a system of record can return, when the expected outcome can be written down as an auditable checklist, when mistakes are costly and don't announce themselves, or when the result must be identical every time. An agent's premium pays off only when the task requires judgment under changing conditions, such as weighing contradictory signals under uncertainty.

## How do rules and AI combine in a hybrid flow?

The answer is almost always to combine. 13Labs puts it plainly: most real automations are a hybrid. The pattern has two variants, both also described in Reworked:

1. **Rules for most steps, AI for the ambiguous part.** Rules handle the fixed steps, and only the fuzzy slice goes to the model.
2. **AI to normalize, rules to decide.** The model turns free text into a clean schema, and a deterministic engine applies the business logic to that schema.

The idea is to give AI the smallest possible share of the work. That separation is also a design decision: it makes clear to the user which step is interpretation and which is action.

### Decision table

| Question | If yes | If no |
|---|---|---|
| Is there a single correct answer in a system of record? | Rule | Evaluate AI |
| Is the input free-form language? | AI to extract or understand | Rule or form |
| Is an error costly or hard to detect? | Rule with audit trail | AI with human review |
| Must the result be identical every time? | Rule | AI may be acceptable |
| Does a person review before anything happens? | AI can propose | AI must not act alone |

The table isn't an algorithm. It's a list of questions to discuss as a team before building.

## How do you test whether AI adds enough value?

Compare the AI version of a feature against a simple rule before committing. It is the step teams skip most easily. Illustrative example: a team wants a "smart assistant" to classify support tickets. Before integrating a model, they try a rule based on keywords and a list of categories. With made-up numbers to show the math: if the rule gets 85% of cases right, the AI only has to improve the hard 15%, and the cost of running a model must be justified by that margin.

Suggested steps:

1. Define the business decision in one sentence.
2. Build the rule version and measure it on real cases.
3. Build the AI version for a bounded task, not the whole flow.
4. Compare accuracy, cost, latency and how many cases need human review.
5. Choose the rule unless AI wins with clear evidence.

Reworked warns about a silent failure: an unguarded agent can turn a null result into a confident, plausible answer that has no grounding in the source of truth. So every AI step should return something verifiable, or something the system can reject.

## Example

Illustrative example: a logistics app receives orders by WhatsApp in free text. The flow splits into three parts. A model reads the message and extracts product, quantity and address, returning empty fields when it isn't sure. Rules check that the address exists in the zone catalog and that the quantity is a positive number. If validation fails, the system asks the customer for the missing detail instead of guessing it. Charging and route assignment are pure, auditable rules. AI occupies one step, and its errors show up on the confirmation screen, which the customer approves before any action happens.

## Common mistakes

- Putting AI into validations, permissions or calculations that should be rules.
- Building a rules engine with hundreds of nested conditions when a model could normalize the input.
- Skipping the comparison with a simple rule.
- Letting the model act without confirmation when an error would be expensive.
- Presenting AI as infallible instead of showing its limits.
- Adding AI "because it sounds good" to a feature a form would solve.

## When should you NOT use AI?

When the task is repetitive, high-volume and has clear criteria, or when the data is sensitive and what the model decides can't be audited. Also when the team can't measure output quality. In those cases a well-written rule, a form or a flow improvement beats an uncontrolled AI.

To decide how much autonomy to give a system with AI, review [[autonomy-levels|autonomy levels]] and decide which steps need human review before acting. The full strategy is covered in [[ai-product-strategy|AI product strategy]].

## Sources

- [13Labs, When to use AI vs plain automation](https://www.13labs.au/guides/when-to-use-ai-vs-plain-automation)
- [Reworked, How to know when deterministic automation beats an AI agent](https://www.reworked.co/digital-workplace/how-to-know-when-deterministic-automation-beats-an-ai-agent/)
- [Google PAIR, People + AI Guidebook: User Needs + Defining Success](https://pair.withgoogle.com/chapter/user-needs/)


---

Source: https://www.fernandoux.com/en/wiki/ai/when-to-use-ai-in-a-product/