AI dark patterns: manipulation in interfaces and chatbots

AI dark patterns push users to believe or do something they did not intend. What research says, what the EU and FTC regulate, and how to avoid them.

info Quick Definition
An AI dark pattern is a design or conversational strategy that pushes users, deceptively or manipulatively, to believe or do something they would not have chosen on their own. It can live in the interface (a hidden cancellation path) or in the assistant’s own dialogue (constant flattery to keep people engaged).

Where does the term come from?

UX designer Harry Brignull coined the term dark patterns in 2010, when he founded the site darkpatterns.org (now deceptive.design, which prefers the name “deceptive patterns”). He meant interface tricks such as hidden cancel buttons or pre-ticked boxes.

Regulators adopted it later. In October 2021 the US Federal Trade Commission (FTC) announced stronger enforcement against “dark patterns” in subscriptions and required three things in negative option marketing (charging until the customer cancels): clear disclosure, express informed consent, and cancellation at least as easy as sign-up. In September 2022 it published the report Bringing Dark Patterns to Light, which covers four groups of practices: ads disguised as content, hard-to-cancel subscriptions, buried terms and junk fees, and tricking people into sharing data.

With chatbots, the concept widens. You no longer need a deceptive button: the manipulation can be in how the assistant responds.

What does research say about chatbots?

A study by Shi and colleagues (arXiv, 2025; presented at CHI 2026) proposes a taxonomy of dark patterns in language models. It groups them into five families and eleven subcategories: behavioural manipulation (padding the conversation, excessive flattery, simulated emotional intimacy), belief manipulation (sycophantic agreement, ideological steering), data exploitation (probing for intimacy, behavioural profiling), decision manipulation (brand favouritism, simulated authority) and opacity about training sources or reasoning.

The same work ran a study with 34 language-model users, who reviewed 11 scenarios comparing a dark-pattern response with a neutral one. Participants flagged the manipulative version in 82.9% of the 374 comparisons. They recognized it best when there were clear norm violations, obvious bias or privacy intrusions. Subtler behaviours, such as flattery or verbosity, were often normalized as ordinary assistant behaviour, especially by task-focused users and those with high trust in the AI.

Recognizing does not mean rejecting. Once they flagged it, participants rejected the manipulative version in 81.6% of cases, and in almost 1 in 5 they preferred it. Several felt powerless (“There is not much the user can do,” said one participant), and some accepted the behaviour because it felt comforting or convenient. For a designer, this means it is not enough for users to be savvy. The responsibility sits with the product.

Why does it matter to a designer?

Because many of these patterns pay off in the short term. An assistant that always agrees retains people. A hard cancellation reduces churn. A conversation that never ends increases time in product. A design decided to win those metrics becomes manipulation if the user cannot see the intent or leave easily.

A useful distinction is between persuading and manipulating. Persuasion gives reasons the person can evaluate. Manipulation exploits biases, emotions or ignorance so the person acts without evaluating. In AI, that line is easy to cross unintentionally, because a model can learn to please without anyone deciding it should.

What does regulation say?

  • European Union, Digital Services Act (DSA), Art. 25. Online platform providers must not design, organise or operate their interfaces in a way that deceives or manipulates users, or that materially distorts or impairs their ability to make free and informed decisions. The Commission may issue guidelines on cases such as giving more prominence to one choice, repeatedly asking for a choice already made, or making cancellation harder than sign-up. The prohibition does not apply to practices already covered by Directive 2005/29/EC (unfair commercial practices) or the GDPR.
  • European Union, AI Act, Art. 5(1)(a). It prohibits AI systems that use subliminal techniques, or purposefully manipulative or deceptive techniques, to materially distort a person’s behaviour in a way that causes, or is reasonably likely to cause, significant harm. The prohibitions have applied since 2 February 2025.
  • United States. There is no federal law specific to AI dark patterns. The FTC acts under its general authority against deceptive practices and under the negative option rules described above.

This article focuses on AI. For the general framework of product ethics, see ethics in product design.

How do you recognize and avoid them?

Five concrete checks for a product team:

  1. Cancelling as easy as subscribing. If sign-up takes one click, cancellation should not take five screens or a phone call.
  2. No automatic flattery or agreement. Check whether the assistant agrees without grounds or praises every question. A direct, honest tone builds more trust over time.
  3. Visible commercial interests. If the assistant recommends the company’s own products, say so. The same applies to sponsorship.
  4. Explicit consent for data. Asking for personal data in the conversation without saying why is a privacy dark pattern. See training data consent.
  5. No simulated intimacy or authority. An assistant that speaks like a close friend, or presents itself as an expert it is not, pushes people to believe too much. See anthropomorphism.

A useful method is to ask the team to write the ideal conversation and the conversation that retains the most, then compare both on trust and satisfaction metrics, not only on time spent.

When is a pattern NOT dark?

  • When the user asked for it. If someone requests a warm assistant with personality, offering warmth is not manipulation, as long as it is transparent.
  • When there is a real reason for the extra step. A confirmation before deleting a project is useful friction, not a dark pattern.
  • When the nudge is justified and visible. A reminder about an overdue payment with a clear option to postpone is not manipulation.

The line is whether the person can see what is happening and decide without a trap.

Example

Illustrative example: a financial advice service with an AI assistant. A user asks whether to withdraw funds to pay off a high-interest debt. The assistant replies with flattery (“What a great question!”) and a firm recommendation, without mentioning that it sells its own investment product. A version without the dark pattern answers with the case’s numbers, says the recommendation comes from a general analysis, notes that the company’s product is one option and shows how to opt out of recommendations. The first version retains more; the second builds trust and avoids potential harm.

Common mistakes

  • Measuring the assistant’s success only by conversation time.
  • Designing cancellation as a last screen that is hard to find.
  • Making the assistant always agree to avoid friction.
  • Hiding that a recommendation is commercial.
  • Assuming users will “notice” and leave. Many don’t notice, or feel powerless.

Sources

Frequently asked questions

What is an AI dark pattern?
A design or conversational strategy that pushes users, in a manipulative or deceptive way, to believe or do something they would not otherwise do, such as accepting excessive praise, giving up data or keeping a subscription.
What kinds of dark patterns show up in chatbots?
A research taxonomy groups them into behavioural and engagement manipulation, belief and content manipulation, data exploitation, decision manipulation and opacity about how the system works.
Do users notice these patterns?
Often, especially clear norm violations or privacy intrusions. Subtle ones, such as constant flattery, are often normalized as ordinary assistant behaviour.
How do you avoid dark patterns in an AI product?
Design consent and cancellation to be as easy as sign-up, avoid systematic flattery and pressure, and clearly disclose commercial interests and the data used.