AI automatic summaries: designing them to be verifiable

AI summaries can omit or invent. What studies show and how to design summaries that show their source, can be edited and can be checked against the original.

info Quick Definition
An automatic AI summary is a short text that a model generates from a long document, a meeting or a conversation. Its value is saving reading time; its risk is that the person accepts it without checking.

What problem does it solve, and what problem does it create?

A summary solves a very concrete problem: a person has twenty pages, an hour-long meeting or a hundred messages, and needs the essentials in two minutes. A good summary gives them the map before the territory.

The problem is that a summary replaces the original in the reader’s head. It can omit a detail that mattered most to that person: a date, a clause, a commitment. It can also state something that was never in the text. A wrong summary that reads well is more dangerous than a clumsy one, because it looks reliable.

This connects to hallucinations: the model does not warn you when it is inventing. The design has to compensate for that limitation. This article focuses on the summary itself; for synthesizing several research interviews, see AI interview synthesis.

How faithful are the summaries a model writes?

Not always very faithful, and the evidence comes from three different directions:

  • Language-processing research. Maynez et al. (ACL 2020) used human annotators to evaluate several neural summarization systems and found, in their words, “substantial amounts of hallucinated content in all model generated summaries”. Those models are older than today’s, but the finding is the reason faithfulness to the source text is still studied.
  • A public-sector trial. In a proof of concept run in January and February 2024, AWS tested Meta’s Llama2-70B for the Australian financial regulator ASIC, summarizing public submissions to a parliamentary inquiry. Five assessors who did not know which summaries came from AI scored the human ones 61 out of 75 (81%) and the model’s 35 out of 75 (47%). They flagged incorrect information, missed key points and strange choices about what to highlight. There are caveats: it was one model, with one week of prompt tuning, and AWS said the results do not necessarily reflect other models. ASIC concluded that AI should be positioned as a tool to support human work, not replace it.
  • A real product. In January 2025, Apple turned off notification summaries for news and entertainment apps in iOS 18.3 after BBC complaints about inaccurately summarized headlines. Summaries that stayed on appeared in italics, and turning the feature on in Settings now showed a notice that it was in beta and could make mistakes.

The lesson for design is not “models summarize badly” but that any summary product should assume some summaries will be wrong and give the user a way to notice.

What do studies say about how people read summaries?

Cheng et al. (NAACL 2022) reviewed 70 papers, defined five types of human-AI interaction in text generation, built summarization prototypes for each and interviewed 16 users. It is an exploratory study with few people, but it leaves useful ideas:

  • Some participants saw the AI summary as an “authority” and hesitated to edit it (“should I question what the AI thinks is important?”). Others feared they could not judge the output on topics they did not know.
  • They wanted information about how the summary was generated, and the ability to correct it.
  • The control they wanted depended on the stakes: more control for a serious text, such as a government bill, and more tolerance for something informal.

A more recent study, Traceable Text (Kambhamettu, Flores and Head, University of Pennsylvania), tested a system that links each summary sentence to the source passage. Twenty-one people took part (one did not finish and was left out of the quantitative analysis) and the material was medical progress notes. When the summary contained hallucinations, correct answers rose from 12.5% without links to 70% with them, and those tasks were faster (1.8 minutes versus 2.9). With correct summaries the gain was from 75% to 90%, which the authors describe as modest and not statistically significant.

They also warn of a risk: a wrong link can lead the reader to overestimate how true a claim is. In a spot check of 10 texts, 129 of 159 links were completely correct, 18 had semantic problems and 12 were incorrect. A link is not a guarantee, it is a verification tool, and it has to be designed as one.

What should a well-designed summary include?

Five elements drawn from the studies above (they are design recommendations, not a list that appears as such in any single paper):

  1. A source per claim. Each sentence or point should lead back to its original passage with a simple gesture: hover, tap or expand.
  2. Editing in place. If the summary omits something or gets it wrong, the person should be able to correct it without starting over, and the corrected version should be marked.
  3. Control over focus and length. Let people ask for a summary aimed at a different audience or purpose: to decide, to share, to study.
  4. A source signal. Make clear that the text is generated. Apple chose italics for its notification summaries and a beta notice in Settings.
  5. One-click access to the original. The summary should invite reading the source, not replace it.

For the citation side, see citations and sources. And if the summary may contain uncertain parts, showing uncertainty explains how to flag them.

When should you NOT summarize with AI?

  • When the exact detail is what matters. Contracts, medication doses, financial figures: a summary can erase the one datum you need.
  • When the text is short. If reading the original takes less time than reviewing the summary, the summary saves nothing.
  • When nobody can verify. If the person has no way to check the summary, the AI becomes an authority with no counterweight.
  • When the summary replaces a human decision. A meeting summary that records commitments no one has confirmed can create obligations nobody accepted.

Example

Illustrative example: a meeting summary tool. When a call ends, the summary shows three decisions and two open items. Each decision has a citation icon: hovering highlights the transcript fragment it came from. One decision says “deliver the prototype on Friday”, but the link leads to a sentence where someone says “maybe Friday”. The person edits the claim to “maybe Friday”, and the tool marks the change. Without the link, the commitment would have reached the team as firm.

Common mistakes

  • Showing the summary as a final text, with no way to reach the original.
  • Presenting the summary with the same confident tone even when the source is ambiguous.
  • Making the source link look like proof of accuracy, when it is only a lead for verification.
  • Hiding edits: if someone corrects the summary, the change should stay visible to whoever reads it later.
  • Designing a single summary length for every case, with no way to choose length or focus.
  • Testing the summary only on easy texts. Test it also on long, technical or ambiguous documents, which is where omissions and inventions show up most.

Sources

Frequently asked questions

What is an AI automatic summary?
It is a short text a model generates from long documents, meetings or conversations so that a person can read it instead of the full original.
Why can an AI summary be risky?
It can omit a critical detail or state something the original does not say, and readers may stop checking the original. In interviews with 16 users, some saw the summary as an authority and hesitated to correct it.
How does showing a source for each sentence help?
It lets people check every claim against the original text. In a study with 21 participants and medical notes, linking phrases to their sources raised correct answers on hallucinated content from 12.5% to 70%. A link can still be wrong, though.
What should a well-designed summary let users do?
See where each claim comes from, edit the summary, adjust its length or focus, and reach the original in one click.
When should you not summarize with AI?
When the exact detail is what matters (contracts, doses, figures), when the text is short, or when nobody can check the summary against the original.