AI Hallucinations: Why AI Makes Things Up

What an AI hallucination is

An AI hallucination is a confident, plausible answer that is false or cannot be verified. In business terms, the AI fills in the blank instead of saying "I don't know." Common examples:

Invented citations: articles, studies, laws or court cases that do not exist.
Wrong prices: an outdated or guessed figure given to a customer.
Fake policies: a refund, warranty or discount your company never offered.
False quotes: words attributed to a person or publication that never said them.
Unsupported product claims: "certified", "compliant" or "guaranteed" with nothing behind it.

Why AI makes things up

Language models are built to produce likely-sounding text, not to prove each statement before answering. They know what a refund policy or a legal citation looks like. When the real fact is missing, the model can still produce text that fits the pattern.

Hallucinations are more likely when:

the question needs facts the tool does not have, such as today's stock, a private contract or an unpublished policy;
the prompt pushes for a complete answer ("don't say you can't");
the topic needs precision: legal, financial, medical, technical or pricing;
the information changed after the model learned it;
nobody checks the answer against a source before it is used.

NIST's Generative AI Profile lists confidently stated false content as a core risk of generative AI. Treat AI as a fast drafter, not an unchecked authority.

Where hallucinations hurt a business most

Customer answers: a chatbot or AI-drafted email promises a refund, discount or delivery date that does not exist. See who pays when a chatbot makes a promise.
Legal and financial content: a misread contract clause, a wrong rule or an invented case.
Quotes and citations: a made-up testimonial or source makes weak content look verified.
Product claims: AI loves words like "approved" and "guaranteed"; unless your documents support them, they become misleading claims repeated across ads, proposals and listings.

A public example: fake cases in a court filing

In Mata v. Avianca, a federal court in New York sanctioned two lawyers and their firm in June 2023 after a filing cited court opinions that did not exist, generated by ChatGPT, and the lawyers kept relying on them after the court questioned them. Courts have since dealt with similar filings.

The lesson reaches far beyond law: if a source matters, open it and confirm it says what the AI claims. Fluent writing passed through professional review because people trusted how it looked.

How to catch hallucinations before they leave the building

Ask for sources, then open them. Confirm the source exists, is current and actually says what the AI claims. No source, no final answer for important facts.
Ground answers in your own documents. Have the AI answer from your help center, price sheet, policies and product sheets instead of general knowledge. Our RAG vs fine-tuning guide explains how grounding works.
Keep a person on anything leaving the business. Customer emails, chatbot answers, proposals, ads, filings and product claims need a named reviewer.
Test with questions it should refuse. Ask for a fake discount, a policy exception, legal advice or a price that is not in the system. A safe setup declines or escalates.
Choose tools that can say "I don't know". For business use, an honest refusal beats a polished guess.

Red flags that an answer may be made up

Very specific dates, names, prices or percentages with no source.
A report, case or policy nobody can find.
Quotation marks with no link or transcript.
Absolute words: "always", "guaranteed", "legally required", "fully compliant".
An answer that conflicts with what your team knows about pricing, policy or features.
A reference to a department, approval or policy number that may not exist.

The dangerous hallucinations are not absurd. They are close enough to reality that busy people accept them, which is why checking beats intuition.

Match the review to the risk

Low risk: brainstorming, outlines, rewriting internal notes. Label it a draft; check any fact before it is reused.
Medium risk: customer emails, help articles, product descriptions, training material. Name the source for important claims and have an accountable reviewer.
High risk: legal, financial, compliance, medical or safety content and public claims. AI drafts from approved material only; an expert approves and the check is recorded.

Useful standing instructions: "Answer only from the documents provided; if the answer is not there, say you do not know" and "Do not invent prices, policies, quotes, citations or guarantees." Train staff on these habits; our AI training guide has a simple plan.

Where ASL fits

Viavitna is our AI knowledge base for customer questions. It answers only from content you approve, cites its sources and declines when your content does not cover a question, the refusal behaviour this guide recommends. For staff, ASL Help Engine answers from reviewed help content. Contact us to test your own risky questions.

Frequently asked questions

What is an AI hallucination?

A confident, plausible answer from an AI tool that is false or unsupported, such as an invented citation, a wrong price or a policy your company never had.

Why does AI make things up?

Language models generate likely-sounding text rather than proving each fact. When the real information is missing, they can still produce an answer that fits the pattern.

Can hallucinations be eliminated?

Not completely, but grounding answers in your own documents, requiring sources, testing refusals and keeping a person on high-risk output catches most of them before they cause harm.

How do I check an AI citation?

Open the source itself. Confirm it exists, is current and says what the AI claims. A citation that looks correctly formatted proves nothing on its own.

Which business uses are riskiest?

Customer promises, legal and financial content, quotes and testimonials, and product claims, because people rely on them and they carry your company's name.

Related guides

Explore the complete product

Read features, current access and limitations

AI Hallucinations: Why AI Makes Things Up and How to Catch It