When AI gets it wrong: check, don’t trust.

Language models write convincingly, even when they are wrong. Anyone using AI needs a review process.

Why models make things up

A language model predicts the next word that fits statistically. It does not access a secured database of facts. That is why an answer sounds fluent even when the source, figure or connection is invented.

Typical patterns

• Invented sources, studies or quotations.
• Wrong figures, dates and names.
• Outdated information without any note on the state of knowledge.
• Confident wording without any uncertainty.

Four checks worth doing

• Ask for the source, and open the source yourself.
• Always verify figures, legal and medical content.
• Have answers broken down into checkable parts.
• Work with your own documents: provide current texts as reference.

Technology alone is not enough

Risk management frameworks such as the NIST AI Risk Management Framework rely on four steps: understand context, name risks, measure impact, apply controls. Human oversight remains decisive, especially for decisions with consequences.

A wrong answer is written quickly. A checked answer builds trust.

Conclusion

Hallucinations are not an outlier but a property of the system. Those who know this verify, and use AI where verification is possible.