AI Can Sound Confident—and Still Be Wrong

A clear, polished answer can still contain invented facts, incorrect instructions, or references that do not exist. NIST identifies this risk as confabulation, often called hallucination. The wording of an answer is not evidence that it is accurate.

Treat AI output as something to evaluate before relying on it. The amount of checking should reflect the consequences of an error: a rough brainstorming note and a production server change need different levels of review.

• Check important claims. Open the cited source and confirm that it supports the answer.
• Verify technical instructions. Compare commands with official documentation and test in an appropriate non-production environment.
• Give uncertainty a place. Allow “not enough information” instead of forcing a confident conclusion.

Office example: If AI suggests a firewall change, an administrator should verify its purpose and test its effect before applying it to a live network.

One action today: Add a source-checking step to one task where your team already uses AI.

Based on: NIST AI 600-1 — Generative AI Profile, especially Section 2.2 on confabulation. The office example is illustrative.