Don't ask AI if it's right

The least useful verification question

You ask an AI assistant for the current command to repair a service. It gives a tidy answer, explains each flag and finishes with a warning to back up first. Something feels uncertain, so you ask, “Are you sure?”

It says yes.

Nothing has been verified. You have asked the same system to grade the answer it just produced, using the same conversation and usually the same missing evidence. The second response may sound more cautious or more confident, but confidence is presentation, not proof.

The better question is not “Are you right?” It is “What would prove this claim wrong?” That turns a conversation about confidence into a verification plan.

Start with the consequence, not the prose

Not every sentence deserves the same checking effort. A generated summary for your private notes and a command that changes a firewall rule do not carry the same cost if they are wrong.

Before checking the answer, classify the next action:

This is a judgment rule, not a scientific scale. Its purpose is to spend attention where an error becomes damage.

• **Low consequence:** brainstorming, reformatting or a disposable first draft. Read it critically and move on.
• **Recoverable:** a local code change with version control, a configuration edit with a tested rollback, or a calculation you can recompute. Check the important parts and exercise the recovery path.
• **High consequence:** health, legal, financial, security, production or outward-facing decisions. Stop and verify against authoritative sources or qualified people before acting.

Break the answer into checkable claims

A polished paragraph can hide several different claim types. Separate them before searching.

Suppose an assistant says: “Version 4.2 fixes the bug, the migration is backward-compatible, and the service can be restarted without downtime.” That is at least three claims:

1. Version 4.2 contains the relevant fix. 2. The migration preserves compatibility with your current data and clients. 3. Your particular restart path causes no interruption.

The first may be settled by official release notes. The second may require the migration guide plus a test against a copy of your data. The third cannot be proved by release notes at all; it depends on your architecture and needs a rehearsal or an observed health check.

This is why asking for “sources” is not enough. A real source can support one sentence while the rest of the answer rides along untested.

Use the right verifier for the claim

The OWASP Gen AI Security Project describes LLM misinformation as false or misleading information that appears credible. It also names overreliance: trusting generated content without verifying its accuracy. Its mitigations include cross-checking against trusted external sources, human fact-checking for critical information and automatic validation of key outputs.

Those are three different tools:

NIST's AI Risk Management Framework Playbook makes the same practical distinction at system level. It recommends testing procedures and metrics that demonstrate whether a system is fit for purpose and functioning as claimed, documenting what cannot be measured, and involving assessors who were not the front-line developers.

For an everyday AI answer, the small version is simple: source what can be sourced, test what can be tested, and give consequential judgment to someone able to disagree.

• **A source check** answers factual questions: does the official document actually say this, and is it current?
• **An execution check** answers behavioural questions: does the command, calculation or workflow produce the required result here?
• **An independent review** challenges framing, omissions and judgment, especially when the author and first checker share the same assumptions.

A worked example: verify a repair command

Imagine an AI proposes a command to repair a failed home-server service. Do not run it because the explanation looks plausible. Build a five-line evidence packet first.

1. **Claim:** write exactly what the command is supposed to change. 2. **Authority:** find the current official manual or project documentation for every non-obvious flag. 3. **Local state:** inspect the installed version, current configuration and service status; documentation for another version is not evidence about yours. 4. **Safe test:** prefer a dry run, validation command, disposable fixture or isolated copy. Record the actual output. 5. **Recovery:** identify the backup, previous configuration or revert command, then verify it exists before mutation.

If the tool has no dry run and the rollback is unclear, that is not a reason to trust the answer harder. It is evidence that the action belongs behind a stronger review or should be redesigned.

Citations can be wrong too

NIST calls confidently stated erroneous content “confabulation” and warns that generated logic or citations can themselves be confabulated. A link-shaped answer is therefore only a lead until you open it.

Check four things at the source:

NIST's profile describes high-integrity information as distinguishing fact from opinion or inference, acknowledging uncertainty, linking to original evidence, and being transparent about its vetting and when it may expire. That is a better answer format than a wall of citations: **fact, inference, unknown, source, checked date**.

• The page exists and belongs to the claimed publisher.
• The cited passage supports the exact claim, not a nearby weaker one.
• The date or version matches the decision you are making.
• The source is primary enough for the consequence: official documentation for product behaviour, legislation or regulator guidance for legal requirements, and live system evidence for your own runtime.

Limits and what we would do next

Verification does not guarantee truth. Official documentation can be wrong or stale. A passing test covers only the conditions you exercised. Independent reviewers can share blind spots, and some questions remain uncertain because no decisive evidence exists.

The honest outcome is sometimes “not verified.” That is useful if it prevents a guess from silently becoming a fact.

What we would do next is take one AI answer you planned to act on today and replace “Are you sure?” with four lines: the material claim, the consequence if wrong, the source or test that could settle it, and the result you actually observed. If you cannot fill in the third line, do not let the confidence of the answer choose the risk for you.

Sources and limits

This article synthesises the sources below into a practical explanation. It is not a security standard, legal advice, or a guarantee that guidance current at review time still applies — check the review date above against your own situation.

1. LLM09:2025 Misinformation — OWASP Gen AI Security Project. Defines LLM misinformation as false or misleading information that appears credible, describes overreliance as failing to verify generated content, and recommends cross-checking against trusted external sources, human fact-checking for critical information and automatic validation of key outputs.
2. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile — National Institute of Standards and Technology. Defines confabulation as confidently presented erroneous or false content, warns that generated logic and citations may also be confabulated, and describes high-integrity information as distinguishing fact from opinion or inference, acknowledging uncertainty, linking to original evidence and making its vetting and likely expiry clear.
3. AI RMF Playbook — Measure — NIST AI Resource Center. Recommends tests and metrics that show whether an AI system is fit for purpose and functioning as claimed, documenting risks that cannot be measured, and involving people independent of front-line development in assessments.

More from Server Attic

• 2026-07-31

  A scheduler is not an owner

  A practical way to design scheduled automation around missed runs, duplicates, evidence, approval and recovery instead of assuming a clock makes work reliable.

• 2026-07-29

  The alert you always ignore

  An alert nobody acts on is not monitoring. It is noise with a notification sound. A practical way to decide which home-server alerts deserve to interrupt you, using precision, recall, detection time and reset time.

• 2026-07-27

  What AdGuard Home can and cannot fix

  What network-wide DNS filtering genuinely solves, the ceiling AdGuard's own documentation sets out, and how encrypted DNS and Encrypted Client Hello change the picture.