Primary sources and the source ladder

By the end, you can

• Split a material sentence in an AI-generated answer into fact, inference, recommendation or unknown, and identify which kind actually needs an external source before it is acted on (AF-5).
• Choose the source closest to a specific fact — a live system, an original document or an official current record — over a secondary summary or restatement, when checking that fact (AF-5).

Before you start

This is Module 5, Lesson 1 — the opening lesson of this course's final module. It assumes Module 1 Lesson 2's two limits on a language model: no automatic lookup, and no built-in self-verification step, so a fluent, confident answer is not, on its own, evidence that it is correct. This lesson picks up exactly where that leaves off. If the AI cannot verify its own answer, something else has to — and the first two questions are which parts of the answer actually need that check, and where the check should come from.

Splitting an answer into what needs checking

Not every sentence in an AI-generated answer carries the same kind of claim, and treating them all the same way — either trusting everything or checking everything with equal effort — wastes the second option and risks the first. It helps to sort each material sentence into one of four kinds before deciding what to do with it:

The habit worth building is asking, sentence by sentence for anything that matters: which of these four is this, really? An AI answer's fluency does not sort these categories for you — a recommendation can be phrased exactly as confidently as a fact, and an unknown can be smoothed over into a sentence that reads like a fact simply because the model had to produce something.

• A **fact** is a specific, checkable claim about the world — a date, a figure, a name, a status — that a named external source could confirm or contradict.
• An **inference** is a reasoned step drawn from stated facts — plausible, but not itself directly confirmed by any one source; it is only as solid as the facts and reasoning underneath it.
• A **recommendation** is a judgement about what to do next — useful as a starting point, but not a fact about the world at all, and not something a source can confirm the way it can confirm a fact.
• An **unknown** is a genuine gap — something the answer does not actually establish, whether or not it is stated with confidence.

The source ladder: what can actually settle a fact

Once a sentence is identified as a fact — the one category an external source can actually confirm or contradict — the next question is which source. Not every source is equally able to answer the question. A useful habit, sometimes called a source ladder, is to prefer the source closest to the fact itself: a live system's current state, an original document, an official current record, or the person actually responsible for the decision — over a summary, a forum post, or an explanation written by someone once removed from the fact itself. This ranking is a practical habit this course recommends, not a rule handed down by any one authority — treat it as guidance to apply with judgement, not a fixed formula.

The Open Worldwide Application Security Project's generative-AI security effort recommends exactly this kind of external check as a mitigation against exactly the failure mode Module 1 described: "Encourage users to cross-check LLM outputs with trusted external sources to ensure the accuracy of the information." Read carefully, that recommendation names two separate things: the output needs cross-checking, not just skimming, and the source doing the checking needs to be trusted, not merely present. A link inside an AI answer is not automatically that trusted source; whether it actually is one is this module's next lesson.

Checking against a source is an external step for a specific, structural reason, not a matter of extra caution. The US National Institute of Standards and Technology frames what an AI system fundamentally does: it is "an engineered or machine-based system that can, for a given set of objectives, generate outputs such as predictions, recommendations, or decisions." Nothing in that description includes checking those outputs against reality — generating an output and verifying one are two different steps, and only the first is what the system itself does. This is the same limit Module 1 named for missing facts, Module 3 named for unstated goals, and Module 4 named for unrecognised sensitive input, now applied to verification: the system does not do this step, so someone else has to.

NIST's own risk-management framework backs up this separation formally: it describes a distinct "measure" function that "employs quantitative, qualitative, or mixed-method tools, techniques, and methodologies to analyze, assess, benchmark, and monitor AI risk and related impacts" — a separate activity from the system generating output in the first place, and the same distinction this lesson draws between an AI producing an answer and a person checking it. Module 1 Lesson 2 introduced NIST's companion profile addressing risks specific to generative AI systems; the verification habit built across this module is a practical, everyday response to exactly that risk territory, not a one-off checklist invented for this course alone.

A worked example: three sentences, three checks

An AI tool, asked to summarise a supplier's status, produces: "The supplier's last delivery was on 3 June. Given that gap, they may be struggling to keep up with demand. You should consider moving to a backup supplier for the next order."

Splitting this by kind: "the supplier's last delivery was on 3 June" is a fact — a specific, checkable claim. "They may be struggling to keep up with demand" is an inference — a reasoned guess built from the date alone, not a separate confirmed fact. "You should consider moving to a backup supplier" is a recommendation — a judgement about what to do, not a claim about the world.

Applying the source ladder to the one fact: the closest source is the delivery record itself — the logistics system or the supplier's own dispatch confirmation — not a general note summarising deliveries, and certainly not the AI's own restatement of a fact it was given. Checking that record either confirms 3 June or corrects it. The inference and the recommendation are not wrong for being unconfirmed — they are a different kind of claim, and treating them as if a source could confirm them the same way a date can is exactly the mistake this lesson's four-way split exists to prevent.

Accessibility notes

This lesson is text-first, with no images, audio, video or downloadable artifacts. The practice exercise's model answer sits behind a native disclosure control that is reachable and operable by keyboard and correctly announced by screen readers. The knowledge check uses native radio-button inputs with a visible question and options, and posts its result to a live status region so assistive technology announces the outcome without a page reload.

Sources and limits

This lesson synthesises the sources below into a practical learning model. It is not a security standard, legal advice or a guarantee that any particular agent design is safe.

1. LLM09:2025 Misinformation — OWASP Gen AI Security Project. Recommends cross-checking LLM outputs against trusted external sources, and defines hallucination as content that seems accurate but is fabricated — together the basis for this lesson's source-ladder habit and its point that an unknown can be smoothed into a sentence that reads like a fact.
2. Artificial Intelligence Risk Management Framework 1.0 — NIST AI Resource Center. Frames an AI system as an engineered system that generates outputs such as predictions, recommendations or decisions — not a system that checks those outputs against reality itself.
3. AI RMF Core — NIST AI Resource Center. Describes the AI RMF's measure function as using tools and methodologies to analyze, assess, benchmark and monitor AI risk — a distinct function from generating output in the first place.
4. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile — NIST. A NIST companion profile addressing risks unique to generative AI systems, referenced here as this course's ongoing generative-AI risk grounding.