Confidence, verification logs and this course's reusable checklist

By the end, you can

• Verify a generated claim against the source closest to it, and record the check as a five-part review-log entry — the claim, its kind, the source, the citation result and the outcome (AF-5).
• Assemble this course's five final-assessment artifacts into one reusable workflow checklist, naming which earlier module and lesson produces each of the first four (AF-5).

Before you start

This is Module 5, Lesson 3 — the last lesson of this course. It assumes Lesson 1's fact/inference/recommendation/unknown split and source ladder, and Lesson 2's citation-quality check. It also assumes Module 2 Lesson 3's stakes question about how costly or reversible acting on a wrong answer would be, and it names, in its closing section, exactly where in this course the rest of this course's final-assessment artifacts live. This lesson does not introduce a new verification technique of its own — it is the assembly step.

Confidence is not evidence, one more time

Module 1 Lesson 2 made a point this course has returned to in every module since: a fluent, confident-sounding AI answer is not, on its own, evidence that it is correct. The Open Worldwide Application Security Project's generative-AI security effort names the failure this produces directly: content that "seems accurate but is fabricated." Nothing about how sure an answer sounds changes whether it has actually been checked — confidence is a property of the writing, not a report on the checking.

Stakes change how much checking is actually worth doing, though, and this course already has the tool for deciding that: Module 2 Lesson 3's question, in effect, is how easy a wrong answer would be to notice and how costly or reversible acting on it would be before anyone checks. The same guidance that defines hallucination states a matching mitigation directly: "Implement tools and processes to automatically validate key outputs, especially output from high-stakes environments." This is the same threshold Module 4 Lesson 3 used for tool choice, from a different source: OWASP's guidance on excessive agency recommends that teams "utilise human-in-the-loop control to require a human to approve high-impact actions before they are taken." Read together with Lesson 1 and Lesson 2's checks, the practical rule is: the higher the stakes of being wrong, the more this module's checks are worth doing in full — source ladder and citation check both — rather than skimmed or skipped because the answer sounded confident.

Writing a review log entry

Checking a claim once, in your head, and moving on leaves no trace a later reader — including you, a week from now — can actually inspect. NIST's AI Risk Management Framework treats this as a formal expectation at a larger scale, not merely a nice-to-have habit: its measure function calls for "rigorous software testing and performance assessment methodologies with associated measures of uncertainty, comparisons to performance benchmarks, and formalized reporting and documentation of results." A single verified claim does not need a testing programme, but it needs the same underlying habit in miniature: a short, written record of what was checked and what was found.

A usable review log entry has five parts: the **claim** being checked, in its own words; the **kind** it was sorted into, from Lesson 1 — fact, inference, recommendation or unknown; the **source** consulted, and where it sits on Lesson 1's source ladder; whether any **citation** attached to the claim actually held up, per Lesson 2's check; and the **result** — confirmed, corrected, or could not be confirmed — with the date the check happened. None of the five parts needs more than a sentence. What matters is that all five are there, and that a reader who was not present for the check can still tell what was actually established.

A worked example: verifying a generated answer and logging it

An AI tool, asked to prepare a short update for a community group's members, produces: "Our new website is now live at the address members were sent last month, and early feedback has been very positive."

Splitting this by Lesson 1's kinds: "the new website is now live at the address members were sent" is a fact — a specific, checkable claim. "Early feedback has been very positive" looks less like a fact than an unsupported claim once checked — nothing in the instruction supplied any actual feedback for the AI to summarise, so this sentence reads as a confident restatement of something that was never actually given to it.

Checking the fact: the closest source, per Lesson 1's ladder, is the live website itself — visiting the actual address, not trusting the AI's restatement of it. The site is live, confirming the claim. The "very positive" feedback sentence has no source to check at all, because no feedback was ever supplied to check against — it does not fail a citation check the way Lesson 2 describes; it never had anything real behind it to check in the first place.

The review log entry: **Claim** — the new website is live at the address sent to members. **Kind** — fact. **Source** — the live site itself, visited directly. **Citation** — not applicable; no citation was attached. **Result** — confirmed, 16 July 2026. A second entry: **Claim** — early feedback has been very positive. **Kind** — originally presented as a fact, reclassified as unsupported. **Source** — none available; no feedback was supplied to the AI or exists to check. **Result** — cannot be confirmed; removed before the update is sent.

This course's five final-assessment artifacts, and where each one lives

The AI Foundations final assessment asks for five things, not one: a task selection and risk rationale; an improved AI task brief; a data-sensitivity checklist; a verification log for a sample AI output; and a reusable workflow checklist. Four of those five already have a specific home in this course — three in earlier modules, and the fourth in this very lesson:

The fifth, the reusable workflow checklist, is not a sixth new thing to learn — it is the other four, assembled into one document you can actually reuse on a new task next week.

• **Task selection and risk rationale** — Module 2 Lesson 3's three-question habit, sorting a task into use directly, use with a named review step, or do not delegate without a named human decision-maker, with the reasoning behind that choice.
• **Improved AI task brief** — Module 3 Lesson 3's complete, testable brief: goal, context, constraints, an example or stated output format, a stop condition and a review criterion.
• **Data-sensitivity checklist** — Module 4 Lesson 3's five-step safe-use checklist: classify, check necessity, redact what remains, check retention and training, and match the tool to what's left.
• **Verification log for a sample AI output** — this lesson, just practised above: a claim, its kind, the source checked, the citation result and the outcome.

Assembling the reusable workflow checklist

1. **Choose the pattern.** Apply Module 2 Lesson 3's three questions and name the pattern: use directly, use with a named review step, or keep this with a named human decision-maker. 2. **Write the brief.** Apply Module 3's building blocks: goal, context, constraints, an example or output format, a stop condition and a review criterion. 3. **Check the data.** Apply Module 4's habit: classify, check necessity, redact what remains, check retention and training, and match the tool to what's left. 4. **Verify the output.** Apply this module's habit: split the answer into fact, inference, recommendation and unknown; check each fact against the closest source on the ladder; check any citation against what its source actually says; and log the claim, kind, source, citation result and outcome. 5. **Record the result.** Keep the brief, the data check and the verification log together, so a later reader, including you, can see not just what the AI produced, but what was actually checked before it was used.

Completing this course's final assessment with all five parts shows that you can carry a task from a deliberate choice about whether to use AI at all, through a checkable brief and a data-sensitivity check, to a verified, documented answer. It is not a credential or an accreditation, and it does not promise anything beyond that specific, practical skill — the same honest limit NIST's own framework names for a trustworthy AI system: being "accountable and transparent, explainable and interpretable" describes a design and practice goal, not a promise that every future answer will be correct. What this course's five artifacts together show is that you checked, not that nothing will ever need checking again.

Accessibility notes

This lesson is text-first, with no images, audio, video or downloadable artifacts. The practice exercise's model answer sits behind a native disclosure control that is reachable and operable by keyboard and correctly announced by screen readers. The knowledge check uses native radio-button inputs with a visible question and options, and posts its result to a live status region so assistive technology announces the outcome without a page reload.

Sources and limits

This lesson synthesises the sources below into a practical learning model. It is not a security standard, legal advice or a guarantee that any particular agent design is safe.

1. LLM09:2025 Misinformation — OWASP Gen AI Security Project. Defines hallucination as content that seems accurate but is fabricated, and recommends implementing tools and processes to automatically validate key outputs, especially in high-stakes environments.
2. AI RMF Core — NIST AI Resource Center. States that the AI RMF's measure function calls for rigorous testing and performance assessment with formalized reporting and documentation of results — the basis for this lesson's review-log habit.
3. Artificial Intelligence Risk Management Framework 1.0 — NIST AI Resource Center. Names being accountable and transparent, explainable and interpretable among the characteristics of trustworthy AI use — the honest, bounded claim this lesson's closing section makes about what completing the course's final assessment shows.
4. LLM06:2025 Excessive Agency — OWASP Gen AI Security Project. Recommends human-in-the-loop control requiring a person to approve high-impact actions before they are taken — the same stakes-based threshold this lesson applies to how much verification effort a claim deserves.