Documenting residual risk

By the end, you can

• State what "residual risk" means for a completed local-agent run — the specific things that remain unresolved, deferred or unverified after a disposition and any rollback or revision — and distinguish an honest residual-risk statement from reassuring vagueness (LA-4).
• Write a residual-risk statement that names the specific unresolved item and the condition under which it would matter, rather than a general reassurance that omits both (LA-4).

Before you start

This is Module 5, Lesson 2 of the Local AI Agents course. It assumes Lesson 1's rollback-versus-revision choice, and Module 4 Lesson 2's point that an honestly recorded unresolved risk is evidence, not a gap to hide. This lesson goes further: what actually belongs in the record once a run — accepted, revised or rolled back — has already finished, and how to tell an honest residual-risk statement from something that only sounds like one.

What stays true after a run ends, whatever the disposition was

Whichever disposition Module 4 Lesson 3's process reaches, something usually remains unresolved: a check that could not be run, a case the fix does not cover, a rollback that restores a prior state without explaining why the problem happened in the first place, a scope the task never touched at all. That remainder has a name. NIST's AI Risk Management Framework defines it directly, in its MANAGE function: "Negative residual risks (defined as the sum of all unmitigated risks) to both downstream acquirers of AI systems and end users are documented." Residual risk is not a sign that something went wrong with the run — it is the ordinary, expected remainder any bounded task leaves behind, because a bounded task, by Module 2's own definition, was never scoped to cover everything.

Even a clean accept, and a rollback that fully restores the prior state, both typically still carry some residual risk. An accept can meet every stated acceptance criterion while an adjacent, uncovered case stays untouched — Module 4 Lesson 2's dependency-bump exercise practised writing exactly that kind of honest unresolved-risk line, naming the downstream consumers of the bumped library that the test suite never exercised, even with every automated check passing. A rollback undoes the symptom, restoring what was there before, but it rarely explains the underlying cause that produced the bad change — that cause is still out there until someone separately investigates it.

Honest residual risk vs reassuring vagueness

Two statements can describe the same run and carry entirely different amounts of information.

Google's Site Reliability Engineering practice built this same discipline for a different kind of write-up — an incident review, not a routine run — but the underlying habit transfers directly. A postmortem works because it stays "blameless": it "must focus on identifying the contributing causes of the incident without indicting any individual or team for bad or inappropriate behavior." A residual-risk line only stays honest under the same condition. It is not admitting a mistake was made; it is stating, plainly and without editorializing, what is still true and unresolved. Vague reassurance is often just as unwilling to name a fault as an honest line is — the difference is that vagueness hides the useful information along with it.

• Reassuring and vague: "should be fine now," "this shouldn't happen again," "mostly resolved." None of these names a specific unresolved item, a condition under which it would matter, or anything a later reader could check. They read as confidence, not as evidence.
• Honest and specific: names exactly what remains unresolved, and the concrete condition under which it would turn into a real problem. "The rollback restores the prior configuration value; the run did not investigate why the bad value was set in the first place, so the same failure could recur if that upstream cause is not separately investigated" tells a reader precisely what is still open and what to watch for. "The notification was corrected with a follow-up message; there is no way to confirm every original reader saw the correction" does the same for Lesson 1's irreversible-action case.

Why this honesty is an accountability requirement, not extra diligence

NIST's Appendix C states the underlying reason plainly: "Human roles and responsibilities in decision making and overseeing AI systems need to be clearly defined and differentiated." Someone has to actually be positioned to own accepting or monitoring a residual risk, and a vague statement makes that ownership impossible to assign — nobody can be responsible for watching for "issues," only for a named, specific thing. The MANAGE function's documentation requirement makes the same point from a different angle: a residual risk that exists only as something someone privately suspects, and never gets written down, is not actually managed, whatever that person happens to remember later.

A worked example: residual risk after a database connection-string rollback

A local agent's change to a service's database connection string caused connection failures; the disposition was roll back, and the prior, known-good connection string was restored from the diff captured when the change was first made. The service is stable again.

A vague version of the record: "Issue resolved, everything back to normal." This is true in the narrowest sense and tells a later reader almost nothing.

The honest version: "The connection string was restored to its prior known-good value, and the service is stable again. This run did not investigate why the bad value was set in the first place — the rollback removes the symptom, not the cause — so the same failure could recur if whatever produced the original bad value is not separately investigated." The honest version costs one extra sentence and changes what a reader actually knows: not just that the service is currently fine, but exactly what has, and has not, been established.

Accessibility notes

This lesson is text-first, with no images, audio, video or downloadable artifacts. The practice exercise's model answer sits behind a native disclosure control that is reachable and operable by keyboard and correctly announced by screen readers. The knowledge check uses native radio-button inputs with a visible question and options, and posts its result to a live status region so assistive technology announces the outcome without a page reload.

Sources and limits

This lesson synthesises the sources below into a practical learning model. It is not a security standard, legal advice or a guarantee that any particular agent design is safe.

1. AI RMF Core — NIST AI Resource Center. Defines residual risk, in subcategory MANAGE 1.4, as the sum of all unmitigated risks, and requires it to be documented for downstream acquirers and end users.
2. Appendix C: AI Risk Management and Human-AI Interaction — NIST AI Resource Center. Calls for explicit human roles and responsibilities in AI oversight — the accountability an honest residual-risk statement exists to support.
3. Site Reliability Engineering, Chapter 15: Postmortem Culture: Learning from Failure — Google SRE Book. Models a written, blameless record of what remains unresolved — the same honesty this lesson asks of a routine run's residual-risk line, not only of a post-incident review.