Local AI Agents · Module 2 · Lesson 2 of 3

Acceptance criteria

Writing acceptance criteria that a task's output can actually be checked against, and telling "looks done" apart from "verifiably done."

Lesson · 15–20 minutes · Text-first

By the end, you can

Write acceptance criteria for a bounded task that are checkable against evidence, not against impression (LA-2).
Distinguish "looks done" from "verifiably done," and identify which one a proposed criterion actually tests (LA-2).

Before you start

This is Module 2, Lesson 2 of the Local AI Agents course. It assumes Module 1's vocabulary and this module's first lesson: a bounded task's inputs, outputs and constraints, written down before the run starts. This lesson adds the next piece of the task brief — the check that decides whether a completed output actually satisfies the task, rather than merely resembling something that would.

Two very different sentences that both claim "done"

"The summary looks good" and "the summary lists every ticket category present in the source data, with counts that match the export" are both claims that a task is finished. Only one of them can be checked by someone who was not in the room when the agent ran.

"Looks done" is a judgement about plausibility: the output reads well, nothing obviously jumps out as wrong, the shape matches what was expected. "Verifiably done" is a judgement about evidence: a specific, checkable statement was tested against the actual output, and it passed. The first can be produced by skimming. The second requires the criterion to have been written precisely enough that skimming would not be enough to answer it.

Acceptance criteria are how a bounded task moves from the first kind of claim to the second.

What makes a criterion checkable

The Scrum Guide's Definition of Done is written for a different kind of work, but the underlying idea transfers directly: it is "a formal description of the state of the Increment when it meets the quality measures required for the product," and until that state is met, the work is not allowed to count as finished — it is not released or presented as done. The value is not the specific checklist; it is the discipline of having a written, shared standard that a person other than the one claiming completion can apply.

A checkable acceptance criterion has three parts:

"The summary is accurate" fails all three: it is not specific, it has no stated threshold, and it names no check. "Every category total in the summary table matches the count of matching rows in the source export" passes all three: it names exactly what to compare, states the pass condition (matches), and names the check (compare the table to the source export).

A specific, observable claim about the output — not a feeling about it.
A pass/fail line — a stated threshold or condition that separates "met" from "not met," not a spectrum to eyeball.
A named check — what evidence gets inspected, and by what method, to decide which side of that line the output falls on.

Acceptance criteria are written before the run, not after

Writing acceptance criteria after a task has already run invites a quiet failure mode: the criteria drift to match whatever the agent produced, instead of testing whether it produced the right thing. A criterion decided in advance cannot be reshaped by the output it is meant to judge — if the output does not meet it, that is information about the output, not a signal to soften the criterion.

This does not mean criteria can never be revised. It means a revision is a visible decision, made and recorded separately from the moment you are looking at a specific result and want it to pass.

A light pointer to evidence, not the full treatment yet

A criterion is only as good as the evidence used to check it, and deciding what evidence a claim needs is a bigger subject than this lesson covers. The adjacent point has already appeared in this course: OpenAI's guidance on human review documents the mechanism — a run pauses so a person or policy can approve or reject a sensitive action before it takes effect — and, as Module 1's approval-request framing put it, that reviewer needs the exact proposed action and its expected result in front of them, because an opaque "Approve?" is not oversight. An acceptance check needs the same kind of concrete evidence, not a summary of the summary. Module 4, later in this course, covers how to capture and structure that evidence — logs, diffs, test results, review notes — in full. For now, it is enough that every criterion you write names what evidence would settle it, even before you have built the process to capture that evidence reliably.

A worked example: acceptance criteria for the ticket-summary task

Building on the previous lesson's bounded ticket-summary task — input: this week's ticket export, five named columns; output: a markdown table plus a two-sentence paragraph — three checkable criteria:

Each line names a claim, a pass condition, and a check — not an impression of quality.

Every distinct category present in the input rows appears as its own row in the output table, with no category merged, split or omitted. Check: compare the table's category list to the distinct values in the export's `category` column.
Each category's count in the table equals the number of input rows with that category. Check: recompute each count directly from the export and compare.
The output contains no customer name, email address or message body text, even though those fields exist in the source export. Check: search the output text for any value that matches a name, email or body field from the source rows.

Accessibility notes

This lesson is text-first, with no images, audio, video or downloadable artifacts. The practice exercise's model answer sits behind a native disclosure control that is reachable and operable by keyboard and correctly announced by screen readers. The knowledge check uses native radio-button inputs with a visible question and options, and posts its result to a live status region so assistive technology announces the outcome without a page reload.

Practice

Turn a vague sign-off into checkable criteria

A task produces a weekly digest of newly published documentation pages. The current sign-off standard is: 'looks complete and reads well.'

Write one checkable acceptance criterion about what the digest must contain, with a named pass condition and a named check.
Write one checkable acceptance criterion about what the digest must not contain or must not do.
Explain, in one sentence, why 'reads well' cannot be an acceptance criterion on its own.

Compare with a bounded first version

A content criterion: every documentation page published in the target week appears as its own entry in the digest, with no page omitted or duplicated — checked by comparing the digest's entries against the publishing system's list of pages published that week. A boundary criterion: the digest links to each page rather than reproducing its full text, and it does not email or post the digest anywhere — checked by inspecting the digest file for full-text reproduction and confirming no send or post action occurred. 'Reads well' cannot be an acceptance criterion on its own because it names no specific claim, no pass/fail line and no check — two reviewers could disagree about whether it passed without either being wrong, which is exactly what a checkable criterion is meant to prevent.

Knowledge check

Try the idea

Low-stakes practice only. This does not score, block progress or create a learner record.

Sources and limits

This lesson synthesises the sources below into a practical learning model. It is not a security standard, legal advice or a guarantee that any particular agent design is safe.

The 2020 Scrum Guide — Scrum.org and Scrum Inc.. Defines the Definition of Done as the formal, shared standard that work must meet before it counts as complete, distinguishing genuinely finished work from work that merely appears finished.
Guardrails and human review — OpenAI API documentation. Documents human review pausing a run so a person or policy can approve or reject a sensitive action before it takes effect — the review moment an acceptance check supplies evidence to.