Autonomy boundaries

By the end, you can

• Classify a proposed agent action as automatic, approval-required or out of scope, and justify the classification (LA-2).
• Explain why an agent's authority boundary is a design decision, not a property of the model (LA-1).

Before you start

This is Module 1, Lesson 2 of the Local AI Agents course. It assumes the previous lesson's vocabulary: the difference between a workflow and an agent, and the five boundaries — task, data, tool, authority, and time-and-cost — that turn raw capability into a controlled, inspectable job. This lesson goes deeper on one of those five: the authority boundary.

Three outcomes, one decision

Every proposed agent action resolves to one of three outcomes before it runs. Getting this decision right, action by action, is most of what "safe autonomy" means in practice.

None of these labels comes from the model. OpenAI's API documentation frames the distinction plainly: automatic guardrails validate input, output or tool behaviour without stopping the run, while human review "pauses the run so a person or policy can approve or reject a sensitive action." The label is a design decision an operator makes about a specific action, in a specific system, before the agent ever runs it.

• Automatic — the agent performs the action without pausing, because a failure is cheap, reversible and contained.
• Approval-required — the agent prepares the action and stops. A person reviews the exact target, expected effect and reasoning, then approves or rejects it.
• Out of scope — the agent must never attempt the action at all, regardless of how confident it is.

Why the autonomy label needs a real reason, not a vibe

"This seems safe enough to automate" is not a classification method — it is a guess wearing a decision's clothes. Two questions do most of the real work:

An action that is reversible, contained to the operator's own sandboxed data, and cheap to check afterwards is a reasonable automatic candidate. An action that is hard to reverse, touches someone else, or is expensive to get wrong belongs in approval-required at minimum — and some actions belong in out of scope permanently, however convenient automating them would be.

OWASP's guidance names the failure mode directly: excessive autonomy is "an LLM-based application or extension [that] fails to independently verify and approve high-impact actions." That failure does not require a malicious prompt or a broken model — it only requires an operator skipping the classification step for one convenient-looking action.

• If this action is wrong, how far does the damage spread, and can it be undone?
• Does this action affect only data or systems the operator already controls, or does it touch another person's inbox, calendar, money, record or reputation?

Autonomy is not free, even when it works

It is tempting to treat a wider automatic set as pure efficiency. Anthropic's engineering guidance pushes back on that instinct directly: "the autonomous nature of agents means higher costs, and the potential for compounding errors," and it recommends "extensive testing in sandboxed environments, along with the appropriate guardrails" before trusting an agent with a wider automatic set.

Compounding errors are the specific risk worth sitting with. A single wrong automatic action is a bounded mistake. An agent that takes several automatic actions in sequence, each depending on the last one having gone as expected, can carry an early error through several more steps before anyone reviews the result. The more steps run unattended, the more that first small mistake gets to compound before a person sees it.

A short decision procedure

Before adding any action to the automatic set, answer these four questions in writing. A "no" to the first three, or a "yes" to the fourth, moves the action to approval-required or out of scope.

NIST's AI Risk Management Framework material frames this as a spectrum rather than a single switch: an AI system can "autonomously make decisions, defer decision making to a human expert, or be used by a human decision maker as an additional opinion," and it stresses that human roles across that spectrum "need to be clearly defined and differentiated." The procedure above is one concrete way to make that definition and differentiation explicit for a single action, rather than leaving it implied.

• Is the effect fully reversible, and do you already know how to reverse it?
• Is the blast radius (everything a wrong action could reach — data, systems and people) contained to data or systems the operator controls?
• Can a wrong result be checked cheaply, without specialist judgement, after the fact?
• Would getting this wrong affect another person's data, money, schedule, reputation or legal position?

A worked case: two versions of the same capability

Consider an agent that can draft and send meeting-reschedule messages to attendees. The capability — reading a calendar conflict and writing a polite reschedule request — is identical in both versions below. Only the authority boundary changes.

Version A treats "send the message" as automatic once the agent detects a conflict. A wrong conflict detection, a wrong attendee, or an overly presumptuous tone reaches another person's inbox before anyone reviews it — and the damage (an unwanted message, a confused colleague, a professional relationship cost) is not something the operator can quietly undo.

Version B treats drafting as automatic and sending as approval-required. The agent still saves the same research and drafting time. The difference is that a human sees the exact recipient, wording and reasoning before anything reaches someone else's inbox — the same check the decision procedure above asks for on any action that touches another person.

Accessibility notes

This lesson is text-first, with no images, audio, video or downloadable artifacts. The practice exercise's model answer sits behind a native disclosure control that is reachable and operable by keyboard and correctly announced by screen readers. The knowledge check uses native radio-button inputs with a visible question and options, and posts its result to a live status region so assistive technology announces the outcome without a page reload.

Sources and limits

This lesson synthesises the sources below into a practical learning model. It is not a security standard, legal advice or a guarantee that any particular agent design is safe.

1. Guardrails and human review — OpenAI API documentation. Distinguishes automatic guardrails from a human-review pause before a sensitive action.
2. Appendix C: AI Risk Management and Human-AI Interaction — NIST AI Resource Center. Frames AI autonomy as a spectrum from human-in-the-loop review to autonomous decisions, and calls for clearly defined human roles.
3. LLM06:2025 Excessive Agency — OWASP Gen AI Security Project. Defines excessive autonomy as an LLM-based system failing to independently verify and approve high-impact actions.
4. Building effective agents — Anthropic Engineering. Warns that an agent's autonomy raises cost and the risk of compounding errors, and recommends sandboxed testing with appropriate guardrails.