Give your AI agent a budget

Permission is only one side of the boundary

When people make an AI agent safer, they often start with permissions: can it read a folder, call an API, send a message or change a device? That is the right question, but it is only one boundary. An agent with read-only access can still consume all available time, make an expensive number of model calls, flood a queue, or keep retrying a task long after its answer stopped being useful.

Think of an agent as having two different kinds of reach:

Least privilege narrows authority. A budget narrows consumption. Neither makes a model correct, and neither replaces review, but together they keep an unproductive run from becoming an incident.

OWASP calls the broader failure mode “Unbounded Consumption”. Its examples include excessive inference, oversized inputs, repeated requests and resource-intensive queries. The guidance is aimed at AI applications, including security and availability risks, rather than a particular home setup. The household translation is simple: decide how much work one run is allowed to attempt before you give it a tool loop.

• **Authority:** what it may access or change.
• **Consumption:** how much time, context, compute, money or downstream work it may use.

Set the budget before you write the prompt

Do not begin with “try until the task is complete”. That sentence hides every limit that matters. Write the stop conditions beside the task instead.

Four small budgets cover most first agent workflows:

These are design controls, not magic numbers. Start with a deliberately small allowance, measure ordinary runs, and change the budget from observed evidence. If the budget is regularly exhausted, do not simply increase it. First ask whether the task is too broad, the input is noisy, the tool result is ambiguous, or the agent lacks a clear success test.

• **Time:** a maximum wall-clock duration, such as two minutes for a local document triage run. When it expires, stop and preserve the partial result rather than silently extending the run.
• **Steps:** a maximum number of model turns or tool calls. A loop that needs 80 calls to answer a supposedly narrow question is telling you something important about the task design.
• **Data:** maximum input size, files, rows or records per run. This protects both compute and privacy; “the whole folder” is rarely a useful default.
• **Downstream work:** maximum queued actions, retries or proposed changes. A run may be allowed to suggest five changes while still being forbidden to apply any of them automatically.

Make stopping a normal result

A safe stop should not look like a crash. Record a compact outcome such as `completed`, `stopped-time-limit`, `stopped-step-limit`, `needs-review` or `failed`. Include the input version, how much of the budget was used, what was produced, and the next human action.

This gives the person operating the workflow something better than a confident paragraph. They can see that an agent inspected 12 of a permitted 20 files, made three calls, found two possible matches and stopped before changing anything. The result is incomplete, but it is legible. An incomplete result that says so is safer than a complete-sounding result that quietly ran out of time.

NIST's AI Risk Management Framework treats testing and monitoring as continuing activities, not a one-time launch ceremony. It also says that limitations should be documented and that systems should be able to fail safely. For a small agent, that means testing the boundary itself: what happens when the clock expires halfway through a tool call, the model returns malformed output, the input limit is reached, or a downstream service rejects a retry?

A worked example: household maintenance triage

Imagine a local agent that helps sort maintenance notes into three labels: investigate, routine and ignore. It may read a user-selected directory of text files, but it may not edit files, contact a service, or control a device.

Before the first run, define the contract:

The last line is an authority boundary. The file, call and time limits are consumption boundaries. If the agent reaches the tenth proposal, it stops even if more files remain. If its output is malformed twice, it stops and shows the raw problem to the operator rather than inventing a label. A human can review the proposed changes, rerun the task with a different budget, or abandon it without having to unwind an unknown number of edits.

This is not a claim that the labels are accurate. Accuracy still needs examples, evaluation and a person who understands the domain. The budget only limits the blast radius of being wrong or inefficient.

• Read at most 30 files, each no larger than a stated size.
• Make at most 10 model calls and run for no longer than three minutes.
• Produce at most 10 proposed labels, each linked to the source filename and a short reason.
• Retry one transient tool failure, then stop with an error state.
• Write the proposal to a new review file; never overwrite the notes it read.

Pair budgets with a recovery path

Every stop condition needs a next action. Time expiry might mean “review the partial report”. A rate limit might mean “wait and retry once after checking the service”. A malformed response might mean “switch to a manual checklist”. A queue limit might mean “leave the rest pending rather than dropping it”.

Do not make retries the default answer to every failure. A retry is another consumption event, and repeating an unsafe action can multiply the harm. Give each retry a reason, a cap and an idempotency check where the downstream system supports one. For outward-facing, destructive or costly actions, keep the proposed action separate from the approved execution even when the agent has plenty of budget left.

OWASP's mitigation guidance names rate limits, timeouts, throttling, logging, graceful degradation and restrictions on queued or total actions. NIST's Manage guidance adds a useful operational habit: define thresholds for bypassing or deactivating a system, and keep a recovery plan when behaviour or performance changes. The durable principle is not “use these exact numbers”. It is “make the stop observable before you need it”.

Know where this advice stops

A budget cannot prevent prompt injection, incorrect reasoning, privacy mistakes or a dangerous tool permission. It can reduce how far a bad or stuck run travels. It also cannot tell you the right cost or latency threshold for a particular household, service or organisation; those are decisions about consequence, resources and acceptable delay.

Local execution may reduce dependence on a hosted service, but it does not remove the need for limits. A local model can still consume memory, fill storage, monopolise a machine, expose data to a tool, or produce a large queue of bad suggestions. “It runs on my own hardware” is not a safety case.

What we would do next is take one agent task and write its four budgets on a card: time, steps, data and downstream work. Add the exact result when each budget is reached and the person or process that owns recovery. Run it once with a small allowance. If the workflow cannot explain how it stops, it is not ready for more authority.

Sources and limits

This article synthesises the sources below into a practical explanation. It is not a security standard, legal advice, or a guarantee that guidance current at review time still applies — check the review date above against your own situation.

1. LLM10:2025 Unbounded Consumption — OWASP Gen AI Security Project. Describes uncontrolled inference as a source of service degradation and unexpected cost, and recommends input limits, rate limits, timeouts, monitoring, graceful degradation and limits on queued or total actions.
2. AI RMF Core — National Institute of Standards and Technology. Describes continuous risk management, testing before deployment and during operation, documenting limitations, safe failure and response or recovery planning.
3. AI RMF Playbook — Manage — National Institute of Standards and Technology. Recommends monitoring, thresholds for bypass or deactivation, recovery planning and change management when AI behaviour or performance changes.

More from Server Attic

• 2026-09-04

  A web page is not permission for your AI agent

  An agent can use a page, document or message as evidence without treating instructions inside it as authority. Design that boundary before connecting powerful tools.

• 2026-08-28

  Automate disk cleanup only after you can explain the growth

  A full disk invites a hurried deletion. A safer home-server workflow identifies the owner, growth rate, reclaimable data, retention need and recovery path before cleanup becomes automatic.

• 2026-08-26

  Local AI stays private only if the whole workflow does

  Running a model on your own machine can remove one remote data path, but prompts, tools, logs, saved outputs, backups and sharing still decide where information travels.