Server AtticLearning Harbour

AI Foundations · Module 2 · Lesson 3 of 3

Choosing when not to use AI

A practical decision habit for matching a task to a safe AI use pattern — including recognising the tasks where AI should not be used unsupervised, and why that recognition needs a habit rather than good intentions alone.

Lesson · 15–20 minutes · Text-first

By the end, you can

  • Apply a three-question decision habit to choose whether a task is safe for AI, safe for AI with a named review step, or not appropriate for AI without a named human decision-maker (AF-2).
  • Explain why overreliance on AI-generated content is a documented, recurring risk rather than just a matter of individual carelessness (AF-1).
  • Select a safe AI use pattern for a set of sample tasks, including at least one task that should not use AI unsupervised (AF-2).

Before you start

This is Module 2, Lesson 3 of the AI Foundations course, the last lesson before Module 3's work on writing clearer AI instructions. It builds on this module's first two lessons — drafting and summarizing (Lesson 1), classification and analysis (Lesson 2) — and on Module 1's point that a fluent AI output is not automatically a correct one. This lesson does not require coding or technical setup.

A three-question decision habit

Every task in this module so far has had a review step: something specific a person checks before the AI's output is used. This lesson turns that pattern into a habit you can apply to a new task before you start, not just after. Ask three questions:

1. **Is everything the AI needs already in front of it?** Does the task stay inside "continue this supplied instruction or source text usefully" (Module 1 Lesson 2), or does it depend on an external, current or private fact the AI has no way to check? 2. **If the AI gets it wrong, how easy is that to notice, and how costly or reversible is acting on it before anyone checks?** A wrong word in a casual draft is easy to notice and cheap to fix. A wrong figure in a report that triggers a spending decision is neither. 3. **Who is actually going to check the output, and against what, before it is used?** "Someone will probably notice" is not a review step. A named person checking a named thing is.

The answers sort a task into one of three patterns: **use directly**, with only a light skim-check, because the stakes of an unnoticed error are low; **use with a named review step**, because the output could plausibly be wrong or the stakes of acting on it are meaningful; or **do not delegate this to AI without a named human decision-maker**, because the task itself requires a specific person's judgement or authority, not just a checked output.

Overreliance: a documented risk, not just carelessness

It is tempting to treat getting this wrong as a matter of one person being careless. The Open Worldwide Application Security Project's generative-AI security effort names this pattern directly, as its own defined risk: **overreliance** "occurs when users place excessive trust in LLM-generated content, failing to verify its accuracy." The same guidance recommends organisations "implement human oversight and fact-checking processes, especially for critical or sensitive information," and encourages cross-checking AI output against trusted external sources. Naming overreliance as a documented, recurring risk — not a one-off individual mistake — is exactly why this lesson teaches a repeatable three-question habit rather than simply asking you to "be careful."

Accountability doesn't transfer to the tool

Recall Module 1's opening definition: an AI system is "an engineered or machine-based system that can, for a given set of objectives, generate outputs such as predictions, recommendations, or decisions." Nothing in that definition includes verifying its own outputs or being accountable for what happens next. The same NIST framework names being "accountable and transparent, explainable and interpretable" among the characteristics of a trustworthy AI system — read that carefully: it describes a design goal for the system, not a transfer of responsibility away from the person using its output. Even a well-designed, well-documented AI tool does not become the decision-maker. If a classification is wrong, a draft misrepresents something, or an analysis is acted on without checking, the responsibility for what happens next sits with the person who used the output — the same principle Module 1 Lesson 3 applied to a convincing synthetic voice message, now applied to every task type in this module.

When not to use AI at all — or not without a named human decision-maker

Three situations call for the third pattern from the decision habit above:

  • **The action is high-stakes and hard to reverse, and no named person is reviewing it before it happens.** Auto-approving a large refund, sending a legally binding commitment, or taking an irreversible financial action without a human check are all in this category — not because AI cannot help draft or analyze toward the decision, but because the decision itself needs a named person's sign-off. This is the same concern OWASP's generative-AI security effort raises about giving a system too much unsupervised autonomy: its guidance on excessive agency recommends that teams "utilise human-in-the-loop control to require a human to approve high-impact actions before they are taken."
  • **The task depends on a fact the AI cannot verify, the stakes of being wrong are real, and there is no way to check it before acting.** This is Module 1 Lesson 2's "no automatic lookup" limit returning with consequences attached — a current figure, a private record, a live status that must be right and cannot be confirmed in time.
  • **A policy, law or company rule assigns this judgement to a specific named role** — a legal sign-off, a safety judgement, a hiring decision. AI can help that person prepare, draft or analyze toward their judgement, but it cannot stand in for their accountability. Module 1 Lesson 2 introduced NIST's companion profile addressing risks specific to generative AI systems; the decision habit in this lesson is a practical, everyday response to exactly that risk territory, not a one-off rule for refunds alone.

A worked example

A team lead is deciding how to use AI across five requests this week: drafting a thank-you note to a colleague; summarizing a 20-page supplier contract before it is signed; classifying support tickets by urgency; deciding whether to auto-approve a large customer refund; and analyzing sales data to suggest which product line to discontinue.

Applying the three-question habit: the thank-you note stays inside supplied text, the stakes of a small error are low, and a quick read before sending is enough — **use directly**. The contract summary stays inside supplied text, but the stakes of missing a clause before signing are high, so it needs a named person reading the actual contract against the summary before signing — **use with a named review step**. Ticket classification is similar: low stakes per item, with a spot-check and an escalation path for low-confidence cases — **use with a named review step**, at the batch level rather than per item. Auto-approving a large refund is high-stakes, hard to reverse, and is exactly the kind of judgement a named role (a manager or finance lead) should retain — **do not delegate this to AI without a named human decision-maker**; AI can flag or summarize refund requests for that person, but should not approve them unsupervised. The sales analysis has real stakes if a product line is discontinued on a wrong conclusion, so the analysis is a useful starting point, but the decision itself needs the underlying figures checked by a named person before it is acted on — **use with a named review step**.

Accessibility notes

This lesson is text-first, with no images, audio, video or downloadable artifacts. The practice exercise's model answer sits behind a native disclosure control that is reachable and operable by keyboard and correctly announced by screen readers. The knowledge check uses native radio-button inputs with a visible question and options, and posts its result to a live status region so assistive technology announces the outcome without a page reload.

Practice

Choose a safe use pattern: an events coordinator's week

An events coordinator preparing a 300-person community festival is deciding how to use an AI tool for five tasks: (1) drafting social-media posts announcing the festival's food stalls, from a list of confirmed stallholders she supplies; (2) summarizing 60 volunteer feedback forms from last year's festival into the top five recurring complaints; (3) classifying incoming vendor enquiry emails into 'catering', 'entertainment' and 'logistics' folders; (4) signing off the crowd-safety plan required by the venue, which the local safety officer must approve; (5) analyzing last year's hourly attendance figures to recommend how many entry stewards to schedule per shift.

  1. For each of the five tasks, choose one pattern: 'use directly', 'use with a named review step', or 'do not delegate to AI without a named human decision-maker'.
  2. For the task you marked 'do not delegate to AI without a named human decision-maker', explain which of this lesson's three situations applies and why.
  3. For one task you marked 'use with a named review step', name the specific person or role who should do the check, and what they should check it against.
  4. Write one sentence explaining to the coordinator why 'the AI sounded very sure' is never, on its own, one of the three questions in the decision habit.
Compare with a bounded first version

Social-media drafts: use directly — the stallholder list is supplied, stakes of a small wording slip are low, and a quick read before posting (checking stall names against the supplied list) is enough. Feedback-form summary: use with a named review step — compression across 60 forms can drop or overweight complaints, so the coordinator should trace each of the five listed complaints back to actual forms before acting on them. Vendor email classification: use with a named review step at the batch level — low stakes per email, but a spot-check plus a person handling anything the tool sorts with low confidence stops misfiled enquiries going unanswered. Crowd-safety plan sign-off: do not delegate to AI without a named human decision-maker — this is the lesson's third situation: a rule assigns this judgement to a specific named role (the safety officer), and it is also high-stakes and hard to reverse; AI can help draft or check the plan's text, but the sign-off itself must stay with the officer. Steward-scheduling analysis: use with a named review step — the recommendation is a starting judgement; the coordinator should check it against the actual attendance figures (and the venue's minimum staffing rules) before publishing the rota, because understaffing an entry gate has real safety consequences. A fair closing sentence: how sure the AI sounds is never one of the three questions — the habit asks what the tool actually had in front of it, what a wrong answer costs, and who checks it, not how confident the answer reads.

Knowledge check

Try the idea

A team is considering letting an AI tool automatically approve customer refund requests over a set amount, with no person reviewing each approval before it happens. Using this lesson's decision habit, what is the most accurate assessment?
Low-stakes practice only. This does not score, block progress or create a learner record.

Sources and limits

This lesson synthesises the sources below into a practical learning model. It is not a security standard, legal advice or a guarantee that any particular agent design is safe.

  1. LLM09:2025 Misinformation — OWASP Gen AI Security Project. Defines overreliance as occurring when users place excessive trust in LLM-generated content without verifying its accuracy, and recommends human oversight and fact-checking, especially for critical or sensitive information.
  2. Artificial Intelligence Risk Management Framework 1.0 — NIST AI Resource Center. Names accountable, transparent, explainable and interpretable operation among the characteristics of trustworthy AI systems, and frames an AI system as an engineered system that generates outputs such as predictions, recommendations or decisions rather than a self-verifying decision-maker.
  3. LLM06:2025 Excessive Agency — OWASP Gen AI Security Project. Recommends human-in-the-loop control requiring a person to approve high-impact actions before an LLM-connected system takes them.
  4. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile — NIST. A NIST companion profile to the AI Risk Management Framework addressing risks unique to generative AI systems, referenced here as this course's ongoing generative-AI risk grounding.