Local AI stays private only if the whole workflow does

Local is a useful property, not a privacy verdict

Running a language model on a computer you control can remove an important remote hop. That is valuable, especially when the alternative sends every prompt to a hosted service. It does not prove that the whole task stayed private.

The model is only one stop. A prompt may begin in a browser extension, pass through an agent, call a web search or document connector, appear in a log, become a saved chat, enter a backup and finish in a message to somebody else. If any one of those steps crosses a boundary you did not intend, “the model ran locally” is still true and “the workflow stayed local” is false.

Ollama's current documentation illustrates the distinction without settling it for every product. It says prompts and data are not seen by Ollama when a model runs locally, while cloud-hosted models process prompts and responses remotely. It also documents how to disable its cloud features. That is evidence about a particular configured runtime, not a promise about every interface, plug-in or tool attached to it.

Follow one prompt, not one label

Imagine asking a local model to turn fictional workshop notes into a short maintenance checklist. The text contains no names, addresses, account details or real infrastructure. Trace that deliberately neutral prompt through six places.

This map separates facts from unknowns. “The runtime binds to this address” is a fact you can inspect. “No extension sends telemetry” is an unknown until you check the extension and observe its traffic or documentation. “Keep the default workflow offline” is Server Attic judgment: a cautious starting point, not a universal technical requirement.

• **Input capture:** Where did you type it? A local terminal and a browser-based chat can use the same model while storing history differently. Check the interface, not just the model name.
• **Model runtime:** Is the selected model actually local, or can the same application select a cloud model? Does the service listen only on the device, or is it reachable from the local network or a proxy?
• **Tools and connectors:** Can the workflow use web search, email, cloud storage or remote document retrieval? A local model can still send a query or extracted passage to a remote tool.
• **Logs and histories:** Which component records prompts, tool arguments, errors or outputs? NCSC guidance treats prompts and logs as security-relevant assets and says logs should be protected as sensitive data.
• **Saved outputs and backups:** Where are conversations, exports, indexes and temporary files written? Which backup copies them, who can restore them and when do they expire?
• **Sharing:** Does the result remain a draft, or is it pasted into a ticket, message or shared folder? Privacy can be lost after perfectly local inference.

A worked audit with a harmless prompt

Start with a prompt that is safe even if your assumptions are wrong:

“Turn these fictional equipment notes into a five-step checklist. Use placeholders for names and locations. Do not call tools or save the result.”

Before running it, disconnect optional connectors and select a model explicitly marked local. Record the application, model and active tools. Run the prompt, then inspect the places your setup says it stores conversations and logs. Check whether a new history, cache or export appeared. If practical, repeat once with history disabled and compare the result.

Next, check network behaviour using the operating system or network controls you already trust. You are looking for an explanation of every connection, not trying to prove the impossible claim that software can never communicate. Model downloads, update checks and time synchronisation may be legitimate; a web-search request containing part of the prompt is a different data path. Document what you observed and what you could not observe.

Finally, delete the test conversation through the normal interface and verify what that action means. Did it remove only the visible history? Is a local file, log rotation, search index, recycle bin or backup copy retained? OWASP recommends clear policies for data retention, use and deletion because an invisible copy changes the real boundary.

The eight-minute privacy checklist

NCSC guidance also explains the tension around logging: input logs can support audit, investigation and remediation, but they remain subject to privacy and data-protection requirements. The answer is not “log everything” or “log nothing.” Keep the smallest record that supports the decision, restrict access and avoid duplicating raw sensitive prompts when metadata or a redacted reference is enough.

• **Minimise the input:** Use a neutral sample first, then include only the data the real task needs.
• **Name the runtime:** Confirm the exact model and whether it is local or cloud-backed at the moment of use.
• **List every tool:** Disable connectors that the task does not require; do not treat “local model” as “offline agent.”
• **Check the listener:** Know which device or network can reach the runtime and what access control protects it.
• **Find the records:** Locate chat history, prompt logs, tool traces, temporary files and generated outputs.
• **Set retention:** Decide what should be kept, for how long, and how deletion and backup expiry work.
• **Protect the output:** Classify generated text by the sensitivity of its inputs before saving or sharing it.
• **Write down unknowns:** If you cannot verify telemetry, plug-in behaviour or deletion, keep sensitive data out until the owner can answer.

What local execution cannot promise

This checklist does not prove anonymity, legal compliance or freedom from compromise. A local device may be shared, infected, poorly backed up or exposed on the network. A model may reproduce sensitive context that the application supplied. An encrypted disk does not protect data already copied into a message, and deleting a chat does not necessarily delete its backups.

The practical claim is narrower: local inference gives you a useful place to remove a remote processor and exercise more control. Privacy comes from verifying the complete route, limiting what enters it and protecting every copy that remains.

What we would do next

Draw one line from prompt to final output for a real low-risk task. Put a box around the interface, runtime, tools, logs, saved files, backups and sharing destination. Mark each connection as local, remote or unknown, then remove one unnecessary path.

For the deeper hands-on version, the Local AI Agents course treats tools, memory, permissions, evidence and recovery as separate decisions. The habit to keep is simpler: before trusting the word “local,” follow the data all the way home.

Sources and limits

This article synthesises the sources below into a practical explanation. It is not a security standard, legal advice, or a guarantee that guidance current at review time still applies — check the review date above against your own situation.

1. FAQ — Ollama. Distinguishes locally run prompts from cloud-hosted model processing, documents a local-only setting and shows that the runtime can be exposed beyond loopback by configuration.
2. Guidelines for secure AI system development — Secure development — National Cyber Security Centre. Treats prompts, data, logs and generated content as assets whose location, sensitivity, access and recovery controls must be understood.
3. Guidelines for secure AI system development — Secure operation and maintenance — National Cyber Security Centre. Explains why input logging may support audit and investigation while remaining subject to privacy and data-protection requirements.
4. LLM02:2025 Sensitive Information Disclosure — OWASP Gen AI Security Project. Describes how sensitive information can enter an AI application's context and later be exposed through output, and recommends clear retention, usage and deletion policies.

More from Server Attic

• 2026-08-28

  Automate disk cleanup only after you can explain the growth

  A full disk invites a hurried deletion. A safer home-server workflow identifies the owner, growth rate, reclaimable data, retention need and recovery path before cleanup becomes automatic.

• 2026-08-24

  Draft first, send second

  AI can help shape an outbound message without deciding who receives it or pressing send. Use a bounded draft, a deliberate human check and a clear correction path.

• 2026-08-14

  Give your AI agent a budget

  Permissions limit what an agent may touch. A separate budget for time, steps, data and queued work limits how much it can consume before it must stop.