What you saw, what you think, what you changed

The fix that fixed nothing

A service on your home server keeps restarting. You paste the last hundred log lines into an AI assistant and ask what is wrong. The answer arrives in seconds: the process is running out of memory. You ask a follow-up about the restart timing, and the reply opens with "since the service is being killed for memory". Two messages later you raise the memory limit, restart the service and wait. The restarts continue.

Nothing in that exchange was a lie. The assistant offered a plausible cause, you did not contradict it, and it carried that cause forward as settled. The failure was structural, not factual: a sentence that began as a guess was read back as a measurement, and a change was made on the strength of it.

Three registers, not one narrative

Troubleshooting is normally described as a loop: from a set of observations and a working understanding of the system, you propose candidate causes and test them. Google's Site Reliability Engineering book sets out that iterative hypothesise-and-test shape in detail. The loop only works while hypotheses stay labelled as hypotheses, and a chat session makes that hard, because observations, guesses and instructions all arrive as the same kind of paragraph.

The repair is to keep three records rather than one story:

Once those columns exist, the earlier mistake is visible on the page: an inference had been filed as an observation, and an action built on top of it.

• **Observation.** Something a named tool reported at a named time, which you could reproduce by running the same thing again. "The service manager recorded six restarts between 02:10 and 02:40."
• **Inference.** A claim about cause or meaning, with an owner and a confidence. "The assistant suggests memory pressure. Medium confidence, untested."
• **Action.** A change made to the system, with the value it replaced and a way back. "Raised the memory limit at 09:14; previous value recorded; revert is a one-line edit and a restart."

Why AI assistance collapses the three

Human troubleshooters have always confused a guess with a finding. Generative assistants make it easier, for reasons worth naming.

NIST's generative-AI profile describes confabulation as a system confidently presenting erroneous or false content, and adds that outputs may include "confabulated logic or citations that purport to justify or explain the system's answer", which can mislead people into trusting the output further. OWASP's LLM09:2025 entry puts the same problem from the reader's side: misinformation is output that is false or misleading while appearing credible, and overreliance is trusting it without checking. NIST also names the habit underneath — automation bias, or "excessive deference to automated systems" — and notes it can make confabulation risk worse.

Two more pressures are specific to the chat shape. Both are our reading of the mechanics rather than published findings, so treat them as inference:

Neither is something a better prompt reliably removes. Both argue for keeping the authoritative record outside the conversation, in a file you control.

• The assistant's earlier replies become part of what it reads next. Its own guess returns to it as context, indistinguishable in form from the log lines you supplied.
• Fluent prose flattens hedges. "This could be memory pressure" and "the memory pressure here" look almost identical when you are skim-reading at 2am.

A worked example, kept in a text file

The following example is invented to show the format. A backup job on a home server has started failing roughly every third night; the log is a plain file with three headings, appended in order:

That last line is the discipline the columns buy. A one-column transcript would already be saying the conflict was found and fixed.

• **O — 21:40.** Backup tool exit code 1 on three of the last nine nightly runs. Failure nights: Tue, Fri, Mon.
• **I — 21:44.** Assistant proposes the destination disk is filling up. Low confidence: no capacity figure has been read yet.
• **O — 21:47.** Destination reports 41% used, and 41% on the morning after a failure night. The capacity hypothesis is not supported.
• **I — 21:50.** Own hypothesis: the failures land on nights when a separate media scan runs. Medium confidence, based on the timing overlap only.
• **O — 21:58.** Scan schedule read from its configuration. It runs Tue, Fri, Mon. Overlap confirmed as a correlation; cause still unproven.
• **A — 22:05.** Moved the scan two hours later. Previous schedule recorded. Revert is one configuration line.
• **O — next day.** One clean backup night. Not yet decisive: the pattern needs more than a single night to distinguish it from chance.

Rules for moving between the columns

The columns only help if promotion between them is governed by something other than how confident the last sentence sounded.

• **An inference becomes an observation only when a new measurement says so.** Not when it is repeated, not when it goes unchallenged, and not when the assistant restates it more confidently.
• **Design a test that splits the hypotheses.** The SRE guidance is to prefer tests with mutually exclusive outcomes, to try the likeliest explanations first, to watch for confounding factors, and to remember that active tests can have side effects that change later results, and that some tests are only suggestive.
• **Change one thing, and record what it was before.** Two simultaneous changes produce one ambiguous result and two things to undo.
• **Keep the record as you go.** The SRE book calls keeping a living incident document the incident commander's most important responsibility, and explicitly accepts a messy one that works over a tidy one written later.
• **Cross-check anything load-bearing.** OWASP's advice to verify generated claims against trusted sources applies to the claim you are about to act on, and to nothing else in the transcript.

Where this stops helping

Sorting a session into three columns proves nothing about the contents of any column. An observation can be wrong: misread metrics and irrelevant symptoms are among the failure modes the SRE book lists, and a correctly filed bad measurement is still a bad measurement. A confirmed correlation is not a cause, however neatly it is logged.

There is also a cost. In a live outage the first duty is triage — making the system work as well as it can under the circumstances — and a complete log is worth less than a running service. Write the record retroactively then, and accept the gaps. No amount of bookkeeping substitutes for understanding the system: the columns organise evidence, they do not supply a model of how the thing works. If the underlying question is whether to believe an AI-generated claim at all, that is a separate verification problem.

What we would do next

Before the next debugging session, open an empty text file with three headings. During the session, every line you write starts with O, I or A, and nothing gets an O without a command you could run again. The first time an assistant's confident sentence has to be filed under I, the habit will have paid for itself.

Sources and limits

This article synthesises the sources below into a practical explanation. It is not a security standard, legal advice, or a guarantee that guidance current at review time still applies — check the review date above against your own situation.

1. Effective Troubleshooting, Site Reliability Engineering — Google. Describes troubleshooting as iteratively hypothesising causes and testing them, lists common failure modes including irrelevant symptoms, misread metrics and spurious correlations, gives test-design guidance, and states that triage means making the system work as well as it can under the circumstances.
2. Managing Incidents, Site Reliability Engineering — Google. States that keeping a living incident document is the incident commander's most important responsibility, that it should be editable by several people concurrently, and that a messy but functional document is acceptable.
3. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1) — National Institute of Standards and Technology. Section 2.2 defines confabulation as confidently presented erroneous content and notes outputs may include confabulated logic or citations that purport to justify an answer; section 2.7 describes automation bias as excessive deference to automated systems that can exacerbate confabulation risk.
4. LLM09:2025 Misinformation — OWASP Gen AI Security Project. Defines misinformation as false or misleading output that appears credible, defines overreliance as excessive trust in generated content without verifying accuracy, and recommends cross-checking against trusted sources and human oversight.

More from Server Attic

• 2026-08-24

  Draft first, send second

  AI can help shape an outbound message without deciding who receives it or pressing send. Use a bounded draft, a deliberate human check and a clear correction path.

• 2026-08-03

  Don't ask AI if it's right

  A confident answer is not evidence. Build a small verification plan around the claim, the consequence of being wrong, and the source or test that can actually settle it.

• 2026-09-04

  A web page is not permission for your AI agent

  An agent can use a page, document or message as evidence without treating instructions inside it as authority. Design that boundary before connecting powerful tools.