When an AI Agent Does Something Wrong, You Need the Tape

A modern AI agent is not a single question and answer. It is a sequence: it reads a request, decides to look something up, calls a tool, reads the result, decides what to do next, calls another, and eventually acts or replies. When the outcome is good, the sequence is invisible and nobody thinks about it. When the outcome is wrong — a bad refund issued, a customer given false information, a record updated incorrectly — the sequence is suddenly the only thing that matters, and if it was not recorded, the honest answer to "why did it do that" is that nobody can say. This is why observability, long a discipline for ordinary software, is being retrofitted onto AI agents with some urgency. The goal is a trace: for each run, what the agent was asked, what it retrieved, which tools it called with which inputs, what those tools returned, what the intermediate decisions were, and what version of the model and the prompts produced them. With that record, a bad outcome becomes a diagnosis — the retrieval returned stale data, the tool was called with the wrong argument, the model misread an ambiguous instruction — and a fixable one. Without it, teams are reduced to re-running the agent and hoping the failure reappears, which for a non-deterministic system is a poor bet. There are two audiences for the record, and conflating them causes trouble. One is engineering, which needs enough detail to debug and improve. The other is compliance and accountability, which needs a durable, tamper-evident account of what an automated system did on a customer's behalf, increasingly because a regulation or a contract requires it. These pull in different directions on retention, on access, and especially on sensitive data: the trace that is most useful for debugging is the one that captured everything, including the personal information that must not simply be logged in the clear and kept forever. The mature deployments decide deliberately what to record, redact what they must, and separate the debugging stream from the audit record. The practical takeaway for anyone putting an agent into production is that the logging is not an afterthought to add once something goes wrong — by then the run that would have explained it is gone. It is part of the initial design, on the same footing as the agent's ability to act at all. An agent you cannot inspect is an agent you cannot debug, cannot audit, and ultimately cannot defend, and "the AI decided" has never been an acceptable answer to a customer, a regulator or a court.