Observability and logging
Also called: tracing, monitoring, audit trail
Recording what the system received, what it did, and why — inputs, model outputs, tool calls, and decisions — so behaviour can be understood after the fact rather than guessed at.
When an AI feature does something wrong, logs are the difference between 'we can see exactly what happened and fix it' and 'we have no idea.' They're also what makes an injected action, a bad output, or a cost spike visible at all — and often what accountability and regulation require.
Storing everything forever. Good observability is deliberate about what it keeps and for how long, balancing the need to explain behaviour against the privacy cost of retaining user content.
- Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1U.S. National Institute of Standards and Technology · 2023-01-26
- ISO/IEC 42001:2023 — Information technology — Artificial intelligence — Management systemInternational Organization for Standardization · 2023-12-18
First-pass citations, limited to primary sources; a reviewer will broaden and verify these before this entry leaves draft.