LLM observability and safe traces
Trace identity
A useful trace connects a request to the versions and outcomes that explain it. Preserve the request ID, prompt revision, source revision, final status and supplied fictional usage units required by the task. Those units are teaching data, not measured provider tokens. A status alone is insufficient. A log line saying only “failed” cannot distinguish retrieval failure from a denied write.
In a fictional freight application, a tool times out after a write request. Preserve the operation identity so recovery can reconcile the uncertain result. Retrying with a new identity can create a duplicate action even when the original trace looks like a failure.
Data minimization
Trace metadata is not permission to store every prompt or document. Redact sensitive fields according to the supplied policy and retain the non-sensitive identifiers needed for investigation. Test nested fields as well as top-level values.
This notebook produces local records from fictional events. It does not connect an observability vendor or measure a production service. Continue with redaction, uncertain tool outcomes and alert denominators.