Seeing What Your AI Service Is Actually Doing

You cannot debug what you did not record

When someone reports a wrong answer, the questions are what was asked, what was retrieved, which prompt and model were used, and what came back. All four have to have been recorded at the time.

Reproducing an AI failure after the fact is nearly impossible without the trace. Recording it costs almost nothing and it is the difference between diagnosing and guessing.

What to record per call

1. The input, or a reference to it
2. The retrieved context — which passages, from which documents
3. The prompt version and model version
4. The raw output, before any post-processing
5. Latency, tokens and cost
6. Any validation failure or retry

Metrics that matter

Correlate with the originating request

A single user action may trigger retrieval, several model calls and a queued job. Without a shared identifier across all of them, tracing what happened means guessing.

Generate it once at the boundary and carry it everywhere, including into queued work.

Mind what you log

• Retrieved passages may contain personal or sensitive data
• Log references rather than content where the material is sensitive
• Apply a retention period automatically
• Remember these logs are in scope for data requests
• Restrict who can read them

Frequently asked questions