A Practical Error-Logging Pattern for Model-Backed Applications

The question

When a model-backed application fails, what does the log actually tell you?

In most codebases I have reviewed, the answer is: the exception that was thrown, at the line that threw it. That is an exception log. It is not an error log for a model-backed system, because the failures that matter most in these systems do not throw.

The empty-but-billed 200 does not throw. The refusal parsed as data does not throw. The silently ignored sampling parameter does not throw. The retry that succeeded after a 6% first-attempt failure rate does not throw — it produces a 200 and a success metric. A logging layer built around exceptions is, by construction, blind to the majority of the failure modes this series has documented.

This post is the logging pattern I use instead: log the decisions, not the exceptions.

The short answer

A model-backed application has four logs, and teams usually have three of them:

The first three describe events. The fourth describes judgments — the classification of an error, the outcome of a validation, the decision to accept or retry, the reason a response was or was not written. Without it, the other three describe what happened but not why the system did what it did, and the gap between those two things is where every incident review stalls.

The pattern below is a structured event, written at each decision boundary, with a stable taxonomy and a correlation ID that ties the four logs together.

Why exception-centric logging fails here

The failures in this series share a structural property: they occur at a point where the code chose to continue. The exception log only fires when code chose to stop.

Seven failure classes, zero exceptions. The exception log is not wrong. It is measuring a different question — did the process continue? — when the operational question is should it have?

The decision record answers the second question, because it is written at the point where the decision was made.

The four decision points to log

There are four points in a model call where the application makes a judgment that a future responder will need to reconstruct. Log all four, always, in the same record.

1. Error classification. When a call fails, the tier matters more than the code. error_tier ∈ {deterministic, transient, ambiguous}, per the retry analysis. This is the classification the code acted on, recorded at the moment it acted.

2. Validation outcome. When a response returns, the gates from the schema validation pattern produce a typed result: parse, schema, domain, acceptance — pass or fail, with the failing gate named. accepted_for_write is the boolean the database depends on.

3. Retry decision. retried_by_client, the attempt number, and — critically — idempotent at the moment of the decision. A retry after a non-idempotent write is a different event from a retry after a read, and the record has to say which one occurred.

4. Acceptance decision. accepted_by_application separates “the API succeeded” from “we chose to use the output.” These are the two fields most often conflated and the two most useful when separated.

Everything else — model ID, token counts, latency, cost — is context for these four judgments. The judgments are the log.

The record

One structured event per logical request, with per-attempt entries nested inside it. This is the shape I use:

json

{
  "schema_version": "1.0",
  "logical_request_id": "req_01J...",
  "trace_id": "4bf92f3577b34da6",
  "service": "extraction-worker",
  "env": "prod",
  "started_at": "2026-10-20T14:02:11.402-05:00",
  "model": {
    "model_id": "gemini-4-argon",
    "call_surface": "interactions.create",
    "sdk": "google-genai/2.30.1",
    "thinking_level": "medium",
    "max_output_tokens": 8192,
    "streaming": false,
    "params_sent": ["temperature", "top_p"],
    "params_effective": []
  },
  "input": {
    "prompt_sha256": "…",
    "dataset_item_id": "…",
    "input_tokens": 0
  },
  "attempts": [
    {
      "attempt": 1,
      "http_status": 503,
      "error_code": "UNAVAILABLE",
      "error_tier": "transient",
      "retryable": true,
      "idempotent": false,
      "elapsed_ms": 0,
      "retry_after_ms": null
    },
    {
      "attempt": 2,
      "http_status": 200,
      "error_code": null,
      "error_tier": null,
      "retryable": null,
      "idempotent": false,
      "elapsed_ms": 0,
      "retry_after_ms": null
    }
  ],
  "outcome": {
    "finish_reason": "STOP",
    "parse_ok": true,
    "schema_valid": true,
    "domain_valid": false,
    "domain_error": "total_cents violates ge=0",
    "refusal_detected": false,
    "output_empty": false,
    "truncated": false,
    "accepted_for_write": false,
    "rejection_reason": "domain"
  },
  "usage": {
    "input_tokens": 0,
    "thinking_tokens": 0,
    "output_tokens": 0,
    "cached_tokens": 0,
    "cost_usd_total": 0.0,
    "cost_usd_first_attempt": 0.0,
    "cost_usd_retries": 0.0
  },
  "duplicate_risk": "write_after_partial",
  "reconciliation_required": false,
  "payload_sha256": "…",
  "payload_retained": true,
  "evidence_level": "runtime"
}

The fields doing the most work are the ones most teams omit:

• params_sent vs. params_effective — on Argon, sampling parameters land in the first list and not the second. That divergence is the compatibility story, and it is invisible if only the request is logged.

• error_tier and idempotent — the two fields that determine whether a retry was correct. Without them, the attempt list is a list of status codes with no reasoning attached.

• rejection_reason — an enum, not a message. It makes failures countable by category, which is the input to the decision of whether the model, the schema, or the mapping needs to change.

• cost_usd_first_attempt — the cost of the attempt that was abandoned. This is what makes the invisible retry visible.

schema_version is not decoration. The record shape will change, and a record whose shape is unknown is a record that cannot be queried historically.

Design rules that keep the pattern usable

Rule 1: log at boundaries, not inside wrappers. A retry wrapper that logs internally produces one log per attempt with no view of the logical request. Log at the caller, where the four decisions are made, and nest the attempts inside. The wrapper should return attempt metadata, not write it.

Rule 2: hash payloads, don’t store them by default. prompt_sha256 and payload_sha256 give you correlation and deduplication without storing user content in a log system with different retention and access rules than your database. Store the raw payload only on rejection (payload_retained: true), where the diagnostic value outweighs the privacy cost, and apply the retention policy of the durable store, not the log store.

Rule 3: sample successes, never sample failures. A successful call is routine and can be sampled (or logged at reduced detail). A failure is the evidence. Sampling failures at 1% means the incident review has 1% of the data it needs.

Rule 4: separate the decision record from the business record. The decision record is for operators; the business record is for the application. If a rejection is written to the business table, it is no longer a rejection — it is data. Keep them in different stores with different schemas.

Rule 5: never log credentials, ever. No API keys, no auth headers, no account identifiers beyond what the audit requires. This is the same rule as the site’s contact policy, applied to your own logs: a log store that contains a key is a credential leak waiting for a permissions mistake.

Rule 6: make the taxonomy an enum in code, not a string in a message. error_tier, rejection_reason, and duplicate_risk should be types the compiler enforces. A free-text field that says “temporary” today and “transient” tomorrow is not a taxonomy.

The three queries the log must support

A logging pattern is only practical if it answers the questions an operator actually asks. Three queries define the minimum:

Query 1 — What is the first-attempt failure rate?

sql

SELECT
  COUNT(*) FILTER (WHERE first_status >= 500) / COUNT(*)::float AS first_attempt_5xx_rate,
  COUNT(*) FILTER (WHERE array_length(attempts, 1) > 1) / COUNT(*)::float AS retried_rate
FROM decision_log
WHERE started_at > now() - interval '24 hours'
  AND model_id = 'gemini-4-argon';

This is the number the invisible retry hides. The reported success rate is computed over logical requests; the first-attempt failure rate is computed over first attempts. They answer different questions, and both are needed.

Query 2 — What is the rejection distribution, by reason?

sql

SELECT rejection_reason, COUNT(*)
FROM decision_log
WHERE accepted_for_write = false
GROUP BY rejection_reason;

A spike in schema suggests a prompt or schema change; a spike in domain suggests a mapping bug; a spike in acceptance suggests a model or prompt behavior change. The distribution, not the total, tells you which.

Query 3 — Which requests have duplicate risk?

sql

SELECT logical_request_id, duplicate_risk, reconciliation_required
FROM decision_log
WHERE duplicate_risk <> 'none'
  AND reconciliation_required = true;

This is the list someone has to work through. Without duplicate_risk recorded at decision time, the list is reconstructed from database diffs, which is not a recovery procedure.

Three queries. If the log cannot answer them without a join across three systems and a manual reconstruction, it is not a decision record — it is an exception log with extra fields.

Failure cases this pattern prevents

The empty-output pass-through. A 200 with an empty output at the streaming ceiling. The exception log was empty. The decision record shows accepted_for_write: false, rejection_reason: "acceptance", output_empty: true. The failure becomes a row in the rejection distribution instead of a gap discovered weeks later.

The invisible retry. A 99% reported success rate over a 6% first-attempt failure rate. The exception log shows one line. The decision record shows two attempts, error_tier: "transient", cost_usd_first_attempt non-zero, and cost_usd_retries that the billing record never attributed. Query 1 surfaces the number the success rate hid.

The duplicate write. A timeout at 30 seconds, a retry at 45, both written. The decision record shows idempotent: false at the moment of the retry decision and duplicate_risk: "write_after_partial". Query 3 lists the request. The uniqueness constraint on logical_request_id prevents the second write, and the record explains why it was needed.

The refusal stored as data. A well-formed envelope containing a safety block, parsed and stored. The decision record shows refusal_detected: true, accepted_for_write: false, rejection_reason: "acceptance". The branch exists because the record shape required the branch to be decided.

The parameter drift. A deprecated parameter begins returning 400 in a later generation. The decision record shows params_sent non-empty and params_effective empty, then error_tier: "deterministic" on the first 400. The failure is attributed to the parameter within one query rather than after a bisect.

None of these are logging improvements. They are decision points that become visible because the log was designed around decisions.

What this pattern does not prove

It does not prove the system is correct. A perfect decision record for a wrong decision is a well-documented error — useful for the postmortem, not a substitute for the fix.

It does not replace metrics, traces, or the business record. It complements them: metrics aggregate, traces follow a request across services, the business record stores what the application chose to keep. The decision record explains the choice.

It does not prove that the taxonomy is complete. The enums are a hypothesis about the failure classes that occur, and they should be revised when a failure occurs that the taxonomy cannot name. A rejection reason of "other" is a signal that the enum needs a new value, not that the log needs more free-text.

It does not solve retention and access. A record containing hashed payloads, user-level identifiers, and model outputs is still sensitive; the retention policy and access controls are a separate decision from the schema.

What it does establish is the minimum that makes the rest of the series’ failure patterns visible in production: four decisions logged per request, a stable taxonomy, a correlation ID, and three queries the log can answer. Without that, the failure modes documented here are not prevented — they are merely possible to describe after the fact.

A working demo is evidence. It is not a production system. An exception log is evidence that something threw. It is not evidence that anything was decided — and in a model-backed system, the decisions are where the failures live.

For AI tool features, rules and availability, refer to their official sites.