A postmortem is not a punishment. It is the document the team writes after an incident to capture what happened, why it happened, and what changes so it does not happen again. Classical Site Reliability Engineering codified the blameless postmortem two decades ago, and the template that emerged works extraordinarily well for traditional software systems: timeline, impact, root cause, contributing factors, action items. For agent incidents the SRE template is necessary but no longer sufficient. Two new fields must be added to capture the failure modes that classical software does not have. "Version drift" — the platform, model, prompt, or tool-schema version that changed without obviously changing — is the field that prevents the same silent regression from recurring. "Eval gap" — the cases the eval set did not cover that allowed the incident to slip through — is the field that converts every incident into a permanent eval set improvement. This lesson is the postmortem template that works for agent incidents in 2026, the two new fields, the workflow that turns the document into changes the team actually makes, and the cultural discipline that keeps the practice blameless.

Why the Blameless Postmortem Still Matters

Twenty years ago, SRE practitioners at Google and elsewhere converged on a discipline: when something breaks, do not ask "who caused this?" Ask "what in our system allowed this?" The question is not academic. Every minute the team spends investigating who is to blame is a minute they are not investigating what to change. And the cultural cost of blame is enormous — engineers stop reporting near-misses, stop volunteering information about their own mistakes, and stop trusting the team to handle problems as a team.

The cultural foundation

Blameless postmortems work because they assume good faith. Every human in the system is doing their best with the information they had at the time. When they made a mistake, the mistake is information about the system, not information about the human. The system allowed the mistake; the system can be changed to prevent recurrence.

This cultural foundation is non-negotiable. A team that nominally writes "blameless" postmortems but in practice points fingers in the meeting room produces postmortems that are theater. The action items get accepted formally and never executed. The next incident hits and the team has learned nothing.

How agents complicate blamelessness

For classical software, the human-in-the-loop is the engineer who wrote the code. The blameless framing is straightforward. For agents, there is a second actor in the loop: the model itself. When the agent does the wrong thing, the question of "who made the mistake" gets weird. The model "decided" to write the wrong refund. The model "followed" the injected instruction. The model "looped" instead of stopping.

The temptation: blame the model. The model is convenient because it does not have feelings. "The model hallucinated" is a sentence with no human cost.

The discipline: do not blame the model. The model is part of the system the team designed. The team chose the model. The team wrote the prompt. The team designed the eval set. The team failed to anticipate the case. The blamelessness applies to the system-the-team-built, not the model.

The postmortem is the artifact that makes incidents pay dividends. Every incident has a cost — service degradation, customer trust, engineering time. The cost is sunk; the postmortem extracts learning from it. Teams that write rigorous postmortems compound that learning across incidents. Teams that skip postmortems pay the same cost over and over for the same incident in different clothes.

The Classical SRE Template

The 2026 agent postmortem inherits the classical SRE structure and adds to it. The inherited sections:

Header

Incident identifier (e.g., INC-2026-05-16-001). Date. Author (the on-call who responded, typically). Reviewers. Status (draft, review, accepted). Severity (S1 through S4, defined consistently across the team).

Summary

One paragraph. What happened, what the impact was, how it was resolved. The summary is the only thing many stakeholders will read. It should stand alone.

Timeline

A chronological record of the incident from detection through resolution. Timestamps in UTC. Each entry: timestamp, actor, action, source-of-truth (the channel message, alert, or log entry that confirms the entry).

The timeline is the foundational artifact. Other sections refer to it. The timeline is reconstructed from the incident channel transcript, the alerting tool's history, and any system logs. Channels that were used for the incident make this easy; DM-driven incidents make it nearly impossible.

Impact

Quantified. How many users affected, for how long, with what measurable consequence. Dollar impact where possible. Customer impact in concrete terms (refunds processed wrong, support backlog, data exfiltrated). Internal impact in concrete terms (engineering hours, opportunity cost, postmortem follow-on cost).

Root cause

The factor that, if changed, would have prevented the incident. The "five whys" technique often surfaces this. The first answer ("the agent hallucinated") is rarely the root cause; the fifth answer ("the eval set did not cover the multi-purchase ambiguity case") usually is.

Contributing factors

Factors that were not the root cause but that compounded the impact or delayed detection. The alerting was tuned slightly too loosely. The on-call was new to the system. The platform vendor's status page was misleading.

Action items

Concrete commitments. Who. What. By when. Tracked in the same system as other team work. Reviewed in the postmortem follow-up.

The Two New Fields for Agent Postmortems

The classical template captures most of what is needed. Two fields are conspicuously missing for agent incidents and should be added explicitly.

Field one: version drift

A complete enumeration of every version in play at the time of the incident. The fields:

• Model version. Exact version string in use. gpt-5-2026-04-15. claude-sonnet-4-5-2026-03-20. The version pinned in the team's code or in the provider's preferred-version pointer.
• Prompt version. The version identifier of the system prompt and any user prompt template. Sha-hash of the prompt file, or a semantic version, or a tag in the prompt-versioning system the team uses.
• Tool schema version. Version identifiers for every tool definition the agent had access to. Tool descriptions, parameter schemas, enum values.
• Platform version. The exact version of every platform component: n8n version, Lindy version, LangChain SDK version, LangGraph version, Anthropic SDK version, OpenAI SDK version, observability platform agent version.
• Eval set version. The version identifier of the eval set in use at the time. Hash or tag.
• Guardrail version. Versions of any pre-call, mid-call, or post-call guardrails (Guardrails AI, Nemo Guardrails, Lakera Guard, custom).

For each version, two timestamps: when it was first deployed, and when it was last validated against the eval set. Version drift means a version that has been deployed for some time but never validated.

Why this field matters: the most common incident pattern in 2026 is a version that changed without anyone noticing. The February n8n incident lived in version drift. A model that silently upgraded behind a provider endpoint lives in version drift. A platform that auto-upgraded over the weekend lives in version drift. The postmortem cannot identify the root cause if the team does not enumerate the version state.

Field two: eval gap

A description of the case (or class of cases) that the eval set did not cover, which allowed the incident to slip through.

Every incident, by definition, slipped past the team's existing detection. Either the eval did not include a case like the failure, or the eval included a case but the case was not adversarial enough, or the eval set covered the case but the production-traffic shadow eval did not catch the regression in time. The eval-gap field forces the team to identify which.

The field has three components:

• Gap description. What case was missing or under-tested? "The eval set had no case for users with multiple recent purchases requesting an ambiguous refund. The case category 'multi-purchase refund disambiguation' is a gap."
• How the gap was discovered. Through the incident itself, through retrospective analysis, through customer complaint. Where in the timeline the gap became evident.
• Closure plan. What cases will be added to the eval set as a direct result of this incident? Specific, with target counts. "We will add five new cases to the eval set covering multi-purchase refund disambiguation: two golden (clear correct answer), two edge (ambiguous answer with right escalation), one adversarial (deliberately confusing input designed to trigger the wrong answer)."

Why this field matters: every incident is an eval improvement opportunity. The eval-gap field converts the incident into permanent eval coverage. Without it, the team fixes the immediate problem but the eval set does not grow; the next similar incident slips past for the same reason.

The Complete Agent Postmortem Template

Combining classical SRE fields with the two new agent-specific fields, the 2026 agent postmortem template:

1. Header. Identifier, date, author, reviewers, status, severity.
2. Summary. One paragraph for stakeholders.
3. Timeline. Chronological record with UTC timestamps and source-of-truth references.
4. Impact. Quantified — users, duration, dollars, downstream effects.
5. Root cause. The single factor that would have prevented the incident.
6. Contributing factors. Compounding and delaying factors.
7. Version drift. Model, prompt, tool schema, platform, eval set, guardrail versions; deployment and last-validation timestamps for each.
8. Eval gap. Missing or under-tested case description; how discovered; closure plan with specific new cases.
9. Detection. How the incident was detected and how it could have been detected sooner. What signal would have caught it earlier?
10. Containment. What actions stopped the bleeding. How quickly. Whether the playbook was followed or improvised.
11. Diagnosis. How root cause was identified. What tools were used (traces, eval reproductions, logs). What slowed or accelerated diagnosis.
12. Remediation. What fix returned the system to normal. Whether the fix was reversible. Whether it required customer communication.
13. Action items. Concrete commitments with named owners and target dates. Categorized into prevention, detection, response, and process.
14. Lessons learned. What patterns, when seen elsewhere, would help the team or other teams.
15. Distribution. Who reads this? Owner, governance, other agent teams, security/legal where applicable.

How long should it be

For a moderate S2 incident: 3-5 pages. For a high-impact S1: 5-10 pages, with supporting trace exports and screenshots as appendices. For a low-impact S3 caught early: 1-2 pages. The format scales with severity; the structure does not.

The Action Item Discipline

The most common postmortem failure: action items written but never executed. The document is accepted. The team moves on. The next incident hits and the same action items appear in the new postmortem because the previous ones were never done.

Action item categorization

The 2026 convention is to categorize action items into four classes:

• Prevention. Changes that prevent the same failure from recurring. New eval cases, prompt strengthening, guardrails, tool changes.
• Detection. Changes that allow the failure to be caught faster next time. New alerts, dashboard additions, shadow eval expansions.
• Response. Changes that improve the team's reaction. Updated runbooks, named-owner clarifications, escalation-path improvements.
• Process. Changes to the team's working pattern. Change-management updates, governance reviews, training.

Each category gets its own action items. The categorization makes it clear that the team is investing in the system, not just patching the immediate issue.

Action item ownership

Every action item has a named owner. Not a team — a specific human. The owner commits to a target date. The action item is tracked in the same backlog system as feature work.

If the action item is genuinely a team responsibility ("review the on-call rotation policy"), the team's lead owns it personally and is accountable for delegating execution. "The team will handle this" is not an action item; "Priya Sharma will draft the policy update by 2026-05-30" is.

Action item review

Postmortem action items get reviewed at a recurring cadence — weekly during the first month after the incident, then in the next postmortem review meeting, then in the quarterly retrospective.

An action item that has been open for more than 60 days without movement is a red flag. Either it is not actually important (in which case close it explicitly), it is blocked (in which case identify the blocker), or it is being deprioritized (in which case the postmortem's lessons are not being absorbed). The 60-day review surfaces all three.

The Postmortem Meeting

The postmortem document is the artifact. The postmortem meeting is the cultural ritual that makes the document real.

Format

60 minutes. Attendees: the on-call who wrote the document, the reviewer, the owner, governance where relevant, and any other team member who was involved in the response. The agenda:

1. Read the document together (10 min). The author or facilitator reads the timeline, impact, and root cause aloud. Everyone has the document in front of them.
2. Open discussion (30 min). What did we get right? What did we get wrong? What surprised us? What changes if we hit this again? The discussion is facilitated to stay on system and not slip into individual blame.
3. Action item review (15 min). Walk through the proposed action items. Refine, add, remove. Confirm owners and dates. Push back on action items that are not actionable.
4. Distribution and close (5 min). Confirm who reads the final document. Schedule the action item review. Close.

The hot-wash variant

For minor incidents (S3, S4) the postmortem can be a 30-minute hot-wash with a condensed document. The fields are the same; the depth is less. The point is that even small incidents go through the discipline.

Cross-incident pattern review

Quarterly, the team reviews the postmortems written in the previous quarter. Are there patterns? Is a particular failure mode recurring? Is the eval set growing in the right places? The cross-incident review is where the team's system-level learning crystallizes.

Case Study: A Hallucinated Refund Postmortem

To make the template concrete, a worked example. Names and details are illustrative.

Header

INC-2026-05-14-002. Severity S2. Author: Alex Chen (on-call builder). Reviewers: Priya Sharma (reviewer), James Hong (VP Customer Success, owner). Status: Accepted.

Summary

On 2026-05-14, the customer-support agent processed four refunds where the wrong purchase was identified as the one to refund. Total over-refund exposure: $1,840. The incident was detected by a customer complaint at 14:32 UTC and contained by 14:47 UTC by pausing the agent's Stripe write authority. Remediation involved reverse-charge processing for three of the four cases (one customer declined the clawback). Root cause: the eval set did not cover the multi-purchase ambiguous-refund case.

Version drift

• Model: claude-sonnet-4-5-2026-03-20 (deployed 2026-03-22; last validated 2026-05-10).
• Prompt: support-agent-v2.18 (deployed 2026-05-08; validated 2026-05-08).
• Tool schema: stripe-mcp-v1.4.2 (deployed 2026-04-01; validated 2026-04-01).
• Platform: n8n 1.84.6 (deployed 2026-05-09; validated 2026-05-09 per platform-upgrade eval).
• Eval set: support-eval-v3.7 (deployed 2026-04-22; last validated 2026-05-13 nightly).
• Guardrails: lakera-guard-2026.05.01 (deployed 2026-05-01).

No version drift — every version was validated within the last 30 days. Conclusion: the regression was not version-driven. The eval-gap field is the operative one.

Eval gap

The eval set support-eval-v3.7 contained 47 refund cases. All 47 were single-purchase scenarios. The agent had never been tested against multi-purchase ambiguity. The case category "multi-purchase refund disambiguation" is a gap. Discovered by retrospective analysis after the incident. Closure plan: add 7 new cases — 3 golden (clear correct answer despite ambiguity), 3 edge (ambiguous case that should escalate), 1 adversarial (deliberately misleading user input designed to trick the agent). Target: support-eval-v3.8 by 2026-05-22.

Root cause

The eval set did not cover multi-purchase ambiguity. The agent's prompt instructed it to "process the refund the customer is requesting" but did not require disambiguation when the request was ambiguous and multiple purchases existed.

Action items (excerpt)

Prevention. (1) Add 7 multi-purchase cases to eval set by 2026-05-22 — Owner: Alex Chen. (2) Update support-agent prompt to require explicit disambiguation when multiple purchases match the time window — Owner: Alex Chen, by 2026-05-23. (3) Add post-call guardrail to validate refund target against the explicit request — Owner: Alex Chen, by 2026-05-30.

Detection. (4) Add cost-per-refund metric to the support-agent dashboard with two-sigma alert — Owner: Alex Chen, by 2026-06-05.

Response. (5) Update runbook page HRW-CONTAIN to include the n8n-specific Stripe write-authority revocation steps — Owner: Priya Sharma, by 2026-05-25.

Process. (6) Add multi-purchase scenarios to the next tabletop exercise — Owner: Priya Sharma, by 2026-06-30.

Distributing Postmortems and Cross-Team Learning

A postmortem read only by the team that wrote it captures only their own learning. The postmortem that propagates across teams multiplies the value.

Internal distribution

Every agent postmortem is distributed to: the agent's owner, the agent's governance contact, the AI Platform team (if one exists), other agent teams in the organization, and the security team where applicable. The distribution list is part of the postmortem template — explicit, not implicit.

The distribution does not mean "everyone reads everything." It means everyone has access. Teams looking for patterns can search; new on-calls onboarding can learn from real incidents; the AI Platform team can identify cross-cutting issues that warrant platform-level fixes.

External communication where appropriate

Some postmortems are appropriate for external sharing — particularly when the failure mode is industry-wide (the February 2026 n8n incident is a perfect example). Anonymized, sanitized versions of postmortems can be shared with the platform vendor, posted to public forums, or contributed to the body of agent-ops knowledge.

The team's legal and PR teams review external sharing. The cost of sharing is low (some review effort); the benefit to the agent-ops community is high (other teams learn from your incident before they live it themselves).

Cultural Anti-Patterns

The performative postmortem

The team writes the postmortem because the process requires it. The document is generic, the action items are vague, the meeting is a check-the-box ritual. No one learns anything. This is the most common failure mode in early agent ops practice.

The fix: rigorous template enforcement plus action-item ownership. Generic postmortems get returned for revision. Vague action items get rejected at the meeting. The discipline takes a quarter to establish but pays for itself within two.

The blame-the-model trap

"The model hallucinated" becomes the universal root cause. Every postmortem ends there. The team learns nothing because the conclusion is always the same and is always outside the team's control.

The fix: prohibit "the model did X" as a root cause. Always require the deeper cause: the eval gap, the prompt weakness, the missing guardrail. The model is part of the system the team chose; the system can be improved.

The skipped postmortem

Minor incidents do not get postmortems because they are "not worth the time." Six months later the team has accumulated thirty minor incidents that share a pattern, and no one noticed because no one wrote them up. The fix: every incident, however small, gets at least a hot-wash. The discipline is the value.

The action-item graveyard

Action items get written, get accepted, and then quietly fade. The 60-day review surfaces this; without the review, the graveyard grows. The fix: explicit review cadence and the discipline to close stale action items rather than letting them die in place.

The closed-team postmortem

The postmortem stays within the team that wrote it. Other teams learn nothing. The same incident plays out in three different teams over six months. The fix: explicit distribution policy; cross-team postmortem review forum; AI Platform team or governance role with responsibility for surfacing patterns.

From Postmortem to Operating Improvement

The point of the postmortem is not the document. The point is the operating improvement that comes from the document.

The cumulative effect

A team that writes 10 rigorous postmortems in a year accumulates: 10 new eval cases (at minimum) per postmortem = 100+ eval cases added, dozens of prompt and guardrail improvements, refined runbooks, sharpened alerts, clearer ownership. The agent is materially more reliable at the end of the year than at the beginning, and the team has the artifacts to prove it.

The compounding curve

The first postmortem feels like overhead. The fifth feels like discipline. The twentieth feels like leverage. The hundredth is the foundation on which the team's operational excellence rests. The compounding is real but only visible in retrospect.

The QBR artifact

At the quarterly business review, the postmortem corpus is the artifact the owner shows. "We had four incidents this quarter. Here are the four postmortems. Here is the action-item completion rate. Here is the eval set growth. Here is the trend on each of our four SLOs." Stakeholders see the team is learning. Trust grows.

Key Takeaways

• The classical SRE blameless postmortem template (header, summary, timeline, impact, root cause, contributing factors, action items) is necessary but not sufficient for agent incidents. Two new fields must be added: version drift and eval gap.
• Version drift enumerates every version in play at incident time: model, prompt, tool schema, platform, eval set, guardrails. For each, when it was deployed and when it was last validated. The field surfaces silent regressions caused by drift.
• Eval gap describes the case or class of cases the eval set did not cover that allowed the incident to slip through. Three components: gap description, how discovered, closure plan with specific new cases to be added. Every incident becomes a permanent eval set improvement.
• The complete template has 15 sections: header, summary, timeline, impact, root cause, contributing factors, version drift, eval gap, detection, containment, diagnosis, remediation, action items, lessons learned, distribution. Scales with severity from 1-2 pages (S3) to 5-10 pages (S1).
• Blamelessness is non-negotiable. "The model hallucinated" is not a root cause — the team chose the model, wrote the prompt, designed the eval. Blameless framing applies to the system the team built, not the model.
• Action items categorize into prevention, detection, response, and process. Every item has a named owner, a target date, and lives in the same backlog as feature work. The 60-day review surfaces graveyard items, closes stale ones, identifies blockers.
• The postmortem meeting is 60 minutes: read together (10), open discussion (30), action item review (15), distribution (5). Hot-wash variant for S3/S4. Quarterly cross-incident pattern review crystallizes system-level learning.
• Distribution policy is explicit: owner, governance, AI Platform team, other agent teams, security where applicable. Anonymized external sharing for industry-wide failures (the February n8n incident is the canonical example).
• Cultural anti-patterns: performative postmortems (process without learning), blame-the-model trap (always-the-same root cause), skipped postmortems (minor incidents accumulate unread), action-item graveyard (no review cadence), closed-team postmortems (no cross-team propagation).
• The cumulative effect compounds: 10 rigorous postmortems in a year = 100+ eval cases added, refined runbooks, sharpened alerts, clearer ownership. The QBR artifact is the postmortem corpus. Trust grows visibly.