Production Incident Post-Mortem & Root-Cause Synthesizer
Convert messy incident Slack logs and alerts into a blameless, rigorous post-mortem with corrective action items.
Compatibility & Specs
Compatible AI Models
ClaudeChatGPTGemini
Last UpdatedApr 1, 2026
Customizable Variables3 parameters
Main Prompt Template
154 words • 1149 characters
You are a Principal Site Reliability Engineer (SRE) and Incident Commander. I need to author an authoritative, blameless Production Incident Post-Mortem based on raw incident timeline notes.
Incident Overview & Context:
- Service Affected & Severity: [incident_severity]
- Total Customer Downtime / Impact Duration: [downtime_duration]
- Raw Incident Timeline & Slack Notes:
[raw_incident_notes]
Deliver the standard SRE Blameless Post-Mortem Report:
1. Executive Summary: High-level non-technical summary of what occurred, total business impact, and key preventative takeaways.
2. Detailed Incident Timeline (UTC): Synthesize raw notes into a standardized chronological sequence (T0 Detection, T+X Mitigation, T+Y Resolution).
3. Root Cause Analysis (5 Whys): Conduct a deep technical 5 Whys deconstruction distinguishing proximate triggers from systemic architecture/process flaws.
4. What Went Well vs. Where We Got Lucky vs. Where We Struggled: Objective assessment of observability, alerts, and runbooks.
5. Corrective Action Items (P0 to P2): Actionable engineering tasks with concrete definitions of done to mechanically prevent recurrence.
Tip:Variables marked in purple brackets can be tailored below.
Customize Prompt
Fill in the variables below. Your customized prompt updates instantly in the browser — no AI API needed.
Parameters (3)* Indicates required field
[incident_severity]
Severity level and impacted service
[downtime_duration]
Total elapsed time from first error to full resolution
[raw_incident_notes]
Paste unformatted incident notes, alerts, and chat snippets
3 parameters active
Live Customized OutputLive Preview
242 wordsYou are a Principal Site Reliability Engineer (SRE) and Incident Commander. I need to author an authoritative, blameless Production Incident Post-Mortem based on raw incident timeline notes.
Incident Overview & Context:
- Service Affected & Severity: SEV-1: Customer Payment Processing Outage
- Total Customer Downtime / Impact Duration: 34 minutes full checkout outage; 14 minutes elevated error rates
- Raw Incident Timeline & Slack Notes:
14:12 UTC - Automated Datadog alert: Stripe webhook consumer backlog exceeds 10,000.
14:16 UTC - On-call engineer notices Redis connection pool saturated (100% capacity).
14:22 UTC - Deploy pipeline locked by an unrelated PR merge.
14:29 UTC - Redis maxclients reached because worker pods auto-scaled from 6 to 48 without connection limits.
14:38 UTC - Hard restart of worker pods with lowered pool sizes restores processing.
14:46 UTC - Backlog cleared, error rate returns to 0.01%.
Deliver the standard SRE Blameless Post-Mortem Report:
1. Executive Summary: High-level non-technical summary of what occurred, total business impact, and key preventative takeaways.
2. Detailed Incident Timeline (UTC): Synthesize raw notes into a standardized chronological sequence (T0 Detection, T+X Mitigation, T+Y Resolution).
3. Root Cause Analysis (5 Whys): Conduct a deep technical 5 Whys deconstruction distinguishing proximate triggers from systemic architecture/process flaws.
4. What Went Well vs. Where We Got Lucky vs. Where We Struggled: Objective assessment of observability, alerts, and runbooks.
5. Corrective Action Items (P0 to P2): Actionable engineering tasks with concrete definitions of done to mechanically prevent recurrence.
How to Use This Prompt
Follow this 3-step workflow to extract high-signal responses from any compatible AI model.
01
1. Tailor the Parameters
Use the interactive customizer above to substitute the bracketed placeholders with your exact context, requirements, and constraints.
02
2. Send to AI Model
Copy the prompt and paste it into Claude, ChatGPT, Gemini, or Copilot. These models follow structured multi-step constraints reliably.
03
3. Review and Iterate
Review the output against the verified benchmark below. Follow up in the conversation to stress-test edge cases or refine tone.
Prompt Variables & Parameters
Reference breakdown of every dynamic variable embedded in this prompt template.
| Placeholder | Parameter Name | Type | Status | Description & Guidance |
|---|---|---|---|---|
| [incident_severity] | Incident Severity & Affected Service | text | Required | Severity level and impacted serviceDefault: SEV-1: Customer Payment Processing Outage |
| [downtime_duration] | Downtime / Degradation Duration | text | Required | Total elapsed time from first error to full resolutionDefault: 34 minutes full checkout outage; 14 minutes elevated error rates |
| [raw_incident_notes] | Raw Incident Notes & Slack Logs | textarea | Required | Paste unformatted incident notes, alerts, and chat snippetsDefault: 14:12 UTC - Automated Datadog alert: Stripe webhook consumer backlog exceeds 10,000. 14:16 UTC - On-call engineer notices Redis connection pool saturated (100% capacity). 14:22 UTC - Deploy pipeline locked by an unrelated PR merge. 14:29 UTC - Redis maxclients reached because worker pods auto-scaled from 6 to 48 without connection limits. 14:38 UTC - Hard restart of worker pods with lowered pool sizes restores processing. 14:46 UTC - Backlog cleared, error rate returns to 0.01%. |
Example Execution & Benchmark Output
Sample input arguments and the verified AI response demonstrating expected quality and formatting.
Sample VariablesInputs
[incident_severity]
SEV-1: Customer Payment Processing Outage
[downtime_duration]
34 minutes full checkout outage; 14 minutes elevated error rates
[raw_incident_notes]
14:12 UTC - Automated Datadog alert: Stripe webhook consumer backlog exceeds 10,000.
14:16 UTC - On-call engineer notices Redis connection pool saturated (100% capacity).
14:22 UTC - Deploy pipeline locked by an unrelated PR merge.
14:29 UTC - Redis maxclients reached because worker pods auto-scaled from 6 to 48 without connection limits.
14:38 UTC - Hard restart of worker pods with lowered pool sizes restores processing.
14:46 UTC - Backlog cleared, error rate returns to 0.01%.
Expected AI ResponseVerified Result
### 1. Executive Summary
On October 12, 2026, from 14:12 UTC to 14:46 UTC (34 minutes full outage, 14 minutes degraded recovery), our payment processing service experienced a complete inability to ingest customer checkout webhooks. Approximately 1,420 transactions were delayed, with zero corrupted financial data. The root cause was an un-capped database connection pooling configuration that exhausted Redis maxclients when background worker pods auto-scaled eightfold during a traffic surge.
### 3. Root Cause Analysis (The 5 Whys)
1. **Why did checkout fail?** The payment worker pods threw timeout exceptions trying to acquire Redis lock connections.
2. **Why were connections unavailable?** Redis reached its hard ceiling of 10,000 concurrent client sockets.
3. **Why were there 10,000 sockets?** Horizontal Pod Autoscaler scaled workers from 6 to 48 pods, and each pod initialized a static pool of 200 persistent connections (48 x 200 = 9,600 + system connections).
4. **Why did each pod initialize 200 connections?** Connection pool sizing was hard-coded in 2024 when the cluster ran only 4 fixed pods.
5. **Why was this not caught in staging?** Staging traffic never reaches thresholds required to trigger pod autoscaling beyond the minimum replica count (Systemic flaw: lack of load-test scale parity).
### 5. Corrective Action Items
- **[P0]** Implement Redis client connection pooling limits tied dynamically to pod replica counts (Owner: SRE | Due: Oct 15).
- **[P1]** Add Datadog threshold alert when Redis client count exceeds 70% of maxclients (Owner: Infra | Due: Oct 16).
Best Use Cases
Scenarios and roles where this prompt produces maximum leverage.
Engineering teams drafting formal post-mortems after production service outages
SREs standardizing incident management and corrective action tracking across squads
Tech leads presenting outage post-mortems to non-technical executive stakeholders
Tips for Best Results
Techniques to elevate response fidelity
- •Provide rich background context rather than one-sentence inputs to receive deep, non-generic responses.
- •Engage in multi-turn conversation: use the initial output as a baseline, then ask the AI to sharpen specific sections.
- •Prompt the model to highlight any hidden assumptions or missing trade-offs in its recommendations.
Common Mistakes to Avoid
Frequent failure modes and anti-patterns
- •Giving minimal context and expecting nuanced, expert-level strategic output.
- •Not validating factual references, citations, or statistical claims with verified primary sources.
- •Skipping the customization step and pasting raw bracketed template variables into the AI chat.
Part of Curated Collections
This prompt is sequenced as part of these goal-oriented workflows
Related AI Prompts
Complementary workflows in Coding
Codingadvanced
Principal Code Reviewer & Architecture Auditor
Conduct rigorous architectural code reviews identifying memory leaks, race conditions, and typing holes.
claudechatgptcopilot
#code-review#clean-code#architecture
Job Interviewadvanced
Behavioral Failure & Workplace Conflict Answer Architect
Structure answers for difficult behavioral questions about interpersonal conflict, failed projects, and bad decisions.
claudechatgptgemini
#behavioral-questions#conflict-resolution#failure-story
Codingintermediate
Exhaustive Edge-Case Unit & Integration Test Generator
Analyze production functions to discover subtle concurrency, boundary, and null pointer edge cases and write unit tests.
claudechatgptgemini
#unit-testing#integration-testing#edge-cases
Related Engineering Guides
Deep-dive playbooks and system prompt methodologies for Coding
Prompt Engineering#promptarchitecture
8 min readHow to Write Better AI Prompts
A comprehensive playbook for crafting high-fidelity prompts: mastering context, roles, objectives, constraints, output schemas, few-shot examples, and systematic iteration.
Mar 25, 2026Read Guide
Software Engineering#engineering
9 min readAI Prompts for Software Developers
Turn modern LLMs into senior engineering peers: tactical prompt patterns for architecture review, edge-case test generation, root-cause debugging, and technical documentation.
Mar 27, 2026Read Guide