Incidents
Articles tagged Incidents — notes and deep-dives grouped under this topic.
-
Blameless postmortem template: what the agent fills in, what only a human can
A model now writes the first postmortem draft from the timeline and the chat. What is left for people are the fields telemetry does not hold.
-
Toil, the 50% rule and the AI agent: what it removes and what it hides
Everyone remembers the 50% rule from the Google SRE book. Two other numbers in the same chapter explain why a four-person rotation never gets below the ceiling.
-
Drift detection in 2026: from the alert to remediation as code
Plan says “no changes” while a security group sits open to the internet. The four blind spots of single-layer detection, and where automatic remediation should stop.
-
HTTP/2 for Internal Traffic: A Well-Known Class of Pitfalls
One connection for every request, a shared flow control window, and a pool that never evicts the dead: why HTTP/2 services hang "without errors" and what Go and gRPC did about it.
-
An incident closed twice: severity, ICS roles and three gates
Why «recovered» is the most expensive word in an incident, and the three checks that must pass before you say it.
-
OOMKilled forensics: from pmap to cgroups memory.stat
Exit code 137 leaves no stack trace and no final log line, while the dashboard swears there was plenty of memory. The tools that answer «where did the memory go» — while the pod is still alive.