Wiki novels do not help at 2 a.m. Short runbooks with commands and owners do.
Wiki novels do not help at 2 a.m.
Write the steps you would need half-asleep: symptoms, checks, mitigations, escalate-to.
Tribal knowledge evaporates when the expert is offline. IT systems last longer than any one engineer’s memory.
Why this shows up in real IT work
Information technology fails in boring, repeated ways: unclear ownership, untested recovery, and tools used without a question. Write the steps you would need half-asleep: symptoms, checks, mitigations, escalate-to. That sentence is the filter for everything below.
Most teams already own the software that could help. What they lack is a sequence they can run under pressure. Wiki novels do not help at 2 a.m. The gap between knowing the tool exists and using it deliberately is where outages and slow weeks live.
You do not need a new platform to start. You need a first check small enough to survive a noisy on-call, then a second check that confirms the first one. That is how operational skill compounds without requiring a conference talk every morning.
The mechanism in plain language
Tribal knowledge evaporates when the expert is offline. IT systems last longer than any one engineer’s memory. When you treat the problem as bad luck, you get folklore. When you treat it as a diagnosable system, you get a checklist.
Notice how often teams wait for a perfect rewrite before they fix the path in front of them. Delivery follows verification more often than the reverse. Write the steps you would need half-asleep: symptoms, checks, mitigations, escalate-to. Prove the symptom, then change one variable.
Also notice the stories people tell mid-incident: that this stack is uniquely cursed, that only one hero can fix it, that documentation is for other people. Those stories are loud and mostly useless. Replace them with a sequence you can run with half a dashboard and a half-honest status page.
A practice you can run this week
- For each critical service, keep a one-page runbook.
- Include exact commands and links to dashboards.
- Name the escalate path and the expected response time.
- Update the runbook in the same PR as behavior changes.
- Delete docs that no longer match reality.
Run the practice once without optimizing it. Optimization is how people postpone the first restore drill or the first key rotation. After three honest reps, adjust one friction point — the alert threshold, the runbook command, or the access path — and keep going.
A scene from ordinary operations
A new hire mitigates a cache outage using the runbook while the author is on a flight. The page had the flush command and the rollback flag.
That scene is intentionally ordinary. IT skills earn their keep in ticket queues, deploy windows, and quiet Sundays — not only in viral outage postmortems.
If your own version of the scene looks messier, that is fine. Skill is not aesthetic. Skill is whether the next similar moment goes slightly better because you had a plan.
The pitfall that quietly undoes the practice
A 40-page architecture PDF with no “what to do when it breaks” section.
If you catch yourself doing that, stop and return to the smallest next verification. Do not scrap the whole system because one change window went sideways. Wiki novels do not help at 2 a.m.
Another quiet undoing: collecting more tooling instead of practicing the tooling you already have. Read this once, then open the terminal or console and run step one. Articles do not change uptime. Reps do.
A short checklist
- Decide what “done enough” looks like before you start.
- Remove one friction from the environment or the runbook.
- Run the smallest verification even on a busy day.
- Review once, without courtroom energy.
- Ask yourself: Could someone who never touched this service mitigate with only the runbook?
Limits and honesty
No single habit fixes underfunding, hostile vendors, or legacy constraints you did not choose. Practices sit on top of reality; they do not erase it. Use what helps, ignore hustle folklore that blames individuals for structural debt, and escalate when the checklist is not enough.
Still, many of the leaks in a week are local and trainable. Access hygiene, DNS checks, backup restores, and clear alerts respond to practice. That is why this belongs under IT on Swiftener: useful moves for systems work, not vibes for performing expertise.
If someone on your team needs a softer entry, share the scene and the checklist — not a lecture. Invitation beats correction.
The one test
Could someone who never touched this service mitigate with only the runbook?
If the answer is no, shrink the practice until it becomes yes. Write the steps you would need half-asleep: symptoms, checks, mitigations, escalate-to.
Then teach the move to someone else in one minute. Teaching is how the skill sticks in your own hands.
Come back to this page when the pager gets loud again. The point is not to memorize a brand of best practice. The point is to leave with a move you can use before the shift ends.