The quiet runtime.
A meditation on uptime, on-call, and the kind of operational software you stop noticing — which is, in our view, the only kind worth shipping.
A short measure
The right way to measure infrastructure is by how often you stop thinking about it.
By that measure, we have been failing for years on most of the platforms we use. The dashboard you log into every morning. The system that paged you twice this week. The integration you have to babysit when a quarter closes. The ETL job that always falls over on the second of the month.
The standard for "good enough" has slid so far that we now treat constant attention as a feature. It is not. It is a tax. The runtime should be quiet.
What quiet looks like
Quiet means the median deploy goes out at 2 p.m. on a Tuesday and nobody notices. Quiet means the on-call rotation pages once a quarter, and when it does, the page is correct. Quiet means the audit packet is generated automatically because every action left a receipt. Quiet means the CFO's reconciliation question is answered by a query, not a working group.
We design for quiet. We do not always achieve it. But the gap between us and the alternative is the entire point of the company.
How we think about pages
Every page on the Anillion runtime is reviewed in the weekly operations meeting. Not as a checklist. As a conversation. Why did this page fire? Was the page correct? Did the response work? What changed?
Pages are a signal of work the system is asking of a human. The first question is always: should the system have done this itself? The second question is: did we learn something we will encode? The third is: is this page now lower priority than it was?
If the answer to all three is no, the page should not have fired.
The unglamorous part
This is, frankly, unglamorous work. Reviewing every page. Tagging every false positive. Owning every silenced alert. Making the queue smaller every week.
But that work compounds. The Anillion environments that have been in production longest are the quietest. They run with smaller on-call rotations. They have shorter incident lists. They are increasingly easy to extend.
We do not have a slide for "quiet quarter." We just keep shipping toward it.
N° 006.X— Continue reading
Send the next memo to my inbox.
One memo a month, no roundups, no marketing. Unsubscribe in one click.