5 posts tagged with "engineering"

We’ve gotten pretty good at collecting observability data, but we’re terrible at making sense of it. Most teams—especially those running complex microservices—still rely on a handful of senior engineers who just know how everything fits together. They’re the rockstars who can look at alerts, mentally trace the dependency graph, and figure out what's actually broken.

When they leave, that knowledge walks out the door with them. That is the observability Bus Factor.

The problem isn't a lack of data; we have petabytes of it. The problem is a lack of context. We need systems that can actually explain what's happening, not just tell us that something is wrong.

This post explores the concept of a "Living Knowledge Base", Where the context is built based on the telemetry data application is emitting, not based on the documentations or confluence docs. Maintaining docs is a nightmare and we cannot always keep up Why not just build a system that will do this

At base14, everyone is always

• shipping
• forward deployed
• helping customers
• on production support

What makes recovery slower — and what disciplined, observable teams do differently.

In reliability engineering, MTTR (Mean Time to Recovery) is one of the clearest indicators of how mature a system — and a team — really is. It measures not just how quickly you fix things, but how well your organization detects, communicates, and learns from failure.

Every production incident is a test of the system's design, the team's reflexes, and the clarity of their shared context. MTTR rises when friction builds up in those connections — between tools, roles, or data. It falls when context flows freely and decisions move faster than confusion.

Last month, I watched a senior engineer spend three hours debugging what should have been a fifteen-minute problem. The issue wasn't complexity—it was context switching between four different monitoring tools, correlating timestamps manually, and losing their train of thought every time they had to log into yet another dashboard. If this sounds familiar, you're not alone. This is the hidden tax most engineering teams pay without realizing there's a better way.

the·a·tre (also the·a·ter) /ˈθiːətər/ noun

: the performance of actions or behaviors for appearance rather than substance; an elaborate pretense that simulates real activity while lacking its essential purpose or outcomes

Example: "The company's security theatre gave the illusion of protection without addressing actual vulnerabilities."

Your organization has invested millions in observability tools. You have dashboards for everything. Your teams dutifully instrument their services. Yet when incidents strike, engineers still spend hours hunting through disparate systems, correlating timestamps manually, and guessing at root causes. When the CEO forwards a customer complaint asking "are we down?", that's when the dev team gets to know about incidents.