Network & Infrastructure / Telemetry, Capacity & Incident Analysis
Error-budget burn alerts for network-dependent services
An isolated packet-loss spike repeatedly pages operators even when users see no meaningful service impact. Conversely, sustained modest failures can erode availability without crossing a dramatic device threshold.
Environment and inputs: Network and infrastructure observability; Prometheus examples describe metric semantics rather than a universal exporter schema. Diagnostic/design example; not tested on production equipment.
Collect the evidence
Define a service-level indicator from successful useful requests and record its objective and window. Compare service error consumption with device loss, latency and routing events.
Explain the result
Device thresholds describe mechanisms; an error budget describes service outcomes over time. Neither should replace the other, but they answer different operational questions.
Plan the fix
Design supported multi-window alerting around actionable budget consumption and keep diagnostic device signals available. Establish response ownership and avoid pretending that one threshold fits every service.
Check the result and rollback
Evaluate historical incidents and controlled failures for detection speed and false pages. Revert if the new rule misses significant user harm or pages on conditions no operator can meaningfully address.