Error budgets and policies
Using the budget
The error budget turns reliability into a shared decision tool between product and engineering:
- Budget healthy → ship features, run experiments, do risky migrations.
- Budget burning fast or exhausted → error budget policy kicks in: freeze non-critical releases, prioritise reliability work, require extra review for changes, until the budget recovers.
A policy should be agreed in advance with product leadership, documented, and applied consistently — otherwise it's ignored in the first crisis.
Burn rate
Burn rate = how fast you're consuming the budget relative to plan. Burn rate 1 = exactly exhausting the budget at the end of the window. With a 30-day window, a burn rate of 14.4 sustained for 1 hour consumes 2% of the monthly budget.
Multi-window, multi-burn-rate alerts (Google SRE workbook) balance fast detection and low noise:
| Severity | Burn rate | Long window | Short window | Budget consumed |
|---|---|---|---|---|
| Page | 14.4 | 1 h | 5 m | 2% |
| Page | 6 | 6 h | 30 m | 5% |
| Ticket | 1 | 3 d | 6 h | 10% |
The short window ensures the alert resets quickly when the problem stops. PromQL sketch for a 99.9% SLO (budget 0.001):
(
sum(rate(http_requests_total{service="checkout",status=~"5.."}[1h]))
/ sum(rate(http_requests_total{service="checkout"}[1h]))
) > (14.4 * 0.001)
and
(
sum(rate(http_requests_total{service="checkout",status=~"5.."}[5m]))
/ sum(rate(http_requests_total{service="checkout"}[5m]))
) > (14.4 * 0.001)
Tools like Sloth, Pyrra or OpenSLO generate these rules from an SLO definition; Grafana, Datadog and CloudWatch Application Signals provide SLO tracking.