SD System Design In Depth
0%
lesson system-designreliabilityslo

SLOs and error budgets

Module 5.1: Production Reliability, SLOs & Resilience Updated Sep 18, 2026
📄 Cheat Sheet
SLIs, SLOs, SLAs, oh my! (class SRE implements DevOps)
Google Cloud Tech SLIs, SLOs, SLAs, oh my! (class SRE implements DevOps) Watch on YouTube ↗

Liz Fong-Jones and Seth Vargo, both from Google, explain SLI vs SLO vs SLA and error budgets in eight minutes. This is the original source that the current video only paraphrases.

0:00 / --:--

SLI, SLO, and SLA Demystified

  • SLI (Service Level Indicator): A carefully defined metric that measures service behavior (e.g., "percentage of HTTP requests returning in < 200ms with a 2xx status").
  • SLO (Service Level Objective): An internal target reliability threshold agreed upon between product and engineering teams (e.g., "99.9% of requests meet the SLI over a 30-day rolling window").
  • SLA (Service Level Agreement): A commercial contract with customers specifying legal and financial penalties (refunds, credits) if reliability drops below a contractual threshold (e.g., 99.5%).

The Error Budget Concept

A 100% target is usually impractical and can suppress useful change. If an availability SLO is 99.9%, its error budget is the remaining 0.1% of eligible events in the stated window. Teams use a written error-budget policy to decide which risky changes to slow or stop when burn is excessive; an automatic freeze of every release is one possible policy, not part of the definition, and urgent security or reliability fixes may still need to ship.

Worked example: what "three nines" actually allows

SLO (30-day window)Error budgetAllowed full downtimeAt 50M requests/month
99%1%7 h 12 min500,000 failed requests
99.9%0.1%43 min 12 s50,000 failed requests
99.95%0.05%21 min 36 s25,000 failed requests
99.99%0.01%4 min 19 s5,000 failed requests

Each extra nine cuts the budget by a factor of ten, and the engineering cost usually rises steeply with it. A 99.99% target leaves roughly four minutes a month, which is less than the time it takes a person to get paged, log in and roll back. At that level, detection and mitigation have to be automated.

Burn-rate alerting

Burn rate is how fast you are consuming the budget relative to an even spend. A burn rate of 1 uses exactly the whole budget over the 30-day window. The Google SRE Workbook recommends multi-window alerts such as:

Burn rateLong windowShort windowBudget consumedAction
14.41 hour5 minutes2%Page
66 hours30 minutes5%Page
13 days6 hours10%Ticket

Where 14.4 comes from: 2% of a 720-hour window is 14.4 hours of budget, and spending it in 1 hour is a 14.4x burn. For a 99.9% SLO, the page fires when the error ratio over the last hour is above 14.4 x 0.1% = 1.44%, and the last 5 minutes are also above that level. The short window makes the alert clear quickly once the problem is fixed.

Writing a good SLI

  • Ratio form: good events divided by valid events. For example, requests to /checkout that returned a non-5xx status within 300 ms, divided by all requests to /checkout excluding health checks.
  • Choose what users feel: availability and latency for APIs, freshness for data pipelines, durability for storage.
  • Low-traffic services: at 100 requests per hour, a single failure is 1% of the hour. Use longer windows, synthetic probes, or combine several services into one SLO.

The error-budget policy

The written policy is what gives the budget any force. A typical policy says that while the budget is exhausted, only reliability fixes and security patches ship, and a postmortem is required for any single incident that used more than 20% of the budget. Product and engineering leadership agree to the policy in advance, so nobody argues about it in the middle of an incident.

Check yourself Question 1 of 3
SLO versus SLA recall
From: SLI, SLO, and SLA Demystified

Your contract promises customers 99.9% monthly availability with service credits below that. How should the internal SLO relate to this SLA?

Error Budget and Multi-Window Burn-Rate Alerts

Burn rate = observed error rate ÷ allowed error rate (1 − SLO). A burn rate of 1 uses exactly the whole budget over the SLO window. Each alert fires only when both its long and short window are at or above the threshold: the long window gives significance, and the short window makes the alert stop soon after the incident ends. Traffic is assumed uniform, with a resolution of one minute.

Error budget remaining at the end of each day
Alert timeline
Architecture Challenge: Error Budget Consumption & Release Freezes
Staff Level
Context / Scenario: Monthly error budget 96% consumed in the first third of the calendar month.
Problem: Your team has an SLO of 99.9% monthly availability (allowing 43.8 minutes of downtime). On Day 10 of the month, an outage consumes 42 minutes of downtime. What is the standard SRE policy?
A Halt all non-critical feature deployments and redirect engineering effort to reliability, testing, and automated guardrails until the budget recovers.
B Increase the SLO to 99.99% so the formula resets.
C Blame the on-call engineer and remove their production deployment permissions.
D Ignore the error budget since it has not reached 100% yet.

Key Takeaways

SLIs measure, SLOs set targets, and SLAs impose commercial consequences.
100% uptime is never the goal; error budgets balance feature velocity against stability.
When the error budget burns too fast, enforce deployment freezes to protect user trust.
Was this lesson helpful? Spot an issue or have a suggestion? Let us know on GitHub.
Report an issue