RTO and RPO Targets
- RPO (Recovery Point Objective): The maximum acceptable age of data that must be recovered after an outage (measures acceptable data loss).
- RTO (Recovery Time Objective): The maximum acceptable duration of downtime before systems are restored to operational status.
Disaster Recovery Topologies
Architectures range in cost and capability: Backup & Restore (often hours to days), Pilot Light (minimal core running in a secondary region), Warm Standby (a scaled-down replica stack), and Active-Active Multi-Region (traffic served in multiple regions). Active-active can target low RTO, but replication lag, conflicts, quorum loss, routing convergence, and shared dependencies can still produce nonzero RPO and RTO; the topology alone proves neither target.
DR strategies side by side
| Strategy | Typical RPO | Typical RTO | Standby cost | What runs in the DR region |
|---|---|---|---|---|
| Backup and restore | Hours (backup interval) | Hours to days | Storage only | Nothing; backups are copied cross-region |
| Pilot light | Seconds to minutes (async replication) | Tens of minutes | Low | Replicated database; application servers off or scaled to zero |
| Warm standby | Seconds | Minutes | Medium | Full stack at reduced size, scaled up on failover |
| Active-active multi-region | Near zero to seconds (depends on replication mode) | Near zero to minutes | High (two or more times) | Full stack serving live traffic |
These ranges follow the AWS disaster recovery whitepaper. They describe what each topology can achieve. Whether you actually achieve them depends on testing.
Worked example: what does failover really cost in time?
This is a warm-standby design with asynchronous database replication. Replication lag is normally 1 to 2 seconds, but reaches about 10 seconds at peak.
- Detect: health checks fail for 3 consecutive 30-second intervals, so about 90 s.
- Decide: automatic failover risks flapping and split-brain, so many teams require a person to approve it. Paging and approval take about 5 to 10 minutes.
- Promote the database: about 1 to 2 minutes. Any writes still inside the replication lag, up to 10 s of data, are lost unless they can be recovered later from the old primary.
- Scale the application tier from 20% to 100%: about 5 minutes with pre-warmed images.
- Move traffic: DNS with a 60 s TTL takes minutes to converge, since some clients ignore TTLs. Anycast or a global load balancer takes seconds.
- Total: RTO of about 15 to 20 minutes and RPO of about 10 s. If the business needs an RPO of zero for payments, those writes need synchronous replication across regions or a consensus-replicated database such as Spanner or CockroachDB. That adds tens of milliseconds of write latency for the cross-region round trip.
Failure modes
- Split brain: both regions believe they are primary and accept conflicting writes. Use fencing, such as a lease or quorum, so the old primary cannot accept writes after failover.
- Failback is forgotten: returning to the original region needs its own replication in reverse and its own runbook.
- Shared fate through dependencies: third-party APIs, secrets managers, CI/CD and the status page itself must not live only in the region that failed.
- Data corruption replicates: DR replication copies bad writes too. You still need point-in-time backups.
How real systems do it
Netflix runs active-active across several AWS regions and regularly evacuates a region on purpose (Chaos Kong) to prove it can. Google's DiRT exercises test disaster scenarios company-wide. Many banks run warm standby with regulator-mandated annual failover tests. Most SaaS companies use pilot light or warm standby for their databases, combined with cross-region backup copies.