Placement: Edge vs. Gateway vs. Service-Level
- CDN / Edge: Stops volumetric DDoS attacks before they reach internal cloud infrastructure. Filters by client IP.
- API Gateway: Protects the overall fleet. Enforces organization-wide rate limits based on API keys, authenticated user IDs, and client tiers.
- Service-Level: Enforces domain-specific limits (e.g., maximum 5 password reset attempts per email per hour) directly within the service.
Choosing the Rate Limit Key
Rate limiting solely by IP address fails because thousands of mobile users on cellular networks or corporate offices share the same public NAT IP. Effective rate limiting uses composite keys: UserId + Action for authenticated users, and IP + DeviceFingerprint for anonymous endpoints. Client-supplied fingerprints and X-Forwarded-For headers can be forged, so take the IP from your own edge, and key IPv6 clients by prefix (for example /64).
Placement options
| Layer | Keys available | Protects | Latency cost | Examples |
|---|---|---|---|---|
| CDN/edge | IP, ASN, country, path, TLS fingerprint | Whole origin against floods | None added to the origin | Cloudflare rate limiting rules, AWS WAF rate-based rules |
| API gateway | API key, tenant, plan tier, authenticated user | Fleet fairness and commercial quotas | Sub-millisecond locally, about 1 ms with a shared Redis | Envoy global rate limit service, Kong, Apigee |
| Service | User + action, resource ID, business fields | Domain invariants (password resets, SMS sends) | Similar to gateway | In-process limiter backed by Redis |
| Client of a dependency | Downstream name | A fragile dependency such as a payment processor or partner API | Local | Concurrency limits, adaptive limiters (Netflix concurrency-limits) |
Worked example: keys for a public API
- Anonymous
GET /search: keyip_prefix + path, 60 per minute. The IPv4 /32 and IPv6 /64 prefixes are computed from the address seen by your edge, not from a header. - Authenticated API: key
api_key, with the limit set by plan: Free 10 requests per second, Pro 100, Enterprise 1,000. Add a separate per-tenant concurrency limit (for example 20 requests in flight) so a few slow expensive queries cannot tie up every worker. POST /password-reset: two limits must both pass:emailat 5 per hour, andip_prefixat 20 per hour. Together they stop both targeted harassment of one account and broad spraying across many.POST /sms/send: keytenant + destination_countrywith a daily cost cap. SMS pumping fraud targets expensive destinations.
What Stripe runs in production
Stripe has described four layers: a request-rate limiter (per user, token bucket), a concurrent-requests limiter (caps in-flight requests per user), a fleet usage load shedder (reserves capacity for critical traffic such as charges), and a worker utilisation load shedder (drops low-priority traffic when workers are saturated). The last two do not protect fairness between users. They protect the system as a whole, and they key on the traffic class rather than on a user.
Failure behaviour
Most APIs fail open, because the limiter should not become a single point of failure, and they keep a coarse in-memory limit on each instance as a backstop. Security-sensitive endpoints such as login, OTP and payments often fail closed.