Under the Hood: L4/L7 Traffic Management Primitives
Engineers often conflate Load Balancers and API Gateways because modern proxies (like Envoy, Nginx, and Traefik) can perform both roles. However, in enterprise architecture, they operate at distinct layers of the OSI stack with different performance profiles and responsibilities.
The Multi-Layer Ingress Architecture
flowchart TD
Client["Client Traffic"] --> L4LB["Layer 4 Load Balancer (AWS NLB / Maglev: TCP/UDP Wire Speed)"]
L4LB --> L7GW["Layer 7 API Gateway (Envoy / Kong: HTTP Parsing)"]
subgraph GatewayFunctions ["API Gateway Cross-Cutting Concerns"]
L7GW --> Auth["JWT Auth & OAuth2 Verification"]
L7GW --> RateLimit["Distributed Rate Limiter (Redis Token Bucket)"]
L7GW --> Transform["Payload Transformation & Header Injection"]
L7GW --> Routing["Path-Based Routing (/v1/users -> User Service)"]
end
GatewayFunctions --> S1["User Microservice"]
GatewayFunctions --> S2["Order Microservice"]
GatewayFunctions --> S3["Payment Microservice"]
Comparison: Load Balancer vs API Gateway
| Dimension | Layer 4 / 7 Load Balancer (AWS NLB/ALB) | API Gateway (Envoy, Kong, Zuul) |
|---|---|---|
| OSI Layer | Layer 4 (TCP/UDP, e.g. AWS NLB) or Layer 7 (HTTP, e.g. AWS ALB, which also routes by host/path and can authenticate users via OIDC) | Layer 7 (Application Layer) |
| Primary Goal | Distribute raw network traffic evenly across servers | Provide unified business API orchestration & security |
| Cross-Cutting Features | Health checks, TLS termination, round-robin | JWT validation, rate limiting, request transformation, monetization |
| Throughput & Latency | Very high throughput at L4 (microsecond overhead); L7 balancers that terminate TLS pay similar TLS costs | Lower per-instance throughput; TLS plus JWT validation, policy plugins and transformations add CPU per request |
Worked example: a request through both
A mobile client calls GET /v1/orders/123.
- DNS resolves
api.example.comto an L4 load balancer (for example AWS NLB or Google Maglev). It picks a gateway instance by hashing the connection 5-tuple and forwards packets without reading HTTP. Overhead is microseconds. - The gateway (for example Envoy, Kong or AWS API Gateway) terminates TLS, validates the JWT signature and expiry, checks the caller's rate limit, and matches the path
/v1/orders/*to the order service. - The gateway picks one of the order service's healthy instances (its own L7 load balancing, for example least-requests) and forwards the request, adding headers such as the authenticated user ID and a trace ID.
- If the order service returns 503, the gateway may retry once on another instance, within a retry budget.
Rate limit arithmetic: if a partner is allowed 100 requests per second and the gateway fleet has 10 instances, either each instance enforces 10 per second locally (cheap, but uneven if traffic is not spread evenly) or all instances consult a shared counter (accurate, but an extra network hop of about 1 ms per request).
Responsibility table
| Capability | L4 load balancer | L7 load balancer | API gateway |
|---|---|---|---|
| Sees | IP, port, TCP or UDP | HTTP method, path, headers | HTTP plus API identity and policy |
| Spread load across instances | Per connection | Per request | Per request, often delegated |
| TLS termination | Optional (pass-through common) | Common | Common |
| Path or host routing | No | Yes | Yes |
| Authentication and API keys | No | Limited | Yes |
| Rate limiting and quotas per client | No | Basic | Yes, often per plan |
| Request or response transformation | No | Headers | Yes |
| Examples | AWS NLB, Google Maglev, IPVS | AWS ALB, NGINX, HAProxy, Envoy | Kong, Apigee, AWS API Gateway, Envoy-based gateways |
Failure modes
- Gateway as bottleneck: CPU for TLS and JWT validation scales with request rate; autoscale it and keep policies cheap.
- Timeouts that do not nest: if the gateway times out at 30 s but clients give up at 10 s, the gateway keeps work alive that nobody is waiting for.
- Health checks that lie: a TCP health check passes even when the HTTP application returns 500 for every request; use an HTTP health endpoint that checks the app's real readiness.