Under the Hood: Synchronous RPC vs. Asynchronous Messaging
When monolithic applications decompose into distributed services, communication transitions from in-memory function pointers (sub-microsecond) to network sockets subject to non-deterministic latency, packet loss, and connection pool exhaustion. Choosing between synchronous RPC and asynchronous messaging fundamentally dictates system resilience.
Communication Topologies Compared
Under the Hood: Production Communication Invariants
1. Distributed Context & Deadline Propagation
Every inbound user request must receive a unique W3C Trace Context (traceparent) and an absolute client deadline (e.g., grpc-timeout: 500m). If an upstream client specifies a 500ms timeout, and service A spends 450ms processing, downstream calls from service A to service B must only receive the remaining 50ms deadline. Continuing to execute downstream work after an upstream caller has already disconnected wastes CPU and database connections.
2. Bounded Connection Pools and Keep-Alive
Creating TCP/TLS handshakes for every inter-service call introduces substantial latency overhead. Production services maintain persistent connection pools with HTTP/2 multiplexing or TCP keep-alive, strictly bounding max connections to avoid overwhelming downstream servers with connection spikes.
Worked example: deadline budget across a call chain
A user request enters the API with a 1,000 ms deadline. The API calls the order service, which calls inventory and then pricing sequentially.
- API spends 50 ms on auth and parsing, then calls order with a remaining deadline of 950 ms.
- Order spends 30 ms, calls inventory. Inventory normally answers in 20 ms but is overloaded and hangs.
- Without propagation: order uses a fixed 2 s client timeout to inventory. The user's request fails at 1,000 ms, but order keeps a thread and a connection blocked for another 1 s, and inventory keeps working on a request nobody wants. Under load, this wasted work is what exhausts thread pools.
- With propagation: order passes the remaining 920 ms. gRPC does this automatically by sending the
grpc-timeoutheader; each hop subtracts elapsed time. When the deadline passes, every hop cancels its work.
A good rule is to set each hop's timeout to the smaller of its own latency target (for example p99.9 plus margin) and the remaining caller deadline.
Choosing a style
| Need | Synchronous (HTTP or gRPC) | Asynchronous (queue or log) |
|---|---|---|
| Caller needs the result to respond | Natural fit | Awkward; needs request/reply correlation |
| Downstream may be down for minutes | Caller fails too unless it has a fallback | Messages wait in the broker |
| Traffic bursts | Downstream must absorb the burst in real time | Broker buffers; consumers work at their own pace |
| Many consumers of one fact | Caller must call each one | Publish once; each subscriber reads independently |
| Debugging | Single trace, simple to follow | Needs correlation IDs across producer and consumer |
| Delivery semantics | Caller knows success or timeout (ambiguous) | At-least-once; duplicates and reordering possible |
Mechanics that matter
- Connection reuse: a new TCP plus TLS 1.3 connection costs at least 2 round trips before the first byte of the request. Pooled keep-alive connections or HTTP/2 multiplexing avoid this per call.
- Bounded pools: cap connections per destination so a slow dependency cannot absorb all of a caller's threads (the bulkhead pattern).
- Trace context: propagate the W3C
traceparentheader on calls and in message headers so one request can be followed through queues. - Transactional outbox: to update a database and publish an event without losing either, write the event to an outbox table in the same transaction and have a relay publish it.
How real systems do it
Google's internal RPC system (Stubby, the predecessor of gRPC) made deadline propagation a default. Uber, Netflix and LinkedIn use Kafka for asynchronous fan-out of events such as trips, plays and profile updates, while keeping request/response RPC for user-facing reads.