Under the Hood: Serialization Formats & Wire Protocols
Selecting between REST (typically JSON over HTTP/1.1 or HTTP/2) and gRPC (Protocol Buffers over HTTP/2) is fundamentally a trade-off between human readability/client reach and binary serialization throughput/CPU efficiency.
Wire Format Comparison: JSON vs Protocol Buffers
Protocol Mechanics: HTTP/1.1 vs HTTP/2 in gRPC
- Head-of-Line Blocking: In HTTP/1.1, a single TCP connection processes one request-response pair at a time. Concurrency requires opening multiple TCP sockets (typically 6 per browser host). gRPC runs over HTTP/2, where binary frames are multiplexed concurrently across a single persistent TCP connection.
- Schema Enforcement: REST over JSON relies on optional runtime validators (JSONSchema). gRPC uses strictly typed
.protofiles compiled into typed language bindings at build time, catching many type mismatches at compile time (proto3 default values, unknown fields and version skew between services can still cause runtime surprises). - Bidirectional Streaming: gRPC natively supports client streaming, server streaming, and bidirectional streaming over HTTP/2, making it ideal for live sensor telemetry and real-time backend synchronization.
When to Use Which?
| Feature | REST (JSON over HTTP/1.1 / HTTP/2) | gRPC (Protobuf over HTTP/2) |
|---|---|---|
| Payload Encoding | Human-readable ASCII text / UTF-8 | Compact, binary-encoded varints |
| Parsing Speed | Moderate to slow (string scanning & memory allocations) | Fast (binary decoding, no text scanning; standard libraries still allocate) |
| Browser Support | Native across 100% of browsers and devices | Requires gRPC-Web proxy translation layer |
| Ideal Use Case | Public APIs, mobile apps, third-party integrations | High-throughput East-West internal microservices |
Worked example: the same message in JSON and Protobuf
A user record {"user_id": 98412, "email": "[email protected]", "active": true} is 56 bytes as compact JSON (61 with the spaces shown). With a Protobuf schema int64 user_id = 1; string email = 2; bool active = 3; the encoding is:
- user_id: 1 tag byte plus a 3 byte varint for 98412, 4 bytes.
- email: 1 tag byte, 1 length byte, 14 bytes of text, 16 bytes.
- active: 1 tag byte plus 1 value byte, 2 bytes.
Total 22 bytes, about 2.5 times smaller. Most of the saving comes from not repeating field names. The string content is identical in both, so text-heavy messages shrink much less, and with gzip applied to both formats the gap narrows further. Parsing is usually faster for Protobuf because it avoids text scanning and number parsing, but standard Protobuf libraries still allocate objects; zero-copy decoding is a property of formats such as FlatBuffers and Cap'n Proto.
HTTP versions in one table
| Version | Transport | Concurrency on one connection | Head-of-line blocking |
|---|---|---|---|
| HTTP/1.1 | TCP | One request at a time (pipelining exists but is effectively unused) | At the HTTP layer; browsers open about 6 connections per host to compensate |
| HTTP/2 | TCP | Many multiplexed streams | Removed at the HTTP layer, but one lost TCP packet stalls all streams |
| HTTP/3 | QUIC over UDP | Many independent streams | A lost packet stalls only the streams whose data it carried |
Choosing
| Situation | Prefer | Why |
|---|---|---|
| Public API for third parties | REST with JSON and OpenAPI | Universal tooling, curl-friendly, HTTP caching |
| Browser front end | REST or GraphQL | Native browser support |
| High-volume internal service calls | gRPC | Generated typed clients, compact encoding, deadlines built in |
| Streaming between services | gRPC streaming | Client, server and bidirectional streams are first class |
| Cacheable reads at a CDN | REST GET | URLs and cache headers work with every cache |
Operational gotchas
- Load balancing gRPC: HTTP/2 keeps one long-lived connection, so an L4 load balancer pins all of a client's calls to one backend. Use L7 (per-request) balancing or client-side balancing.
- Schema evolution: never reuse a Protobuf field number; mark removed numbers as reserved.
- Debuggability: binary payloads need tools like grpcurl and server reflection.