Under the Hood: The Enterprise Product Event Bus
In a microservices organization, business changes (e.g. OrderPlaced, UserSignedUp, SubscriptionCanceled) must be broadcast to dozens of downstream services (search indexing, email marketing, fraud detection, analytics, and accounting). An enterprise event bus coordinates these product events safely.
The Event-Driven Topology with Outbox & Schema Registry
Under the Hood: Solving the Dual-Write Disaster
A fatal anti-pattern in distributed systems is writing to a database and publishing to a message broker in application code:
// THE DUAL-WRITE DISASTER: NEVER DO THIS
await db.orders.insert(order);
await kafka.publish("order_events", order); // What if this network call fails or server crashes?
If the Kafka write fails or the process crashes between lines, the database has the order but downstream workers never receive the event. If you reverse the order, the event is emitted but the database write could fail.
The Transactional Outbox Pattern: The application writes both the order record and an outbox event into the same local database transaction. A Change Data Capture (CDC) engine such as Debezium tails committed database changes and publishes them to Kafka. This closes the application dual-write window, but publication can be delayed or repeated, so consumers still need schema handling, idempotent effects, monitoring, and retention sufficient for recovery.
Worked example: an outbox row to three consumers
BEGIN;
INSERT INTO orders (id, user_id, total_cents, status)
VALUES ('o_981', 'u_42', 4599, 'PLACED');
INSERT INTO outbox (id, aggregate_type, aggregate_id, type, payload)
VALUES ('evt_7f3', 'order', 'o_981', 'OrderPlaced',
'{"orderId":"o_981","userId":"u_42","totalCents":4599,"schemaVersion":3}');
COMMIT;Debezium reads the committed outbox insert from the Postgres WAL and routes it to topic order.events, keyed by o_981 so that all events for this order stay in one partition, in order. Search, fraud and email each consume in their own consumer group. If Debezium crashes after publishing but before recording its WAL position, it republishes evt_7f3 on restart. Consumers deduplicate on the event ID.
Event styles (Fowler's taxonomy)
| Style | Payload | Consumer coupling | Trade-off |
|---|---|---|---|
| Event notification | ID plus type only | Consumers call back the producer's API | Small events, but a callback load spike on the producer |
| Event-carried state transfer | Full relevant state | Consumers keep local copies | No callbacks, but larger events and schema weight |
| Event sourcing | Every state change is the source of truth | Rebuild state by replay | Full audit trail, but hard schema evolution |
Schema evolution rules (Avro or Protobuf with a registry)
- Backward compatible (new consumers read old events): you may delete fields, or add fields that have defaults.
- Forward compatible (old consumers read new events): you may add fields, or delete fields that have defaults.
- Never change a field's type or reuse a Protobuf field number.
- Include an envelope: event ID, type, source, occurrence time, schema version, and optionally a trace ID. CloudEvents standardises these attributes.
Failure modes
- Outbox table growth: delete or partition rows once they are published, or the CDC-read table bloats.
- WAL retention: a stopped Debezium replication slot makes Postgres keep WAL indefinitely, which can fill the disk. Alert on slot lag.
- Event storms: a backfill publishes 50 million events and swamps consumers. Throttle backfills, or give them a separate topic.