CDC, outbox & projections

Senior loops punish naive dual-write (update DB + fire-and-forget cache/search). This page is the standard fix vocabulary: CDC, transactional outbox, and projections.

Quick mental model

Transactional outbox (microservices default story)

1. Begin transaction.
2. Persist domain change + outbox row (topic, payload, schema_version).
3. Commit (atomic).
4. Relay process publishes to broker (Kafka/SNS/SQS/Rabbit) and marks outbox sent or deletes with tombstone policy.
5. Consumers process idempotently (natural key / dedupe table).

Node.js: relay can be separate worker process; use SELECT FOR UPDATE SKIP LOCKED class patterns for concurrent relays where supported.

CDC (change data capture)

Sources: Postgres logical replication, Debezium on Kafka Connect, DMS, RDS native feeds—vendor-specific.

Flow: WAL/binlog → capture → stream → warehouse, Elasticsearch, analytics.

Pros: No second write in request path; fewer partial states.
Cons: lag, ordering per partition, schema evolution, ops burden.

┌────────┐    WAL / binlog     ┌──────────┐     ┌─────────────┐
│  OLTP  │ ─────────────────►  │  CDC     │ ──► │ Warehouse / │
│  (RDS) │     (ordered log)   │ connector│     │ search proj.│
└────────┘                     └──────────┘     └─────────────┘

Projections & read models

Projection = materialized view for a read path (search doc, cache warm list, dashboard aggregates).

Rules staff state aloud:

• Rebuild path if projection drifts (replay from outbox or CDC).
• Version events; tolerate duplicate delivery.
• Backpressure consumers when downstream slow.

When to choose which

Diagram — compare three integration shapes

See also

• AWS data services & Node.js at scale
• /backend/topics/idempotency-at-least-once
• Application caching & consistency