Skip to main content
CDC reads changes from PostgreSQL’s write-ahead log (WAL) through a logical replication slot. It captures inserts, updates, deletes, and truncates, and needs no cursor column. CDC is available for PostgreSQL sources. Set it up from Source → Change data capture, then choose CDC as a stream’s sync mode. See PostgreSQL CDC setup.

​
How a CDC run works

Each run is a micro-batch:
  1. Arcyria notes the current WAL position as the run’s target.
  2. It reads every change up to that target, in whole transactions, capped at 200,000 changes per run.
  3. When a row changed several times, only its latest state is kept.
  4. Changes are transformed by your models and upserted into the destination. Deletes are applied as you choose.
  5. Only after delivery succeeds does Arcyria confirm the batch on the source, so the source can release that WAL.
If a run fails at any point, nothing is confirmed. The next run reads the same changes again, and because delivery is an upsert, the destination ends in the same state. When more changes are waiting than one run can take, the next run starts right away instead of waiting for the schedule.

​
Latency

CDC runs follow your schedule. With a frequent schedule, changes typically arrive within a few minutes: each run has to start, transform, and deliver. This is near real-time, not continuous streaming.

​
First load

When a stream starts using CDC, it first loads the existing table in primary-key order, in chunks of 500,000 rows. Larger tables continue in follow-up runs. Changes made during the load are applied after it, so nothing is missed. While the first load runs, the source keeps WAL for the slot. Start CDC for large tables when you can let the load finish promptly.

​
CDC or Incremental?

​
What the source keeps

A replication slot makes PostgreSQL keep WAL until Arcyria confirms it. If CDC stops running, the source keeps WAL indefinitely and can run out of disk. Arcyria checks every slot every 5 minutes and alerts you when the WAL it holds passes your warning or critical threshold. Disabling CDC, or deleting the pipeline, drops the slot.