Skip to main content
This guide covers pipelines that read from ClickHouse. ClickHouse is available as a source; ClickHouse as a destination is coming soon.

​
Validated connector routes

The following route was exercised through the Arcyria browser workflow with real source previews and destination verification: This is a functional verification, not a throughput benchmark. Validate the route on your own clusters before promoting it to production. Other destinations are not yet validated for ClickHouse sources.

​
Required access

The source user needs read-only access to the source tables. See the ClickHouse source connection guide for the arcyria_reader setup. For the PostgreSQL destination, grant SELECT, INSERT, and UPDATE on the destination schema.

​
Connection setup

Enable Secure / TLS for cloud and remote deployments. The HTTP port defaults to 8443 for secure connections and 8123 for plain local development.

​
Full Table flow

  1. Open Connections and create the ClickHouse source and PostgreSQL destination connections.
  2. Open Pipelines and click + New Pipeline.
  3. Choose the ClickHouse connection and select one or more tables.
  4. Choose the destination connection and enter the published output target.
  5. Open Transformations and create a SQL model that maps the source columns to the destination columns.
  6. Save, validate, preview, and publish the revision.
  7. Run the pipeline once and compare the rows written value with the destination count.

​
Incremental flow

  1. Set the stream’s sync mode to Incremental and pick a cursor column. Prefer a timestamp that is part of the table’s sorting or partition key.
  2. Run the pipeline twice and confirm that the second run only delivers the new rows.
Incremental runs do not see ClickHouse updates made with ALTER UPDATE or rows removed by deletes, unless your writes also change the cursor column.

​
ClickHouse table behavior

  • Queue tables (Kafka, RabbitMQ, NATS, FileLog, S3Queue, AzureQueue) are never read, because reading them would consume messages. Read the table a materialized view writes to instead.
  • ReplacingMergeTree and other merge engines can return more than one row per key until ClickHouse merges them. Turn on Read with FINAL in the stream’s incremental settings to read one row per key; reads are slower.
  • First loads of ClickHouse tables run in one pass and cannot resume, because ClickHouse keys are not unique. With Read with FINAL on and the table’s ORDER BY columns as the stream key, the first load resumes in chunks.

​
Troubleshooting

  • Connection refused: verify host, HTTP port, network allowlist, and firewall rules.
  • Authentication failed: verify username, password, and database name.
  • TLS connection failure: check the Secure / TLS toggle and the HTTP port. For self-hosted ClickHouse on TLS, make sure the certificate is publicly trusted or supply the CA certificate.
  • No tables discovered: grant SELECT on the source database and tables. System databases are filtered out by design.
  • A table cannot be read: the table uses a queue or streaming engine; see ClickHouse table behavior.
See also the ClickHouse source guide and the PostgreSQL incremental example.