Skip to main content
MongoDB is available as an Available Now destination connector. Arcyria delivers only the published SQL model’s output through a custom dlt destination backed by PyMongo. Delivery uses bounded unordered bulk writes and removes internal dlt fields before documents reach the destination collection.

​
Required access

Use a dedicated writer scoped to the destination database:
The writer needs permission to read index metadata and insert or update documents. If your operating policy requires pre-created collections, validators, or indexes, create those with a separate administrative identity.

​
Connection fields

The complete URI is encrypted as one secret and masked after saving. URI query options such as authSource, replicaSet, and tls=true are passed to PyMongo.

​
Create the connection

  1. Open Connections.
  2. Click + New Connection.
  3. Set the role to Destination.
  4. Choose MongoDB.
  5. Enter the complete URI and destination database.
  6. Click Test Connection.
  7. Save after the test succeeds.

​
Prepare the collection

Create the collection before the first run. Arcyria never creates or drops collections, and a run whose target collection does not exist stops before reading the source, with the command to create it:
Create a unique index for the selected Upsert key:
Without an index, Upsert remains correct but may perform poorly as the collection grows. Arcyria reports a warning when it cannot verify a supporting index.

​
Configure delivery

  1. Open Destinations and click Add destination.
  2. Choose the MongoDB connection and target database.
  3. Open Transformations, select the MongoDB destination, and create the SQL model.
  4. Save, validate, preview, and publish the revision.
  5. Set the published output target to database.collection, such as analytics.orders.
  6. Select one or more stable, non-null Upsert keys.
  7. Save the destination and run the pipeline.
The source and destination cannot be the same logical MongoDB database and collection in one pipeline.

​
Write behavior

The public MongoDB destination uses Upsert. Every output row must contain a non-null value for each selected merge key. Matching documents are updated; new documents are inserted.
  • _id is the default stable key for MongoDB-to-MongoDB pipelines.
  • A business identifier such as external_id can be used for SQL sources.
  • Internal fields whose names begin with _dlt are never written.
  • A rerun with the same keys updates existing documents instead of duplicating them.
Internal APPEND support exists for controlled runtime operations but is not advertised in the public UI. REPLACE is disabled because a portable atomic collection swap cannot be guaranteed.

​
Deletes

When the source stream reports deletes (CDC, or delete detection on an Incremental stream), Arcyria deletes the matching documents from the collection:
  • Only when the collection’s Upsert key is the same as the source key. Otherwise deletes are skipped and the run records why.
  • Keys are converted the same way as upserted values, so a delete matches the document the upsert wrote. If the stored key type differs, the run warns.
  • Deleting a document that is already gone is not an error, so a retried run gives the same result.
  • Hard delete only. Soft delete is skipped with a warning, because it would add fields your model does not output.
  • Truncate the destination removes every document from the target collection. The collection, its indexes, and its validator stay; other collections are never touched.
  • Without an index on the Upsert key, deletes scan the collection. Arcyria warns when it cannot find a supporting index.

​
Document limits and BSON mapping

Documents are BSON-encoded before network I/O. A document exceeding MongoDB’s 16 MiB limit is rejected with the collection and a bounded record identifier. Common SQL values map to BSON-safe strings, numbers, decimals, dates, binary values, arrays, and nested documents.

​
Verify delivery

The duplicate aggregation should return no rows. Compare the collection count with the run’s rows written value, accounting for transformation filters.

​
Current limitations

  • REPLACE and Change Streams are disabled.
  • Atlas, replica-set, sharded-cluster, and live TLS destination certification remains incomplete.
  • PostgreSQL-to-MongoDB and MongoDB-to-MongoDB have automated evidence. MySQL, ClickHouse, and manual UI pairings remain pending owner validation.
  • DuckDB connections are Coming Soon and cannot be selected for new pipelines.

​
Troubleshooting

See MongoDB source and the MongoDB and PostgreSQL verified paths.