From d7cedba4276b0e53e78c3e3e35122b883d119d1b Mon Sep 17 00:00:00 2001 From: colmsnowplow Date: Tue, 7 Jul 2026 17:36:06 +0100 Subject: [PATCH 1/2] Add ClickHouse event forwarding integration page Document the ClickHouse forwarding destination: prerequisites, creating the destination table, configuration, validation, delivery/deduplication semantics, identity, and a schema reference for the default field mapping. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../integrations/clickhouse/index.md | 91 +++++++++++++++++++ .../forwarding-events/integrations/index.md | 1 + .../Schemas/clickhouse.json | 56 ++++++++++++ 3 files changed, 148 insertions(+) create mode 100644 docs/destinations/forwarding-events/integrations/clickhouse/index.md create mode 100644 src/components/EventForwardingSchemaTable/Schemas/clickhouse.json diff --git a/docs/destinations/forwarding-events/integrations/clickhouse/index.md b/docs/destinations/forwarding-events/integrations/clickhouse/index.md new file mode 100644 index 000000000..e6179bcdd --- /dev/null +++ b/docs/destinations/forwarding-events/integrations/clickhouse/index.md @@ -0,0 +1,91 @@ +--- +title: "Forward events to ClickHouse" +sidebar_label: "ClickHouse" +description: "Send Snowplow events to ClickHouse for real-time analytics, inserting events into a table you define through the ClickHouse HTTP interface." +sidebar_position: 4 +keywords: ["clickhouse", "event forwarding", "real-time analytics", "database", "columnar"] +date: "2026-07-07" +--- + +```mdx-code-block +import EventForwardingSchemaTable from '@site/src/components/EventForwardingSchemaTable'; +import clickhouseSchema from '@site/src/components/EventForwardingSchemaTable/Schemas/clickhouse.json'; +``` + +Send Snowplow events to ClickHouse to store behavioral data in a fast, column-oriented database for real-time analytics. The integration inserts events into a table you define using ClickHouse's [HTTP interface](https://clickhouse.com/docs/en/interfaces/http). + +## Prerequisites + +Before setting up the forwarder in Console, you'll need the following from your ClickHouse Cloud service: + +- **Endpoint host**: your ClickHouse Cloud HTTPS endpoint, including the port (usually `8443`), found under **Connect** in the ClickHouse Cloud console. Enter it as `host:port`, without the `https://` scheme. +- **Database**: the database containing your destination table. +- **Table**: the table to insert events into. You must create this table before forwarding, as described below. +- **Username** and **Password**: a ClickHouse user with `INSERT` privileges on the table. + +## Getting started + +Set up your ClickHouse table first, then create the connection and forwarder in Console. + +### Create the destination table + +ClickHouse does not create or alter tables for you, so create the destination table before forwarding. Each field you map must match a column in the table: a mapped field with no matching column fails the insert rather than being silently dropped. Map only the columns your table defines, and add columns for any other fields you want to forward. + +The default field mapping targets the eight columns below. A table that matches it looks like this: + +```sql +CREATE TABLE default.snowplow_events +( + event_id String, + event_name String, + app_id String, + user_id String, + collector_tstamp DateTime64(3), + web_page_id String, + domain_userid String, + domain_sessionid String +) +ENGINE = MergeTree +ORDER BY (collector_tstamp, event_id) +``` + +Timestamp fields are sent as ISO 8601 strings and parsed by ClickHouse, so `DateTime` and `DateTime64` columns accept them directly. + +Database and table names must be valid ClickHouse identifiers: they start with a letter or underscore and contain only letters, digits, and underscores (matching `^[A-Za-z_][A-Za-z0-9_]*$`). Names containing dots, dashes, spaces, or other characters are not supported. + +### Configure the destination + +To create the connection and forwarder, follow the steps in [Creating forwarders](/docs/destinations/forwarding-events/creating-forwarders/index.md). + +When configuring the connection, select **ClickHouse** for the connection type and enter your endpoint host, database, table, username, and password. When configuring the forwarder, map each field to a column in your table, using the default mapping in the schema reference below as a starting point. + +### Validate the integration + +You can confirm events are arriving by querying the table from the ClickHouse Cloud SQL console or any ClickHouse client: + +```sql +SELECT count() FROM default.snowplow_events +``` + +To inspect recent rows: + +```sql +SELECT * FROM default.snowplow_events ORDER BY collector_tstamp DESC LIMIT 10 +``` + +## Delivery and deduplication + +Event forwarding delivers events at least once. When a request fails transiently and is retried, a batch that ClickHouse already inserted can be written again, which produces duplicate rows. Design your table and queries to tolerate duplicates, keyed on `event_id`: + +- Deduplicate at query time, for example with `SELECT ... FROM snowplow_events GROUP BY event_id` or by selecting the latest row per `event_id` +- Use a [`ReplacingMergeTree`](https://clickhouse.com/docs/en/engines/table-engines/mergetree-family/replacingmergetree) engine ordered by `event_id` so duplicate rows collapse during background merges. Merges run asynchronously, so use `FINAL` or query-time deduplication when you need duplicate-free results immediately. + +## Identity management + +ClickHouse is a database rather than an identity-resolution destination, so there is no identify or alias concept. Identity is whichever identifier columns you map. The default mapping includes `user_id` (business identifier), `domain_userid` (first-party device cookie), and `domain_sessionid` (session identifier), and `event_id` serves as the natural deduplication key. Map whichever identifier columns match your table. + +## Schema reference + +This section contains the fields the default mapping sends to ClickHouse, including field names, data types, and default Snowplow mapping expressions. All fields are optional and fully configurable: remove any field your table does not define, and add columns for any other fields you want to forward. + + diff --git a/docs/destinations/forwarding-events/integrations/index.md b/docs/destinations/forwarding-events/integrations/index.md index 4fbe2963d..5efba271d 100644 --- a/docs/destinations/forwarding-events/integrations/index.md +++ b/docs/destinations/forwarding-events/integrations/index.md @@ -15,4 +15,5 @@ Snowplow event forwarding supports the following destinations: - [Amplitude](/docs/destinations/forwarding-events/integrations/amplitude/index.md) - [Bloomreach](/docs/destinations/forwarding-events/integrations/bloomreach/index.md) - [Braze](/docs/destinations/forwarding-events/integrations/braze/index.md) +- [ClickHouse](/docs/destinations/forwarding-events/integrations/clickhouse/index.md) - [Mixpanel](/docs/destinations/forwarding-events/integrations/mixpanel/index.md) diff --git a/src/components/EventForwardingSchemaTable/Schemas/clickhouse.json b/src/components/EventForwardingSchemaTable/Schemas/clickhouse.json new file mode 100644 index 000000000..54c5ced0c --- /dev/null +++ b/src/components/EventForwardingSchemaTable/Schemas/clickhouse.json @@ -0,0 +1,56 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "type": "object", + "required": [], + "additionalProperties": true, + "properties": { + "event_id": { + "type": "string", + "description": "Unique identifier for the Snowplow event. Suggested ClickHouse column type: String. A natural primary or deduplication key.", + "consoleDefault": "event.event_id", + "consoleRequired": false + }, + "event_name": { + "type": "string", + "description": "Name of the Snowplow event. Suggested ClickHouse column type: String.", + "consoleDefault": "event.event_name", + "consoleRequired": false + }, + "app_id": { + "type": "string", + "description": "Application identifier the event was tracked from. Suggested ClickHouse column type: String.", + "consoleDefault": "event.app_id", + "consoleRequired": false + }, + "user_id": { + "type": "string", + "description": "Business or user-supplied user identifier. Suggested ClickHouse column type: String.", + "consoleDefault": "event.user_id", + "consoleRequired": false + }, + "collector_tstamp": { + "type": "string", + "description": "Timestamp the event was received by the Collector, in ISO 8601 format. Parsed by ClickHouse using flexible date-time parsing. Suggested ClickHouse column type: DateTime64(3).", + "consoleDefault": "event.collector_tstamp", + "consoleRequired": false + }, + "web_page_id": { + "type": "string", + "description": "Page view identifier from the Snowplow web page entity. Suggested ClickHouse column type: String. A missing entity produces an omitted field rather than an error.", + "consoleDefault": "event.contexts_com_snowplowanalytics_snowplow_web_page_1?.[0]?.id", + "consoleRequired": false + }, + "domain_userid": { + "type": "string", + "description": "First-party cookie identifier for the user's device. Suggested ClickHouse column type: String.", + "consoleDefault": "event.domain_userid", + "consoleRequired": false + }, + "domain_sessionid": { + "type": "string", + "description": "First-party cookie identifier for the user's session. Suggested ClickHouse column type: String.", + "consoleDefault": "event.domain_sessionid", + "consoleRequired": false + } + } +} From ea5d0c1e44896784b8fddd9c20784737e7744070 Mon Sep 17 00:00:00 2001 From: colmsnowplow Date: Wed, 8 Jul 2026 10:30:36 +0100 Subject: [PATCH 2/2] Trim redundant column-mapping guidance on ClickHouse page Co-Authored-By: Claude Opus 4.8 (1M context) --- .../forwarding-events/integrations/clickhouse/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/destinations/forwarding-events/integrations/clickhouse/index.md b/docs/destinations/forwarding-events/integrations/clickhouse/index.md index e6179bcdd..d768c6da7 100644 --- a/docs/destinations/forwarding-events/integrations/clickhouse/index.md +++ b/docs/destinations/forwarding-events/integrations/clickhouse/index.md @@ -29,7 +29,7 @@ Set up your ClickHouse table first, then create the connection and forwarder in ### Create the destination table -ClickHouse does not create or alter tables for you, so create the destination table before forwarding. Each field you map must match a column in the table: a mapped field with no matching column fails the insert rather than being silently dropped. Map only the columns your table defines, and add columns for any other fields you want to forward. +ClickHouse does not create or alter tables for you, so create the destination table before forwarding. Each field you map must match a column in the table: a mapped field with no matching column fails the insert rather than being silently dropped. The default field mapping targets the eight columns below. A table that matches it looks like this: