feat: Add ClickHouse as a new accepted sink format

[malon64]

Summary

Add ClickHouse as a new accepted sink format, enabling Floe to write quality-gated data directly into a ClickHouse table.

Motivation

ClickHouse is one of the most widely adopted OLAP databases in data platforms. It is a natural landing target for a data quality gate: raw data enters Floe, gets validated, and the accepted rows land directly in a ClickHouse table ready for analytics — without an intermediate file stage.

Technical Feasibility

The official clickhouse Rust crate supports inserting Arrow data via the HTTP interface (FORMAT ArrowStream), which means the existing Arrow conversion layer in Floe is directly reusable.

Additionally, ClickHouse's merge tree engines (ReplacingMergeTree, AggregatingMergeTree) provide server-side upsert/deduplication semantics that complement Floe's merge_scd1 intent at the engine level.

Proposed Implementation

1. New adapter

Create crates/floe-core/src/io/write/clickhouse.rs implementing AcceptedSinkAdapter:

use clickhouse::Client;

struct ClickHouseAcceptedAdapter;

impl AcceptedSinkAdapter for ClickHouseAcceptedAdapter {
    fn write_accepted(&self, target, df, mode, ..., entity) -> FloeResult<AcceptedWriteOutput> {
        let cfg = entity.sink.accepted.clickhouse.as_ref()
            .ok_or("missing clickhouse config")?;
        let batch = dataframe_to_record_batch(df)?;
        // Serialize RecordBatch as Arrow IPC stream
        // POST to ClickHouse HTTP endpoint: INSERT INTO {table} FORMAT ArrowStream
        let client = Client::default().with_url(&cfg.url).with_database(&cfg.database);
        client.insert_arrow(&cfg.table, batch).await?;
    }
}

2. Dispatch registration

// io/format.rs
"clickhouse" => Ok(io::write::clickhouse::clickhouse_accepted_adapter()),

3. Config extension

Add a clickhouse: Option<ClickHouseSinkConfig> field to SinkTarget:

pub struct ClickHouseSinkConfig {
    pub url: String,        // e.g. "http://localhost:8123"
    pub database: String,
    pub table: String,
    pub credentials: Option<String>, // storage reference for user/password
}

User config:

sink:
  accepted:
    format: clickhouse
    clickhouse:
      url: http://my-clickhouse-host:8123
      database: analytics
      table: events

4. Target enum note

ClickHouse is a network target, not a file-path target. SinkTarget.path is unused for this format. The connection details live entirely in ClickHouseSinkConfig. This is the same pattern that would apply to any database sink.

Write Modes

Mode	Support
append	✅ INSERT INTO
overwrite	✅ TRUNCATE TABLE then INSERT INTO
merge_scd1	⚠️ Deferred — can be delegated to ReplacingMergeTree engine semantics
merge_scd2	❌ Out of scope

Known Constraints

• No Target enum involvement: ClickHouse is addressed by URL, not by a cloud storage path.
• Credentials management: user/password should reference an existing Floe storage credentials entry to avoid plaintext secrets in config.
• Schema enforcement: the target ClickHouse table must pre-exist with a compatible schema. Floe will not create or alter ClickHouse tables.

Out of Scope (for now)

• Automatic table creation / DDL management
• ClickHouse Cloud native interface (HTTP endpoint is compatible)
• SCD2 merge support
• Rejected rows sink to ClickHouse