Problem
The native chart-compute path crosses two different Apache Arrow implementations:
packages/engine-server encodes query results with apache-arrow (tableFromIPC).- The render side (Mosaic /
@uwdata/vgplot) decodes with @uwdata/flechette (a transitive dep of @uwdata/mosaic-core).
They agree on the Arrow IPC wire format but differ on JS value semantics — e.g. Date32/Date64 columns, where apache-arrow and flechette's .get() can hand back different JS shapes (Date object vs epoch-days vs epoch-millis). Today the knowledge of how the two libraries differ is scattered across native-engine.ts, the connector, and verified-by-hand comments rather than owned in one place.
This surfaced during review of the desktop native-chart work (#76): a Date branch in native-engine.ts was flagged as suspicious and had to be hand-verified to be correct. The next Arrow type with a semantics gap (decimals, timestamp-with-timezone, nested) will reintroduce the same bug class.
Proposal
Introduce a single anti-corruption boundary — one module/function that is the only place engine-produced Arrow values are translated into what the chart layer consumes. Every type with a cross-library semantics gap (Date32/64, decimal, timestamp-tz, nested) is handled once, there, with a test pinning value-equality across the boundary. The two Arrow libraries stay; only the translation knowledge is centralized.
Trigger (Rule of Three)
Not yet — currently only one type (Date) needs special handling and it's verified. Extract the bridge when a third Arrow type needs cross-library special-casing in the engine→chart path. Doing it now is premature abstraction; doing it never lets the value-semantics knowledge stay scattered.
Acceptance
- A single typed boundary (
toChartRows / encodeForChart / arrow-bridge) owns all apache-arrow→chart value translation.
- Each special-cased type has a test asserting value-equality across the engine→chart boundary.
- Fits the Execution Pipeline transport contract (one engine API, one typed transport boundary).
Context
Follow-up from #76 (desktop native chart compute). Not blocking — the current path works and the Date case is verified.
Problem
The native chart-compute path crosses two different Apache Arrow implementations:
packages/engine-serverencodes query results withapache-arrow(tableFromIPC).@uwdata/vgplot) decodes with@uwdata/flechette(a transitive dep of@uwdata/mosaic-core).They agree on the Arrow IPC wire format but differ on JS value semantics — e.g.
Date32/Date64columns, whereapache-arrowand flechette's.get()can hand back different JS shapes (Date object vs epoch-days vs epoch-millis). Today the knowledge of how the two libraries differ is scattered acrossnative-engine.ts, the connector, and verified-by-hand comments rather than owned in one place.This surfaced during review of the desktop native-chart work (#76): a Date branch in
native-engine.tswas flagged as suspicious and had to be hand-verified to be correct. The next Arrow type with a semantics gap (decimals, timestamp-with-timezone, nested) will reintroduce the same bug class.Proposal
Introduce a single anti-corruption boundary — one module/function that is the only place engine-produced Arrow values are translated into what the chart layer consumes. Every type with a cross-library semantics gap (Date32/64, decimal, timestamp-tz, nested) is handled once, there, with a test pinning value-equality across the boundary. The two Arrow libraries stay; only the translation knowledge is centralized.
Trigger (Rule of Three)
Not yet — currently only one type (Date) needs special handling and it's verified. Extract the bridge when a third Arrow type needs cross-library special-casing in the engine→chart path. Doing it now is premature abstraction; doing it never lets the value-semantics knowledge stay scattered.
Acceptance
toChartRows/encodeForChart/arrow-bridge) owns all apache-arrow→chart value translation.Context
Follow-up from #76 (desktop native chart compute). Not blocking — the current path works and the Date case is verified.