Skip to content

TCP J4: Add Dynamic support - #460

Open
alex-clickhouse wants to merge 14 commits into
tcp/epic-j3-variantfrom
tcp/epic-j4-dynamic
Open

TCP J4: Add Dynamic support#460
alex-clickhouse wants to merge 14 commits into
tcp/epic-j3-variantfrom
tcp/epic-j4-dynamic

Conversation

@alex-clickhouse

@alex-clickhouse alex-clickhouse commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

Adds the ClickHouse Dynamic column type over the native protocol, stacked on #459 (Variant).

Dynamic is a column whose per-row value type is discovered at runtime. This reads and writes only the FLATTENED serialization (wire version 3), gated on the query setting output_format_native_use_flattened_dynamic_and_json_serialization = 1; the leading UInt64 version word is validated and any non-flat encoding rejected as a protocol error. (A follow-up will have the client set that setting automatically — tracked in the TODO.)

What's here

  • Read + write of top-level Dynamic (FLATTENED v3): version + runtime type-name list + per-type prefixes as the state prefix, then discriminators (width scales with type count; NULL = num_types) + one dense run per type. Verified against a real 26.6 server; the documented-bytes unit tests use a real server capture.
  • Dense DynamicColumn (type-name list + widened discriminators + per-type child columns) — the zero-copy read/write source; an ergonomic IColumn<object> write source infers each value's ClickHouse type via a self-contained DynamicTypeInference (the TCP project can't reference the main driver): scalars, IPv4/IPv6 by address family, date-times, decimals, and Array/Map/Tuple recursion, normalizing each value to its codec's element type.
  • The J0b write-state contract: IColumnCodec.BeginWrite + IColumnWriteState, with state-aware WriteStatePrefix/WriteColumn overloads that default to the existing state-free ones (leaf codecs untouched). The block writer runs BeginWrite → prefix → body → Dispose, so a data-dependent prefix (the Dynamic type list) and the element-flattening composites do their flatten/scatter once across both phases.
  • Full nesting, both directions: Array/Tuple/Map/Nested now project their flattened inner sub-column into the child's prefix and body, so Array(Dynamic), Tuple(...Dynamic...), Map(K, Dynamic), Nested(...Dynamic...) and composite values inside a Dynamic all round-trip. Variant rejects a Dynamic alternative (server-disallowed).
  • Like Variant/Array/Tuple/Map/Nested, there is no Nullable(Dynamic) — NULL rides the discriminator.

Testing

  • Codec-level documented-bytes unit tests (write→bytes, bytes→read, dense round-trip, version-not-3 rejection, discriminator-past-count, width boundaries) and inference-mapping unit tests.
  • Integration round-trip cases (CREATE/INSERT/SELECT against a live server): top-level scalars + NULL, a "(basically) every type + composites" mega-case, DateTimeOffset/DateTime/decimal inference, and Array(Dynamic), Tuple(Dynamic, String), Tuple(Dynamic, Dynamic), Map(String, Dynamic), Nested(a Dynamic, b String), Array(Tuple(Dynamic, String)).
  • Full suite green (869 tests); DynamicColumnCodec ~90% line coverage (remaining gaps are the >255-type discriminator-width branches).

Notes / deferred

  • Composite-of-coercion-needing-element (e.g. DateTimeOffset[] inside a Dynamic) throws a clear NotSupportedException; use the canonical CLR type (ClickHouseDateTime64/ClickHouseDecimal).
  • A dictionary value is not inferred as a Map; pass KeyValuePair<K, V>[], the shape the map codec reads back. Inferring Map(K, V) from any IDictionary/IReadOnlyDictionary was tried as an experiment and dropped together with the dictionary write path in TCP I4: Add Map(K, V) support #443 — the interface walk needed a per-type plan cache and several rejections (ambiguous key/value pairs, multidimensional arrays) to stay correct, which was not worth it for a second spelling of a type the client already accepts.

🤖 Generated with Claude Code

@alex-clickhouse
alex-clickhouse requested a review from Copilot July 23, 2026 10:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown

Triage

Category: featureRisk: high

Summary
This PR adds read/write support for the ClickHouse Dynamic column type over the native TCP protocol, stacked on #459 (Variant). It introduces a new DynamicColumnCodec implementing only the "flattened" serialization (version 3, gated on the server query setting output_format_native_use_flattened_dynamic_and_json_serialization=1), a DynamicColumn dense-column shape, and a self-contained DynamicTypeInference helper that derives ClickHouse type names from CLR values at write time via recursive inference. The PR also threads a BeginWrite/IColumnWriteState write-state contract through composite codecs (Array, Tuple, Map, Nested) so the data-dependent type list is computed once and shared across the prefix and body write phases — enabling full nesting of Dynamic in both read and write directions. Coverage includes codec unit tests against captured server bytes, inference mapping tests, and integration round-trip cases for scalars, NULLs, every supported type in one column, composite nesting, datetime/decimal inference, and overflow buckets.

What this impacts

  • Binary protocol (TCP) — new serialization layout for Dynamic: version word, varuint type count, type-name list, per-type state prefixes, variable-width discriminators (1/2/4 bytes scaled by type count), and one dense run per type
  • ClickHouse.Driver.Tcp/Types/Codecs/ — new DynamicColumnCodec (613 lines), DynamicTypeInference (217 lines), DynamicWire (32 lines)
  • ClickHouse.Driver.Tcp/Types/ — new DynamicColumn and IDynamicColumn interface (210 lines)
  • ClickHouse.Driver.Tcp/Types/ColumnCodecRegistry.cs — Dynamic factory registration
  • ClickHouse.Driver.Tcp.Tests/ — 308 lines of new codec and inference unit tests, plus 178 lines of new InsertRoundTripCase entries covering full nesting

Concerns

  • Reflection in write pathDynamicColumnCodec.FlatBuilderFor uses MethodInfo.MakeGenericMethod/CreateDelegate via ConcurrentDictionary to build per-element-type flat-column builders. The result is cached (one-time cost per CLR element type), but reflection fires the high-risk performance rule; a reviewer should confirm the cache is sufficient and contention on ConcurrentDictionary is negligible.
  • Unbounded recursion in type inferenceDynamicTypeInference.Infer recurses into composite elements (Array → element, Map → key/value, Tuple → fields) with no depth guard, and composites may themselves contain composites. An adversarially or accidentally deeply nested input (e.g. Array(Array(Array(...)))) could overflow the stack; the high-risk recursion-on-unbounded-input rule fires.
  • DRAFT + stacked on TCP J3: Add Variant(...) support #459 — this diff is not standalone; the IColumnWriteState/BeginWrite interface and the updated composite codecs live in TCP J3: Add Variant(...) support #459. The full risk surface cannot be assessed without that stack, and this PR should not be merged before TCP J3: Add Variant(...) support #459.
  • Server opt-in required — only flattened serialization (version 3) is implemented; a TODO defers automatic injection of output_format_native_use_flattened_dynamic_and_json_serialization=1, meaning callers must set it manually today.

Required reviewer action

  • PR body must include an architectural description before review (high-risk policy). Additionally, TCP J3: Add Variant(...) support #459 must land and be reviewed first; review this PR's diff against that branch, not main.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (2)

ClickHouse.Driver.Tcp/Types/Codecs/TupleColumnCodec.cs:367

  • In the flat tuple write path, BuildState returns pooled childBoxed buffers in finally, but if one of the child codecs’ BeginWrite(...) calls throws, any earlier child states are leaked (they may hold rented buffers) because they’re never disposed on the exceptional path.
        try
        {
            for (int row = 0; row < length; row++)
            {
                var tuple = (ITuple)column.GetValue(start + row);
                for (int i = 0; i < arity; i++)
                {
                    childBoxed[i][row] = tuple[i];
                }
            }

            for (int i = 0; i < arity; i++)
            {
                childColumns[i] = childFlatBuilders[i](column.Name, children[i].TypeName, childBoxed[i], length);
                childStates[i] = children[i].BeginWrite(childColumns[i], 0, length);
            }

            return new TupleWriteState { ChildColumns = childColumns, ChildStart = 0, Length = length, ChildStates = childStates };

ClickHouse.Driver.Tcp/Types/Codecs/MapColumnCodec.cs:392

  • In the jagged map path, if one of the inner BeginWrite(...) calls throws (especially valueCodec.BeginWrite after keyState is created), the already-created inner state is leaked. The exception path should dispose any created inner state before returning the pooled key/value buffers.
            var keyColumn = ArrayColumn<TKey>.OverBuffer(column.Name, keyCodec.TypeName, flatKeys, total);
            var valueColumn = ArrayColumn<TValue>.OverBuffer(column.Name, valueCodec.TypeName, flatValues, total);
            IColumnWriteState keyState = keyCodec.BeginWrite(keyColumn, 0, total);
            IColumnWriteState valueState = valueCodec.BeginWrite(valueColumn, 0, total);
            return new MapWriteState(keyColumn, valueColumn, pairBase: 0, total, keyState, valueState, flatKeys, flatValues);
        }
        catch
        {
            ArrayPool<TKey>.Shared.Return(flatKeys, clearArray: RuntimeHelpers.IsReferenceOrContainsReferences<TKey>());
            ArrayPool<TValue>.Shared.Return(flatValues, clearArray: RuntimeHelpers.IsReferenceOrContainsReferences<TValue>());
            throw;

Comment thread ClickHouse.Driver.Tcp/Types/ColumnCodecExtensions.cs
Comment thread ClickHouse.Driver.Tcp/Types/Codecs/TupleColumnCodec.cs Outdated
Comment on lines +332 to +340
var fieldColumns = new IColumn[children.Length];
var fieldStates = new IColumnWriteState[children.Length];
for (int f = 0; f < children.Length; f++)
{
children[f].WriteColumn(writer, nested.GetField(f), elementBase, elementCount);
fieldColumns[f] = nested.GetField(f);
fieldStates[f] = children[f].BeginWrite(fieldColumns[f], elementBase, elementCount);
}

return new NestedWriteState { FieldColumns = fieldColumns, ElementBase = elementBase, ElementCount = elementCount, FieldStates = fieldStates };
Comment thread ClickHouse.Driver.Tcp/Types/Codecs/MapColumnCodec.cs Outdated
@alex-clickhouse

Copy link
Copy Markdown
Collaborator Author

Thanks @copilot — all four inline findings plus the two low-confidence ones were valid (the same pooled-buffer / partial-child-state leak class). Addressed in the latest push:

  • ColumnCodecExtensions.WriteFull — now returns early for a zero-row column, matching BlockWriter (which skips prefix+body for empty slices), so it can't emit bytes the block writer never would or trip a codec that rejects empty ranges.
  • TupleColumnCodec.BuildState (dense + flat paths) — both loops now dispose the child write states already built (via a DisposeStates helper) if a later child's BeginWrite throws.
  • NestedColumnCodec.BuildState — the field-state loop now disposes already-created field states on a mid-loop throw.
  • MapColumnCodec MapShape.BeginWrite (dense + jagged paths) — a keyState created before valueCodec.BeginWrite throws is now disposed before rethrowing (jagged path also still returns the pooled key/value buffers).

These mirror the try/catch cleanup already applied to DynamicColumnCodec's BuildScatteredState/BuildDenseState. Full suite green (869 tests).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Comment thread ClickHouse.Driver.Tcp/Types/Codecs/DynamicTypeInference.cs Outdated
Comment on lines +173 to +182
private static string InferFromClrType(Type type)
{
if (type is not null && Scalars.TryGetValue(type, out string scalar))
{
return scalar;
}

throw new NotSupportedException(
$"No ClickHouse type is inferred for a Dynamic array whose element CLR type is '{type?.ToString() ?? "unknown"}' with no element to disambiguate it.");
}
Comment thread ClickHouse.Driver.Tcp/Types/Codecs/DynamicColumnCodec.cs
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-j4-dynamic branch 2 times, most recently from 5925968 to c4f7c01 Compare August 10, 2026 09:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

Suppressed comments (6)

ClickHouse.Driver.Tcp/Types/Codecs/DynamicTypeInference.cs:102

  • Returning the original Array is unsafe when value-based inference chooses a different CLR element shape. For example, object[] { 1 } infers Array(Int32) but remains an object[]; the write planner later builds the Int32[] child and BuildFlatColumn<int[]> throws InvalidCastException. int?[] { 1, null } similarly infers non-nullable Array(Int32). Canonicalize the array to the inferred codec shape (including nullability), or reject these shapes here with a clear error.
        if (value is Array array)
        {
            return (InferArrayOrMap(array), value);

ClickHouse.Driver.Tcp/Types/Codecs/DynamicTypeInference.cs:117

  • Map inference uses only KeyValuePair<K,V>'s declared generic arguments, so supported value-dependent/null-capable shapes cannot be inferred even from populated maps. For example, KeyValuePair<string, IPAddress>[] throws because InferFromClrType(IPAddress) cannot distinguish IPv4/IPv6, and KeyValuePair<string, uint?>[] also throws. Inspect present pairs and validate a homogeneous inferred key/value type, with declared-type fallback only for empty maps.
        Type elementType = array.GetType().GetElementType();
        if (elementType is not null && elementType.IsGenericType && elementType.GetGenericTypeDefinition() == typeof(KeyValuePair<,>))
        {
            Type[] pair = elementType.GetGenericArguments();
            return $"Map({InferFromClrType(pair[0])}, {InferFromClrType(pair[1])})";

ClickHouse.Driver.Tcp/Types/DynamicColumn.cs:129

  • Values materializes and caches rows independently, while all current round-trip/decoder assertions call GetValue, so this new accessor is never executed. AGENTS.md:165-170 requires dedicated coverage for independently materialized Values caches. Add a test asserting value order, NULL handling, and repeated cached access.
    public ReadOnlySpan<object> Values

ClickHouse.Driver.Tcp/Types/ColumnCodecRegistry.cs:162

  • Registering Dynamic adds user-visible TCP client behavior, but this PR does not update either release document. AGENTS.md:303-305 requires every behavioral client change to update both CHANGELOG.md and RELEASENOTES.md; add a concise TCP Dynamic support entry to each.
        AddFactory("Dynamic", static (TypeNode node, in ResolveContext context, ColumnCodecRegistry registry) => DynamicColumnCodec.Create(node, context, registry));

ClickHouse.Driver.Tcp/Types/Codecs/DynamicTypeInference.cs:176

  • The suggested raw-long workaround cannot represent DateTime64 inside a Dynamic composite: Infer(long) always selects Int64, so DateTimeOffset[] is rejected while long[] becomes Array(Int64), not Array(DateTime64(9)). There is also no ClickHouseDateTime64 carrier in the TCP project. This leaves the workaround promised in the PR description unavailable; add an unambiguous carrier/typed value mechanism or support recursive coercion.
        (string typeName, object canonical) = Infer(element);
        if (canonical.GetType() != element.GetType())
        {
            throw new NotSupportedException(
                $"A Dynamic composite element of CLR type '{element.GetType()}' would be coerced to '{canonical.GetType()}', which is not supported inside a composite. Use the canonical element type directly (e.g. a raw long count for a DateTime64, ClickHouseDecimal for a decimal).");

ClickHouse.Driver.Tcp/Types/Codecs/DynamicColumnCodec.cs:93

  • This validation accepts repeated arguments and values outside ClickHouse's valid range, e.g. Dynamic(max_types=1, max_types=2), Dynamic(max_types=-1), and Dynamic(max_types=255). ClickHouse accepts at most one unsigned max_types value in the range 0–254, so these codecs pass client validation only to fail at the server. Enforce both arity and range here.
        foreach (TypeNode argument in node.Arguments)
        {
            if (!TryParseMaxTypes(argument.Name, out _))
            {
                throw new FormatException(
                    $"Dynamic type '{node}' has unsupported argument '{argument.Name}'; only 'max_types=N' is recognized.");
            }
        }

Comment thread ClickHouse.Driver.Tcp/Types/Codecs/DynamicTypeInference.cs
Comment thread ClickHouse.Driver.Tcp/Types/Codecs/DynamicTypeInference.cs
alex-clickhouse and others added 13 commits August 16, 2026 19:48
Adds the ClickHouse `Dynamic` column type — a column whose per-row value
type is discovered at runtime — over the native protocol, reading and
writing only the FLATTENED serialization (wire version 3), gated on
`output_format_native_use_flattened_dynamic_and_json_serialization = 1`.
The leading version word is validated and any non-flat encoding rejected.

- Dense `DynamicColumn` (runtime type-name list + widened discriminators +
  per-type child columns) as the zero-copy read/write source; an ergonomic
  `IColumn<object>` write source infers each value's ClickHouse type via a
  self-contained `DynamicTypeInference` (scalars, IPv4/IPv6 by address
  family, date-times, decimals, and Array/Map/Tuple recursion), normalizing
  a value to its codec's element type so it round-trips.
- Introduces the per-operation write-state contract (`IColumnCodec.BeginWrite`
  + `IColumnWriteState` + state-aware `WriteStatePrefix`/`WriteColumn`
  overloads that default to the state-free ones, so leaf codecs are
  untouched). The block writer runs BeginWrite -> prefix -> body -> Dispose.
  This lets a data-dependent prefix (the Dynamic type list) and the
  element-flattening composites do their flatten/scatter once across both
  phases.
- Full nesting both directions: the Array/Tuple/Map/Nested codecs now project
  their flattened inner sub-column into the child's prefix and body, so
  `Array(Dynamic)`, `Tuple(...Dynamic...)`, `Map(K, Dynamic)`,
  `Nested(...Dynamic...)` and composite values inside a Dynamic all
  round-trip. Variant rejects a Dynamic alternative (server-disallowed).
- Like Variant, there is no `Nullable(Dynamic)` — NULL rides the
  discriminator (value `num_types`, not 255).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Dynamic infers its alternatives from the data at write time, so it cannot densify
up front (its Densify stays the identity) — but the per-type columns it builds
from the boxed values are the ergonomic form, and the resolved codecs now write
and measure only the dense wire shape. So both data-dependent paths densify each
per-type column before delegating: BuildState densifies each bucket before the
child's BeginWrite (storing the dense column for the body phase), and the
per-row measure densifies its one-row probe before pricing it. This makes a
Dynamic value that is itself an Array/Tuple/Nullable round-trip end-to-end.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Dynamic discovers its per-type children from the ergonomic source and
writes each straight through its codec, with no densify pre-pass or byte
measurement.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PR 460 feedback:
- Reject a heterogeneous Dynamic array (e.g. IPv4 mixed with IPv6 in an
  IPAddress[]) with a clear NotSupportedException at inference time,
  rather than letting a later element fail the bucket cast mid-write.
- Infer an empty nested composite (an empty Array(Array(T)) or
  Array(Map(...))) structurally from its CLR element type instead of
  throwing, so empty nested arrays are usable inside Dynamic.
- Null-check the BuildFlatColumn reflection lookup in FlatBuilderFor,
  throwing an actionable InvalidOperationException like VariantColumnCodec
  rather than a NullReferenceException.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ReadColumn_DocumentedBytes_ReconstructsValuesAndNull asserted only
RowCount + GetValue on the golden vector, which the "Dynamic [scalars + null]"
case in InsertRoundTripCase reproduces against a real server with the same
{String, UInt64} type list and NULL discriminator.

Rather than delete it, folded it into
ReadColumn_DocumentedBytes_SurfacesTheRuntimeTypeList -- same setup, and
keeping the value assertions next to the type list and discriminators guards
the decoder against drifting even if a future server emits a different type
ordering. Net one test, no coverage lost.

Added the Dynamic(max_types=2) case. That is the one shape where the server
reshapes the dynamic structure -- types past N go to the shared/overflow
bucket -- and Create_MaxTypesArgument_IsAccepted only checks the TypeName
string, so the client's read of the reshaped structure had no coverage at all.

Verified against ClickHouse 26.6 -- 1009 tests pass, integration included.

Co-Authored-By: Claude <noreply@anthropic.com>
Same defect as Variant, which Dynamic's indexer mirrors: it read discriminators[row] out of a
buffer that is normally a pooled array longer than the column, so an out-of-range row's
behavior depended on the leftover value — a stale NULL discriminator reported the row as an
existing NULL, anything else fell through to the exactly-sized local index and threw. Indexing
through the RowCount-sliced Discriminators span makes both a bounds failure, and the
constructor now validates the buffer rather than letting the local-index walk fault on a short
one.

Co-Authored-By: Claude <noreply@anthropic.com>
IDynamicColumn already described the wire layout, so this promotes it and
documents the two things that distinguish Dynamic from Variant for a consumer
reading it columnar: the runtime type list is discovered per block rather than
declared, so TypeNames is how a caller knows which typed column to read a child
as; and because the list is discovered, NULL is encoded as TypeCount — one past
the last type — instead of Variant's fixed 255.

Like Variant, Dynamic has no useful materialized element type, so its
IColumn<T> surface is IColumn<object> and the columnar view is the typed way in.
Same regression as the variant branch: making IDynamicColumn public turned the
dense write path's trust boundary into a public extension point. The planner
trusts invariants only DynamicColumn's constructor establishes — the type-name
list matches the child-column count, and every discriminator is a valid type
index or the NULL marker — so gate on the concrete class, as every other
composite does. The interface stays public as a read view.

Two consequences of the same exposure, fixed alongside:

TypeNames returned the backing string[] as IReadOnlyList<string>, which is
castable back and mutable. A caller could rewrite a decoded column's type list,
after which a re-insert resolves a codec from the rewritten name while the child
column still holds the original values. Hand out a read-only wrapper instead.

BuildDenseState rented its discriminator slice before the per-type walk that
indexes by discriminator, with nothing between the rent and the try that would
return it. Reordered so the rent comes after the fallible indexing. Unreachable
again now the gate is concrete, but it cost nothing to stop relying on that.
Same two additions as the variant branch, plus a note that TypeNames is read-only
and does not hand out its backing storage.
Last of the six composites.

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Drop the null-state fallback from the Dynamic codec, as the write-state
contract now takes the state as required in the state-aware overloads.

Cover the dense slice plan, which had no test at all. The plan derives each
type's child-column run start from the local index of that type's first
in-slice row, and at start 0 every one of those is 0 - so a slice starting at
row 0 cannot tell a correct plan from one that ignores the offset. Both new
tests slice past row 0:

- a codec test asserting the exact bytes of rows [3, 5) of a five-row column
  whose two types each hold two values, so each run is written from offset 1.
  Replacing the run start with 0 fails this test and nothing else.
- an integration case inserting the dense column as blocks of two rows, the
  Dynamic counterpart of the Variant case beside it.

Co-Authored-By: Claude <noreply@anthropic.com>
DynamicWire held three members that only DynamicColumnCodec used: the
flattened serialization version, the type-count ceiling, and the
discriminator-width rule. A separate file for them added a hop without
adding a boundary, so move them into the codec — the version and the
ceiling as private constants, the width rule as an internal static
method the tests still reach.

The width test moves with it, from DynamicTypeInferenceTests to
DynamicColumnCodecTests, where it now belongs.

Co-Authored-By: Claude <noreply@anthropic.com>
Comment thread ClickHouse.Driver.Tcp/Types/Codecs/DynamicTypeInference.cs
Inference for a Map inside a Dynamic used only the CLR generic arguments
of KeyValuePair<K, V>. That helper knows the scalar table and arrays, so
a type that needs a value to resolve — an IPAddress (address family) or a
ClickHouseDecimal (scale) — threw NotSupportedException as a map key or
value, although the same type works as an array element or a tuple
element.

Scan the present pairs instead, as the array branch already does, and
keep the CLR types as the fallback for an empty map. The two branches now
share one Agree helper, which also gives the map the mixed-type rejection
the array had: without it, a map of mixed IPv4 and IPv6 keys would trade
a clear error for an InvalidCastException in the bucket projection.

The pair scan runs through a cached per-pair-type delegate, the pattern
DynamicColumnCodec already uses, so it does not box each pair or reflect
over Key/Value once per entry.

Also correct a comment that named ClickHouseDateTime64, a type that does
not exist; the coerced value is the raw Int64 nanosecond count.

Co-Authored-By: Claude <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 55416b7. Configure here.

{
Type[] pair = elementType.GetGenericArguments();
(string key, string value) = PairScannerFor(elementType)(array);
return $"Map({key ?? InferFromClrType(pair[0])}, {value ?? InferFromClrType(pair[1])})";

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Boxed map write type mismatch

Medium Severity

Map inference now derives key and value ClickHouse types from present pair values, so a KeyValuePair&lt;string, object&gt;[] can resolve to something like Map(String, IPv4). Infer still returns that original pair array, whose CLR type does not match MapColumnCodec.ElementType (KeyValuePair&lt;string, IPAddress&gt;[]). The Dynamic write path then fails with an InvalidCastException in BuildFlatColumn when projecting the bucket, instead of a clear inference rejection. Strongly typed pair arrays still work; the boxed-value path newly succeeds at inference and breaks on write.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 55416b7. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants