refactor(rust): give float columns their own raw model by CommanderStorm · Pull Request #1599 · maplibre/maplibre-tile-spec

CommanderStorm · 2026-09-01T08:37:45Z

Float columns move off RawScalar, which holds exactly one stream, onto a per-encoding enum carrying each variant's stream set, so a dictionary-encoded column can hold two. No wire changes

github-actions · 2026-09-01T08:57:23Z

Performance Comparison

Performance Comparison `main` → `stack-v2-float-raw-model`

Total Elapsed Time: 2.16s → 2.13s (-1.4%)
CPU Baseline: 89.58µs → 90.22µs (+0.7%)
Benchmark ID: mlt-convert

timing - Function execution time metrics.

+--------------------------------------+------------------------------+--------------------------------+--------------------------------+---------------------------------+------------------------------+
| Function                             | Calls                        | Avg                            | P95                            | Total                           | % Total                      |
+--------------------------------------+------------------------------+--------------------------------+--------------------------------+---------------------------------+------------------------------+
| optimizer::encode                    | 14096 → 14096 (+0.0%)        | 523.87µs → 511.71µs (-2.3%)    | 2.29ms → 2.23ms (-2.6%)        | 7.38s → 7.21s (-2.3%)           | 342.94% → 339.49% (-1.0%)    |
+--------------------------------------+------------------------------+--------------------------------+--------------------------------+---------------------------------+------------------------------+
| optimizer::encode_into               | 28192 → 28192 (+0.0%)        | 158.52µs → 157.35µs (-0.7%)    | 667.13µs → 666.11µs (-0.2%)    | 4.47s → 4.44s (-0.7%)           | 207.55% → 208.78% (+0.6%)    |
+--------------------------------------+------------------------------+--------------------------------+--------------------------------+---------------------------------+------------------------------+
| encode01::write_to                   | 28192 → 28192 (+0.0%)        | 123.77µs → 122.99µs (-0.6%)    | 587.77µs → 583.68µs (-0.7%)    | 3.49s → 3.47s (-0.6%)           | 162.05% → 163.19% (+0.7%)    |
+--------------------------------------+------------------------------+--------------------------------+--------------------------------+---------------------------------+------------------------------+
| writer::with                         | 1573034 → 1573034 (+0.0%)    | 1.38µs → 1.38µs (+0.0%)        | 2.15µs → 2.15µs (+0.0%)        | 2.18s → 2.17s (-0.5%)           | 101.13% → 102.01% (+0.9%)    |
+--------------------------------------+------------------------------+--------------------------------+--------------------------------+---------------------------------+------------------------------+
| mlt::main                            | 1 → 1 (+0.0%)                | 2.15s → 2.12s (-1.4%)          | 2.15s → 2.13s (-0.9%)          | 2.15s → 2.12s (-1.4%)           | 100.00% → 100.00% (+0.0%)    |
+--------------------------------------+------------------------------+--------------------------------+--------------------------------+---------------------------------+------------------------------+
| optimizer::analyze                   | 14096 → 14096 (+0.0%)        | 136.29µs → 129.28µs (-5.1%)    | 698.88µs → 654.85µs (-6.3%)    | 1.92s → 1.82s (-5.2%)           | 89.22% → 85.77% (-3.9%)      |
+--------------------------------------+------------------------------+--------------------------------+--------------------------------+---------------------------------+------------------------------+
| shared_dict::group_string_properties | 14096 → 14096 (+0.0%)        | 131.66µs → 124.33µs (-5.6%)    | 671.74µs → 629.76µs (-6.2%)    | 1.86s → 1.75s (-5.9%)           | 86.19% → 82.49% (-4.3%)      |
+--------------------------------------+------------------------------+--------------------------------+--------------------------------+---------------------------------+------------------------------+
| streams::dict_may_be_beneficial      | 28192 → 28192 (+0.0%)        | 52.09µs → 51.59µs (-1.0%)      | 230.66µs → 229.89µs (-0.3%)    | 1.47s → 1.45s (-1.4%)           | 68.20% → 68.46% (+0.4%)      |
+--------------------------------------+------------------------------+--------------------------------+--------------------------------+---------------------------------+------------------------------+
| encode::write_prop                   | 116572 → 116572 (+0.0%)      | 7.75µs → 7.68µs (-0.9%)        | 17.33µs → 17.41µs (+0.5%)      | 903.23ms → 895.17ms (-0.9%)     | 41.95% → 42.13% (+0.4%)      |
+--------------------------------------+------------------------------+--------------------------------+--------------------------------+---------------------------------+------------------------------+
| tile::from_tile                      | 28192 → 28192 (+0.0%)        | 20.51µs → 19.74µs (-3.8%)      | 79.94µs → 79.10µs (-1.1%)      | 578.16ms → 556.52ms (-3.7%)     | 26.85% → 26.19% (-2.5%)      |
+--------------------------------------+------------------------------+--------------------------------+--------------------------------+---------------------------------+------------------------------+
| shared_dict::write_shared_dict       | 24002 → 24002 (+0.0%)        | 22.11µs → 21.88µs (-1.0%)      | 51.39µs → 51.49µs (+0.2%)      | 530.73ms → 525.24ms (-1.0%)     | 24.65% → 24.72% (+0.3%)      |
+--------------------------------------+------------------------------+--------------------------------+--------------------------------+---------------------------------+------------------------------+
| strings::write_str_col               | 57070 → 57070 (+0.0%)        | 5.35µs → 5.28µs (-1.3%)        | 12.38µs → 12.40µs (+0.2%)      | 305.53ms → 301.37ms (-1.4%)     | 14.19% → 14.18% (-0.1%)      |
+--------------------------------------+------------------------------+--------------------------------+--------------------------------+---------------------------------+------------------------------+
| sort::curve_params                   | 14096 → 14096 (+0.0%)        | 17.22µs → 15.42µs (-10.5%)     | 80.51µs → 72.38µs (-10.1%)     | 242.75ms → 217.35ms (-10.5%)    | 11.27% → 10.23% (-9.2%)      |
+--------------------------------------+------------------------------+--------------------------------+--------------------------------+---------------------------------+------------------------------+
| strings::fsst_try_train              | 40536 → 40536 (+0.0%)        | 3.56µs → 3.44µs (-3.4%)        | 80.00ns → 80.00ns (+0.0%)      | 144.48ms → 139.65ms (-3.3%)     | 6.71% → 6.57% (-2.1%)        |
+--------------------------------------+------------------------------+--------------------------------+--------------------------------+---------------------------------+------------------------------+
| streams::build_hilbert_dict          | 362 → 362 (+0.0%)            | 223.90µs → 226.02µs (+0.9%)    | 429.06µs → 418.05µs (-2.6%)    | 81.05ms → 81.82ms (+1.0%)       | 3.76% → 3.85% (+2.4%)        |
+--------------------------------------+------------------------------+--------------------------------+--------------------------------+---------------------------------+------------------------------+

alloc-bytes - Exclusive allocation bytes by each function (excluding nested calls).

+--------------------------------------+------------------------------+------------------------------+--------------------------------+--------------------------------+----------------------------+
| Function                             | Calls                        | Avg                          | P95                            | Total                          | % Total                    |
+--------------------------------------+------------------------------+------------------------------+--------------------------------+--------------------------------+----------------------------+
| tile::from_tile                      | 28192 → 28192 (+0.0%)        | 44.1 KB → 44.1 KB (+0.0%)    | 238.5 KB → 238.5 KB (+0.0%)    | 1.2 GB → 1.2 GB (+0.0%)        | 23.23% → 23.23% (+0.0%)    |
+--------------------------------------+------------------------------+------------------------------+--------------------------------+--------------------------------+----------------------------+
| optimizer::encode                    | 14096 → 14096 (+0.0%)        | 86.2 KB → 86.2 KB (+0.0%)    | 159.2 KB → 159.2 KB (+0.0%)    | 1.2 GB → 1.2 GB (+0.0%)        | 22.69% → 22.69% (+0.0%)    |
+--------------------------------------+------------------------------+------------------------------+--------------------------------+--------------------------------+----------------------------+
| writer::with                         | 1573034 → 1573034 (+0.0%)    | 723 B → 723 B (+0.0%)        | 25.0 KB → 25.0 KB (+0.0%)      | 1.1 GB → 1.1 GB (+0.0%)        | 20.74% → 20.74% (+0.0%)    |
+--------------------------------------+------------------------------+------------------------------+--------------------------------+--------------------------------+----------------------------+
| strings::fsst_try_train              | 40536 → 40536 (+0.0%)        | 16.8 KB → 16.8 KB (+0.0%)    | 3.6 MB → 3.6 MB (+0.0%)        | 664.5 MB → 664.5 MB (+0.0%)    | 12.70% → 12.70% (+0.0%)    |
+--------------------------------------+------------------------------+------------------------------+--------------------------------+--------------------------------+----------------------------+
| shared_dict::group_string_properties | 14096 → 14096 (+0.0%)        | 30.3 KB → 30.3 KB (+0.0%)    | 171.6 KB → 171.6 KB (+0.0%)    | 416.9 MB → 416.9 MB (+0.0%)    | 7.97% → 7.97% (+0.0%)      |
+--------------------------------------+------------------------------+------------------------------+--------------------------------+--------------------------------+----------------------------+
| strings::write_str_col               | 57070 → 57070 (+0.0%)        | 4.1 KB → 4.1 KB (+0.0%)      | 11.0 KB → 11.0 KB (+0.0%)      | 227.7 MB → 227.7 MB (+0.0%)    | 4.35% → 4.35% (+0.0%)      |
+--------------------------------------+------------------------------+------------------------------+--------------------------------+--------------------------------+----------------------------+
| encode01::write_to                   | 28192 → 28192 (+0.0%)        | 7.5 KB → 7.5 KB (+0.0%)      | 35.7 KB → 35.7 KB (+0.0%)      | 206.0 MB → 206.0 MB (+0.0%)    | 3.94% → 3.94% (+0.0%)      |
+--------------------------------------+------------------------------+------------------------------+--------------------------------+--------------------------------+----------------------------+
| shared_dict::write_shared_dict       | 24002 → 24002 (+0.0%)        | 6.6 KB → 6.6 KB (+0.0%)      | 37.4 KB → 37.4 KB (+0.0%)      | 154.9 MB → 154.9 MB (+0.0%)    | 2.96% → 2.96% (+0.0%)      |
+--------------------------------------+------------------------------+------------------------------+--------------------------------+--------------------------------+----------------------------+
| strings::write_raw_str_data          | 137744 → 137744 (+0.0%)      | 122 B → 122 B (+0.0%)        | 9.0 KB → 9.0 KB (+0.0%)        | 16.1 MB → 16.1 MB (+0.0%)      | 0.31% → 0.31% (+0.0%)      |
+--------------------------------------+------------------------------+------------------------------+--------------------------------+--------------------------------+----------------------------+
| streams::build_morton_dict           | 362 → 362 (+0.0%)            | 44.6 KB → 44.6 KB (+0.0%)    | 89.9 KB → 89.9 KB (+0.0%)      | 15.8 MB → 15.8 MB (+0.0%)      | 0.30% → 0.30% (+0.0%)      |
+--------------------------------------+------------------------------+------------------------------+--------------------------------+--------------------------------+----------------------------+
| optimizer::analyze                   | 14096 → 14096 (+0.0%)        | 1.0 KB → 1.0 KB (+0.0%)      | 6.5 KB → 6.5 KB (+0.0%)        | 14.0 MB → 14.0 MB (+0.0%)      | 0.27% → 0.27% (+0.0%)      |
+--------------------------------------+------------------------------+------------------------------+--------------------------------+--------------------------------+----------------------------+
| streams::build_hilbert_dict          | 362 → 362 (+0.0%)            | 25.9 KB → 25.9 KB (+0.0%)    | 90.7 KB → 90.7 KB (+0.0%)      | 9.2 MB → 9.2 MB (+0.0%)        | 0.18% → 0.18% (+0.0%)      |
+--------------------------------------+------------------------------+------------------------------+--------------------------------+--------------------------------+----------------------------+
| streams::dict_may_be_beneficial      | 28192 → 28192 (+0.0%)        | 256 B → 256 B (+0.0%)        | 256 B → 256 B (+0.0%)          | 6.9 MB → 6.9 MB (+0.0%)        | 0.13% → 0.13% (+0.0%)      |
+--------------------------------------+------------------------------+------------------------------+--------------------------------+--------------------------------+----------------------------+
| strings::write_str_plain             | 57070 → 57070 (+0.0%)        | 69 B → 69 B (+0.0%)          | 240 B → 240 B (+0.0%)          | 3.8 MB → 3.8 MB (+0.0%)        | 0.07% → 0.07% (+0.0%)      |
+--------------------------------------+------------------------------+------------------------------+--------------------------------+--------------------------------+----------------------------+
| encode::write_prop                   | 116572 → 116572 (+0.0%)      | 27 B → 27 B (+0.0%)          | 256 B → 256 B (+0.0%)          | 3.1 MB → 3.1 MB (+0.0%)        | 0.06% → 0.06% (+0.0%)      |
+--------------------------------------+------------------------------+------------------------------+--------------------------------+--------------------------------+----------------------------+

Threads

No threads to compare

Generated with hotpath-rs

Per-tile encoded size diff

zoom	tiles	prev_total	curr_total
0	1	71471	71471
1	4	220986	220986
2	16	615003	615003
3	64	1653851	1653851
4	256	3828904	3828904
5	1024	9716821	9716821
6	4096	32401914	32401914
ALL	5461	48508950	48508950

Top 3 improvements

Top 3 degradations

codecov · 2026-09-01T12:26:44Z

Codecov Report

❌ Patch coverage is 97.05882% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 70.06%. Comparing base (cde4843) to head (5a59982).

Files with missing lines	Patch %	Lines
rust/mlt-core/src/decoder/iterators.rs	66.66%	1 Missing ⚠️

Additional details and impacted files

@@                       Coverage Diff                        @@
##             stack-v2-encoding-families    #1599      +/-   ##
================================================================
- Coverage                         73.40%   70.06%   -3.34%     
================================================================
  Files                               338      108     -230     
  Lines                             35333    18953   -16380     
  Branches                           3649        0    -3649     
================================================================
- Hits                              25935    13279   -12656     
+ Misses                             8205     5674    -2531     
+ Partials                           1193        0    -1193

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copilot

Pull request overview

This PR refactors the Rust decoder’s raw property model so float columns (F32/F64) no longer use RawScalar and instead use a dedicated RawFloats type with an encoding enum, aligning floats with the “encoding carries stream set” pattern used by strings.

Changes:

Introduce RawFloats / RawFloatsEncoding and switch RawProperty::{F32,F64} to use it.
Update v01/v02 layer parsing to construct float properties via RawFloats::single(...).
Update decoding + analysis + iterator plumbing to handle the new float raw model.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
rust/mlt-core/src/decoder/root02.rs	Switch v02 float property parsing from `RawScalar::new` to `RawFloats::single`.
rust/mlt-core/src/decoder/root01.rs	Switch v01 float property parsing from `RawScalar::new` to `RawFloats::single`.
rust/mlt-core/src/decoder/property/model.rs	Add `RawFloats` and `RawFloatsEncoding`; change `RawProperty` float variants to use the new type.
rust/mlt-core/src/decoder/property/decode.rs	Add float decoding helper on `RawFloats` and route `RawProperty::{F32,F64}` decode through it.
rust/mlt-core/src/decoder/mod.rs	Re-export the new float raw model types from the decoder module.
rust/mlt-core/src/decoder/iterators.rs	Update property-name iteration to accommodate float variants no longer being `RawScalar`.
rust/mlt-core/src/decoder/analyze.rs	Add `Analyze` impls for float raw model and update `RawProperty` stream enumeration accordingly.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

`RawScalar` holds exactly one `RawStream`, which is all a float column has needed so far, but a dictionary-encoded one needs two. Floats therefore move to a per-encoding enum carrying each variant's stream set, the shape string columns already use. No wire change, and no public API break: neither `RawProperty` nor `RawScalar` is exported, and both are reachable only through the `__private` test and bench feature. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

CommanderStorm force-pushed the stack-v2-float-raw-model branch from 2c6e23b to a97f6d2 Compare September 1, 2026 10:24

CommanderStorm force-pushed the stack-v2-float-raw-model branch from a97f6d2 to 7b5e8e8 Compare September 1, 2026 12:04

CommanderStorm marked this pull request as ready for review September 1, 2026 13:17

Copilot AI lite review requested due to automatic review settings September 1, 2026 13:17

Copilot started reviewing on behalf of CommanderStorm September 1, 2026 13:18 View session

Copilot AI reviewed Sep 1, 2026

View reviewed changes

Comment thread rust/mlt-core/src/decoder/property/model.rs

CommanderStorm force-pushed the stack-v2-float-raw-model branch from 7b5e8e8 to 17f1b57 Compare September 1, 2026 14:09

CommanderStorm force-pushed the stack-v2-float-raw-model branch from 17f1b57 to a666f36 Compare September 2, 2026 13:20

CommanderStorm force-pushed the stack-v2-float-raw-model branch from a666f36 to a20b293 Compare September 3, 2026 14:28

CommanderStorm force-pushed the stack-v2-float-raw-model branch from a20b293 to 450ada5 Compare September 3, 2026 15:16

CommanderStorm force-pushed the stack-v2-float-raw-model branch from 450ada5 to 65829db Compare September 3, 2026 17:42

CommanderStorm force-pushed the stack-v2-float-raw-model branch from 65829db to 534abef Compare September 3, 2026 17:53

CommanderStorm force-pushed the stack-v2-float-raw-model branch from 534abef to 391d6de Compare September 4, 2026 08:53

CommanderStorm force-pushed the stack-v2-float-raw-model branch from 391d6de to 5a59982 Compare September 4, 2026 09:20

CommanderStorm force-pushed the stack-v2-float-raw-model branch 2 times, most recently from c7a828d to fa9c4d8 Compare September 4, 2026 12:25

CommanderStorm force-pushed the stack-v2-float-raw-model branch from fa9c4d8 to 0f181d5 Compare September 4, 2026 13:43

CommanderStorm force-pushed the stack-v2-float-raw-model branch from 0f181d5 to 384f7e0 Compare September 4, 2026 14:03

CommanderStorm force-pushed the stack-v2-float-raw-model branch from 384f7e0 to ce1c8b7 Compare September 4, 2026 14:15

CommanderStorm force-pushed the stack-v2-float-raw-model branch from ce1c8b7 to 248820e Compare September 4, 2026 23:59

CommanderStorm force-pushed the stack-v2-float-raw-model branch 2 times, most recently from 2d80114 to 4a6b740 Compare September 5, 2026 13:42

CommanderStorm force-pushed the stack-v2-float-raw-model branch from 4a6b740 to 299e1e7 Compare September 7, 2026 11:06

Navigation Menu

Uh oh!

refactor(rust): give float columns their own raw model - #1599

refactor(rust): give float columns their own raw model#1599
CommanderStorm wants to merge 1 commit into
stack-v2-encoding-familiesfrom
stack-v2-float-raw-model

CommanderStorm commented Sep 1, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 1, 2026 •

edited

Loading

Performance Comparison `main` → `stack-v2-float-raw-model`

timing - Function execution time metrics.

alloc-bytes - Exclusive allocation bytes by each function (excluding nested calls).

Threads

Top 3 improvements

Top 3 degradations

Uh oh!

codecov Bot commented Sep 1, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

CommanderStorm commented Sep 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Performance Comparison main → stack-v2-float-raw-model

timing - Function execution time metrics.

alloc-bytes - Exclusive allocation bytes by each function (excluding nested calls).

Threads

Top 3 improvements

Top 3 degradations

Uh oh!

codecov Bot commented Sep 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

CommanderStorm commented Sep 1, 2026 •

edited

Loading

github-actions Bot commented Sep 1, 2026 •

edited

Loading

Performance Comparison `main` → `stack-v2-float-raw-model`

codecov Bot commented Sep 1, 2026 •

edited

Loading