feat(smart-gen): bound baked meta, Cube-native descriptions, idempotent reconciles#64
Merged
Merged
Conversation
…nt reconciles Generated models were ~2/3 per-field meta: volatile value snapshots (ranges, 60-value enum lists, exact cardinalities) that rot as data arrives, leak identifier values (PII) into schema files, and defeat the reconciler's no-change guard so every reconcile minted a version. The model carries shape + semantics; the data plane answers value questions (live queries, /meta/dynamic). Changes: - drop write-only keys: range, raw_type, max_array_length, known_keys (superseded by /meta/dynamic), cube refresh_cadence - unique_values: exact <= 12, else 2 significant figures (consumers only threshold at 7/12/50/500/2000) - lc_values / known_values: baked only as a COMPLETE (<= 12) set that is not identifier-shaped (UUID/email/opaque-token guard) - descriptions move to the Cube-native description property (member + cube); expanded map keys no longer repeat the parent column's text - titles baked only where the acronym-aware form differs from Cube's derived default (Entity GID vs Entity Gid) - merger/templateMerger shed legacy generator keys instead of preserving them forever as presumed user keys Verified live (dev stack, doc-anchor datasource): per-field meta 9.0 -> 3.8 lines; applied model queryable (store_format counts match doc anchors); reconcile-team run 2 = all skipped_no_change; cxs2 079 e2e US1 5/5 + US2 6/6 against this backend. Unit suite 620/622 (2 failures pre-exist on main, partition.test.js). Known follow-up: probe key inventory can flap on rare map keys (profiler sampling) - now the only churn source. Scope + consumer map: docs/plans/2026-07-07-smart-gen-meta-trim-scope.md Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Generated models were ~⅔ per-field
metaby volume — redundant type/provenance echoes plus volatile value snapshots (ranges, 60-value enum lists, exact cardinalities) that rot as data arrives, bake identifier values (PII) into schema files, and defeat the reconciler's no-change guard so nearly every reconcile minted a no-op version.Principle: the model carries shape + semantics; the data plane answers value questions (live queries,
/meta/dynamic).Changes (all in
utils/smart-generation/)range,raw_type,max_array_length,known_keys(superseded by 014/meta/dynamic), cube-levelrefresh_cadenceunique_values: exact ≤ 12, else rounded to 2 significant figures — consumers only threshold (7 pie / 12 facet / 50 searchable / 500 list / 2000 lazy), exact large counts were pure churnlc_values/known_values: baked only as a complete (≤ 12 distinct) set that is not identifier-shaped (UUID/email/opaque-token guard — fixes user UUIDs baked intouser_id.meta.lc_values)descriptionproperties now (member + cube;meta.descriptiongone) — surfaces in standard Cube tooling; expanded map keys no longer repeat the parent column's description dozens of timesEntity GIDkept,Store Formatdropped)refresh_cadence, cubemeta.description) instead of preserving them forever as presumed user keysFull key-by-key consumer map + dispositions:
docs/plans/2026-07-07-smart-gen-meta-trim-scope.md(in this PR).Verification (live dev stack, real ClickHouse)
partition.test.js) pre-exist on main (verified in a clean worktree)event='Sales Report Submitted')account_id);store_formatkeeps its complete 8-value enum; 158,604 →unique_values: 160000store_format× count matches doc anchors exactly (Discount Supermarket = 82,180)/internal/reconcile-team×3, somi team)updated), run 2 allskipped_no_changecxs2 needs no code change (all meta reads are null-guarded with thresholds preserved); its editor meta-conventions + the semantic-layer skill docs were synced separately.
Known follow-up (pre-existing, out of scope)
Probe key inventory can flap on rare map keys (profiler deep-pass sampling —
holiday_nameappeared in one reconcile probe, not the next). With meta now byte-stable, this is the only remaining reconcile churn source. Candidate fix: occurrence-thresholded or unsampled key discovery.🤖 Generated with Claude Code