Skip to content

feat(api-gateway): granularities config and /v1/granularities endpoint - #10918

Open
igorlukanin wants to merge 22 commits into
masterfrom
igor/cub-2567-granularities-update-cube-core-tesseract-cube-runtime-sdk
Open

feat(api-gateway): granularities config and /v1/granularities endpoint#10918
igorlukanin wants to merge 22 commits into
masterfrom
igor/cub-2567-granularities-update-cube-core-tesseract-cube-runtime-sdk

Conversation

@igorlukanin

@igorlukanin igorlukanin commented May 20, 2026

Copy link
Copy Markdown
Member

Summary

Engine-side foundation for configurable, deployment-wide granularities:

  • New granularities shape on time dimensions. Accepts a dict { includes?, excludes?, custom? } alongside the legacy flat array. includes/excludes (string list or '*') select from the enabled set; custom defines per-dimension granularities. Mutually exclusive except for includes: '*' + excludes: [...].
  • Deployment-wide granularities config. Configurable via CUBEJS_GRANULARITIES (comma-separated list), per-custom env vars CUBEJS_GRANULARITIES_<NAME>_INTERVAL|TITLE|OFFSET|ORIGIN, or config.granularities (list or context-dependent function). File config fully replaces env vars.
  • GET /v1/granularities. New read-only endpoint returning the resolved set (built-ins + customs) for the request's security context.
  • /v1/meta enrichment. Each time-dimension granularity now carries type ('built-in' / 'custom'), title, format (d3-time-format), and interval. Per-dimension includes/excludes are resolved against the deployment config so the response reflects the effective set.
  • /v1/load + /v1/cubesql annotations. Same type/format enrichment flows through prepareAnnotation to query responses.
  • Client SDK types widened. Granularity / GranularityAnnotation and the Rust transport struct in @cubejs-backend/native now carry type and format.

Out of scope (separate follow-ups):

  • allow_no_granularity parameter end-to-end
  • XMLA / DAX exposure of custom granularities
  • Elasticsearch + MS Fabric driver SQL for custom granularities
  • AI Engineer prompts
  • Documentation, telemetry, release notes

Test plan

  • Added unit tests covering every row of the new resolution table (granularities-shape.test.ts) and global config precedence (granularities-config.test.ts).
  • Extended cube-validator.test.ts for the new dict shape: includes/excludes mutual-exclusion, built-in shadowing, legacy form acceptance.
  • Extended prepare-annotation.test.ts for the new type / format fields on the granularity annotation.
  • Existing pre-agg matching tests still pass (no hash-affecting changes).
  • Manual end-to-end against a Postgres scratch model:
    • /v1/granularities returns 8 built-ins + customs sourced from CUBEJS_GRANULARITIES.
    • /v1/meta for a time dim with granularities: { includes: ['year', 'quarter'], custom: [...] } returns exactly those built-ins plus the local custom; a dim with the legacy flat array gets all enabled built-ins plus its locals.
    • /v1/load with granularity: 'week' returns { type: 'built-in', title: 'Week', interval: '1 week', format: '%b %-d, %Y' }.
  • CI: lint + tsc + unit + integration.

@igorlukanin
igorlukanin requested review from a team as code owners May 20, 2026 14:36
@github-actions github-actions Bot added client:core Issues relating to the JavaScript client SDK rust Pull requests that update Rust code javascript Pull requests that update Javascript code labels May 20, 2026
Comment thread packages/cubejs-api-gateway/src/gateway.ts Dismissed
@igorlukanin
igorlukanin force-pushed the igor/cub-2567-granularities-update-cube-core-tesseract-cube-runtime-sdk branch 3 times, most recently from 02bf1d7 to b7ac787 Compare May 21, 2026 15:24
@igorlukanin
igorlukanin force-pushed the igor/cub-2567-granularities-update-cube-core-tesseract-cube-runtime-sdk branch from b7ac787 to 898dec8 Compare May 21, 2026 15:43
@igorlukanin igorlukanin changed the title feat(api-gateway): global granularities catalog, dict-shape data model, /v1/granularities endpoint feat(api-gateway): granularities config and /v1/granularities endpoint May 21, 2026
@igorlukanin
igorlukanin force-pushed the igor/cub-2567-granularities-update-cube-core-tesseract-cube-runtime-sdk branch from 898dec8 to 19ef48a Compare May 21, 2026 16:06
@codecov

codecov Bot commented May 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 83.43195% with 56 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.58%. Comparing base (6088edf) to head (da0d7cb).

Files with missing lines Patch % Lines
...cubejs-schema-compiler/src/compiler/CubeSymbols.ts 62.85% 12 Missing and 1 partial ⚠️
...compiler/src/compiler/GlobalGranularitiesConfig.ts 88.42% 11 Missing ⚠️
...ackages/cubejs-server-core/src/core/CompilerApi.ts 67.85% 9 Missing ⚠️
packages/cubejs-backend-shared/src/env.ts 0.00% 5 Missing ⚠️
...chema-compiler/src/compiler/GranularityResolver.ts 91.37% 3 Missing and 2 partials ⚠️
...ema-compiler/src/compiler/CubeToMetaTransformer.ts 83.33% 0 Missing and 4 partials ⚠️
...ubejs-schema-compiler/src/compiler/YamlCompiler.ts 0.00% 4 Missing ⚠️
packages/cubejs-api-gateway/src/gateway.ts 80.00% 2 Missing ⚠️
...bejs-api-gateway/src/helpers/prepare-annotation.ts 86.66% 0 Missing and 2 partials ⚠️
...js-schema-compiler/src/compiler/PrepareCompiler.ts 87.50% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #10918      +/-   ##
==========================================
- Coverage   84.06%   79.58%   -4.49%     
==========================================
  Files         257      486     +229     
  Lines       81582   100027   +18445     
  Branches        0     3755    +3755     
==========================================
+ Hits        68581    79605   +11024     
- Misses      13001    19863    +6862     
- Partials        0      559     +559     
Flag Coverage Δ
cube-backend 59.76% <83.43%> (?)
cubesql 84.06% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@igorlukanin
igorlukanin force-pushed the igor/cub-2567-granularities-update-cube-core-tesseract-cube-runtime-sdk branch from b438640 to d0745d7 Compare July 16, 2026 16:40
@claude

claude Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Claude finished @igorlukanin's task in 4m 28s —— View job


Code Review — 1 high, 2 medium

Review details, checklist, and findings
  • Checked existing review threads (only the CodeQL rate-limiting note; not re-raising)
  • Reviewed gateway.ts (new endpoint + resolver wiring)
  • Reviewed GlobalGranularitiesConfig.ts & GranularityResolver.ts
  • Reviewed CubeValidator.ts
  • Reviewed CubeSymbols.ts / CubeToMetaTransformer.ts / YamlCompiler.ts
  • Reviewed prepare-annotation.ts, env config, tests
  • Posted inline comments

Overall this is a clean, well-scoped change. The two-shape validation, the lazy per-dim resolver in the load path, and the "block is internal; strip before responding" pattern in /v1/meta are all sensible. Test coverage on the resolution table and validator is thorough. Below are the substantive concerns.

Findings

# Severity File Concern
1 High packages/cubejs-schema-compiler/src/compiler/GranularityResolver.ts:96-102 Overriding a built-in via config.granularities: [{ name: 'year', title: 'Anno' }] yields type: 'custom' in the resolved output. resolveFromList puts year into both enabledBuiltIns and customGranularities; here the custom loop runs after the built-in loop and unconditionally overwrites out.year with type: 'custom'. Since buildBuiltInsCatalog already folds the override into allBuiltInsCatalog, the custom loop should skip names that shadow a built-in. Missing test in granularities-shape.test.ts — worth adding one that asserts type: 'built-in' survives an override.
2 Medium packages/cubejs-api-gateway/src/gateway.ts:832-834 /v1/meta-extended leaks the internal granularitiesBlock (includes/excludes/custom) on every time dimension. transformDimension in helpers/transform-meta-extended.ts:58 spreads the dimension verbatim, and unlike /v1/meta there's no stripping step. Also causes shape divergence with /v1/meta.
3 Medium packages/cubejs-schema-compiler/src/compiler/utils.ts:10-12 IGNORE_CAMELIZE[2].custom = true is applied globally at depth 2 across measures / dimensions / preAggregations / cubes, not scoped to granularities. Works today (no other feature has a level-2 custom field), but it's a fragile foothold — a future nested custom: { ... } block anywhere else would silently inherit identifier preservation. Consider a path-scoped mechanism.

Minor / nits (not filed inline)

  • resolveEffectiveGranularitiesForDim (gateway.ts:2438) writes offset/origin: undefined explicitly into localCustom, then spread-merges over block.custom.<name>. Works because spread doesn't delete keys, but the "explicit undefineds override" surface is easy to trip on later — consider building localCustom with a helper that omits undefined fields.
  • CubeToMetaTransformer.ts:334-341 still doesn't emit format in the legacy dim.granularities array. Fine given the deprecation note and effectiveGranularities carrying it, but worth confirming that no legacy client currently reads granularities[*].format from that field.
  • CodeQL rate-limiting note on GET /v1/granularities (packages/cubejs-api-gateway/src/gateway.ts:483) already raised by github-advanced-security; endpoint uses the same userMiddlewares + assertApiScope('meta') gating as /v1/meta so the risk profile matches an existing endpoint.
  • GlobalGranularitiesConfig.ts:47 casts opts to any with a comment about getEnv typing. The underlying Parameters<> typing mismatch is worth fixing at the source rather than repeating the workaround for each new env var group, but that's out of scope for this PR.
• [Branch](https://github.com/cube-js/cube/tree/igor/cub-2567-granularities-update-cube-core-tesseract-cube-runtime-sdk)

Comment thread packages/cubejs-schema-compiler/src/compiler/GranularityResolver.ts
Comment thread packages/cubejs-schema-compiler/src/compiler/utils.ts Outdated
@claude

claude Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Claude finished @igorlukanin's task in 1m 59s —— View job


Re-review after 5d91cf7 / d0745d7 — 1 medium remaining

Details

Findings

Medium — /v1/meta-extended still leaks the internal granularitiesBlock. CubeToMetaTransformer.ts:345 emits granularitiesBlock on every time-dim config, and packages/cubejs-api-gateway/src/helpers/transform-meta-extended.ts:58-66 spreads ...dimension verbatim. There's no stripping step equivalent to the { granularitiesBlock, ...rest } = dim in applyGlobalGranularitiesToMetaCubes (gateway.ts:2535). Result: /v1/meta-extended responses include the raw { includes, excludes, custom } block on every time dimension, and the shape diverges from /v1/meta (which now carries effectiveGranularities and omits the block). Attempted an inline comment but the file isn't in the PR diff, so filing here.

Suggested fix in transform-meta-extended.ts:

function transformDimension(dimension: any, cubeDefinitions: any) {
  const { cubeName, memberName } = getMemberPath(dimension.name);
  const { granularitiesBlock, ...rest } = dimension;
  return {
    ...rest,
    sql: stringifyMemberSql(cubeDefinitions[cubeName]?.dimensions?.[memberName]?.sql),
    case: handleDimensionCaseCondition(cubeDefinitions[cubeName]?.dimensions?.[memberName]?.case),
  };
}

Separately (and out of scope for this PR if you'd rather punt it): /v1/meta-extended doesn't attach effectiveGranularities either. Not a leak, but the shape gap is worth confirming is intentional.

Notes on the new architecture (d0745d7)

  • The split — legacy granularities (deprecated, custom-only, no type/format) kept as-is, effectiveGranularities added alongside on /v1/meta, resolved on demand for /v1/load via buildGranularityResolver — is cleaner than the previous "rewrite the whole meta" approach and matches what DEPRECATION.md promises.
  • buildGranularityResolver correctly falls back to BUILT_IN_GRANULARITIES defaults for a queried built-in that global config disabled but that the planner still executes — nice defensive touch.
  • The isDictForm check in normalizeGranularitiesBlock (GranularityResolver.ts:47-54) — validating that includes/excludes values look like inclusion lists before treating the object as the dict form — correctly rescues a legacy custom granularity literally named includes/excludes/custom from being misparsed. Worth a targeted test if not already covered.
  • The resolveEffectiveGranularitiesForDim local-custom merge writes offset: undefined, origin: undefined explicitly, then relies on the caller's def.offset !== undefined ? ... : {} filter to drop them from the response. Works today, but a localCustom builder that omits undefined keys would be less fragile if that filter ever moves.
• [Branch](https://github.com/cube-js/cube/tree/igor/cub-2567-granularities-update-cube-core-tesseract-cube-runtime-sdk)

@claude

claude Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Claude finished @igorlukanin's task in 54s —— View job


Re-review after db2126e — no substantive change

Details
  • Diffed db2126e vs. prior re-review state
  • Confirmed my finding MUTEX for requests #2 (/v1/meta-extended leaks granularitiesBlock) is still open

db2126e is a pure lint fix in GranularityResolver.ts (replacing continue with a shadowsBuiltIn guard in the same condition). Behavior for prior finding #1 is preserved — an overridden built-in still emits type: 'built-in'.

Still open

Medium — /v1/meta-extended leaks the internal granularitiesBlock. No change since my previous re-review. packages/cubejs-api-gateway/src/helpers/transform-meta-extended.ts:58-66 still spreads ...dimension verbatim with no stripping of granularitiesBlock, and CubeToMetaTransformer.ts:345 still emits it on every time-dim config. /v1/meta-extended responses continue to expose the raw { includes, excludes, custom } block; shape also diverges from /v1/meta (no effectiveGranularities).

Fix as previously suggested:

function transformDimension(dimension: any, cubeDefinitions: any) {
  const { cubeName, memberName } = getMemberPath(dimension.name);
  const { granularitiesBlock, ...rest } = dimension;
  return {
    ...rest,
    sql: stringifyMemberSql(cubeDefinitions[cubeName]?.dimensions?.[memberName]?.sql),
    case: handleDimensionCaseCondition(cubeDefinitions[cubeName]?.dimensions?.[memberName]?.case),
  };
}

Fix this →

• • Branch

@claude

claude Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Claude finished @igorlukanin's task in 1m 0s —— View job


Re-review after da17307 — 1 medium still open

Details
  • Diffed da17307 vs. the prior re-review state
  • Re-checked CubeToMetaTransformer.ts:335-345 (now conditionally emits granularitiesBlock)
  • Re-checked /v1/meta-extended path in transform-meta-extended.ts and gateway wiring
  • Confirmed /v1/meta still strips the block via applyGlobalGranularitiesToMetaCubes (gateway.ts:2535)

da17307 makes granularitiesBlock conditional on presence in CubeToMetaTransformer.ts:345 and refreshes snapshots. It does not change the /v1/meta-extended path, so the previously-flagged leak is narrowed (only time dims that actually use the dict form) but not closed.

Still open

Medium — /v1/meta-extended leaks granularitiesBlock for time dims that use the new dict form. packages/cubejs-api-gateway/src/helpers/transform-meta-extended.ts:58-66 still spreads ...dimension verbatim with no stripping step. When a time dim is declared with granularities: { includes, excludes, custom }, CubeToMetaTransformer.ts:345 now attaches granularitiesBlock on that dim, and /v1/meta-extended passes it through raw. /v1/meta strips it (gateway.ts:2535) and attaches effectiveGranularities in its place — so the two endpoints diverge in shape as well as leaking internal state on the extended one.

I attempted to file this inline, but transform-meta-extended.ts isn't in this PR's diff so GitHub rejects the comment. Suggested fix:

function transformDimension(dimension: any, cubeDefinitions: any) {
  const { cubeName, memberName } = getMemberPath(dimension.name);
  const { granularitiesBlock, ...rest } = dimension;

  return {
    ...rest,
    sql: stringifyMemberSql(cubeDefinitions[cubeName]?.dimensions?.[memberName]?.sql),
    case: handleDimensionCaseCondition(cubeDefinitions[cubeName]?.dimensions?.[memberName]?.case),
  };
}

Separately (out of scope if you'd rather punt): /v1/meta-extended doesn't attach effectiveGranularities either. Not a leak, but worth confirming the shape gap vs /v1/meta is intentional.

Notes

  • da17307's snapshot updates look right — the pre-existing snapshot that still asserted granularitiesBlock: undefined on non-dict-form dims was correctly removed.
  • No behavior change for the two previously-resolved findings (type: 'built-in' on override, granularities.custom-scoped camelize guard).

• • Branch

@claude

claude Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Claude finished @igorlukanin's task in 3m 41s —— View job


Re-review after 314311d — no blocking findings

314311d is a substantial refactor: /v1/meta granularity enrichment moves out of the gateway and into CubeToMetaTransformer (baked at compile for env/static configs) and CompilerApi.selectGranularityVariant (per-request variant cache for the function form). Prior open finding on /v1/meta-extended is now resolved as a side effect — granularitiesBlock is no longer emitted anywhere in meta output.

Details, checklist, prior findings
  • Diffed 314311d vs. da17307 (16 files, +943/−192)
  • Reviewed CompilerApi.selectGranularityVariant / buildGranularityVariant / variant LRU
  • Reviewed CubeToMetaTransformer (baked path + granularityInputs map for function form)
  • Reviewed GranularityConfigHash + hash test
  • Verified prior finding on /v1/meta-extended granularitiesBlock leak — RESOLVED (block no longer emitted; also gets effectiveGranularities now via selectGranularityVariant)
  • Verified prior finding on metaConfigExtended(context, ctx) mis-passing — fix in gateway.ts:1230 swaps to metaConfigExtended(ctx, { requestId: ctx.requestId }) with an explanatory comment
  • Checked granularity-variants.test.ts covers dedup / LRU / failure-retry / recompile-invalidates paths
  • Confirmed prior claude threads all resolved

Prior open finding — resolved by this commit

/v1/meta-extended granularitiesBlock leak (medium). CubeToMetaTransformer.ts used to emit granularitiesBlock on time dims that used the dict form; /v1/meta-extended's transformDimension spread it verbatim. In 314311d the block is dropped entirely from the wire (CubeToMetaTransformer.ts:424 now emits effectiveGranularities in its place), so there is nothing left for transform-meta-extended.ts:58-66 to leak. As a bonus, metaConfigExtended now routes through selectGranularityVariant (CompilerApi.ts:1263), so /v1/meta-extended gains effectiveGranularities parity with /v1/meta.

New code — notes

Variant cache correctness. selectGranularityVariant (CompilerApi.ts:1123-1163) is well-designed:

  • Cache lives on the Compiler object → recompile discards it implicitly.
  • MAX_GRANULARITY_VARIANTS = 16 with LRU refresh via delete+set; eviction is keys().next().value (Map insertion order = LRU order). ✅
  • Concurrent misses of one hash dedup because the promise is set synchronously before the await, and the async build is deferred via Promise.resolve().then(...). Test concurrent identical-config misses dedup to a single build locks this in.
  • Failed builds self-evict via variant.catch(...), and the guard cache.get(hash) === variant prevents evicting a newer variant that raced in. ✅
  • Base cube identity is preserved for non-time-dim cubes (test uncustomized time dimensions share the default set within a variant asserts this).

Hash canonical form. granularityConfigHash treats enabledBuiltIns and custom insertion order as significant (they affect wire output). Non-string field values are silently dropped by asHashableString; today CubeValidator enforces string shapes, so this is defensive rather than lossy. interval/offset/origin/title/format all participate — test distinguishes an absent field from a present one covers the JSON-stringify absent-vs-undefined edge.

compilerVersion composition. Env/static configs fold _gran_<hash> into compilerVersion (CompilerApi.ts:250) so an env change forces a full recompile. Function form intentionally does not — it churns per request via the variant cache instead. Test resolved config hash is folded into compilerVersion confirms.

compilerId composition. mixInMaskHash is factored out and both visibility + granularity hashes now mix in with a documented fixed order (CompilerApi.ts:1240-1249). Test granularity variant selects first, visibility patches on top, compilerId mixes both covers the interaction. Note that metaConfigExtended intentionally does not compose granularity hash into any returned id (it returns {metaConfig, cubeDefinitions}, no id).

granularityInputsForDimension (CubeToMetaTransformer.ts:472-493). Local custom titles resolve to short-title form here (matching the legacy granularities array) so the two shapes stay consistent. sql is never spread through — only the whitelisted fields (title, interval, offset, origin, format). ✅

Bug fix in pre-agg job path. gateway.ts:1230 now passes ctx (job-merged context) as the request context to metaConfigExtended, not the outer context. Prior code was silently selecting visibility/granularities for the wrong tenant on pre-agg job status polls. Good catch, worth calling out in the PR description if not already.

Minor / nits (not filing inline)

  • selectGranularityVariant calls the user's granularities function on every request (needed to compute the hash), even on a cache hit. Unavoidable given the design; worth documenting that users should keep the function cheap.
  • Object.entries(config.customGranularities) in granularityConfigHash relies on Map/object iteration order — the comment explains this is intentional, but a user config that reorders semantically-equal customs would thrash the variant cache. Not a bug, just a footgun worth a docs mention if the function form gets user-facing docs.
  • buildGranularityVariant scans dimensions twice per time-dim cube (some then map). Trivial.

Rate-limiting on /v1/granularities

Still unchanged from the CodeQL note by github-advanced-security; the endpoint reuses userMiddlewares + assertApiScope('meta'), so the risk profile matches /v1/meta. Not re-raising.

• • Branch

@igorlukanin

Copy link
Copy Markdown
Member Author

Reworked where effective granularities are resolved: moved from per-request gateway enrichment into the compiler layer.

  • Env/static granularities configs are now baked into the compiled meta once (their hash is folded into the compiler version, so a config change recompiles). The context-function form resolves per request against a bounded (16-entry) promise-valued LRU of enriched meta variants keyed by a canonical config hash, owned by the compiled model so a recompile drops it. Concurrent misses dedup to one build; failed builds self-evict.
  • compilerId now composes base → visibility mask → granularity hash, so CubeSQL and other compilerId-keyed caches are tenant-correct for context-dependent configs.
  • Steady-state granularity cost on a 6k-cube model drops from ~2 ms/request to ~0.01 ms (baked/cache hit); a new tenant config costs one ~2 ms build.
  • Time dimensions without local customization share a single default set instance per config instead of allocating identical arrays per dimension.
  • /v1/meta?extended no longer includes the internal granularitiesBlock (transport-only field, never documented); effectiveGranularities appears instead. Non-extended /v1/meta, /v1/load, /v1/cubesql, /v1/granularities shapes are unchanged, except local custom granularities now correctly carry their format and title always falls back to the granularity name.
  • Fixed a pre-existing arg swap in the pre-agg job status path (metaConfigExtended(context, ctx)metaConfigExtended(ctx, …)) that selected visibility for the wrong tenant.

@claude

claude Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Claude finished @igorlukanin's task in 40s —— View job


Re-review after 56b8b82 — no blocking findings

56b8b82 is a pure tuning change: MAX_GRANULARITY_VARIANTS 16 → 64 with an expanded rationale on the constant, plus a test refactor that derives the loop bound from the constant instead of hard-coding 17. No behavior change to the eviction, dedup, or self-eviction paths I checked in the prior review.

Details
  • Diffed 56b8b82 vs. 314311d — only CompilerApi.ts and granularity-variants.test.ts touched
  • Confirmed the cache mechanics (insertion-order LRU via keys().next().value, promise-valued entries, cache.get(hash) === variant guard on failed-build eviction) are unchanged
  • Confirmed the LRU-eviction test still asserts the same three invariants (bound-sized cache after overflow, warning logged once, LRU entry rebuilds while newest stays cached), now parameterised on the constant
  • Confirmed all prior claude threads on this PR are resolved

Sizing note

The new bound (64) sits under CubeSQL's LRU-100 compiler cache, which the comment now calls out explicitly — reasonable ceiling given each entry is roughly a full enriched meta copy (~2 MB on a 3k-cube model per the commit rationale ⇒ ~128 MB worst-case for this cache alone). If a real deployment's granularities function ever keys on something high-cardinality (tenant id, user id), you'd hit steady eviction well before OOM but with meaningful churn; a follow-up metric on variantCache hit rate would make that observable. Not blocking.

Still worth confirming (unchanged from prior notes)

  • The context function is invoked on every request to compute the hash, even on cache hits — users should keep it cheap. Worth documenting when the function form gets public docs.
  • Semantically-equal customs in a different insertion order thrash the cache (documented tradeoff in granularityConfigHash); another docs-time note.

Nothing here needs a code change.

• • Branch

@igorlukanin
igorlukanin force-pushed the igor/cub-2567-granularities-update-cube-core-tesseract-cube-runtime-sdk branch from 904ee5f to a7a7604 Compare August 13, 2026 10:36
@claude

claude Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Claude finished @igorlukanin's task in 41s —— View job


Re-review after a7a7604 — rebase only; 3 prior findings (1 high, 2 medium) still apply

Details, checklist, verification
  • Determined what changed: a7a7604 is the rebased equivalent of the previously-reviewed 904ee5f (itself a rebase of 41a77d0 / 6216c54 / e076da0 / 7a77487 / cf1638a) — same subject (refactor(granularities): resolve and apply global config in CubeSymbols instead of a bespoke CompilerApi compile flow) and an identical change surface (9 files, +229/−213, same file list). 904ee5f is no longer reachable in this checkout, confirming a rebase rather than new work layered on top.
  • Re-verified each finding's anchor in the current tree:
    • packages/cubejs-schema-compiler/src/compiler/CubeSymbols.ts:673this.mergeGlobalCustomsIntoDimension(dim); still runs inside phase-1 normalizeDimensionGranularities, ahead of CubeValidator.
    • packages/cubejs-schema-compiler/src/compiler/CubeToMetaTransformer.ts:346granularitiesBlock?.custom ?? extendedDimDef.granularities unchanged (thread anchored at 344).
    • packages/cubejs-schema-compiler/src/compiler/CubeSymbols.ts:1666 — the unconditional fallback to globalGranularities.customGranularities[granName] unchanged (thread anchored at 1663).
  • Listed existing review threads. The three claude threads from the prior review are unresolved and not outdated, and you've confirmed each (decisions deferred to the planning doc). Re-affirming them rather than resolving or re-posting duplicates. The two older claude threads (built-in override type, IGNORE_CAMELIZE scoping) remain resolved.
  • Could not run lint/tsc/unit tests — no node_modules in this checkout, so verification is static only.

Still open (unchanged)

# Severity File Concern
1 High packages/cubejs-schema-compiler/src/compiler/CubeSymbols.ts:673 Global-custom merge runs before CubeValidator, so a config-level custom that resolveFromList accepts ({ name: 'retail_period', interval: '5 days' }, or a name failing identifierRegex) is validated through the stricter per-dimension CustomGranularityEntrySchema and fails compilation on every plain time dimension. Repro confirmed by you; the choice between config-time validation and moving the merge past validation is recorded in the planning doc.
2 Medium packages/cubejs-schema-compiler/src/compiler/CubeToMetaTransformer.ts:346 A time dim declaring no granularities has no granularitiesBlock, so the deprecated granularities meta field falls through to the merged map and now advertises global customs — a wire-shape change on the deprecated field, and those dims lose the shared defaultSet fast path. Confirmed.
3 Medium packages/cubejs-schema-compiler/src/compiler/CubeSymbols.ts:1666 The unconditional global fallback in resolveGranularity undoes the per-dimension includes/excludes filtering the merge performs — an excluded global still resolves in SQL. Product call on display-only vs. enforced; test to be pinned once settled.

Nothing new in this push. The prior review stands verbatim for a7a7604, and the three threads above are left unresolved since the underlying decisions are still pending.

• [Branch](https://github.com/cube-js/cube/tree/igor/cub-2567-granularities-update-cube-core-tesseract-cube-runtime-sdk)

@igorlukanin
igorlukanin force-pushed the igor/cub-2567-granularities-update-cube-core-tesseract-cube-runtime-sdk branch from a7a7604 to da0d7cb Compare August 18, 2026 00:20
@claude

claude Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Claude finished @igorlukanin's task in 35s —— View job


Re-review after da0d7cb — rebase only; 3 prior findings (1 high, 2 medium) still apply

Details, checklist, verification
  • Determined what changed: da0d7cb is the rebased equivalent of the previously-reviewed a7a7604 (itself a rebase of 904ee5f / 41a77d0 / 6216c54 / e076da0 / 7a77487 / cf1638a) — same subject (refactor(granularities): resolve and apply global config in CubeSymbols instead of a bespoke CompilerApi compile flow) and an identical change surface (9 files, +229/−213, same file list). a7a7604 is no longer reachable in this checkout, confirming a rebase rather than new work layered on top.
  • Re-verified each finding's anchor in the current tree:
    • packages/cubejs-schema-compiler/src/compiler/CubeSymbols.ts:673this.mergeGlobalCustomsIntoDimension(dim); still runs inside phase-1 normalizeDimensionGranularities, ahead of CubeValidator.
    • packages/cubejs-schema-compiler/src/compiler/CubeToMetaTransformer.ts:346granularitiesBlock?.custom ?? extendedDimDef.granularities unchanged (thread anchored at 344), with the comment above it still asserting the global merge leaves the legacy field untouched.
    • packages/cubejs-schema-compiler/src/compiler/CubeSymbols.ts:1663-1667 — the hasOwnProperty-guarded unconditional fallback to globalGranularities.customGranularities[granName] unchanged.
  • Listed existing review threads. The three claude threads from the prior review are unresolved and not outdated, and you've confirmed each (decisions deferred to the planning doc). Re-affirming them rather than resolving or re-posting duplicates. The two older claude threads (built-in override type, IGNORE_CAMELIZE scoping) remain resolved.
  • Could not run lint/tsc/unit tests — no node_modules in this checkout, so verification is static only.

Still open (unchanged)

# Severity File Concern
1 High packages/cubejs-schema-compiler/src/compiler/CubeSymbols.ts:673 Global-custom merge runs before CubeValidator, so a config-level custom that resolveFromList accepts ({ name: 'retail_period', interval: '5 days' }, or a name failing identifierRegex) is validated through the stricter per-dimension CustomGranularityEntrySchema and fails compilation on every plain time dimension. Repro confirmed by you; the choice between config-time validation and moving the merge past validation is recorded in the planning doc.
2 Medium packages/cubejs-schema-compiler/src/compiler/CubeToMetaTransformer.ts:346 A time dim declaring no granularities has no granularitiesBlock, so the deprecated granularities meta field falls through to the merged map and now advertises global customs — a wire-shape change on the deprecated field, and those dims lose the shared defaultSet fast path. Confirmed.
3 Medium packages/cubejs-schema-compiler/src/compiler/CubeSymbols.ts:1663 The unconditional global fallback in resolveGranularity undoes the per-dimension includes/excludes filtering the merge performs — an excluded global still resolves in SQL. Product call on display-only vs. enforced; test to be pinned once settled.

Nothing new in this push. The prior review stands verbatim for da0d7cb, and the three threads above are left unresolved since the underlying decisions are still pending.

• [Branch](https://github.com/cube-js/cube/tree/igor/cub-2567-granularities-update-cube-core-tesseract-cube-runtime-sdk)

…; harden resolution (proto keys, built-in interval, view block, validator, config validation, hash)
…and SQL paths; drop empty-interval env customs; non-array function return uses default catalog
…aritiesFor + GRANULARITY_STRING_FIELDS constant
…ut a local block (share one global-custom map by reference)
…fs map; memoize proxy bound methods; dedup definitions cache; hash only output-affecting built-in fields; fix custom-named/proto-name bugs; skip compilerId when unneeded
…ile time, bake into compiled model (CUB-2567)
… CompilerApi, tighten comments (CUB-2567 review)
…ls instead of a bespoke CompilerApi compile flow
@igorlukanin
igorlukanin force-pushed the igor/cub-2567-granularities-update-cube-core-tesseract-cube-runtime-sdk branch from da0d7cb to dba5209 Compare August 19, 2026 07:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

client:core Issues relating to the JavaScript client SDK javascript Pull requests that update Javascript code python rust Pull requests that update Rust code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants