Context
Typed xsd:decimal and overflow xsd:integer values are NumBig-arena-keyed per (graph, predicate), with handles minted at index build. Overlay translation now resolves values already in the arena read-only (BinaryIndexStore::find_numbig_handle, used by value_to_otype_okey's BigInt/Decimal arms in fluree-db-query/src/binary_scan.rs), which covers every retract and every re-assert of an indexed value.
A value the index has never seen still has no handle, so translation returns Unsupported and collect_resolved_overlay_ops declines the predicate's whole batched probe lane (fast_path_common.rs) until the next reindex. On decimal-assert-heavy novelty (e.g. frequent updates writing new budgetAmount values), the batched join lanes never engage for that predicate.
Severity note: the TranslatedOverlayCache memo (landed via #1308) removed the old O(rows × novelty) per-row re-translation cliff, so the cost of a decline today is "memoized per-row scans" rather than quadratic blowup. This issue is the remaining step to keep the batched lanes fully live.
Proposal
Mirror the ephemeral-p_id pattern used for novelty-only predicates:
- During per-predicate ops collection, assign ephemeral handles above the arena watermark for arena-miss values, keyed by value so the same decimal maps to one handle within a translation.
- Thread a decode side-table (ephemeral handle → value) to consumers, the way
ephemeral_preds / ephemeral_p_id_to_sid already flows through binary_range and TranslatedOverlayOps.
Soundness: the arena is equality-only (no value order in o_key), and a value absent from the arena can never identity-match a base row — so ephemeral handles only ever participate in overlay-only assert injection and intra-overlay identity, never in base-row reconciliation.
Acceptance
- A novelty assert of a never-indexed decimal keeps the batched subject/object lanes engaged (extend
it_join_batched_overlay.rs::batched_join_decimal_novelty phase B, which currently asserts the decline-and-fallback behavior).
- Ephemeral handles decode back to the exact
BigDecimal/BigInt for projection and FILTER evaluation.
Context
Typed
xsd:decimaland overflowxsd:integervalues are NumBig-arena-keyed per(graph, predicate), with handles minted at index build. Overlay translation now resolves values already in the arena read-only (BinaryIndexStore::find_numbig_handle, used byvalue_to_otype_okey'sBigInt/Decimalarms influree-db-query/src/binary_scan.rs), which covers every retract and every re-assert of an indexed value.A value the index has never seen still has no handle, so translation returns
Unsupportedandcollect_resolved_overlay_opsdeclines the predicate's whole batched probe lane (fast_path_common.rs) until the next reindex. On decimal-assert-heavy novelty (e.g. frequent updates writing newbudgetAmountvalues), the batched join lanes never engage for that predicate.Severity note: the
TranslatedOverlayCachememo (landed via #1308) removed the old O(rows × novelty) per-row re-translation cliff, so the cost of a decline today is "memoized per-row scans" rather than quadratic blowup. This issue is the remaining step to keep the batched lanes fully live.Proposal
Mirror the ephemeral-p_id pattern used for novelty-only predicates:
ephemeral_preds/ephemeral_p_id_to_sidalready flows throughbinary_rangeandTranslatedOverlayOps.Soundness: the arena is equality-only (no value order in
o_key), and a value absent from the arena can never identity-match a base row — so ephemeral handles only ever participate in overlay-only assert injection and intra-overlay identity, never in base-row reconciliation.Acceptance
it_join_batched_overlay.rs::batched_join_decimal_noveltyphase B, which currently asserts the decline-and-fallback behavior).BigDecimal/BigIntfor projection and FILTER evaluation.