Skip to content

Property-path traversal can't be combined with multi-ledger datasets — blocks cross-ledger graph crawl #1405

Description

@Jackamus29

Property-path / graph-crawl traversal and multi-ledger datasets are each documented, first-class
features, but they cannot be composed: a transitive path (p+ / p*) cannot run over a
multi-ledger dataset. The natural single-query forms all fail — one is rejected by a guard, and
the suggested GRAPH workaround crashes on indexed data and silently drops rows on cross-graph
joins. Net effect: "traverse a hierarchy in one ledger and join instances in another" — graph
crawl across ledgers — can't be expressed in a single query.

Re-verified on v4.1.1 (running server + a runnable, scrubbed regression test). Distinct
from #1295
(that's cross-ledger explicit projection dropping foreign-namespace predicates — a
formatter bug; this is traversal / joins in the execution layer).

Why this should compose (it's documented)

  • Property paths / graph crawl are documented first-class (query/graph-crawl.md,
    query/sparql.md). The docs' own "Not Yet Supported" list for property paths contains only
    p? and !p — the multi-graph restriction below is not listed anywhere.
  • Multi-ledger datasets + cross-ledger joins are documented first-class (query/datasets.md:
    "queries across multiple graphs and ledgers simultaneously … powerful data integration"; the
    "Multi-Ledger Datasets" and "Cross-Ledger Joins" sections).

So a developer reading the docs reasonably expects transitive traversal across ledgers to work.

Motivating use case

A node in ledger A references a node in ledger B, and A also carries the edge that
bridges into B's hierarchy
— e.g. a user-defined term broader→ a canonical term in a shared
taxonomy ledger. Answering "what is reachable via broader+, joined to instances?" needs a path
whose hops span A and B, plus a join back to A's instances. There is no single-query way to
express this today.

The three failures (verified v4.1.1)

Shape Result Class
1 property path over a multi-graph from (union) rejected — Property paths over multi-graph datasets are not supported; use GRAPH to select a single graph guard (undocumented limitation)
2 GRAPH-scoped path over an indexed multi-ledger dataset internal error — EncodedSid/EncodedPid reached stamp_provenance bug
3 variable join across a GRAPH boundary, divergent namespace codes silently empty bug

⚠️ Triggers matter — a naive repro won't fire. 2 only reproduces when the data is
indexed (binary store), not novelty-only. 3 only reproduces when the cross-graph join
key's namespace has divergent codes across the two ledgers. The regression test below
reproduces both deterministically.

Crucially, the motivating use case needs form 1 (the bridge hop spans ledgers, so the path is
inherently graph-spanning) — and GRAPH-scoping cannot express a graph-spanning path. So even
with 2/3 fixed, lifting the guard (1) is what actually unblocks the use case.

1 — union path is guarded (intentional, but an undocumented limitation)

SELECT ?x FROM <a:main> FROM <b:main> WHERE { ex:start ex:broader* ?x }
# → Invalid query: Property paths over multi-graph datasets are not supported; use GRAPH ...

The path operator is a single-snapshot BFS (property_path.rs)
gated by require_single_graph (context.rs:756-770);
it cannot walk across ledger indexes without per-hop SID re-encoding. The guard is the right
call vs. returning a silent subset — but the restriction isn't in the docs' property-path
limitations list, and it blocks the documented composition above.

2 — GRAPH-scoped path over an indexed multi-ledger dataset → internal error (bug)

SELECT DISTINCT ?thing FROM <catalog:main> FROM NAMED <taxonomy:main>
WHERE { ?thing ex:category ?c . GRAPH <taxonomy:main> { ?c ex:broader* ex:top } }
# → EncodedSid/EncodedPid reached stamp_provenance — binary store should have been disabled
#   for multi-ledger datasets

(On v4.1.1 this is now surfaced as err:db/InvalidQuery / 400 rather than a 500 — reclassified,
but the internal-invariant message still leaks.) Root cause: inside a GRAPH block over a
multi-ledger dataset, the GRAPH operator forces eager materialization and stamps inner results
with IriMatch provenance so SIDs decode against the right ledger
(graph.rs:176-248); that stamping assumes a decodable
Binding::Sid, but the property-path operator emits late EncodedSid/EncodedPid bindings that
bypass the contract, so stamp_binding hits its invariant
(dataset_operator.rs:273-287). At minimum
this should be a clean "unsupported" error, not an internal invariant.

3 — cross-graph join drops rows under namespace divergence (silent bug)

SELECT DISTINCT ?thing FROM NAMED <catalog2:main> FROM NAMED <taxonomy:main>
WHERE { GRAPH <catalog2:main> { ?thing cat:category ?c }
        GRAPH <taxonomy:main>  { ?c ex:broader ex:mid } }
# → []   when the join-key namespace has a different code in catalog2 than in taxonomy

With aligned namespace codes the identical query returns the expected row — so it's not a
join-logic error: a SID bound in one graph is compared against another graph's SIDs without
re-encoding across the namespace tables
. Same family as #1295, but on the GRAPH-join
execution/seed path rather than the formatter. Silent, so the more dangerous of the two.

Root cause map

  • 2 + the guard (1): single-snapshot path operator — a cross-ledger path needs a
    cross-snapshot BFS that re-encodes the frontier SID at each ledger boundary.
  • 3 (and Cross-ledger explicit projection silently drops foreign-namespace predicates #1295): cross-ledger namespace-code divergence — a join key isn't re-encoded against
    the target graph's namespace table. Validated: a deterministic "vocabulary warm-up" that aligns
    namespace codes across ledgers makes 3 pass. Today user namespaces are allocated lazily,
    per-ledger, in first-use order, so shared namespaces drift apart — a way to declare namespace
    codes at ledger genesis
    would fix this class by construction.

What would resolve it

  1. Bug 2: make the GRAPH-scoped path honor the eager-materialization contract (or disable
    the binary-store path for it); failing a fix, a clean error rather than an internal invariant.
  2. Bug 3: re-encode the join key's SID against the target graph's namespace table at the
    GRAPH boundary.
  3. The capability (lifts the guard, 1): cross-snapshot path evaluation — the enhancement that
    actually unblocks graph crawl across ledgers. (Non-trivial; honest about the cost.)

Reproduction

Runnable, scrubbed regression test (generic taxonomy/catalog ledgers — a broader hierarchy

  • an instance referencing a deep node): fluree-db-api/tests/it_multi_graph_property_path.rs
    (--features native). Single-graph path + aligned-namespace join pass; the indexed GRAPH-path
    (2) and divergent-namespace join (3) fail. Re-verified on v4.1.1 against a running server. Happy
    to open a PR with the test.

Current workaround

Decompose into single-graph steps: a single-graph path query on the taxonomy ledger to collect an
IRI set, then a multi-graph projecting query using ["values", …] set-membership (no path).
Correct, but 2–3 round-trips where one should do — and it can't express the bridging case as a
single query.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions