Skip to content

Commit f9c3188

Browse files
added semantic cards: schema discovery (#340)
1 parent bdce4a0 commit f9c3188

74 files changed

Lines changed: 7111 additions & 449 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

CHANGELOG.md

Lines changed: 110 additions & 52 deletions
Large diffs are not rendered by default.

docs/assets/graflo-ontology-viz/embed.html

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
<html lang="en">
33
<head>
44
<meta charset="utf-8" />
5-
<title>GraFlo Ontology (v1.3.0)</title>
5+
<title>GraFlo Ontology (v1.4.0)</title>
66
<link rel="stylesheet" href="graph-view.css" />
77
</head>
88
<body class="embed">
@@ -1490,13 +1490,17 @@
14901490
"label": "targetNamespace",
14911491
"range": "string"
14921492
},
1493+
{
1494+
"label": "unit",
1495+
"range": "string"
1496+
},
14931497
{
14941498
"label": "version",
14951499
"range": "string"
14961500
}
14971501
],
14981502
"universalRoot": "https://ontology.growgraph.dev/graflo/GrafloArtifact",
1499-
"version": "1.3.0"
1503+
"version": "1.4.0"
15001504
};</script>
15011505
<script src="graph-view.js"></script>
15021506
</body>

docs/assets/graflo-ontology-viz/graph-data.json

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1460,11 +1460,15 @@
14601460
"label": "targetNamespace",
14611461
"range": "string"
14621462
},
1463+
{
1464+
"label": "unit",
1465+
"range": "string"
1466+
},
14631467
{
14641468
"label": "version",
14651469
"range": "string"
14661470
}
14671471
],
14681472
"universalRoot": "https://ontology.growgraph.dev/graflo/GrafloArtifact",
1469-
"version": "1.3.0"
1473+
"version": "1.4.0"
14701474
}

docs/assets/graflo-ontology-viz/index.html

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,13 +2,13 @@
22
<html lang="en">
33
<head>
44
<meta charset="utf-8" />
5-
<title>GraFlo Ontology (v1.3.0)</title>
5+
<title>GraFlo Ontology (v1.4.0)</title>
66
<link rel="stylesheet" href="graph-view.css" />
77
</head>
88
<body>
99
<div class="layout">
1010
<aside class="sidebar">
11-
<h1>GraFlo Ontology (v1.3.0)</h1>
11+
<h1>GraFlo Ontology (v1.4.0)</h1>
1212
<p>Class graph from <code>graflo.ttl</code>, grouped into the blocks the manifest composes. Within a block, depth runs left to right: each class sits right of whatever contains or generalises it. Taxonomy is shown by default; select a class to reveal its properties.</p>
1313
<div class="control">
1414
<label for="search">Search classes</label>
@@ -1508,13 +1508,17 @@ <h1>GraFlo Ontology (v1.3.0)</h1>
15081508
"label": "targetNamespace",
15091509
"range": "string"
15101510
},
1511+
{
1512+
"label": "unit",
1513+
"range": "string"
1514+
},
15111515
{
15121516
"label": "version",
15131517
"range": "string"
15141518
}
15151519
],
15161520
"universalRoot": "https://ontology.growgraph.dev/graflo/GrafloArtifact",
1517-
"version": "1.3.0"
1521+
"version": "1.4.0"
15181522
};</script>
15191523
<script src="graph-view.js"></script>
15201524
</body>

docs/concepts/schema/ontology.md

Lines changed: 23 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,8 +28,8 @@ flowchart TB
2828
| Role | IRI |
2929
|------|-----|
3030
| Ontology document | `https://ontology.growgraph.dev/graflo` |
31-
| Version IRI | `https://ontology.growgraph.dev/graflo/1.3.0` |
32-
| Version info | `1.3.0` |
31+
| Version IRI | `https://ontology.growgraph.dev/graflo/1.4.0` |
32+
| Version info | `1.4.0` |
3333
| Vocabulary prefix `gf:` | `https://ontology.growgraph.dev/graflo/` |
3434

3535
The Turtle source lives in the package at `graflo/rdf/ontology/graflo.ttl`. Constants are also exposed in Python as `graflo.rdf.namespace` (`GF_ONTOLOGY_IRI`, `GF_VERSION`, `GF_VERSION_IRI`, `GF_BASE`).
@@ -92,6 +92,27 @@ If the embedded viewer is blank in an IDE browser preview, use **Open full scree
9292
- `gf:Bindings`, and `gf:BoundConnector` with one subclass per connector model: `gf:FileConnector`, `gf:TableConnector`, `gf:SparqlConnector`, `gf:APIConnector`, `gf:KafkaConnector`
9393
- `gf:ResourceConnectorBinding`, `gf:ConnectorConnectionBinding`, `gf:StagingProxyBinding`
9494

95+
**Semantic grounding** (optional, added in 1.4.0)
96+
97+
An element may be anchored to an external vocabulary through a `semantics:` block on the schema metadata, a vertex, an edge, or a field:
98+
99+
```yaml
100+
vertices:
101+
- name: person
102+
identity: [email]
103+
semantics:
104+
iri: https://schema.org/Person
105+
exact_match: [http://xmlns.com/foaf/0.1/Person]
106+
synonyms: [individual, human]
107+
properties:
108+
- name: speed
109+
type: FLOAT
110+
semantics:
111+
unit: m/s
112+
```
113+
114+
This maps to `gf:semanticIri`, `skos:exactMatch`, `skos:altLabel`, and — fields only — `gf:unit`. The block is purely descriptive: identity, storage naming and ingestion behave identically whether or not it is present. `unit` is rejected outside a field, where it would be meaningless.
115+
95116
**Enumerations** (named individuals): `gf:DBType` (ArangoDB, Neo4j, …), transform target/strategy, key-selection mode, edge duplicate policy, bound source kind.
96117

97118
**PROV-O hooks**: `gf:GraphManifest` ⊑ `prov:Entity`, `gf:ProtoTransform` ⊑ `prov:Activity` (subclasses such as `gf:Transform` inherit this; for lineage tooling).

docs/guides/importing.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,10 +14,16 @@ L0 graflo.onto enums and core vocabulary (DBType, identity selectors
1414
1515
L1 graflo.filter FilterExpression, SelectSpec, JoinClause
1616
graflo.architecture.graph_types containers, contexts, identifiers, EdgeDerivation
17+
graflo.architecture.onto_sql leaf pydantic models (SQL introspection)
18+
graflo.architecture.util helpers over graph_types only
1719
1820
L2 graflo.architecture.schema Schema, VertexConfig, EdgeConfig,
1921
DatabaseProfile, db-aware projections,
2022
identity_uuid helpers
23+
graflo.architecture.query read contract (QueryCaps, NodeQuery,
24+
NeighborQuery, TraverseQuery, AggregateQuery,
25+
and QueryResult)
26+
graflo.architecture.onto_sample leaf sample models (ResourceSample, etc.)
2127
2228
L3 graflo.connections connection *configs*: DBConfig + per-backend configs
2329
(onto), source configs (sources), DBType mapping

graflo/architecture/__init__.py

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,38 +47,82 @@
4747
"Transform": "graflo.architecture.contract",
4848
"DatabaseProfile": "graflo.architecture.schema.database_features",
4949
"Index": "graflo.architecture.graph_types",
50+
"Budget": "graflo.architecture.schema.context",
51+
"BudgetAccounting": "graflo.architecture.schema.context",
52+
"ElisionReport": "graflo.architecture.schema.context",
53+
"RankingWeights": "graflo.architecture.schema.context",
54+
"SchemaCard": "graflo.architecture.schema.context",
55+
"SchemaGraph": "graflo.architecture.schema.context",
56+
"SchemaNeighborhood": "graflo.architecture.schema.context",
57+
"SchemaPath": "graflo.architecture.schema.context",
58+
"build_card": "graflo.architecture.schema.context",
59+
"subschema": "graflo.architecture.schema.context",
60+
"SubschemaSelection": "graflo.architecture.schema.projection",
61+
"build_subschema": "graflo.architecture.schema.projection",
62+
"select_induced": "graflo.architecture.schema.projection",
63+
"AggregateQuery": "graflo.architecture.query",
64+
"CapExceededError": "graflo.architecture.query",
65+
"GraphQuery": "graflo.architecture.query",
66+
"HARD_CAPS": "graflo.architecture.query",
67+
"NeighborQuery": "graflo.architecture.query",
68+
"NodeQuery": "graflo.architecture.query",
69+
"QueryCaps": "graflo.architecture.query",
70+
"QueryResult": "graflo.architecture.query",
71+
"TraverseQuery": "graflo.architecture.query",
5072
}
5173

5274
__all__ = [
75+
"HARD_CAPS",
5376
"APIConnector",
77+
"AggregateQuery",
5478
"Bindings",
5579
"BoundSourceKind",
80+
"Budget",
81+
"BudgetAccounting",
82+
"CapExceededError",
5683
"CoreSchema",
5784
"DatabaseProfile",
5885
"Edge",
5986
"EdgeConfig",
6087
"EdgeConfigDBAware",
88+
"ElisionReport",
6189
"FieldType",
6290
"FileConnector",
6391
"GraFloIndex",
6492
"GraFloOutput",
6593
"GraphManifest",
6694
"GraphMetadata",
6795
"GraphModel",
96+
"GraphQuery",
6897
"Index",
6998
"IngestionModel",
7099
"KafkaConnector",
100+
"NeighborQuery",
101+
"NodeQuery",
71102
"ProtoTransform",
103+
"QueryCaps",
104+
"QueryResult",
105+
"RankingWeights",
72106
"Resource",
73107
"ResourceConnector",
74108
"Schema",
109+
"SchemaCard",
75110
"SchemaDBAware",
111+
"SchemaGraph",
112+
"SchemaNeighborhood",
113+
"SchemaPath",
76114
"SparqlConnector",
115+
"SubschemaSelection",
77116
"TableConnector",
78117
"Transform",
118+
"TraverseQuery",
79119
"Vertex",
80120
"VertexConfig",
81121
"VertexConfigDBAware",
122+
"build_card",
123+
"build_subschema",
124+
"select_induced",
125+
"subschema",
82126
]
83127

84128

Lines changed: 42 additions & 81 deletions
Original file line numberDiff line numberDiff line change
@@ -1,102 +1,63 @@
1-
"""Pure planning for :class:`~graflo.architecture.evolution.ops.ProjectManifestOp`."""
1+
"""Pure planning for :class:`~graflo.architecture.evolution.ops.ProjectManifestOp`.
2+
3+
Selector validation and manifest unwrapping live here; the induced-connectivity
4+
kernel itself lives at layer 2 in
5+
:mod:`graflo.architecture.schema.projection` so manifest projection and schema
6+
context projection cannot drift apart.
7+
"""
28

39
from __future__ import annotations
410

5-
from graflo.architecture.base import ConfigBaseModel
611
from graflo.architecture.contract.manifest import GraphManifest
7-
from graflo.architecture.graph_types import EdgeId
12+
from graflo.architecture.schema.projection import SubschemaSelection, select_induced
813

914
from .ops import EdgeSelector, ProjectManifestOp
1015

1116

12-
class ProjectionPlan(ConfigBaseModel):
13-
"""Survivor and removal sets computed from a projection op."""
14-
15-
surviving_vertices: set[str]
16-
surviving_edge_ids: set[EdgeId]
17-
removed_vertices: set[str]
18-
removed_edge_ids: set[EdgeId]
17+
def _selector_edge_ids(
18+
selectors: list[EdgeSelector],
19+
) -> set[tuple[str, str, str | None]]:
20+
return {selector.edge_id() for selector in selectors}
1921

2022

21-
def _selector_edge_ids(selectors: list[EdgeSelector]) -> set[EdgeId]:
22-
return {selector.edge_id() for selector in selectors}
23+
def _validate_strict(manifest: GraphManifest, op: ProjectManifestOp) -> None:
24+
"""Raise when strict mode is on and a selector names something undeclared."""
25+
schema = manifest.require_schema()
26+
all_vertices = schema.core_schema.vertex_config.vertex_set
27+
all_edge_ids = {edge.edge_id for edge in schema.core_schema.edge_config.edges}
28+
29+
if op.keep_vertices:
30+
missing_vertices = sorted(set(op.keep_vertices) - all_vertices)
31+
if missing_vertices:
32+
raise ValueError(f"Unknown vertices in keep_vertices: {missing_vertices}")
33+
if op.keep_edges:
34+
missing_edges = sorted(_selector_edge_ids(op.keep_edges) - all_edge_ids)
35+
if missing_edges:
36+
raise ValueError(
37+
"Unknown edges in keep_edges: "
38+
+ ", ".join(
39+
f"({source!r}, {target!r}, {relation!r})"
40+
for source, target, relation in missing_edges
41+
)
42+
)
2343

2444

2545
def compute_projection(
2646
manifest: GraphManifest, op: ProjectManifestOp
27-
) -> ProjectionPlan:
47+
) -> SubschemaSelection:
2848
"""Compute survivor/removal sets without mutating *manifest*."""
2949
schema = manifest.graph_schema
3050
if schema is None:
3151
raise ValueError("project_manifest requires graph_schema")
3252

33-
all_vertices = schema.core_schema.vertex_config.vertex_set
34-
all_edges = schema.core_schema.edge_config.edges
35-
all_edge_ids = {edge.edge_id for edge in all_edges}
36-
3753
if op.strict:
38-
if op.keep_vertices:
39-
missing_vertices = sorted(set(op.keep_vertices) - all_vertices)
40-
if missing_vertices:
41-
raise ValueError(
42-
f"Unknown vertices in keep_vertices: {missing_vertices}"
43-
)
44-
if op.keep_edges:
45-
requested = _selector_edge_ids(op.keep_edges)
46-
missing_edges = sorted(requested - all_edge_ids)
47-
if missing_edges:
48-
raise ValueError(
49-
"Unknown edges in keep_edges: "
50-
+ ", ".join(
51-
f"({source!r}, {target!r}, {relation!r})"
52-
for source, target, relation in missing_edges
53-
)
54-
)
55-
56-
if op.keep_edges is not None:
57-
keep_edge_ids = _selector_edge_ids(op.keep_edges)
58-
surviving_edge_ids = keep_edge_ids & all_edge_ids
59-
else:
60-
surviving_edge_ids = set(all_edge_ids)
61-
62-
keep_vertex_set = set(op.keep_vertices) if op.keep_vertices is not None else None
63-
64-
if keep_vertex_set is not None:
65-
surviving_edge_ids = {
66-
edge_id
67-
for edge_id in surviving_edge_ids
68-
if edge_id[0] in keep_vertex_set and edge_id[1] in keep_vertex_set
69-
}
70-
71-
surviving_vertices: set[str] = set()
72-
for source, target, _relation in surviving_edge_ids:
73-
surviving_vertices.add(source)
74-
surviving_vertices.add(target)
75-
76-
if keep_vertex_set is not None:
77-
surviving_vertices &= keep_vertex_set
78-
if op.connectivity == "induced_prune":
79-
connected_in_keep = {
80-
vertex
81-
for vertex in keep_vertex_set
82-
if any(
83-
source == vertex or target == vertex
84-
for source, target, _relation in surviving_edge_ids
85-
)
86-
}
87-
surviving_vertices = connected_in_keep
88-
surviving_edge_ids = {
89-
edge_id
90-
for edge_id in surviving_edge_ids
91-
if edge_id[0] in surviving_vertices and edge_id[1] in surviving_vertices
92-
}
93-
94-
removed_vertices = all_vertices - surviving_vertices
95-
removed_edge_ids = all_edge_ids - surviving_edge_ids
96-
97-
return ProjectionPlan(
98-
surviving_vertices=surviving_vertices,
99-
surviving_edge_ids=surviving_edge_ids,
100-
removed_vertices=removed_vertices,
101-
removed_edge_ids=removed_edge_ids,
54+
_validate_strict(manifest, op)
55+
56+
return select_induced(
57+
schema.core_schema,
58+
keep_vertices=op.keep_vertices,
59+
keep_edge_ids=(
60+
_selector_edge_ids(op.keep_edges) if op.keep_edges is not None else None
61+
),
62+
connectivity=op.connectivity,
10263
)
Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
"""DB-agnostic read contract: what an agent may ask of a live graph.
2+
3+
Layer 2. Imports `filter/` and `graph_types/` (layer 1) and nothing from `db/` —
4+
enforcement has to be expressible without a driver, or "enforced in core" would
5+
mean "enforced wherever a driver happens to be imported".
6+
7+
Not in `contract/`, which is the write-side manifest at layer 3: mixing a read
8+
contract into it would muddy a boundary the layering test already guards.
9+
"""
10+
11+
from graflo.architecture.query.caps import HARD_CAPS, CapExceededError, QueryCaps
12+
from graflo.architecture.query.models import (
13+
AggregateQuery,
14+
GraphQuery,
15+
NeighborQuery,
16+
NodeQuery,
17+
TraverseQuery,
18+
)
19+
from graflo.architecture.query.result import QueryResult
20+
21+
__all__ = [
22+
"HARD_CAPS",
23+
"AggregateQuery",
24+
"CapExceededError",
25+
"GraphQuery",
26+
"NeighborQuery",
27+
"NodeQuery",
28+
"QueryCaps",
29+
"QueryResult",
30+
"TraverseQuery",
31+
]

0 commit comments

Comments
 (0)