Skip to content

Commit 7068314

Browse files
44 add doc fields that are lists (#321)
* added proxy field to DBConfig * added list type * version bump
1 parent fda19e9 commit 7068314

26 files changed

Lines changed: 701 additions & 120 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,14 @@ All notable changes to this project will be documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8+
## [1.8.13]
9+
10+
### Added
11+
12+
- **`FieldType.LIST`** with required scalar **`item_type`** — homogeneous one-level list properties on `Field` (manifest round-trip preserves `type` + `item_type`). LIST fields cannot be identity / hash-identity sources.
13+
- **Backend type support policy** (`graflo.db.field_type_support`) — native list storage or **raise** (`UnsupportedFieldTypeError`); no silent JSON/`STRING` downgrade. TigerGraph DDL emits `LIST<T>`; Postgres emitter uses `T[]`; Nebula define/DDL raises; Cypher/Arango validate at define.
14+
- Docs: supported vs planned field-type matrix and per-backend LIST table in [core components](docs/concepts/architecture/core_components.md).
15+
816
## [1.8.12]
917

1018
### Added

‎docs/concepts/architecture/core_components.md‎

Lines changed: 110 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -83,17 +83,124 @@ A `Vertex` describes vertices and their logical identity. It supports:
8383
- Single or compound identity fields (e.g., `["first_name", "last_name"]` instead of `"full_name"`)
8484
- Property definitions with optional type information
8585
- Fields can be specified as strings (backward compatible) or typed `Field` objects
86-
- Supported types: `INT`, `FLOAT`, `BOOL`, `STRING`, `DATETIME`
87-
- Type information enables better validation and database-specific optimizations
88-
- Duplicate property declarations are normalized by field name
86+
- Untyped fields (`type: null`) remain valid for schema-agnostic backends
87+
- Duplicate property declarations are normalized by field name
8988
- Same type duplicates merge into one field
9089
- If one duplicate is typed and the other is untyped, the typed definition wins
9190
- Conflicting non-null types for the same field name are rejected
91+
- **LIST-typed properties cannot be identity / hash-identity sources**
9292
- Filtering conditions
9393
- **`blank: true`** — placeholder vertex with no natural key; identity defaults to **`id`** when omitted
9494
- **`hash_identity_properties`** — when non-empty, SHA256 hash of these source fields produces a deterministic synthetic **`id`** (see [Vertex identity modes](../schema/vertex_identity.md))
9595
- **`Vertex.identity_mode`** — derived runtime mode: **`natural`** (upsert on `identity`), **`hash`**, or **`blank`**
9696

97+
#### Supported field types
98+
99+
| `FieldType` | Shape | Notes |
100+
|-------------|-------|-------|
101+
| `INT` `UINT` `FLOAT` `DOUBLE` `BOOL` `STRING` `DATETIME` | scalar | Existing scalar types |
102+
| `LIST` | + required `item_type` (scalar above) | Homogeneous, **one level** only — no `LIST[LIST[…]]`, no object schemas |
103+
104+
Declare types on each property as a mapping (`name` / `type` / optional `item_type`).
105+
String shorthand (`properties: [id, name]`) still works and leaves types unset.
106+
107+
**Example — article vertex with scalar + list properties:**
108+
109+
```yaml
110+
schema:
111+
metadata:
112+
name: demo
113+
version: "1.0.0"
114+
graph:
115+
vertex_config:
116+
vertices:
117+
- name: article
118+
# Identity must be a scalar (or untyped) field — never LIST
119+
identity: [doi]
120+
properties:
121+
- name: doi
122+
type: STRING
123+
- name: title
124+
type: STRING
125+
- name: year
126+
type: INT
127+
# Homogeneous list of strings → TigerGraph LIST<STRING>, Neo4j list, PG TEXT[]
128+
- name: tags
129+
type: LIST
130+
item_type: STRING
131+
# Homogeneous list of floats
132+
- name: topic_scores
133+
type: LIST
134+
item_type: FLOAT
135+
edge_config:
136+
edges:
137+
- source: article
138+
target: article
139+
relation: cites
140+
properties:
141+
- name: contexts
142+
type: LIST
143+
item_type: STRING
144+
db_profile: {}
145+
```
146+
147+
Equivalent compact forms (same semantics):
148+
149+
```yaml
150+
# Inline dicts in a list
151+
properties:
152+
- { name: doi, type: STRING }
153+
- { name: tags, type: LIST, item_type: STRING }
154+
155+
# Untyped (schema-agnostic backends); still fine when you do not need DDL typing
156+
properties: [doi, title, tags]
157+
```
158+
159+
**Invalid (rejected at model validation):**
160+
161+
```yaml
162+
# LIST without item_type
163+
- { name: tags, type: LIST }
164+
165+
# Nested / non-scalar item_type
166+
- { name: matrix, type: LIST, item_type: LIST }
167+
168+
# item_type on a non-LIST field
169+
- { name: title, type: STRING, item_type: STRING }
170+
171+
# LIST used as identity
172+
- name: article
173+
identity: [tags]
174+
properties:
175+
- { name: tags, type: LIST, item_type: STRING }
176+
```
177+
178+
For mixed or nested payloads that are not a homogeneous scalar list, author an
179+
explicit `STRING` field and store JSON yourself — that is never an automatic
180+
fallback from `type: LIST`.
181+
182+
#### Backend support (LIST)
183+
184+
Policy: **native storage or raise** — no soft conversion to `STRING`/JSON.
185+
186+
| Backend | LIST as storable property | Behavior |
187+
|---------|---------------------------|----------|
188+
| TigerGraph | Yes — `LIST<T>` attribute | DDL emits `LIST<STRING>`, `LIST<INT>`, … |
189+
| Neo4j / Memgraph / FalkorDB | Yes — homogeneous list of primitives | Validated at define; lists serialize as properties |
190+
| ArangoDB | Yes — document array | Validated at define; pass through |
191+
| Postgres | Yes — SQL arrays | Emitter uses `T[]` for typed LIST columns |
192+
| NebulaGraph | **No** — composites are query-only | Define/DDL **raises** `UnsupportedFieldTypeError` |
193+
194+
#### Planned field types
195+
196+
Not in `FieldType` yet — do not author these in manifests:
197+
198+
| Type | Status | Sketch |
199+
|------|--------|--------|
200+
| `UUID` | PR2 | Logical scalar; TigerGraph/Nebula still store as `STRING` |
201+
| `MAP` | follow-up | `key_type` + `value_type` (scalar); native TG/Arango; Cypher targets raise (maps are not storable node/rel properties) |
202+
| `SET` | follow-up (low) | TigerGraph-oriented; prefer `LIST` elsewhere |
203+
97204
Identity defaults at schema level (`VertexConfig`):
98205

99206
- **`identity_from_all_properties: true`** (default) — vertices without explicit **`identity`** use all **`properties`** names as the logical key.

‎docs/getting_started/creating_manifest.md‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,18 @@ Defines the graph contract.
6666
- `graph.edge_config`: source/target relationships, optional `relation`, optional **`directed`** (default `true`), edge **`properties`**, `identities`
6767
- `db_profile`: DB-specific physical behavior (indexes, naming, **`default_property_values`** for TigerGraph GSQL `DEFAULT` on vertex/edge attributes, backend details)
6868

69+
**Typed properties:** use mappings with `type` (and `item_type` for lists), not only bare name strings:
70+
71+
```yaml
72+
- name: article
73+
identity: [doi]
74+
properties:
75+
- { name: doi, type: STRING }
76+
- { name: tags, type: LIST, item_type: STRING } # homogeneous list only
77+
```
78+
79+
`LIST` requires a scalar `item_type`; list fields cannot be identity keys. Full matrix and invalid cases: [Core components — field types](../concepts/architecture/core_components.md#supported-field-types).
80+
6981
Use `schema` for **what graph exists**.
7082

7183
### `ingestion_model`

‎graflo/architecture/schema/db_aware.py‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -117,7 +117,12 @@ def properties(self, vertex_name: str) -> list[Field]:
117117
return props
118118
# TigerGraph needs explicit scalar defaults for schema definition.
119119
return [
120-
Field(name=f.name, type=FieldType.STRING if f.type is None else f.type)
120+
Field(
121+
name=f.name,
122+
type=FieldType.STRING if f.type is None else f.type,
123+
item_type=f.item_type,
124+
description=f.description,
125+
)
121126
for f in props
122127
]
123128

0 commit comments

Comments
 (0)