diff --git a/CHANGELOG.md b/CHANGELOG.md index 95e18093..d337804e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,26 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [1.10.1] + +### Added + +- **Meta-ontology `1.1.0`** (`graflo/rdf/ontology/graflo.ttl`) — `gf:assigned` (datatype, boolean), `gf:hasHashIdentity` (object property reusing `gf:Identity`), `gf:SecondaryIdentity` (class) and `gf:hasSecondaryIdentity`. `gf:hasIdentity`'s domain widens to include `gf:SecondaryIdentity`, so a secondary identity's field list reuses the existing identity-chain vocabulary rather than adding a parallel one. Mirrored in `graflo-context.jsonld` and `rdf/namespace.py`; `docs/assets/graflo-ontology-viz/` regenerated. +- **Resource sampling** — a connector-driven sampling primitive, the shared input stage for schema inference (algorithmic or agentic). Previously `infer_manifest` performed this privately for PostgreSQL and nothing else could reach it. +- **`architecture.onto_sample`** (new, L2 leaf models) — `ResourceSample` holds documents **verbatim as JSON**: flat rows for tables, arbitrarily nested for APIs, nothing flattened at the boundary. `connector` records where the documents came from, so the relation that later becomes a resource plus its `resource_connector` binding survives the round trip. `SourceSample.samples_by_resource` returns `dict[str, list[dict]]` — the shape cross-resource identity inference consumes, so no adapter is needed. `ForeignKeyHint` carries *declared* references, which are ground truth for edge inference rather than a name-suffix guess. +- **Sampling and profiling are separate operations.** `ResourceSampler` (`hq/sampler.py`) pulls documents; `profile_sample` derives the description. `ResourceProfile` is **path-keyed** (`address.city`, `items[].sku`) with a `FieldType`, null rate, cardinality and nesting depth, so hierarchical and tabular sources take one code path — a flat column list cannot represent a nested response at all. `ResourceProfile.flat_docs` projects nested documents into flat records, which is what makes `IdentityInferencer` (flat-records only) usable on an API source. +- **`GraphEngine.sample_resources(...)`** — accepts a `PostgresConfig`, a `Bindings` block, or a file/directory path. Bindings-driven sampling reuses `RegistryBuilder.discover_files` and the `resource_connector` mapping, so provenance comes for free instead of being reconstructed. PostgreSQL sampling carries `primary_key` and `foreign_keys` through from introspection. Documents are normalized to JSON-safe values (`datetime`, `Decimal`, `memoryview`, numpy scalars), capped by `max_docs` and `max_cell_chars`, with `truncated` set when anything was dropped or clipped. +- **`PostgresConfig.from_dsn(dsn)`** — a bare `uri=` derives only host and port, leaving `username`/`password`/`database` unset, so `to_sqlalchemy_connection_string()` raised. Parses credentials, database and a `search_path` from either `?schema=` or libpq `?options=-csearch_path=`. +- Concept page: [Sampling and profiling](docs/concepts/schema/sampling_and_profiling.md), cross-linked from the identity inference guide. + +### Fixed + +- **RDF round-trip no longer silently degrades a vertex's identity mode** (`CORE-RDF-001`). The serializer wrote only `gf:blank` and `gf:identityName`, so `assigned`, `hash_identity_properties` and `secondary_identities` were dropped and every vertex read back as `natural` — a wrong-but-valid schema, which is the worst failure shape for a documented round-trip format. All four identity modes (`natural`, `hash`, `blank`, `assigned`) now survive, and `examples/16-secondary-identities` round-trips to canonical equality. +- **Identity field order survives.** Identity nodes are `BNode`s and RDF triples are unordered, so a multi-field `identity` could come back permuted. They now carry `gf:artifactIndex` and are read through `_ordered_nodes`; graphs written by the previous serializer still parse (a missing index degrades to arbitrary order rather than failing). +- **`strict_references=True` now rejects a pipeline `vertex:` step naming an undeclared vertex.** `filter_vertex_config_for_resource` intersects a resource's vertex names with the schema's and silently drops unknowns, so a resource that ingested nothing validated clean — a name mismatch between the vertex definition and the step was invisible. **Behaviour change:** manifests that previously passed under `strict_references=True` may now fail, which is the point; lenient validation is unchanged. +- `ChunkerFactory._guess_chunker_type` raises the documented `ValueError` for a file with no extension instead of `IndexError`, so callers scanning a directory can skip it like any other unknown type. +- The CSV/TSV chunker no longer raises `RuntimeError: generator raised StopIteration` (PEP 479) on an empty file; it yields no rows. + ## [1.10.0] diff --git a/docs/assets/graflo-ontology-viz/embed.html b/docs/assets/graflo-ontology-viz/embed.html index da83e1b4..7c27f656 100644 --- a/docs/assets/graflo-ontology-viz/embed.html +++ b/docs/assets/graflo-ontology-viz/embed.html @@ -2,7 +2,7 @@
-