release: v0.5.0 grounded AI narratives

CHANGELOG.md

@@ -14,6 +14,39 @@ https://semver.org/
 
 ---
 
+## [0.5.0] - 2026-08-04
+
+### Added
+
+- Optional grounded AI narratives through
+  `edf.generate_agentic_eda_narrative(...)`.
+- Provider-neutral narrative adapter contract and optional OpenAI Responses API
+  adapter, installable with `eazydatafix[openai]`.
+- JSON-ready `AgenticEDANarrative`, `NarrativeClaim`, `NarrativeEvidence`, and
+  `AgenticEDANarrativeConfig` models.
+- Optional inclusion of an already validated narrative in Agentic EDA HTML,
+  JSON, and Markdown reports.
+- Evidence-reference sections in human-readable HTML and Markdown reports.
+
+### Reliability
+
+- Narrative providers receive only compact evidence from a completed
+  deterministic workflow, never the raw dataset.
+- Every generated narrative claim must cite supplied deterministic evidence.
+  Invalid JSON, uncited claims, duplicate citations, unknown citations,
+  invented numbers, unsupported causal language, insufficient lexical support,
+  and over-limit sections fail with explicit errors.
+- Narrative evidence is immutable during provider execution, and each narrative
+  is bound to its exact workflow using a SHA-256 fingerprint.
+- Narrative generation never reruns or modifies the deterministic workflow.
+
+### Limitations
+
+- Deterministic grounding checks are guardrails, not proof of semantic truth;
+  AI-written narratives still require human review.
+
+---
+
 ## [0.4.0] - 2026-08-01
 
 ### Added

README.md

@@ -18,8 +18,9 @@ planning, modular execution, traceable findings, and reproducible reporting.
 The same package also supports data-quality assessment, validation, cleaning,
 preparation, and exploratory data analysis.
 
-> EazyDataFix v0.4.0 adds reproducible notebook export and explicit human
-> approval checkpoints to the deterministic Agentic EDA workflow.
+> EazyDataFix v0.5.0 adds optional evidence-cited AI narratives to completed
+> deterministic Agentic EDA workflows. Deterministic analysis remains the
+> authoritative source of metrics and continues to work without an LLM.
 
 Install with `pip install eazydatafix` ·
 [Documentation](https://eazydatafix.com/docs) ·
@@ -126,8 +127,47 @@ Caller DataFrames are not mutated by the deterministic EDA workflow.
 
 ### AI Optional
 
-v0.4.0 does not require an LLM. Optional grounded narratives are planned for a
-future release.
+The deterministic workflow does not require an LLM. The v0.5.0 release adds
+optional grounded narratives through a provider adapter; existing workflows
+continue to run without an API key or AI dependency.
+
+## Optional grounded AI narrative
+
+Create a business-facing narrative only after deterministic analysis is complete.
+The provider receives an immutable, compact evidence brief, not the raw dataset.
+Every generated statement must cite one or more evidence IDs from that brief.
+EazyDataFix rejects malformed or unknown citations, invented numbers,
+unsupported causal language, and claims without sufficient lexical support in
+their cited evidence. The narrative is bound to the exact workflow by a SHA-256
+fingerprint, so it cannot be exported with a different or modified workflow.
+
+These deterministic checks reduce unsupported output but cannot prove the
+semantic truth of AI-written text. Review the narrative before using it for a
+decision. HTML and Markdown reports include an evidence-reference section for
+that review.
+
+```python
+import eazydatafix as edf
+from eazydatafix.narratives import OpenAINarrativeProvider
+
+workflow = edf.run_agentic_eda("employees.csv")
+provider = OpenAINarrativeProvider(model="your-openai-model")
+
+narrative = edf.generate_agentic_eda_narrative(workflow, provider)
+
+report = edf.export_agentic_eda_report(
+    workflow,
+    output_dir="eda-report",
+    formats=["html", "json", "markdown"],
+    narrative=narrative,
+)
+```
+
+Install the adapter only when needed:
+
+```bash
+pip install "eazydatafix[openai]"
+```
 
 ## Workflow
 
@@ -249,6 +289,7 @@ visualisation recommendations.
 | `edf.resume_agentic_eda(...)` | Resume an approved plan after dataset fingerprint validation. |
 | `edf.export_agentic_eda_report(...)` | Export Agentic EDA reports and recommended visualisations. |
 | `edf.export_agentic_eda_notebook(...)` | Export a reproducible, ready-to-run Jupyter Notebook. |
+| `edf.generate_agentic_eda_narrative(...)` | Generate a cited optional AI narrative from deterministic workflow evidence. |
 | `edf.fix(...)` | Apply the existing configurable dataset-cleaning pipeline. |
 | `edf.prepare(...)` | Prepare types and columns for downstream analysis. |
 | `edf.analysis_ready(...)` | Clean and prepare a dataset in one workflow. |
@@ -270,7 +311,7 @@ Detailed API documentation is maintained on the
 
 ## Project status
 
-- Current stable version: v0.4.0
+- Current stable version: v0.5.0
 - Development status: Beta
 - Python support: 3.10–3.13
 - Licence: MIT
@@ -281,7 +322,7 @@ The public API may continue evolving before v1.0.
 
 - **v0.3.0 — Deterministic Agentic EDA Foundation — Released**
 - **v0.4.0 — Notebook Export and Human Approval — Released**
-- **v0.5.0 — Optional Grounded AI Narratives — Planned**
+- **v0.5.0 — Optional Grounded AI Narratives — Released**
 - **v1.0.0 — Stable Production API — Goal**
 
 See the [full roadmap](ROADMAP.md) for milestone details.

RELEASE_NOTES.md

@@ -1,99 +1,76 @@
-# EazyDataFix 0.4.0 Release Notes
+# EazyDataFix 0.5.0 Release Notes
 
-EazyDataFix 0.4.0 adds reproducible Jupyter Notebook export and explicit human
-approval checkpoints to the deterministic Agentic EDA workflow. These features
-remain LLM-free, preserve existing public APIs, and support Python 3.10–3.13.
+EazyDataFix 0.5.0 adds optional, evidence-cited AI narratives to completed
+deterministic Agentic EDA workflows. Calculated metrics remain authoritative,
+existing non-AI workflows remain unchanged, and no API key or AI dependency is
+required unless the narrative feature is used.
 
-## Deterministic Jupyter Notebook export
+## Optional grounded AI narratives
 
-`edf.export_agentic_eda_notebook(...)` converts an existing
-`AgenticEDAResult` into an unexecuted, ready-to-run notebook-format v4 file.
-The notebook contains stable Markdown and executable code cells for dataset
-loading, understanding, planning, execution, orchestration, findings,
-recommendations, unresolved questions, and report export.
-
-Notebook documents are generated as deterministic JSON using the Python
-standard library, so Jupyter and `nbformat` are not required runtime
-dependencies. File-based datasets use portable path references. DataFrame
-inputs generate a deterministic JSON companion dataset beside the notebook,
-allowing the exported notebook to execute independently without mutating the
-caller-owned DataFrame.
+`edf.generate_agentic_eda_narrative(...)` converts a completed
+`AgenticEDAResult` into a concise, business-facing narrative. Providers receive
+an immutable compact evidence brief rather than the raw dataset, and every
+claim must cite evidence IDs supplied by EazyDataFix.
 
 ```python
 import eazydatafix as edf
+from eazydatafix.narratives import OpenAINarrativeProvider
 
 workflow = edf.run_agentic_eda("employees.csv")
-notebook = edf.export_agentic_eda_notebook(
+provider = OpenAINarrativeProvider(model="your-openai-model")
+
+narrative = edf.generate_agentic_eda_narrative(workflow, provider)
+
+report = edf.export_agentic_eda_report(
     workflow,
-    dataset="employees.csv",
-    output_path="agentic-eda.ipynb",
+    output_dir="eda-report",
+    formats=["html", "json", "markdown"],
+    narrative=narrative,
 )
-
-print(notebook.generated_files)
 ```
 
-The JSON-ready `AgenticEDANotebookResult` reports the notebook path, generated
-companion files, cell count, notebook format version, and export status.
-
-## Human Approval Checkpoints
+The narrative layer is provider-neutral. The built-in OpenAI Responses API
+adapter is optional and installed through the `openai` extra.
 
-The new two-phase approval workflow separates deterministic understanding and
-planning from analysis execution:
+## Grounding and integrity guardrails
 
-- `edf.prepare_agentic_eda_approval(...)` creates a pending checkpoint without
-  executing analysis steps.
-- `edf.approve_agentic_eda_plan(...)` approves all selected steps or an
-  explicit subset.
-- `edf.reject_agentic_eda_plan(...)` records an explicit rejection that cannot
-  be resumed.
-- `edf.resume_agentic_eda(...)` executes only an approved plan and returns the
-  existing `AgenticEDAResult` type.
-
-```python
-checkpoint = edf.prepare_agentic_eda_approval("employees.csv")
-
-# Review checkpoint.eda_result and checkpoint.eda_plan.
-approved = edf.approve_agentic_eda_plan(
-    checkpoint,
-    approved_step_ids=None,
-    reviewer="Data owner",
-    notes="Approved for deterministic execution",
-)
-
-workflow = edf.resume_agentic_eda("employees.csv", approved)
-```
+- Every claim must cite supplied deterministic evidence.
+- Unknown, duplicate, missing, or malformed citations are rejected.
+- Invented numbers, unsupported causal language, and insufficient lexical
+  support are rejected before a narrative result is returned.
+- Evidence is immutable while a provider runs.
+- A SHA-256 workflow fingerprint prevents a narrative generated for one
+  workflow from being attached to another.
+- HTML and Markdown reports include an evidence-reference section for human
+  review.
 
-Subset approval is constrained to steps selected by the original planner.
-Unknown, duplicate, skipped, and unplanned IDs fail clearly. Dependencies must
-be approved explicitly; incomplete subsets fail before execution and no
-additional steps are approved implicitly.
+These checks are deterministic guardrails, not proof that AI-written text is
+semantically true. Narratives should still be reviewed before they are used for
+decisions.
 
-Each frozen, JSON-ready `AgenticEDAApprovalCheckpoint` contains copied dataset
-understanding, the original plan, configuration, ordered decisions, reviewer
-metadata, and deterministic summaries. SHA-256 fingerprints protect the
-dataset and complete checkpoint decision state. Resume rejects changed
-datasets, modified snapshots, approval-field tampering, and pending or rejected
-checkpoints while reusing the stored understanding and approved plan.
+## Compatibility
 
-The existing `edf.run_agentic_eda(...)` one-call workflow remains unchanged for
-users who do not need an approval gate.
+- Python 3.10, 3.11, 3.12, and 3.13 are supported.
+- No existing public APIs were removed or renamed.
+- Deterministic EDA, reporting, notebook export, and approval checkpoints do
+  not require an LLM or API key.
+- Narrative generation does not rerun or modify the deterministic workflow.
+- Caller-owned DataFrames remain unmodified.
 
 ## Install
 
 ```bash
-pip install eazydatafix==0.4.0
+pip install eazydatafix==0.5.0
 ```
 
-Parquet support remains optional:
+Install the optional OpenAI adapter with:
 
 ```bash
-pip install "eazydatafix[parquet]==0.4.0"
+pip install "eazydatafix[openai]==0.5.0"
 ```
 
-## Compatibility
+Parquet support remains optional:
 
-- Python 3.10, 3.11, 3.12, and 3.13 are supported.
-- No existing public APIs were removed or renamed.
-- Notebook and approval workflows preserve caller-owned DataFrames.
-- Notebook export remains compatible with existing deterministic report
-  export.
+```bash
+pip install "eazydatafix[parquet]==0.5.0"
+```

ROADMAP.md

@@ -39,8 +39,8 @@
 - [x] Full-plan approval, subset approval, and explicit rejection
 - [x] Explicit dependency-completeness validation for subset approvals
 
-### Next Milestone: Grounded Presentation
-- [ ] Optional LLM narratives grounded exclusively in deterministic metrics
+### Milestone 7: Grounded Presentation — Released in v0.5.0
+- [x] Optional LLM narratives grounded exclusively in deterministic metrics
 
 ## Version 0.1
 

architecture/04_Public_API.md

@@ -103,6 +103,29 @@ HTML and JSON are default formats. Markdown is optional. The dataset argument
 is optional and is used only for chart types that honestly require raw
 observations.
 
+## Optional Grounded AI Narrative
+
+Generate a presentation only after the deterministic workflow is complete:
+
+```python
+from eazydatafix.narratives import OpenAINarrativeProvider
+
+provider = OpenAINarrativeProvider(model="your-openai-model")
+narrative = edf.generate_agentic_eda_narrative(workflow, provider)
+```
+
+Providers receive only an immutable compact deterministic evidence brief. Each
+generated claim must cite supplied evidence IDs. Citation, numeric,
+causal-language, and lexical-support validation rejects common unsupported
+outputs. A SHA-256 workflow fingerprint prevents a narrative from being
+attached to a different or modified workflow. These checks do not prove
+semantic truth, so AI-written text still requires human review. Pass the
+resulting object to
+`edf.export_agentic_eda_report(..., narrative=narrative)` to include it in HTML,
+JSON, and Markdown reports; human-readable formats include the cited evidence
+details. Install the OpenAI adapter only when required with `pip install
+"eazydatafix[openai]"`.
+
 ## Agentic EDA Notebook Export
 
 ```python
@@ -147,6 +170,8 @@ All supported sources route through the shared datasource loading system.
 - `AgenticEDAConfig`
 - `AgenticEDAApprovalCheckpoint`
 - `AgenticEDANotebookResult`
+- `AgenticEDANarrative` and `AgenticEDANarrativeConfig`
+- `NarrativeClaim` and `NarrativeEvidence`
 - `AgenticEDAResult`
 - `AgenticEDAReportResult`
 - `EDAResult`

architecture/06_Agentic_EDA_Roadmap.md

@@ -1,7 +1,8 @@
 # Agentic EDA Roadmap
 
-Deterministic notebook export and human approval checkpoints are delivered in
-EazyDataFix 0.4.0. The next milestone remains optional grounded presentation.
+Deterministic notebook export and human approval checkpoints were delivered in
+EazyDataFix 0.4.0. EazyDataFix 0.5.0 adds optional evidence-cited presentation
+grounded in completed deterministic workflows.
 
 ## Foundation: Deterministic EDA
 
@@ -90,13 +91,26 @@ follow-up decision pipeline.
 The original `eazydatafix.run_agentic_eda(...)` one-call workflow remains
 unchanged for callers that do not require an approval gate.
 
-## Next Milestone: Grounded Presentation
-
-The next phase will build optional presentation capabilities on deterministic
-workflow outputs without changing calculated metrics.
-
-## Future Milestone: LLM Narratives
-
-LLM-generated explanations will be optional and will use the deterministic EDA
-result as their source of truth. Generated text will not replace calculated
-metrics.
+## Milestone 8: Grounded Presentation
+
+`eazydatafix.generate_agentic_eda_narrative(...)` builds an optional
+business-facing presentation from a completed deterministic workflow without
+changing calculated metrics. Providers receive only a compact deterministic
+evidence brief, never the raw dataset. Every generated claim must cite supplied
+evidence IDs; malformed, uncited, duplicate, unknown, numerically unsupported,
+causally unsupported, or lexically unanchored claims fail before a narrative
+result is returned. Evidence is immutable during provider execution. A SHA-256
+fingerprint binds the narrative to the exact workflow, and HTML and Markdown
+reports expose cited evidence details. These deterministic checks are
+guardrails, not proof of semantic truth, so AI-written text requires review.
+The resulting JSON-ready narrative can be included in HTML, JSON, and Markdown
+Agentic EDA reports.
+
+The built-in OpenAI Responses API adapter is optional and installed through
+`eazydatafix[openai]`. No API key or AI dependency is required for deterministic
+EDA, reports, notebooks, or approval checkpoints.
+
+## Future Hardening
+
+Future releases may add provider-independent semantic verification while
+keeping generated text optional and calculated metrics authoritative.