DanteBurbano27 · Sep 21, 2026 · Sep 21, 2026 · Sep 21, 2026
diff --git a/.env.example b/.env.example
@@ -1,4 +1,5 @@
 # GitHub
+# Optional for public repositories; recommended for higher API rate limits.
 GITHUB_TOKEN=
 GITHUB_API_BASE_URL=https://api.github.com
 GITHUB_API_VERSION=2026-03-10
@@ -13,4 +14,4 @@ BIGQUERY_LOCATION=US
 
 # Application
 DEVFLOW_ENV=development
-LOG_LEVEL=INFO
+LOG_LEVEL=INFO
diff --git a/README.md b/README.md
diff --git a/config/repositories.evidence.json b/config/repositories.evidence.json
@@ -0,0 +1,8 @@
+{
+  "repositories": [
+    {
+      "owner": "DanteBurbano27",
+      "name": "devflow-engineering-analytics"
+    }
+  ]
+}
diff --git a/docs/analytics/analytics_architecture.md b/docs/analytics/analytics_architecture.md
@@ -2,7 +2,7 @@
 
 ## 1. Purpose and Overview
 
-The **Analytics Layer** in DevFlow Intelligence serves as the analytical contract, quality assurance gate, and derived metrics engine bridging raw GitHub metadata ingestion and downstream cloud data warehousing (Google BigQuery) and consumption (Looker Studio - Planned V2 / APIs).
+The **Analytics Layer** in DevFlow Intelligence is the implemented analytical contract, quality gate, and derived-metrics engine for GitHub metadata. BigQuery SQL and Looker Studio appear here as target-state consumers: the SQL is static-tested, while no BigQuery dataset or Looker Studio dashboard is deployed.
 
 It transforms point-in-time operational metadata into structured, reproducible analytical records with derived activity indicators, technical ratios, and fleet aggregations.
 
@@ -53,7 +53,7 @@ flowchart TD
 1. **Explicit Data Contracts**: Upstream API changes or schema drift cannot silently propagate. Records are validated against an immutable analytical contract schema (`RepositoryContract`).
 2. **Non-Destructive Quality Gates**: The `DataQualityEngine` evaluates data integrity without mutating incoming records. Errors halt or isolate tainted records; warnings flag anomalies.
 3. **Reproducibility & Determinism**: All temporal calculations (e.g. `days_since_last_push`) evaluate relative to `extracted_at` unless explicitly overridden, eliminating non-deterministic temporal drift across execution environments.
-4. **Separation of Concerns**: Python implements contract enforcement, programmatic validation, and rapid local summary analytics. BigQuery SQL models implement warehouse-scale transformations, incremental materializations, and partitioning/clustering.
+4. **Separation of Concerns**: Python implements contract enforcement, programmatic validation, and local summary analytics. The BigQuery SQL files design warehouse transformations, incremental materializations, and partitioning/clustering; live execution remains unverified.
 5. **Zero External Framework Overhead**: The analytics engine utilizes standard Python library primitives (`dataclasses`, `datetime`, `enum`, `statistics`, `typing`), maximizing execution speed and eliminating brittle framework dependencies.
 
 ---

diff --git a/docs/analytics/warehouse_models.md b/docs/analytics/warehouse_models.md
@@ -2,7 +2,7 @@
 
 ## 1. Overview and Architecture
 
-DevFlow Intelligence transforms raw metadata into structured analytical layers inside **Google BigQuery**. The models are located under `analytics/sql/` and follow modern analytics engineering conventions.
+This document describes the intended **Google BigQuery** warehouse shape encoded by the SQL files under `analytics/sql/`. The models are static-tested for contract shape and critical BigQuery-dialect statements; they have not been compiled or executed against a provisioned BigQuery dataset.
 
 ```
                     devflow_raw.raw_repositories
@@ -85,4 +85,4 @@ devflow_analytics.dim_repositories   devflow_analytics.fct_repository_snapshots
 
 1. **Partition Pruning**: Always filter analytical queries on `snapshot_date` when querying `fct_repository_snapshots` to minimize BigQuery slot usage and query costs.
 2. **Clustering Efficiency**: Clustering on `(repository_id, language)` ensures that multi-repository filters avoid scanning irrelevant columnar blocks.
-3. **Idempotent Loads**: Deduplication via `QUALIFY ROW_NUMBER() ...` guarantees idempotency during pipeline backfills or replay runs.
+3. **Designed Replay Behavior**: Deduplication via `QUALIFY ROW_NUMBER() ...` is intended to make backfills and replay runs idempotent. This remains unverified in a live BigQuery dataset.
diff --git a/docs/architecture/system_architecture.md b/docs/architecture/system_architecture.md
@@ -1,8 +1,11 @@
 # DevFlow Intelligence — System Architecture
 
+> [!IMPORTANT]
+> This document is the target-state architecture. The implemented V1 boundary is GitHub repository metadata -> local raw/normalized JSONL -> contract and 16-rule quality gate -> local analytics report and manifest. BigQuery is represented by a mock-tested adapter and static-tested SQL; GH Archive, dbt, Airflow, deployed BigQuery datasets, and Looker Studio are not implemented or deployed.
+
 ## 1. Architecture purpose
 
-This document describes how DevFlow Intelligence collects, stores, transforms, validates and exposes public GitHub engineering activity.
+This document describes the intended architecture for collecting, storing, transforming, validating, and exposing public GitHub engineering activity.
 
 The architecture is designed as a daily batch pipeline focused on:
 
@@ -15,7 +18,7 @@ The architecture is designed as a daily batch pipeline focused on:
 
 ---
 
-## 2. High-level architecture
+## 2. Target-state architecture
 
 ```mermaid
 flowchart LR

diff --git a/docs/execution_evidence.md b/docs/execution_evidence.md
@@ -0,0 +1,100 @@
+# Verified local execution
+
+This evidence was captured from an actual anonymous GitHub REST API run on 2026-09-21 UTC. It exercises the implemented V1 path from API request through raw and normalized persistence, contract validation, all 16 quality rules, metric calculation, analytics publication, and manifest enrichment. It does not exercise BigQuery or Looker Studio.
+
+## Command
+
+```powershell
+$env:GITHUB_TOKEN = ""
+python -m scripts.run_devflow `
+  --config config/repositories.evidence.json `
+  --output-root data
+```
+
+## Safe configuration
+
+```json
+{
+  "repositories": [
+    {
+      "owner": "DanteBurbano27",
+      "name": "devflow-engineering-analytics"
+    }
+  ]
+}
+```
+
+## Sanitized console result
+
+The request returned HTTP 200 on its first attempt. The terminal result was:
+
+```json
+{
+  "status": "success",
+  "exit_code": 0,
+  "repositories_succeeded": 1,
+  "repositories_failed": 0,
+  "stages": [
+    {"name": "configuration", "status": "success"},
+    {"name": "client", "status": "success"},
+    {"name": "service", "status": "success"},
+    {"name": "batch", "status": "success"},
+    {"name": "quality", "status": "success"},
+    {"name": "analytics", "status": "success"}
+  ],
+  "error_type": null,
+  "error_message": null
+}
+```
+
+The run ID and timestamps are omitted because they are generated for every execution. No token, request header, or raw API payload is included in this document.
+
+## Generated file structure
+
+```text
+data/
+├── raw/github/repositories/extraction_date=2026-09-21/run_id=<UTC_ID>/repositories.jsonl
+├── normalized/github/repositories/extraction_date=2026-09-21/run_id=<UTC_ID>/repositories.jsonl
+├── analytics/github/repositories/extraction_date=2026-09-21/run_id=<UTC_ID>/report.json
+└── manifests/github/repositories/extraction_date=2026-09-21/run_id=<UTC_ID>/manifest.json
+```
+
+## Representative report excerpt
+
+```json
+{
+  "meta": {
+    "total_input_records": 1,
+    "processed_records": 1
+  },
+  "quality": {
+    "summary": {
+      "is_valid": true,
+      "has_errors": false,
+      "has_warnings": false,
+      "total_records": 1,
+      "passed_records": 1,
+      "failed_records": 0,
+      "rules_evaluated": 16,
+      "error_count": 0,
+      "warning_count": 0
+    }
+  },
+  "portfolio": {
+    "summary_metrics": {
+      "total_repositories": 1
+    },
+    "activity_health": {
+      "active_count": 1
+    },
+    "languages": [
+      {
+        "language": "Python",
+        "repository_count": 1
+      }
+    ]
+  }
+}
+```
+
+The full generated files remain local and ignored because GitHub API responses are mutable snapshots. This stable excerpt proves the stage outcomes and output contract without presenting volatile repository counters as permanent portfolio results.
diff --git a/docs/integration_acceptance_checklist.md b/docs/integration_acceptance_checklist.md
@@ -126,7 +126,7 @@ The analytics layer accepts either `ingestion.github.repository_metadata.Reposit
 
 ## 6. BigQuery SQL Warehouse Models
 
-All SQL models in `analytics/sql/` are validated for Google Cloud BigQuery compatibility:
+The SQL models in `analytics/sql/` are static-tested for expected schema references and selected BigQuery-dialect statements. They have not been compiled or executed by BigQuery:
 
 1. **`stg_github_repositories.sql`** (Staging):
    - Type casting and null coalescing.
@@ -150,16 +150,16 @@ All SQL models in `analytics/sql/` are validated for Google Cloud BigQuery compa
 
 - [x] **Test Isolation**: 100% offline, zero network requests, zero GitHub tokens, zero cloud credentials.
 - [x] **Synthetic Fixtures**: All tests run with deterministic synthetic data fixtures.
-- [x] **Pytest Pass**: All 109+ tests pass in $< 1.0\text{s}$.
+- [x] **Pytest Pass**: All 155 collected tests pass locally on Python 3.12.
 - [x] **Linter Compliance**: `ruff check .` passes with zero errors or warnings.
 - [x] **Formatting Compliance**: `ruff format --check .` passes with line length $\le 88$.
 
 ---
 
 ## 8. Security & Environment Governance
 
-- [x] **Zero Hardcoded Secrets**: No API tokens, passwords, or GCP service account keys in codebase.
-- [x] **Environment Variable Ingestion**: Relies strictly on `GITHUB_TOKEN` from environment.
+- [x] **Credential Scan**: No known API tokens, passwords, or GCP service-account keys are tracked.
+- [x] **Environment Variable Ingestion**: `GITHUB_TOKEN` is optional for public repositories and is read from the environment when authenticated rate limits are needed.
 - [x] **Standard Library Core**: Analytics engine requires zero external packages beyond standard Python library, minimizing supply chain vulnerabilities.
 
 ---

diff --git a/docs/production_readiness.md b/docs/production_readiness.md
@@ -1,7 +1,7 @@
-# Production Readiness Audit
+# Implementation Readiness Audit
 
 ## Audit Summary
-This audit certifies the status of the DevFlow Intelligence pipeline components handled by the Antigravity Analytics team.
+This audit records the test-backed implementation status of DevFlow Intelligence. It is not a production deployment or operational certification.
 
 ### 1. Data Contracts
 - **Status**: **READY**

diff --git a/docs/project_scope.md b/docs/project_scope.md
@@ -40,19 +40,19 @@ DevFlow Intelligence will consolidate this information into structured analytica
 
 ---
 
-## 3. Project objective
+## 3. Target-state objective
 
-Build a reproducible batch data pipeline that:
+The broader roadmap would build a reproducible batch data pipeline that:
 
 1. Extracts public repository activity from GitHub.
 2. Preserves raw source data for traceability.
-3. Loads the data into Google BigQuery.
-4. Cleans and transforms the data using dbt.
+3. Loads the data into Google BigQuery (not deployed in V1).
+4. Cleans and transforms the data using dbt (not implemented in V1).
 5. Creates a basic dimensional model.
 6. Generates software delivery metrics.
 7. Validates data quality.
-8. Automates pipeline execution using Apache Airflow.
-9. Presents the results through an analytical dashboard.
+8. Automates pipeline execution using Apache Airflow (planned V2).
+9. Presents the results through an analytical dashboard (planned V2).
 
 ---
 

diff --git a/docs/release_checklist.md b/docs/release_checklist.md
@@ -1,15 +1,15 @@
 # Release Checklist
 
-## Pre-Release (Antigravity Analytics)
-- [ ] Ensure 100% test coverage using `py -3.14 -m pytest`.
-- [ ] Validate linting rules using `py -3.14 -m ruff check .`.
-- [ ] Confirm code formatting using `py -3.14 -m ruff format --check .`.
+## Pre-Release (Analytics)
+- [ ] Run the complete collected test suite using Python 3.12 or newer: `python -m pytest`.
+- [ ] Validate linting rules using `python -m ruff check .`.
+- [ ] Confirm code formatting using `python -m ruff format --check .`.
 - [ ] Run Integration Simulation (`tests/test_integration_simulation.py`) against Golden Data.
-- [ ] Verify BigQuery SQL scripts compile cleanly.
+- [ ] When a BigQuery project is available, compile and execute SQL against a disposable dataset; V1 tests are static only.
 
 ## Release Execution
-- [ ] Merge `team/antigravity-analytics-quality` to integration branch.
-- [ ] Validate downstream Codex outputs using the simulation constraints.
+- [ ] Review the release diff and CI result on the release PR.
+- [ ] Validate normalized outputs using the simulation constraints.
 
 ## Post-Release
-- [ ] Monitor metric deviation and quality error rates for the first 24 hours.
+- [ ] For a deployed scheduler, define and validate monitoring before enabling recurring runs.
diff --git a/docs/runbook.md b/docs/runbook.md
@@ -1,12 +1,12 @@
 # Operational Runbook
 
 ## Overview
-This runbook covers the standard operating procedures for the Antigravity Analytics plane in DevFlow Intelligence.
+This runbook separates current V1 local operations from target-state warehouse operations.
 
 ## Daily Operations
 - Monitor execution logs for `ContractValidationError`. A spike in these errors indicates an upstream change in the GitHub API or Codex ingestion logic.
-- Verify that `DataQualityEngine` reject rates remain stable (historically < 1%).
-- Check BigQuery Dataform/dbt runs for the models in `analytics/sql/`.
+- Review `DataQualityEngine` errors and warnings against the 16-rule catalog; no historical reject-rate baseline is claimed.
+- **Target state only:** after a BigQuery deployment exists, monitor the jobs that execute models in `analytics/sql/`. V1 has no deployed Dataform/dbt run.
 
 ## Service Level Objectives (SLOs)
 - **TARGET (Phase 3)**: 99.9% of valid records processed within 5 seconds.

diff --git a/ingestion/common/config.py b/ingestion/common/config.py
@@ -18,7 +18,7 @@ class ConfigurationError(RuntimeError):
 class Settings:
     """Runtime settings required by the ingestion application."""
 
-    github_token: str
+    github_token: str | None
     github_api_base_url: str
     github_api_version: str
     github_timeout_seconds: float
@@ -31,12 +31,7 @@ def from_env(cls) -> Settings:
         """Build application settings from environment variables."""
         load_dotenv()
 
-        github_token = os.getenv("GITHUB_TOKEN", "").strip()
-
-        if not github_token:
-            raise ConfigurationError(
-                "GITHUB_TOKEN is missing. Add it to the local .env file."
-            )
+        github_token = os.getenv("GITHUB_TOKEN", "").strip() or None
 
         try:
             github_api_base_url = normalize_https_base_url(

diff --git a/ingestion/common/environment.py b/ingestion/common/environment.py
@@ -3,7 +3,7 @@
 import sys
 from collections.abc import Sequence
 
-MINIMUM_PYTHON_VERSION = (3, 11)
+MINIMUM_PYTHON_VERSION = (3, 12)
 
 
 def is_supported_python(

diff --git a/ingestion/github/client.py b/ingestion/github/client.py
@@ -43,11 +43,11 @@
 
 
 class GitHubClient:
-    """Client responsible for sending authenticated requests to GitHub."""
+    """Client for authenticated or rate-limited public GitHub requests."""
 
     def __init__(
         self,
-        token: str,
+        token: str | None = None,
         base_url: str = "https://api.github.com",
         api_version: str = "2026-03-10",
         timeout_seconds: float = 30.0,
@@ -57,10 +57,7 @@ def __init__(
         sleeper: Sleeper = time.sleep,
     ) -> None:
         """Initialize the GitHub API client."""
-        clean_token = token.strip()
-
-        if not clean_token:
-            raise ValueError("A GitHub token is required.")
+        clean_token = token.strip() if token else None
 
         if timeout_seconds <= 0:
             raise ValueError("timeout_seconds must be greater than zero.")
@@ -78,14 +75,14 @@ def __init__(
         self._session = session or requests.Session()
         self._sleeper = sleeper
 
-        self._session.headers.update(
-            {
-                "Accept": "application/vnd.github+json",
-                "Authorization": f"Bearer {clean_token}",
-                "X-GitHub-Api-Version": api_version,
-                "User-Agent": "devflow-engineering-analytics",
-            }
-        )
+        headers = {
+            "Accept": "application/vnd.github+json",
+            "X-GitHub-Api-Version": api_version,
+            "User-Agent": "devflow-engineering-analytics",
+        }
+        if clean_token:
+            headers["Authorization"] = f"Bearer {clean_token}"
+        self._session.headers.update(headers)
 
     def get(
         self,

diff --git a/requirements.txt b/requirements.txt
@@ -4,4 +4,4 @@ python-dotenv
 google-cloud-bigquery
 google-cloud-storage
 pandas
-pyarrow
+pyarrow
-Original file line number
+Diff line change
@@ Expand Up / @@ -4,4 +4,4 @@ python-dotenv @@
     google-cloud-bigquery
     google-cloud-storage
     pandas
-    pyarrow
+    pyarrow