[codex] Version domain descriptors and provider contracts

docs/architecture/provider-contracts.md

@@ -9,6 +9,10 @@ state, ingress/TLS, or production hardening services.
 
 ## Contract Source Of Truth
 
+Provider contract exports carry `schema_version: "1.0.0"`. Consumers reject
+unknown versions rather than guessing field meanings. This version is
+independent of deployment and component versions.
+
 Terraform is the source of truth for provider contracts. The local, Azure, and
 AWS platform roots normalize explicit contract objects in their `contracts.tf`
 files and validate them with Terraform `check` blocks. Runtime scripts can read
@@ -63,6 +67,22 @@ Product-owned runtime assets use logical aliases. Local and Azure resolve
 AWS resolves them to S3 buckets. Local and Azure resolve `iceberg_catalog` to
 Polaris; AWS resolves it to Glue.
 
+Domain descriptors use `apiVersion: openlakeforge.io/v1alpha1` and
+`kind: Domain`; the machine-readable schema is
+[`docs/schema/domain.schema.json`](../schema/domain.schema.json). Descriptors
+contain logical product and table names only. Provider contracts derive the
+physical catalog/database/schema FQNs at runtime, so changing catalog adapters
+does not require editing business metadata.
+
+### Compatibility and migration
+
+The `v1alpha1` descriptor and provider contract versions are strict. Migrate a
+legacy descriptor by adding the version envelope, removing
+`silver_tables.schema`, `gold_tables.schema`, and physical asset FQNs, then
+validating against the schema before deployment. A future incompatible shape
+must publish a new API/version and migration guide; deployments fail closed
+when either version is unknown.
+
 ## Catalog Contract
 
 The catalog contract describes an Iceberg catalog implementation. The local

docs/schema/domain.schema.json

@@ -0,0 +1,95 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://openlakeforge.io/schemas/domain/v1alpha1",
+  "title": "OpenLakeForge Domain",
+  "type": "object",
+  "additionalProperties": true,
+  "required": ["apiVersion", "kind", "name", "displayName", "description", "status", "data_products"],
+  "properties": {
+    "apiVersion": {"const": "openlakeforge.io/v1alpha1"},
+    "kind": {"const": "Domain"},
+    "name": {"type": "string", "pattern": "^[a-z][a-z0-9_]*$"},
+    "displayName": {"type": "string", "minLength": 1},
+    "description": {"type": "string"},
+    "status": {"type": "string", "minLength": 1},
+    "data_products": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "required": ["id", "name", "displayName", "description", "status"],
+        "properties": {
+          "id": {"type": "string", "minLength": 1},
+          "name": {"type": "string", "minLength": 1},
+          "displayName": {"type": "string", "minLength": 1},
+          "description": {"type": "string"},
+          "status": {"type": "string", "minLength": 1},
+          "asset_prefix": {"type": "string", "minLength": 1},
+          "domain": {"type": "string", "minLength": 1},
+          "domains": {
+            "type": "array",
+            "minItems": 1,
+            "items": {"type": "string", "minLength": 1}
+          },
+          "bronze": {
+            "type": "array",
+            "items": {"$ref": "#/$defs/bronzeEntry"}
+          },
+          "silver_tables": {"$ref": "#/$defs/tableGroup"},
+          "gold_tables": {"$ref": "#/$defs/tableGroup"},
+          "assets": {
+            "type": "array",
+            "items": {
+              "type": "object",
+              "required": ["name"],
+              "properties": {
+                "name": {"type": "string", "minLength": 1},
+                "type": {"const": "table"}
+              },
+              "not": {
+                "anyOf": [
+                  {"required": ["fqn"]},
+                  {"required": ["fullyQualifiedName"]}
+                ]
+              }
+            }
+          }
+        }
+      }
+    }
+  },
+  "$defs": {
+    "tableGroup": {
+      "type": "object",
+      "required": ["tables"],
+      "properties": {
+        "tables": {
+          "type": "array",
+          "items": {
+            "type": "object",
+            "required": ["name"],
+            "properties": {
+              "name": {"type": "string", "minLength": 1},
+              "description": {"type": "string"}
+            },
+            "not": {
+              "anyOf": [
+                {"required": ["fqn"]},
+                {"required": ["fullyQualifiedName"]}
+              ]
+            }
+          }
+        }
+      },
+      "not": {"required": ["schema"]}
+    },
+    "bronzeEntry": {
+      "type": "object",
+      "required": ["name", "path"],
+      "properties": {
+        "name": {"type": "string", "minLength": 1},
+        "path": {"type": "string", "minLength": 1},
+        "description": {"type": "string"}
+      }
+    }
+  }
+}

domains/README.md

@@ -7,7 +7,12 @@ product has side-by-side assets under domain capability folders: raw examples,
 dlt loaders, Floe contracts, dbt projects, Dagster modules, Superset reports,
 tests, and documentation.
 
-`domains/<domain>/domain.yaml` is the single source for domain, data-product,
+`domains/<domain>/domain.yaml` is a versioned (`openlakeforge.io/v1alpha1`,
+`kind: Domain`) single source for domain, data-product,
 Bronze, Silver, Gold, and OpenMetadata table metadata.
 
+Keep this metadata provider-neutral: catalog/database/schema identities are
+derived from the environment provider contract. Validate descriptors against
+`docs/schema/domain.schema.json` before deployment.
+
 The current seed domains are `sales` and `supply_chain`.

domains/sales/domain.yaml

@@ -1,3 +1,5 @@
+apiVersion: openlakeforge.io/v1alpha1
+kind: Domain
 name: sales
 displayName: Sales
 domainType: Source-aligned
@@ -38,7 +40,6 @@ data_products:
         path: s3://lakehouse-bronze/sales/order_revenue/promotions
         description: Raw CSV promotion dimension.
     silver_tables:
-      schema: polaris.lakehouse_dev.sales_order_revenue_silver
       tables:
         - name: orders
           description: Validated sales order headers.
@@ -51,21 +52,13 @@ data_products:
         - name: promotions
           description: Validated promotion dimension.
     gold_tables:
-      schema: polaris.lakehouse_dev.sales_order_revenue_gold
       tables:
         - name: mart_order_revenue_by_day
           description: Daily order revenue, units, and discount amount by region.
         - name: mart_order_revenue_by_channel
           description: Net revenue and discount amount by sales channel and promotion type.
         - name: mart_order_revenue_margin_by_product
           description: Product revenue, cost, and gross margin for fulfilled order lines.
-    assets:
-      - type: table
-        fqn: polaris.lakehouse_dev.sales_order_revenue_gold.mart_order_revenue_by_day
-      - type: table
-        fqn: polaris.lakehouse_dev.sales_order_revenue_gold.mart_order_revenue_by_channel
-      - type: table
-        fqn: polaris.lakehouse_dev.sales_order_revenue_gold.mart_order_revenue_margin_by_product
   - id: customer_health
     name: sales_customer_health
     displayName: Sales Customer Health
@@ -86,7 +79,6 @@ data_products:
         path: s3://lakehouse-bronze/sales/customer_health/nps_responses
         description: Raw CSV NPS responses.
     silver_tables:
-      schema: polaris.lakehouse_dev.sales_customer_health_silver
       tables:
         - name: accounts
           description: Validated account dimension.
@@ -97,18 +89,10 @@ data_products:
         - name: nps_responses
           description: Validated NPS response facts.
     gold_tables:
-      schema: polaris.lakehouse_dev.sales_customer_health_gold
       tables:
         - name: mart_customer_health_score
           description: Account-level health score using subscription, support, and NPS signals.
         - name: mart_churn_risk_by_segment
           description: Churn risk and ARR exposure grouped by account segment and region.
         - name: mart_support_sla_by_customer
           description: Support volume, resolution hours, and SLA rate by customer and priority.
-    assets:
-      - type: table
-        fqn: polaris.lakehouse_dev.sales_customer_health_gold.mart_customer_health_score
-      - type: table
-        fqn: polaris.lakehouse_dev.sales_customer_health_gold.mart_churn_risk_by_segment
-      - type: table
-        fqn: polaris.lakehouse_dev.sales_customer_health_gold.mart_support_sla_by_customer

domains/supply_chain/domain.yaml

@@ -1,3 +1,5 @@
+apiVersion: openlakeforge.io/v1alpha1
+kind: Domain
 name: supply_chain
 displayName: Supply Chain
 domainType: Source-aligned
@@ -41,7 +43,6 @@ data_products:
         path: s3://lakehouse-bronze/supply_chain/inventory_reliability/stockout_events
         description: Raw CSV stockout events.
     silver_tables:
-      schema: polaris.lakehouse_dev.supply_chain_inventory_reliability_silver
       tables:
         - name: warehouses
           description: Validated warehouse dimension.
@@ -56,18 +57,10 @@ data_products:
         - name: stockout_events
           description: Validated stockout event facts.
     gold_tables:
-      schema: polaris.lakehouse_dev.supply_chain_inventory_reliability_gold
       tables:
         - name: mart_inventory_position
           description: Latest inventory position and reorder status by product and warehouse.
         - name: mart_supplier_delivery_reliability
           description: Supplier delivery lateness, fill rate, and on-time performance.
         - name: mart_stockout_risk
           description: Stockout risk by product and warehouse using current inventory and stockout history.
-    assets:
-      - type: table
-        fqn: polaris.lakehouse_dev.supply_chain_inventory_reliability_gold.mart_inventory_position
-      - type: table
-        fqn: polaris.lakehouse_dev.supply_chain_inventory_reliability_gold.mart_supplier_delivery_reliability
-      - type: table
-        fqn: polaris.lakehouse_dev.supply_chain_inventory_reliability_gold.mart_stockout_risk

infra/terraform/environments/aws-poc/contracts.tf

@@ -291,6 +291,7 @@ locals {
   }
 
   provider_contracts = {
+    schema_version      = "1.0.0"
     foundation          = local.foundation_contract
     kubernetes_platform = local.kubernetes_platform_contract
     cluster             = local.kubernetes_platform_contract

infra/terraform/environments/azure-poc/contracts.tf

@@ -300,6 +300,7 @@ locals {
   }
 
   provider_contracts = {
+    schema_version      = "1.0.0"
     foundation          = local.foundation_contract
     kubernetes_platform = local.kubernetes_platform_contract
     cluster             = local.kubernetes_platform_contract

infra/terraform/environments/local/contracts.tf

@@ -284,6 +284,7 @@ locals {
   }
 
   provider_contracts = {
+    schema_version      = "1.0.0"
     foundation          = local.foundation_contract
     kubernetes_platform = local.kubernetes_platform_contract
     cluster             = local.kubernetes_platform_contract

scripts/test/check-contracts.sh

@@ -35,6 +35,21 @@ azure_main_text = azure_main_tf.read_text()
 aws_main_tf = Path("infra/terraform/environments/aws-poc/main.tf")
 aws_main_text = aws_main_tf.read_text()
 
+for descriptor_path in sorted(Path("domains").glob("*/domain.yaml")):
+    descriptor_text = descriptor_path.read_text()
+    if "apiVersion: openlakeforge.io/v1alpha1" not in descriptor_text or "kind: Domain" not in descriptor_text:
+        errors.append(f"{descriptor_path}: unsupported or missing domain descriptor version")
+    if "polaris.lakehouse_dev" in descriptor_text or "fqn:" in descriptor_text:
+        errors.append(f"{descriptor_path}: physical catalog identities must come from provider contracts")
+
+for contracts_path, contracts_body in [
+    (contracts_tf, text),
+    (azure_contracts_tf, azure_text),
+    (aws_contracts_tf, aws_text),
+]:
+    if 'schema_version      = "1.0.0"' not in contracts_body:
+        errors.append(f"{contracts_path}: provider contracts must declare schema_version 1.0.0")
+
 required_contracts = [
     "foundation_contract",
     "kubernetes_platform_contract",

tools/olf/olf/contracts.py

@@ -25,6 +25,12 @@
 from collections.abc import Mapping
 from typing import Any
 
+PROVIDER_CONTRACT_SCHEMA_VERSION = "1.0.0"
+
+
+class ProviderContractError(ValueError):
+    """Raised when Terraform returns an unsupported provider contract version."""
+
 # Kept as literal compact-JSON strings for byte parity with the previous bash
 # defaults.
 _DEFAULT_CATALOG_NAMESPACES_JSON = (
@@ -67,7 +73,15 @@ def load_provider_contracts(terraform_dir: str) -> dict[str, Any] | None:
         contracts = json.loads(result.stdout)
     except json.JSONDecodeError:
         return None
-    return contracts if isinstance(contracts, dict) else None
+    if not isinstance(contracts, dict):
+        return None
+    schema_version = contracts.get("schema_version")
+    if schema_version != PROVIDER_CONTRACT_SCHEMA_VERSION:
+        raise ProviderContractError(
+            f"provider_contracts.schema_version {schema_version!r} is unsupported; "
+            f"expected {PROVIDER_CONTRACT_SCHEMA_VERSION!r}"
+        )
+    return contracts
 
 
 class _Env:
@@ -350,6 +364,12 @@ def build_contract_env(
     env = _Env(base)
     _apply_default_contract_env(env, base)
     if contracts is not None:
+        schema_version = contracts.get("schema_version")
+        if schema_version != PROVIDER_CONTRACT_SCHEMA_VERSION:
+            raise ProviderContractError(
+                f"provider_contracts.schema_version {schema_version!r} is unsupported; "
+                f"expected {PROVIDER_CONTRACT_SCHEMA_VERSION!r}"
+            )
         _apply_provider_contracts(env, contracts)
         _apply_default_contract_env(env, base)