retail_planning: align to predictive skills PR #21 + add registry split

v1/retail_planning/README.md

@@ -18,9 +18,11 @@ tags:
 
 ## What this template is for
 
-Retailers face interconnected decisions: which items will sell, which customers are at risk of leaving, what discounts to offer, and how much inventory to stock. Traditionally these are solved in isolation -- demand forecasting in one silo, pricing optimization in another, supply planning in a third. This template shows how to unify them in a single predict-then-optimize pipeline using RelationalAI.
+Retailers face interconnected decisions: which items will sell, which customers are at risk of leaving, what discounts to offer, and how much inventory to stock. Traditionally these are solved in isolation -- demand forecasting in one silo, pricing optimization in another, supply planning in a third. This template shows how to unify them in a single **predict-then-optimize** pipeline using RelationalAI.
 
-Three GNN models learn directly from the H&M transaction graph: one predicts article-level sales, another predicts customer churn, and a third predicts which articles each customer will purchase. All three predictions are aggregated into adjusted demand estimates -- churn risk discounts demand while purchase propensity uplifts it -- that feed two downstream optimization problems: a markdown optimizer that chooses weekly discount schedules to maximize revenue, and a demand planner that sets production quantities to minimize cost. The entire pipeline runs on one semantic model, with GNN outputs flowing seamlessly into prescriptive constraints and objectives.
+**Start with `retail_planning_local.py`** -- it trains a real sales-regression GNN on a bundled H&M subset (CPU, no external data), aggregates predictions per article, and runs both optimizers. A few minutes end-to-end. It's the quickest way to see the whole pattern working.
+
+**Then adapt the pattern to your own Snowflake data** using `retail_planning.py` as a reference. It trains three GNNs (sales regression, customer-churn classification, user-article link prediction) against the full Kaggle H&M dataset in Snowflake, aggregates all three signals into an adjusted demand estimate, and feeds that into the same two optimizers. The H&M pipeline is the worked example -- the structure (graph concepts → GNN tasks → aggregation bridge → prescriptive constraints) is what carries over to your own retail, pricing, or demand-planning data.
 
 ## Who this is for
 
@@ -41,20 +43,34 @@ Assumes familiarity with Python, basic ML concepts (classification, regression,
 
 ## What's included
 
+- **Runners**:
+  - `retail_planning_local.py` -- **primary, runnable out of the box.** Trains a sales-regression GNN on the bundled HM_MINI subset and solves both optimizers.
+  - `retail_planning.py` -- **reference pattern** for adapting the same pipeline to your own Snowflake data. Trains three GNNs (sales, churn, purchase) against a full H&M dataset in Snowflake.
 - **Model**: Three GNN tasks on the H&M knowledge graph (Customer, Article, Transaction), two prescriptive problems consuming their output
-- **Runner**: `retail_planning.py` -- single script executing the full pipeline
-- **Sample data**: CSV files for optimization parameters (discounts, weeks, article inventory, production capacity)
+- **Sample data**:
+  - `data/hm_mini/` -- bundled H&M subset (~10K customers / 5K articles / 9.6K transactions) with sales task splits. This is what the local runner trains on.
+  - `data/*.csv` -- optimizer parameters: discounts, weeks, article inventory, production capacity.
 - **Outputs**: GNN evaluation metrics, optimal discount schedules, production plans, cost/revenue summaries
 
 ## Prerequisites
 
 ### Access
 
-- A Snowflake account with the RAI Native App installed
-- H&M dataset loaded in Snowflake (from [RelBench](https://relbench.stanford.edu/datasets/rel-hm/)):
-  - Core tables: `CUSTOMERS`, `ARTICLES`, `TRANSACTIONS`
-  - Task tables: churn (`TRAIN`, `VAL`, `TEST`), sales (`TRAIN`, `VAL`, `TEST`), purchase (`TRAIN_EXPLODED`, `VALIDATION_EXPLODED`, `TEST_EXPLODED`)
-- A GPU-enabled engine for GNN training
+**To run the local demo (`retail_planning_local.py`)** you need any Snowflake
+account with the RAI Native App. No H&M Snowflake data, no GPU. The bundled
+`data/hm_mini/` CSVs ship with the template; the sales-regression GNN trains
+on CPU in a few minutes.
+
+**To adapt to your own Snowflake pipeline (`retail_planning.py` as reference)**
+you'll additionally need:
+
+- A dataset in Snowflake analogous to the H&M schema -- customer, item, and
+  transaction tables, plus pre-built train/val/test split tables for whatever
+  predictive tasks you need. The Kaggle [H&M Personalized Fashion
+  Recommendations](https://www.kaggle.com/competitions/h-and-m-personalized-fashion-recommendations/data)
+  dataset (with [RelBench rel-hm](https://relbench.stanford.edu/datasets/rel-hm/)
+  task splits) is the one `retail_planning.py` targets as-shipped.
+- A GPU-enabled RAI engine for GNN training at scale.
 
 ### Tools
 
@@ -89,55 +105,79 @@ Assumes familiarity with Python, basic ML concepts (classification, regression,
    rai init
    ```
 
-5. Update Snowflake settings in `retail_planning.py`:
-   ```python
-   DATABASE = "HM_DB"           # your Snowflake database
-   SCHEMA = "HM_SCHEMA"         # schema with core H&M tables
-   TASK_CHURN_SCHEMA = "HM_CHURN"
-   TASK_SALES_SCHEMA = "HM_SALES"
-   TASK_PURCHASE_SCHEMA = "HM_PURCHASE"
+5. Run the local demo on the bundled H&M subset (CPU, a few minutes):
+   ```bash
+   python retail_planning_local.py
    ```
 
-6. Run:
+### Adapting to your own Snowflake data
+
+`retail_planning.py` is a reference for wiring this pattern against a real
+Snowflake dataset (customers, items, transactions + train/val/test task splits
+for the tasks you care about). To adapt it:
+
+1. Point the table references at your data:
+   ```python
+   DATABASE = "YOUR_DB"
+   SCHEMA = "YOUR_SCHEMA"        # schema with core tables (Customer / Item / Transaction)
+   TASK_SALES_SCHEMA = "..."     # schema with sales train/val/test tables
+   TASK_CHURN_SCHEMA = "..."
+   TASK_PURCHASE_SCHEMA = "..."
+   ```
+2. Adjust the PropertyTransformer to match your columns and drop your PKs/FKs.
+3. Run against a GPU-enabled RAI engine:
    ```bash
    python retail_planning.py
    ```
+The as-shipped `retail_planning.py` targets the Kaggle H&M dataset + RelBench
+task splits (see Prerequisites).
 
-7. Expected output (abbreviated):
-   ```text
-   === Item Sales Predictions (sample) ===
-    article_id  predicted_value
-           100            12.45
-          5000             8.73
-    ...
-
-   === Markdown: Selected Discounts by Article-Week ===
-    article  week  discount_pct
-    Rib Top     1           0.0
-    Rib Top     2          10.0
-    ...
-
-   Pipeline Complete
-   Predictive: 3 GNN models trained (item-sales, user-churn, user-item-purchase)
-   Prescriptive A (Markdown): discount schedule optimized for revenue
-   Prescriptive B (Demand Planning): production plan optimized for cost
-   ```
+### Expected output (local run, abbreviated)
+
+```text
+=== Sales target profile (train split) ===
+  n=7648  min=0.0004237  max=0.5915  mean=0.0286  stddev=0.02121
+
+=== Adjusted Demand per Article (from sales GNN, aggregated) ===
+  article_id                      name  adjusted_demand
+          74          3p Sneaker Socks        21.95
+       53892  Jade HW Skinny Denim TRS       147.64
+       ...
+
+Markdown Status: OPTIMAL
+Total revenue (sales + salvage): $62,096.89
+
+Demand Planning Status: OPTIMAL
+Total cost (production + holding + unmet penalty): $8,985.53
+```
 
 ## Template structure
 
 ```text
 .
 ├── README.md                    # this file
 ├── pyproject.toml               # dependencies
-├── retail_planning.py           # main runner (full pipeline)
+├── retail_planning_local.py     # primary: real GNN on bundled HM_MINI CSVs + both optimizers
+├── retail_planning.py           # reference pattern: same pipeline against full H&M in Snowflake
 └── data/
     ├── discounts.csv            # discount levels with demand lifts
     ├── weeks.csv                # planning weeks with seasonal multipliers
-    ├── articles_inventory.csv   # article pricing/inventory for markdown
-    └── production_capacity.csv  # production caps/costs for demand planning
+    ├── articles_inventory.csv   # article pricing/inventory (full-pipeline scope)
+    ├── production_capacity.csv  # production caps/costs (full-pipeline scope)
+    └── hm_mini/                 # HM_MINI subset used by retail_planning_local.py
+        ├── customers.csv        #   10K customers from H&M Kaggle
+        ├── articles.csv         #   5K articles
+        ├── transactions.csv     #   9.6K transactions
+        ├── train_sales.csv      #   RelBench sales task: 7.6K train rows
+        ├── val_sales.csv        #   1.1K val rows
+        ├── test_sales.csv       #   806 test rows
+        ├── articles_inventory.csv     # 12-article optimizer scope (real HM_MINI IDs)
+        └── production_capacity.csv    # matching production params
 ```
 
-**Start here**: `retail_planning.py` runs end-to-end.
+**Start here**: `retail_planning_local.py` (CPU, no external setup). Use
+`retail_planning.py` (requires GPU) as the adaptation reference when you wire
+this pattern into your own Snowflake data.
 
 ## Sample data
 
@@ -159,15 +199,20 @@ The H&M core data (customers, articles, transactions) comes from Snowflake, sour
 ### Pipeline stages
 
 ```text
-Snowflake tables
+Customer / Article / Transaction data (Snowflake tables or bundled CSVs)
   → GNN item-sales (regression on Article)
-  → GNN user-churn (classification on Customer)
-  → GNN user-item-purchase (link prediction Customer→Article)
-  → Bridge: adjusted demand per article (churn + purchase propensity)
+  → GNN user-churn (classification on Customer)     [full pipeline only]
+  → GNN user-item-purchase (link prediction)        [full pipeline only]
+  → Bridge: adjusted demand per article
   → Markdown optimization (MILP, maximize revenue)
   → Demand/inventory planning (LP, minimize cost)
 ```
 
+`retail_planning_local.py` trains only the sales GNN (the most demonstrative
+task) on the bundled HM_MINI CSVs — HM_MINI does not ship churn or purchase
+splits. Churn and purchase are omitted from the local aggregation step.
+`retail_planning.py` runs all three GNNs against the full HM_PYREL data.
+
 ### Concepts
 
 **OptArticle** -- Articles in the optimizer's scope, linking GNN predictions to pricing/inventory data.
@@ -214,15 +259,19 @@ Snowflake tables
 
 ### 1. Train GNN models on the H&M knowledge graph
 
-Three separate GNN models are trained using the Graph/Relationship/PropertyTransformer API:
+Three separate GNN models are trained using the Graph / Relationship / PropertyTransformer API. All three share the same graph and feature configuration; only the task relationships and task-type differ:
 
 ```python
 # Item-sales regression
-SalesTrain = Relationship(f"{Article} at {Any:timestamp} has {Any:target}")
+SalesTrain = Relationship(f"{Article} at {Any:timestamp} has {Any:sales}")
 sales_gnn = GNN(
-    graph=sales_graph, pt=sales_pt,
+    exp_database=GNN_EXP_DATABASE, exp_schema=GNN_EXP_SCHEMA,
+    graph=graph, property_transformer=pt,
     train=SalesTrain, validation=SalesVal,
-    task_type="regression", eval_metric="rmse", ...
+    task_type="regression", eval_metric="rmse",
+    has_time_column=True, stream_logs=STREAM_LOGS, seed=SEED,
+    device="cuda", n_epochs=20, train_batch_size=256, lr=0.005, head_layers=2,
+    temporal_strategy="last", max_iters=500,
 )
 sales_gnn.fit()
 Article.sales_predictions = sales_gnn.predictions(domain=SalesTest)
@@ -279,7 +328,6 @@ dp.minimize(prod_cost_total + hold_cost_total + unmet_cost_total)
 
 ### Tune parameters
 
-- **GNN hyperparameters**: `n_epochs`, `lr`, `train_batch_size`, `head_layers` in each GNN constructor. More epochs improve accuracy but increase training time.
 - **Churn discount weight** (`CHURN_DISCOUNT_WEIGHT`): controls how much churn risk reduces demand. 0 = ignore churn, 1 = full reduction.
 - **Purchase propensity weight** (`PURCHASE_PROPENSITY_WEIGHT`): controls how much predicted purchase demand uplifts demand. 0 = ignore, higher = stronger uplift.
 - **Unmet demand penalty** (`UNMET_PENALTY`): higher values force the demand planner to fulfill more demand at the cost of higher production.
@@ -298,7 +346,6 @@ dp.minimize(prod_cost_total + hold_cost_total + unmet_cost_total)
 <summary>GNN training fails or is very slow</summary>
 
 - Ensure a GPU-enabled engine is available. GNN training on CPU is significantly slower.
-- Reduce `n_epochs` or `train_batch_size` for faster iteration during development.
 - Check that the task tables (TRAIN, VAL, TEST) are populated and the foreign keys match the core tables.
 </details>
 
@@ -322,9 +369,38 @@ dp.minimize(prod_cost_total + hold_cost_total + unmet_cost_total)
 
 - Ensure the GNN training completed successfully (check for fit() errors).
 - Verify that the test set tables contain rows and that foreign keys link correctly to the core entity tables.
-- Try increasing `n_epochs` -- very few epochs may not converge.
+- If val-RMSE is at or above `stddev(target)`, the regression model has
+  collapsed to the mean — revisit the PropertyTransformer and task setup.
+</details>
+
+<details>
+<summary><code>has_time_column=True</code> fails validation</summary>
+
+Known limitation flagged in the rai-predictive-training skill: when the concept
+carrying `time_col` (here, `Transaction`) is used only as an edge intermediary,
+validation can fail with "no time column defined in data tables." Workaround:
+set `has_time_column=False` on the affected GNN and remove the `"at"` clause
+from its Relationship templates until the GNN team resolves this.
 </details>
 
+<details>
+<summary>Sales regression R² is low or negative</summary>
+
+R² < 0 early in training is normal — it means the model is doing worse than
+predicting the target mean. See the "Sales target profile (train split)" block
+printed before training: if val-RMSE prints below the target's stddev, the
+GNN is learning signal. If it plateaus at or above the stddev, re-check the
+PropertyTransformer and task setup.
+</details>
+
+<details>
+<summary>Spinner floods the log when running in CI / non-TTY</summary>
+
+Set `STREAM_LOGS = False` at the top of the script (the default). The GNN
+continues training server-side; only the client-side log stream is suppressed.
+</details>
+
+
 <details>
 <summary>rai init fails or connection errors</summary>