feat(customer-analytics): synthetic causal telco data model

.gitignore

@@ -38,6 +38,9 @@ logs/
 build/
 output/
 
+# Generated synthetic datasets — reproduce from the generator (seeded), don't commit
+tracks/customer-analytics/telco-customer-intelligence/data-model/data/
+
 # Claude agent instructions (internal — this repo is destined to be public)
 CLAUDE.md
 .claude/

README.md

@@ -30,7 +30,7 @@ something under active maintenance.
 |---|---|
 | [`data-governance/`](tracks/data-governance) | Catalog-aware agents: reading metadata, and using it as context before touching data |
 | [`governed-ai/`](tracks/governed-ai) | LLM agents with grounded access to data — SQL, RAG, classification |
-| [`customer-analytics/`](tracks/customer-analytics) | *(empty)* segmentation, churn, next-best-offer, ARPU, incrementality |
+| [`customer-analytics/`](tracks/customer-analytics) | Segmentation, churn, next-best-offer, ARPU, incrementality — on one synthetic, causal, reproducible telco data model |
 | [`data-privacy/`](tracks/data-privacy) | *(empty)* consent, minimisation, subject rights |
 
 ## Projects
@@ -41,6 +41,7 @@ something under active maintenance.
 | [`agent-data-analyst`](tracks/data-governance/agent-data-analyst) | data-governance | Analyses the *data* using the catalog as governance context: who owns the table, did it pass quality, what the codes mean | `reference` · paused | 47 commits · 2026-02 → 2026-06 |
 | [`agent-sql-khipu_ai`](tracks/governed-ai/agent-sql-khipu_ai) | governed-ai | LLM agent with SQL access and a RAG knowledge base, local models | `archived snapshot` | 8 commits · 2025-01 → 2025-11 |
 | [`agent_text_classification`](tracks/governed-ai/agent_text_classification) | governed-ai | Sentiment classification with a local LLM | `archived snapshot` | 8 commits · 2025-02 |
+| [`telco-customer-intelligence`](tracks/customer-analytics/telco-customer-intelligence) | customer-analytics | Synthetic, causal, reproducible telco data model + the analytics cases it feeds (segmentation, churn, NBO, ARPU, incrementality) | `work in progress` | data model built & tested; cases in progress |
 
 **The two agents in `data-governance/` are a progression, not a duplicate.**
 `openmetadata-mcp-agent` talks *to the catalog*. `agent-data-analyst` analyses *the data*,
@@ -91,13 +92,17 @@ database document what they need in their own `.env.example` — never real valu
 
 Stated plainly rather than hidden behind a green badge:
 
-- **No test suite yet** in any track. CI runs the linter; the pytest job activates itself
-  the moment a `tracks/<track>/<project>/tests/` directory appears. The `test_*.py` files
+- **Tests exist only for `telco-customer-intelligence`** so far (schema, referential
+  integrity, no-leakage, causal signal — standard-library only, so CI runs them without
+  installing anything). The migrated agent tracks still have none; CI's pytest job collects
+  every `tracks/<track>/<project>/tests/` directory as it appears. The `test_*.py` files
   under some `scripts/` folders are **manual integration scripts** — they need a live
   catalog, a database and credentials — so CI deliberately does not collect them.
-- `customer-analytics/` and `data-privacy/` are empty.
-- Dependencies in the migrated projects are declared with ranges, not pinned. A fresh
-  install today may not resolve to what the author ran.
+- `data-privacy/` is empty; `customer-analytics/` has its data model built and its cases
+  in progress (see [#64](https://github.com/ronaldmego/site-ronaldmego/issues/64)).
+- Dependencies in the migrated agent projects are declared with ranges, not pinned. A fresh
+  install today may not resolve to what the author ran. (The customer-analytics data model
+  has no third-party dependencies — standard library only.)
 
 ## License
 

tracks/customer-analytics/telco-customer-intelligence/README.md

@@ -0,0 +1,48 @@
+# Telco Customer Intelligence
+
+Customer-analytics evidence on a single **synthetic, causal, reproducible**
+telco dataset: one data model, feeding a season of cases — segmentation, churn,
+next-best-offer, ARPU and campaign incrementality.
+
+> **Maturity: `work in progress`.** The data model (below) is built, tested and
+> reproducible. The five cases are being added on top of it, one at a time.
+> Everything here is synthetic — no employer or customer data. See
+> [`data-model/DATA_CARD.md`](data-model/DATA_CARD.md).
+
+## Why this track
+
+The public brand lists **Customer & Digital Analytics** as a pillar; this track
+is its open reference implementation. The hard part in customer intelligence
+isn't the algorithm — it's the *data model and the governance around the
+decision*: churn without leakage, next-best-offer with consent and eligibility,
+uplift that isn't confounded by who you targeted. So the foundation is a dataset
+built to make those problems real.
+
+## The foundation: [`data-model/`](data-model/)
+
+A seeded generator emits ~12 related tables with an explicit causal structure,
+so the cases are demonstrable rather than circular. See
+[`data-model/README.md`](data-model/README.md) for the schema, the causal design
+and the no-leakage guarantee.
+
+```bash
+cd data-model && uv run generate.py    # reproducible CSVs, byte-for-byte by seed
+```
+
+Tested on every push (schema, referential integrity, no-leakage, causal signal).
+
+## Cases (roadmap)
+
+Each case will produce a reproducible notebook/pipeline, a visible result, and a
+permanent write-up — the reusable evidence. (Distribution posts/video are a
+separate, downstream concern.)
+
+| # | Case | The real problem it shows |
+|---|---|---|
+| 01 | Actionable segmentation | segments carry behaviour, need, risk, eligible offer, consent and a suggested action — not just RFM clusters |
+| 02 | Churn without leakage | temporal split, calibration, explainable drivers, prioritisation by value — accuracy alone isn't success |
+| 03 | Governed next-best-offer | propensity/uplift **and** eligibility, consent, exclusions, contact policy |
+| 04 | ARPU / value decomposition | where revenue per user comes from and moves |
+| 05 | Campaign incrementality | true uplift vs the confounded naive read (targeting was not random) |
+
+Tracked in [`ronaldmego/site-ronaldmego#64`](https://github.com/ronaldmego/site-ronaldmego/issues/64).

tracks/customer-analytics/telco-customer-intelligence/data-model/DATA_CARD.md

@@ -0,0 +1,44 @@
+# Data card — telco customer-intelligence (synthetic)
+
+**This dataset is 100% synthetic.** It is generated by a seeded program
+(`generate.py`) from statistical assumptions. It contains **no real customers,
+no employer data, and no data derived from any real telecom operator.** Any
+resemblance to a real person or account is coincidental.
+
+## Why it exists
+
+To demonstrate customer-intelligence methods (segmentation, churn, next-best-
+offer, ARPU, incrementality) on a shared, realistic-but-open dataset — without
+exposing any employer's or customer's data. The narrative is grounded in the
+kind of work the author has done; the *data* is fabricated on purpose.
+
+## What it is
+
+- **Population:** configurable (default 5,000 customers), a mix of prepaid and
+  postpaid plans across four fictional regions.
+- **History:** configurable (default 24 months) of usage, billing, digital
+  engagement, support and campaign exposure — all before a single observation
+  cutoff.
+- **Target:** a 90-day churn label living *after* the cutoff.
+
+## How it was made
+
+A deterministic generator with an explicit causal structure (documented in
+`README.md`). Churn is caused by observable trajectories plus unobserved
+satisfaction and noise; retention campaigns carry a true but confounded uplift.
+
+## Known limitations — read before using
+
+- **Not a real distribution.** Base rates are chosen to be *plausible and
+  instructive*, not to match any specific market. Do not cite any figure from
+  it as a real-world statistic.
+- **Simplifications:** usage/billing/engagement are aggregated to monthly grain;
+  one active subscription spell per customer; a single churn cutoff.
+- **The causal parameters are known** (they live in `config.py`). That is a
+  feature for teaching — you can check whether a model recovers them — but it
+  means the dataset is not a blind benchmark.
+
+## License & reuse
+
+Synthetic data, generated by code in this repository, under the repository's
+MIT license. Reuse freely; keep this card with it.

tracks/customer-analytics/telco-customer-intelligence/data-model/README.md

@@ -0,0 +1,99 @@
+# Telco customer-intelligence — data model
+
+A **synthetic, causal, reproducible** dataset that the five customer-analytics
+cases (segmentation, churn, next-best-offer, ARPU, incrementality) all read
+from. One data model, many cases — not a fresh dataset per notebook.
+
+Standard library only, no numpy/pandas: the generator stays auditable and CI can
+validate it without installing anything. Output is plain CSV.
+
+## Generate
+
+```bash
+# from this directory
+uv run generate.py                       # defaults: 5000 customers, 24 months, seed 42
+uv run generate.py --customers 2000 --seed 7 --out data/
+python generate.py --customers 300 --months 12   # stdlib only — no deps required
+```
+
+Same seed → **byte-for-byte identical** CSVs on any machine. The `data/` output
+is git-ignored; reproduce it from the generator rather than committing it.
+
+## Why monthly, not daily
+
+`#64` sketched `usage_daily`. This model aggregates usage, billing and digital
+engagement to **customer × month**. Churn, ARPU and RFM are modelled at monthly
+grain in practice; daily rows would multiply the dataset ~30× and bloat the CSVs
+without adding analytical value. The grain is a deliberate modelling decision,
+documented here rather than hidden.
+
+## Tables
+
+Reference dimensions are tiny and fixed; facts scale with the population.
+
+| Table | Grain | Key | Notes |
+|---|---|---|---|
+| `plans` | one row per plan | `plan_id` | prepaid/postpaid, fee, data/voice caps, tier |
+| `offers` | one row per offer | `offer_id` | discount / data_bundle / upgrade |
+| `campaigns` | one row per campaign | `campaign_id` | objective (retention/upsell/crosssell) → `offer_id` |
+| `customers` | one row per customer | `customer_id` | signup, region, channel, age band, current plan, tenure |
+| `subscriptions` | one spell per customer | `subscription_id` | active at cutoff (churn is modelled as future) |
+| `usage_monthly` | customer × month | (`customer_id`,`period_month`) | data_gb, voice, sms, active_days |
+| `billing` | customer × month | `invoice_id` | billed/paid, paid_date, days_late, status (paid/late/failed) |
+| `digital_monthly` | customer × month | (`customer_id`,`period_month`) | app_logins, self_service, occasional NPS |
+| `support_interactions` | one ticket | `ticket_id` | reason, channel, escalated, resolved, csat |
+| `consent` | customer × channel | (`customer_id`,`channel`) | opt-in per email/sms/push/call |
+| `campaign_exposures` | customer × campaign | `exposure_id` | **exposed vs control**, responded |
+| `churn_labels` | one row per customer | `customer_id` | `churned_next_90d`, `observation_cutoff`, `churn_date` |
+
+## The causal structure (what makes the cases real)
+
+The label is **generated from observable trajectories** plus unobserved
+satisfaction and noise — not sprinkled at random. That is what lets a churn
+model recover signal, never fit perfectly, and never leak:
+
+- **usage decline** — a downward usage trend raises churn odds;
+- **payment problems** — failed/late invoices raise churn odds;
+- **unresolved escalations** — the strongest single driver;
+- **weak digital engagement** — low app logins raise churn odds;
+- **early life** — short-tenure customers churn more;
+- **plan misfit** — paying for a plan that doesn't fit;
+- **retention response** — a genuine responder to a retention campaign churns
+  *less* (a real, negative uplift);
+- **latent satisfaction + gaussian noise** — deliberately unobserved, so the
+  achievable model performance is bounded (no leakage, no perfect fit).
+
+**No leakage by construction:** every fact stops at the observation cutoff (the
+last month of history); the churn outcome lives in the *next 90 days*. There is
+no post-outcome fact to leak — features are strictly pre-cutoff. The test suite
+enforces this.
+
+**Incrementality has real confounding:** retention campaigns are *targeted at
+high-risk customers* (selection bias), so a naive "responders churn less" read
+is wrong — responders and the held-out **control** both sit above the base rate
+because both were selected for risk. The control group is what lets the
+incrementality case recover the true uplift instead of the confounded one.
+
+Reference signals from the default run (seed 42, 5000 customers) — **re-derive,
+don't quote**; they move with the seed:
+
+- 90-day churn base rate ≈ 13%
+- churn with an unresolved escalation ≈ 28% vs ≈ 10% without
+- retention responders ≈ 20% vs held-out control ≈ 31% (both > base → confounding)
+
+## Layout
+
+```
+data-model/
+├── generate.py          # CLI entrypoint
+├── telco/
+│   ├── config.py        # seed, scale, base rates (every knob, documented)
+│   ├── model.py         # the causal generator, table by table
+│   └── writer.py        # dict-of-tables → CSV
+├── README.md            # this file
+└── DATA_CARD.md         # synthetic-data disclosure
+```
+
+Tests live one level up in `../tests/` and run on the standard library, so CI
+validates schema, referential integrity, the no-leakage property and the
+presence of causal signal on every push.

tracks/customer-analytics/telco-customer-intelligence/data-model/generate.py

@@ -0,0 +1,42 @@
+#!/usr/bin/env python3
+"""Generate the synthetic telco customer-intelligence dataset.
+
+Reproducible: same seed -> byte-for-byte identical CSVs on any machine.
+
+    uv run generate.py                      # defaults: 5000 customers, 24 months
+    uv run generate.py --customers 2000 --seed 7 --out data/
+    python generate.py --customers 300 --months 12   # stdlib only, no deps needed
+"""
+
+from __future__ import annotations
+
+import argparse
+from pathlib import Path
+
+from telco import Config, generate, write_tables
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--customers", type=int, default=Config.n_customers, help="population size")
+    ap.add_argument("--months", type=int, default=Config.n_months, help="months of history before the cutoff")
+    ap.add_argument("--seed", type=int, default=Config.seed, help="RNG seed (determinism)")
+    ap.add_argument("--out", type=Path, default=Path(__file__).parent / "data", help="output directory")
+    args = ap.parse_args()
+
+    cfg = Config(seed=args.seed, n_customers=args.customers, n_months=args.months)
+    tables = generate(cfg)
+    counts = write_tables(tables, args.out)
+
+    churn = tables["churn_labels"]
+    churn_rate = sum(r["churned_next_90d"] for r in churn) / max(1, len(churn))
+    print(f"Wrote {len(counts)} tables to {args.out}/ (seed={cfg.seed}, "
+          f"{cfg.n_customers} customers, {cfg.n_months} months)")
+    width = max(len(n) for n in counts)
+    for name, n in counts.items():
+        print(f"  {name:<{width}}  {n:>9,} rows")
+    print(f"90-day churn base rate: {churn_rate:.1%}")
+
+
+if __name__ == "__main__":
+    main()

tracks/customer-analytics/telco-customer-intelligence/data-model/telco/__init__.py

@@ -0,0 +1,11 @@
+"""Synthetic telco customer-intelligence data model.
+
+The shared, causal, reproducible dataset that the five customer-analytics cases
+(segmentation, churn, next-best-offer, ARPU, incrementality) all read from.
+"""
+
+from .config import Config
+from .model import generate
+from .writer import write_tables
+
+__all__ = ["Config", "generate", "write_tables"]

tracks/customer-analytics/telco-customer-intelligence/data-model/telco/config.py

@@ -0,0 +1,85 @@
+"""Generation parameters for the synthetic telco data model.
+
+Everything the generator needs to be *reproducible* lives here: a single seed,
+the size of the population, the length of history, and the base rates that shape
+the causal structure. Change the seed and you get a different-but-consistent
+world; keep it and the output is byte-for-byte identical on any machine.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+
+@dataclass(frozen=True)
+class Config:
+    """All knobs for one generation run.
+
+    The defaults produce a mid-sized, analytically interesting dataset
+    (~5k customers over 24 months). Tests override ``n_customers`` and
+    ``n_months`` down to keep CI fast.
+    """
+
+    seed: int = 42
+    n_customers: int = 5000
+    n_months: int = 24  # months of history, all <= the churn observation cutoff
+
+    # Calendar anchor for month 0. Kept fixed so dates are reproducible without
+    # depending on the machine clock.
+    start_year: int = 2024
+    start_month_of_year: int = 1  # January
+
+    # --- Base rates that drive the causal structure -----------------------
+    # These are the levers the downstream cases are meant to *recover*, so they
+    # live in one place and are documented in data-model/README.md.
+
+    # Fraction of customers on prepaid (vs postpaid) plans.
+    prepaid_share: float = 0.55
+
+    # Baseline probability a billed invoice is paid late / fails, before the
+    # per-customer price-sensitivity adjustment.
+    base_late_rate: float = 0.08
+    base_fail_rate: float = 0.03
+
+    # Support: expected tickets per active year, and the chance a ticket ends
+    # escalated-and-unresolved (the strongest churn driver).
+    tickets_per_year: float = 1.4
+    unresolved_escalation_rate: float = 0.12
+
+    # Churn: the intercept sets the overall base rate; the weights set how much
+    # each observed driver moves the log-odds. Downstream churn models should be
+    # able to recover the *sign and rough magnitude* of these — not the exact
+    # value, because latent satisfaction and noise are deliberately unobserved.
+    churn_intercept: float = -3.6  # tuned to a ~13% 90-day base rate (realistic for telco)
+    w_usage_decline: float = 1.6
+    w_payment_problems: float = 1.1
+    w_unresolved_support: float = 1.4
+    w_low_engagement: float = 0.9
+    w_early_life: float = 0.8
+    w_plan_misfit: float = 0.7
+    w_retention_response: float = 1.5  # subtracted: a real retention uplift
+    churn_noise_sd: float = 0.6
+
+    # Campaigns: size of the true retention uplift (reduction in churn log-odds
+    # for a genuine responder) and the selection bias that makes the naive
+    # estimate wrong — retention campaigns preferentially target high-risk
+    # customers, so a naive "responders churn less" read is confounded.
+    retention_selection_bias: float = 1.8  # how strongly targeting favours risk
+    control_share: float = 0.30  # held-out control per campaign (for uplift)
+
+    def output_tables(self) -> tuple[str, ...]:
+        """Canonical table order — also the order they are written and tested."""
+        return (
+            "plans",
+            "offers",
+            "campaigns",
+            "customers",
+            "subscriptions",
+            "usage_monthly",
+            "billing",
+            "digital_monthly",
+            "support_interactions",
+            "consent",
+            "campaign_exposures",
+            "churn_labels",
+        )