Update documentation: enhance README, add maintenance guidelines, and implement 404 error page

Author: ChrisW09

07_industry_applications/README.md

@@ -73,16 +73,36 @@ The first four close loops the core notebooks deliberately leave open. NB 23, NB
 
 **A9–A14 are growth & operations** — and they exist because the first eight appendices, for all their range, share a shape: predict something, threshold it, act. These six break that mould in ways the module needed. **A9** is the missing commercial function: A1 owns price and A7 owns the sales motion, but nothing owned the marketing budget, which needs two effects no other notebook has — carryover and diminishing returns — and delivers an *allocation* rather than a score. **A10** sets capacity instead of ranking within it, and brings the course its only queueing theory; its punchline is that the conversion from volume to headcount is non-linear, which is why a plan built on daily averages hits its target on paper and fails in half of all intervals. **A11** is the module's first genuine *plan under constraints* — an assignment and a sequence, where the lesson is that an optimiser answers the question you actually asked, and that the schedule which looks best on paper is the one that collapses on Tuesday. **A12** takes A3's regulated-pricing idiom somewhere A3 cannot go: a loss that is a count *times* an amount, a severity with a real tail, and a strategic trap (refuse to differentiate and your good risks leave) that has no analogue in approve/decline. **A13** works upstream of NB 23, in cohorts rather than customer-months, and asks the question that decides how a product team spends its quarter — is this metric *diagnostic* or *actionable*? **A14** closes the module's longest argument: NB 23 targeted with a model, A3 found its own decisions in its training data, A4 measured the damage, A2 randomized to fix it — and A14 shows what happens when the randomization itself adapts, which fixes the cost of learning and re-breaks the inference.
 
+All fourteen run ~2 h 45 m at ⭐⭐ (stretch ⭐⭐⭐), and all are independent of each other — pick by the question you actually have:
+
+| Track | Appendices | Pick one when you want… |
+|---|---|---|
+| **Closing the loops** | [A1](#a1--pricing-elasticity--promotion-roi--a1_pricing_promotionsipynb) · [A2](#a2--experiment-design--ab-testing-for-business-decisions--a2_experiments_ab_testingipynb) · [A3](#a3--credit-risk-scorecards-expected-loss--the-approvedecline-decision--a3_credit_risk_scorecardsipynb) · [A4](#a4--causal-inference--uplift-who-to-target-not-who-will-churn--a4_causal_upliftipynb) | the questions NB 23–26 raise and then defer — pricing, the experiment they keep recommending, regulated lending, and the causal machinery behind "risk ≠ persuadability" |
+| **The business-function tour** | [A5](#a5--people-analytics-attrition-survival--pay-equity--a5_people_analyticsipynb) · [A6](#a6--finance-late-invoices-collections--the-13-week-cash-forecast--a6_finance_ar_cashflowipynb) · [A7](#a7--sales-lead-scoring-pipeline-truth--the-quarter-forecast--a7_sales_pipelineipynb) · [A8](#a8--procurement-spend-supplier-scorecards--total-cost--a8_procurement_spendipynb) | the same toolkit in the four offices the core notebooks never visit: HR, finance, sales, procurement |
+| **Growth & operations** | [A9](#a9--marketing-mix--incrementality--a9_marketing_mixipynb) · [A10](#a10--service-operations-arrivals-erlang-c--the-staffing-plan--a10_service_operationsipynb) · [A11](#a11--from-prediction-to-allocation-routing-dispatch--slack--a11_routing_allocationipynb) · [A12](#a12--pricing-risk-frequency--severity--adverse-selection--a12_insurance_pricingipynb) · [A13](#a13--product-analytics-funnels-cohort-curves--the-activation-metric--a13_product_analyticsipynb) · [A14](#a14--bandits--adaptive-allocation-learning-while-you-earn--a14_bandits_adaptiveipynb) | a deliverable that is not a score: a budget, a headcount, a schedule, a rate table, a roadmap call, a policy |
+
+#### Closing the loops — A1–A4
+
 | Appendix | Notebook | ⏱ Time | Difficulty | Business problem | What you'll build |
 |---|---|---|---|---|---|
 | A1 | `A1_pricing_promotions.ipynb` | ~2 h 45 m | ⭐⭐ (stretch ⭐⭐⭐) | What should we charge for three very different SKUs, and was last year's promo calendar worth running? | Confound-corrected price elasticities, the headroom rule and inverse-elasticity price, a bootstrap + support-range guardrail that produces *hold / raise / test* rather than three prices, and a break-even discount rule that settles the promo P&L |
 | A2 | `A2_experiments_ab_testing.ipynb` | ~2 h 45 m | ⭐⭐ (stretch ⭐⭐⭐) | Three teams want to test something. Which of these tests can actually answer its question? | MDE-first test planning that *cancels* one test before it runs, CUPED variance reduction that doubles power for free, a peeking simulation and a boundary calibrated by Monte Carlo, SRM + multiple-comparisons validity checks, and the winner's curse quantified at two power levels |
 | A3 | `A3_credit_risk_scorecards.ipynb` | ~2 h 45 m | ⭐⭐ (stretch ⭐⭐⭐) | Who gets trade credit, how much, and what do we do about the applicants we have never approved? | A WOE/IV scorecard with IV screen, sign check and points transform, a calibration check, the `PD* = m/(m+LGD)` cutoff and profit curve, risk-banded limits priced as an overlay, reason codes for decline letters, and the selective-labels problem measured in euros |
 | A4 | `A4_causal_uplift.ipynb` | ~2 h 45 m | ⭐⭐ (stretch ⭐⭐⭐) | Last quarter's retention report says the discount backfired. Did it — and who should get it this quarter? | The naive-vs-causal decomposition computed exactly (effect + selection bias), the four uplift segments priced in euros, a T-learner with a break-even targeting depth, a difference-in-differences + placebo analysis of an un-randomized rollout, and adjustment methods shown working — then failing with the wrong sign — on targeted data |
+
+#### The business-function tour — A5–A8
+
+| Appendix | Notebook | ⏱ Time | Difficulty | Business problem | What you'll build |
+|---|---|---|---|---|---|
 | A5 | `A5_people_analytics.ipynb` | ~2 h 45 m | ⭐⭐ (stretch ⭐⭐⭐) | Who is leaving, what does attrition actually cost — and is the pay gap real? | A hand-built Kaplan–Meier survival curve that exposes the censoring trap, an attrition bill per function (the highest *rate* is not the biggest *bill*), an honest leaves-within-12-months model with a per-function break-even threshold and a budgeted retention list, and a raw-vs-adjusted pay-gap decomposition with the mediator caveat spelled out |
 | A6 | `A6_finance_ar_cashflow.ipynb` | ~2 h 45 m | ⭐⭐ (stretch ⭐⭐⭐) | When will these invoices actually pay, whom should the two collectors chase, and will we need the credit line? | A days-late model built on as-of features (persistence is the signal), a collections queue ranked by expected cash acceleration rather than amount, a Monte-Carlo 13-week cash forecast that turns into a credit-line decision with a probability attached, and a walk-forward backtest that prices the due-date spreadsheet's optimism in euros |
 | A7 | `A7_sales_pipeline.ipynb` | ~2 h 45 m | ⭐⭐ (stretch ⭐⭐⭐) | Are the CRM's own numbers any good — and what actually lands this quarter? | An as-of snapshot rebuild that deflates a leaked AUC 1.000 to an honest 0.853, a reliability curve that prices rep-entered probabilities, a calibrated quarter forecast with Monte-Carlo bands backtested over six quarters, and an EV-ranked lead queue with a break-even calling depth |
 | A8 | `A8_procurement_spend.ipynb` | ~2 h 45 m | ⭐⭐ (stretch ⭐⭐⭐) | Where does the €11M actually go, which suppliers are actually good, and should we consolidate the tail? | A spend cube with Pareto/ABC and a maverick-spend bill, a naive savings claim collapsed to its honest number, a supplier scorecard where lead-time *variance* is priced via NB 26's safety stock, total cost of ownership that dethrones the cheapest invoice, and a consolidation plan with payback and the dual-sourcing premium worth paying |
+
+#### Growth & operations — A9–A14
+
+| Appendix | Notebook | ⏱ Time | Difficulty | Business problem | What you'll build |
+|---|---|---|---|---|---|
 | A9 | `A9_marketing_mix.ipynb` | ~2 h 45 m | ⭐⭐ (stretch ⭐⭐⭐) | Which channels actually create demand, and where should next year's €2.4M go? | Adstock and saturation curves fitted and recovered against planted truth, last-click's 12× ROAS on branded search set beside its true 1.72×, a marginal-return reallocation worth +€193k a year on the same budget, identification limits priced in euros, and a geo holdout that buys the answer |
 | A10 | `A10_service_operations.ipynb` | ~2 h 45 m | ⭐⭐ (stretch ⭐⭐⭐) | How many agents does next quarter need, at what service level — and what does the deflection bot really save? | Interval-level arrival forecasting, Erlang C from scratch checked against simulation, the occupancy cliff and shrinkage in FTE and euros, a cost-optimal (not maximal) service level, and an AI-deflection business case walked from the vendor's 30% down to an honest 11.2% |
 | A11 | `A11_routing_allocation.ipynb` | ~2 h 45 m | ⭐⭐ (stretch ⭐⭐⭐) | Forty jobs, six vans, two-hour promises — what does tomorrow's schedule look like? | A travel-time model whose *error* matters more than its mean, the distance-optimal plan that misses 27 of 40 windows, a euro-optimal assignment (`linear_sum_assignment` + greedy insertion + 2-opt) that drives further and costs €4,836 less, and slack chosen from a 400-day simulation because the on-paper optimum always picks zero |

MAINTENANCE.md

@@ -17,6 +17,7 @@ Work through the table; most quarters most rows need nothing.
 | **EU AI Act section** | Obligation dates phrased as future become past as they arrive (next milestone: Annex I embedded high-risk, Aug 2027) | [`16_business_ai/52_bpm_governance_poc_mvp.ipynb`](./16_business_ai/52_bpm_governance_poc_mvp.ipynb) §5 |
 | **Colab claims** | "Colab ships PyTorch preinstalled" and friends still true | root `README.md`, Module 6 |
 | **Optional requirements** | Commented pins in `requirements.txt` still install cleanly on a fresh venv (spot-check the ones you touch) | [`requirements.txt`](./requirements.txt) |
+| **Tools named in honest sections** | Module 7's appendices name the tools real teams reach for (Robyn / PyMC-Marketing in A9, OR-Tools in A11, `lifelines` in A5, off-policy evaluation libraries in A14). No code depends on them, but the claim "this is what the industry uses" ages | [`07_industry_applications/`](./07_industry_applications/) |
 | **External links** | `make -C docs_site linkcheck` output is empty (exclusions live in `conf.py`) | `docs_site/` |
 
 Anything *not* on this list — synthetic-data lessons, statistics, sklearn/pandas idioms — only needs attention when a library's own API deprecates something (CI's execution sweep will surface that).
@@ -32,6 +33,24 @@ Anything *not* on this list — synthetic-data lessons, statistics, sklearn/pand
 5. **Counts:** `python3 scripts/check_course_counts.py` — notebook/checkpoint/appendix totals in the README, docs index and 00b must match the tree (CI enforces this too).
 6. **Docs:** `make -C docs_site html` builds with `-W`; `make -C docs_site linkcheck` output should be empty.
 
+## Adding a notebook or a module
+
+Most of the gates above are self-explanatory once they fail. Three are not, because they enforce
+conventions encoded in file *names* and in the docs sidebar:
+
+- **Appendix filenames** must be `A<n>_<slug>.ipynb` in a module directory. `check_course_counts.py`
+  globs `A[0-9]*_*.ipynb`, so two-digit appendices (`A10_…`) count correctly — it globbed `A[0-9]_*`
+  until Module 7 grew past nine appendices, which silently under-counted rather than failing.
+- **A new module directory** must be added by hand to a `{toctree}` group in `docs_site/index.md`;
+  the sidebar is grouped by theme rather than globbed, so `generate.py` refuses to build if the two
+  disagree and names the module it could not place.
+- **Every notebook needs a Colab row** in the root README's index — `check_course_counts.py` compares
+  the number of unique Colab links against the tree and fails if one is missing.
+
+When a module gains notebooks, the counts in the root README (badge, headline, "across the course",
+appendix totals, Colab footnote), `docs_site/index.md`, and `00b_course_overview.ipynb` all move
+together. Run gate 5 rather than trying to remember the list; it names each stale location.
+
 ## Editing rules that keep the course honest
 
 - **Prose numbers must match printed output.** If a cell prints `+1.8 pp`, the paragraph below it says +1.8, not +1.9. After re-execution, re-read the surrounding prose.

docs_site/README.md

@@ -20,7 +20,7 @@ relative links so they resolve in the built site. Those directories and
 `_build/` are generated — edit the module READMEs in the repository instead,
 then rebuild.
 
-Two things `generate.py` does that are worth knowing when you edit a module:
+Three things `generate.py` does that are worth knowing when you edit a module:
 
 - **Chapter order follows the README.** A mini-book module's sidebar lists its
   chapters in the order the module README first links to them, not
@@ -31,6 +31,23 @@ Two things `generate.py` does that are worth knowing when you edit a module:
   become GitHub URLs; links to another module's directory or README become
   that module's page on this site; `../README.md` ("🏠 Course home") becomes
   this site's home page.
+- **The sidebar is grouped by hand, and checked.** `index.md` sorts the twenty
+  modules into themed `{toctree}` blocks ("Python foundations", "Machine
+  learning & applications", …) rather than one 20-entry `:glob:`, because a flat
+  list of twenty is a wall rather than a table of contents. The cost is that a
+  new module could be left out — so `generate.py` compares the tree against
+  `index.md` first and aborts with the offending module's name if they differ.
+
+Beyond the module guides the site also publishes the course-wide pages
+(fast track, quizzes, datasets) and [`MAINTENANCE.md`](../MAINTENANCE.md) as
+*For maintainers* — the same quarterly-currency checklist and verification
+gates contributors run locally. Add another by putting it in `EXTRAS` in
+`generate.py` and giving it a `{toctree}` entry in `index.md`.
+
+`docs_site/root_files/` is copied verbatim to the site root (`html_extra_path`).
+It holds `404.html`, which GitHub Pages serves for any unknown path: it is
+deliberately a standalone file with absolute URLs and inline CSS, because a
+themed page served from `/a/b/c/` cannot resolve its assets by relative path.
 
 `make html` builds with `-W`, so an unresolved cross-reference fails the build
 rather than shipping a dead link. `make linkcheck` additionally verifies that

docs_site/conf.py

@@ -56,6 +56,14 @@
 
 html_theme = "furo"
 html_title = "Python for AI-Driven Automation<br>& Business Data Science"
+# Every page is regenerated from the repository markdown on each build, so this
+# stamp is the publish date — which is the useful thing for a reader deciding
+# whether a model name or a price is still current.
+html_last_updated_fmt = "%d %B %Y"
+# Files copied to the site root verbatim: 404.html needs to sit there (GitHub
+# Pages serves it for any unknown path) and must not be wrapped in a template,
+# because a page served at /a/b/c/ cannot resolve theme assets by relative path.
+html_extra_path = ["root_files"]
 # "assets" holds the hand-written design files (tracked); "_static" is rebuilt
 # by generate.py each run. Sphinx merges both into the output _static/.
 html_static_path = ["_static", "assets"]

docs_site/generate.py

@@ -35,6 +35,7 @@ def github_url(rel, is_dir: bool) -> str:
     "fast_track": ROOT / "fast_track" / "README.md",
     "quizzes": ROOT / "quizzes" / "README.md",
     "datasets": ROOT / "data" / "README.md",
+    "maintenance": ROOT / "MAINTENANCE.md",
 }
 
 LINK_RE = re.compile(r"(?<!!)\[([^\]]*)\]\(([^)\s]+)\)")
@@ -116,7 +117,33 @@ def chapter_order(mod: Path, readme: str) -> list[str]:
     return ordered + sorted(files - set(ordered))
 
 
+def check_index_covers_modules() -> None:
+    """Fail the build if index.md's sidebar has drifted from the module tree.
+
+    The home page groups the modules into captioned sections by hand, because a
+    single 20-entry `:glob:` list is a wall rather than a table of contents.
+    The cost of writing them out is that adding a module could silently leave it
+    out of the sidebar — so check here instead, where `make html` runs first.
+    """
+    index = (HERE / "index.md").read_text()
+    listed = set(re.findall(r"^modules/(\d{2}_[a-z0-9_]+)/index\s*$", index, re.M))
+    actual = {m.name for m in MODULE_DIRS}
+    missing, extra = actual - listed, listed - actual
+    if missing or extra:
+        problems = []
+        if missing:
+            problems.append("not listed in index.md: " + ", ".join(sorted(missing)))
+        if extra:
+            problems.append("listed but absent from the tree: " + ", ".join(sorted(extra)))
+        raise SystemExit(
+            "docs_site/index.md is out of step with the module tree —\n  "
+            + "\n  ".join(problems)
+            + "\nAdd the module to the appropriate `{toctree}` group in index.md."
+        )
+
+
 def main() -> None:
+    check_index_covers_modules()
     for out in (HERE / "modules", HERE / "extras", HERE / "_static"):
         shutil.rmtree(out, ignore_errors=True)
         out.mkdir(parents=True)