From adf4911ca7614ccb4b884e9ebf1305aa54645f9e Mon Sep 17 00:00:00 2001 From: SAURAV SINGLA Date: Fri, 17 Jul 2026 14:28:15 +0530 Subject: [PATCH 1/9] Add anomaly detection project checklist --- docs/project-checklist.md | 91 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 91 insertions(+) create mode 100644 docs/project-checklist.md diff --git a/docs/project-checklist.md b/docs/project-checklist.md new file mode 100644 index 0000000..cacad12 --- /dev/null +++ b/docs/project-checklist.md @@ -0,0 +1,91 @@ +# Anomaly Detection Project Checklist + +Use this checklist when completing a course exercise, adapting a notebook, or presenting an anomaly-detection project. It is designed to make the analysis reproducible and easier to review. + +## 1. Problem definition + +- [ ] State what an anomaly means in the problem domain. +- [ ] Explain whether the task is outlier detection, novelty detection, or supervised rare-event classification. +- [ ] Describe the practical cost of a false positive and a false negative. +- [ ] Identify who will review or act on an alert. + +## 2. Data understanding + +- [ ] Record the data source, collection period, unit of observation, and target population. +- [ ] Check missing values, duplicates, impossible values, and inconsistent units. +- [ ] Examine skew, heavy tails, seasonality, and subgroup differences. +- [ ] Confirm that no confidential, personal, or regulated data is committed to the repository. + +## 3. Experimental design + +- [ ] Separate training, validation, and test data before fitting preprocessing steps. +- [ ] Use a time-based split when future observations must not influence the past. +- [ ] Fix random seeds where the implementation supports them. +- [ ] Record Python and package versions. +- [ ] Establish a simple baseline before trying complex methods. + +## 4. Preprocessing + +- [ ] Explain how missing values are handled. +- [ ] Fit scaling, encoding, and feature selection only on training data. +- [ ] Justify any logarithmic or power transformation. +- [ ] Review whether distance-based methods are being distorted by feature scale. +- [ ] Keep an auditable mapping from transformed features to their original meaning. + +## 5. Detector selection + +- [ ] Match each selected detector to the structure of the data. +- [ ] Document the main assumptions and important hyperparameters. +- [ ] Compare at least two credible approaches when practical. +- [ ] Confirm whether larger or smaller scores represent more anomalous observations. +- [ ] Avoid treating a library default contamination value as a known anomaly rate. + +## 6. Threshold selection + +- [ ] Select the threshold using validation evidence, domain prevalence, or an alert-review budget. +- [ ] Report how results change under nearby threshold values. +- [ ] State the final expected alert volume. +- [ ] Keep score generation separate from the business decision threshold. + +## 7. Evaluation + +- [ ] Do not rely on accuracy when anomalies are rare. +- [ ] Report precision, recall, F1, and average precision where labels are available. +- [ ] Include precision at k when investigation capacity is limited. +- [ ] Measure runtime and memory when scalability matters. +- [ ] Inspect representative false positives and false negatives. +- [ ] Compare performance across meaningful subgroups when appropriate. + +## 8. Interpretation + +- [ ] Explain why selected observations received high anomaly scores. +- [ ] Distinguish data errors from valid but unusual observations. +- [ ] Avoid automatically deleting detected outliers. +- [ ] Document limitations and cases where the method is likely to fail. + +## 9. Reproducibility + +- [ ] Run the notebook from the first cell in a clean kernel. +- [ ] Remove hidden state and machine-specific file paths. +- [ ] Keep dependencies in `pyproject.toml` or an environment file. +- [ ] Save only outputs that help the learner understand the result. +- [ ] Verify that another person can follow the setup instructions. + +## 10. Production considerations + +- [ ] Define monitoring for data drift, score drift, and alert volume. +- [ ] Record retraining or recalibration triggers. +- [ ] Add human review for high-stakes decisions. +- [ ] Define fallback behaviour when data or model checks fail. +- [ ] Establish privacy, security, access-control, and incident-response requirements. + +## Suggested project conclusion + +A strong conclusion should answer four questions: + +1. Which observations were flagged and how many? +2. Why was the selected detector and threshold appropriate? +3. What did the error analysis reveal? +4. What should a decision-maker do next? + +Keep the conclusion understandable to a reader who has not seen the notebook code. \ No newline at end of file From 9757cd8d81264ac68bd0db2eaecbc991d5626b9d Mon Sep 17 00:00:00 2001 From: SAURAV SINGLA Date: Fri, 17 Jul 2026 14:30:19 +0530 Subject: [PATCH 2/9] Link student project checklist --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 13e0bb7..6c3e5fd 100644 --- a/README.md +++ b/README.md @@ -39,7 +39,7 @@ The goal is to help learners move from understanding an algorithm in a notebook | Learn the subject independently | Follow the [student guide](docs/student-guide.md) and [learning roadmap](docs/course-roadmap.md) | | Try a working example quickly | Run [`examples/quickstart.py`](examples/quickstart.py) | | Compare detectors | Run the [benchmark](#reproducible-benchmark) | -| Practise and build a portfolio project | Complete the [exercises and capstone](exercises/README.md) | +| Practise and build a portfolio project | Complete the [exercises and capstone](exercises/README.md), then use the [project checklist](docs/project-checklist.md) | | Reuse the implementation | Explore [`src/outlier_detection/`](src/outlier_detection/) | | Fix an environment or notebook error | Read the [troubleshooting guide](docs/troubleshooting.md) | @@ -230,7 +230,7 @@ High-stakes use cases require human review, subgroup analysis, privacy safeguard ## Student practice -The [`exercises`](exercises/README.md) progress from basic statistical detection to multivariate modelling and a complete capstone. For every exercise, learners should document: +The [`exercises`](exercises/README.md) progress from basic statistical detection to multivariate modelling and a complete capstone. Use the [`project checklist`](docs/project-checklist.md) to review the final work before sharing it. For every exercise, learners should document: - the business or analytical objective; - preprocessing choices; @@ -319,4 +319,4 @@ Repository source code is released under the [MIT License](LICENSE). Course vide **Learn the assumptions. Run the experiment. Inspect the mistakes. Document the decision.** - + \ No newline at end of file From 1a3b53cb23e9a771437f0ebbf5e0fc632750186b Mon Sep 17 00:00:00 2001 From: SAURAV SINGLA Date: Fri, 17 Jul 2026 14:31:28 +0530 Subject: [PATCH 3/9] Add documentation index --- docs/README.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 docs/README.md diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..666c7b1 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,25 @@ +# Documentation + +Use this page as the starting point for the learning and project documentation in this repository. + +## Course learning path + +- [Course syllabus](course-syllabus.md) — maps the original Udemy lectures to the repository notebooks. +- [Student guide](student-guide.md) — explains setup, notebook usage and a practical study workflow. +- [Course roadmap](course-roadmap.md) — provides an extended path beyond the original course. +- [Troubleshooting](troubleshooting.md) — covers common installation, dependency and notebook problems. + +## Project work + +- [Project checklist](project-checklist.md) — reviews problem definition, data preparation, modelling, evaluation, interpretation and reproducibility before a project is shared. +- [Exercises and capstone](../exercises/README.md) — provides progressive practice tasks and a portfolio-oriented capstone. + +## Recommended order + +1. Set up the environment using the student guide. +2. Follow the course syllabus alongside the lecture notebooks. +3. Complete the exercises after each algorithm family. +4. Run the benchmark to compare methods consistently. +5. Use the project checklist before publishing a capstone or portfolio project. + +The original lecture notebooks remain unchanged where possible so that existing Udemy students can continue to follow the recorded course. \ No newline at end of file From f35a2d51274192dae87b5da43968f99b972e3912 Mon Sep 17 00:00:00 2001 From: SAURAV SINGLA Date: Fri, 17 Jul 2026 14:31:42 +0530 Subject: [PATCH 4/9] Add notebook issue template --- .github/ISSUE_TEMPLATE/notebook-problem.md | 40 ++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100644 .github/ISSUE_TEMPLATE/notebook-problem.md diff --git a/.github/ISSUE_TEMPLATE/notebook-problem.md b/.github/ISSUE_TEMPLATE/notebook-problem.md new file mode 100644 index 0000000..dabdbe7 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/notebook-problem.md @@ -0,0 +1,40 @@ +--- +name: Notebook problem +about: Report an error or unexpected result in a course notebook +title: "[Notebook] " +labels: bug +assignees: "" +--- + +## Notebook + +Provide the exact notebook file name and, where possible, the section or cell number. + +## What happened? + +Describe the error or unexpected result clearly. + +## Expected behaviour + +Explain what you expected to happen. + +## Environment + +- Operating system: +- Python version: +- Jupyter Notebook or JupyterLab version: +- Relevant package versions: + +## Error output + +Paste the complete traceback inside a code block. + +```text +Paste traceback here +``` + +## Steps already tried + +Mention whether you restarted the kernel, ran the notebook from the first cell, recreated the environment or checked the troubleshooting guide. + +Do not include private, confidential or regulated data in the issue. \ No newline at end of file From 983dd96ffa5fd2b6141a2a9ea035cf917e2f8878 Mon Sep 17 00:00:00 2001 From: SAURAV SINGLA Date: Fri, 17 Jul 2026 14:31:52 +0530 Subject: [PATCH 5/9] Add documentation issue template --- .../documentation-improvement.md | 25 +++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 .github/ISSUE_TEMPLATE/documentation-improvement.md diff --git a/.github/ISSUE_TEMPLATE/documentation-improvement.md b/.github/ISSUE_TEMPLATE/documentation-improvement.md new file mode 100644 index 0000000..53e996f --- /dev/null +++ b/.github/ISSUE_TEMPLATE/documentation-improvement.md @@ -0,0 +1,25 @@ +--- +name: Documentation improvement +about: Suggest a clearer explanation, example or learning resource +title: "[Docs] " +labels: documentation +assignees: "" +--- + +## Area to improve + +Link to the notebook, document or section that could be clearer. + +## Current difficulty + +Describe what is confusing, incomplete or difficult for a learner to reproduce. + +## Suggested improvement + +Explain the change that would make the material easier to understand. + +## Learner context + +Mention whether you are following the Udemy course, studying independently or using the repository for a project. + +Please avoid including copyrighted course material, private data or confidential examples. \ No newline at end of file From d407f4549947abd089545c0b1f103a8fa20931b9 Mon Sep 17 00:00:00 2001 From: SAURAV SINGLA Date: Fri, 17 Jul 2026 14:32:04 +0530 Subject: [PATCH 6/9] Add pull request checklist --- .github/pull_request_template.md | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 .github/pull_request_template.md diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 0000000..221b8ca --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,27 @@ +## Summary + +Describe the learning, correctness or reproducibility improvement made by this pull request. + +## Type of change + +- [ ] Notebook correction +- [ ] Documentation improvement +- [ ] Exercise or example +- [ ] Reusable code change +- [ ] Test or CI improvement + +## Validation + +- [ ] I ran the affected notebook or example from a clean start. +- [ ] I added or updated tests where reusable code changed. +- [ ] I checked that existing Udemy lecture references still work. +- [ ] I did not include private, confidential or regulated data. +- [ ] I cited external datasets, papers or code where applicable. + +## Learner impact + +Explain how this change helps a student understand, reproduce or apply the material. + +## Additional notes + +Include relevant screenshots, output summaries or limitations. Avoid committing large generated files unless they are essential. \ No newline at end of file From c0ae5bad1a4420435bc5c3c4a7fbaee9c7d5009c Mon Sep 17 00:00:00 2001 From: SAURAV SINGLA Date: Fri, 17 Jul 2026 14:32:25 +0530 Subject: [PATCH 7/9] Document notebook quality standards --- docs/notebook-quality-guide.md | 60 ++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) create mode 100644 docs/notebook-quality-guide.md diff --git a/docs/notebook-quality-guide.md b/docs/notebook-quality-guide.md new file mode 100644 index 0000000..09765a4 --- /dev/null +++ b/docs/notebook-quality-guide.md @@ -0,0 +1,60 @@ +# Notebook quality guide + +Use these checks when updating a lecture notebook or adding a new educational example. + +## Before running + +- Use a clean Python environment supported by the repository. +- Restart the kernel and run all cells from top to bottom. +- Fix random seeds where the library supports them. +- Keep datasets public, synthetic or properly attributed. + +## Explanations + +A useful notebook should explain: + +1. the learning objective; +2. the dataset and feature meaning; +3. preprocessing choices; +4. the detector assumptions; +5. score direction and label convention; +6. threshold or contamination selection; +7. evaluation metrics; +8. observed failure cases; +9. the practical conclusion. + +Avoid presenting one detector as universally best. Results depend on the anomaly definition, feature space, scaling, class balance and review cost. + +## Code + +- Keep cells focused and executable in sequence. +- Avoid hidden state from cells run out of order. +- Use clear variable names and brief comments for non-obvious steps. +- Separate data preparation, model fitting, scoring and evaluation. +- Do not suppress warnings without explaining why. +- Avoid absolute local file paths. + +## Results + +- Show more than accuracy for rare-event problems. +- Prefer precision, recall, F1, average precision and operational measures such as precision at k. +- Explain whether higher scores mean more or less anomalous. +- Report runtime when comparing computationally different methods. +- Discuss false positives and false negatives rather than showing only an aggregate score. + +## Visuals + +Every chart should have a descriptive title, labelled axes and a short interpretation. Use visualisations to support a conclusion, not only for decoration. + +## Final review + +Before committing a notebook: + +- restart and run all cells; +- remove accidental debug output; +- confirm that links and paths work; +- check that no private data or credentials are present; +- record any important version-dependent behaviour; +- keep compatibility with the original Udemy lecture flow where relevant. + +For a complete student project, also use the [project checklist](project-checklist.md). \ No newline at end of file From ab13c40747856ab6063d2e1405c1fe28e1b126f6 Mon Sep 17 00:00:00 2001 From: SAURAV SINGLA Date: Fri, 17 Jul 2026 14:32:38 +0530 Subject: [PATCH 8/9] Link notebook quality guide --- docs/README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/README.md b/docs/README.md index 666c7b1..d873698 100644 --- a/docs/README.md +++ b/docs/README.md @@ -12,6 +12,7 @@ Use this page as the starting point for the learning and project documentation i ## Project work - [Project checklist](project-checklist.md) — reviews problem definition, data preparation, modelling, evaluation, interpretation and reproducibility before a project is shared. +- [Notebook quality guide](notebook-quality-guide.md) — provides standards for clear, reproducible and course-compatible notebooks. - [Exercises and capstone](../exercises/README.md) — provides progressive practice tasks and a portfolio-oriented capstone. ## Recommended order @@ -20,6 +21,7 @@ Use this page as the starting point for the learning and project documentation i 2. Follow the course syllabus alongside the lecture notebooks. 3. Complete the exercises after each algorithm family. 4. Run the benchmark to compare methods consistently. -5. Use the project checklist before publishing a capstone or portfolio project. +5. Use the notebook quality guide when updating an example. +6. Use the project checklist before publishing a capstone or portfolio project. The original lecture notebooks remain unchanged where possible so that existing Udemy students can continue to follow the recorded course. \ No newline at end of file From 6e3bc767b76feeba212ce1a10cb1d9638233bad5 Mon Sep 17 00:00:00 2001 From: SAURAV SINGLA Date: Thu, 30 Jul 2026 08:47:54 +0530 Subject: [PATCH 9/9] Fix quickstart lint failure --- examples/quickstart.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/examples/quickstart.py b/examples/quickstart.py index 07453be..045b8c5 100644 --- a/examples/quickstart.py +++ b/examples/quickstart.py @@ -22,7 +22,10 @@ def build_dataset(): random_state=7, ) x = __import__("numpy").vstack([normal, anomalies]) - y = __import__("numpy").r_[__import__("numpy").zeros(len(normal)), __import__("numpy").ones(len(anomalies))] + y = __import__("numpy").r_[ + __import__("numpy").zeros(len(normal)), + __import__("numpy").ones(len(anomalies)), + ] return train_test_split(x, y, test_size=0.35, stratify=y, random_state=42)