Anomaly Detection Playground — Time Series

A hands-on project comparing time-series anomaly detection techniques across three benchmark datasets, plus a case study on threshold-exceedance forecasting and honest temporal validation.

This started life as a study notebook and was restructured into a small, testable Python package so each idea can be run, tweaked, and reasoned about independently instead of scrolling through one long notebook.

What's inside

Dataset	Context	What counts as an anomaly
NASA Turbofan (C-MAPSS)	Aircraft engines degrading to failure	Points near end-of-life (low remaining useful life)
SKAB	Sensors on an industrial test bench (pump, motor, valves)	Deliberately injected faults
UCR Anomaly Archive	Diverse univariate signals	One known anomalous region per series

Five detection methods are compared where relevant:

1. Classical statistics (rolling Z-score)
2. Isolation Forest
3. One-Class SVM
4. Local Outlier Factor (LOF)
5. Autoencoder (deep learning, optional)

Every dataset loader tries to download the real data first and transparently falls back to a synthetic generator that mimics the same statistical properties if the download fails — so every script always runs, online or not.

There's also a bonus case study (scripts/run_pollution_case_study.py) that shifts from reactive anomaly detection to predictive threshold exceedance forecasting (e.g. "will PM2.5 cross a regulatory limit in the next few hours?") and walks through the validation mistakes that quietly inflate time-series metrics: data leakage, and the variance a single fixed train/test split hides that walk-forward validation reveals.

Project layout

src/anomaly_playground/
  data/         dataset loaders (real download + synthetic fallback)
  features/     rolling-window stats, time-delay embedding (windowing)
  detectors/    Z-score, Isolation Forest, One-Class SVM, LOF, Autoencoder
  evaluation/   precision/recall/F1/ROC-AUC, POD/FAR/CSI, plotting helpers
  drift/        Kolmogorov-Smirnov drift check
  validation/   leakage demonstrations, walk-forward folds with embargo gap
scripts/        one runnable entry point per dataset/case study
tests/          pytest unit tests for the reusable logic

Setup

python3 -m venv .venv
source .venv/bin/activate
pip install -e .          # core dependencies
pip install -e '.[deep]'  # optional: adds TensorFlow for the autoencoder
pip install -e '.[dev]'   # optional: adds pytest

Running it

Each script prints metrics to the console and saves plots under outputs/.

python scripts/run_nasa_turbofan.py --rul-threshold 30
python scripts/run_skab.py --window 15
python scripts/run_ucr.py --z-threshold 3.0 --window 64
python scripts/run_ucr.py --autoencoder          # needs the `deep` extra
python scripts/run_drift_monitoring.py
python scripts/run_pollution_case_study.py --horizon 5 --n-folds 4

Run the tests with:

pytest

How to study with this repo

This project is built to be learned by tinkering, through three layers:

• STUDY_GUIDE.md — guided experiments in a predict → run → check format, with the explanations hidden behind spoilers so you form your own answer first.
• Module docstrings — each detector and feature module opens with the intuition behind the method, when it shines vs. struggles, and which parameters are worth experimenting with. Read the docstring before the code.
• exercises/ — skeleton functions for you to implement (range-based metrics, detector ensembles, adaptive thresholds), with tests already written: pytest exercises -v starts all-skipped and turns green as you solve them.

The scripts also narrate what they're doing as they run — each stage prints the concept in play and what to watch for in the output that follows.

A few results worth pointing out

The most complex method doesn't always win. On the UCR series, the Z-score baseline catches the point spikes but misses the sustained regime change (low recall), because the rolling mean "follows" a long anomalous stretch. Isolation Forest over sliding windows sees the shape of the window rather than individual points, so it catches the whole episode:

SKAB anomalies are temporal, not just extreme values — rolling-window features (mean/std/min/max) are what let a detector see a fault developing across several sensors at once:

Drift monitoring matters after deployment. A Kolmogorov-Smirnov test on reference vs. production distributions flags when a sensor starts drifting away from what the model was trained on:

A fixed train/test split can hide a lot of variance. For the pollution exceedance case study, walk-forward validation across 4 folds gives a lower mean F-beta(2) and a much wider spread than the single fixed split suggested — because rare events land unevenly depending on where the data happens to be cut:

Key lessons

1. Accuracy lies on imbalanced data — use precision/recall/F1 instead.
2. There's no universal best method — it depends on the data and the business cost of false positives vs. false negatives.
3. Start simple (Z-score) before reaching for deep learning.
4. Feature engineering often matters more than the algorithm — rolling windows and time-delay embedding turn "is this point weird?" into "is this shape weird?".
5. Deployment isn't the finish line — drift detection and retraining triggers are part of the system.
6. Exceedance forecasting ≠ anomaly detection — one is predictive, one is reactive, and they call for different metrics (POD/FAR/CSI vs. precision/recall/F1).
7. Validate honestly — walk-forward with an embargo gap beats a single fixed split, and leakage hides in global .fit() calls and any shift(-k)/rolling(center=True) that looks into the future.

Ideas to extend this

• Systematic hyperparameter search over contamination, n_neighbors, rolling window — plot how F1 responds per dataset.
• Ensemble the detector scores (average or vote across Isolation Forest, LOF, Autoencoder) and see if it beats any single method.
• Swap the dense autoencoder for an LSTM autoencoder, built for sequences.
• Implement a range-based detection metric (a hit counts if it falls anywhere inside the known anomalous window, not just point-by-point).
• Add a dynamic/adaptive threshold instead of a fixed one, and see if it helps under drift.

Acknowledgments

The three datasets referenced are: NASA's C-MAPSS Turbofan Engine Degradation Simulation, the Skoltech Anomaly Benchmark (SKAB), and the UCR Time Series Anomaly Archive. All data loaders fall back to synthetic generators if the real files aren't reachable, so no dataset download is required to run this project.

License

Licensed under the GNU Affero General Public License v3.0 (AGPL-3.0) — see LICENSE.

Copyright (C) 2026 Flávio Manoel Santos Hemerli

You may use, modify and redistribute this code, including commercially, but any derivative work — including software you run as a networked service — must be released as open source under the same AGPL-3.0 terms.