An educational, code-first collection of handwritten Python implementations, visual method notes, and historical experiments by Ma Xiao (MaXiao).
This repository records how I explored several early tabular anomaly-detection methods with Python—especially PCA / Kernel PCA reconstruction error and RobustPCC. The historically preserved repository materials—code, formula notes, explanations, figures, and experiment records—are the main content of the project.
Historical origin (around 2018): this project began as my personal anomaly-detection study notebook—handwritten Python implementations, formula notes, paper walkthroughs, and exploratory experiments accumulated while I was learning the field.
Historical code policy: version-specific APIs, naming, comments, and implementation style are part of that record. They are documented rather than silently rewritten. Readers who want to run a historical script on a modern stack can adapt a copy—manually or with modern coding tools—while keeping the original file as the reference.
The repository is now being revived incrementally. The goal is to add tests, compatibility work, clearer English documentation, and carefully scoped maintained interfaces around the historical work—not to replace it with a wholesale rewrite.
Authorship boundary: for methods originating in cited papers, my contribution is the Python implementation, explanatory notes, and exploratory experiments, not the invention of the underlying algorithm. KADOA is separately labeled as an author-proposed experimental variation of ADOA.
The PCA study is one of the most complete threads in this repository. It connects paper reading, matrix reconstruction, anomaly scoring, a NumPy/SVD implementation, Kernel PCA experiments, and historical comparisons.
| Implementation | Python entry point | What it explores | Current status |
|---|---|---|---|
| Linear PCA reconstruction | recon_error_pca.py |
Weighted reconstruction errors across an increasing sequence of principal components | Historical implementation · Maintained candidate |
| NumPy/SVD reconstruction | recon_error_pca_svd.py |
The same reconstruction idea implemented directly with NumPy and SVD rather than scikit-learn PCA | Historical implementation · Characterization pending |
| Kernel PCA reconstruction | recon_error_kpca.py |
Reconstruction error in nonlinear feature spaces using different kernels | Historical implementation · 2018-era dependency API |
The detailed derivation and original learning path remain in the PCA walkthrough.
For each component count k, the historical implementation reconstructs the standardized data with the first k principal components. It then weights the corresponding reconstruction error by the cumulative explained-variance ratio and adds the weighted errors into one anomaly score.
Historical reconstruction formula preserved in the original PCA study notes. R^k is the matrix reconstructed with the first k principal components.
Historical score formula preserved in the original notes. The Chinese word “其中” means “where”; ev(k) is the cumulative explained-variance ratio used as the weight for reconstruction k.
The linear implementation is being considered for a future maintained API. The Kernel PCA script remains visible in its historical form; in the audited scikit-learn 1.9 environment, its former lambdas_ attribute is no longer available. This version boundary is documented instead of being silently rewritten into the original file.
robustpcc.py is my paper-guided Python implementation of a principal-component classifier for anomaly detection.
The implementation keeps the method's main steps visible:
- standardize the training data;
- use a Mahalanobis-style score to trim a small proportion of potential extremes;
- fit PCA on the remaining observations;
- identify major components that explain roughly the first 50% of variance;
- identify minor components whose eigenvalues fall below the historical threshold;
- calculate normalized deviations on both component groups;
- classify a sample as anomalous when either deviation exceeds its quantile threshold.
In the interpretation used by the cited RobustPCC method, these component groups provide two complementary anomaly views:
- major-component deviation: observations with extreme values in the original variables;
- minor-component deviation: observations that are unusual because their correlation structure differs from the training data.
The PCA and RobustPCC notes preserve the original formulas, terminology, implementation choices, and experiment discussion.
Current evidence status: RobustPCC is an important historical implementation in this repository, not a production-ready detector. Its threshold semantics and numerical edge cases are still being characterized before any maintained API decision.
max_ev_decrease.py records an additional experiment from the original PCA study. It compares the PCA eigenvalues before and after removing observations detected as anomalies.
Historical output from ten synthetic datasets. In these recorded runs, the three largest relative eigenvalue decreases always included index 0 or 19. This is an experiment snapshot, not a claim that the pattern holds for every dataset.
The following figure is preserved because it shows the range of algorithms explored in the original Python experiments: Isolation Forest, LOF, Mahalanobis distance, PCA/KPCA reconstruction error, RobustPCC, and two One-Class SVM configurations.
Historical F1 snapshot from ten synthetic datasets. The original environment and complete nine-series generation code have not yet been reconstructed, so the figure should be read as part of the repository's experiment history—not as a current benchmark or a general ranking of the methods.
View the corresponding historical runtime snapshot
Runtime depends on the original hardware, software environment, parameters, and implementation details. These values should not be used as current performance estimates.
The historical chart contains linear- and RBF-kernel One-Class SVM series, but their generating code has not been recovered from the reachable repository history. A future sklearn baseline must therefore be presented as new maintenance work, not reconstructed historical code.
The repository contains both unsupervised and partially supervised study implementations.
| Learning setting | Method | Repository material | Evidence status |
|---|---|---|---|
| Unsupervised | PCA reconstruction error | Linear PCA, NumPy/SVD, and method notes | Historical implementation · Linear version is a maintained candidate |
| Unsupervised | Kernel PCA reconstruction error | Kernel PCA code and nonlinear-kernel experiments | Historical implementation · 2018-era dependency API |
| Unsupervised | RobustPCC | Python implementation and major/minor-component explanation | Historical implementation · Experimental validation |
| Unsupervised | Mahalanobis distance | Direct covariance-inverse form, PCA-space variant, and an equivalence experiment | Historical implementation · Maintained candidate |
| Unsupervised | Isolation Forest | Paper notes and a scikit-learn example | Historical example · 2018-era dependency API |
| Unsupervised | Local Outlier Factor | Visual method notes and train/novelty examples | Historical example |
| Partially supervised | ADOA | Paper-guided implementation, clustering helper, and visual explanation | Historical implementation · Characterization pending |
| Partially supervised | KADOA | Author-proposed variation using Kernel PCA reconstruction error in the ADOA workflow | Author-proposed · Experimental |
| Positive-unlabeled | PU Learning | Spy/two-step strategy, Biased SVM, weighted logistic regression, papers, and notes | Historical code and notes · End-to-end execution not yet verified |
Status terms in this README describe the repository evidence, not the importance of a method:
- Historical implementation: original code is preserved and readable, but is not yet covered by a supported package and version matrix.
- Maintained candidate: a future maintained interface is under review; it is not yet a stable API on
master. - 2018-era dependency API: the code preserves the library interface used when it was written; modern adaptations should be documented separately rather than silently replacing the historical source.
- Experimental: behavior or empirical conclusions remain preliminary and should not be generalized.
Known characterization findings: the historical RobustPCC code selects all components when no eigenvalue meets its minor-component threshold, and the shared ADOA/KADOA clustering helper can break coordinate pairings by sorting cluster centers column by column. Their end-to-end impact has not yet been quantified.
The current master branch does not yet expose an installable stable package. This small example uses the historical class directly and scores the same matrix supplied at construction time:
import sys
import numpy as np
sys.path.insert(0, "UnSupervised-Based on PCA")
from recon_error_pca import PCA_Recon_Error
rng = np.random.RandomState(2018)
X = rng.normal(size=(100, 4))
detector = PCA_Recon_Error(X, contamination=0.05)
scores = detector.get_anomaly_score()
labels = detector.predict() # anomaly = 1, normal = 0
print(scores.shape)
print(labels.sum())This is an ordinary-input historical smoke example, not a supported-version guarantee. It demonstrates the historical interface only. A future maintained detector will require an explicit, tested fit / score_samples / predict contract before it is presented as stable.
- Start with the featured work: PCA reconstruction and RobustPCC.
- Follow the unsupervised collection: Isolation Forest → Mahalanobis distance → Local Outlier Factor.
- Explore partially observed anomalies: ADOA → KADOA.
- Explore positive-unlabeled learning: PU Learning notes and implementations.
- Read the full original Chinese presentation: README.zh-CN.md.
The current master branch is still a historical learning collection rather than a stable installable package. Maintenance is deliberately incremental: preserve original paths, characterize behavior before changing it, and keep compatibility notes or maintained interfaces separate from the historical source.
RobustPCC and Kernel PCA remain central historical implementations even though they are not in the first maintained API candidate set. See the revival roadmap for the current evidence boundary.
KADOA is presented here as an experimental variation proposed by Ma Xiao. It retains the broader clustering-and-weighting structure of ADOA while replacing the Isolation Forest component with Kernel PCA reconstruction error.
KADOA has not been presented as a peer-reviewed standalone method, and the historical repository experiment does not establish universal superiority over ADOA. If you use or discuss it, describe it as an experimental repository method, acknowledge its relationship to ADOA, and cite this repository.
Bug reports, compatibility notes, documentation corrections, and carefully scoped algorithm contributions are welcome through GitHub Issues.
For reproducibility reports, please include the Python and dependency versions, the exact script, a minimal input, and the observed traceback or output.
Use the software citation metadata in CITATION.cff. The project does not currently claim a DOI, formal release version, or independent KADOA publication.
The author-owned code and documentation in this repository are licensed under the MIT License.
Committed papers, publisher-formatted excerpts, and other third-party materials may have separate licenses. The repository's MIT License does not relicense those materials.