Transcriptomic Biomarker Discovery in Endometriosis

An end-to-end transcriptomics pipeline that merges multiple real public gene-expression datasets, removes cross-dataset batch effects, performs differential-expression analysis, classifies endometriosis vs. healthy endometrium with machine learning, and identifies candidate gene biomarkers with biological annotation. Built with bash + Python + Biopython + R.

Motivation

Endometriosis affects roughly 1 in 10 women of reproductive age, yet it takes on average 7–10 years to diagnose and remains chronically under-researched relative to its prevalence. Non-invasive, expression-based biomarkers could help close that diagnostic gap. This project builds a reproducible computational pipeline to (1) find genes differentially expressed in endometriosis and (2) test whether expression alone can distinguish patients from healthy controls.

Datasets (all real, all public — NCBI GEO)

Accession	Samples	Platform
GSE51981	34 normal + 77 endometriosis (111)	Affymetrix GPL570
GSE7305	10 normal + 10 endometriosis (20)	Affymetrix GPL570
GSE11691	9 normal + 9 endometriosis (18)	Affymetrix GPL96

Merging datasets across microarray platforms introduces batch effects — systematic technical differences that can swamp the biological signal. A central part of this project is removing them with ComBat, so that condition, not cohort, drives the downstream analysis.

Pipeline

GEO download (GEOparse)
      │
      ▼
Probe → gene-symbol mapping (platform annotation)
      │
      ▼
Merge 3 datasets on shared genes
      │
      ▼
Batch-effect removal — ComBat            ◀── PCA before/after (Fig 1–2)
      │
      ▼
Differential expression — limma (R)      ◀── volcano plot (Fig 3), heatmap (Fig 4)
      │
      ▼
ML classification — LogReg / RF / SVM    ◀── ROC curves (Fig 5)
      │
      ▼
Biomarker discovery (RF importance)      ◀── biomarker ranking (Fig 6)
      │
      ▼
Biological annotation — Biopython Entrez

Key results

• Batch correction: PCA shows the three datasets in fully separate clusters before ComBat and well-mixed afterwards, while the endometriosis-vs-normal separation is preserved — a clean demonstration of correct cross-dataset integration.
• Differential expression: limma identifies a set of significantly up- and down-regulated genes (adjusted p < 0.05, |log2FC| > 1), visualised in the volcano plot and heatmap.
• Classification: logistic-regression, random-forest, and SVM classifiers distinguish endometriosis from healthy endometrium with strong cross-validated AUC.
• Biomarkers: the most discriminative genes are ranked and annotated with their NCBI gene descriptions, producing a shortlist of candidate diagnostic markers.

Hierarchical clustering of the top differentially expressed genes cleanly separates endometriosis from healthy samples, and recovers known endometriosis-associated genes (e.g. AEBP1, TIMP1, TIMP2, VCAM1, WNT2B, AMHR2):

Repository structure

endometriosis-transcriptomics/
├── notebook/
│   └── endometriosis_transcriptomics.ipynb   # full pipeline
├── figures/                                   # output plots
├── results/                                   # DE tables, biomarkers, metrics
├── requirements.txt
└── README.md

How to run

Open the notebook in Google Colab and run top to bottom. It installs all Python/R dependencies, downloads the GEO datasets automatically, and mixes Python, bash, and R cells (via rpy2). No GPU required.

Tools

bash · Python (pandas, scikit-learn, GEOparse, ComBat, statsmodels) · Biopython (Entrez annotation) · R (limma, Bioconductor) · Matplotlib · seaborn

Honest scope and limitations

• Sample labels are curated from GEO metadata, which is inconsistent across datasets and requires manual verification.
• The data are microarray (not RNA-seq) expression profiles.
• Identified genes are candidate biomarkers; the findings are associative and would require independent and wet-lab validation before any diagnostic use.
• Batch correction with a modest number of cohorts cannot remove all confounding; results should be read as a demonstration of the integrated workflow on real data.

References

• Tamaresis, J. S., et al. (2014). Molecular classification of endometriosis and disease stage using high-dimensional genomic data. Endocrinology, 155(12), 4986–4999. (GSE51981)
• Hever, A., et al. (2007). Human endometriosis is associated with plasma cells and overexpression of B lymphocyte stimulator. PNAS, 104(30), 12451–12456. (GSE7305)
• Ritchie, M. E., et al. (2015). limma powers differential expression analyses for RNA-sequencing and microarray studies. Nucleic Acids Research, 43(7), e47.
• Johnson, W. E., Li, C., & Rabinovic, A. (2007). Adjusting batch effects in microarray expression data using empirical Bayes methods (ComBat). Biostatistics, 8(1), 118–127.
• Cock, P. J. A., et al. (2009). Biopython. Bioinformatics, 25(11), 1422–1423.
• Pedregosa, F., et al. (2011). Scikit-learn. JMLR, 12, 2825–2830.

License

MIT License (see LICENSE).