LFQ Preprocessing Sensitivity

Reproducible Python analysis repository for:

Impact of missing-value imputation and normalization choices on differential abundance results in public label-free proteomics datasets.

This is a workflow-sensitivity study, not a new imputation-method paper and not a biological discovery project. The goal is to identify proteins whose differential-abundance calls are robust across preprocessing workflows versus proteins whose calls are sensitive to normalization, imputation, and missingness.

Datasets

• PXD000279_dynamicrange: dynamic range benchmark, UPS1 vs UPS2, 4 vs 4 replicates.
• PXD001819: UPS1-in-yeast spike-in benchmark, UPS1_50amol vs UPS1_50000amol, 3 vs 3 replicates. UPS/human proteins are expected positives and yeast proteins are expected background.
• PXD001584: biological perturbation dataset, WT_20 vs D8_20, with technical replicates averaged to biological replicate level when configured.

Required Inputs

Place MaxQuant proteinGroups.txt files and tab-separated metadata here:

data/raw/PXD000279_dynamicrange/proteinGroups.txt
data/raw/PXD001819/proteinGroups.txt
data/raw/PXD001584/proteinGroups.txt
data/metadata/PXD000279_dynamicrange_metadata.tsv
data/metadata/PXD001819_50_vs_50000_metadata.tsv
data/metadata/PXD001584_metadata_WT20_vs_D8_20_technical.tsv

Metadata must contain at least sample and group. Sample names must match cleaned MaxQuant LFQ column names after removing LFQ intensity and suffixes such as .raw.

Environment

conda env create -f environment.yml
conda activate lfq-preprocessing-sensitivity

Or with pip:

pip install -r requirements.txt

Run One Dataset

python scripts/01_check_inputs.py --dataset PXD000279_dynamicrange
python scripts/02_make_qc.py --dataset PXD000279_dynamicrange
python scripts/03_run_method_grid.py --dataset PXD000279_dynamicrange
python scripts/05_make_figures.py --dataset PXD000279_dynamicrange

Run All Available Datasets

python scripts/06_run_all.py

Datasets with missing input files are skipped, so PXD001819 can wait until MaxQuant finishes.

Outputs

Per dataset, tables are written to results/<dataset>/tables/:

• clean_metadata.tsv
• missingness_per_sample.tsv
• missingness_per_protein.tsv
• filtered_matrix.tsv
• differential_<method_id>.tsv
• method_summary.tsv
• pairwise_method_metrics.tsv
• protein_decision_instability.tsv
• ground_truth_metrics.tsv for applicable spike-in benchmarks

Figures are written to results/<dataset>/figures/:

• missingness_per_sample.png, missingness_per_protein.png, missingness_heatmap.png
• intensity_distributions.png, pca.png, sample_correlation.png
• significant_counts_by_workflow.png, jaccard_heatmap.png
• instability_vs_missingness.png, instability_vs_group_missingness.png
• fold_change_scatter.png (baseline norm-none__imp-none vs norm-median__imp-low_intensity_random)
• sensitivity_vs_false_positive.png for spike-in benchmarks with ground truth (PXD001819)

Combined outputs are written to results/combined/ after at least one completed dataset, and are most useful after two or more datasets.

Interpreting Key Tables

method_summary.tsv has one row per workflow. For each normalization and imputation combination it reports the number of proteins tested, number significant, median absolute log2FC, minimum adjusted p-value, PCA separation score, and sample correlation summaries.

protein_decision_instability.tsv has one row per protein. fraction_methods_significant reports how often the protein is significant across workflows. decision_instability is highest near 0.5, meaning the protein flips decisions across workflows. Missingness columns help identify preprocessing-sensitive proteins driven by missing values.

Tests

pytest tests/

tests/conftest.py adds src/ to sys.path automatically, so tests work from the repo root.

Troubleshooting

• "Metadata samples do not match cleaned LFQ columns" — open data/raw/<dataset>/proteinGroups.txt, look at any LFQ intensity <sample> column, drop the LFQ intensity prefix and any .raw suffix, and make sure those exact strings appear in the sample column of the metadata TSV. Sample names are matched as strings.
• PXD001584 technical replicates — the differential test must be done at biological-replicate level. Keep average_technical_replicates: true in config/datasets.yaml and provide technical_replicate and biological_replicate columns in the metadata.
• PXD001819 ground-truth labels — the classifier looks at Protein IDs, Majority protein IDs, Gene names, and Protein names columns from proteinGroups.txt. If your FASTA headers use a non-standard convention, extend classify_pxd001819 in src/lfqstudy/metrics.py.

Reproducibility Notes

All random imputation uses a fixed seed from config/methods.yaml. Raw data are never overwritten. Workflow definitions live in YAML config files. The initial differential method is Welch's t-test; the code is modular so limma or MSstats can be added later.