Performer DNA Classifier

Code needed to train/evaluate the Performer-based DNA classifier on FLIBASE-style cancer vs normal sequences.

Problem

Binary sequence-level classification: given a DNA sequence, decide if it originates from a cancer or a normal sample. Constraints:

• Long, heterogeneous sequences (kilobase scale) with sparse cancer cues.
• Batch effects across centres/chemistries; labels are noisy.
• Must run on a single GPU with reproducible, auditable outputs.

Solution sketch:

• Clean/QC sequences, then build balanced splits (all cancer, sampled normals, 60/20/20).
• Tokenise into overlapping 6-mers, cut into windows (default 384 tokens, stride 192).
• Feed windows to a Performer encoder (FAVOR+ linear attention), pool with learnable gates, classify via MLP.

Model architecture (script)

• 6-mer embeddings; padding and mask handling.
• Performer stack: 6 layers, 8 heads, hidden width 512, random features m=256 (FAVOR+).
• Gated pooling across windows to weight informative regions.
• Classification head: 2-layer MLP → logits for cancer/normal.
• Training: AdamW + cosine LR, dropout 0.1, grad clip 0.5, mixed precision (bfloat16), balanced sampling.

Main script: PerformerTry/dna_performer_classifier.py.

Requirements

• Python 3.8+
• PyTorch with CUDA
• transformers, performer-pytorch, pandas, scikit-learn, matplotlib, numpy

Install (example):

pip install torch transformers performer-pytorch pandas scikit-learn matplotlib numpy

Data

Expected CSVs with one sequence per line:

• FLIBASE-ALL-NORMAL-SRT-sequences.csv
• FLIBASE-ALL-CANCER-SRT-sequences.csv

By default the script looks for these files in the repo root. Override with env vars PANCREAS_NORMAL and PANCREAS_CANCER.

Running the Performer

Basic run on the full data (balanced 60/20/20 split, 6-mer tokens, windows 384 with stride 192):

TRANSFORMERS_NO_TORCHVISION=1 \
TOKENS=384 \
STRIDE_TOKENS=192 \
PERF_DIM=512 \
PERF_DEPTH=6 \
PERF_HEADS=8 \
PERF_NB_FEAT=256 \
BATCH=2 VAL_BATCH=4 \
EPOCHS=8 \
LR=3e-5 \
GRAD_CLIP=0.5 \
DROP=0.1 \
AMP=1 \
SAVE_WINDOWS=1 \
PROB_OUT_DIR=PerformerTry/perf_probs_run \
python PerformerTry/dna_performer_classifier.py

Key env knobs:

• TOKENS / STRIDE_TOKENS: window length / stride (tokens).
• PERF_DIM, PERF_DEPTH, PERF_HEADS, PERF_NB_FEAT: model width/layers/heads/random features.
• BATCH, VAL_BATCH: sequence batch sizes (windows are packed inside).
• LIMIT_PER_CLASS: optionally subsample each class (e.g., 1000) for quick tests.
• TRAIN_*, VAL_*, TEST_* (six vars): point to pre-split CSVs if you already have train/val/test files.
• PROB_OUT_DIR: where to dump per-split probabilities and per-window outputs (if SAVE_WINDOWS=1).

Notes

• Mixed precision (AMP=1) is recommended on RTX A6000/3090-class GPUs; disable if you see instability.
• The script prints class counts per split and saves probability CSVs for downstream plotting.