BirdCLEF+ 2026 — What Actually Works: An Honest Ablation Study

Companion code for our Kaggle notebook and CLEF 2026 working note.
Every number in this repo is reproducible. Every claim is backed by CV output.

---

Motivation

Most BirdCLEF notebooks are either:

• A black-box ensemble with no ablation ("trust me, it scores 0.93X")
• An academic pipeline too complex to reproduce on a CPU-only Kaggle notebook

This repo takes a different approach: measure everything, claim nothing you haven't tested.

We start from the Google Perch v2 pretrained model and add components one at a time, reporting honest 5-fold CV macro-AUC at each step. If something doesn't help, we say so.

---

Results Tracker

Experiment	CV AUC	ΔBaseline	Notes
W1 · Perch logits → sigmoid baseline	TBD	—	Weekend 1
W2 · + Labeled soundscapes (in-domain)	TBD	TBD	Weekend 2
W3 · + MLP head on Perch embeddings	TBD	TBD	Weekend 3
W3 · + Prototype head (ProtoSSM)	TBD	TBD	Weekend 3
W4 · + Site/hour priors	TBD	TBD	Weekend 4
W4 · + Temperature scaling	TBD	TBD	Weekend 4
W5 · Ensemble (best heads)	TBD	TBD	Weekend 5

Results updated each weekend. All CV runs use the same 5-fold GroupKFold split on filename.

---

Repository Structure

birdclef2026/
├── src/
│   ├── data.py          # Audio loading, windowing, label parsing
│   ├── perch.py         # Perch inference engine (ONNX + TF fallback)
│   ├── heads.py         # MLP, ProtoSSM, and baseline sigmoid heads
│   ├── evaluate.py      # Macro-AUC, per-class AUC, fold reporting
│   ├── priors.py        # Site/hour prior tables
│   └── postprocess.py   # Temperature scaling, smoothing, blending
├── notebooks/
│   ├── week1_baseline.ipynb       # Perch baseline + honest CV
│   ├── week2_soundscapes.ipynb    # In-domain labeled data ablation
│   ├── week3_heads.ipynb          # MLP vs ProtoSSM head ablation
│   ├── week4_postprocess.ipynb    # Post-processing ablation
│   └── week5_ensemble.ipynb       # Final ensemble + submission
├── configs/
│   └── base.yaml        # All hyperparameters in one place
├── results/
│   └── cv_results.csv   # Append-only log of all CV runs
├── scripts/
│   └── build_cache.py   # Pre-compute Perch embeddings offline
└── README.md

---

Reproducibility

All experiments use:

• Same random seed: 42
• Same split: 5-fold GroupKFold on filename
• Same metric: macro-averaged ROC-AUC (skipping zero-positive classes), matching competition eval
• Same Perch version: perch_v2_cpu/1 from Google's TF Hub

To reproduce Weekend 1:

git clone https://github.com/YOUR_USERNAME/birdclef2026
cd birdclef2026
pip install -r requirements.txt
python scripts/build_cache.py   # pre-compute Perch embeddings
jupyter notebook notebooks/week1_baseline.ipynb

---

CLEF 2026 Working Note

We are submitting a working note to the BirdCLEF+ 2026 track at CLEF. The paper draft is in paper/ and follows the ablation structure of this repo.

Key claims we will defend in the paper:

1. Perch v2 logits alone are a strong baseline — no fine-tuning needed
2. Labeled soundscapes add measurable signal for underrepresented species
3. Post-processing (priors + temperature) contributes more than head complexity
4. Honest null results: what we tried that didn't work

---

Authors

• Fernando Arizmendi

---

Citation

If you use this code or findings, please cite:

@inproceedings{fer2026birdclef,
  title     = {What Actually Works in BirdCLEF+ 2026: An Honest Ablation Study},
  author    = {Fernando Arizmendi},
  booktitle = {Working Notes of CLEF 2026},
  year      = {2026}
}