fix(data_preprocess): GPQA answer preprocess() strips load-bearing brackets

[glorgao]

Problem

preprocess() in data_preprocess/stem/gpqa_diamond.py and data_preprocess/stem/gpqa.py (adopted from EleutherAI's lm-evaluation-harness GPQA utils, per the code comment) runs re.sub("\[.*?\]", "", text) on every answer-choice field. It's meant to strip citation-style [1]/[title] markers, but GPQA's answer fields are heavily organic-chemistry/physics, where [...] is load-bearing content:

• IUPAC ring-locants: spiro[4.5]decan-6-ol → spirodecan-6-ol
• SMILES stereocenters: [C@H]/[C@@H] deleted outright, breaking the SMILES string
• LaTeX: \left[\kappa\right] → \left_{M} (the symbol inside is lost)
• Plain math grouping: ln(2) = [ (T_1 - T_2) / (T1*T2)] → ln(2) = (the entire answer content)

Impact (found via downstream eval-data audit)

Audited all 198 gpqa_diamond rows against the raw Correct Answer/Incorrect Answer 1-3 fields:

• 12/198 questions had at least one corrupted answer choice.
• One row (ln(2) = [...]) had all four choices collapse to the identical string "ln(2) = " — unanswerable from the text alone. Worse, the pipeline's own ground-truth letter (all_choices.index(correct), also a by-value lookup) hit the exact same collision and silently defaulted to the wrong letter.
• Cross-checked gpqa_main (448 rows): 19 more answer choices with the same kind of content.
• Found zero genuine citation markers in either dataset — every match this regex ever fires on turns out to be real content it shouldn't touch.

Fix

Dropped the re.sub line in both files. .replace(" [title]", ". ") (a separate, targeted substitution for a known artifact from another benchmark's raw text) is left untouched.

This brings us in line with the current upstream reference these files cite: lm_eval/tasks/gpqa/zeroshot/utils.py no longer has the re.sub("\[.*?\]", "", text) line either — its preprocess() is exactly .strip() → .replace(" [title]", ". ") → .replace(" ", " "), i.e. what this PR changes our two files to match. The # adopted from comment in our files must predate that line's removal upstream.

Validation

• Reconstructed all 198 gpqa_diamond prompts from raw source columns with the fixed preprocess(); confirmed correct restoration against the corrupted downstream eval file for all 12 affected rows (11 via direct text matching, 1 via replaying the original random.shuffle(seed=42) sequence, cross-validated 100% against the 186 unaffected rows' known shuffle order and the 11 already-fixed rows' full A/B/C/D arrangement).
• python -m py_compile on both changed files.
• No dedicated data_preprocess test suite currently exists in this repo to run.

Co-Authored-By: Claude Sonnet 5 noreply@anthropic.com