Skip to content

feat: add protein sampling benchmark#729

Draft
lwalew wants to merge 1 commit into
ddmms:mainfrom
lwalew:feat/add-protein-sampling-benchmark
Draft

feat: add protein sampling benchmark#729
lwalew wants to merge 1 commit into
ddmms:mainfrom
lwalew:feat/add-protein-sampling-benchmark

Conversation

@lwalew

@lwalew lwalew commented Jul 23, 2026

Copy link
Copy Markdown

Pre-review checklist for PR author

  • I've confirmed the contribution guidelines.
  • I have reviewed and understand all AI-generated code in this PR.
  • I have added human-written tests for the new logic.
  • I have properly cited any upstream algorithms or libraries the AI utilized.
  • I have disclosed significant AI tool usage in the PR description.

Summary

Migrates the protein sampling benchmark from the MLIP Audit suite into ml-peg,
introducing a new top-level biomolecules category.

The benchmark runs short molecular dynamics simulations for a set of small proteins
(chignolin 1UAO, Trp-cage 2JOF, orexin beta 1CQ0), collects the sampled backbone
(phi/psi) dihedral angles, and compares their distributions to reference
distributions. Three metrics are reported (all lower-is-better, averaged over residue
types and stable systems):

  • Backbone Dihedral RMSD — RMSD between sampled and reference backbone dihedral
    distributions.
  • Backbone Hellinger Distance — Hellinger distance between the same distributions.
  • Backbone Outliers Ratio — fraction of sampled backbone dihedrals lying far from
    any reference data point.

This is a table-only benchmark (no primary plot), consistent with the migration spec.

The underlying SamplingBenchmark logic lives in the pinned
MLIP Audit package (mlpeg-migration
branch); ml-peg wires it up through a thin MlPegSamplingBenchmark adapter that
disables skip_if_elements_missing (ml-peg's ASE calculators do not expose the set of
supported elements).

Notes for reviewers

  • This PR adds the new biomolecules category (app YAML, docs category page, docs
    toctree entry).
  • It also adds the shared mlipaudit wiring (pyproject.toml optional extra +
    conflict + tool.uv.sources, and the mlip_audit framework entry in
    frameworks.yml). This wiring overlaps with other in-flight MLIP Audit migration
    PRs
    (e.g. ring planarity, protein folding) and is expected to be reconciled at
    merge.

Linked issue

Resolves #726

Progress

  • Calculations
  • Analysis
  • Application
  • Documentation

Testing

Static/local validation only so far: all new Python modules compile and the full
pre-commit suite passes (ruff-check, ruff-format, numpydoc-validation, etc.).
End-to-end runs across models are pending upload of the input data to S3
(inputs/biomolecules/protein_sampling/protein_sampling.zip), which has not been
performed as part of this PR.

New decorators/callbacks

None. Uses the existing build_table decorator and BaseApp table-only layout.

@ElliottKasoar ElliottKasoar added external Contribution from external framework new benchmark Proposals and suggestions for new benchmarks labels Jul 23, 2026
@lwalew
lwalew force-pushed the feat/add-protein-sampling-benchmark branch from 41a533e to aae2a03 Compare July 23, 2026 14:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external Contribution from external framework new benchmark Proposals and suggestions for new benchmarks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

New benchmark: protein sampling

2 participants