Skip to content

feat: add protein folding stability benchmark#730

Draft
lwalew wants to merge 1 commit into
ddmms:mainfrom
lwalew:feat/add-protein-folding-benchmark
Draft

feat: add protein folding stability benchmark#730
lwalew wants to merge 1 commit into
ddmms:mainfrom
lwalew:feat/add-protein-folding-benchmark

Conversation

@lwalew

@lwalew lwalew commented Jul 23, 2026

Copy link
Copy Markdown

Pre-review checklist for PR author

Summary

Adds the protein folding stability benchmark, migrated from the MLIP Audit suite. For a small set of proteins (chignolin, tryptophan cage, and an orexin/hypocretin fragment) an NVT molecular dynamics simulation is run at 300 K starting from the native folded reference conformation, and the model's ability to keep each protein folded is measured along the trajectory.

Reported metrics:

  • RMSD — C-alpha RMSD from the native reference structure, averaged over the trajectory and across proteins (lower is better).
  • TM score — TM score of each frame against the native reference, averaged over the trajectory and across proteins (closer to 1 is better).
  • Rgyr deviation — maximum absolute deviation of the radius of gyration from the initial folded state, across proteins (lower is better).

The interactive app shows a line plot of the RMSD from the reference structure along the trajectory, averaged across proteins, for each model.

This benchmark introduces a new top-level biomolecules category (app tab, category YAML, and docs page). It also adds the shared mlipaudit wiring (optional extra, grace conflict entry, and [tool.uv.sources] git pin to the mlpeg-migration branch) plus the mlip_audit framework entry. This wiring is shared with other in-flight MLIP Audit migration PRs (e.g. ring planarity, water RDF, protein sampling), so overlap on pyproject.toml, frameworks.yml, and the biomolecules category files is expected and will be resolved at merge.

Linked issue

Resolves #725

Progress

  • Calculations
  • Analysis
  • Application
  • Documentation

Testing

End-to-end model runs are pending the upload of the input data to S3 (inputs/biomolecules/protein_folding_stability/protein_folding_stability.zip). All pre-commit hooks (ruff check, ruff format, numpydoc-validation, file checks) pass locally, and the new modules compile cleanly. The implementation follows the established MLIP Audit migration pattern (see the ring planarity and water RDF benchmarks).

New decorators/callbacks

None. The RMSD trajectory line plot reuses the existing @plot_scatter decorator (show_line=True, show_markers=False) and the plot_from_table_column callback.

@ElliottKasoar ElliottKasoar added external Contribution from external framework new benchmark Proposals and suggestions for new benchmarks labels Jul 23, 2026
@lwalew
lwalew force-pushed the feat/add-protein-folding-benchmark branch from 492b66e to aa89298 Compare July 23, 2026 14:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external Contribution from external framework new benchmark Proposals and suggestions for new benchmarks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

New benchmark: protein folding

2 participants