Skip to content

Close validation gaps: broaden cross-checks, benchmark, and document - #69

Merged
galjos merged 1 commit into
mainfrom
close-validation-gaps
Jul 6, 2026
Merged

Close validation gaps: broaden cross-checks, benchmark, and document#69
galjos merged 1 commit into
mainfrom
close-validation-gaps

Conversation

@galjos

@galjos galjos commented Jul 5, 2026

Copy link
Copy Markdown
Member

Follow-up to the end-to-end validation (which found no bugs) — turns each remaining coverage gap into a durable artifact. No behaviour changes: every source edit is a docstring. Every reference number was produced by a real calculation and adversarially re-verified.

CI-safe tests (no backend)

  • test_heat_capacity_cv.py (new) — settles the Cv-vs-Cp question: the reported heat capacity is the ideal-gas constant-volume Cv, cross-checked non-circularly against the closed-form Cv and pinned values (also confirmed tool Cv == native-xtb Cp − R).
  • test_S_H_G_match_ase — broadens the ASE IdealGasThermo cross-check from entropy-only/6 molecules to S + H + G across 14 (incl. charged NH₄⁺/OH⁻ and larger benzene/ethane/methanol), masses pinned, agreement < 2e-3.
  • test_quasi_rrho_matches_independent_grimme_and_is_invariant — qRRHO vs an independent hand-coded Grimme formula; confirms enthalpy and Cv are invariant under quasi_rrho.
  • test_dftb_setup.py — solvent→GBSA param plumbing for water/dmso/acetonitrile/thf.

Skippable integration tests (need dftb+/tblite)

  • xTB and DFTB+ gas-phase entropy vs experiment (H₂O/CH₄/N₂, within 2 cal/mol/K) — closes the "no experimental accuracy" gap.
  • qRRHO end-to-end on a real floppy molecule (n-octane, embedded geometry) with engine-generated sub-100 cm⁻¹ modes (28/59/71) → ΔS ≈ −4 cal/mol/K.
  • same-machine reproducibility for both engines.
  • DFTB+ solvation across dmso/thf/water: GBSA block applied, dmso/thf stabilise, and the lowest frequency shifts (solvation enters the Hessian).

Docs

  • total_heat_capacity now documents Cv vs Cp; dftbplus_thermo/xtb_thermo get reproducibility + solvation-accuracy notes; Thermo pressure unit corrected to Pa.
  • docs/benchmarks/entropy_accuracy.md records the measured accuracy vs experiment (xTB MAD 0.17, DFTB+ 3ob MAD 0.14 cal/mol/K).

Validation

Offline suite 241 passed / 10 skipped; with real DFTB+ + tblite 251 passed / 0 skipped. Source changes are docstring-only, so codecov/patch has no new coverable lines.

Follow-up to the end-to-end validation, which found no bugs but left coverage
gaps. Turns each gap into a durable artifact; no behaviour changes (all source
edits are docstrings).

CI-safe tests (no backend):
- test_heat_capacity_cv.py: the reported heat capacity is the ideal-gas Cv,
  cross-checked non-circularly against the closed-form Cv and pinned values.
- test_thermo.py::test_S_H_G_match_ase: broaden the ASE IdealGasThermo
  cross-check from entropy-only/6 molecules to S + H + G over 14 (incl. charged
  NH4+/OH- and larger benzene/ethane/methanol), masses pinned, within 2e-3.
- test_thermo.py::test_quasi_rrho_matches_independent_grimme_and_is_invariant:
  qRRHO vs an independent hand-coded Grimme formula; enthalpy and Cv invariant.
- test_dftb_setup.py: solvent -> GBSA param plumbing for water/dmso/mecn/thf.

Skippable integration tests (need dftb+/tblite):
- xTB and DFTB+ gas-phase entropy vs experiment (H2O/CH4/N2, within 2 cal/mol/K).
- qRRHO end-to-end on a real floppy molecule (n-octane) with engine-generated
  sub-100 cm^-1 modes.
- same-machine reproducibility for both engines.
- DFTB+ solvation across dmso/thf/water: GBSA block applied, stabilisation, and
  a frequency shift (solvation enters the Hessian).

Docs: total_heat_capacity documents Cv vs Cp; api reproducibility + solvation
caveats; Thermo pressure unit corrected to Pa; docs/benchmarks/entropy_accuracy.md
records the measured accuracy vs experiment (xTB MAD 0.17, DFTB+ 3ob MAD 0.14).
@codecov

codecov Bot commented Jul 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.57%. Comparing base (fd6cac2) to head (724ae84).

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #69   +/-   ##
=======================================
  Coverage   96.57%   96.57%           
=======================================
  Files          23       23           
  Lines        1459     1459           
=======================================
  Hits         1409     1409           
  Misses         50       50           
Flag Coverage Δ
unittests 96.57% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@galjos
galjos merged commit 168ed3a into main Jul 6, 2026
3 checks passed
@galjos
galjos deleted the close-validation-gaps branch July 6, 2026 09:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant