Add broader thermochemistry benchmarks

README.md

@@ -159,12 +159,27 @@ the PSI4 VQM24 dataset:
 python scripts/validate_vqm24.py
 ```
 
+Validate zero-point energies across 255 model chemistries:
+
+```bash
+python scripts/validate_model_chemistry_zpe.py
+```
+
+Validate complete DFTB+ and GFN2-xTB entropy workflows against an experimental
+gas-phase compilation:
+
+```bash
+python scripts/validate_experimental_entropy.py --engine both
+```
+
 These commands verify their downloaded archives and keep the external data out
 of the repository. See the
 [anthraquinone reference benchmark](docs/benchmarks/anthraquinone_workflow.rst)
 and the thermochemistry benchmarks for
 [QM9](docs/benchmarks/qm9_thermochemistry.rst) and
-[VQM24 transition states](docs/benchmarks/vqm24_transition_states.rst) for
+[VQM24 transition states](docs/benchmarks/vqm24_transition_states.rst), plus
+the [experimental entropy](docs/benchmarks/entropy_accuracy.rst) and
+[model-chemistry ZPE](docs/benchmarks/model_chemistry_zpe.rst) comparisons, for
 their scope and interpretation.
 
 Run linting:

docs/benchmarks/entropy_accuracy.rst

@@ -1,94 +1,87 @@
-Gas-phase entropy accuracy
-==========================
+Experimental gas-phase entropy
+==============================
 
-Standard molar entropies S°(298.15 K, 1 bar) from ThermoScreening compared with
-experiment, for both engines. These are method accuracies (GFN2-xTB and
-DFTB3/3ob), not tool errors. The thermochemistry implementation itself matches
-ASE ``IdealGasThermo`` to within 0.0003 cal/(mol K).
+ThermoScreening is checked against the experimental entropy compilation
+published by Bains, Petkowski, Zhan, and Seager. The source combines 1,403
+reported entropy values in eight source columns, including NIST, JANAF, and
+Yaws.
 
-.. list-table::
-   :header-rows: 1
+The benchmark downloads the original workbook from
+`Zenodo record 4661783 <https://zenodo.org/records/4661783>`__, verifies its
+checksum, and keeps it outside the repository.
+
+Scope
+-----
+
+The comparison covers 29 neutral gas-phase molecules containing H, C, N, O, F,
+P, S, and Cl. It includes linear and nonlinear molecules, triplet oxygen,
+doublet nitric oxide, high-symmetry species, and flexible molecules with
+low-frequency torsions.
+
+For each molecule, the reference value is the median of the available
+experimental columns in the workbook. This avoids selecting a single
+transcription when the compilation contains conflicting entries, as it does
+for methane. Nitric oxide is the documented exception: its two entries differ
+by almost 10 J/(mol K), so the JANAF value is used. Values are converted from
+J/(mol K) to cal/(mol K).
+
+Both engines start from the corresponding ASE G2 geometry and run a complete
+optimization, Hessian, symmetry analysis, and quasi-RRHO thermochemistry
+calculation at 298.15 K and 1 bar. GFN2-xTB uses a force threshold of
+0.001 eV/A because methyl torsions remain sensitive at the normal screening
+tolerance.
+
+Run the benchmark
+-----------------
+
+DFTB+ and the 3ob parameter set are required for the default run:
+
+.. code-block:: bash
 
-   * - Molecule
-     - Symmetry number
-     - xTB (GFN2)
-     - DFTB+ 3ob
-     - Experiment
-   * - H₂O
-     - 2
-     - 45.08
-     - 45.07
-     - 45.1
-   * - CH₄
-     - 12
-     - 44.44
-     - 44.47
-     - 44.5
-   * - NH₃
-     - 3
-     - 45.96
-     - 46.00
-     - 46.0
-   * - N₂
-     - 2
-     - 45.79
-     - 45.76
-     - 45.8
-   * - CO₂
-     - 2
-     - 51.23
-     - 51.53
-     - 51.1
-   * - CO
-     - 1
-     - 47.23
-     - 47.14
-     - 47.2
-   * - CH₃OH
-     - 1
-     - 56.63
-     - 57.00
-     - 57.3
-   * - C₂H₆
-     - 6
-     - 54.38
-     - 54.54
-     - 54.8
-
-Values are in cal/(mol K).
-
-Deviation from experiment
--------------------------
+   python scripts/validate_experimental_entropy.py
+
+Run both independently implemented engines with:
+
+.. code-block:: bash
+
+   python scripts/validate_experimental_entropy.py --engine both
+
+The xTB comparison requires the optional ``tblite`` package. The script uses a
+separate working directory for every molecule and engine so fixed-name Hessian
+and vibration files cannot leak between cases.
+
+Results
+-------
 
 .. list-table::
    :header-rows: 1
 
    * - Engine
-     - MAD
+     - MAE
      - RMSD
      - Maximum absolute deviation
-     - Mean signed deviation
-   * - xTB (GFN2)
-     - 0.17
-     - 0.28
-     - 0.67 (CH₃OH)
-     - -0.13
-   * - DFTB+ 3ob
-     - 0.14
-     - 0.21
-     - 0.43 (CO₂)
-     - -0.04
-
-Both engines reproduce experimental gas-phase entropies to well under
-1 cal/(mol K).
-
-Notes
------
+   * - DFTB3/3ob
+     - 0.377
+     - 0.635
+     - 2.317 (ethanol)
+   * - GFN2-xTB
+     - 0.457
+     - 0.777
+     - 2.387 (ethanol)
+
+All deviations are in cal/(mol K). The regression gate requires an MAE no
+greater than 0.75 and a maximum deviation no greater than 3.0 cal/(mol K).
+
+Interpretation
+--------------
+
+This is an end-to-end method benchmark: it exercises geometry optimization,
+frequencies, automatic rotational symmetry numbers, open-shell handling,
+quasi-RRHO, and entropy assembly. The remaining deviations include the
+accuracy limits of DFTB3/3ob and GFN2-xTB and are not numerical errors in the
+thermochemistry formulas. Those formulas are separately tested against ASE
+``IdealGasThermo``.
 
-- Symmetry numbers are detected correctly for all eight molecules.
-- The values use ``pressure=100000`` (1 bar). The default pressure of 101325 Pa
-  raises the entropy by 0.026 cal/(mol K).
-- Residual error for methanol and ethane comes from treating low-frequency
-  torsions as harmonic oscillators; ``quasi_rrho=True`` softens this.
-- The Linux release gate runs a mandatory DFTB+ calculation. Broader xTB and
-  solvent checks remain optional integration tests.
+The source workbook is not redistributed. See the
+`dataset publication <https://doi.org/10.3390/data7030033>`__ for provenance,
+experimental-source details, and limitations.

docs/benchmarks/model_chemistry_zpe.rst

@@ -0,0 +1,62 @@
+Model-chemistry zero-point energies
+===================================
+
+The zero-point energy implementation is checked against the quantum-chemistry
+outputs published with the model-chemistry benchmark by Green, Wu, Doner, and
+Pang. The source spans density-functional, double-hybrid, perturbation, and
+composite methods across many basis sets.
+
+The validation script downloads ``qm_outputs.json`` from
+`Zenodo record 14733712 <https://zenodo.org/records/14733712>`__, verifies its
+checksum, and keeps the 55 MB source file outside the repository.
+
+Run the benchmark
+-----------------
+
+.. code-block:: bash
+
+   python scripts/validate_model_chemistry_zpe.py
+
+No external electronic-structure engine is needed. ThermoScreening recomputes
+the zero-point energy directly from each source frequency spectrum.
+
+Coverage
+--------
+
+The script scans all 41,492 frequency records across 255 model chemistries.
+Eleven source rows have fewer modes than their geometries require; their exact
+identities are pinned and excluded. The remaining 41,481 records must be
+structurally valid.
+
+A deterministic 1,000-record sample covers every one of the 255 model
+chemistries, evenly spaced records, and a seeded random remainder. The sample
+contains molecules with H, C, N, O, F, S, Cl, and Br and up to 26 atoms.
+
+Results
+-------
+
+.. list-table::
+   :header-rows: 1
+
+   * - Quantity
+     - Mean absolute deviation
+     - Maximum absolute deviation
+   * - Zero-point energy
+     - 0.345971 microhartree
+     - 1.459415 microhartree
+
+The regression gate is 0.5 microhartree for the mean and 2.0 microhartree for
+the maximum. The residual is consistent with constants and rounding in the
+source outputs.
+
+Interpretation
+--------------
+
+This benchmark validates frequency counting, unit conversion, and zero-point
+energy assembly over a much broader method and element matrix than QM9. It
+does not assess whether a particular electronic-structure method predicts an
+experimentally accurate frequency.
+
+See the
+`dataset publication <https://pubs.acs.org/doi/10.1021/acs.jpca.5c00615>`__
+for the model-chemistry definitions and scientific scope.

docs/index.rst

@@ -108,6 +108,7 @@ Features
    configuration
    api
    benchmarks/entropy_accuracy
+   benchmarks/model_chemistry_zpe
    benchmarks/anthraquinone_workflow
    benchmarks/qm9_thermochemistry
    benchmarks/vqm24_transition_states