Switch to Using Heavy Matches Heavy Atom mappings

[badisa]

Warning

Note that in #9 there was an issue with the OpenFE preparation, so it is different in notable ways to the previous sets.

Summary

Overall the total nanoseconds simulated changed from +2% nanoseconds to -30% nanoseconds simulated. The average reduction in nanoseconds appears to be about 10%.

The overall rank correlation held out for larger datasets, but suffered for the smaller sets which are more sensitive to run to run variability

One such example is Factor Xa, where a minor change in the prediction swaps the ranking of two compounds and produces a -1 correlation

The one set that stood out from this was BACE, which is covered in more detail below.

BACE

Looking through the results the BACE dataset stood out as a set that got worse. To investigate, multiple replicates were run with and without the new maps to see how the performance changes. Despite a significant change to the previous BACE results, the replicates with the old and new maps produced similar results.

Reference (5 replicates)	Heavy matches Heavy (6 replicates)

Looking closer it became apparent that the preparation of the datasets from OpenFE was wrong and tearing out more than intended. In the case of BACE there are crystal waters that were removed. Re-running the performance looks more in line with previous references

Reference (5 replicates)	Heavy matches Heavy (3 replicates)

The performance of the heavy matches heavy is still slightly worse than the previous atom mapping, but for consistency will stay with the new map. The atom mapping protocol may be useful to tweak for different systems, which is already an expectation.

Next Steps

• Fix the preparation issues and re-run the benchmarks.