[MrOlm]

Summary

Adds an opt-in --primary_skani_min_af_both flag that stops plasmid datasets from collapsing into one giant primary cluster (#311). Also bumps the version to 4.0.3, which publishes to PyPI automatically on merge.

The problem. A pair becomes a primary-clustering edge if its aligned fraction passes --primary_skani_min_af in either direction. Plasmids share mobile elements, and a small plasmid that is mostly a transposon or resistance cassette aligns ~100% to every larger plasmid carrying it. Under single linkage, a few of those plasmids chain thousands of unrelated plasmids together.

In #311, 27,237 Acinetobacter plasmids ended up in 1 primary cluster. Secondary clustering then built every pair in it: 741,854,169 = 27,237² Ndb rows. That step is single-threaded and was still running after 22 h at 350 GB of RAM.

The fix. --primary_skani_min_af_both requires the aligned fraction in both directions. This doesn't change the final clusters, as long as --primary_skani_min_af ≤ 100 × --cov_thresh and -pa ≤ -sa:

• Secondary clustering zeroes the ANI of any direction aligning ≤ --cov_thresh, then averages the two directions.
• So average linkage can only join two groups if at least one pair between them is within -sa and aligns in both directions.
• Any such pair is a primary edge under the new rule, so primary clustering never separates genomes that secondary would group.

The flag is opt-in because the either-direction 15% default was tuned to reproduce the Mash partition on UHGG.

Changes

• drep/d_cluster/union_find.py: cluster_edges(..., cov_both=False), plus a vectorized _reverse_coverage that looks up the other direction's aligned fraction. A pair with only one direction present fails the both-directions check.
• drep/d_cluster/compare_utils.py: passes the flag through, and the log line says "in both directions".
• drep/argumentParser.py: adds the --primary_skani_min_af_both flag.
• Docs:
  • docs/module_descriptions.rst gets the new flag in both -h dumps.
  • docs/choosing_parameters.rst gets a plasmid paragraph in the non-bacterial section.
• CHANGELOG.md: 4.0.3 entry. It also records that releases now publish automatically and that 4.0.1 never reached PyPI.
• drep/VERSION: 4.0.3.

Testing

• New unit test test_cov_both_stops_small_genomes_chaining_clusters. A small "transposon" plasmid chains two families into one cluster by default, but not with cov_both. A pair with only one direction present is not an edge.
• End to end on 215 simulated plasmids: 8 unrelated families of 25 sharing an 8 kb transposon, plus 15 small plasmids made mostly of that transposon. With the exact command from dRep becomes very slow when dereplicating a large plasmid dataset #311:

	primary clusters	Ndb rows	final clusters / winners vs. original
original command	1	46,225	—
+ --primary_skani_min_af_both	9	5,225	identical / identical
+ -pa 0.98 --primary_skani_min_af 50 --primary_skani_min_af_both	9	5,225	identical / identical
-pa 0.98 --primary_skani_min_af 50 without the new flag	1	46,225	identical

• Local tests/tests: 85 passed, 12 skipped, 4 failed, all for reasons unrelated to this change:
  • test_goANI, test_goANI2 and test_dereplicate_3 need nsimscan, which isn't installed locally.
  • test_drep_scaffold_level fails the same way on master locally.
• CI on the branch push: the Tests workflow passed.

🤖 Generated with Claude Code

https://claude.ai/code/session_013UpcPFwkh6QMusnD6ApThL

---

Generated by Claude Code