fix(suggest): gate out-of-dataset species matches#31
Merged
Conversation
The species suggester ranked every candidate by Levenshtein distance and returned the top matches unconditionally. For a name absent from the bacteria-only collection (e.g. the archaeal "Sulfolobus acidocaldarius"), that surfaced the nearest-spelled bacterium as a "Did you mean", which only misled the user (issue #25). Add a plausibility gate in Suggest: a candidate qualifies only as a substring match either way, or within a quarter of the longer name's rune length in edit distance. Typos and GTDB clade renames pass; unrelated names are dropped. Edit distance alone cannot tell a same-genus different-species name from a different-genus similar name, so distant same-genus matches (P. aeruginosa for a P. fluorescens query) are dropped too, in favour of coherent near-spellings only. When nothing is close, the query command now states that no close match exists in a bacteria-only collection and points at --species-like to browse, instead of staying silent.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Issue #25: species queries for names outside AllTheBacteria returned a
misleading "Did you mean" hint. The suggester ranked candidates by
Levenshtein distance and always returned the top matches, so an archaeal
name like "Sulfolobus acidocaldarius" (absent from this bacteria-only
collection) surfaced the nearest-spelled bacterium
("Alicyclobacillus acidocaldarius") as if it were a match.
Change
internal/suggest: add a plausibility gate toSuggest. A candidatequalifies only as a substring match either way (catching typed
prefixes and GTDB clade suffixes like "Pseudomonas_E fluorescens"), or
when its edit distance is within a quarter of the longer name's rune
length. Legitimate typos and GTDB renames pass; out-of-dataset names
are dropped.
internal/cli: when nothing is close,printSpeciesSuggestionsnowstates that no close match exists in a bacteria-only collection and
points at
--species-liketo browse, instead of staying silent.Tradeoff (worth a reviewer's eye)
Edit distance cannot distinguish "same genus, different species" from
"different genus, similar spelling" - both differ by a whole word.
Measured, "Pseudomonas aeruginosa" (0.435 normalised distance) is
farther from a "Pseudomonas fluorescens" query than the archaeal false
positive "Alicyclobacillus acidocaldarius" (0.355) is from "Sulfolobus
acidocaldarius". So no single threshold can keep distant same-genus
suggestions while rejecting the archaeal noise. The gate deliberately
admits only near-spellings (GTDB renames, typos) and drops the rest - a
coherent "suggest what you probably misspelled, otherwise say it is not
here". Same-genus different-species names are no longer offered.
Test
Test-first (TDD), each watched RED before implementing:
TestSuggestRejectsOutOfDataset: the archaeal query returns nosuggestions (was RED: returned the nearest bacterium).
TestPrintSpeciesSuggestionsOutOfDataset: the out-of-dataset messageand
--species-likebrowse hint are printed, with no "Did you mean".TestPrintSpeciesSuggestionsupdated: the distant same-genus name isno longer offered.
Validation
go build ./...andgo vet ./internal/suggest/ ./internal/cli/cleango test ./...passesmake docsidempotent (no generated-reference drift), so the docsstaleness gate stays green
Scope
suggest.Suggest(one caller) andprintSpeciesSuggestions(twocallers, both stderr). No signatures changed; the behaviour change is
confined to the zero-result species path.