Repository navigation
Python: chroma collection_exists catches too much, ensure_collection_deleted too little - #14510
Eric Del Orefice (ericdelorefice) wants to merge 4 commits into
Conversation
…deleted too little pyproject allows chromadb >= 0.5, < 1.6, and the error for a missing collection is not the same across that range. 0.5 has no NotFoundError at all and raised ValueError; 1.5.9 raises chromadb.errors.NotFoundError, which is not a ValueError. Both symptoms follow from that one fact. collection_exists catches bare Exception, so an unreachable server or a rejected request reports that the collection is not there. Three places already say that is not the rule: _get_collection nine lines above raises RuntimeError from the identical client.get_collection call, ensure_collection_deleted raises VectorStoreOperationException for anything that is not a missing collection, and VectorStore.collection_exists in data/vector.py narrows to except VectorStoreOperationException when it wraps this method. ensure_collection_deleted catches only ValueError, the 0.5-era signal, so on chromadb 1.x a genuinely absent collection is re-raised as a VectorStoreOperationException and the "doesn't exist" path is unreachable. One module-level tuple resolves the version difference, behind a guarded import so chromadb 0.5 keeps working. collection_exists narrows to it and ensure_collection_deleted widens to it. Four tests, no server. They resolve the not-found error themselves rather than importing the new constant, so the file runs against the code before and after. Two fail on main, one per symptom, and two pass there.
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Modern Chroma still treats unrelated ValueError exceptions as missing collections.
Review effort: Balanced
Findings: 1
What changed in this PR
Updates Chroma missing-collection handling across supported versions.
Changes:
- Adds version-compatible not-found exception handling.
- Adds tests for missing collections and connection failures.
| File | Description |
|---|---|
python/semantic_kernel/connectors/chroma.py |
Narrows exception handling for lookup and deletion. |
python/tests/unit/connectors/memory/test_chroma.py |
Tests not-found and operational failure behavior. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Review point taken. Keeping ValueError in the tuple alongside NotFoundError reinstated the broad-catch symptom on modern Chroma in a narrower form, and in ensure_collection_deleted it is worse than untidy: an unrelated ValueError would be logged as "could not be deleted because it doesn't exist" and the delete silently skipped. That is the defect this PR is about, introduced by the fix for it. COLLECTION_NOT_FOUND is now (NotFoundError,) where the import succeeds and (ValueError,) only on chromadb < 1.0, where ValueError was the signal. Two tests pin it, skipped on chromadb < 1.0 where a ValueError genuinely does mean absent. Against this branch's previous commit both fail, the second with the captured log showing the swallow. Against upstream all four error-path tests fail, two per symptom.
There was a problem hiding this comment.
This is a community review and does not clear the merge gate. I ran this on Windows against upstream main at 64005c7, with real chromadb 1.5.9 for the behavior checks.
The fix does what it says on chromadb 1.x. I have one behavior change to flag and one piece of code that cannot run.
What I ran with a real in-process Chroma client (chromadb.EphemeralClient()):
- On main,
ensure_collection_deleted()on a collection that does not exist raisesVectorStoreOperationException: Failed to delete collection nope .... On the branch it is tolerated.get_collectionon a missing name raiseschromadb.errors.NotFoundError, which is not aValueError(checked), so main'sexcept ValueErrorcould never match on 1.x. collection_exists()on a missing collection returnsFalseon both main and the branch.
- Behavior change at the store level.
VectorStore.collection_existsindata/vector.pyonly catchesVectorStoreOperationException, and this PR re-raises the raw exception. With a mocked client whoseget_collectionraisesConnectionError("connection refused"),ChromaStore(client=client).collection_exists("c")returnsFalseon main and raisesConnectionErroron the branch. That may be what you want, and the PR text says so. But it is a different contract from the Cosmos change in #14515, which wraps inVectorStoreOperationException(and so still reads as absent at the store level). It would be good to pick one on purpose. If you want a typed error, wrap it. If you want it raw, a short comment would help. - The
except ImportErrorfallback cannot run. The connector's own line 11 importschromadb.api.collection_configuration, and I installed chromadb 0.6.3 and checked that this module does not exist there: importingsemantic_kernel.connectors.chromafails withModuleNotFoundError. So the connector does not import on 0.6.3. I did not check each version between 0.6.3 and 1.0, so I cannot say exactly where the import starts working. Because of that, on the versions I checked the(ValueError,)branch,COLLECTION_NOT_FOUND_IS_VALUE_ERRORand the twoskipiftests never execute. So thechromadb >= 0.5pin inpyproject.tomlthat the PR text relies on does not hold for this connector, at least on 0.6.3. I would drop the fallback, or leave it and say it is defensive.
I did not run the new unit tests myself, only the probes above, and I did not test against an HTTP Chroma server, so the exception type for a server-side 404 is unverified.
The connector only imports on chromadb 1.x (it needs chromadb.api.collection_configuration), so the ValueError fallback for chromadb < 1.0 could never run; it is removed, with the two tests it gated. collection_exists now wraps any other error in VectorStoreOperationException, matching ensure_collection_deleted here and the Cosmos NoSQL change in microsoft#14515.
|
Thank you for testing this against a real Chroma client, and for checking 0.6.3. Both points are right. The fallback could never run. Line 11 imports On the contract, I agree it should be one choice, and I've taken the one from #14515: any other error in Your review also surfaced that the The Chroma tests pass on the branch, the two changed ones fail against the previous commit, and ruff 0.9 is clean. 🤖 Addressed by Claude Code |

Motivation and Context
ChromaCollectionhas two handlers for the same condition, and the error type they were written against changed underneath them.pyproject.tomlallowschromadb >= 0.5, < 1.6. Across that range the error for a missing collection is not the same:Both symptoms follow from that one fact, and they point in opposite directions.
collection_existscatches too much.except Exception: return False, so an unreachable server, a rejected request or a timeout all report that the collection is not there. Three places in this repo already say that is not the rule._get_collection, nine lines above, raisesRuntimeErrorfrom the identicalclient.get_collection(...)call.ensure_collection_deleted, in the same class, raisesVectorStoreOperationExceptionfor anything that is not a missing collection. AndVectorStore.collection_existsindata/vector.pywraps this method and narrows toexcept VectorStoreOperationException, so the layer above already assumes only a recognised failure becomesFalse.ensure_collection_deletedcatches too little. Itsexcept ValueErroris the 0.5-era signal. On chromadb 1.x a genuinely absent collection raisesNotFoundError, which is not aValueError, so it falls through toexcept Exceptionand is re-raised asVectorStoreOperationException. The "could not be deleted because it doesn't exist" path is unreachable on any chromadb 1.x, which is most of the supported range.Description
One module-level tuple resolves the version difference once:
collection_existsnarrows fromExceptionto that tuple.ensure_collection_deletedwidens fromValueErrorto the same tuple. Nothing else changes, and the guarded import keepschromadb 0.5working, whereNotFoundErrordoes not exist.Contribution Checklist
Four unit tests, no Chroma server. They resolve the not-found error themselves rather than importing the new constant, so the same file runs against the code before and after this change.
Against unmodified
main, two fail - one per symptom - and two pass:The two that pass on
mainpin what must not move: a not-found still returnsFalse, and a real failure during delete still raises.Note
This is the connector I named as the correct one in #14473, and it is correct in
ensure_collection_deleted, which was the method that issue was about.collection_existsunderneath it is not, which I only saw when I went back to check the claim.RedisCollection.collection_existshas the broad-catch half of this and is already covered by #14474.CosmosNoSqlCollection.collection_existscatchesCosmosHttpResponseError, which includes 401, 403 and 5xx, so it may want the same narrowing - left out here because I have not read it closely enough to say.