test: add unit tests for memory search ranking - #13
Conversation
- Add tests for vector similarity scoring - Add tests for BM25 keyword matching - Add tests for combined ranking with 70/30 weights - Add tests for temporal boost calculation - Add tests for tier computation Closes ben4mn#8
ben4mn
left a comment
There was a problem hiding this comment.
Hey, thanks for putting this together! The TestComputeTemporalBoost and TestComputeTier tests are really solid — they exercise the actual functions with good edge case coverage, and the bounds checking is a nice touch.
A couple things I'd love to see adjusted before merging:
Ranking tests need to hit real code. The three tests in TestRetrieveMemoriesRanking currently hardcode the formula and verify the arithmetic in isolation — they never actually call into memory_service. If the formula changes in the source, these tests would still pass. Could you rework these to call the real functions (mocking the DB layer is totally fine)?
Placeholder tests. The pass-only tests in TestSearchMemoriesWithFilters and TestFindSimilarMemories would be better removed for now. Empty tests tend to give a false sense of coverage, and we can always add them in a follow-up when the integration test setup is ready.
Happy to re-review once those are addressed. Thanks again for the contribution!
Summary
Adds comprehensive unit tests for the memory search ranking functionality in
backend/services/memory_service.py.Changes
Test Coverage
TestComputeTemporalBoost (6 tests)
test_temporal_boost_timeless_stays_near_one- Timeless memories should not decaytest_temporal_boost_temporary_decays_with_age- Temporary memories should decay when staletest_temporal_boost_evolving_considers_frequency- Evolving memories consider both recency and frequencytest_temporal_boost_bounds- Boost should always be in [0.7, 1.3] rangetest_temporal_boost_none_last_accessed- Handle None last_accessed (treat as very old)TestComputeTier (4 tests)
test_tier_observation_for_low_count- Low reinforcement count → observation tiertest_tier_belief_for_medium_count- Medium reinforcement count → belief tiertest_tier_knowledge_for_high_count- High reinforcement count → knowledge tiertest_tier_progression- Tier should progress as reinforcement increasesTestMemoryDataclass (2 tests)
test_memory_creation- Create a Memory with all fieldstest_memory_defaults- Create a Memory with default valuesTestRetrieveMemoriesRanking (3 tests)
test_combined_ranking_70_30_weights- Test 70% vector + 30% keyword weight formulatest_importance_affects_ranking- Higher importance increases scoretest_tier_boost_affects_ranking- Tier multipliers affect final scoreTestSearchMemoriesWithFilters (3 tests)
TestFindSimilarMemories (2 tests)
Technical Details
Formula tested:
Tier progression:
Temporal boost range: [0.7, 1.3]
How to Run
Related Issue
Closes #8
Checklist