Skip to content

test: add unit tests for memory search ranking - #13

Open
by22Jy wants to merge 1 commit into
ben4mn:mainfrom
by22Jy:test/memory-search-ranking
Open

test: add unit tests for memory search ranking#13
by22Jy wants to merge 1 commit into
ben4mn:mainfrom
by22Jy:test/memory-search-ranking

Conversation

@by22Jy

@by22Jy by22Jy commented Mar 20, 2026

Copy link
Copy Markdown

Summary

Adds comprehensive unit tests for the memory search ranking functionality in backend/services/memory_service.py.

Changes

  • Vector similarity scoring tests - Test combined ranking formula with 70% vector weight
  • BM25 keyword matching tests - Test keyword search integration
  • Combined ranking tests - Test 70/30 weight distribution and importance multiplier
  • Temporal boost tests - Test recency and frequency effects on ranking
  • Tier computation tests - Test observation/belief/knowledge tier progression
  • Memory dataclass tests - Test Memory object creation and defaults

Test Coverage

TestComputeTemporalBoost (6 tests)

  • test_temporal_boost_timeless_stays_near_one - Timeless memories should not decay
  • test_temporal_boost_temporary_decays_with_age - Temporary memories should decay when stale
  • test_temporal_boost_evolving_considers_frequency - Evolving memories consider both recency and frequency
  • test_temporal_boost_bounds - Boost should always be in [0.7, 1.3] range
  • test_temporal_boost_none_last_accessed - Handle None last_accessed (treat as very old)

TestComputeTier (4 tests)

  • test_tier_observation_for_low_count - Low reinforcement count → observation tier
  • test_tier_belief_for_medium_count - Medium reinforcement count → belief tier
  • test_tier_knowledge_for_high_count - High reinforcement count → knowledge tier
  • test_tier_progression - Tier should progress as reinforcement increases

TestMemoryDataclass (2 tests)

  • test_memory_creation - Create a Memory with all fields
  • test_memory_defaults - Create a Memory with default values

TestRetrieveMemoriesRanking (3 tests)

  • test_combined_ranking_70_30_weights - Test 70% vector + 30% keyword weight formula
  • test_importance_affects_ranking - Higher importance increases score
  • test_tier_boost_affects_ranking - Tier multipliers affect final score

TestSearchMemoriesWithFilters (3 tests)

  • Placeholders for integration tests with filters

TestFindSimilarMemories (2 tests)

  • Placeholders for similarity search integration tests

Technical Details

Formula tested:

base_score = (0.7 * vector_score + 0.3 * keyword_score) * (1 + importance * 0.2)
combined_score = base_score * temporal_boost * tier_multiplier

Tier progression:

  • 0-1 reinforcements → observation (1.0x multiplier)
  • 2-4 reinforcements → belief (1.1x multiplier)
  • 5+ reinforcements → knowledge (1.2x multiplier)

Temporal boost range: [0.7, 1.3]

  • Timeless: stays ~1.0
  • Temporary: decays to 0.7 when stale
  • Evolving: moderate blend of recency + frequency

How to Run

pytest tests/unit/test_memory_service.py -v

Related Issue

Closes #8

Checklist

  • Tests follow project structure
  • All test functions have clear docstrings
  • Tests cover all three ranking aspects (vector, keyword, combined)
  • Tests verify temporal boost and tier computation logic
  • No external dependencies required (all tests use pure Python logic)

- Add tests for vector similarity scoring
- Add tests for BM25 keyword matching
- Add tests for combined ranking with 70/30 weights
- Add tests for temporal boost calculation
- Add tests for tier computation

Closes ben4mn#8

@ben4mn ben4mn left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, thanks for putting this together! The TestComputeTemporalBoost and TestComputeTier tests are really solid — they exercise the actual functions with good edge case coverage, and the bounds checking is a nice touch.

A couple things I'd love to see adjusted before merging:

Ranking tests need to hit real code. The three tests in TestRetrieveMemoriesRanking currently hardcode the formula and verify the arithmetic in isolation — they never actually call into memory_service. If the formula changes in the source, these tests would still pass. Could you rework these to call the real functions (mocking the DB layer is totally fine)?

Placeholder tests. The pass-only tests in TestSearchMemoriesWithFilters and TestFindSimilarMemories would be better removed for now. Empty tests tend to give a false sense of coverage, and we can always add them in a follow-up when the integration test setup is ready.

Happy to re-review once those are addressed. Thanks again for the contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add unit tests for memory search ranking

2 participants