Add a tei-reranker service (cross-encoder) to the qq memory stack

[OriNachum]

Context

The qq memory stack runs one TEI container (qq-tei, services/embeddings) serving the embedding model Qwen/Qwen3-Embedding-0.6B on :8101. There is no reranker in the RAG loop — retrieval goes MongoDB + Neo4j → straight to the Nemotron LLM, with no cross-encoder re-scoring step. TEI serves rerankers too (cross-encoders, /rerank endpoint), so adding one is a second TEI container, not a new tool.

Ask

Add a tei-reranker service that serves a cross-encoder reranker via TEI's /rerank, and wire it into the memory retrieval pipeline to re-score candidates before the LLM answer step.

Compose block (mirror the existing tei)

  tei-reranker:
    build: ./services/embeddings        # same image; Qwen3-Reranker support is recent, hence a custom build
    container_name: qq-tei-reranker
    restart: unless-stopped
    ports:
      - "8102:80"
    environment:
      - MODEL_ID=Qwen/Qwen3-Reranker-0.6B   # or BAAI/bge-reranker-v2-m3
    volumes:
      - ./data/tei-reranker:/root/.cache/huggingface
    healthcheck:
      test: ["CMD", "python", "-c", "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://localhost:80/health').status==200 else 1)"]
      interval: 10s
      timeout: 5s
      retries: 5

Wiring

• Add a reranker section to config.json (base_url: http://localhost:8102, model: …) alongside embeddings.
• In the retrieval path (memory_context.py / the Mongo+Neo4j candidate gather), POST the (query, candidates) to /rerank, keep top-K by score, then build the LLM prompt from the reranked set.
• Extend check_services.py to health-check :8102.

Notes

• Same warmup-once profile as the embedder (~1s cold, ~0.2s warm) — model loads at container start.

• 0.6B cross-encoder is cheap; only re-scores the already-retrieved shortlist, so it adds little latency.

• The genuinely slow leg remains the Nemotron-30B answer step (:8000), not embed/rerank.

• Claude