Skip to content

Verify document scope parity when swapping from the legacy retrieval backend #40

Description

@iliia-veselov

What is the problem this feature will solve?

Deployments migrating from the legacy pgvector-based retrieval backend to this service need the searchable document set to be unchanged — same documents, same selection logic. That is not automatic, and it is not currently checked anywhere.

It is also the safety net for statgpt-backend #536 (removing the deprecated DIAL file RAG backend): if scope silently narrowed during the swap, that removal would take answers with it, and the change would look like a quality regression rather than a missing-document problem.

What is the proposed feature or solution?

  1. Enumerate the document set indexed under the legacy backend and the set indexed here.
  2. Diff them and account for every difference.
  3. Confirm selection logic — filters, document types, date handling — behaves equivalently.
  4. Record the result, and use it as an input to the retrieval regression evaluation so a scope difference is not misread as a quality difference.

Acceptance criteria

  • Both document sets are enumerated and diffed.
  • Every difference is explained or corrected.
  • Selection-logic equivalence is confirmed.
  • The result is written down and referenced from the regression evaluation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions