Harden LLM benchmark stability for v0.2.2

[HuRuilizhen]

Summary

This PR hardens the local benchmark path for v0.2.2, with a focus on making both cloud-backed and local-model LLM benchmark runs stable enough to complete the same 100-sample evaluation workflow.

The main goal is to eliminate avoidable benchmark instability without changing the core public pipeline API.

Included

• hardens benchmark defaults for local Ollama runs

  • uses a conservative default batch size of 1 for ollama when the benchmark caller does not explicitly override --batch-size
  • keeps the existing cloud-provider default batch size unchanged

• adds optional benchmark artifact controls

  • allows benchmark runs to opt into LLM extraction artifact recording for debugging
  • supports explicit artifact directories and opt-in recording of raw response text and paragraph content
  • keeps artifact recording disabled by default for normal benchmark runs

• improves local-model JSON recovery in the LLM extractor

  • adds a narrow recovery path for lightly malformed single-paragraph local-model output
  • keeps the recovery logic scoped so it does not broaden into arbitrary JSON repair

• adds focused regression coverage

  • covers provider-aware benchmark batch-size defaults
  • covers benchmark artifact configuration wiring
  • covers malformed single-paragraph JSON recovery in LLMConceptExtractor

Validation

Repository checks:

• ruff check .
• pyright
• pytest -q

Targeted checks:

• pytest -q tests/test_benchmark.py tests/test_llm_extraction.py

Manual benchmark verification:

• reran the 100-sample TechQA benchmark input with heuristic, spacy, llm:ollama:qwen3.5:4b, and llm:mistral:mistral-small
• confirmed that both the local Ollama path and the Mistral path now complete the same 100-sample benchmark workflow

Notes

• this PR is a benchmark-stability follow-up, not a release-prep PR
• benchmark inputs and outputs remain local development assets under experiment/
• the next step after this branch is to move into release/0.2.2-prep

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f499426572

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".