You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Once a local, offline vision-language model provider exists (#18), add an optional stage that runs it in parallel with the primary cloud provider
(Claude/GPT/Gemini) and produces a consensus / second-opinion comparison:
where both agree, confidence is reinforced; where they diverge, the report flags
it for the radiologist. This complements the heuristic cross-validation in #52
with a second independent model.
Why (vision)
Two independent models disagreeing on a finding is exactly the signal a reviewing
radiologist most wants surfaced. A cloud frontier model and a medical-domain
local model have different failure modes; comparing them is cheap defense in
depth. It also gives users who cannot send data to the cloud (consent gate /
offline, see #27) a fully local path that still benefits from cross-checking.
Recommended model (license-verified — corrects #18)
Lingshu-7B — MIT license (code + weights), explicitly supports MRI
among 12+ modalities. This is the recommended default for Add local LLaVA-Med vision model provider #18 instead of LLaVA-Med, whose weights are research-only ("any deployed use case —
commercial or otherwise — is out of scope") and therefore unsuitable as a
default in an Apache-2.0 tool. Alternatives: HuatuoGPT-Vision-7B-Qwen2.5VL
(Apache-2.0). See comment on Add local LLaVA-Med vision model provider #18.
Runs behind the optional local-models extra (torch + transformers).
New optional step src/medcheck/pipeline/consensus.py → ConsensusStep(PipelineStep), OR a flag on vision_analysis to run a second
registered provider and diff the results.
Extend PipelineContext with second_opinion: AnalysisResult | None and a consensus: list[...] comparison.
Report renderers show an "Independent model comparison" section (agreement per
structure, divergences highlighted).
Part of #51. Depends on #18 (local VLM provider).
Summary
Once a local, offline vision-language model provider exists (#18), add an
optional stage that runs it in parallel with the primary cloud provider
(Claude/GPT/Gemini) and produces a consensus / second-opinion comparison:
where both agree, confidence is reinforced; where they diverge, the report flags
it for the radiologist. This complements the heuristic cross-validation in #52
with a second independent model.
Why (vision)
Two independent models disagreeing on a finding is exactly the signal a reviewing
radiologist most wants surfaced. A cloud frontier model and a medical-domain
local model have different failure modes; comparing them is cheap defense in
depth. It also gives users who cannot send data to the cloud (consent gate /
offline, see #27) a fully local path that still benefits from cross-checking.
Recommended model (license-verified — corrects #18)
among 12+ modalities. This is the recommended default for Add local LLaVA-Med vision model provider #18 instead of
LLaVA-Med, whose weights are research-only ("any deployed use case —
commercial or otherwise — is out of scope") and therefore unsuitable as a
default in an Apache-2.0 tool. Alternatives: HuatuoGPT-Vision-7B-Qwen2.5VL
(Apache-2.0). See comment on Add local LLaVA-Med vision model provider #18.
local-modelsextra (torch + transformers).Where
LocalLLMProviderfrom Add local LLaVA-Med vision model provider #18 (currently a stub insrc/medcheck/llm/local.py).src/medcheck/pipeline/consensus.py→ConsensusStep(PipelineStep), OR a flag onvision_analysisto run a secondregistered provider and diff the results.
PipelineContextwithsecond_opinion: AnalysisResult | Noneand aconsensus: list[...]comparison.structure, divergences highlighted).
Design constraints
consent; the cloud model already requires
--allow-cloud-llm. Alocal-only consensus (e.g. two local models, or local vs. cloud only when
consented) must be possible.
local-modelsisn't installed (the coresingle-provider path is unaffected).
LLMRouter/call_with_retriesresiliency.Acceptance criteria
ConsensusStep(orvision_analysisflag) compares two providers'structured findings and records agreements/divergences.
model download in CI.
Effort
~1 day on top of #18.