金融研究助手的离线质量评测见 assistant assessment。
quant-assistant-eval 提供隐藏答案的证据导出、确定性基线、独立标注评分及可读报告;
单独记录评分器回归与基线表现,不宣称测量了在线模型质量。
项目级完整尝试清单与显式复核见 研究证据联邦。
Studio 可通过独立进程调用 quant-research-workflow federate;Lab 保持只读。
独立研究业务层:见 机器学习、稳健性、研究生命周期与证据关系。
入口 quant-research-workflow;只读消费 Lab/Factors,研究状态不授予真实执行权限。
研究可信度升级:接口、使用示例、验收及限制见 11–20 使用说明。
Studio的项目证据解释接口见工作台研究助手,支持离线整理及显式授权的模型建议。
LangGraph review layer for the PureSaber quant stack. It reads completed run outputs from sibling repos (starting with a-share-multifactor), applies deterministic QA rules, optionally calls an LLM for interpretation, and writes experiment notes plus a review_manifest.json back into the run directory.
| Does | Does not |
|---|---|
| Post-backtest review of IC / stats / IC decay CSVs | Fetch market data |
| Validate the immutable standard run contract before review | Repair mutated run artifacts |
| Rule-based QA (NaN factors, weak IC, backtest stats, IC decay, run diff) | Run backtests or replace quant-data-kit |
| Optional LLM explain + skeptic nodes | Embed inside multifactor |
Write markdown to experiment-log/ |
Require API keys in CI (--offline) |
Emit review_manifest.json for quant-lab scanning |
cd quant-agent
python -m pip install --no-deps -r requirements.lock
python -m pip check
python -m pip install -e . --no-deps --no-build-isolation
python -m pip checkrequirements.lock is the audited Python3.10-3.12 environment. It includes the base runtime,
the optional LLM runtime, development tools, and editable-build requirements. Do not install extras
on top of this environment because that would resolve a second, unaudited dependency graph.
# offline review (default, no API key)
quant-review run --project multifactor --run-dir <validated-run-dir>
# auto-detect project from run dir
quant-review detect <validated-run-dir>
# enable LLM (needs OPENAI_API_KEY)
quant-review run --project multifactor --run-dir PATH --llm
# draft only; preserves costs, risk and holdout and never starts a run
quant-propose paper.txt --template recipe.yaml --output draft --study-id momentum-v2 \
--database studies/experiments.db --studies-root studies
# retrieve cited study results and failures with artifact verification
quant-research-history "momentum low volatility" --database studies/experiments.db \
--studies-root studies --recipe recipe.yaml --output advice.jsonExit code 2 when deterministic rules report an error severity finding.
The UI-facing quant_agent.research_advice(...) API returns read-only historical matches,
re-verifiable citations, completed results, failures, artifact checks, a minimal paired comparison
and explicit missing-data notes. It does not select a winner, modify governed recipe fields or run
an experiment. See the research assistant guide.
Studio可先展示历史证据和清除旧输出的Notebook源码,再显式调用:
python -m quant_agent.workbench_assistant --context context.json --notebook source.ipynb --output answer.json默认离线返回证据整理与空patches。在线生成需要显式--model、
QUANT_AGENT_LLM_OK=1和已有供应商凭据;发送内容包括预览的代码及历史摘要。
每个修改建议绑定原单元格序号、SHA-256和证据编号。拒绝未知证据、
错误源码哈希、重复单元格及超限响应,Studio展示逐单元格diff供审阅。
助手不会执行代码或改写草稿,建议仍需研究者验证。输入源码上限60KB,
合并证据上限120KB;输出文件必须是新文件。
测试使用离线进程与可控模型响应,不代表在线供应商已验收。
load → rules → explain → skeptic → write_report
- load: adapter validates
standard/run_manifest.json, then reads project-specific QA files - rules: Python checks (IC quality, alt-data NaN, PIT config)
- explain / skeptic: LLM when
--llm, else offline templates - write:
experiment-log/review_*.md+review_manifest.json
This repository declares the orchestration layer and consumes standard/v2@2.0.0 through
[tool.quant-workspace]. The declared external lock is requirements.lock; a release audit records
its SHA-256. Internal runtime dependencies use immutable commit pins or published annotated tags
in both project metadata and the lock; their verified commits are:
quant-labresearch integrity and objective contract at commit4c48f8a678c8d29a7dc46294c99f0e668cd6c52b;quant-factorsrestricted research expressions at commit1c798bc737bc92fa5eca85dd0663dc608104f721;quant-workspace v0.3.1at commit537388a4d9548b612fa1e4b306c482c04b45c433.
For a run containing standard/v2, the agent first calls the version-independent quant-lab
validator. Only after all manifest, hash, schema, lineage, profile, and file-set checks succeed does
it read the manifest-listed config.json and metrics.json. Root-level project CSV files and an
external --config cannot override a v2 run. If v2 exists but is invalid, review fails; it never
downgrades to v1. When v2 is absent, the immutable v1 reader and historical project layouts remain
supported permanently.
V2 producers may put agent inputs in the validated metrics.json object as ic_summary,
backtest_stats, and ic_decay arrays of objects. Recognized scalar performance metrics are also
projected from the top level. Factor or strategy names come from validated config first and the
manifest strategy IDs otherwise.
Review changes to pyproject.toml and requirements-constraints.txt first. Run the rebuild under
Python3.10, the lowest supported interpreter, so its conditional tomli runtime remains explicit;
then regenerate with the pinned resolver used for this release:
python -m pip install "pip-tools==7.6.1"
pip-compile --extra dev --extra llm --build-deps-for editable --allow-unsafe --strip-extras \
--resolver backtracking --index-url https://pypi.org/simple \
--constraint requirements-constraints.txt \
--output-file requirements.lock pyproject.tomlInstall the result in clean Python3.10,3.11, and3.12 environments using the commands in
Install. CI runs pip check both before and after the no-dependency/no-build-isolation editable
install, then runs Ruff check/format, the complete test suite with at least80% branch coverage, and
the artifact-consumption core with at least90% pure branch coverage. Update the declaration,
constraints, and lock as one review unit; never hand-edit an isolated transitive pin.
Migration is producer-owned: publish a new immutable standard/v2 directory beside v1, including
its complete manifest, manifest checksum, config, metrics, Parquet artifacts, and lineage. Do not
modify v1 and do not copy unvalidated private CSVs into the v2 review path. Validate the new run with
the pinned quant-lab validator before invoking quant-review; a failed migration remains a failed v2 run and
must be repaired by publishing a new run rather than by forcing v1 fallback.
Rollback this governance change with a Git revert that restores pyproject.toml, the constraints,
and requirements.lock together. Existing runs and v1 data require no rewrite. Never move, delete,
or recreate historical tags; if a dependency resolution must change, publish a new version and a
new lock hash.
Notebook proposals request a final JSON object directly, without a program or tool output
that generates JSON. Quotes must come from the cited context.evidence.text, not from a
code cell. Local schema, quotation and cell-hash validation still reject incompatible
provider output; no code is executed and uncertain generations are not automatically retried.
Prompt-emulated JSON mode is not a guarantee of provider conformance.
configs/agent_review.yaml — thresholds, rule toggles, LLM model.
Implement RunAdapter in src/quant_agent/adapters/ and register in get_adapter(). Planned: sklearn-stock-trend, futures spread.
ruff check src tests
ruff format --check src tests
coverage run --branch --source=quant_agent -m pytest -q
coverage report --show-missing --fail-under=80