Skip to content

Commit c9f0bef

Browse files
public release: ControlMind project, CC-BY-4.0, full reproducibility
- LICENSE: CC-BY-4.0 standard full text - README.md / README.zh.md: rewritten for public release - requirements.txt: cleaned 5 unused deps, 28 packages - .gitignore: unblock _final_submission_by_track, add corpus/chunks - Track 1-3 reports: MorningStar0709/ControlMind -> control-sci, Apache 2.0 -> CC-BY-4.0 - All PPTs: URL + license corrected - All absolute paths (D:\WorkPlace) replaced with relative paths - CITATION.cff, npm packages, notebooks: URL consistency
1 parent c1c7de8 commit c9f0bef

28,760 files changed

Lines changed: 1286273 additions & 1137 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.gitignore

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,6 @@ docs/*
5555
# --- PPT 内部素材 ---
5656
_ppt_materials/
5757
_ppt_materials_tracks/
58-
_final_submission_by_track/
5958

6059
# --- 运行时 & 衍生产物 ---
6160
benchmark/eval/reports/
@@ -86,7 +85,6 @@ data/processed/
8685

8786
# --- MinerU 解析产物(可重建,量大) ---
8887
corpus/processed/
89-
corpus/chunks/
9088
data/sources_medical/chunks/
9189
data/sources_medical/pdf/
9290

@@ -133,6 +131,8 @@ data/sources_medical/medbench/*.zip
133131

134132
# --- Pipeline 临时文件 ---
135133
pipeline/_bat_tmp/
134+
pipeline/*.checkpoint.json
135+
pipeline/*.checkpoint.*.json
136136

137137
# --- OS ---
138138
.DS_Store
@@ -142,6 +142,10 @@ Thumbs.db
142142
# Keep these after broad allow-list rules such as !docs/submissions/**.
143143
docs/submissions/data_trace_bundle/09_medical_rag_large/
144144
docs/submissions/data_trace_bundle/10_charts_large/
145-
_final_submission_by_track/
146145
_ppt_materials_tracks/
147146
build/
147+
148+
# --- Submission package: exclude rebuildable binary caches ---
149+
_final_submission_by_track/**/embeddings_cache.npy
150+
_final_submission_by_track/**/medical.index
151+
_final_submission_by_track/**/bm25.pkl

README.md

Lines changed: 95 additions & 128 deletions
Original file line numberDiff line numberDiff line change
@@ -1,180 +1,147 @@
11
# ControlMind
22

3-
ControlMind is a MinerU-based scientific document intelligence project for the 2026 MinerU Data Intelligence Challenge. It connects three independent tracks into one auditable chain: scientific corpus construction, autonomous data processing, and source-grounded RAG.
3+
**A MinerU-powered scientific document intelligence system: 500-question cross-modal benchmark, 14-intent data agent, and local-first medical RAG — all from raw PDFs.**
44

5-
[Chinese version](README.zh.md)
5+
[Chinese version](README.zh.md) | [CC-BY-4.0](LICENSE)
66

77
```text
8-
Track 1: Sci-Align dataset and benchmark
9-
Track 2: Data Agent execution protocol
10-
Track 3: Medical literature RAG
8+
Track 1 Sci-Align — 4-dimension control-science evaluation benchmark
9+
Track 2 Data Agent — 14-intent autonomous corpus agent with 4-path scheduling
10+
Track 3 Medical RAG — local-first evidence-grounded clinical literature Q&A
1111
```
1212

13-
The project uses MinerU to turn scientific PDFs into structured text, formulas, tables, figures, chunks, indexes, benchmark questions, Agent traces, and RAG evidence cards.
13+
---
1414

15-
## Public Entry Points
16-
17-
| Item | Link |
18-
|:---|:---|
19-
| Cloud demo | [https://demo.askiler.com/](https://demo.askiler.com/) |
20-
| Demo access code | `ControlMind@2026` |
21-
| GitHub | [MorningStar0709/ControlMind](https://github.com/MorningStar0709/ControlMind) |
22-
| HuggingFace dataset | [MorningStar0709/control-sci-corpus](https://huggingface.co/datasets/MorningStar0709/control-sci-corpus) |
23-
| Submission quickstart | [docs/submissions/quickstart.md](docs/submissions/quickstart.md) |
24-
| Submission README | [docs/submissions/README.md](docs/submissions/README.md) |
25-
| Data trace | [docs/submissions/shared/DATA-TRACE.md](docs/submissions/shared/DATA-TRACE.md) |
26-
| Reproducibility | [REPRODUCIBILITY.md](REPRODUCIBILITY.md) |
27-
28-
## What Is Included
29-
30-
| Track | Main artifact | Start here |
31-
|:---|:---|:---|
32-
| Track 1: Sci-Align | 500-question control-science benchmark with AI-ready schema | [track1_sci_align_report.md](docs/submissions/track1_sci_align_report.md) |
33-
| Track 2: Data Agent | 14-intent Agent protocol with scheduling, logs, fallback, replay, and validation | [track2_agent_report.md](docs/submissions/track2_agent_report.md) |
34-
| Track 3: Medical RAG | Source-grounded medical literature RAG with Chinese Ask, claim support, refusal boundary, and local deployment | [track3_medical_rag_report.md](docs/submissions/track3_medical_rag_report.md) |
15+
![System Architecture](docs/submissions/shared/assets/task2/system_architecture.png)
3516

36-
Representative sample packs:
17+
*ControlMind system architecture: a single RTX 5090 runs the full pipeline — MinerU parsing, tri-engine inference (API / Ollama / vLLM), and multi-index RAG.*
3718

38-
- [Track 1 20 cases](docs/submissions/shared/track1_sci_align_20_cases.md)
39-
- [Track 2 20 cases](docs/submissions/shared/track2_agent_20_cases.md)
40-
- [Track 3 20 cases](docs/submissions/shared/track3_medical_rag_20_cases.md)
19+
---
4120

42-
## Key Results
21+
## What This Project Does
4322

44-
| Area | Result |
45-
|:---|:---|
46-
| Scientific corpus | 362 control-science documents, 253,012 LaTeX formulas, 28K-level chunks |
47-
| Track 1 benchmark | 500 balanced questions, A/B/C/D = 125 each, 9-model leaderboard |
48-
| Cross-modal traceability | 500/500 questions linked to source chunks; image/formula statistics preserved |
49-
| Track 2 Agent | 14 intents, ResourceScheduler, LogStep schema, artifact validation, 391-second flywheel replay |
50-
| Track 3 Medical RAG | 97 parsed PMC papers, 3,348 medical chunks, FAISS/BM25/vision indexes |
51-
| Chinese Ask | BGE-M3 fixed trace with full claim support and citation coverage in saved evaluation |
52-
| Local-first boundary | Medical chunks, indexes, QLoRA data, and RAG context stay local by default |
23+
| Track | What You Get | One Command |
24+
|:---|:---|:---|
25+
| **Sci-Align** | A 500-question 4-dimension benchmark (A: Concept Recall, B: Multi-step Reasoning, C: Condition Sensitivity, D: Open Design) with 9-model leaderboard and full source traceability. Loadable via `load_dataset()`. | `load_dataset("MorningStar0709/control-sci-corpus")` |
26+
| **Data Agent** | A 14-intent autonomous agent that searches arXiv, parses PDFs with MinerU, audits cross-modal alignment, builds benchmarks, evaluates models, and self-corrects on failure — with unified logging and checkpoint recovery. | `controlmind track2 validate --artifact all` |
27+
| **Medical RAG** | A local-first evidence Q&A system over 97 parsed PMC papers, with IMRAD-aware chunking, hybrid FAISS+BM25 retrieval, Chinese-to-English query bridging, visual injection, and safety-refusal boundaries. | `controlmind track3 eval --case-set zh_ask` |
5328

54-
All quantitative claims in the submission reports point back to files, commands, or hashes in [DATA-TRACE.md](docs/submissions/shared/DATA-TRACE.md).
29+
---
5530

5631
## Quick Start
5732

58-
Use an existing `myenv` environment when available.
59-
60-
```powershell
61-
conda run -n myenv python demo/cli/controlscidemo all --quick
62-
```
63-
64-
Run per-track quick views:
65-
66-
```powershell
67-
conda run -n myenv python demo/cli/controlscidemo track1 --quick
68-
conda run -n myenv python demo/cli/controlscidemo track2 --quick
69-
conda run -n myenv python demo/cli/controlscidemo track3 --quick
33+
```bash
34+
pip install -r requirements.txt
35+
pip install -e .
36+
controlmind doctor
7037
```
7138

72-
Run reviewer-oriented minimal verification:
39+
Load the Sci-Align benchmark dataset:
7340

74-
```powershell
75-
.\run_reviewer_demo.ps1 -Track All -SkipApiHealth
76-
```
77-
78-
Run JSON-oriented checks. Use `--output` when saving files on Windows; it lets the Python CLI write UTF-8 JSON directly and avoids shell redirection encoding issues.
41+
```python
42+
from datasets import load_dataset
7943

80-
```powershell
81-
conda run --no-capture-output -n myenv python -m controlsci.cli doctor --output _scratch/doctor.json
82-
conda run --no-capture-output -n myenv python -m controlsci.cli track2 validate --artifact all --output _scratch/track2_validate.json
83-
conda run --no-capture-output -n myenv python -m controlsci.cli track3 eval --case-set zh_ask --output _scratch/track3_eval_zh_ask.json
44+
core = load_dataset("MorningStar0709/control-sci-corpus", "core", split="train")
45+
print(len(core)) # 500
46+
print(core[0]["question"])
8447
```
8548

86-
Install the public CLI wrapper locally:
49+
Run per-track quick checks:
8750

88-
```powershell
89-
pip install -e .
90-
controlmind doctor
51+
```bash
52+
controlmind track1 validate --sample 4
9153
controlmind track2 validate --artifact all
9254
controlmind track3 eval --case-set zh_ask
9355
```
9456

95-
Optional Node.js launcher:
57+
> **Windows users:** prepend `conda run -n myenv python -m controlsci.cli` if not using `pip install -e .`. PowerShell scripts (`run_reviewer_demo.ps1`, `run_frontend.ps1`) are also provided.
9658
97-
```powershell
98-
npm install -g ./npm/controlmind
99-
controlmind wrapper-doctor
100-
controlmind track2 validate --artifact all
101-
```
102-
103-
The npm package is only a thin launcher. It locates the repository, selects `CONTROLMIND_PYTHON`, `conda run -n myenv python`, or system Python, and forwards commands to `python -m controlsci.cli`.
59+
---
10460

105-
## Load The Sci-Align Dataset
106-
107-
```python
108-
from datasets import load_dataset
109-
110-
core = load_dataset("MorningStar0709/control-sci-corpus", "core", split="train")
111-
print(len(core))
112-
print(core[0]["question"])
113-
114-
full = load_dataset("json", data_files="benchmark/dataset/full.json", field="questions", split="train")
115-
```
61+
## Key Results
11662

117-
Local JSON files:
63+
| Metric | Value |
64+
|:---|---:|
65+
| Documents parsed | 362 (23 textbooks + 339 arXiv papers) |
66+
| Structured chunks | 28,514 |
67+
| LaTeX formulas extracted | 253,012 |
68+
| Image-formula co-occurrence pairs | 4,996 (9,207 audit judgments) |
69+
| Benchmark questions | 500 (A/B/C/D = 125 each, 14 sub-domains) |
70+
| Models evaluated | 9 |
71+
| PMC medical papers | 97 parsed, 3,348 medical chunks |
72+
| QLoRA fine-tuning | 4B/9B variants, perplexity-probed |
11873

119-
```text
120-
benchmark/dataset/core.json
121-
benchmark/dataset/full.json
122-
benchmark/dataset/schema.json
123-
```
74+
---
12475

125-
Dataset documentation:
76+
## Leaderboard — ControlSci Sci-Align Benchmark
12677

127-
- [benchmark/dataset/README.md](benchmark/dataset/README.md)
128-
- [HuggingFace dataset card](https://huggingface.co/datasets/MorningStar0709/control-sci-corpus)
78+
| Rank | Model | Overall | A: Concept | B: Reasoning | C: Sensitivity | D: Design |
79+
|:---:|:---|:---:|:---:|:---:|:---:|:---:|
80+
| 1 | **MiMo-v2-flash** | **0.647** | 0.610 | 0.606 | 0.636 | 0.736 |
81+
| 2 | DeepSeek-v4-flash | 0.632 | 0.634 | 0.631 | 0.714 | 0.550 |
82+
| 3 | Qwen3.5-9B | 0.625 | 0.569 | 0.610 | 0.662 | 0.659 |
83+
| 4 | DeepSeek-v4-pro | 0.619 | 0.627 | 0.590 | 0.742 | 0.514 |
84+
| 5 | MiniMax-M2.5-highspeed | 0.602 | 0.638 | 0.519 | 0.624 | 0.626 |
85+
| 6 | MiniMax-M2.7-highspeed | 0.574 | 0.605 | 0.485 | 0.612 | 0.593 |
86+
| 7 | MiMo-v2.5-pro | 0.539 | 0.595 | 0.523 | 0.602 | 0.436 |
87+
| 8 | MiMo-v2-pro | 0.514 | 0.638 | 0.490 | 0.560 | 0.369 |
88+
| 9 | MiMo-v2.5 | 0.440 | 0.608 | 0.466 | 0.528 | 0.156 |
12989

130-
## Local Demo And Deployment
90+
All scores verified by LLM-as-Judge with cross-validation. Full results and analysis in [`benchmark/eval/results/`](benchmark/eval/results/).
13191

132-
Start the local frontend/backend workbench:
92+
![Leaderboard](docs/submissions/shared/assets/task1/track1_leaderboard_scores.png)
13393

134-
```powershell
135-
.\run_frontend.ps1 -StartBackend
136-
```
94+
---
13795

138-
Run the RAG API-backed reviewer check after the local API is available:
96+
## Reports
13997

140-
```powershell
141-
.\run_reviewer_demo.ps1 -Track All -ApiPort 17001
142-
```
98+
Each track has a companion technical report with detailed methodology, experiments, and traceable evidence:
14399

144-
Track-specific deployment notes:
100+
| Track | Report | Key Evidence |
101+
|:---|:---|:---|
102+
| Track 1 Sci-Align | [track1_sci_align_report.md](docs/submissions/track1_sci_align_report.md) | 500-question schema, 9-model leaderboard, QLoRA results |
103+
| Track 2 Data Agent | [track2_agent_report.md](docs/submissions/track2_agent_report.md) | 14-intent protocol, dry-run logs, failure recovery cases |
104+
| Track 3 Medical RAG | [track3_medical_rag_report.md](docs/submissions/track3_medical_rag_report.md) | 97 PMC papers, Chinese Ask traces, MedBench comparison |
145105

146-
- [Track 2 Agent deploy notes](docs/submissions/shared/track2_agent_deploy.md)
147-
- [Track 3 Medical RAG deploy notes](docs/submissions/shared/track3_medical_deploy.md)
148-
- [Cloud demo boundary](docs/submissions/shared/public_cloud_boundary.md)
106+
Every quantitative claim in the reports points back to source files, commands, or hashes in [`DATA-TRACE.md`](docs/submissions/shared/DATA-TRACE.md).
149107

150-
The public cloud demo is for public or sanitized examples. Private documents, medical chunks, indexes, model adapters, and RAG contexts are handled through local/private paths.
108+
---
151109

152-
## Reproducibility Boundary
110+
## Public Entry Points
153111

154-
The recommended verification path uses packaged public artifacts and local indexes. arXiv/PMC online downloading remains available as a corpus expansion capability, but it is intentionally not a prerequisite for minimal validation because public services may apply rate limits, browser checks, or short-lived cookies. Local run outputs should go under `_scratch/`; that directory is ignored and can be deleted or regenerated without affecting the audited source data.
112+
| Item | Link |
113+
|:---|:---|
114+
| Cloud Demo | [demo.askiler.com](https://demo.askiler.com/) (code: `ControlMind@2026`) |
115+
| HuggingFace Dataset | [MorningStar0709/control-sci-corpus](https://huggingface.co/datasets/MorningStar0709/control-sci-corpus) |
116+
| Reproducibility Guide | [REPRODUCIBILITY.md](REPRODUCIBILITY.md) |
117+
| Evidence Bundle | [docs/submissions/data_trace_bundle/](docs/submissions/data_trace_bundle/) |
155118

156-
For the full layered policy, see [REPRODUCIBILITY.md](REPRODUCIBILITY.md). In short: smoke checks and minimal real-chain validation are designed for a public checkout; report-level conclusions are auditable through `docs/submissions/data_trace_bundle/`; full-scale rebuilds may require GPU, network access, and external model/API credentials.
119+
---
157120

158121
## Repository Map
159122

160123
```text
161-
benchmark/ Track 1 benchmark and Track 2 Agent artifacts
162-
benchmark/dataset/ ControlSci core/full/schema JSON files
163-
benchmark/eval/ evaluation, leaderboard, medical RAG eval scripts
164-
controlsci/ Python package and controlmind CLI
165-
npm/controlmind/ optional Node.js launcher for the Python CLI
166-
data/sources_medical/ Track 3 medical corpus, chunks, indexes, vision artifacts
167-
docs/submissions/ public submission reports, quickstart, evidence bundle
168-
starboard/ local/cloud demo frontend
169-
tools/ MinerU and utility scripts
124+
benchmark/ Sci-Align benchmark & Data Agent code
125+
benchmark/dataset/ Core/full/schema JSON, multimodal index
126+
benchmark/eval/ Evaluation, leaderboard & medical RAG scripts
127+
controlsci/ Python package (controlmind CLI)
128+
data/sources_medical/ Medical corpus, chunks, FAISS/BM25 indexes
129+
docs/submissions/ Technical reports, evidence bundle, DATA-TRACE
130+
starboard/ Local & cloud demo frontend (Next.js)
131+
tools/ MinerU utilities & analysis scripts
132+
npm/controlmind/ Optional Node.js CLI launcher
170133
```
171134

172-
## License And Data Boundary
135+
---
136+
137+
## License & Data Boundary
138+
139+
- This project is released under **CC-BY-4.0**. See [LICENSE](LICENSE).
140+
- Public PMC/arXiv source documents retain their original licenses and attribution.
141+
- Patient-level private data is **not** included.
142+
- Cloud demo inputs are limited to public or sanitized materials; medical chunks, indexes, and RAG contexts are local-first by default.
173143

174-
- The ControlSci dataset is released under **CC-BY-4.0**. See [LICENSE](LICENSE).
175-
- Public PMC/arXiv/source documents retain their original licenses and attribution requirements.
176-
- Patient-level private data is not included in this repository or submission package.
177-
- Cloud demo inputs are limited to public or sanitized materials; private RAG assets are local-first.
144+
---
178145

179146
## Citation
180147

@@ -183,7 +150,7 @@ tools/ MinerU and utility scripts
183150
title = {ControlMind: MinerU-based Scientific Document Intelligence for Sci-Align, Data Agent, and Medical RAG},
184151
author = {MorningStar},
185152
year = {2026},
186-
howpublished = {\url{https://github.com/MorningStar0709/ControlMind}},
187-
note = {ControlSci dataset released under CC-BY-4.0}
153+
howpublished = {\url{https://github.com/MorningStar0709/control-sci}},
154+
note = {ControlSci benchmark released under CC-BY-4.0}
188155
}
189156
```

0 commit comments

Comments
 (0)