Harden OpenAlphaStack data and execution integrity

Author: 44-99

.gitignore

@@ -26,6 +26,9 @@ data/output/
 data/memory/
 data/state/
 data/test_tmp/
+data/**/*.sqlite3
+data/**/*.sqlite3-wal
+data/**/*.sqlite3-shm
 data/*.json
 data/*.md
 *.log

AGENT_GUIDE.md

@@ -12,7 +12,7 @@ validation, state, risk rules, backtests, and mechanical paper execution.
 3. Put Agent workflow instructions in `skills/`.
 4. Put live data and domain actions behind typed MCP tools.
 5. Keep risk limits, T+1 behavior, fees, state transitions, and idempotency in Python.
-6. MCP mutations are paper-only. Never expose a live-order tool before an explicit safety design and separate approval.
+6. Public engine modes are paper and backtest only. MCP mutations are paper-only; historical live runs remain read-only.
 7. Missing or stale plans keep the engine in observation mode; the engine must not invent a plan.
 8. The Dashboard is an observability surface, not an Agent terminal.
 
@@ -22,7 +22,8 @@ validation, state, risk rules, backtests, and mechanical paper execution.
 - Do not expose arbitrary shell execution through HTTP, WebSocket, or MCP.
 - Do not commit `.env` or runtime data under `data/`.
 - Do not reveal private Feishu conversations or local secrets.
-- Preserve atomic writes and append-only ledgers.
+- Treat per-run SQLite as canonical; JSON/JSONL files are human-readable projections.
+- Preserve atomic state-plus-ledger commits and append-only ledger semantics.
 
 ## Development rules
 

README.md

@@ -23,8 +23,6 @@ missing trading plan.
 
 > Paper trading only. OpenAlphaStack does not place real orders and does not promise investment returns.
 
-![OpenAlphaStack Dashboard](docs/assets/dashboard-workbench.jpg)
-
 ## Why OpenAlphaStack?
 
 - **One plugin, two extension layers** — package reusable domain Skills and a
@@ -78,10 +76,22 @@ Requirements:
 ```powershell
 git clone https://github.com/44-99/OpenAlphaStack.git
 cd OpenAlphaStack
-pip install -r requirements.txt
 pip install -e .
+openalphastack doctor
+```
+
+The base install supports the Codex plugin, MCP contracts, and offline Demo path.
+Install only the surfaces you use:
+
+```powershell
+pip install -e ".[market]"             # AkShare-backed market providers
+pip install -e ".[engine]"             # paper/backtest Parquet runtime
+pip install -e ".[dashboard]"          # FastAPI Dashboard
+pip install -e ".[all]"                # complete local development runtime
+
 npm install
 npm run dashboard:build
+openalphastack doctor
 openalphastack app start
 ```
 
@@ -94,6 +104,22 @@ Use $market-analyzer to assess today's A-share market, cite the MCP data used,
 and finish with risks and invalidation conditions.
 ```
 
+### Offline first run
+
+Market providers may be unavailable outside trading hours or behind a restricted
+network. To verify the complete Skill → MCP path without treating sample values
+as market facts, ask Codex:
+
+```text
+Use $market-analyzer in offline demo mode. Read the market_overview and
+market_news demo datasets, show their schema version, source, as-of time and
+freshness status, then produce a short report clearly labelled as synthetic data.
+```
+
+The `read_demo_dataset` MCP tool is read-only and deterministic. Skills must not
+publish Demo-derived values into a paper plan. Available datasets cover market
+overview, screening, quote, technical, fundamentals, news and a baseline backtest.
+
 The repository is also a Codex plugin: `.codex-plugin/plugin.json` discovers the
 Skills and `.mcp.json` registers the stdio server. After installing the Python
 package, install/open the plugin in Codex and verify the `open-alpha-stack` MCP
@@ -105,6 +131,13 @@ Start the MCP server manually for diagnostics:
 openalphastack mcp serve
 ```
 
+Check the local installation at any time:
+
+```powershell
+openalphastack doctor
+openalphastack doctor --json
+```
+
 ## Codex Skills
 
 - `$market-analyzer`: market environment, sentiment, sectors, and leaders.
@@ -121,13 +154,35 @@ Codex Desktop to remain running.
 Read tools expose paper/backtest runs, market data, indicators, news, screens,
 risk calculations, and deterministic baseline backtests.
 
+Every MCP tool returns a versioned envelope:
+
+```json
+{
+  "schema_version": "openalphastack.mcp/v1",
+  "ok": true,
+  "data": {},
+  "meta": {
+    "source": "provider-or-demo-dataset",
+    "as_of": "2026-07-23T10:00:00+08:00",
+    "freshness": {"status": "fresh"},
+    "demo": false
+  }
+}
+```
+
+Callers must check `ok` before reading `data`. Failures use a stable
+`error.code` and do not expose provider exception text. JSON schemas are
+available through `get_contracts` and `openalphastack://contracts/v1`.
+
 The only plan mutations are:
 
 1. `save_plan_draft` — writes a non-executable `plan.codex-draft.json`.
 2. `publish_paper_plan` — validates, requires an idempotency key, checks the
    expected plan version, and atomically updates a paper run only.
 
-Live runs are hidden from MCP mutations. No shell, arbitrary file-write, or live-order tool is exposed.
+There is no public live mode: the CLI cannot start or resume one, and MCP has no
+live-order tool. Historical `live_*` directories remain read-only for migration
+and audit. No shell or arbitrary file-write tool is exposed.
 
 ## Engine commands
 
@@ -146,6 +201,16 @@ The paper engine can stay running outside trading hours. It idles according to
 the trading calendar and remains observation-only until Codex publishes a valid
 plan for the current date.
 
+Each run uses `run.sqlite3` as the transactional source of truth for account
+state, the active plan, and ledger events. `state.json`, `plan.json`, and
+`ledger.jsonl` are human-readable projections. A trade updates account state and
+its matching ledger event in one SQLite transaction.
+
+Backtests require real cached or provider minute bars. Missing intraday data now
+fails closed instead of fabricating bars from daily OHLC. Backtest output remains
+experimental evidence—not a profitability claim—until walk-forward,
+out-of-sample, and baseline comparisons are implemented.
+
 ## Verification
 
 ```powershell

docs/architecture.md

@@ -27,9 +27,10 @@ Scheduled premarket task
   -> validate_paper_plan
   -> save_plan_draft
   -> publish_paper_plan (paper only, idempotent, optimistic concurrency)
-  -> paper engine refreshes newer plan.json
+  -> paper engine refreshes the newer validated plan from run.sqlite3
   -> FastLane applies deterministic rules
-  -> append ledger and workflow events
+  -> atomically commit account state + ledger event to run.sqlite3
+  -> refresh human-readable JSON/JSONL projections and workflow events
   -> scheduled postclose prompt reviews facts without mutation
 ```
 
@@ -40,7 +41,10 @@ Scheduled premarket task
 - The engine may execute a valid current plan; it may not call a model.
 - Emergency handling is deterministic and notification-based.
 - A missing, stale, or invalid plan produces observation mode.
-- Ledger records are append-only.
+- SQLite is the per-run source of truth; JSON and JSONL are projections.
+- Account mutations and their ledger events commit in one SQLite transaction.
+- Missing intraday bars fail closed; backtests never synthesize minute bars from daily OHLC.
+- Public engine modes are paper and backtest only. Historical live runs are read-only.
 
 ## MCP surface
 
@@ -63,6 +67,36 @@ Mutation group:
 Every publication requires an idempotency key. Optional `expected_updated`
 provides optimistic concurrency against a plan changed since the Agent read it.
 
+### Versioned response contract
+
+All MCP tools return the `openalphastack.mcp/v1` envelope. Consumers check `ok`
+before reading `data`; failures contain a stable `error.code`, retryability and
+non-sensitive details. Market responses expose `meta.source`, `meta.as_of` and
+`meta.freshness`. Plans and run snapshots use the separate
+`openalphastack.plan/v1` and `openalphastack.run-snapshot/v1` contracts.
+
+Resources:
+
+- `openalphastack://contracts/v1`
+- `openalphastack://demo/catalog`
+- `openalphastack://demo/{dataset}`
+- `openalphastack://runs/{run_id}/snapshot`
+- `openalphastack://runs/{run_id}/ledger`
+
+The bundled Demo datasets are static synthetic fixtures. They support offline
+Skill verification and cannot mutate or publish a trading plan. Dashboard Demo
+account, plan, and ledger fixtures share the same `demo_data` ownership boundary;
+chart and workflow fixtures remain explicitly UI-only presentation data.
+
+## Persistence boundary
+
+Every paper or backtest run owns a `run.sqlite3` database containing runtime
+state, the active validated plan, and append-only ledger events. SQLite WAL,
+full synchronous commits, and immediate write transactions provide the
+cross-record atomicity boundary. `state.json`, `plan.json`, and `ledger.jsonl`
+exist for operator inspection and compatibility; corrupt or stale projections
+must not override a valid database.
+
 ## Scheduling boundary
 
 Codex scheduled tasks compose the domain Skills for premarket research,

docs/roadmap.md

@@ -11,17 +11,22 @@
 - ✅ Paper engine reloads externally published plans and stays observation-only without one.
 - ✅ Embedded Agent terminal and in-process Agent scheduling removed from the main path.
 - ✅ Backtests no longer invoke an Agent internally.
+- ✅ Public live start/resume paths removed; historical live runs are read-only.
 
 ## Next: MCP contract hardening
 
-- Add explicit JSON schemas and version fields for plans and run snapshots.
-- Add source freshness and provenance to every market-data response.
-- Add structured MCP errors rather than provider-specific exception text.
-- Add resource URIs for plan, state, ledger, reports, and strategy metrics.
-- Add a read-only demo dataset for Skill forward tests.
+- ✅ Added explicit JSON schemas and version fields for plans and run snapshots.
+- ✅ Added source freshness and provenance to every MCP market-data response.
+- ✅ Added structured MCP errors without provider exception text.
+- ✅ Added contract, Demo, run snapshot, and ledger resource URIs.
+- ✅ Added a read-only synthetic Demo dataset and Skill forward-contract tests.
+- Add dedicated report and strategy-metric resource URIs after their persisted schemas stabilize.
 
 ## Next: deterministic runtime
 
+- ✅ Persist state, active plans, and ledger events in a per-run SQLite source of truth.
+- ✅ Commit account mutations and matching ledger events atomically.
+- ✅ Reject missing intraday data instead of synthesizing minute bars from daily OHLC.
 - Run the paper engine as a long-lived local service that idles outside exchange sessions.
 - Make plan activation a first-class state transition with an audit event.
 - Add plan expiry and automatic observation fallback.
@@ -38,13 +43,13 @@
 
 ## Next: product and distribution
 
-- Replace the old screenshot with a Dashboard image without the Agent terminal.
+- Remove the obsolete Agent-terminal screenshot and capture a verified current Dashboard image.
 - Record a short Codex Scheduled task -> Skill -> MCP -> paper plan demo.
 - Publish honest benchmark and failure reports rather than return promises.
 - Publish plugin installation and scheduled-task composition examples.
 
 ## Live trading
 
-Live trading remains unimplemented and inaccessible through MCP. It requires a
+Live trading remains unimplemented and inaccessible through the CLI and MCP. It requires a
 separate BrokerAdapter, explicit human confirmation, order idempotency, restart
 recovery, kill switches, authenticated deployment, and a new security review.

pyproject.toml

@@ -8,15 +8,36 @@ version = "0.1.0"
 description = "Open-source Codex plugin stack for auditable A-share research and paper trading"
 requires-python = ">=3.10"
 dependencies = [
+    "mcp>=1.2.0,<2.0.0",
+    "pydantic>=2.10.0,<3.0.0",
+    "python-dotenv==1.0.1",
+    "pandas>=2.2.0",
+    "requests>=2.32.0",
+]
+
+[project.optional-dependencies]
+market = [
+    "akshare>=1.16.0",
+]
+engine = [
+    "pyarrow>=16.0.0",
+]
+dashboard = [
     "fastapi==0.115.6",
     "uvicorn==0.34.0",
+    "pyarrow>=16.0.0",
+]
+feishu = [
     "httpx==0.28.1",
-    "pandas>=2.2.0",
-    "akshare>=1.16.0",
-    "python-dotenv==1.0.1",
     "lark-oapi>=1.4.0",
+]
+all = [
+    "akshare>=1.16.0",
     "pyarrow>=16.0.0",
-    "mcp>=1.2.0,<2.0.0",
+    "fastapi==0.115.6",
+    "uvicorn==0.34.0",
+    "httpx==0.28.1",
+    "lark-oapi>=1.4.0",
 ]
 
 [project.scripts]

requirements.txt

@@ -1,3 +1,5 @@
+# Full local development/runtime environment. End users should prefer
+# `pip install -e .` or `pip install -e ".[all]"` as documented in README.md.
 fastapi==0.115.6
 uvicorn==0.34.0
 httpx==0.28.1
@@ -7,3 +9,5 @@ python-dotenv==1.0.1
 lark-oapi>=1.4.0
 pyarrow>=16.0.0
 mcp>=1.2.0,<2.0.0
+pydantic>=2.10.0,<3.0.0
+requests>=2.32.0

skills/market-analyzer/SKILL.md

@@ -17,3 +17,10 @@ description: 分析 A 股大盘、市场情绪、板块轮动和龙头线索。
 6. 输出市场环境、情绪阶段、板块线索、龙头证据、数据限制和适用的风险暴露区间。
 
 不要把市场研判直接当作买卖指令，也不要承诺收益。
+
+## MCP 响应契约
+
+- 每次调用先检查 `schema_version` 和 `ok`，只有 `ok=true` 才读取 `data`。
+- 引用数据时保留 `meta.source`、`meta.as_of` 和 `meta.freshness.status`；状态为 `stale` 或 `unknown` 时降低结论置信度。
+- `ok=false` 时报告结构化的 `error.code`，不得把缺失数据补写成事实。
+- 用户要求离线演示或实时数据全部不可用时，依次调用 `read_demo_dataset(dataset="market_overview")` 和 `read_demo_dataset(dataset="market_news")`；必须明确标记 `meta.demo=true`，不得将 Demo 数值解释为今日行情。

skills/stock-analyzer/SKILL.md

@@ -19,3 +19,10 @@ description: 对具体 A 股进行技术面、基本面、新闻、位置和风
 8. 输出趋势、信号、位置、风险、研究建议、数据时间和缺失信息。
 
 价格区间、止损和止盈必须说明依据。不得绕过 MCP 直接写 `plan.json`，不得提交真实订单，也不得把一次分析表述为收益保证。
+
+## MCP 响应契约
+
+- 每次调用先检查 `schema_version` 和 `ok`，分析字段只从 `data` 读取。
+- 报告数据来源时引用 `meta.source`、`meta.as_of` 和 `meta.freshness.status`；过期或时间未知的数据必须进入风险部分。
+- `ok=false` 时依据 `error.code` 决定重试或停止，不得把供应商异常当作个股事实。
+- 离线演示可读取 `stock_quote`、`stock_technical`、`stock_fundamentals` 和 `stock_news` Demo 数据集；必须注明是合成数据，不能形成真实股票建议。

skills/stock-screener/SKILL.md

@@ -20,3 +20,10 @@ description: 使用确定性规则筛选 A 股候选标的。用户要求选股
 6. 输出代码、名称、筛选依据、复核证据、风险和数据时间；结果不足时不要补齐数量。
 
 如需进一步分析某只股票，转入 `$stock-analyzer`。如需形成模拟盘计划，必须另外读取目标 paper run、校验计划并通过 MCP 保存草稿；本 Skill 不直接发布计划。
+
+## MCP 响应契约
+
+- 每次调用先检查 `schema_version` 和 `ok`，候选数据只从 `data` 读取。
+- 输出必须携带 `meta.source`、`meta.as_of` 和 `meta.freshness.status`；`stale` 或 `unknown` 数据不能支持强结论。
+- `ok=false` 时使用 `error.code` 说明失败，不得为了凑足候选数量而虚构结果。
+- 离线演示可依次读取 `screen_candidates`、`stock_quote`、`stock_technical`、`stock_news` Demo 数据集，并显式标注 `meta.demo=true`；Demo 候选不能转成模拟盘计划。