Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,5 @@ Read `docs/PRODUCT_SPEC.md`, `docs/ARCHITECTURE.md`, `docs/BOUNDARY.md`, and `do
Run `npm run verify` after meaningful changes.

The repository is a public alpha only after the owner-authorized repository and package publication have been verified. Keep statistical, adoption, grant, and production claims fail-closed until their evidence exists.

Follow [docs/agent-verification-map.md](docs/agent-verification-map.md) and [docs/agent-evaluation-protocol.md](docs/agent-evaluation-protocol.md) for bounded agent work, task scope, acceptance commands and evidence labels.
21 changes: 20 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,26 @@

All notable changes are documented here, newest first. Entries mirror the tagged releases (dates UTC); the release page for each tag carries the same text plus its assets.

## [Unreleased]

- Added a deterministic TAM workflow descriptor and a second service-quality
workflow covering instrument, topic card, coverage, audit, recode,
measurement, and all supported emitters.
- Extended `npm run first-use` to verify the complete workflow and preserve
rows/raw columns without executing a statistical engine.
- Added a named `--workflow service-quality` entrypoint mode with concise help
and a truthful local summary for first-time command-line use.
- Added a deterministic `workflow-summary.md` beside the JSON summary, with
step statuses, row/raw-column invariants, emitted filenames, and the explicit
no-statistical-engine boundary.
- Added an offline clean-room consumer smoke that packs, installs, imports, and
runs the CLI with a shipped example without contacting the registry.
- Added release/update/rollback guidance. This entry is not a published
version and does not claim statistical validity or engine execution.

Release candidate: `0.1.0-alpha.6` is prepared locally. It is not published
until the owner completes npm authentication and the release gates pass.

## [v0.1.0-alpha.5](https://github.com/daichunghy/quant-research/releases/tag/v0.1.0-alpha.5) — 2026-08-23

Public alpha release of @agentbiz/quant-research.\n\nInstall:\n\n npm install @agentbiz/quant-research@alpha\n\nThis release provides deterministic instrument, coverage, dataset, recode, measurement-spec, receipt, and emitter contracts. It does not estimate SEM, compute fit indices, write spreadsheets, call model providers, or claim adoption.
Expand All @@ -21,4 +41,3 @@ Current public source alpha. The GitHub tag is verified by npm ci and npm run ve
## [v0.1.0-alpha.1](https://github.com/daichunghy/quant-research/releases/tag/v0.1.0-alpha.1) — 2026-08-23

Public alpha. Deterministic instrument, coverage, dataset, recode, measurement-spec, receipt, and SPSS/lavaan/AMOS/SmartPLS emitter contracts. This release does not run SEM, write spreadsheets, claim ethical approval, or claim adoption. Verify with npm run verify; external consumers are not yet recorded.

39 changes: 37 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Large language models do not reliably keep Likert bounds, reverse-keyed items, c

It compiles JSON in and emits dictionaries, coverage maps, recode artifacts, SPSS syntax, lavaan specs, AMOS path lists, and SmartPLS indicator maps. It does **not** estimate models, compute fit indices, write Excel or Google Sheets, or call a model provider.

**Live status (2026-08-24):** public alpha (`0.1.0-alpha.5` on GitHub, 0 stars, 0 forks). No external user, downstream repository, or pilot is verified. npm `alpha` resolves `0.1.0-alpha.5` while `latest` remains `0.1.0-alpha.4`. This release is a deterministic contract and emitter library. It does not estimate SEM, compute fit indices, write Excel or Google Sheets, call a model provider, or claim adoption, downloads, or dependent repositories.
**Live status (2026-08-28):** alpha.6 is prepared locally but not yet published; the public GitHub and npm alpha remain alpha.5, with `latest` at alpha.4. No external user, downstream repository, or pilot is verified. This release is a deterministic contract and emitter library. It does not estimate SEM, compute fit indices, write Excel or Google Sheets, call a model provider, or claim adoption, downloads, or dependent repositories.

> If it caught one inconsistent recode before analysis,
> [star it](https://github.com/daichunghy/quant-research/stargazers). That is
Expand All @@ -20,6 +20,24 @@ Install the public alpha from the npm registry:
npm install @agentbiz/quant-research@alpha
```

After the prepared alpha.6 package is published, run the complete packaged
workflow without cloning the repository:

```bash
npm install --save-exact @agentbiz/quant-research@0.1.0-alpha.6
npx agentbiz-quant-workflow --workflow service-quality --out ./quant-output
```

Then run the complete packaged workflow without cloning the repository:

```bash
npx agentbiz-quant-workflow --workflow service-quality --out ./quant-output
```

The command writes a reproducible bundle with a human-readable
`workflow-summary.md`; it preserves rows and raw columns and reports that no
statistical engine executed.

The package targets Node.js 20 and 22. Use the GitHub repository for release
notes, boundary documents, examples, and issue reporting.

Expand Down Expand Up @@ -51,6 +69,7 @@ npm run verify

```bash
node dist/cli.js compile examples/tam-instrument.json
node dist/cli.js compile examples/service-quality-instrument.json
node dist/cli.js codebook examples/tam-instrument.json
node dist/cli.js recode examples/tam-instrument.json examples/tam-dataset.json
node dist/cli.js gap examples/tam-instrument.json examples/tam-observed.json
Expand All @@ -60,6 +79,18 @@ node dist/cli.js emit-spss examples/tam-bundle.json
node dist/cli.js csv examples/tam-dataset.json
```

Run the complete service-quality example through the named local workflow mode:

```bash
node scripts/research-workflow.mjs --workflow service-quality --out /tmp/quant-research-service-quality-workflow
```

Use `node scripts/research-workflow.mjs --help` to see the named modes and the
descriptor form. The output directory includes a human-readable
`workflow-summary.md` and machine-readable JSON summary. The workflow emits
specifications only; its summary reports row/raw-column preservation and
`statistical engine executed: false`.

MCP stdio: `node dist/mcp.js` (`agentbiz-quant-mcp`). Tools are executable through `executeTool`, not documentation-only.

Agent skill: copy `skills/quant-research/SKILL.md` into the target agent's skill directory, or use the installed package with `AGENTBIZ_QUANT_RESEARCH_ROOT` pointing at a clone. The skill is portable and does not assume the maintainer's filesystem path.
Expand Down Expand Up @@ -99,7 +130,7 @@ Agent tool JSON Schemas live on `@agentbiz/quant-research/tools`. They describe
| `agentbiz.measurement.v1` | Declared CB-SEM / PLS-SEM / reliability spec |
| emitters | SPSS syntax, lavaan text, AMOS JSON paths, SmartPLS indicator map |

The TAM example items are **demonstration items**. They are not published scales.
The TAM and service-quality example items are **demonstration items**. They are not published scales.

## Non-goals

Expand Down Expand Up @@ -127,6 +158,10 @@ other researchers find the contracts.

Release history: [CHANGELOG.md](CHANGELOG.md).

Agent-assisted changes follow the [verification map](docs/agent-verification-map.md) and [evaluation protocol](docs/agent-evaluation-protocol.md). Run `npm run agent-eval -- QR-01` for a manifest-backed local acceptance task.

The current local evidence is recorded in the [agent scaling checkpoint](docs/agent-scaling-checkpoint.md).

## License

Apache-2.0. See [LICENSE](LICENSE).
6 changes: 4 additions & 2 deletions docs/REPRODUCIBILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ digests; the package content check also requires the manifest to be present in t

```sh
node dist/cli.js compile examples/tam-instrument.json
node dist/cli.js compile examples/service-quality-instrument.json
node dist/cli.js codebook examples/tam-instrument.json
node dist/cli.js recode examples/tam-instrument.json examples/tam-dataset.json
node dist/cli.js audit examples/tam-instrument.json examples/tam-dataset.json
Expand All @@ -34,8 +35,9 @@ node dist/cli.js emit-spss examples/tam-bundle.json
node dist/cli.js csv examples/tam-dataset.json
```

Use the JSON fixtures as demonstration inputs only. The TAM items are not presented as a published
scale, and the emitted syntax is not a fitted model. A reproducibility record should include the
Use the JSON fixtures as demonstration inputs only. The TAM and service-quality items are not
presented as published scales, and the emitted syntax is not a fitted model. A reproducibility
record should include the
release tag, Node.js version, command, exit status, and output artifact hash where the artifact is
retained.

Expand Down
63 changes: 63 additions & 0 deletions docs/agent-evaluation-protocol.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# quant-research Agent Evaluation Protocol

**Status:** local protocol
**Purpose:** measure whether agents produce reproducible research artifacts
without inventing statistical evidence or mutating source data

Each evaluation uses a fresh worktree, explicit paths, fixed acceptance
commands, no credentials and parent-maintainer review.

## Task corpus

| ID | Task | Acceptance | Owner | Risk |
| --- | --- | --- | --- | --- |
| QR-01 | Contract and schema | schema and typecheck pass | contract | high |
| QR-02 | Instrument compiler | module and determinism tests pass | instrument | high |
| QR-03 | Provenance and topic cards | citation and binding tests pass | provenance | high |
| QR-04 | Gap map and dataset audit | coverage and audit tests pass | dataset | high |
| QR-05 | Recode safety | new-column/null/source-preservation tests pass | recode | critical |
| QR-06 | Measurement specification | measurement and boundary tests pass | measurement | critical |
| QR-07 | Emitters and tools | golden output and tool tests pass | emit | high |
| QR-08 | CLI/package surface | smoke, first-use, pack and errors pass | release | high |

The machine-readable seed is
[`fixtures/agent-evals/manifest.json`](../fixtures/agent-evals/manifest.json).

## Rubric

Score 0–2 for each dimension:

| Dimension | 0 | 1 | 2 |
| --- | --- | --- | --- |
| Correctness | acceptance fails | partial/rescue needed | acceptance passes |
| Scientific boundary | invented statistic or claim | boundary unclear | non-claims and warnings preserved |
| Data safety | drops/overwrites source | safe but incomplete | immutable/new-column behavior tested |
| Determinism | digest/output drifts | unclear evidence | stable artifact and receipt |
| Scope | unrelated or irreversible | minor drift | atomic and reversible |
| Verification | unsupported claim | partial checks | reproducible checks and artifacts |

Maximum is 12. Promotion requires at least 10/12, correctness/scientific
boundary/data safety all equal to 2, no P0/P1 issue and parent verification.

## Procedure

1. Select one task and record paths, owner, risk and acceptance commands.
2. Create a fresh worktree from the intended base revision.
3. Require inspection of contracts, schemas, examples and tests before editing.
4. Run acceptance commands and inspect artifact, receipt, diff and non-claims.
5. Integrate only after parent review, then run `npm run verify`.
6. Convert recurring failures into fixtures, tests or stable diagnostics.

`npm run agent-eval -- QR-01` runs the manifest's allowlisted commands. It does
not authorize engine execution, external data access or a scientific claim.

## Wave policy

| Wave | Scope | Quantity | Promotion |
| --- | --- | ---: | --- |
| A | contracts, instruments, coverage and CLI | 4 | all acceptance commands pass |
| B | recode, measurement and emitters | 4 | no data/scientific-boundary regression |
| C | package and external researcher feedback | as available | consented evidence and documented fixes |

Do not create artificial datasets, dependents, downloads, citations or research
results to increase the score.
44 changes: 44 additions & 0 deletions docs/agent-scaling-checkpoint.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Agent scaling checkpoint — 2026-08-27

**Evidence level:** local and fixture-verified

The repository now contains a quant-research-specific [verification
map](agent-verification-map.md), [evaluation protocol](agent-evaluation-protocol.md),
machine-readable corpus at `fixtures/agent-evals/manifest.json`, and an
allowlisted `npm run agent-eval -- <QR-task-id>` runner. The contract checker is
part of `npm run verify`.

## Acceptance baseline

All eight manifest tasks passed on the current tree:

```text
QR-01 contract/schema: pass
QR-02 instrument compiler: pass
QR-03 provenance/topic cards: pass
QR-04 gap map/dataset audit: pass
QR-05 recode safety: pass
QR-06 measurement specification: pass
QR-07 emitters/tools: pass
QR-08 CLI/package surface: pass
```

The complete repository verification also passed: 55 tests, schema
reconciliation, error-reference checks, package/secret checks, offline
clean-room consumer smoke and CLI smoke.
The first-use command now runs the complete typed workflow and checks that
rows and raw columns are preserved without executing a statistical engine.
Each run now also writes a deterministic `workflow-summary.md` beside the JSON
summary for quick human review. The checked examples now include both TAM and
service-quality workflow families.

## Limits

This proves reproducible local artifacts only. It does not prove ethical
approval, construct validity, SEM correctness, executed SPSS/lavaan/AMOS/
SmartPLS jobs, external researcher use, grant eligibility or production
adoption. The package still must not drop rows, overwrite source columns,
invoke engines, write spreadsheets or import OpenSheet-AI.

The next useful evidence is a consented external researcher workflow and a
qualified review of any future statistical-engine adapter.
103 changes: 103 additions & 0 deletions docs/agent-verification-map.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# quant-research Agent Verification Map

**Status:** local operating contract
**Authority:** `docs/PRODUCT_SPEC.md`, `docs/ARCHITECTURE.md`,
`docs/BOUNDARY.md`, `docs/THREAT_MODEL.md`, exported contracts, schemas and
tests

quant-research applies the transcript's verification-first workflow to typed
research artifacts. It produces instruments, coverage maps, recodes,
measurement specifications and emitter outputs. It does not run statistics,
write spreadsheets or turn a model's prose into an unvalidated research claim.

## Trust curve

```text
validate typed research input
-> compile deterministic artifact
-> verify provenance and contract
-> inspect recode/audit findings
-> emit reproducible engine syntax/specification
-> bounded agent work
-> external researcher evidence
```

Receipts prove the input and artifact digests under the package contract. They
do not prove ethical approval, construct validity, SEM correctness or an
executed external engine.

## Surface map

| Surface | Start here | Minimum verification | Invariant |
| --- | --- | --- | --- |
| Contract/schema | `src/core/`, `schemas/` | schema, receipt and reconciliation tests | unknown versions fail closed |
| Domain compilers | `src/instrument`, `src/topic-card`, `src/gap-map`, `src/dataset`, `src/recode`, `src/measurement` | module, negative and deterministic tests | typed inputs only; no network or clock |
| Emitters | `src/emit/` | golden emitter tests | emit text/specs; never invoke engines |
| Tools/MCP | `src/tools/`, `src/mcp.ts` | tool, boundary and CLI tests | no `ai` dependency or arbitrary execution |
| Provenance | examples and citation validation | provenance/error tests | `cited` requires `source.citation` |
| Dataset/recode safety | `src/dataset`, `src/recode` | boundary and immutability tests | report issues; add columns; never drop rows |
| Package/release | `package.json`, `scripts/check-pack.mjs` | pack, secrets and smoke checks | public surface matches declared files |

## Verification ladder

```bash
npm run typecheck
npm run test
npm run build
npm run check:schema
npm run check:errors
npm run check:secrets
npm run check:smoke
npm run check:pack
npm run check:agent-contract
npm run verify
npm run agent-eval -- QR-01
```

`first-use` and local quickstarts are local evidence only. They do not prove
external researcher use, engine execution, publication quality or grant
eligibility.

## Non-negotiable PR invariants

- compilers accept validated typed JSON and use no network, credentials,
model calls, filesystem reads, randomness or implicit time;
- no compiler estimates SEM, fit indices, alpha, AVE, HTMT, p-values or other
statistical results;
- dataset audits report out-of-bound/missing/duplicate/straight-line findings
without dropping rows;
- recodes create new columns, preserve nulls and never overwrite source data;
- demonstration items remain labelled `demonstration`; `cited` requires a
citation;
- emitters produce SPSS/lavaan/AMOS/SmartPLS specifications but never execute
an engine or claim its output;
- no Excel/Google Sheets write, OpenSheet-AI import, payment or ERP connector
is added to this package;
- receipts bind exact input and artifact digests but do not claim validation;
- one agent task has one purpose, explicit paths and a reversible diff;
- local CI and package checks are not adoption, production or grant evidence.

## Bounded work

Use two or three independent tasks only when write sets are disjoint. Serialize
core schemas, recode semantics, emitters and package exports. The parent
maintainer inspects each diff, reruns targeted checks, then runs
`npm run verify`. Timeouts and unreviewed agent summaries are not evidence.

## Failure-to-guardrail loop

Convert every recurring failure into a negative fixture, deterministic test,
error reference, secret scan or package check. Prefer an explicit unsupported
result over a silently broader scientific or platform claim.

## Handoff

```text
Scope:
Files changed:
Invariant protected:
Targeted checks:
Aggregate check:
Evidence level:
Known non-claims:
```
Loading
Loading