ATELIER (MVP)

Agentic Textual Exploration for Licensed, Iterative, Ephemeral Research — a supervised corpus-access and analysis workbench, prototyped as a module within Sinographia Diachronica.

This is a runnable MVP of the pipeline: research question → structured plan → policy check → human approval → bounded retrieval → ephemeral store → analysis (KWIC / collocation / diachronic comparison) → provenance log → raw text deleted → derived outputs kept.

Core principle: the agent decides what evidence is needed; deterministic tools decide how it's retrieved; a human authorises access; raw text lives only in ephemeral storage; outputs preserve provenance, not copied corpora. Not a crawler — a supervised workbench.

Run it

No API key, no network, no credentials needed — the demo runs on a bundled public-domain fixture corpus.

pip install pyyaml
# from the repo root (the directory holding run_demo.py and atelier/):
python run_demo.py

Outputs land in outputs/: report.md, diachronic_summary.csv, kwic.csv, provenance_log.jsonl.

The demo case study is 寂寞 across Tang→Qing. Even on the tiny fixture the expected pattern shows: Tang collocates skew spatial (空/山/夜), Song toward interiority (獨/愁/心), Ming–Qing toward interpersonal/psychological loneliness (人/情/淚/孤).

Layout

run_demo.py                    end-to-end 寂寞 case study (entry point)
atelier/                       the package
├── schemas/models.py          structured plan / evidence / provenance types
├── agents/
│   ├── planner.py             question -> structured SearchPlan (rule-based; LLM drop-in noted)
│   └── policy_guard.py        allowlist, volume caps, truncation, approval flags
├── adapters/
│   ├── base.py                the search()/fetch() interface
│   ├── mock_adapter.py        bundled public-domain fixture — the default source
│   └── licensed_db_adapter.py STUB (see below)
├── storage/
│   ├── ephemeral_store.py     in-memory SQLite + temp dir, deleted on exit
│   └── provenance.py          JSONL audit trail
├── analysis/analyze.py        KWIC, collocation-by-period, diachronic summary
├── pipeline.py                the orchestrator (with the human-approval gate)
└── policy.yaml                the enforced constraints

The one deliberate boundary: data sources

licensed_db_adapter.py is a stub on purpose, and it's the only piece that needs a human decision before any code goes in it.

A working version would reuse a saved authenticated browser session to pull records out of an access-restricted, licensed database. That is automated retrieval against a source whose licence almost always forbids exactly that, and the realistic consequence is the vendor revoking access for your whole institution and a breach of the agreement your library signed. The per-query caps and ephemeral storage here keep authorised use bounded — they do not turn unauthorised automated access into authorised use.

Point a real adapter at a source you're actually permitted to automate:

• Open corpora — CBETA (open access), Kanripo, the Daizhige/殆知閣 corpus, or the Chinese Text Project (ctext) API under its stated terms.
• Official TDM APIs — many paywalled vendors offer a text-and-data-mining API or research agreement precisely for this; ask your librarian. Implement against that endpoint, not the reading-room interface.
• Material you already hold rights to — manual uploads, your own digitisations, institutional repositories.

Any authorised adapter only needs to implement search() and fetch() and set source_name / access_mode so the provenance log records honestly how each record was obtained. The rest of the pipeline doesn't change.

Notes / extension points

• LLM planner: agents/planner.py is rule-based so the MVP runs offline. The LLM version prompts a model to emit JSON matching SearchPlan, then validates it — the schema is what keeps that output machine-checkable.
• KWIC window size is a licensing knob: on short texts a wide window can cover most of a line, so tune it to what your source permits.
• Sense clustering: analyze.py works at the character level (no segmentation needed to be useful). A segmenter (CKIP/jieba/pkuseg) or embedding-based clustering of context windows slots in without disturbing the pipeline.
• Human approval is a callback here so the demo runs unattended; in a Streamlit/Jupyter build it becomes an interactive pause (LangGraph's interrupt mechanism is a natural fit if you go that route).