Audit implementation: correctness, streaming honesty, speed, and a smaller core

.claude/skills/konfai-experiments/SKILL.md

@@ -34,7 +34,7 @@ This is the tool order verified by the segmentation and synthesis end-to-end tes
 the discovery steps only when the dataset and task are already understood.
 
 **Route first (cheapest fit wins)**
-0. `list_apps` → `describe_app` → `run_app_infer`: when the user wants a RESULT, check whether a published app already solves it BEFORE authoring and training from scratch. `run_app_infer` / `run_app_evaluate` / `run_app_uncertainty` / `run_app_pipeline` run the app AS PUBLISHED (no config editing), and `fine_tune_app` adapts it to the user's dataset. `import_app` is the modify-then-run path: it copies the app into the session so it runs as a normal experiment (`run_prediction`, or `run_resume` with `weights_only=True` to fine-tune from its weights). `run_resume` (without `weights_only`) continues an interrupted session training.
+0. `list_apps` → `describe_app` → `run_app_infer`: when the user wants a RESULT, check whether a published app already solves it BEFORE authoring and training from scratch. `run_app_infer` / `run_app_evaluate` / `run_app_uncertainty` / `run_app_pipeline` run the app AS PUBLISHED (no config editing), and `fine_tune_app` is the one-call path to adapt it to the user's dataset. `import_app` is the full-control tier: it copies the app into the session so it runs as a normal experiment (`run_prediction`, or `run_resume` with `weights_only=True` to fine-tune with custom losses or config surgery). `run_resume` (without `weights_only`) continues an interrupted session training.
 
 **Discover (dataset-driven)**
 1. `browse_dataset` → `inspect_dataset`: choose the real dataset root, see groups + sampled stats (`include_stats=False` for a fast structural peek; `groups=[...]` when you need intensity ranges for normalization).

.claude/skills/konfai-experiments/references/tool-reference.md

@@ -72,7 +72,7 @@ Use to EXPORT the full reproducibility record of one run: the job manifest (comm
 
 ### `fine_tune_app`
 
-Use to TRAIN by starting from a published app instead of a blank slate: fine-tune an existing app's checkpoint(s) on the user's dataset, WITHOUT authoring or editing a config. This is the middle option between run_app_infer (use as-is, no training) and design_config_strategy (author a config and train from scratch); it is also the safer alternative to import_app + run_resume(weights_only=True), which needs the copied Config.yml to be edited by hand. It launches a tracked training job and writes a resolvable app bundle (config + code + fine-tuned checkpoint) to the output directory, which you can then run with run_app_infer. TRUST GATE: resolving the app imports its Python code and pip-installs its requirements, so pass allow_untrusted_code=True to confirm you trust the source. Local and HuggingFace apps only. It does not author a config or adapt the dataset layout for you. Training knobs are first-class parameters (epochs, it_validation, lr, batch_size); set_parameters is for the app's MODEL tunables (bare names) or any config key by its full dotted path. Outputs: a job payload (status, resources, next_actions) plus the bundle output path. Next: wait_for_job, then run_app_infer on the produced bundle (then run_app_evaluate to score and rank this fine-tune against other training trials via leaderboard / compare_runs).
+Use to TRAIN by starting from a published app instead of a blank slate: fine-tune an existing app's checkpoint(s) on the user's dataset, WITHOUT authoring or editing a config. This is the middle option between run_app_infer (use as-is, no training) and design_config_strategy (author a config and train from scratch). The produced training runs like any app fine-tune; for full control (custom losses, config surgery) use import_app + run_resume(weights_only=True) instead. It launches a tracked training job and writes a resolvable app bundle (config + code + fine-tuned checkpoint) to the output directory, which you can then run with run_app_infer. TRUST GATE: resolving the app imports its Python code and pip-installs its requirements, so pass allow_untrusted_code=True to confirm you trust the source. Local and HuggingFace apps only. It does not author a config or adapt the dataset layout for you. Training knobs are first-class parameters (epochs, it_validation, lr, batch_size); set_parameters is for the app's MODEL tunables (bare names) or any config key by its full dotted path. Outputs: a job payload (status, resources, next_actions) plus the bundle output path. Next: wait_for_job, then run_app_infer on the produced bundle (then run_app_evaluate to score and rank this fine-tune against other training trials via leaderboard / compare_runs).
 
 ### `generate_folds`
 
@@ -88,7 +88,7 @@ Use to read the FULL evaluation metrics (per-case values + aggregates) of ONE na
 
 ### `import_app`
 
-Use to RUN a published KonfAI app as a NORMAL experiment in this session: the single path to use a local or HuggingFace app. It copies the app's config(s), custom code, and .pt checkpoints into the session root and pip-installs its requirements, so predict / fine-tune / evaluate then go through the ordinary run_prediction / run_resume / run_evaluation tools (no app-specific wrapper, no extra sub-folder). The copied checkpoints are returned so run_prediction can pass them as models, and run_resume(weights_only=True) warm-starts a fine-tune from them. TRUST GATE: copying+running the app's Python code and installing its requirements is the trust boundary, so you MUST pass allow_untrusted_code=True to confirm you trust the source. Local/HuggingFace apps only: a remote server keeps its code remote and cannot be imported (drive a remote app with konfai-apps directly). Outputs: imported_to, files, checkpoints, configs, next_actions. Next: run_prediction (pass checkpoints as models) / run_resume (fine-tune) / run_evaluation.
+Use to RUN a published KonfAI app as a NORMAL experiment in this session. Prefer run_app_* when the app is used exactly as published; import_app is the full-control tier: editing the config, fine-tuning with custom losses or config surgery (run_resume with weights_only=True; fine_tune_app is the one-call path when the app trains as published), or wiring the app into a larger experiment. It copies the app's config(s), custom code, and .pt checkpoints into the session root and pip-installs its requirements, so predict / fine-tune / evaluate then go through the ordinary run_prediction / run_resume / run_evaluation tools (no app-specific wrapper, no extra sub-folder). The copied checkpoints are returned so run_prediction can pass them as models, and run_resume(weights_only=True) warm-starts a fine-tune from them. TRUST GATE: copying+running the app's Python code and installing its requirements is the trust boundary, so you MUST pass allow_untrusted_code=True to confirm you trust the source. Local/HuggingFace apps only: a remote server keeps its code remote and cannot be imported (drive a remote app with konfai-apps directly). Outputs: imported_to, files, checkpoints, configs, next_actions. Next: run_prediction (pass checkpoints as models) / run_resume (fine-tune) / run_evaluation.
 
 ### `import_experiment`
 
@@ -220,7 +220,7 @@ Use after train config review and validation succeed. This launches a training j
 
 ### `run_transform`
 
-Use to apply a transform chain to a dataset with NO model: read, transform, write. Read plan_transform first: this writes a dataset, and the plan is what says how much and how. This launches a transform job from the session Transform.yml and returns structured job resources. The run replans, stores the plan, and refuses outright when a case can neither stream nor fit memory_budget. A case whose output already exists is skipped, so an interrupted run resumes; pass overwrite to recompute. Outputs: job payload with resources and next_actions; or a blocker payload when a prerequisite is missing. Next: wait_for_job then inspect_dataset on what it wrote.
+Use to batch-process a dataset with a transform chain: read, transform, write. Read plan_transform first: this writes a dataset, and the plan is what says how much and how. This launches a transform job from the session Transform.yml and returns structured job resources. Pass gpu to run the chain on a GPU (the resample and intensity stages move with it; every write still lands on the host). The run replans, stores the plan, and refuses outright when a case can neither stream nor fit memory_budget. A case whose output already exists is skipped, so an interrupted run resumes; pass overwrite to recompute. Outputs: job payload with resources and next_actions; or a blocker payload when a prerequisite is missing. Next: wait_for_job then inspect_dataset on what it wrote.
 
 ### `set_live_tunables`
 

.github/workflows/konfai_apps_ci.yml

@@ -54,11 +54,15 @@ jobs:
         uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
         with:
           python-version: ${{ matrix.python-version }}
+          cache: "pip"
+          cache-dependency-path: pyproject.toml
 
       - name: Install core package
+        # Linux adds the CPU torch index: the default PyPI torch wheel bundles the multi-GB CUDA
+        # stack a CPU-only runner never uses (macOS/Windows PyPI wheels are already CPU-only).
         run: |
           python -m pip install -U pip
-          pip install -e ".[dev]"
+          pip install -e ".[dev]" ${{ runner.os == 'Linux' && '--extra-index-url https://download.pytorch.org/whl/cpu' || '' }}
 
       - name: Restore Hugging Face cache
         uses: actions/cache@caa296126883cff596d87d8935842f9db880ef25 # v5
@@ -73,3 +77,17 @@ jobs:
       - name: Run apps tests
         run: |
           python -m pytest -q konfai-apps/tests
+
+      - name: Install app bundles
+        # Editable, so each bundle's scm-derived `konfai==`/`konfai-apps==` pin resolves against
+        # the editable installs above (both sides derive the same version from this checkout).
+        shell: bash
+        run: |
+          for app in apps/*/; do pip install -e "./$app"; done
+
+      - name: Run app bundle tests
+        # The five PyPI-published bundles. impact_reg's integration test gates itself
+        # (KONFAI_IMPACTREG_REPO + fireants + CUDA) and skips here; its unit tests stub the runtime.
+        shell: bash
+        run: |
+          for app in apps/*/; do python -m pytest -q "${app}tests"; done

.github/workflows/konfai_ci.yml

@@ -10,7 +10,7 @@ on:
       - "konfai/**"
       - "konfai-apps/**"
       - "tests/**"
-      - "docs/source/config_guide/**"
+      - "docs/**"
       - "examples/Transform/**"
       - "pyproject.toml"
       - "README.md"
@@ -20,7 +20,7 @@ on:
       - "konfai/**"
       - "konfai-apps/**"
       - "tests/**"
-      - "docs/source/config_guide/**"
+      - "docs/**"
       - "examples/Transform/**"
       - "pyproject.toml"
       - "README.md"
@@ -45,15 +45,20 @@ jobs:
         uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
         with:
           python-version: ${{ matrix.python-version }}
+          cache: "pip"
+          cache-dependency-path: pyproject.toml
 
       - name: Install dependencies
+        # Linux adds the CPU torch index: the default PyPI torch wheel bundles the multi-GB CUDA
+        # stack a CPU-only runner never uses (macOS/Windows PyPI wheels are already CPU-only).
         run: |
           python -m pip install -U pip
-          pip install -e ".[dev]"
+          pip install -e ".[dev]" ${{ runner.os == 'Linux' && '--extra-index-url https://download.pytorch.org/whl/cpu' || '' }}
 
       - name: Run pytest
+        # Same xdist mode as the pixi `test` task; loadfile keeps per-file session fixtures on one worker.
         run: |
-          pytest -q tests
+          pytest -q -n auto --dist loadfile tests
 
   lint:
     runs-on: ubuntu-latest
@@ -68,6 +73,8 @@ jobs:
         uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
         with:
           python-version: "3.12"
+          cache: "pip"
+          cache-dependency-path: pyproject.toml
 
       - name: Install ruff
         run: pip install ruff==0.15.2
@@ -88,13 +95,47 @@ jobs:
         uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
         with:
           python-version: "3.12"
+          cache: "pip"
+          cache-dependency-path: pyproject.toml
 
       - name: Install ruff
         run: pip install ruff==0.15.2
 
       - name: Check formatting
         run: ruff format --check konfai konfai-apps/konfai_apps tests
 
+  docs:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
+        with:
+          # This job runs code the pull request controls; the token has no business staying
+          # in .git/config while it does.
+          persist-credentials: false
+          # setuptools-scm derives both packages' versions from the tag history.
+          fetch-depth: 0
+
+      - name: Set up Python
+        uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
+        with:
+          python-version: "3.12"
+          cache: "pip"
+          cache-dependency-path: |
+            pyproject.toml
+            docs/requirements.txt
+
+      - name: Install dependencies
+        # autodoc imports konfai and konfai_apps, so both install from this checkout; the imaging
+        # extra keeps the guarded backend modules importable. docs/requirements.txt is the single
+        # source for the Sphinx toolchain (what ReadTheDocs installs).
+        run: |
+          python -m pip install -U pip
+          pip install -e ".[imaging]" -e ./konfai-apps -r docs/requirements.txt --extra-index-url https://download.pytorch.org/whl/cpu
+
+      - name: Build the docs strictly
+        # -W turns any warning into a failure; --keep-going still reports them all.
+        run: python -m sphinx -b html -W --keep-going docs/source /tmp/docs-html
+
   build:
     runs-on: ubuntu-latest
     steps:
@@ -109,13 +150,15 @@ jobs:
         uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
         with:
           python-version: "3.12"
+          cache: "pip"
+          cache-dependency-path: pyproject.toml
 
       - name: Install dependencies
         # The dev extra carries pytest + build; setuptools/setuptools-scm/wheel are the declared
         # build requirements the wheel test needs for a --no-isolation build.
         run: |
           python -m pip install -U pip
-          pip install -e ".[dev]" setuptools setuptools-scm wheel
+          pip install -e ".[dev]" setuptools setuptools-scm wheel --extra-index-url https://download.pytorch.org/whl/cpu
 
       - name: Build sdist and wheel
         run: python -m build --sdist --wheel

.github/workflows/konfai_mcp_ci.yml

@@ -39,11 +39,17 @@ jobs:
         uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
         with:
           python-version: ${{ matrix.python-version }}
+          cache: "pip"
+          cache-dependency-path: |
+            pyproject.toml
+            konfai-mcp/pyproject.toml
 
       - name: Install core package (with imaging backends)
+        # The CPU torch index: the default PyPI torch wheel bundles the multi-GB CUDA stack this
+        # CPU-only runner never uses.
         run: |
           python -m pip install -U pip
-          pip install -e ".[dev,imaging]"
+          pip install -e ".[dev,imaging]" --extra-index-url https://download.pytorch.org/whl/cpu
 
       - name: Install standalone apps package
         run: |

.github/workflows/konfai_studio_ci.yml

@@ -39,6 +39,10 @@ jobs:
         uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
         with:
           python-version: ${{ matrix.python-version }}
+          cache: "pip"
+          cache-dependency-path: |
+            pyproject.toml
+            studio/pyproject.toml
 
       - name: Set up Node
         uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
@@ -55,9 +59,11 @@ jobs:
           test -f ../konfai_studio/web/index.html
 
       - name: Install core package
+        # The CPU torch index: the default PyPI torch wheel bundles the multi-GB CUDA stack this
+        # CPU-only runner never uses.
         run: |
           python -m pip install -U pip
-          pip install -e ".[dev]"
+          pip install -e ".[dev]" --extra-index-url https://download.pytorch.org/whl/cpu
 
       - name: Install standalone apps package
         run: |
@@ -71,6 +77,18 @@ jobs:
         run: |
           pip install -e ./studio
 
+      - name: Build a throwaway studio wheel and verify it ships the front
+        # The wheel is the only artifact that carries web/ (git-ignored, swept in by package-data),
+        # so only inspecting a built wheel catches a packaging regression before release day.
+        run: |
+          python -m build --wheel studio
+          python -c "
+          import glob, sys, zipfile
+          names = zipfile.ZipFile(glob.glob('studio/dist/*.whl')[0]).namelist()
+          ok = 'konfai_studio/web/index.html' in names and any(n.startswith('konfai_studio/web/assets/') for n in names)
+          sys.exit(0 if ok else 'the studio wheel is missing konfai_studio/web/index.html or assets/')
+          "
+
       - name: Lint and format check
         run: |
           ruff check studio/konfai_studio