Fix STXM line-scan setup and widget UI

.github/workflows/ci.yml

@@ -0,0 +1,21 @@
+name: CI
+
+on:
+  push:
+    branches: [main, master]
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v5
+        with:
+          enable-cache: true
+      - name: Install dependencies
+        run: uv sync --all-groups
+      - name: Ruff
+        run: uv run ruff check src tests
+      - name: Pytest
+        run: uv run pytest -q

.gitignore

@@ -15,3 +15,5 @@ src/stxm.egg-info/PKG-INFO
 src/stxm.egg-info/requires.txt
 src/stxm.egg-info/SOURCES.txt
 src/stxm.egg-info/top_level.txt
+.cursor/hooks/state/continual-learning-index.json
+.cursor/hooks/state/continual-learning.json

AGENTS.md

@@ -1,17 +1,21 @@
 ## Learned User Preferences
 
 - Plot and figure must appear in the same output as the widget buttons and update when Load or other actions run; avoid separate plt.show() so the figure is embedded with controls (e.g. fig.canvas in VBox).
+- Prefer production-grade, numerically sound code with no placeholder text, no emojis, and no inline comments; use NumPy-style docstrings for documentation.
+- For Python projects use uv as the package manager; for TypeScript/JavaScript projects use bun instead of npm.
 
 ## Learned Workspace Facts
 
 - STXM toolkit processes beamtime line-scan data from .hdr/.xim files in an experiment folder.
-- Widget takes parent directory of experiments; use experiment dropdown to pick folder, then file dropdown lists only valid NEXAFS line scans (Type = "NEXAFS Line Scan" with loadable 2D .xim). Selecting a file loads it.
+- Widget takes parent directory of experiments; experiment dropdown lists subdirs sorted by date (names like yyyy-mm(Month) or yyyy_mm(Month)), latest first. File dropdown lists only valid NEXAFS line scans (Type = "NEXAFS Line Scan" with loadable 2D .xim). Selecting a file loads it.
 - Line-scan file list is filtered via list_nexafs_line_scans: only .hdr with Type = "NEXAFS Line Scan" and correct 2D shape; Image Scan, Focus Scan, and stacks are excluded.
-- Sample and izero regions are set per scan via draggable bars; 3-region segmentation (sample, edge, izero; edge thinnest) initializes bar bounds via bar_bounds_from_three_regions. Multiple sample regions supported with add/remove and per-region spot labels.
-- NEXAFS OD = ln(I0/I); dataset normalization is pre-edge baseline subtraction then scale so post-edge mean = 1.
-- Experiment parquet stores NEXAFS columns plus formula, scan_path, sample_name, spot_label, film_region_name; optional OD_normalized and derived columns (mass_absorption, mass_absorption_err, beta, beta_err). Create parquet and parent dirs if missing when appending.
-- Setup tab: directory, experiment dropdown, export file name (defaults to experiment.parquet), and sample config JSON. Config can be loaded from JSON path or auto-loaded when config.json exists in the selected experiment directory.
-- Reduction tab: sample from config dropdown, film region, line-scan file, region spot labels with add region and per-row remove (trash) and edit (pencil) buttons, then image and OD plot. Sample choice sets formula for export from config map.
-- Views tab: display mode (OD, mass absorption, beta), mass-abs fit option, spectrum plot, and Export button. No batch Process all in current widget.
-- When loading a new image, use a single consistent update path (set_data, set_extent, set_clim, axes/line updates) to avoid display corruption.
+- Sample and izero regions (A-B, C-D bars) are set per scan; defaults come from three-region segmentation (sample, edge, izero via bar_bounds_from_three_regions, edge thinnest); bars remain draggable. Pre-edge and post-edge (eV) define dataset normalization ranges.
+- NEXAFS OD = ln(I0/I); normalization modes are `pre_edge_scale` (pre-edge baseline + post-edge scale to 1) and `scale_shift` (adds energy shift to align post-edge when samples drift).
+- Experiment parquet stores NEXAFS columns plus formula, scan_path, optional OD_normalized, and optional derived columns mass_absorption, mass_absorption_err, beta, and beta_err; parquet and parent dirs are created if missing when appending. In Setup, parquet and sample config are filenames only (e.g. experiment.parquet, samples.json); resolved against the selected experiment directory. Auto-load prefers samples.json, then config.json.
+- Interactive line-scan widget has two top-level tabs: Dashboard and Ingestion. Dashboard has a shared header (parent dir, experiment, parquet path, store root, refresh) and nested sub-tabs: Preview spectra (parquet/store browser, scan checkboxes, sample/region filters, overlay plot) and LC fitting (component-based LCF: target blend spectrum, film component rows with material name, reference spectrum, initial/min/max %, fixed checkbox, live preview, Run LCF, composition %, fit + residual panels). LCF uses spectra from the current reduction, loaded parquet, and store catalog; normalization basis follows Ingestion Raw vs Normalized OD settings. Ingestion tab: two-column layout (left: regions list with add button, narrow line scan map, export; right: spectrum controls and wide OD plot), line scan select, draggable region bars, normalization mode, SciencePlots-styled spectra with draggable legends; export to parquet and optional store.
+- Process all batch processing is not available in the current widget; export works on the currently reduced scan and its defined regions only.
+- When loading a new image, use a single consistent update path (set_data, set_extent, set_clim via plotting.apply_line_scan_image_clim with grayscale and percentile limits on raw counts) to avoid display corruption.
 - File dropdown defaults to the first valid line scan so a selection is ready and changing selection loads that scan.
+- line_scan_processor displays the widget and does not return a dataframe; read exported spectra with load_experiment_parquet.
+- Export can write legacy experiment.parquet and an append-only partitioned spectrum store (store.py) when a store root is set; region averaging uses selectable WeightingMode (default POISSON_MLE).
+- Region ROI bar bounds and spot labels persist per scan in experiment-dir `regions.json` (keyed by `.hdr` basename); the Ingestion tab restores saved regions on load and debounced auto-saves on drag, add/remove, label edit, and export.

pyproject.toml

@@ -17,17 +17,25 @@ dependencies = [
     "pyarrow",
     "scikit-image>=0.26.0",
     "scikit-learn>=1.8.0",
-    "panel>=1.8.9",
-    "holoviews>=1.22.1",
-    "hvplot>=0.12.2",
-    "jupyter-bokeh>=4.0.5",
-    "ipywidgets-bokeh>=1.6.0",
+    "scipy>=1.17.1",
+    "scienceplots>=2.2.1",
 ]
 
 [tool.setuptools.packages.find]
 where = ["src"]
 
 [dependency-groups]
 dev = [
+    "pytest>=9.0.3",
     "ruff>=0.15.6",
 ]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+
+[tool.ruff]
+line-length = 100
+target-version = "py313"
+
+[tool.ruff.lint]
+select = ["E", "F", "I", "UP"]

src/stxm/__init__.py

@@ -1,30 +1,10 @@
-from stxm.io import (
-    load_stxm,
-    read_hdr,
-    read_xim,
-    is_valid_line_scan,
-    is_nexafs_line_scan_type,
-    is_nexafs_line_scan,
-    list_nexafs_line_scans,
-)
-from stxm.regions import (
-    sample_izero_masks,
-    auto_sample_izero_regions,
-    bar_bounds_from_three_regions,
-    segment_spatial_regions,
-)
-from stxm.nexafs import nexafs_beer_lambert
 from stxm.absorption import (
-    mass_absorption_cm2_per_g,
     fit_bare_atom_background,
+    mass_absorption_cm2_per_g,
     od_to_beta,
 )
-from stxm.normalization import (
-    energy_region_mask,
-    pre_edge_subtract,
-    post_edge_normalize,
-    normalize_nexafs,
-)
+from stxm.demix import Decomposition, demix_nmf, demix_svd
+from stxm.estimators import WeightingMode, region_mean_and_sigma
 from stxm.experiment import (
     CHEMICAL_FORMULA_COLUMN,
     FILM_REGION_NAME_COLUMN,
@@ -34,7 +14,36 @@
     load_experiment_parquet,
     process_experiment_folder,
 )
-from stxm.ui_panel import line_scan_processor, interactive_izero_split
+from stxm.io import (
+    is_nexafs_line_scan,
+    is_nexafs_line_scan_type,
+    is_valid_line_scan,
+    list_nexafs_line_scans,
+    load_stxm,
+    read_hdr,
+    read_xim,
+)
+from stxm.lcf import LCFResult, Spectrum, fit_lcf, preview_lcf_model
+from stxm.nexafs import nexafs_beer_lambert
+from stxm.normalization import (
+    NormalizationMode,
+    apply_normalization_mode,
+    energy_region_mask,
+    normalize_nexafs,
+    normalize_nexafs_with_metadata,
+    post_edge_normalize,
+    pre_edge_subtract,
+)
+from stxm.plotting import image_display_limits, make_draggable_legend, style_axes, use_science_style
+from stxm.reduction import RegionSpectrum, reduce_loaded_scan_two_region
+from stxm.regions import (
+    auto_sample_izero_regions,
+    bar_bounds_from_three_regions,
+    sample_izero_masks,
+    segment_spatial_regions,
+)
+from stxm.store import Provenance, list_manifest, query_spectra, write_spectrum
+from stxm.ui import line_scan_processor
 
 __all__ = [
     "load_stxm",
@@ -48,14 +57,36 @@
     "auto_sample_izero_regions",
     "bar_bounds_from_three_regions",
     "segment_spatial_regions",
+    "WeightingMode",
+    "region_mean_and_sigma",
     "nexafs_beer_lambert",
+    "RegionSpectrum",
+    "reduce_loaded_scan_two_region",
+    "Provenance",
+    "write_spectrum",
+    "query_spectra",
+    "list_manifest",
+    "Spectrum",
+    "LCFResult",
+    "fit_lcf",
+    "preview_lcf_model",
+    "Decomposition",
+    "demix_svd",
+    "demix_nmf",
     "mass_absorption_cm2_per_g",
     "fit_bare_atom_background",
     "od_to_beta",
+    "NormalizationMode",
+    "apply_normalization_mode",
     "energy_region_mask",
     "pre_edge_subtract",
     "post_edge_normalize",
     "normalize_nexafs",
+    "normalize_nexafs_with_metadata",
+    "image_display_limits",
+    "make_draggable_legend",
+    "style_axes",
+    "use_science_style",
     "CHEMICAL_FORMULA_COLUMN",
     "FILM_REGION_NAME_COLUMN",
     "SAMPLE_NAME_COLUMN",
@@ -64,5 +95,4 @@
     "load_experiment_parquet",
     "process_experiment_folder",
     "line_scan_processor",
-    "interactive_izero_split",
 ]