diff --git a/CLAUDE.md b/CLAUDE.md index 4e33c5e..ec3d93f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -131,7 +131,7 @@ If the current branch was cut from a feature branch (not from `main`), open the When opening or merging a PR, add a summary entry to `MIGRATION_LOG.md` under `## Done` (newest first). -Before opening a PR, check [TODO.md](TODO.md) for related open items to update or remove. Also check and update [docs/logic_flows.md](docs/logic_flows.md) if any execution paths changed. +Before opening a PR, check [TODO.md](TODO.md) for related open items to update or remove. Also check and update [docs/logic_flows/](docs/logic_flows/) if any execution paths changed (edit_session.md for edit flows, picking.md for pick flows, etc.). ## Current Architecture @@ -148,7 +148,7 @@ Before opening a PR, check [TODO.md](TODO.md) for related open items to update o ### ParaView Backend Path -- `paraview_backend.py`: owns ParaView sources/displays, pipeline nodes, filters, coloring, display controls, picking, selection overlays, saving/export. `set_edit_target_dataset()` builds four remap caches at session begin (`_edit_source_dataset`, `_edit_source_point_indexes`, `_edit_target_cell_map`, `_edit_point_id_map`) so picks avoid per-pick Fetch/rebuild — see `docs/logic_flows.md §5b`. +- `paraview_backend.py`: owns ParaView sources/displays, pipeline nodes, filters, coloring, display controls, picking, selection overlays, saving/export. At edit session begin, `export_active_dataset_for_editing()` creates a real pipeline node (kind="edit") that reads the same backing data as the original node, so pick cell IDs directly index `working_dataset` with no remapping — see [docs/logic_flows/edit_session.md](docs/logic_flows/edit_session.md). - `paraview_runtime.py`: synchronizes backend state into Trame state, handles render pushes, edit-session overlay sync. Forwards event normalization calls to `paraview_event_utils.py`. - `paraview_event_utils.py`: pure helpers for normalizing ParaView picking event payloads (`normalize_edit_selection_ids`, `summarize_edit_event`, coordinate mapping). No Trame dependency. - `paraview_controllers.py`: user actions from the UI: pipeline actions, filters, edit sessions, selection, field creation, display/color controls. @@ -263,7 +263,7 @@ conda run -n coral-paraview pytest -q tests/ --ignore-glob=tests/test_e2e*.py ## Debugging Notes -- `docs/logic_flows.md` has detailed flow diagrams. +- [docs/logic_flows/](docs/logic_flows/) has detailed flow diagrams split by domain (`edit_session.md`, `picking.md`, `color.md`, etc.). - If `pytest` or `python` resolves to the wrong environment (ParaView unavailable): a stale virtualenv may be active and its `PATH` entry wins. Run `deactivate` if a venv is active (`deactivate` is only defined while a venv is sourced — if not found, no venv is active), then `conda activate coral-paraview`. - E2E test meshes live in `test_data/`; use them instead of writing into `data/` unless needed. - `tests/test_e2e_edit_selection_playwright.py` has helpers for normalized box drags and switch state checks. diff --git a/MIGRATION_LOG.md b/MIGRATION_LOG.md index e119b7d..9434fda 100644 --- a/MIGRATION_LOG.md +++ b/MIGRATION_LOG.md @@ -1,4 +1,4 @@ -# CODEX_LOG +# MIGRATION_LOG ## Goal @@ -6,6 +6,33 @@ Evolve the current VTK/trame viewer toward a ParaView-backed application while k ## Done +- Edit pipeline node for zero-remap picks (Step 2, issue #34, branch `perf/step2-edit-node`): + - replaced the per-pick remap-cache approach (Step 1, commit `298a3df`) with a real + pipeline node (kind="edit") that reads the same backing data as the original node, + so pick cell IDs directly index `working_dataset` with no coordinate-based remapping + - a dedicated reader proxy approach (originally planned for Step 2) was explored but + dismissed: it caused regressions in coloring, LUT, and representation handling because + those code paths all expect a regular pipeline node; a real node fits the existing + architecture with zero property overrides and no special-casing + - removed all four remap-cache fields (`_edit_source_dataset`, `_edit_source_point_indexes`, + `_edit_target_cell_map`, `_edit_point_id_map`) and five remap methods + - edit node label "✏ Editing: {original}", icon `mdi-pencil-outline`, visible in pipeline panel + - root readers reuse the same backing file (no write); filter nodes write a temp VTU + - `can_edit_active_node()` replaces the expensive export probe with a cheap `GetDataInformation` call + - surface picks skip a redundant `Fetch(source)` during an active session (use `working_dataset` directly) + - cached surface-pick geometry at session begin (B5, B5b): `set_edit_target_dataset` builds + `_edit_source_point_indexes` (O(n_points) coordinate index) and `_edit_boundary_elements` + (O(n_cells×faces) boundary map) once; + - the pipeline node model is also a better foundation for future improvements: real-time + display updates when cell values change (update the edit node in place), and server-side + field-value mutations via ParaView proxies to preserve MPI distribution without collapsing + all ranks through `Fetch()` + - `docs/logic_flows/` (split from monolithic `docs/logic_flows.md`), `CLAUDE.md`, and `TODO.md` updated + +- Eliminate per-pick Fetch and map rebuild with edit session remap cache (Step 1, issue #34, commit `298a3df`): + - pre-built four coordinate-based maps at session begin to avoid O(n) rebuild on every pick + - subsequently superseded by Step 2 above (edit node approach makes remapping unnecessary) + - UI restructure, centralized notifications, save dialog, and categorical LUT fixes (PR #32): - moved edit-panel controls into the right inspector Edit tab; added `docs/ui_structure.md` - replaced 9 scattered alert state pairs with a single `VSnackbar` driven by `notifications.py` diff --git a/TODO.md b/TODO.md index cfe7415..2518cca 100644 --- a/TODO.md +++ b/TODO.md @@ -2,11 +2,6 @@ ## Priorita alta -- **Performance selezione su dataset grandi** — bottleneck documentati in - `docs/logic_flows.md` §5a e §6. Piano a step progressivi in [issue #34](https://github.com/2listic/coral-visualizer/issues/34); ogni step - è indipendente e lascia i test esistenti verdi. Step 1 può essere fatto direttamente - su `paraview_backend.py` senza refactor preventivo. - - Separare `paraview_backend.py` in servizi di dominio più piccoli (3900 righe). Iniziare da moduli con dipendenze unidirezionali (`backend/display.py`, `backend/coloring.py`, `backend/pipeline.py`) — più self-contained e a minor rischio diff --git a/docs/logic_flows.md b/docs/logic_flows.md deleted file mode 100644 index 4173000..0000000 --- a/docs/logic_flows.md +++ /dev/null @@ -1,597 +0,0 @@ -# Logic Flows - -A walkthrough of the main execution paths in the Trame/ParaView visualizer. - -## 1. Startup (`app.py`) - -``` -app.py - ├── configure_app() → parse CLI flags, devtools, hot-reload - ├── create_runtime_context() → instantiate ParaViewBackend, initialize_view(), EditSession - ├── get_server() → Trame websocket server (Vue 2) - ├── initialize_state() → seed all Trame state variables with defaults - ├── build_ui() → declare Vuetify layout, bind state vars and ctrl methods - ├── ParaViewRuntime(...) → create Trame-aware runtime wrapper (state + view_controls now available) - └── register_app_handlers() - ├── register_paraview_controllers() → ctrl.add("pv_*") — user action handlers - ├── register_state_handlers() → @state.change("...") — reactive callbacks - └── register_common_controllers() → upload, refresh-files, etc. -``` - -Two-phase construction is intentional: `ParaViewBackend` / `EditSession` are created *before* Trame -(no `state` dependency), then `ParaViewRuntime` wraps them *after* (needs `state`). - ---- - -## 2. File Load - -Triggered when the user picks a file in the left drawer → `state.selected_file` changes. - -``` -state.selected_file change - └── on_file_change() [state_handlers.py:24] - └── paraview_runtime.load_file(path) [paraview_runtime.py:327] - ├── refresh_runtime_message(clear=True) — reset VTK output window offset - ├── pv_backend.load_file(path) — ParaView Reader → pipeline node - ├── apply_representation("Surface with Edges") - ├── apply_coloring(__solid__) - ├── reset_view() - ├── update_ui_state() — sync all pipeline/array/display state - └── render_and_push() - ├── pv_backend.render() - └── call_view_update() — push frame to browser -``` - ---- - -## 3. UI State Sync (`update_ui_state`) - -Called after almost every action. It is the "flush everything" step. - -``` -paraview_runtime.update_ui_state() [paraview_runtime.py:198] - ├── pv_backend.get_ui_state() — collects pipeline_items, arrays, display props, time info - ├── writes ~30 state.* variables — pipeline_items, selected_array, color_controls_*, etc. - ├── state.can_edit_active — probes whether export-for-editing is possible - └── sync_edit_session_state() — overlays edit-mode specific state on top -``` - ---- - -## 4. Controller → Backend Pattern - -Every button click in the UI calls a `ctrl.add("pv_*")` action registered in -`paraview_controllers.py`. The pattern is always: - -``` -UI click → ctrl.pv_something() [paraview_controllers.py] - ├── pv_backend.do_thing() — ParaView mutation - ├── update_paraview_ui_state() — refresh all derived state - └── render_and_push() — render + send frame to browser -``` - -Example — adding a filter: - -``` -ctrl.pv_add_filter(filter_key) [paraview_controllers.py:689] - ├── pv_backend.add_filter(filter_key) - ├── update_paraview_ui_state() - └── render_and_push() -``` - ---- - -## 5. Edit Session Flow - -``` -[Begin] -ctrl.pv_begin_edit_session() - ├── pv_backend.export_active_dataset_for_editing() — pulls dataset from ParaView server into a local VTK object (no further server involvement) - ├── edit_session.begin(node_id, label, filename, dataset) — DeepCopy into working_dataset, reset all session state, precompute CellCenters array - ├── pv_backend.set_edit_target_dataset(working_dataset) — registers working_dataset so picks can be remapped onto it - ├── state.pick_mode = True, mainViewMode = "remote" — enables hardware picking; forces server-side rendering - └── sync_edit_session_state() + render_and_push() — flush edit state to UI and send first frame - -[Selection — click or box drag] -ctrl.pv_edit_click_selection(event) / ctrl.pv_edit_box_selection(event) — entry point for click and box drag gestures - ├── normalize_edit_selection_ids(event) — extract screen coords or composite IDs - ├── _pick_edit_ids_at_coords(mode, x, y) / _pick_edit_ids_in_rect(mode, ...) — dispatch to click or rect pick - │ └── pv_backend.pick_visible_cell_ids(x, y) — ParaView hardware pick - └── _apply_selection_ids(picked_ids, ...) — apply selection mode (replace/add/subtract/flip) and update overlay - ├── edit_session.replace/add/subtract/flip_selection(picked_ids) — mutate selected_cell_ids / surface_keys / point_ids - ├── sync_edit_session_state() — update selection count and field readiness in Trame state - ├── sync_paraview_edit_selection_overlay() — highlight selected cells in view - └── render_and_push() — send updated frame to browser - -[Assign value] -ctrl.pv_apply_edit_field() — triggered by "Apply" button in Edit Tools panel - └── _apply_edit_field() — resolves field, runs expression, writes values into working_dataset - ├── _parse_field_choice() — "cell:FieldName" → (association, name) - ├── edit_session.assign_to_selected(field_name, association, expression) — evaluate expression via vtkArrayCalculator and write results to selected cells/points - └── sync_edit_session_state() — refresh dirty flag and field state in UI - -[Commit] -ctrl.pv_commit_edit_session() — opens save dialog; actual write happens in _commit_edit_session - └── _commit_edit_session() — save, tear down session, reload result as new pipeline node - ├── save_paraview_output() — write modified dataset to disk - ├── edit_session.clear() — reset all session state and release working_dataset - ├── pv_backend.load_file(output_path) — add saved file back as new pipeline node - ├── update_paraview_ui_state() — full state flush: arrays, display, pipeline tree - └── render_and_push() — send final frame showing new node -``` - -### 5a. Data movement, process model, and bottlenecks - -The diagram below tracks what happens to the mesh data across the same four phases -as section 5. Read section 5 to trace function calls; read this to understand data -ownership, copies, and where the performance costs land. - -The app is **single-process** (one Python process hosts Trame, ParaView pipeline, and -edit logic). VTK C++ filter work uses multi-thread internally via `vtkMultiThreader`; -Python orchestration is single-threaded. There is no MPI: all data lives in one process. -If a pvserver were connected via `simple.Connect()`, `servermanager.Fetch()` would -still collapse all distributed ranks into this one process at the points marked below. - -``` -─── SESSION BEGIN ────────────────────────────────────────────────────────────────── - [paraview_controllers.py:721 pv_begin_edit_session] - [paraview_backend.py:400 export_active_dataset_for_editing] - [edit_session.py:71 EditSession.begin] - - ParaView in-process pipeline (proxy layer, vtkSMProxy stubs) - source proxy → UpdatePipeline() - │ - ▼ - servermanager.Fetch(source) ⚠ BOTTLENECK: entire mesh transferred into - [paraview_backend.py:410] one Python vtkUnstructuredGrid object. - │ In MPI mode all ranks would be merged here. - ▼ - vtkUnstructuredGrid (local Python) - │ - ▼ - vtkUnstructuredGrid.DeepCopy() ⚠ full mesh copy — all points, all cells of - [edit_session.py:85] all dimensions, all point/cell data arrays. - ▼ - edit_session.working_dataset independent copy; field assignments, - │ selections, and expressions operate only - │ on this dataset — picks still go through - │ the ParaView source and are remapped here. - │ - ├── pv_backend.set_edit_target_dataset(working_dataset, source_dataset=exported["dataset"]) - │ [paraview_backend.py:156] - │ registers copy so subsequent picks can be remapped onto it; - │ also pre-builds four remap caches from the already-fetched source: - │ _edit_source_dataset — the fetched source (avoids re-Fetch per pick) - │ _edit_source_point_indexes — coord → point_id index for the source - │ _edit_target_cell_map — cell geometry → cell_id map for working_dataset - │ _edit_point_id_map — source point_id → working_dataset point_id map - │ source_dataset is passed in from export_active_dataset_for_editing so no - │ extra Fetch is needed here - │ - └── _ensure_cell_centers_array() - [edit_session.py:1167] - adds CellCenters cell-data array (XYZ per cell centroid) so - vtkArrayCalculator can reference spatial coords in expressions - -─── DURING SESSION — two parallel structures ────────────────────────────────────── - - ParaView side (pipeline proxy) Edit session side (working_dataset) - ──────────────────────────────── ────────────────────────────────────── - original source rendered in viewport holds all mutations (assigned field values) - picked by SelectSurfaceCells() tracks selected_cell_ids / surface_keys / - selected_point_ids - - cell IDs from rendered source - │ - ▼ - _remap_cell_ids_to_edit_target_dataset() Uses pre-built caches: - [paraview_backend.py:2752] _edit_source_dataset → no Fetch per pick - _remap_surface_keys_to_edit_target_dataset() _edit_target_cell_map → no map rebuild - [paraview_backend.py:2812] _edit_point_id_map → O(1) point lookup - │ - │ Fallback: if caches are absent (e.g. tests - │ that set _edit_target_dataset directly), - │ both methods rebuild the structures inline — - │ same correctness, O(n) per pick as before. - ▼ - IDs on working_dataset - - CellCenters array (cell data) GetPoint() calls (raw mesh geometry) - └─ vtkArrayCalculator reads it └─ _ensure_surface_element_vectors() - when evaluating user expressions [edit_session.py:995] - (e.g. "CellCenters[0] > 2.5") computes unit normals/tangents for - Stripped from output on save. dihedral angle filter during grow. - Never stored as an array. - -─── COMMIT ───────────────────────────────────────────────────────────────────────── - [paraview_controllers.py:108 _commit_edit_session] - - edit_session.working_dataset - │ - ├── _remove_internal_edit_arrays() strips CellCenters before write - │ [edit_session.py:1200] - ▼ - vtkXMLUnstructuredGridWriter.Write() ⚠ full mesh write to disk - │ - ▼ - pv_backend.load_file(output_path) ⚠ full mesh re-read into ParaView pipeline - │ ParaView owns it again as a new node. - ▼ - new pipeline node — normal visualization resumes, edit bridge torn down -``` - -Key constraints that follow from this model: - -- **Edit mode cannot be distributed**: `Fetch()` collapses all data to one process. - For very large meshes the `Fetch` + `DeepCopy` at session begin is the dominant cost. -- **Remap on every pick** *(mitigated)*: coordinate-based cell matching between source - and working dataset was rebuilt on every pick event. Now the structures are built once - at session begin in `set_edit_target_dataset` and reused per pick (O(1) lookups). The - fallback rebuild path remains for correctness when caches are absent. -- **Commit writes the whole mesh**: there is no delta/patch write — the full working - dataset is serialized even if only a handful of cell values changed. -- **Normal visualization (no edit) is unaffected**: filters run in ParaView's proxy - layer with multithread VTK SMP parallelism; only `Fetch()` calls (rare, for metadata) - touch the single-process ceiling. - ---- - -### 5b. Dataset naming conventions - -The edit-session code uses several "dataset" variables and cache fields with similar names. -This is the reference for what each one is, who owns it, how long it lives, and why it exists. - -| Name | Type | Owner | Lifetime | Role | -|---|---|---|---|---| -| `source` | ParaView proxy (`vtkSMProxy`) | `ParaViewBackend` (property) | permanent | Active pipeline node on the server side. Used for hardware picks (`SelectSurfaceCells`) and `Fetch` calls. Never mutated. | -| `source_dataset` | `vtkUnstructuredGrid` | local var (caller) | per call | Result of `servermanager.Fetch(source)`. A read-only local copy used to build indexes or remap IDs. Fetched on demand wherever needed; not stored — except see `_edit_source_dataset`. | -| `working_dataset` | `vtkUnstructuredGrid` | `EditSession` | session | `DeepCopy` of `source_dataset` made once at session begin. The **mutation target**: field assignments, expression results, and selection tracking all land here. | -| `_edit_target_dataset` | reference to `working_dataset` | `ParaViewBackend` | session | Registered via `set_edit_target_dataset()` so remap methods know where to translate picks onto. Same Python object as `working_dataset` — not a copy. | -| `_edit_source_dataset` | `vtkUnstructuredGrid` | `ParaViewBackend` | session | The one `source_dataset` kept alive past its call. Cached at session begin to avoid re-Fetching on every pick. Passed in from `export_active_dataset_for_editing` which already has it. | -| `_edit_source_point_indexes` | `list[dict]` | `ParaViewBackend` | session | Coordinate → point-ID index derived from `_edit_source_dataset`. Lets `_surface_keys_from_selected_dataset` match surface-pick coordinates back to source point IDs without rebuilding on every pick. | -| `_edit_target_cell_map` | `dict` | `ParaViewBackend` | session | Cell-geometry → cell-ID map derived from `working_dataset`. Lets `_remap_cell_ids_between_datasets` translate source cell IDs onto working-dataset cell IDs in O(1) per cell rather than O(n_cells) per pick. | -| `_edit_point_id_map` | `dict[int, int]` | `ParaViewBackend` | session | Source point-ID → working-dataset point-ID map. Lets `_remap_surface_keys_to_edit_target_dataset` translate surface picks from source space to working-dataset space without rebuilding on every pick. | -| `edit_dataset` | alias (local var) | `_pick_surface_keys_native` | per call | Convenience alias within that one method: equals `_edit_target_dataset` when a session is active, falls back to a fresh `source_dataset` fetch otherwise. Read-only. | -| `selected_dataset` | `vtkUnstructuredGrid` | pick methods (local var) | per pick | Result of `ExtractSelection` + `Fetch` after a hardware pick. A small subset of source/GeometryFilter cells that ParaView determined were hit. Deleted after each pick. | - -The four session-scoped `ParaViewBackend` fields (`_edit_source_dataset`, `_edit_source_point_indexes`, -`_edit_target_cell_map`, `_edit_point_id_map`) are built together by `set_edit_target_dataset()` at -session begin and cleared together by `clear_edit_target_dataset()` at session end. - -**Identity chain:** - -``` -source (proxy — server side) - │ servermanager.Fetch() ← pulls data across the proxy boundary - ▼ -source_dataset (vtkUG, read-only) ← also cached as _edit_source_dataset - │ DeepCopy() at session begin - ▼ -working_dataset (vtkUG, mutable) ← also referenced as _edit_target_dataset -``` - -The remap exists because picks go through `source` (proxy) and return IDs relative to -`source_dataset`, but mutations must land on `working_dataset`. Both datasets have the -same geometry but are independent Python objects with independent ID spaces, so -coordinate-based maps (`_edit_target_cell_map`, `_edit_point_id_map`) are needed to -translate IDs from one to the other. These maps are now built once at session begin -(see §5a) rather than on every pick. - ---- - -## 6. Selection / Picking Flow - -The full path from a user gesture to a list of selected cell (or surface) IDs. -Picking always runs inside an active edit session; the results feed back into -`edit_session` and update the gold overlay in the view. - -### 6a. Click pick — cell or point mode - -``` -UI click event - └── ctrl.pv_edit_click_selection(event) [paraview_controllers.py] - ├── normalize_edit_selection_ids(event) [paraview_runtime.py] - │ — extracts (x, y) screen coords from the raw Trame event payload, - │ preferring repicked coords over composite toggle IDs - └── _pick_edit_ids_at_coords(mode, x, y) - └── pv_backend.pick_visible_cell_ids(x, y, radius=1) - │ [paraview_backend.py:1764] - ├── _candidate_pick_positions(x, y) - │ — generates 2–4 candidate pixel coords to handle Y-axis - │ convention differences between Trame events and ParaView - │ display space (raw, inverted, normalized variants) - │ - ├── clear_edit_selection_overlay() - │ — removes the gold overlay proxy before picking so it - │ cannot interfere with ParaView's hardware pick query - │ - ├── for each candidate (px, py): ← tries until non-empty result - │ ├── _clear_selection_state(source) - │ ├── simple.SelectSurfaceCells(Rectangle=[px±r, py±r], View=view) - │ │ — fires ParaView's hardware selection on the rendered surface - │ └── _fetch_selected_original_cell_ids(source) - │ ├── simple.ExtractSelection(Input=source) - │ ├── servermanager.Fetch(extract) — transfer to client - │ ├── cell_data.GetArray("vtkOriginalCellIds") — happy path - │ └── _source_cell_ids_from_selected_dataset() — geometry fallback - │ — matches selected cells back to source by point coordinates - │ when the original-IDs array is absent (e.g. after a filter) - │ - ├── _clear_selection_state(source) + render() - │ — clears ParaView's purple selection highlight before returning - │ - └── _remap_cell_ids_to_edit_target_dataset(picked, source) - — maps source cell IDs onto working_dataset cell IDs; - uses _edit_source_dataset (no Fetch) and _edit_target_cell_map - (no rebuild) when caches are populated at session begin -``` - -### 6b. Box drag pick — cell or point mode - -``` -UI box drag event - └── ctrl.pv_edit_box_selection(event) [paraview_controllers.py] - └── _pick_edit_ids_in_rect(mode, x0, y0, x1, y1, behavior) - └── pv_backend.pick_visible_cell_ids_in_rect(x0, y0, x1, y1, behavior) - ├── clear_edit_selection_overlay() - ├── simple.SelectSurfaceCells(Rectangle=[x0,y0,x1,y1], View=view) - ├── _fetch_selected_original_cell_ids(source) (same as click path) - ├── _clear_selection_state(source) + render() - ├── _remap_cell_ids_to_edit_target_dataset() - └── if behavior == "inside": - _filter_cell_ids_inside_rect(source, picked, rect) - — projects each candidate cell's vertices to display space - and keeps only cells fully contained in the drag rectangle -``` - -### 6c. Surface key pick (boundary face mode) - -Used when the selected field is a surface/boundary field. The pick target is -a visible boundary face (a codimension-1 entity), not a volumetric cell. - -``` -pv_backend.pick_visible_surface_keys(x, y, radius=2) - └── _pick_surface_keys_in_rect(rect) [paraview_backend.py:1967] - │ - ├── [native path — 3D meshes only] - │ _pick_surface_keys_native(rect) - │ ├── _surface_selection_helper_for(source) - │ │ — creates (or reuses) a cached GeometryFilter proxy, which - │ │ extracts boundary PolyData and is a valid SelectSurfaceCells - │ │ target; kept hidden in the view, shown only during the pick - │ ├── hide original display, show surface helper temporarily - │ ├── simple.SelectSurfaceCells(rect, on temp_source) - │ ├── simple.ExtractSelection(Input=temp_source) + servermanager.Fetch() - │ ├── _surface_keys_from_selected_dataset(selected, source_dataset, - │ │ │ source_point_indexes=_edit_source_point_indexes) - │ │ ├── _iter_leaf_datasets() — flatten composite blocks - │ │ ├── coordinate index: uses pre-built source_point_indexes when provided - │ │ │ (avoids O(n_source_points) rebuild per pick); lazy-builds if absent - │ │ ├── for each selected cell: map XYZ corners → source point IDs via - │ │ │ coordinate lookup (neither PassThroughPointIds nor PassThroughCellIds - │ │ │ is used: in ParaView 6.1 both produce output-geometry indices through - │ │ │ the proxy/Fetch round-trip, not source topology IDs — see PR #31, issue #34) - │ │ └── _normalize_surface_keys_to_source_boundary() - │ │ — ParaView may triangulate quads; resolves partial triangle keys - │ │ back to the canonical quad key in the source boundary map - │ └── _remap_surface_keys_to_edit_target_dataset(keys, source_dataset) - │ — maps source point IDs onto working_dataset point IDs; - │ uses _edit_point_id_map cache when populated at session begin - │ - └── [fallback path — 2D meshes, or if native path returns None] - ├── servermanager.Fetch(source) - ├── _cached_boundary_elements(dataset) - │ — finds codimension-1 entities (faces for 3D, edges for 2D) that - │ appear exactly once: these are the visible boundary elements - ├── for each boundary element: - │ ├── _surface_element_is_visible() — depth-buffer check at centroid - │ ├── _project_points_to_display() — world → pixel coords via renderer - │ └── _polyline_intersects_rect() or all-inside check - └── _remap_surface_keys_to_edit_target_dataset(picked, source_dataset=dataset) - — maps source point IDs onto working_dataset point IDs; - uses _edit_point_id_map cache when populated at session begin -``` - -### 6d. Overlay update (after any successful pick) - -``` -_apply_selection_ids(picked_ids, mode, behavior) [paraview_controllers.py] - ├── edit_session.replace/add/subtract_selection(ids) - ├── sync_edit_session_state() — update count, field readiness - └── sync_paraview_edit_selection_overlay() - └── pv_backend.update_edit_selection_overlay(selected_dataset) - [paraview_backend.py:1717] - ├── clear_edit_selection_overlay() — delete stale TrivialProducer proxy - ├── simple.TrivialProducer("__edit_selection__") - │ — lightweight proxy that wraps a local vtkDataSet without a reader - ├── overlay.GetClientSideObject().SetOutput(selected_cells_dataset) - ├── simple.Show(overlay, view) - ├── style: gold fill, black edges, Pickable=0, LineWidth=4, PointSize=10 - │ RelativeCoincidentTopologyPolygonOffsetParameters=[-2,-2] — always on top - └── render() -``` - ---- - -## 7. Filter System - -Filters are defined in `paraview_filter_catalog.py` and wired through -`paraview_controllers.py` → `paraview_backend.py`. - -### Add-filter flow - -``` -UI "Filter" menu → ctrl.pv_add_filter(filter_key) [paraview_controllers.py:689] - └── pv_backend.add_filter(filter_key) [paraview_backend.py:~270] - ├── resolve spec from SUPPORTED_FILTERS or experimental catalog - ├── getattr(simple, spec["factory"])(Input=source) — create ParaView proxy - ├── simple.Show(filter_proxy, view) - ├── inherit parent visibility, representation, selected_array - ├── hide parent display - ├── append new node to pipeline_nodes - └── apply_coloring() + render() -``` - -All filters use the same `Input=source` pattern — no special per-filter setup in the controller. - -### Supported filters (all fully working, no extra setup required) - -| Key | Label | ParaView factory | -|---|---|---| -| `calculator` | Calculator | `Calculator` | -| `cell_centers` | Cell Centers | `CellCenters` | -| `clip` | Clip | `Clip` | -| `contour` | Contour | `Contour` | -| `coordinates` | Coordinates | `Coordinates` | -| `glyph` | Glyph | `Glyph` | -| `reflect` | Reflect | `Reflect` | -| `slice` | Slice | `Slice` | -| `stream_tracer` | Streamline | `StreamTracer` | -| `threshold` | Threshold | `Threshold` | -| `transform` | Transform | `Transform` | -| `tube` | Tube | `Tube` | -| `warp_by_scalar` | Warp by Scalar | `WarpByScalar` | -| `warp_by_vector` | Warp by Vector | `WarpByVector` | - -### Experimental filters - -When `--show-experimental-filters` is passed, `ParaViewFilterCatalog` scans -`paraview.simple` at startup for any callable matching a discovery keyword list -(clip, contour, glyph, threshold, warp, etc.) and not in an exclusion list -(readers, extractors, ghost, selection, source…). Their keys are prefixed -`factory:{FactoryName}`. They appear in a separate "Experimental" section of the -filter menu and go through the same `add_filter` code path as curated filters. - ---- - -## 8. Color / Color-Bar Logic - -### Color-by change - -Triggered when the user picks an array in the "Color by" selector. - -``` -state.selected_array change - └── on_array_change() [state_handlers.py:39] - ├── pv_backend.apply_coloring(value) - │ ├── ColorBy(display, (assoc, name)) — binds array to display - │ ├── _ensure_display_lookup_table() — creates LUT if missing - │ ├── _restore_scalar_bar_visibility() — reapplies user visibility flag - │ └── render() - ├── update_color_state() — targeted 6-key color flush - └── call_view_update() -``` - -### Representation change - -Triggered when the user picks a display mode (Surface, Wireframe, Points, …). - -``` -state.representation change - └── on_representation_change() [state_handlers.py:46] - ├── pv_backend.apply_representation(value) - │ ├── display.SetRepresentationType() — switches ParaView display mode - │ ├── _apply_representation_to_extract_displays() - │ └── render() - ├── update_ui_state() — full flush: display_properties vary by mode - └── call_view_update() -``` - -### Scalar bar visibility — two-layer preservation - -Scalar bar visibility is tracked in two places that must stay in sync: - -- `pv_backend._scalar_bar_visible` — Python flag, the authoritative user intent -- `state.color_bar_visible` — Trame state, what the UI toggle reflects - -**Why preservation is needed at all:** ParaView's `RescaleTransferFunctionToDataRange` -internally calls `UpdateScalarBars` as a side effect, which hides the scalar bar. -This is the confirmed case from the CODEX_LOG. `ApplyPreset` and -`RescaleTransferFunction` are treated the same way defensively. - -**Layer 1 — backend** (`_restore_scalar_bar_visibility` [paraview_backend.py:961]): -Called at the end of `apply_color_map_preset`, `apply_color_range`, and -`rescale_color_range_to_data`. Immediately re-asserts `_scalar_bar_visible` -back onto the display via `SetScalarBarVisibility` after the ParaView call. -By the time the backend method returns, ParaView-side visibility is already correct. - -**Layer 2 — controller** (`_refresh_color_state` [paraview_controllers.py]): -Calls `update_color_state()` (a targeted 6-key Trame flush) then `render_and_push()`. -`update_color_state()` reads `_scalar_bar_visible` from the backend — since layer 1 -already corrected it before returning, this always writes the right value. -The old save/restore of `state.color_bar_visible` was removed because it was always -a no-op once layer 1 was present for all LUT-mutating operations. - -``` -update_color_state() — reads _scalar_bar_visible → writes 6 color state keys -render_and_push() -``` - -### Color-bar control actions - -``` -UI action → ctrl.pv_*() [paraview_controllers.py] - -pv_apply_color_map_preset(preset) - ├── pv_backend.apply_color_map_preset(preset) - │ ├── lut.ApplyPreset(candidate, True) — modifies LUT RGB points - │ ├── _restore_scalar_bar_visibility() — layer 1 - │ └── render() - └── _refresh_color_state() — layer 2: targeted color flush - -pv_apply_color_range() - ├── pv_backend.apply_color_range(min, max) - │ ├── lut.RescaleTransferFunction(min, max) - │ ├── _restore_scalar_bar_visibility() - │ └── render() - └── _refresh_color_state() - -pv_rescale_color_range_to_data() - ├── pv_backend.rescale_color_range_to_data() - │ ├── display.RescaleTransferFunctionToDataRange() ← confirmed: hides scalar bar - │ ├── _restore_scalar_bar_visibility() — layer 1 fix - │ └── render() - └── _refresh_color_state() - -pv_rescale_color_range_over_time() - ├── pv_backend.rescale_color_range_over_time() - │ └── (same pattern as above) - └── _refresh_color_state() - -pv_set_categorical_coloring(enabled) - ├── pv_backend.set_categorical_coloring() - │ ├── lut.InterpretValuesAsCategories / IndexedLookup = ... - │ ├── _restore_scalar_bar_visibility() — layer 1 (added to match other ops) - │ └── render() - └── _refresh_color_state() - -pv_set_scalar_bar_visible(visible) - ├── pv_backend.set_scalar_bar_visible() - │ ├── display.SetScalarBarVisibility(view, visible) - │ └── _scalar_bar_visible = visible — updates the flag directly - └── render_and_push() — no color flush needed; state written before call - -pv_set_orientation_axes_visible(visible) - ├── pv_backend.set_orientation_axes_visible() — view.OrientationAxesVisibility - └── render_and_push() — no color flush needed; state written before call -``` - -### `get_color_control_state` — reading color state from the backend - -Called inside both `get_ui_state()` (full flush) and `update_color_state()` (targeted flush). - -``` -pv_backend.get_color_control_state() [paraview_backend.py:773] - ├── _get_selected_array() — check if a non-solid array is active - ├── _active_lookup_table() [paraview_backend.py:1003] - │ └── GetColorTransferFunction(array_name) on active display - ├── _lookup_table_range(lut) — lut.RGBPoints[0] and lut.RGBPoints[-4] - ├── _lookup_table_categorical(lut) — lut.Annotations != [] - └── returns { - color_controls_enabled, — False when solid color or no display - color_range_min/max, — empty strings when disabled - color_bar_visible, — _scalar_bar_visible instance flag - orientation_axes_visible, — view.OrientationAxesVisibility - categorical_coloring — bool - } -``` diff --git a/docs/logic_flows/color.md b/docs/logic_flows/color.md new file mode 100644 index 0000000..bd96f22 --- /dev/null +++ b/docs/logic_flows/color.md @@ -0,0 +1,131 @@ +# Color / Color-Bar Logic + +## Color-by change + +Triggered when the user picks an array in the "Color by" selector. + +``` +state.selected_array change + └── on_array_change() [state_handlers.py] + ├── pv_backend.apply_coloring(value) + │ ├── ColorBy(display, (assoc, name)) — binds array to display + │ ├── _ensure_display_lookup_table() — creates LUT if missing + │ ├── _restore_scalar_bar_visibility() — reapplies user visibility flag + │ └── render() + ├── update_color_state() — targeted 6-key color flush + └── call_view_update() +``` + +## Representation change + +Triggered when the user picks a display mode (Surface, Wireframe, Points, …). + +``` +state.representation change + └── on_representation_change() [state_handlers.py] + ├── pv_backend.apply_representation(value) + │ ├── display.SetRepresentationType() — switches ParaView display mode + │ ├── _apply_representation_to_extract_displays() + │ └── render() + ├── update_ui_state() — full flush: display_properties vary by mode + └── call_view_update() +``` + +## Scalar bar visibility — two-layer preservation + +Scalar bar visibility is tracked in two places that must stay in sync: + +- `pv_backend._scalar_bar_visible` — Python flag, the authoritative user intent +- `state.color_bar_visible` — Trame state, what the UI toggle reflects + +**Why preservation is needed at all:** ParaView's `RescaleTransferFunctionToDataRange` +internally calls `UpdateScalarBars` as a side effect, which hides the scalar bar. +This is the confirmed case from the CODEX_LOG. `ApplyPreset` and +`RescaleTransferFunction` are treated the same way defensively. + +**Layer 1 — backend** (`_restore_scalar_bar_visibility` [paraview_backend.py]): +Called at the end of `apply_color_map_preset`, `apply_color_range`, and +`rescale_color_range_to_data`. Immediately re-asserts `_scalar_bar_visible` +back onto the display via `SetScalarBarVisibility` after the ParaView call. +By the time the backend method returns, ParaView-side visibility is already correct. + +**Layer 2 — controller** (`_refresh_color_state` [paraview_controllers.py]): +Calls `update_color_state()` (a targeted 6-key Trame flush) then `render_and_push()`. +`update_color_state()` reads `_scalar_bar_visible` from the backend — since layer 1 +already corrected it before returning, this always writes the right value. +The old save/restore of `state.color_bar_visible` was removed because it was always +a no-op once layer 1 was present for all LUT-mutating operations. + +``` +update_color_state() — reads _scalar_bar_visible → writes 6 color state keys +render_and_push() +``` + +## Color-bar control actions + +``` +UI action → ctrl.pv_*() [paraview_controllers.py] + +pv_apply_color_map_preset(preset) + ├── pv_backend.apply_color_map_preset(preset) + │ ├── lut.ApplyPreset(candidate, True) — modifies LUT RGB points + │ ├── _restore_scalar_bar_visibility() — layer 1 + │ └── render() + └── _refresh_color_state() — layer 2: targeted color flush + +pv_apply_color_range() + ├── pv_backend.apply_color_range(min, max) + │ ├── lut.RescaleTransferFunction(min, max) + │ ├── _restore_scalar_bar_visibility() + │ └── render() + └── _refresh_color_state() + +pv_rescale_color_range_to_data() + ├── pv_backend.rescale_color_range_to_data() + │ ├── display.RescaleTransferFunctionToDataRange() ← confirmed: hides scalar bar + │ ├── _restore_scalar_bar_visibility() — layer 1 fix + │ └── render() + └── _refresh_color_state() + +pv_rescale_color_range_over_time() + ├── pv_backend.rescale_color_range_over_time() + │ └── (same pattern as above) + └── _refresh_color_state() + +pv_set_categorical_coloring(enabled) + ├── pv_backend.set_categorical_coloring() + │ ├── lut.InterpretValuesAsCategories / IndexedLookup = ... + │ ├── _restore_scalar_bar_visibility() — layer 1 (added to match other ops) + │ └── render() + └── _refresh_color_state() + +pv_set_scalar_bar_visible(visible) + ├── pv_backend.set_scalar_bar_visible() + │ ├── display.SetScalarBarVisibility(view, visible) + │ └── _scalar_bar_visible = visible — updates the flag directly + └── render_and_push() — no color flush needed; state written before call + +pv_set_orientation_axes_visible(visible) + ├── pv_backend.set_orientation_axes_visible() — view.OrientationAxesVisibility + └── render_and_push() — no color flush needed; state written before call +``` + +## `get_color_control_state` — reading color state from the backend + +Called inside both `get_ui_state()` (full flush) and `update_color_state()` (targeted flush). + +``` +pv_backend.get_color_control_state() [paraview_backend.py] + ├── _get_selected_array() — check if a non-solid array is active + ├── _active_lookup_table() [paraview_backend.py] + │ └── GetColorTransferFunction(array_name) on active display + ├── _lookup_table_range(lut) — lut.RGBPoints[0] and lut.RGBPoints[-4] + ├── _lookup_table_categorical(lut) — lut.Annotations != [] + └── returns { + color_controls_enabled, — False when solid color or no display + color_range_min/max, — empty strings when disabled + color_bar_visible, — _scalar_bar_visible instance flag + orientation_axes_visible, — view.OrientationAxesVisibility + categorical_coloring — bool + } +``` diff --git a/docs/logic_flows/edit_session.md b/docs/logic_flows/edit_session.md new file mode 100644 index 0000000..1679c87 --- /dev/null +++ b/docs/logic_flows/edit_session.md @@ -0,0 +1,312 @@ +# Edit Session Flow + +## 5. Edit Session Lifecycle + +### 5a. High-level call flow + +``` +[Begin] +ctrl.pv_begin_edit_session() + ├── pv_backend.export_active_dataset_for_editing() + │ ├── creates a real edit pipeline node (kind="edit", label="✏ Editing: {original}") + │ │ root reader → OpenDataFile(same backing file) — no temp file write + │ │ filter node → write temp VTU, then OpenDataFile — avoids MPI-collapsing Fetch + │ ├── hides all existing pipeline nodes (Visibility=0) + │ └── Fetch(edit_source) → one local vtkUnstructuredGrid passed back as "dataset" + ├── edit_session.begin(node_id, label, filename, dataset) + │ — DeepCopy into working_dataset, reset all session state, + │ pre-compute CellCenters array for expression evaluation + ├── pv_backend.set_edit_target_dataset(working_dataset) + │ — stores reference so surface picks use working_dataset as source_dataset + │ without a redundant Fetch; also builds two session-scoped geometry caches: + │ _edit_source_point_indexes — coordinate index over all source points (#34 B5 fix) + │ _edit_boundary_elements — boundary codim-1 face map over all cells (B5b fix) + ├── update_paraview_ui_state() — sync pipeline panel to show the edit node + ├── state.pick_mode = True, mainViewMode = "remote" + │ — enables hardware picking; forces server-side rendering + └── sync_edit_session_state() + render_and_push() + +[Selection — click or box drag] +ctrl.pv_edit_click_selection(event) / ctrl.pv_edit_box_selection(event) + ├── normalize_edit_selection_ids(event) — extract screen coords or composite IDs + ├── _pick_edit_ids_at_coords / _pick_edit_ids_in_rect — dispatch to click or rect pick + │ └── pv_backend.pick_visible_cell_ids(...) — ParaView hardware pick + └── _apply_selection_ids(picked_ids, ...) + ├── edit_session.replace/add/subtract/flip_selection(ids) + ├── sync_edit_session_state() — update selection count and field readiness + ├── sync_paraview_edit_selection_overlay() — highlight selected cells in view + └── render_and_push() + +[Assign value] +ctrl.pv_apply_edit_field() [paraview_controllers.py] + └── pv_apply_edit_field() → _apply_edit_field() + ├── _parse_field_choice() — "cell:FieldName" → (association, field_name) + ├── _apply_geometry_options_for_association(association) + │ — sets edit_session.geometry_mode; forces "point" for point fields + ├── edit_session.assign_to_selected(field_name, association, expression) + │ — see §5d for full detail + └── sync_edit_session_state() — refresh dirty flag and field state in UI + +[Commit] +ctrl.pv_commit_edit_session() + └── opens save dialog (state.save_dialog = True, save_dialog_action = "commit") + +User confirms in dialog → _commit_edit_session(filename, overwrite) + ├── save_paraview_output(filename, overwrite) [file_operations.py] + │ ├── resolve_paraview_output_path() — path-safe, confines to data_directory + │ ├── edit_session.save(output_path) — see §5e for full detail + │ └── refresh_available_files() — update file browser + ├── edit_session.clear() + ├── pv_backend.clear_edit_target_dataset() + │ — deletes edit node proxy, restores pre-edit node visibility, + │ clears _edit_source_point_indexes, _edit_boundary_elements, _edit_target_dataset + ├── pv_backend.load_file(output_path) — add saved file as new pipeline node + ├── update_paraview_ui_state() — full flush: new node visible in pipeline + └── render_and_push() + +[Discard] +ctrl.pv_discard_edit_session() [paraview_controllers.py] + ├── edit_session.clear() — reset all session state, release working_dataset + ├── pv_backend.clear_edit_target_dataset() + │ — deletes edit node, restores pre-edit node, clears geometry caches + ├── update_paraview_ui_state() — full state flush: back to pre-edit pipeline + ├── sync_edit_session_state() — edit_session_active = False, etc. + ├── sync_paraview_edit_selection_overlay() — remove gold overlay + ├── state resets (save_filename, selection_count, inspector_tab, edit_selection_event) + └── call_view_update(reset_camera=True) + render_and_push() +``` + +--- + +### 5b. Data movement, process model, and bottlenecks + +The diagram below tracks what happens to the mesh data across the same phases as §5a. +Read §5a to trace function calls; read this to understand data ownership, copies, and +where the performance costs land. + +The app is **single-process** (one Python process hosts Trame, ParaView pipeline, and +edit logic). VTK C++ filter work uses multi-thread internally via `vtkMultiThreader`; +Python orchestration is single-threaded. There is no MPI: all data lives in one process. +If a pvserver were connected via `simple.Connect()`, `servermanager.Fetch()` would +still collapse all distributed ranks into this one process at the points marked below. + +``` +─── SESSION BEGIN ────────────────────────────────────────────────────────────────── + [paraview_controllers.py pv_begin_edit_session] + [paraview_backend.py export_active_dataset_for_editing] + [edit_session.py EditSession.begin] + + export_active_dataset_for_editing(): + ├── All existing pipeline nodes hidden (Visibility=0) + ├── Edit source created (kind="edit" pipeline node): + │ root reader → OpenDataFile(same backing file) — zero extra I/O + │ filter node → XMLUnstructuredGridWriter → temp .vtu → OpenDataFile + │ ⚠ BOTTLENECK (filter case): entire filter output written to disk + └── servermanager.Fetch(edit_source) + ⚠ BOTTLENECK: entire mesh transferred into one Python vtkUnstructuredGrid. + In MPI mode all ranks would be merged here. + │ + ▼ + vtkUnstructuredGrid (local Python, discarded after begin) + │ + ▼ DeepCopy() + │ ⚠ full mesh copy — all points, all cells, all point/cell data arrays + ▼ + edit_session.working_dataset independent mutable copy; field assignments, + selections, and expressions operate only here + + set_edit_target_dataset(working_dataset): + ├── stores reference to working_dataset so surface picks (_pick_surface_keys_native) + │ can use it as source_dataset for coordinate mapping without a second Fetch. + ├── _edit_source_point_indexes = _point_coordinate_indexes(working_dataset) + │ — O(n_points) scan built once; avoids per-pick rebuild (B5 fix) + └── _edit_boundary_elements = _boundary_codim_elements(working_dataset) + — O(n_cells × faces) scan built once; avoids per-pick rebuild (B5b fix) + + _ensure_cell_centers_array() [edit_session.py]: + adds CellCenters cell-data array (XYZ per cell centroid) so + vtkArrayCalculator can reference spatial coordinates in expressions. + +─── DURING SESSION ───────────────────────────────────────────────────────────────── + + Edit node IS the active pipeline source. + Picks (SelectSurfaceCells) run against the edit source. + The edit source reads the same file as the original node (or the temp VTU written + from filter output), so pick cell IDs already index working_dataset directly — + no coordinate-based remapping is needed. + + ParaView side (edit_source proxy) Edit session side (working_dataset) + ──────────────────────────────── ────────────────────────────────────── + renders in viewport holds all mutations (assigned field values) + target of SelectSurfaceCells() tracks selected_cell_ids / surface_keys / + selected_point_ids + + cell IDs from hardware pick + │ (same IDs, same geometry) + ▼ + working_dataset cell IDs no remapping step + + CellCenters array (cell data) GetPoint() calls (raw mesh geometry) + └─ vtkArrayCalculator reads it └─ _ensure_surface_element_vectors() + when evaluating user expressions computes unit normals/tangents for + (e.g. "CellCenters[0] > 2.5") dihedral angle filter during grow. + Stripped from output on save. Never stored as an array. + +─── COMMIT ───────────────────────────────────────────────────────────────────────── + [paraview_controllers.py _commit_edit_session] + + edit_session.working_dataset + │ + ├── materialize_surface_selection() — insert missing boundary face cells (surface mode) + ├── _remove_internal_edit_arrays() — strips CellCenters before write + ▼ + vtkXMLUnstructuredGridWriter.Write() ⚠ full mesh write to disk + │ + ▼ + pv_backend.clear_edit_target_dataset() deletes edit node, restores pre-edit node, + │ clears geometry caches + ▼ + pv_backend.load_file(output_path) ⚠ full mesh re-read into ParaView pipeline + │ ParaView owns it again as a new node. + ▼ + new pipeline node — normal visualization resumes, edit bridge torn down +``` + +Key constraints that follow from this model: + +- **Edit mode cannot be distributed**: `Fetch()` collapses all data to one process. + For very large meshes the `Fetch` + `DeepCopy` at session begin is the dominant cost. +- **No per-pick remapping**: because the edit source reads the same backing data as the + original node, pick cell IDs directly index `working_dataset` — no coordinate maps are + built or maintained during the session. +- **Commit writes the whole mesh**: there is no delta/patch write — the full working + dataset is serialized even if only a handful of cell values changed. +- **Normal visualization (no edit) is unaffected**: filters run in ParaView's proxy + layer with multithread VTK SMP parallelism; only `Fetch()` calls (rare, for metadata) + touch the single-process ceiling. + +--- + +### 5c. Dataset naming conventions + +The edit-session code uses several "dataset" variables with similar names. +This is the reference for what each one is, who owns it, how long it lives, and why it exists. + +| Name | Type | Owner | Lifetime | Role | +|---|---|---|---|---| +| `source` | ParaView proxy (`vtkSMProxy`) | `ParaViewBackend` (property) | permanent | During an edit session this is the **edit node** proxy (kind="edit"). Used for hardware picks (`SelectSurfaceCells`) and `Fetch` calls. Never mutated. | +| `source_dataset` | `vtkUnstructuredGrid` | local var (caller) | per call | Result of `servermanager.Fetch(source)`. A read-only local copy used for coordinate mapping in surface picks. Fetched on demand; not stored between calls. | +| `working_dataset` | `vtkUnstructuredGrid` | `EditSession` | session | `DeepCopy` of the dataset fetched from the edit source at session begin. The **mutation target**: field assignments, expression results, and selection tracking all land here. | +| `_edit_target_dataset` | reference to `working_dataset` | `ParaViewBackend` | session | Registered via `set_edit_target_dataset()`. Used by `_pick_surface_keys_native` as `source_dataset` for coordinate mapping, avoiding a redundant `Fetch(source)`. Same Python object as `working_dataset` — not a copy. | +| `_edit_source_point_indexes` | `list[dict]` | `ParaViewBackend` | session | Pre-built at session begin by `set_edit_target_dataset()`. Coordinate index over all source points; passed to `_surface_keys_from_selected_dataset` to avoid O(n_points) rebuild per pick (B5 fix). | +| `_edit_boundary_elements` | `dict` | `ParaViewBackend` | session | Pre-built at session begin by `set_edit_target_dataset()`. Boundary codim-1 face map over all cells; passed to `_normalize_surface_keys_to_source_boundary` to avoid O(n_cells×faces) rebuild per pick (B5b fix). | +| `_edit_node_id` | `str` | `ParaViewBackend` | session | ID of the edit pipeline node. Used to locate and delete the node on session end. | +| `_edit_pre_node_id` | `str` | `ParaViewBackend` | session | ID of the original node that was active before the session began. Restored to visible on discard/commit. | +| `_edit_temp_file` | `str \| None` | `ParaViewBackend` | session | Path to the temp `.vtu` written for filter-node edit sources. Deleted on session end. `None` for root-reader sessions. | +| `edit_dataset` | alias (local var) | `_pick_surface_keys_native` | per call | Convenience alias within that one method: equals `_edit_target_dataset` when a session is active, falls back to a fresh `Fetch(source)` otherwise. Read-only. | +| `selected_dataset` | `vtkUnstructuredGrid` | pick methods (local var) | per pick | Result of `ExtractSelection` + `Fetch` after a hardware pick. A small subset of source/GeometryFilter cells that ParaView determined were hit. Deleted after each pick. | + +**Identity chain:** + +``` +edit_source (proxy — reads same backing file as original node) + │ servermanager.Fetch() ← one transfer at session begin + ▼ +fetched_dataset (vtkUG, read-only, local temp) + │ DeepCopy() in edit_session.begin() + ▼ +working_dataset (vtkUG, mutable) ← also referenced as _edit_target_dataset +``` + +Because `edit_source` reads the same backing data as the original node, pick cell IDs +returned by `SelectSurfaceCells` already index `working_dataset` directly — no +coordinate-based remapping is needed. + +--- + +### 5d. Assign value — full detail + +``` +edit_session.assign_to_selected(field_name, association, expression) + [edit_session.py] + ├── guards: session active, field exists, is scalar + ├── _selected_ids_for_association(association) + │ ├── [point mode] sorted(selected_point_ids) + │ ├── [surface mode] + │ │ ├── materialize_surface_selection() — see §5e + │ │ └── _selected_surface_cell_ids() + │ │ — scans ALL codim-1 cells in working_dataset (pre-existing + newly inserted), + │ │ builds key→cell_id map, returns IDs matching selected_surface_keys. + │ │ Pre-existing boundary cells are found here too, so skipping them in + │ │ materialize_surface_selection never causes them to miss the assignment. + │ └── [volume mode] sorted(selected_cell_ids) + ├── _evaluate_expression(association, expression) + │ ├── [cell] _ensure_cell_centers_array() — adds CellCenters to cell data if absent + │ ├── vtkArrayCalculator(working_dataset) + │ │ — registers all scalar/vector arrays by name so they can appear in expression + │ │ — evaluates expression; result stored as temporary "__edit_result__" array + │ │ — supports spatial coordinates via CellCenters (cell) or coords (point) + │ └── reads result array → returns list[float], one value per cell or point + └── for each selected_id: + existing.SetComponent(idx, 0, computed[idx]) + — writes scalar value directly into working_dataset's field array + target_data.SetActiveScalars(field_name) + working_dataset.Modified() + self.dirty = True +``` + +--- + +### 5e. Surface materialization — full detail + +`materialize_surface_selection()` makes selected boundary faces *physical* — writes them +as real cells in `working_dataset` so they can carry field data and be serialized. + +Called from `_selected_ids_for_association` before resolving cell IDs (surface mode), +and from `EditSession.save()` before writing to disk. + +``` +materialize_surface_selection() [edit_session.py] + │ + ├── guards: active, dataset not None, geometry_mode == "surface", + │ selected_surface_keys not empty, top_dim >= 2 + │ + ├── _remove_internal_edit_arrays() + │ — strips CellCenters and other scratch arrays before modifying dataset structure + │ + ├── _surface_boundary_map_for_top_cells() (cached) + │ — iterates all top-dimensional cells (e.g. tetras), enumerates every face/edge, + │ keeps only faces appearing exactly once across all cells (true boundary). + │ — returns {sorted_point_ids_tuple: (cell_type, ordered_point_ids, owner_cell_id)} + │ + ├── _existing_codim_keys(top_dim - 1) (cached) + │ — scans working_dataset for cells already at codim-1 dimension (e.g. triangles + │ for a tet mesh), builds their key set to avoid double-insertion. + │ + ├── missing_keys = [k for k in sorted(selected_surface_keys) + │ if k in boundary_map and k not in existing] + │ — only faces that are: selected AND confirmed boundary AND not yet physical cells. + │ Sorted for deterministic insertion order; idempotent on repeated calls. + │ → returns 0 early if no faces need inserting + │ + ├── for each missing key: + │ ├── retrieve (cell_type, point_ids, owner_cell_id) from boundary_map + │ ├── dataset.InsertNextCell(cell_type, id_list) + │ └── record new_cell_id → owner_cell_id in owner_cell_ids map + │ + ├── _extend_cell_data_for_new_cells(dataset, old_cell_count, owner_cell_ids) + │ — for every cell data array, appends a tuple copied from the owner volume cell's row, + │ keeping array lengths consistent with the new cell count and giving the new face + │ the same field values as its parent + │ + ├── dataset.Modified() + ├── self.dirty = True + ├── _invalidate_geometry_caches() — dataset shape changed; boundary/adjacency caches stale + └── return len(missing_keys) +``` + +**Why pre-existing faces are handled correctly:** +`materialize_surface_selection` skips faces already in `existing` (no duplicate insertion), +but `_selected_surface_cell_ids` (called right after) scans *all* codim-1 cells and +returns IDs for every selected key regardless of when the cell was inserted. +So a pre-existing face still receives the field assignment — `existing` only guards structural integrity. diff --git a/docs/logic_flows/filters.md b/docs/logic_flows/filters.md new file mode 100644 index 0000000..1a92dc2 --- /dev/null +++ b/docs/logic_flows/filters.md @@ -0,0 +1,48 @@ +# Filter System + +Filters are defined in `paraview_filter_catalog.py` and wired through +`paraview_controllers.py` → `paraview_backend.py`. + +## Add-filter flow + +``` +UI "Filter" menu → ctrl.pv_add_filter(filter_key) [paraview_controllers.py] + └── pv_backend.add_filter(filter_key) [paraview_backend.py] + ├── resolve spec from SUPPORTED_FILTERS or experimental catalog + ├── getattr(simple, spec["factory"])(Input=source) — create ParaView proxy + ├── simple.Show(filter_proxy, view) + ├── inherit parent visibility, representation, selected_array + ├── hide parent display + ├── append new node to pipeline_nodes + └── apply_coloring() + render() +``` + +All filters use the same `Input=source` pattern — no special per-filter setup in the controller. + +## Supported filters (all fully working, no extra setup required) + +| Key | Label | ParaView factory | +|---|---|---| +| `calculator` | Calculator | `Calculator` | +| `cell_centers` | Cell Centers | `CellCenters` | +| `clip` | Clip | `Clip` | +| `contour` | Contour | `Contour` | +| `coordinates` | Coordinates | `Coordinates` | +| `glyph` | Glyph | `Glyph` | +| `reflect` | Reflect | `Reflect` | +| `slice` | Slice | `Slice` | +| `stream_tracer` | Streamline | `StreamTracer` | +| `threshold` | Threshold | `Threshold` | +| `transform` | Transform | `Transform` | +| `tube` | Tube | `Tube` | +| `warp_by_scalar` | Warp by Scalar | `WarpByScalar` | +| `warp_by_vector` | Warp by Vector | `WarpByVector` | + +## Experimental filters + +When `--show-experimental-filters` is passed, `ParaViewFilterCatalog` scans +`paraview.simple` at startup for any callable matching a discovery keyword list +(clip, contour, glyph, threshold, warp, etc.) and not in an exclusion list +(readers, extractors, ghost, selection, source…). Their keys are prefixed +`factory:{FactoryName}`. They appear in a separate "Experimental" section of the +filter menu and go through the same `add_filter` code path as curated filters. diff --git a/docs/logic_flows/logic_flows.md b/docs/logic_flows/logic_flows.md new file mode 100644 index 0000000..c5a07a1 --- /dev/null +++ b/docs/logic_flows/logic_flows.md @@ -0,0 +1,13 @@ +# Logic Flows + +Execution paths and data flows in the Trame/ParaView visualizer. +One file per domain — open the relevant file to trace a specific flow. + +| File | Contents | +|---|---| +| [startup_load.md](startup_load.md) | App startup, file load, UI state sync, controller→backend pattern | +| [edit_session.md](edit_session.md) | Edit session lifecycle: begin, selection, assign, materialize, commit, discard | +| [picking.md](picking.md) | Selection / picking flow: click, box drag, surface key pick, overlay update | +| [filters.md](filters.md) | Filter system: add-filter flow, supported filter catalogue | +| [color.md](color.md) | Color-by change, representation change, scalar bar preservation, color-bar controls | +| [rendering.md](rendering.md) | Rendering pipeline, deployment modes (X11/EGL/OSMesa), scene graph overhead | diff --git a/docs/logic_flows/picking.md b/docs/logic_flows/picking.md new file mode 100644 index 0000000..97ed490 --- /dev/null +++ b/docs/logic_flows/picking.md @@ -0,0 +1,131 @@ +# Selection / Picking Flow + +The full path from a user gesture to a list of selected cell (or surface) IDs. +Picking always runs inside an active edit session; the results feed back into +`edit_session` and update the gold overlay in the view. + +## 6a. Click pick — cell or point mode + +``` +UI click event + └── ctrl.pv_edit_click_selection(event) [paraview_controllers.py] + ├── normalize_edit_selection_ids(event) [paraview_runtime.py] + │ — extracts (x, y) screen coords from the raw Trame event payload, + │ preferring repicked coords over composite toggle IDs + └── _pick_edit_ids_at_coords(mode, x, y) + └── pv_backend.pick_visible_cell_ids(x, y, radius=1) + │ [paraview_backend.py] + ├── _candidate_pick_positions(x, y) + │ — generates 2–4 candidate pixel coords to handle Y-axis + │ convention differences between Trame events and ParaView + │ display space (raw, inverted, normalized variants) + │ + ├── clear_edit_selection_overlay() + │ — removes the gold overlay proxy before picking so it + │ cannot interfere with ParaView's hardware pick query + │ + ├── for each candidate (px, py): ← tries until non-empty result + │ ├── _clear_selection_state(source) + │ ├── simple.SelectSurfaceCells(Rectangle=[px±r, py±r], View=view) + │ │ — fires ParaView's hardware selection on the rendered surface + │ └── _fetch_selected_original_cell_ids(source) + │ ├── simple.ExtractSelection(Input=source) + │ ├── servermanager.Fetch(extract) — transfer to client + │ ├── cell_data.GetArray("vtkOriginalCellIds") — happy path + │ └── _source_cell_ids_from_selected_dataset() — geometry fallback + │ — matches selected cells back to source by point coordinates + │ when the original-IDs array is absent (e.g. after a filter) + │ + └── _clear_selection_state(source) + render() + — clears ParaView's purple selection highlight before returning + — returned IDs already index working_dataset (edit source reads + the same backing file; no remapping needed) +``` + +## 6b. Box drag pick — cell or point mode + +``` +UI box drag event + └── ctrl.pv_edit_box_selection(event) [paraview_controllers.py] + └── _pick_edit_ids_in_rect(mode, x0, y0, x1, y1, behavior) + └── pv_backend.pick_visible_cell_ids_in_rect(x0, y0, x1, y1, behavior) + ├── clear_edit_selection_overlay() + ├── simple.SelectSurfaceCells(Rectangle=[x0,y0,x1,y1], View=view) + ├── _fetch_selected_original_cell_ids(source) (same as click path) + ├── _clear_selection_state(source) + render() + │ — IDs already index working_dataset; no remapping step + └── if behavior == "inside": + _filter_cell_ids_inside_rect(source, picked, rect) + — projects each candidate cell's vertices to display space + and keeps only cells fully contained in the drag rectangle +``` + +## 6c. Surface key pick (boundary face mode) + +Used when the selected field is a surface/boundary field. The pick target is +a visible boundary face (a codimension-1 entity), not a volumetric cell. + +``` +pv_backend.pick_visible_surface_keys(x, y, radius=2) + └── _pick_surface_keys_in_rect(rect) [paraview_backend.py] + │ + ├── [native path — 3D meshes only] + │ _pick_surface_keys_native(rect) + │ ├── _surface_selection_helper_for(source) + │ │ — creates (or reuses) a cached GeometryFilter proxy, which + │ │ extracts boundary PolyData and is a valid SelectSurfaceCells + │ │ target; kept hidden in the view, shown only during the pick + │ ├── hide original display, show surface helper temporarily + │ ├── simple.SelectSurfaceCells(rect, on temp_source) + │ ├── simple.ExtractSelection(Input=temp_source) + servermanager.Fetch() + │ ├── _surface_keys_from_selected_dataset( + │ │ selected, + │ │ source_dataset=_edit_target_dataset, — working_dataset reference + │ │ source_point_indexes=_edit_source_point_indexes, — pre-built (#34 B5 fix) + │ │ prebuilt_boundary=_edit_boundary_elements — pre-built (#34 B5b fix) + │ │ ) + │ │ ├── _iter_leaf_datasets() — flatten composite blocks + │ │ ├── coordinate lookup via source_point_indexes + │ │ │ — built once at session begin; O(1) per point lookup during pick. + │ │ │ Neither PassThroughPointIds nor PassThroughCellIds is used: + │ │ │ in ParaView 6.1 both produce output-geometry indices through the + │ │ │ proxy/Fetch round-trip — see PR #31, issue #34. + │ │ └── _normalize_surface_keys_to_source_boundary( + │ │ keys, prebuilt_boundary=_edit_boundary_elements + │ │ ) + │ │ — ParaView may triangulate quads; resolves partial triangle keys + │ │ back to the canonical quad key in the source boundary map. + │ │ Uses pre-built boundary map; no per-pick rebuild. + │ └── returned keys already index working_dataset — no remapping step + │ (edit source and working_dataset share the same backing geometry) + │ + └── [fallback path — 2D meshes, or if native path returns None] + ├── servermanager.Fetch(source) + ├── _cached_boundary_elements(dataset) + │ — finds codimension-1 entities (faces for 3D, edges for 2D) that + │ appear exactly once: these are the visible boundary elements + ├── for each boundary element: + │ ├── _surface_element_is_visible() — depth-buffer check at centroid + │ ├── _project_points_to_display() — world → pixel coords via renderer + │ └── _polyline_intersects_rect() or all-inside check + └── returned keys already index working_dataset — no remapping step +``` + +## 6d. Overlay update (after any successful pick) + +``` +_apply_selection_ids(picked_ids, mode, behavior) [paraview_controllers.py] + ├── edit_session.replace/add/subtract_selection(ids) + ├── sync_edit_session_state() — update count, field readiness + └── sync_paraview_edit_selection_overlay() + └── pv_backend.update_edit_selection_overlay(selected_dataset) + [paraview_backend.py] + ├── clear_edit_selection_overlay() — delete stale TrivialProducer proxy + ├── simple.TrivialProducer("__edit_selection__") + │ — lightweight proxy that wraps a local vtkDataSet without a reader + ├── overlay.GetClientSideObject().SetOutput(selected_cells_dataset) + ├── simple.Show(overlay, view) + ├── style: gold fill, black edges, Pickable=0, LineWidth=4, PointSize=10 + │ RelativeCoincidentTopologyPolygonOffsetParameters=[-2,-2] — always on top + └── render() +``` diff --git a/docs/logic_flows/rendering.md b/docs/logic_flows/rendering.md new file mode 100644 index 0000000..6504ab2 --- /dev/null +++ b/docs/logic_flows/rendering.md @@ -0,0 +1,100 @@ +# Rendering Pipeline and Deployment + +## How Trame renders to the browser + +Trame does not render natively inside the browser. The user sees a **live screenshot +stream** of a server-side ParaView render window pushed over a WebSocket: + +``` +vtkRenderWindow (server-side) + │ + │ render() — GPU draws the scene into the window framebuffer + ▼ +pixel readback → JPEG/PNG compression + │ + ▼ +WebSocket push → browser element updated +``` + +There is one `vtkRenderWindow` for the whole application lifetime. Multiple pipeline +sources share it: each `Show(source, view)` call registers a display actor in the +single render view. `Visibility = 0` removes an actor from the draw list but keeps +it in the scene graph (bounds, picking eligibility, actor traversal overhead remain). + +## The visible server-side window + +On a workstation with an X11 or Wayland display, the `vtkRenderWindow` opens as a +**real desktop window** — the familiar grey ParaView viewport. This is the "second +window" visible on the server desktop while the user works through the browser. + +On a headless HPC node, the window is an offscreen framebuffer with no display +system involvement (EGL or OSMesa — see below). + +## Why hiding the server window causes slowdown + +When the X11/Wayland window is occluded (covered by another application) or +minimised, the compositor marks its framebuffer region as not needing repaint. +GPU drivers use this signal to: + +- Defer `glFlush()` / `glFinish()` completion +- Reduce framebuffer sync priority +- In Wayland compositors with damage tracking: refuse to composite the hidden + surface until it becomes visible again + +The result: `render()` returns on the Python side, but the GPU has not finished +writing the framebuffer. When Trame reads pixels back for the JPEG, it either +gets a stale frame or blocks waiting for GPU sync — causing round-trip latency +to spike and the browser viewport to stall. + +**This is not a bug in Trame or ParaView.** It is standard compositor behaviour +for on-screen OpenGL windows. + +## EGL vs X11/OSMesa — deployment modes + +``` +X11 path (workstation dev): + vtkRenderWindow (on-screen X11) + │ compositor can throttle GPU sync for hidden windows + ▼ + pixel readback → JPEG → WebSocket + +EGL path (production HPC, recommended): + vtkRenderWindow (offscreen GPU FBO, no window system) + │ no compositor; GPU always flushes on render() + ▼ + pixel readback → JPEG → WebSocket + +OSMesa path (no GPU / CI): + vtkRenderWindow (CPU Mesa software rasteriser) + │ always synchronous; slower but portable + ▼ + pixel readback → JPEG → WebSocket +``` + +Production `pvserver` deployments should use EGL +(`pvserver --force-offscreen-rendering` or a build with `-DVTK_USE_X=OFF +-DVTK_OPENGL_HAS_EGL=ON`). With EGL, window visibility has zero effect on +rendering throughput. + +## Scene graph overhead from multiple proxies + +Every `Show(source, view)` call registers a display actor in the render view. +Actors with `Visibility = 0` are skipped during the draw pass but still +traversed during: + +- Actor bounds computation (used by `ResetCamera`) +- Picking pass setup +- Render pass state machine + +Sources that add invisible actors during normal operation: + +| Source | When present | Visibility | +|--------|-------------|-----------| +| Edit node display (`edit_source`) | During edit session | 1 (sole visible source) | +| All other pipeline node displays | During edit session | 0 (hidden at begin; restored on discard/commit) | +| GeometryFilter helper | During surface pick setup | 0 → 1 → 0 (toggled per pick) | +| `__edit_selection__` overlay | When selection is non-empty | 1 | +| `ExtractCellsByType` extracts | When cell-dimension visibility is split | 0 or 1 | + +On EGL this overhead is negligible. On the X11 path it compounds with the +compositor throttling described above. diff --git a/docs/logic_flows/startup_load.md b/docs/logic_flows/startup_load.md new file mode 100644 index 0000000..5455706 --- /dev/null +++ b/docs/logic_flows/startup_load.md @@ -0,0 +1,78 @@ +# Startup, File Load, and State Sync + +## 1. Startup (`app.py`) + +``` +app.py + ├── configure_app() → parse CLI flags, devtools, hot-reload + ├── create_runtime_context() → instantiate ParaViewBackend, initialize_view(), EditSession + ├── get_server() → Trame websocket server (Vue 2) + ├── initialize_state() → seed all Trame state variables with defaults + ├── build_ui() → declare Vuetify layout, bind state vars and ctrl methods + ├── ParaViewRuntime(...) → create Trame-aware runtime wrapper (state + view_controls now available) + └── register_app_handlers() + ├── register_paraview_controllers() → ctrl.add("pv_*") — user action handlers + ├── register_state_handlers() → @state.change("...") — reactive callbacks + └── register_common_controllers() → upload, refresh-files, etc. +``` + +Two-phase construction is intentional: `ParaViewBackend` / `EditSession` are created *before* Trame +(no `state` dependency), then `ParaViewRuntime` wraps them *after* (needs `state`). + +--- + +## 2. File Load + +Triggered when the user picks a file in the left drawer → `state.selected_file` changes. + +``` +state.selected_file change + └── on_file_change() [state_handlers.py] + └── paraview_runtime.load_file(path) [paraview_runtime.py] + ├── refresh_runtime_message(clear=True) — reset VTK output window offset + ├── pv_backend.load_file(path) — ParaView Reader → pipeline node + ├── apply_representation("Surface with Edges") + ├── apply_coloring(__solid__) + ├── reset_view() + ├── update_ui_state() — sync all pipeline/array/display state + └── render_and_push() + ├── pv_backend.render() + └── call_view_update() — push frame to browser +``` + +--- + +## 3. UI State Sync (`update_ui_state`) + +Called after almost every action. It is the "flush everything" step. + +``` +paraview_runtime.update_ui_state() [paraview_runtime.py] + ├── pv_backend.get_ui_state() — collects pipeline_items, arrays, display props, time info + ├── writes ~30 state.* variables — pipeline_items, selected_array, color_controls_*, etc. + ├── state.can_edit_active — probes whether export-for-editing is possible + └── sync_edit_session_state() — overlays edit-mode specific state on top +``` + +--- + +## 4. Controller → Backend Pattern + +Every button click in the UI calls a `ctrl.add("pv_*")` action registered in +`paraview_controllers.py`. The pattern is always: + +``` +UI click → ctrl.pv_something() [paraview_controllers.py] + ├── pv_backend.do_thing() — ParaView mutation + ├── update_paraview_ui_state() — refresh all derived state + └── render_and_push() — render + send frame to browser +``` + +Example — adding a filter: + +``` +ctrl.pv_add_filter(filter_key) [paraview_controllers.py] + ├── pv_backend.add_filter(filter_key) + ├── update_paraview_ui_state() + └── render_and_push() +``` diff --git a/edit_session.py b/edit_session.py index f63f26e..ce3c27d 100644 --- a/edit_session.py +++ b/edit_session.py @@ -149,6 +149,11 @@ def save(self, output_path: str | Path) -> str: writer.SetInputData(self.working_dataset) if writer.Write() != 1: raise RuntimeError(f"Failed to write edited dataset to {output_path}") + if not Path(output_path).exists(): + raise RuntimeError( + f"VTK writer reported success but no file was created at {output_path}. " + "The data directory may not be writable." + ) return str(output_path) def available_cell_variables(self) -> list[str]: @@ -630,6 +635,8 @@ def build_selected_dataset(self) -> vtkUnstructuredGrid | None: def materialize_surface_selection(self) -> int: """Append selected boundary faces/edges as missing codim-1 cells.""" + # Guard: nothing to do if session inactive, no data, wrong mode, nothing selected, + # or mesh is 1D (codim-1 of edges has no meaning). if not self.active or self.working_dataset is None: return 0 if self.geometry_mode != "surface": @@ -642,9 +649,15 @@ def materialize_surface_selection(self) -> int: if top_dim < 2: return 0 + # Strip scratch arrays (e.g. CellCenters) before modifying dataset structure. self._remove_internal_edit_arrays() + # Faces appearing exactly once across all top cells — the true boundary. + # {sorted_point_ids_tuple: (cell_type, ordered_point_ids, owner_cell_id)} boundary_map = self._surface_boundary_map_for_top_cells() + # Codim-1 cells already present in the dataset — skip to avoid duplicates. existing = self._existing_codim_keys(top_dim - 1) + # Only insert faces that are selected, confirmed boundary, and not yet physical cells. + # Sorted for deterministic insertion order; idempotent on repeated calls. missing_keys = [ key for key in sorted(self.selected_surface_keys) @@ -654,6 +667,7 @@ def materialize_surface_selection(self) -> int: return 0 old_cell_count = dataset.GetNumberOfCells() + # new_cell_id → owner_volume_cell_id; used below to copy field values. owner_cell_ids = {} for key in missing_keys: cell_type, point_ids, owner_cell_id = boundary_map[key] @@ -663,10 +677,13 @@ def materialize_surface_selection(self) -> int: dataset.InsertNextCell(int(cell_type), id_list) owner_cell_ids[dataset.GetNumberOfCells() - 1] = int(owner_cell_id) + # Copy every cell data array tuple from the owner volume cell to the new face, + # keeping array lengths consistent with the new cell count. self._extend_cell_data_for_new_cells(dataset, old_cell_count, owner_cell_ids) dataset.Modified() self.dirty = True + # Dataset shape changed — invalidate boundary/adjacency caches. self._invalidate_geometry_caches() return len(missing_keys) @@ -1245,3 +1262,11 @@ def _remove_internal_edit_arrays(self) -> None: def is_supported_dataset(dataset) -> bool: """Return True when the dataset type is currently editable.""" return isinstance(dataset, vtkUnstructuredGrid) + + @staticmethod + def is_supported_dataset_type(type_string: str) -> bool: + """Return True when a DataInformation type string is currently editable. + + Used for the cheap probe in can_edit_active_node() — avoids a Fetch. + """ + return type_string == "vtkUnstructuredGrid" diff --git a/paraview_backend.py b/paraview_backend.py index 63794c2..ca7b240 100644 --- a/paraview_backend.py +++ b/paraview_backend.py @@ -5,6 +5,7 @@ import bisect import importlib.util import os +import tempfile import time from math import isfinite from typing import Any, TypedDict @@ -143,16 +144,19 @@ def __init__(self, data_directory=None, show_experimental_filters=True, state=No self._edit_selection_overlay = None self._edit_selection_display = None self._scalar_bar_visible = False - # Edit-session dataset references and remap caches — see docs/logic_flows.md §5b. + # Edit-session dataset reference self._edit_target_dataset: vtkDataSet | None = None # ref to working_dataset - self._edit_source_dataset: vtkDataSet | None = ( - None # fetched source, cached at begin - ) - self._edit_source_point_indexes: list[dict] | None = None # coord→pt_id index - self._edit_target_cell_map: dict | None = None # cell_key→cell_id - self._edit_point_id_map: dict[int, int] | None = None # src_pt_id→target_pt_id + # Edit pipeline node — real pipeline entry created at session begin. + # Picks go through the edit node source → IDs already index working_dataset. + self._edit_node_id: str | None = None # ID of the synthetic edit node + self._edit_pre_node_id: str | None = None # ID of the node active before edit + self._edit_temp_file: str | None = None # temp .vtu path (filter-active case) + # Surface-pick geometry caches — built once at session begin, cleared at end. + self._edit_source_point_indexes: list | None = None + self._edit_boundary_elements: dict | None = None self._surface_selection_helper = None self._last_selection_backend_timing = [] + self._last_session_begin_timing = [] self._boundary_cache = {} self._cell_type_name_aliases = { "Quad": "Quadrilateral", @@ -160,44 +164,67 @@ def __init__(self, data_directory=None, show_experimental_filters=True, state=No "QuadraticTetra": "QuadraticTetrahedron", } - def set_edit_target_dataset( - self, dataset: vtkDataSet | None, source_dataset: vtkDataSet | None = None - ) -> None: - """Set the edit-session dataset and pre-build remap caches. - - source_dataset may be passed to avoid a redundant Fetch when the caller - already holds the fetched source (e.g. pv_begin_edit_session). - """ + def set_edit_target_dataset(self, dataset: vtkDataSet | None) -> None: + """Register the edit-session working dataset.""" + # Edit node reads same backing data → pick IDs directly index dataset. self._edit_target_dataset = dataset - self._clear_surface_selection_helper() - self._clear_boundary_cache() - if source_dataset is None and self.source is not None: - try: - source_dataset = self.servermanager.Fetch(self.source) - except Exception: - source_dataset = None - self._edit_source_dataset = source_dataset + # Build surface-pick caches once + t = time.perf_counter() self._edit_source_point_indexes = ( - self._point_coordinate_indexes(source_dataset) - if source_dataset is not None + ParaViewBackend._point_coordinate_indexes(dataset) + if dataset is not None else None ) - self._edit_target_cell_map = ( - self._build_target_cell_map(dataset) if dataset is not None else None - ) - self._edit_point_id_map = ( - self._point_id_map_between_datasets(source_dataset, dataset) - if source_dataset is not None and dataset is not None + self._last_session_begin_timing = [ + ("point_indexes", (time.perf_counter() - t) * 1000.0) + ] + t = time.perf_counter() + self._edit_boundary_elements = ( + ParaViewBackend._boundary_codim_elements(dataset) + if dataset is not None else None ) + self._last_session_begin_timing.append( + ("boundary_elements", (time.perf_counter() - t) * 1000.0) + ) + self._clear_surface_selection_helper() + self._clear_boundary_cache() + + def consume_session_begin_timing(self) -> list: + """Return and clear timing phases from the last session begin.""" + timing = list(self._last_session_begin_timing or []) + self._last_session_begin_timing = [] + return timing def clear_edit_target_dataset(self) -> None: - """Clear edit-session dataset normalization context.""" + """Clear edit-session dataset context and delete the edit pipeline node.""" + if self._edit_node_id is not None: + pre_node_id = self._edit_pre_node_id + # delete_node() hides/deletes the proxy, removes it from pipeline_nodes, + # and auto-selects pipeline_nodes[-1] as the new active node + # (which is _edit_pre_node_id — no new nodes can be added during a session). + self.delete_node(self._edit_node_id) + self._edit_node_id = None + self._edit_pre_node_id = None + + # Restore the pre-edit node to visible so the user sees their mesh again. + if pre_node_id is not None: + pre_node = self._find_node(pre_node_id) + if pre_node is not None: + self._set_proxy_visibility(pre_node["display"], True) + self._set_extract_displays_visibility(pre_node, True) + pre_node["visibility"] = True + + if self._edit_temp_file: + try: + os.remove(self._edit_temp_file) + except OSError: + pass + self._edit_temp_file = None + self._edit_target_dataset = None - self._edit_source_dataset = None self._edit_source_point_indexes = None - self._edit_target_cell_map = None - self._edit_point_id_map = None + self._edit_boundary_elements = None self._clear_surface_selection_helper() self._clear_boundary_cache() @@ -441,30 +468,153 @@ def save_active_data(self, output_path): self.simple.SaveData(output_path, proxy=source) return output_path - def export_active_dataset_for_editing(self) -> EditSessionSource: - """Fetch the active pipeline result as a local VTK dataset for editing.""" + def can_edit_active_node(self) -> bool: + """Cheap probe: True when the active node output is editable. + + Uses GetDataInformation (no Fetch) so it never stalls on large meshes. + Returns False during an active edit session to prevent nested sessions. + """ + if self._edit_node_id is not None: + return False source = self.source - node = self._get_active_node() - if source is None or node is None: + if source is None: + return False + try: + data_info = source.GetDataInformation() + type_string = ( + data_info.GetDataSetTypeAsString() if data_info is not None else "" + ) + return EditSession.is_supported_dataset_type(type_string) + except Exception: + return False + + def export_active_dataset_for_editing(self) -> EditSessionSource: + """Create a synthetic edit pipeline node and fetch its dataset for editing. + + Two cases at session begin: + - Root reader (no parent_id): OpenDataFile on the same backing file. + Two Reader proxies on the same file produce independent in-process datasets. + - Filter active (parent_id set): XMLUnstructuredGridWriter writes the filter + output server-side to _edit_temp_.vtu; OpenDataFile loads it. + Avoids the MPI-rank collapse of Fetch(source) on pvserver deployments. + + All existing nodes are hidden; the edit node becomes the sole visible source. + Picks go through the edit source → cell IDs already index working_dataset. + """ + pre_node = self._get_active_node() + if pre_node is None: raise RuntimeError("No active pipeline item available for editing") from paraview import servermanager + source = pre_node["source"] source.UpdatePipeline() - dataset = servermanager.Fetch(source) + + # Cheap type check before any proxy creation or file I/O + data_info = source.GetDataInformation() + type_string = ( + data_info.GetDataSetTypeAsString() if data_info is not None else "" + ) + if not EditSession.is_supported_dataset_type(type_string): + raise TypeError( + f"Edit mode currently supports only vtkUnstructuredGrid outputs, " + f"got {type_string!r}." + ) + + # Create the edit source proxy (same file for root readers, temp file for filters) + edit_source, temp_file = self._create_edit_source(pre_node) + edit_source.UpdatePipeline() + + # One Fetch — produces working_dataset; picks on edit_source return IDs in this space + dataset = servermanager.Fetch(edit_source) if not EditSession.is_supported_dataset(dataset): + try: + self.simple.Delete(edit_source) + except Exception: + pass + if temp_file: + try: + os.remove(temp_file) + except OSError: + pass data_type = type(dataset).__name__ if dataset is not None else "Unknown" raise TypeError( - f"Edit mode currently supports only vtkUnstructuredGrid outputs, got {data_type}." + f"Edit mode currently supports only vtkUnstructuredGrid outputs, " + f"got {data_type}." ) + + # Capture current coloring before switching active node + current_array = self._get_selected_array() + + # Create display for the edit source + edit_display = self.simple.Show(edit_source, self.view) if self.view else None + + # Build the pipeline node descriptor + original_label = pre_node.get("label") or os.path.basename( + pre_node.get("filename") or "source" + ) + edit_node = self._make_node( + edit_source, + edit_display, + pre_node.get("filename") or "", + "edit", + f"✏ Editing: {original_label}", + parent_id=pre_node["id"], + ) + + # Hide all existing pipeline nodes so only the edit node is visible + for node in self.pipeline_nodes: + self._set_proxy_visibility(node["display"], False) + self._set_extract_displays_visibility(node, False) + node["visibility"] = False # keep node dict in sync with display state + + # Register the edit node and make it active + self.pipeline_nodes.append(edit_node) + self._edit_node_id = edit_node["id"] + self._edit_pre_node_id = pre_node["id"] + self._edit_temp_file = temp_file + self.active_node_id = edit_node["id"] + self.simple.SetActiveSource(edit_source) + if self.view is not None: + self.simple.SetActiveView(self.view) + + # Re-apply coloring to the edit display + self.apply_coloring(current_array) + return { - "node_id": node["id"], - "label": node.get("label") - or os.path.basename(node.get("filename") or "source"), - "filename": node.get("filename") or "", + "node_id": pre_node["id"], # original node ID for edit session tracking + "label": original_label, + "filename": pre_node.get("filename") or "", "dataset": dataset, } + def _create_edit_source(self, node: PipelineNode): + """Return (edit_source_proxy, temp_file_path_or_None) for session begin. + + Root reader case (no parent_id): opens a second Reader on the same file. + Filter-active case (parent_id set): writes the filter output server-side + to a temp .vtu, then opens that file — avoids Fetch(source) MPI collapse. + """ + filename = node.get("filename") or "" + parent_id = node.get("parent_id") + + if not parent_id and filename: + reader = self.simple.OpenDataFile(sanitized_vtk_xml_path(filename)) + return reader, None + + temp_path = self._resolve_temp_edit_file(node["id"]) + writer = self.simple.XMLUnstructuredGridWriter(Input=node["source"]) + writer.FileName = temp_path + writer.UpdatePipeline() + self.simple.Delete(writer) + reader = self.simple.OpenDataFile(temp_path) + return reader, temp_path + + def _resolve_temp_edit_file(self, node_id: str) -> str: + """Return the path for the server-side temp edit file (filter-active case).""" + safe_id = node_id.replace(":", "_") + return os.path.join(tempfile.gettempdir(), f"_edit_temp_{safe_id}.vtu") + def set_active_node(self, node_id): """Make the given pipeline node active.""" node = self._find_node(node_id) @@ -1846,15 +1996,17 @@ def pick_visible_cell_ids(self, x, y, radius=1): if picked_result: break - # Always clear and RENDER to hide the native ParaView purple selection + # Always clear and RENDER to hide the native ParaView purple selection highlight. + # ParaView renders a blue/purple tint over hardware-selected cells; without an + # explicit clear + render the tint persists on screen even after the selection + # is logically complete. self._clear_selection_state(source) self.render() - remapped = self._remap_cell_ids_to_edit_target_dataset(picked_result, source) print( "[selection-debug] backend.pick.click.result " - f"final_count={len(remapped)} final={self._preview_values(remapped)}" + f"final_count={len(picked_result)} final={self._preview_values(picked_result)}" ) - return remapped + return picked_result def pick_visible_cell_ids_in_rect(self, x0, y0, x1, y1, behavior="touch"): """Return selected visible cell ids inside a display-space rectangle.""" @@ -1884,12 +2036,7 @@ def pick_visible_cell_ids_in_rect(self, x0, y0, x1, y1, behavior="touch"): print( "[selection-debug] backend.pick.box " f"behavior={behavior!r} rect={rect} raw_count={len(raw_picked)} raw={self._preview_values(raw_picked)} " - f"accepted_count={len(picked)} accepted={self._preview_values(picked)}" - ) - picked = self._remap_cell_ids_to_edit_target_dataset(picked, source) - print( - "[selection-debug] backend.pick.box.result " - f"behavior={behavior!r} final_count={len(picked)} final={self._preview_values(picked)}" + f"final_count={len(picked)} final={self._preview_values(picked)}" ) if not picked or behavior != "inside": return picked @@ -2069,15 +2216,11 @@ def _pick_surface_keys_in_rect(self, rect, behavior="touch"): else: if self._polyline_intersects_rect(projected, rect): picked.append(key) - remapped = self._remap_surface_keys_to_edit_target_dataset( - picked, source_dataset=dataset - ) print( "[selection-debug] backend.pick.surface.fallback " - f"behavior={behavior!r} rect={rect} raw_count={len(picked)} raw={self._preview_values(picked)} " - f"final_count={len(remapped)} final={self._preview_values(remapped)}" + f"behavior={behavior!r} rect={rect} final_count={len(picked)} final={self._preview_values(picked)}" ) - return remapped + return picked def _pick_surface_keys_native(self, rect, behavior="touch"): """Native ParaView selection path for visible boundary faces (3D).""" @@ -2093,17 +2236,11 @@ def record_phase(name, start): return None edit_dataset = self._edit_target_dataset - phase_start = time.perf_counter() - source_dataset = None - if edit_dataset is None: - try: - source_dataset = self.servermanager.Fetch(source) - except Exception: - return None - edit_dataset = source_dataset - record_phase("dataset", phase_start) + # Guard defensively against out-of-session calls. if edit_dataset is None: return None + phase_start = time.perf_counter() + record_phase("dataset", phase_start) if not hasattr(edit_dataset, "GetNumberOfCells") or not hasattr( edit_dataset, "GetCell" ): @@ -2118,7 +2255,8 @@ def record_phase(name, start): default=0, ) record_phase("top_dimension", phase_start) - # Keep old custom path for 2D/1D where ExtractSurface does not target boundary edges. + # GeometryFilter on a 2D mesh returns the 2D cells, not their 1D boundary edges. + # Use the projection-based fallback instead. if top_dim != 3: return None @@ -2163,18 +2301,12 @@ def record_phase(name, start): phase_start = time.perf_counter() selected_dataset = self.servermanager.Fetch(extract) record_phase("fetch_selection", phase_start) - if source_dataset is None: - phase_start = time.perf_counter() - try: - source_dataset = self.servermanager.Fetch(source) - except Exception: - source_dataset = edit_dataset - record_phase("fetch_source_dataset", phase_start) phase_start = time.perf_counter() keys = self._surface_keys_from_selected_dataset( selected_dataset, - source_dataset=source_dataset, + source_dataset=edit_dataset, source_point_indexes=self._edit_source_point_indexes, + prebuilt_boundary=self._edit_boundary_elements, ) record_phase("map_selection_keys", phase_start) if keys is None: @@ -2191,7 +2323,7 @@ def record_phase(name, start): inside_keys = [] for key in keys: projected = self._project_points_to_display( - source_dataset, key, renderer + edit_dataset, key, renderer ) if projected and all( self._point_in_rect(point, rect) for point in projected @@ -2199,17 +2331,11 @@ def record_phase(name, start): inside_keys.append(key) keys = inside_keys record_phase("inside_refine", phase_start) - phase_start = time.perf_counter() - remapped = self._remap_surface_keys_to_edit_target_dataset( - keys, source_dataset=source_dataset - ) - record_phase("remap_to_edit_target", phase_start) print( "[selection-debug] backend.pick.surface.native.result " - f"behavior={behavior!r} rect={rect} raw_count={len(keys or [])} raw={self._preview_values(keys or [])} " - f"final_count={len(remapped)} final={self._preview_values(remapped)}" + f"behavior={behavior!r} rect={rect} final_count={len(keys or [])} final={self._preview_values(keys or [])}" ) - return remapped + return keys except Exception: return None finally: @@ -2316,18 +2442,14 @@ def _surface_keys_from_selected_dataset( selected_dataset: vtkDataSet | None, source_dataset: vtkDataSet | None = None, source_point_indexes: list[dict] | None = None, + prebuilt_boundary: dict | None = None, ) -> list | None: """Map selected GeometryFilter cells back to original source point-id keys. Builds a coordinate index from source_dataset and maps each selected cell's - XYZ corners back to source point IDs. Neither vtkOriginalPointIds nor - vtkOriginalCellIds is used: in ParaView 6.1 both PassThroughPointIds and - PassThroughCellIds on GeometryFilter produce output-geometry indices through - the proxy/server-fetch round-trip rather than source topology IDs (see PR #31 - and issue #34). - - source_point_indexes may be passed as a pre-built index (cached at edit-session - begin, B4) to skip the O(n_source_points) build on each pick. + XYZ corners back to source point IDs. + + source_point_indexes and prebuilt_boundary may be passed as pre-built caches """ if source_dataset is None: return None @@ -2376,16 +2498,22 @@ def coordinate_key(block, cell): seen.add(key) deduped.append(key) return ParaViewBackend._normalize_surface_keys_to_source_boundary( - deduped, source_dataset=source_dataset + deduped, source_dataset=source_dataset, prebuilt_boundary=prebuilt_boundary ) @staticmethod - def _normalize_surface_keys_to_source_boundary(keys, source_dataset=None): + def _normalize_surface_keys_to_source_boundary( + keys, source_dataset=None, prebuilt_boundary=None + ): """Resolve partial picked keys to canonical source boundary keys when possible.""" if not keys or source_dataset is None: return keys or [] - boundary = ParaViewBackend._boundary_codim_elements(source_dataset) + boundary = ( + prebuilt_boundary + if prebuilt_boundary is not None + else ParaViewBackend._boundary_codim_elements(source_dataset) + ) if not boundary: return keys @@ -2800,99 +2928,6 @@ def _fetch_selected_original_cell_ids(self, source): return selected_ids - def _remap_cell_ids_to_edit_target_dataset( - self, cell_ids: list[int], source: Any - ) -> list[int]: - """Normalize picked source cell IDs onto the active edit-session dataset.""" - picked_ids = [int(cell_id) for cell_id in (cell_ids or [])] - target_dataset = self._edit_target_dataset - if target_dataset is None or not picked_ids: - return picked_ids - - source_dataset = self._edit_source_dataset - if source_dataset is None: - try: - source_dataset = self.servermanager.Fetch(source) - except Exception: - source_dataset = None - remapped = self._remap_cell_ids_between_datasets( - picked_ids, - source_dataset, - target_dataset, - _target_cell_map=self._edit_target_cell_map, - ) - print( - "[selection-debug] backend.remap.cells " - f"input_count={len(picked_ids)} input={self._preview_values(picked_ids)} " - f"remapped_count={len(remapped)} remapped={self._preview_values(remapped)}" - ) - return remapped if remapped else picked_ids - - def _remap_surface_keys_to_edit_target_dataset( - self, keys: list, source_dataset: vtkDataSet | None = None - ) -> list: - """Normalize picked source boundary keys onto edit-session point IDs.""" - picked_keys = [tuple(key) for key in (keys or []) if key] - target_dataset = self._edit_target_dataset - if target_dataset is None or not picked_keys: - return picked_keys - if source_dataset is target_dataset: - return picked_keys - - point_id_map = self._edit_point_id_map - if point_id_map is None: - if source_dataset is None: - return picked_keys - point_id_map = self._point_id_map_between_datasets( - source_dataset, target_dataset - ) - if not point_id_map: - return picked_keys - - remapped = [] - for key in picked_keys: - mapped = [point_id_map.get(int(source_point_id)) for source_point_id in key] - if any(point_id is None for point_id in mapped): - continue - remapped.append(tuple(sorted(mapped))) - - if not remapped: - return picked_keys - normalized = self._dedupe_ordered(remapped) - print( - "[selection-debug] backend.remap.surface " - f"input_count={len(picked_keys)} input={self._preview_values(picked_keys)} " - f"remapped_count={len(remapped)} remapped={self._preview_values(remapped)} " - f"normalized_count={len(normalized)} normalized={self._preview_values(normalized)}" - ) - return normalized - - @classmethod - def _point_id_map_between_datasets( - cls, source_dataset: vtkDataSet | None, target_dataset: vtkDataSet | None - ) -> dict[int, int]: - """Map source point IDs to target point IDs by tolerant coordinates.""" - if source_dataset is None or target_dataset is None: - return {} - try: - source_count = source_dataset.GetNumberOfPoints() - target_indexes = cls._point_coordinate_indexes(target_dataset) - except Exception: - return {} - if not target_indexes: - return {} - - point_id_map = {} - for source_point_id in range(source_count): - try: - point = source_dataset.GetPoint(source_point_id) - except Exception: - continue - target_point_id = cls._lookup_point_coordinate(target_indexes, point) - if target_point_id is not None: - point_id_map[int(source_point_id)] = int(target_point_id) - return point_id_map - @staticmethod def _dedupe_ordered(values): deduped = [] @@ -3001,78 +3036,6 @@ def _cell_coordinate_key(dataset, cell): return None return tuple(sorted(coords)) - @staticmethod - def _build_target_cell_map(target_dataset: vtkDataSet) -> dict: - """Build cell-geometry key → cell ID map once for caching the remap lookup.""" - if not hasattr(target_dataset, "GetCell"): - return {} - cell_map = {} - for target_cell_id in range(target_dataset.GetNumberOfCells()): - target_cell = target_dataset.GetCell(target_cell_id) - key = ParaViewBackend._cell_coordinate_key(target_dataset, target_cell) - if key is None: - continue - cell_map.setdefault(key, int(target_cell_id)) - return cell_map - - @staticmethod - def _remap_cell_ids_between_datasets( - cell_ids: list[int], - source_dataset: vtkDataSet | None, - target_dataset: vtkDataSet | None, - *, - _target_cell_map: dict | None = None, - ) -> list[int]: - """Map source-dataset cell ids to target-dataset ids by cell geometry.""" - if not cell_ids: - return [] - if source_dataset is None or target_dataset is None: - return [int(cell_id) for cell_id in cell_ids] - if not hasattr(source_dataset, "GetCell") or not hasattr( - target_dataset, "GetCell" - ): - return [int(cell_id) for cell_id in cell_ids] - - target_count = target_dataset.GetNumberOfCells() - if _target_cell_map is not None: - target_cell_map = _target_cell_map - else: - target_cell_map = {} - for target_cell_id in range(target_count): - target_cell = target_dataset.GetCell(target_cell_id) - key = ParaViewBackend._cell_coordinate_key(target_dataset, target_cell) - if key is None: - continue - target_cell_map.setdefault(key, int(target_cell_id)) - - remapped = [] - for cell_id in cell_ids: - mapped_id = None - try: - source_cell_id = int(cell_id) - except (TypeError, ValueError): - continue - if 0 <= source_cell_id < source_dataset.GetNumberOfCells(): - source_cell = source_dataset.GetCell(source_cell_id) - source_key = ParaViewBackend._cell_coordinate_key( - source_dataset, source_cell - ) - if source_key is not None: - mapped_id = target_cell_map.get(source_key) - if mapped_id is None and 0 <= source_cell_id < target_count: - mapped_id = int(source_cell_id) - if mapped_id is not None: - remapped.append(int(mapped_id)) - - deduped = [] - seen = set() - for cell_id in remapped: - if cell_id in seen: - continue - seen.add(cell_id) - deduped.append(cell_id) - return deduped - @staticmethod def _source_cell_ids_from_selected_dataset(selected_dataset, source_dataset): """Best-effort map selected cells back to source cell ids by geometry.""" @@ -3478,7 +3441,12 @@ def _active_kind_label(self): node = self._get_active_node() if node is None: return "Reader Type" - return "Filter Type" if node.get("kind") == "filter" else "Reader Type" + kind = node.get("kind") + if kind == "filter": + return "Filter Type" + if kind == "edit": + return "Edit Session" + return "Reader Type" def _active_parent_label(self): """Return the parent pipeline label for the active node when available.""" diff --git a/paraview_controllers.py b/paraview_controllers.py index 71b231d..85509f7 100644 --- a/paraview_controllers.py +++ b/paraview_controllers.py @@ -723,17 +723,20 @@ def pv_begin_edit_session(): try: debug_view("pv_begin_edit_session.start", mode=state.mainViewMode) - exported = pv_backend.export_active_dataset_for_editing() - edit_session.begin( - exported["node_id"], - exported["label"], - exported["filename"], - exported["dataset"], - ) - pv_backend.set_edit_target_dataset( - edit_session.working_dataset, - source_dataset=exported["dataset"], - ) + timing = SelectionTiming("session_begin") + with timing.phase("export"): + exported = pv_backend.export_active_dataset_for_editing() + with timing.phase("edit_begin"): + edit_session.begin( + exported["node_id"], + exported["label"], + exported["filename"], + exported["dataset"], + ) + with timing.phase("set_target"): + pv_backend.set_edit_target_dataset(edit_session.working_dataset) + for name, ms in pv_backend.consume_session_begin_timing(): + timing.add_phase(f"target.{name}", ms) pv_backend.apply_representation("Surface with Edges") state.pick_mode = True @@ -749,9 +752,11 @@ def pv_begin_edit_session(): state.selection_count = 0 state.inspector_tab = 3 _set_selection_mode("replace") + update_paraview_ui_state() sync_edit_session_state() sync_paraview_edit_selection_overlay() render_and_push() + timing.emit(state, state_key="session_begin_timing") debug_view("pv_begin_edit_session.end", mode=state.mainViewMode) notify(state, f"Edit session initialized for {exported['label']}.", "info") except Exception as exc: @@ -765,6 +770,7 @@ def pv_discard_edit_session(): debug_view("pv_discard_edit_session.start", mode=state.mainViewMode) edit_session.clear() pv_backend.clear_edit_target_dataset() + update_paraview_ui_state() sync_edit_session_state() sync_paraview_edit_selection_overlay() state.save_filename = pv_backend.default_output_filename() diff --git a/paraview_filter_catalog.py b/paraview_filter_catalog.py index 7e8a5c9..d9b35a8 100644 --- a/paraview_filter_catalog.py +++ b/paraview_filter_catalog.py @@ -199,11 +199,14 @@ def experimental_filter_spec(self, filter_key): def pipeline_icon(self, node): """Return a kind-aware icon for a pipeline entry.""" - if node.get("kind") == "filter": + kind = node.get("kind") + if kind == "filter": filter_key = node.get("filter_key") if filter_key in self.supported_filters: return self.supported_filters[filter_key]["icon"] return "mdi-filter-outline" + if kind == "edit": + return "mdi-pencil-outline" return "mdi-database-outline" def _discover_experimental_filters(self): diff --git a/paraview_runtime.py b/paraview_runtime.py index 2a4dd70..9aac92b 100644 --- a/paraview_runtime.py +++ b/paraview_runtime.py @@ -327,11 +327,7 @@ def update_ui_state(self): else "Active pipeline result" ) ) - try: - self.pv_backend.export_active_dataset_for_editing() - self.state.can_edit_active = True - except Exception: - self.state.can_edit_active = False + self.state.can_edit_active = self.pv_backend.can_edit_active_node() self.sync_edit_session_state() def load_file(self, selected_file): diff --git a/selection_timing.py b/selection_timing.py index ef8ee47..01031ef 100644 --- a/selection_timing.py +++ b/selection_timing.py @@ -28,7 +28,7 @@ def update(self, **metadata): def add_phase(self, name, duration_ms): self._phases.append((name, float(duration_ms))) - def emit(self, state=None, status="ok"): + def emit(self, state=None, status="ok", state_key="selection_timing"): total_ms = (time.perf_counter() - self._start) * 1000.0 phases = [ {"name": name, "ms": round(duration_ms, 3)} @@ -43,8 +43,8 @@ def emit(self, state=None, status="ok"): } print("[selection-timing] " + json.dumps(payload, sort_keys=True, default=str)) if state is not None: - state.selection_timing_payload = payload - state.selection_timing_last = self._format_summary(total_ms) + setattr(state, f"{state_key}_payload", payload) + setattr(state, f"{state_key}_last", self._format_summary(total_ms)) def _format_summary(self, total_ms): phase_text = ", ".join( diff --git a/state_handlers.py b/state_handlers.py index 52c1856..50a5449 100644 --- a/state_handlers.py +++ b/state_handlers.py @@ -60,6 +60,10 @@ def on_active_pipeline_item_change(active_pipeline_item, **kwargs): if not active_pipeline_item: return if edit_session.active: + if active_pipeline_item == paraview_runtime.pv_backend.active_node_id: + # Programmatic sync (e.g. update_ui_state updating to the edit node) — + # no user action, no warning needed. + return notify( state, "Cannot switch pipeline node during an edit session.", "warning" ) diff --git a/state_setup.py b/state_setup.py index 66f16b4..fabd19a 100644 --- a/state_setup.py +++ b/state_setup.py @@ -143,6 +143,8 @@ def initialize_state( state.edit_selection_event = "" state.selection_timing_last = "" state.selection_timing_payload = {} + state.session_begin_timing_last = "" + state.session_begin_timing_payload = {} state.edit_selection_mode = "replace" state.edit_selection_mode_options = [ {"text": "Replace", "value": "replace"}, diff --git a/tests/test_e2e_edit_selection_playwright.py b/tests/test_e2e_edit_selection_playwright.py index d8f94a8..54ae0f5 100644 --- a/tests/test_e2e_edit_selection_playwright.py +++ b/tests/test_e2e_edit_selection_playwright.py @@ -652,6 +652,7 @@ def test_paraview_categorical_annotations_updated_on_file_switch(shared_browser) _wait_for_switch_checked( page, "Interpret values as categories", True, timeout_s=8 ) + time.sleep(1.0) file_a_switch_png = _wait_for_stable_viewport(viewport) _set_switch(page, "Interpret values as categories", False) @@ -667,6 +668,7 @@ def test_paraview_categorical_annotations_updated_on_file_switch(shared_browser) _wait_for_switch_checked( page, "Interpret values as categories", True, timeout_s=8 ) + time.sleep(1.0) file_b_switch_png = _wait_for_stable_viewport(viewport) _set_switch(page, "Interpret values as categories", False) @@ -1431,3 +1433,125 @@ def _try_box(fx0, fy0, fx1, fy1, label): proc.wait(timeout=10) except subprocess.TimeoutExpired: proc.kill() + + +def test_paraview_edit_session_shows_edit_node_in_pipeline_panel(shared_browser): + """After entering edit mode a pipeline node labelled '✏ Editing: …' must appear.""" + if not is_paraview_available(): + pytest.skip("ParaView is not installed") + + port = _free_tcp_port() + url = f"http://127.0.0.1:{port}" + proc = subprocess.Popen( + [ + sys.executable, + "app.py", + "--backend", + "paraview", + "--server", + "--data-directory", + str(TEST_DATA_DIR), + "--file", + str(TEST_GRID), + "--host", + "127.0.0.1", + "--port", + str(port), + ], + cwd=ROOT_DIR, + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + text=True, + ) + _drain_proc_stdout(proc) + + try: + _wait_for_http_ready(url) + context = shared_browser.new_context(viewport={"width": 1920, "height": 1080}) + page = context.new_page() + page.goto(url, wait_until="domcontentloaded") + page.wait_for_selector("button:has-text('Enter Edit Mode')", timeout=40000) + + page.click("button:has-text('Enter Edit Mode')") + page.wait_for_selector("text=Edit Tools", timeout=40000) + time.sleep(1.0) + + # The edit pipeline node must appear in the left pipeline panel. + edit_items = page.locator("div.v-list-item__title:has-text('✏ Editing')") + edit_items.first.wait_for(state="visible", timeout=10000) + assert ( + edit_items.count() > 0 + ), "Edit node must appear in the pipeline panel with a '✏ Editing: …' label" + + context.close() + finally: + if proc.poll() is None: + proc.terminate() + try: + proc.wait(timeout=10) + except subprocess.TimeoutExpired: + proc.kill() + + +def test_paraview_edit_session_discard_restores_visible_mesh(shared_browser): + """After discarding an edit session the original node is restored and the mesh is visible.""" + if not is_paraview_available(): + pytest.skip("ParaView is not installed") + + port = _free_tcp_port() + url = f"http://127.0.0.1:{port}" + proc = subprocess.Popen( + [ + sys.executable, + "app.py", + "--backend", + "paraview", + "--server", + "--data-directory", + str(TEST_DATA_DIR), + "--file", + str(TEST_GRID), + "--host", + "127.0.0.1", + "--port", + str(port), + ], + cwd=ROOT_DIR, + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + text=True, + ) + _drain_proc_stdout(proc) + + try: + _wait_for_http_ready(url) + context = shared_browser.new_context(viewport={"width": 1920, "height": 1080}) + page = context.new_page() + page.goto(url, wait_until="domcontentloaded") + page.wait_for_selector("button:has-text('Enter Edit Mode')", timeout=40000) + + viewport = page.locator(".coral-main-viewport") + _, before_bbox = _wait_for_foreground(viewport, timeout_s=12) + assert before_bbox is not None, "Mesh must be visible before entering edit mode" + + page.click("button:has-text('Enter Edit Mode')") + page.wait_for_selector("text=Edit Tools", timeout=40000) + time.sleep(1.0) + + page.click("button:has-text('Discard')") + # After discard, the original node is restored — the mesh must be visible again. + page.wait_for_selector("button:has-text('Enter Edit Mode')", timeout=20000) + _, after_bbox = _wait_for_foreground(viewport, timeout_s=12) + assert after_bbox is not None, ( + "Mesh must be visible after discarding an edit session — " + "original node visibility must be restored" + ) + + context.close() + finally: + if proc.poll() is None: + proc.terminate() + try: + proc.wait(timeout=10) + except subprocess.TimeoutExpired: + proc.kill() diff --git a/tests/test_edit_session.py b/tests/test_edit_session.py index fb95564..853758b 100644 --- a/tests/test_edit_session.py +++ b/tests/test_edit_session.py @@ -233,3 +233,11 @@ def test_surface_mode_save_legacy_vtk_omits_internal_cell_centers(tmp_path): 42.0 ) assert loaded.GetCellData().GetArray("CellCenters") is None + + +def test_is_supported_dataset_type_accepts_only_unstructured_grid(): + assert EditSession.is_supported_dataset_type("vtkUnstructuredGrid") is True + assert EditSession.is_supported_dataset_type("vtkPolyData") is False + assert EditSession.is_supported_dataset_type("vtkStructuredGrid") is False + assert EditSession.is_supported_dataset_type("") is False + assert EditSession.is_supported_dataset_type("vtkUnstructuredGridBase") is False diff --git a/tests/test_paraview_backend.py b/tests/test_paraview_backend.py index cb427fd..f8ccb85 100644 --- a/tests/test_paraview_backend.py +++ b/tests/test_paraview_backend.py @@ -450,10 +450,9 @@ def make_backend(): backend._edit_selection_display = None backend._scalar_bar_visible = False backend._edit_target_dataset = None - backend._edit_source_dataset = None - backend._edit_source_point_indexes = None - backend._edit_target_cell_map = None - backend._edit_point_id_map = None + backend._edit_node_id = None + backend._edit_pre_node_id = None + backend._edit_temp_file = None backend._surface_selection_helper = None backend._boundary_cache = {} backend._last_selection_backend_timing = [] @@ -1761,249 +1760,3 @@ def GetCell(self, cell_id): cell_ids = ParaViewBackend._source_cell_ids_from_selected_dataset(selected, source) assert cell_ids == [0] - - -def test_set_edit_target_dataset_builds_remap_caches(): - source_ds = FakeSurfaceDataset( - points=[(0.0, 0.0, 0.0), (1.0, 0.0, 0.0), (1.0, 1.0, 0.0)], - cells=[(0, 1, 2)], - ) - target_ds = FakeSurfaceDataset( - points=[(1.0, 1.0, 0.0), (1.0, 0.0, 0.0), (0.0, 0.0, 0.0)], - cells=[(2, 1, 0)], - ) - backend = make_backend() - - backend.set_edit_target_dataset(target_ds, source_dataset=source_ds) - - assert backend._edit_source_dataset is source_ds - assert backend._edit_source_point_indexes is not None - assert isinstance(backend._edit_target_cell_map, dict) - assert len(backend._edit_target_cell_map) == 1 - assert isinstance(backend._edit_point_id_map, dict) - assert len(backend._edit_point_id_map) == 3 - - -def test_set_edit_target_dataset_clears_caches_on_clear(): - source_ds = FakeSurfaceDataset( - points=[(0.0, 0.0, 0.0), (1.0, 0.0, 0.0), (1.0, 1.0, 0.0)], - cells=[(0, 1, 2)], - ) - target_ds = FakeSurfaceDataset( - points=[(1.0, 1.0, 0.0), (1.0, 0.0, 0.0), (0.0, 0.0, 0.0)], - cells=[(2, 1, 0)], - ) - backend = make_backend() - backend.set_edit_target_dataset(target_ds, source_dataset=source_ds) - - backend.clear_edit_target_dataset() - - assert backend._edit_target_dataset is None - assert backend._edit_source_dataset is None - assert backend._edit_source_point_indexes is None - assert backend._edit_target_cell_map is None - assert backend._edit_point_id_map is None - - -def test_remap_cell_ids_uses_cached_source_without_fetch(): - fetched = [] - source_ds = FakeSurfaceDataset( - points=[(0.0, 0.0, 0.0), (1.0, 0.0, 0.0), (1.0, 1.0, 0.0)], - cells=[(0, 1, 2)], - ) - target_ds = FakeSurfaceDataset( - points=[(1.0, 1.0, 0.0), (1.0, 0.0, 0.0), (0.0, 0.0, 0.0)], - cells=[(2, 1, 0)], - ) - backend = make_backend() - backend._edit_target_dataset = target_ds - backend._edit_source_dataset = source_ds - backend._edit_target_cell_map = ParaViewBackend._build_target_cell_map(target_ds) - backend.servermanager = SimpleNamespace( - Fetch=lambda _: fetched.append(1) or source_ds - ) - - backend._remap_cell_ids_to_edit_target_dataset([0], object()) - - assert ( - len(fetched) == 0 - ), "Fetch must not be called when _edit_source_dataset is cached" - - -def test_remap_surface_keys_uses_cached_point_id_map(): - # Without a cached map, source_dataset=None triggers an early return (keys unchanged). - # With a cached map, remapping proceeds even when source_dataset is not provided. - # The map shifts each source point ID by 1, so the remapped key differs from the - # input — proving the cache path was taken rather than the early return. - target_ds = FakeSurfaceDataset( - points=[(1.0, 1.0, 0.0), (1.0, 0.0, 0.0), (0.0, 0.0, 0.0)], - cells=[(2, 1, 0)], - ) - # source pt 0 → target pt 1, source pt 1 → target pt 2, source pt 2 → target pt 3 - pre_built = {0: 1, 1: 2, 2: 3} - backend = make_backend() - backend._edit_target_dataset = target_ds - backend._edit_point_id_map = pre_built - - remapped = backend._remap_surface_keys_to_edit_target_dataset( - [(0, 1, 2)], source_dataset=None - ) - - # (0,1,2) → [1,2,3] → sorted → (1,2,3); early return would give (0,1,2) - assert remapped == [(1, 2, 3)] - - -def test_remap_cell_ids_between_datasets_matches_by_geometry_coordinates(): - source = FakeSurfaceDataset( - points=[ - (0.0, 0.0, 0.0), - (1.0, 0.0, 0.0), - (1.0, 1.0, 0.0), - (0.0, 1.0, 0.0), - (2.0, 0.0, 0.0), - (2.0, 1.0, 0.0), - ], - cells=[(0, 1, 2, 3), (1, 4, 5, 2)], - ) - target = FakeSurfaceDataset( - points=[ - (2.0, 1.0, 0.0), - (2.0, 0.0, 0.0), - (1.0, 0.0, 0.0), - (1.0, 1.0, 0.0), - (0.0, 1.0, 0.0), - (0.0, 0.0, 0.0), - ], - cells=[(2, 1, 0, 3), (5, 2, 3, 4)], - ) - - remapped = ParaViewBackend._remap_cell_ids_between_datasets([0, 1], source, target) - - assert remapped == [1, 0] - - -def test_remap_surface_keys_to_edit_target_dataset_uses_point_coordinates(): - backend = make_backend() - source = FakeSurfaceDataset( - points=[ - (0.0, 0.0, 0.0), - (1.0, 0.0, 0.0), - (1.0, 1.0, 0.0), - (0.0, 1.0, 0.0), - ], - cells=[(0, 1, 2, 3)], - ) - target = FakeSurfaceDataset( - points=[ - (1.0, 1.0, 0.0), - (1.0, 0.0, 0.0), - (0.0, 0.0, 0.0), - (0.0, 1.0, 0.0), - ], - cells=[(2, 1, 0, 3)], - ) - backend._edit_target_dataset = target - backend._normalize_surface_keys_to_source_boundary = ( - lambda keys, source_dataset=None: keys - ) - - remapped = backend._remap_surface_keys_to_edit_target_dataset( - [(0, 1, 2, 3)], source_dataset=source - ) - - assert remapped == [(0, 1, 2, 3)] - - -def test_selected_original_source_ids_remap_to_edit_target_surface_keys(): - backend = make_backend() - source = FakeSurfaceDataset( - points=[ - (10.0, 0.0, 0.0), - (11.0, 0.0, 0.0), - (11.0, 1.0, 0.0), - (10.0, 1.0, 0.0), - ], - cells=[(0, 1, 2, 3)], - ) - target = FakeSurfaceDataset( - points=[ - (10.0, 1.0, 0.0), - (11.0, 1.0, 0.0), - (11.0, 0.0, 0.0), - (10.0, 0.0, 0.0), - ], - cells=[(3, 2, 1, 0)], - ) - selected = FakeSurfaceDataset( - points=[ - (10.0, 0.0, 0.0), - (11.0, 0.0, 0.0), - (11.0, 1.0, 0.0), - ], - cells=[(0, 1, 2)], - ) - backend._edit_target_dataset = target - - source_keys = ParaViewBackend._surface_keys_from_selected_dataset( - selected, source_dataset=source - ) - remapped = backend._remap_surface_keys_to_edit_target_dataset( - source_keys, source_dataset=source - ) - - assert source_keys == [(0, 1, 2)] - assert remapped == [(1, 2, 3)] - - -def test_remap_surface_keys_to_edit_target_dataset_skips_identity_remap(): - backend = make_backend() - dataset = FakeSurfaceDataset( - points=[ - (0.0, 0.0, 0.0), - (1.0, 0.0, 0.0), - (1.0, 1.0, 0.0), - ], - cells=[(0, 1, 2)], - ) - backend._edit_target_dataset = dataset - backend._point_coordinate_indexes = lambda _dataset: (_ for _ in ()).throw( - AssertionError("identity remap should not build coordinate indexes") - ) - - remapped = backend._remap_surface_keys_to_edit_target_dataset( - [(0, 1, 2)], source_dataset=dataset - ) - - assert remapped == [(0, 1, 2)] - - -def test_remap_surface_keys_to_edit_target_dataset_tolerates_coordinate_noise(): - backend = make_backend() - source = FakeSurfaceDataset( - points=[ - (0.0, 0.0, 0.0), - (1.0, 0.0, 0.0), - (1.0, 1.0, 0.0), - (0.0, 1.0, 0.0), - ], - cells=[(0, 1, 2, 3)], - ) - target = FakeSurfaceDataset( - points=[ - (1.0 + 1e-11, 1.0, 0.0), - (1.0, 1e-11, 0.0), - (0.0, 0.0, 0.0), - (0.0, 1.0 - 1e-11, 0.0), - ], - cells=[(2, 1, 0, 3)], - ) - backend._edit_target_dataset = target - backend._normalize_surface_keys_to_source_boundary = ( - lambda keys, source_dataset=None: keys - ) - - remapped = backend._remap_surface_keys_to_edit_target_dataset( - [(0, 1, 2, 3)], source_dataset=source - ) - - assert remapped == [(0, 1, 2, 3)] diff --git a/tests/test_paraview_runtime.py b/tests/test_paraview_runtime.py index 67b5286..5d5f83a 100644 --- a/tests/test_paraview_runtime.py +++ b/tests/test_paraview_runtime.py @@ -101,6 +101,10 @@ def get_ui_state(self): def default_output_filename(self): return "pipeline_result.vtu" + def can_edit_active_node(self): + self.calls.append("can_edit_active_node") + return True + def export_active_dataset_for_editing(self): self.calls.append("export_active_dataset_for_editing") return {"dataset": object()} diff --git a/tests/test_state_handlers.py b/tests/test_state_handlers.py index 729b6b2..8c63732 100644 --- a/tests/test_state_handlers.py +++ b/tests/test_state_handlers.py @@ -234,6 +234,47 @@ def test_pipeline_node_switch_blocked_during_edit_session(): ), "UI selection must snap back to current node" +def test_pipeline_node_switch_allows_programmatic_sync_to_same_node(): + """Programmatic sync (same node ID) during edit session must not warn or snap back.""" + switched = [] + pv_backend = SimpleNamespace( + source=object(), + display=object(), + apply_coloring=lambda v: None, + apply_representation=lambda v: None, + set_active_node=lambda node_id: switched.append(node_id) or True, + set_interactor_rotation=lambda enabled: None, + active_node_id="edit-node-1", # backend already tracks the edit node + ) + state = FakeState( + notification_message="", + notification_type="info", + notification_show=False, + active_pipeline_item="edit-node-1", # UI already shows the edit node + edit_session_active=True, # session is active + interactive_quality=0, + interactive_ratio=0, + ) + + _register( + state, + pv_backend=pv_backend, + edit_session=_make_edit_session(active=True), + render_and_push=lambda: None, + ) + + # Simulate update_ui_state() writing the same node ID back (programmatic sync) + state._handlers["active_pipeline_item"]("edit-node-1") + + assert ( + not state.notification_show + ), "Programmatic sync to the same node must not show a warning" + assert ( + state.active_pipeline_item == "edit-node-1" + ), "Programmatic sync must not snap the UI selection back" + assert switched == [], "set_active_node must not be called for a same-node sync" + + def test_pick_mode_in_edit_session_disables_rotation(): calls = [] pv_backend = SimpleNamespace( diff --git a/ui.py b/ui.py index d629def..e3cc56f 100644 --- a/ui.py +++ b/ui.py @@ -549,13 +549,6 @@ def _build_selection_tools_panel(ctrl): dense=True, disabled=("!group_select",), ) - with vuetify.VListItem(v_show=("selection_timing_last",)): - with vuetify.VListItemContent(): - vuetify.VListItemSubtitle("Last selection timing") - vuetify.VListItemTitle( - "{{ selection_timing_last }}", - style="font-family: monospace; white-space: normal; font-size: 0.78rem;", - ) def _build_edit_assign_panel(ctrl): @@ -1348,6 +1341,22 @@ def _build_paraview_inspector_panel(ctrl): _build_selection_tools_panel(ctrl) vuetify.VDivider(classes="my-4") _build_edit_assign_panel(ctrl) + vuetify.VDivider(classes="my-4") + with vuetify.VList(dense=True): + with vuetify.VListItem(v_show=("session_begin_timing_last",)): + with vuetify.VListItemContent(): + vuetify.VListItemSubtitle("Last session begin timing") + vuetify.VListItemTitle( + "{{ session_begin_timing_last }}", + style="font-family: monospace; white-space: normal; font-size: 0.78rem;", + ) + with vuetify.VListItem(v_show=("selection_timing_last",)): + with vuetify.VListItemContent(): + vuetify.VListItemSubtitle("Last selection timing") + vuetify.VListItemTitle( + "{{ selection_timing_last }}", + style="font-family: monospace; white-space: normal; font-size: 0.78rem;", + ) def _build_property_list(ctrl, state_key):