Skip to content

Provide a general mechanism for analytical metrics #25

Description

@JulesGuillou

Context

Holoflow currently handles analytical metrics through specialized paths. For example, Zernike coefficients have
dedicated processing and plotting behavior.

This approach will become difficult to maintain as more metrics are introduced. We need a consistent way for users
to create, discover, manage, visualize, save, and export analytical metrics.

Here, metrics means scientific or analytical measurements such as Zernike coefficients—not runtime, profiling,
throughput, or scheduler metrics.

Goal

Define and deliver a general user-facing workflow for analytical metrics.

The mechanism should ultimately let users:

  • Compute metrics from an entire rendered output or from a selected region of interest.
  • Use pipeline nodes that directly produce metrics.
  • Discover and manage available metrics from the GUI.
  • Explore metric values using suitable visualizations.
  • Work with multiple metrics without requiring a dedicated interface for each one.
  • Save metric configurations and restore them later.
  • Export collected metric data together with the context needed to interpret it.

The design should also account for a future in which users can compose arbitrary metrics from the GUI. A complete
user-defined formula or graph builder is not necessarily required in the first iteration.

Representative workflows

  • A user selects a rendered output, optionally selects a region, chooses a metric, and explores its evolution.
  • A processing node produces one or more metrics that become available in the GUI.
  • A user configures and compares several metrics during an acquisition.
  • A user saves the configuration, restores it in a later session, and exports the collected values.
  • Existing Zernike coefficient plots are supported through the general workflow rather than remaining the model for
    additional specialized displays.

Discovery work

This issue intentionally does not prescribe an implementation. The work should first establish:

  • The user-facing definition and lifecycle of a metric.
  • How metrics are created from rendered outputs, regions, and pipeline nodes.
  • Which metric data forms and metadata must be represented.
  • How users discover, name, configure, organize, visualize, and remove metrics.
  • How configurations and collected data are saved and exported.
  • How sampling, history, units, provenance, unavailable data, and invalid configurations are presented.
  • Which additional representative metrics should validate the design alongside Zernike coefficients.
  • How the terminology and interfaces remain distinct from profiling metrics.

Follow-up implementation issues may be created once the specification exposes independently deliverable pieces.

Acceptance criteria

  • A reviewed product specification describes the intended workflows and initial delivery scope.
  • The agreed workflow supports whole-output and region-based metrics.
  • Metrics produced by processing nodes can be managed and explored through the same user-facing model.
  • Users can manage and visualize multiple metrics from the GUI.
  • Metric configurations can be saved and restored.
  • Collected metric data can be exported with sufficient metadata to remain meaningful.
  • Existing Zernike coefficients and at least two additional, meaningfully different metric use cases validate the
  • The design explicitly preserves a path toward arbitrary user-composed metrics.
  • Analytical metrics remain clearly separated from runtime and profiling metrics.
  • Relevant behavior is documented and covered by appropriate tests.

Out of scope

  • Choosing the concrete architecture before the discovery work.
  • Requiring every possible metric or visualization in the first delivery.
  • Reworking runtime or profiling metrics.
  • Requiring a complete arbitrary-metric editor in the initial iteration.

Interface and testing note

The issue deliberately defines no public APIs, schemas, or widget structure. Those decisions belong to its discovery
phase. Validation should be based on the representative workflows and acceptance criteria above.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions