diff --git a/changelog.d/capabilities-spatial-units.fixed.md b/changelog.d/capabilities-spatial-units.fixed.md new file mode 100644 index 000000000..fd227e578 --- /dev/null +++ b/changelog.d/capabilities-spatial-units.fixed.md @@ -0,0 +1,4 @@ +`solprop.capabilities.output` now states a spatial file's field in the run's stored unit +(V m radially, V/m on a Cartesian or line grid, sqrt(W/m) per y frequency on a planar +one) instead of sqrt(W), and no longer states a modal power, energy or spectral density +for it, whose readers refuse a spatial file (#425). diff --git a/docs/dev/gui/design.md b/docs/dev/gui/design.md index 03dfbba73..689fae770 100644 --- a/docs/dev/gui/design.md +++ b/docs/dev/gui/design.md @@ -1378,7 +1378,7 @@ Propagate step into its own mode (12.7). | M2 Explorer (in progress, #422) | The waveguide explorer, standalone and in the medium page; pinned pressure; multimode capillary. Built as: a Dispersion view among the views (n_eff − 1, group index, β₂ or D, β₃, loss, A_eff and γ against wavelength, frequency or photon energy; modes on and off; ZDW and pump marked; a sweep of one keyword overlaid; a table at the pump; CSV), fed by a kernel `modes` request over `Recipe.modes` (P4) and brought forward on the Geometry and Medium steps; the start panel's "Waveguide explorer" opens a four-mode helium capillary; pins on pressure, radius, energy and peak power edited in their rows ("pinned": target, search range, root, solved value, every root on request through `Recipe.pin_roots`), a pin and a value of one keyword replacing each other in one undo step; the multimode capillary card; the launch table, labelled by the mode preview; the explorer at a position along the guide (z, for tapers and gradients); the design tool (Tools menu and the explorer's "Design…"), which solves the pressure or radius and the energy (or peak power) from a ZDW or RDW target and a soliton order with solprop's two pin stages on a copy of the setup and writes them as literals in one undo step; and the ZDW chart, the pressure `tools.pressure_zdw` gives each zero-dispersion wavelength against the radius for the setup's fill, the 2-D map of 7.4 drawn as its contours; each mode's phase mismatch to the launch soliton as a quantity and its phase-matched RDW marked (from the mode preview, P5, #445); figures exported as PNG, SVG or PDF, drawn by Matplotlib in the kernel from the view's own description. | | M3 Guided routes (built locally) | Antiresonant, full-transverse capillary, GNLSE (Taylor, step-index, tables), rectangular, slab. Built as: every keyword that names a registered type is edited in that type's own rows (the kernel's `describe` carries the metadata of every type a route can name, P2 #409), a list of them (pulses, a waveguide's modes) as a column of such editors; cards for the antiresonant fibre, the hollow waveguide of supplied modes and the GNLSE fibre, each starting from a solprop example (`hollow_rectangular`, #402, is new) and recognised by its preset and marked keywords; the GNLSE's Taylor coefficients as one row per order with its units; a slab set's energy and peak power per metre of y (#404); summary numbers with SI prefixes; the explorer on the GNLSE's one specified mode (P5), without the quantities a Taylor fibre does not have. Not built: step-index and tabulated GNLSE fibres (`IndexTable` and mode objects are not registered types), metal claddings (#403). | | M4 Results depth (in progress, local) | All views of 11.3 under result capabilities, view specifications and export through `solprop.plotting`, richer compare. Built so far: a Plane view (a slider over the field planes, mode selection; the spectral density, power and spectral phase from `solprop.processing`, the phase drawn only where the spectrum is within 30 dB of its peak; the spectrogram drawn by `solprop.plotting.spectrogram`; the best Taylor-phase compression by `compress_taylor` over the power with the transform limit) and a Statistics view (every recorded statistic against z with the catalogue's summary and unit, a column per mode; the step size with rejected steps); figures of `prop_2d`, `spec_1d`, `time_1d`, `spectrogram` and `plot_stats` written by solprop. Still to do: modal energy views, the spatial views (M5), the other compressors, coherence, data export (CSV/NPZ/HDF5, FieldPlane), layouts. | -| M5 Free space (in progress, local) | Radial, Cartesian, line; spatial views. Built so far: the Cartesian free-space and planar-hybrid cards, starting from `cartesian_beam` and `planar_hybrid`, their `[build]` tables edited in the registered types' rows (M3); on a spatial file the Plane view draws the plane's fluence (`node_fluence` on the transverse grid rebuilt from the run's recipe, a map in x and y or a profile in r or x), labels the power and spectrum as the coordinate quantities `capabilities.output` says they are, and disables compression with solprop's reason. Still to do: the radial card (needs a radial example, #405), far field and beam-radius views beyond the recorded statistics, spatial maps along z (r-t, r-lambda), physical spatial power (#406). | +| M5 Free space (in progress, local) | Radial, Cartesian, line; spatial views. Built so far: the Cartesian free-space and planar-hybrid cards, starting from `cartesian_beam` and `planar_hybrid`, their `[build]` tables edited in the registered types' rows (M3); on a spatial file the Plane view draws the plane's fluence (`node_fluence` on the transverse grid rebuilt from the run's recipe, a map in x and y or a profile in r or x), labels the power and spectrum as coordinate (per-column) quantities, and disables compression with solprop's reason. Still to do: the radial card (needs a radial example, #405), far field and beam-radius views beyond the recorded statistics, spatial maps along z (r-t, r-lambda), physical spatial power (#406). | | M6 Scans (built; moved to its own stage after W6) | Scan setup, runs, scan browser. Since W6: a Scan stage beside Setup, Run and Analyse holds the `[scan]` editor, the scan's progress point by point, and its results, a recorded quantity over the first variable with a line for each value of the second (failed points marked, quantities the scan does not move left out). *Scan this…* on a sheet row opens it with that keyword as a variable. A run from the sheet leaves `[scan]` out and runs the setup once; *Run scan* runs the scan; a `scan.h5` opened from Home comes to this stage. After the first hand review: a progress bar per variable, outermost first, with the value it is at, and one for the whole scan; what is also recorded at each point, as solprop reductions the front end declares (energy and peak power within a band, and over one variable the spectrum or the pulse at a distance, held on axes fixed by the recipe so the points line up and drawn as a map); and a point of a profile given as a list (`pressure[0]` of a gradient) offered as a variable, its values read back in their unit. Not built since W6 either: a map over two variables, a third variable's slider, a cost estimate, and opening a point's run. Built first in the step window as: a Scan section on the Propagate step edits the recipe's `[scan]` table (its name; each variable a path, offered from the route's keywords or typed, with a list of values or a range of start, stop, number and spacing; the number of points), as one undo step; Run runs a recipe with a scan through `Recipe.scan` and `run_scan` in the run's worker, a `ScanReporter` streaming each point's start (its values) and end, Stop leaving the points not started; the scan browser draws any result the points recorded (`ScanOutput.as_grid`, each point's value at its last plane summed over its modes) as a line over one variable or a map over two, failed points marked. Every numeric field's context menu has *Scan this…*, which makes it a variable from its value. Not built: nested fields' scan toggle, reductions chosen in the editor, opening a point's run from the browser (fields are not retained by default), executors other than `Local`. In its place, a right click on the results offers *Open this point as a setup*: the recipe the scan's file records, without `[scan]` and with the nearest point's values (the third variable on at its first), opened as a new sheet to run or change. | | M7 Service out of process (first part, local; replaced by D21) | Option C, re-attach, then SSH-tunnel remote. Built as: `service/server.py`, a process of its own that owns one kernel and the run workers and speaks the same requests, replies, runs and events in `Envelope` messages over a Unix socket (a named pipe on Windows) guarded by a random token in a private state file; `service/remote.py`, a window's connection that finds the service or starts it detached; the bridge uses it with `SOLPROP_GUI_SERVICE=1` (opt-in while it is new; option B stays the default), a window that closes leaves its run going, and a new window lists the service's runs at start and re-attaches to the latest one going (its rows replayed by the worker; also Run → Re-attach). The service ends itself after ten idle minutes. Not built: several windows controlling one run (the last attached sees its events alongside the starter), the remote service over SSH with Slurm submission (deferred to #480), a service started at login. | | M7 Other machines (second part, #480, built) | The design of 13.4 (D19, D20). Built as: the service moved into solprop as `solprop.service`, with its tests in solprop's `tests/service`; a run's record (`record.py`: `job.json`, `events.bin` as frames of JSON and raw arrays, `status.json`, the stop file), written by any run whose spec asks for it, and always by a run started without a window (`python -m solprop.service.job FOLDER`); launchers (`launch.py`: a detached process through a short-lived starter, or `sbatch` from the machine's template, with `--signal=B:TERM@120`); states read from folders (`runs.py`); the helper (`agent.py`, `python -m solprop.service.agent`, frames on its standard input and output, hosting a kernel for analysis on the machine); the window's connection (`machines.py`: hosts from the SSH configuration, the system `ssh` with a master connection, reconnection with growing pauses, requests that wait for a machine coming up, a version check that lets development builds through with a warning); setting a machine up (`install.py`); a scan run as a job runs its points in a `Pool`. In the window: the machine on the Runs on line, the Machines dialog with Set up, the Runs window, and the bridge sending a remote run's file requests to that machine's kernel (`ui/machines_hub.py`, `ui/machines_dialog.py`, `ui/setup_dialog.py`, `ui/runs_window.py`, `askpass.py`). Checked by hand on a workstation (directly and through its Slurm partition, through a jump host) and a cluster (Slurm): runs followed to their ends, a session killed mid-run and recovered with no row lost or repeated, a stop by the stop file leaving a resumable file, a run and a Pool scan submitted with the window gone and found again from their folders, and solprop 0.2.0 set up from PyPI. A run on this computer keeps its record too (`RunPanel.record_local`), its cost measured on 2026-11-07 against the same run without it, a short capillary saving 100 to 400 planes at 2 to 8 ms a plane (the worst case, since the record costs per plane), seven alternating pairs per case, in a worker over pipes as the window runs it: macOS (Apple M5 Max, a shared machine) −1.0 to +1.3 % with runs spreading 9 to 17 %; Linux (2× Xeon Gold 6240, an exclusive job) +0.5 to +1.7 % with a 1.5 % spread, and +0.2 to +0.3 % on the march alone; Windows Server 2025 (16 Zen 5 vCPUs) −1.8 to +0.6 %, −0.07 % in its steadiest case (1.2 % spread); live rows reached the window at the same pace with and without it. The record writes a row in about 18 µs, on a thread of its own, and writes `job.json` from that thread too, as stamping the process (`ps` on macOS) had delayed the first row by up to 77 ms. Then D21: the in-memory service of the first part and the window's own worker over pipes deleted (`server.py`, `remote.py`, `outbox.py`, `RunClient`, about 2,400 lines with their tests); a continued run sets its earlier record aside (`events-1.bin`, ...); the window follows a run only once its machine has started it; a remote run's data files are copied beside it first (`data_copies`, `MachineHub.call_after_copies`), Resume continues an ended run from the Runs window once solprop there says it can, and a publication figure is drawn as bytes (`result_figure` with `bytes`) and written here, with a copy of the results file fetched or copied. Not built: scans across many nodes (#297); a helper on Windows hosting a kernel (its kernel, started from a process whose standard streams are the frames' pipes, never answered on the CI's Windows runners; a machine is a Linux or macOS server, and on Windows the window reads its own runs with its own kernel). Found on the way: the CI native wheel needs glibc 2.35 and the system FFTW (the PyPI wheels do not), so a development build is set up from wheels or a Python the user names. | diff --git a/docs/guide/numerics/choosing-a-stepper.md b/docs/guide/numerics/choosing-a-stepper.md index 095c3f942..23bc30fd2 100644 --- a/docs/guide/numerics/choosing-a-stepper.md +++ b/docs/guide/numerics/choosing-a-stepper.md @@ -145,7 +145,12 @@ There: - for a dense, multimode, Raman-active run such as a comb-to-continuum capillary, use `Multistep(carrier_frame="scalar")`: the fastest method measured, about half the - default's and IP-DOPRI5's time; + default's and IP-DOPRI5's time, when the observable is the populated band or the + developed spectrum. It loses a weak driven band that is still forming: near the + launch, a comb-to-continuum's blue side and a Raman self-frequency shift's red band + came out a few hundred times less accurate than with the default at matched + total-field error, and the step-size norm does not see it. Where such a band is the + observable, keep the default or check the band against a tighter run; - for a single-mode dense-gas run (an argon blue shift, a soliton self-frequency shift), use `Multistep(formulation="interaction")`; - at a loose tolerance, `RungeKutta()` or `rtol <= 1e-4` with the default. diff --git a/docs/guide/results/output.md b/docs/guide/results/output.md index 95a8f109b..2baf1b57e 100644 --- a/docs/guide/results/output.md +++ b/docs/guide/results/output.md @@ -366,15 +366,19 @@ recipe, leaves it as it is and is recorded in its own execution record. reader may offer: the file's basis (`modal`, `general_modal` or `spatial`); whether its power, energy, spectral density and field are **physical fluxes** (an orthogonal modal basis) or **coordinate quantities** (a general modal basis, whose physical energy is -`physical_energy`, and a spatial grid, per transverse column); which saved planes keep -their field and which only their statistics; which of the basis-dependent accessors and -compression its readers refuse, with the refusal they give, read from the same gates -(a spatial file admits `energy`, whose statistic is the beam's physical energy); in -`refused_readers`, which processing and plotting readers refuse it (on a spatial file -every reader that would read its columns as modes, such as `get_power`); -what a general modal reconstruction needs that the file only identifies (the guide and -the reference tables); whether it records a recipe; and, for a path, its -[resumability](resuming.md#what-a-file-allows). +`physical_energy`), in W, J, J s/rad and √W. A spatial file states only its field, a +coordinate quantity per transverse column in the run's stored unit (V m radially, V/m +on a Cartesian or line grid, √(W/m) per y frequency on a planar one): the readers of +its power, energy and spectral density read the columns as modes and refuse it, and +the beam's flux is a [spatial functional](../free-space/flux.md#using-the-measure). +It also says which saved planes keep their field and which only their statistics; +which of the basis-dependent accessors and compression its readers refuse, with the +refusal they give, read from the same gates (a spatial file admits `energy`, whose +statistic is the beam's physical energy); in `refused_readers`, which processing and +plotting readers refuse it (on a spatial file every reader that would read its +columns as modes, such as `get_power`); what a general modal reconstruction needs +that the file only identifies (the guide and the reference tables); whether it +records a recipe; and, for a path, its [resumability](resuming.md#what-a-file-allows). ### The writer-lock sidecar diff --git a/src/solprop/capabilities.py b/src/solprop/capabilities.py index beea81f6e..b0b379c96 100644 --- a/src/solprop/capabilities.py +++ b/src/solprop/capabilities.py @@ -44,9 +44,11 @@ capability_of, reading_refusal, refusal_for, + spatial_field_unit, withheld_from, ) from solprop.output.hdf5 import open_file +from solprop.output.metadata import RunMetadata from solprop.output.resumable import Resumability, resumability from solprop.output.store import Output from solprop.physdata.gases import Gas, as_gas, is_molecule @@ -112,7 +114,11 @@ class OutputCapabilities: The kind of run, as its record names it. quantities : collections.abc.Mapping :class:`Quantity` for ``power``, ``energy``, ``spectral_density`` and - ``field``. + ``field``. A spatial file has only ``field``, a coordinate quantity per + transverse column in its route's stored unit (V m radially, V/m on a + Cartesian or line grid, sqrt(W/m) per y frequency on a planar one); the + readers of its power, energy and spectral density (``get_power``, + ``energy`` and ``get_spectral_density``) refuse it, in ``refused_readers``. field_planes : tuple of int The saved planes that keep their field. statistics_planes : tuple of int @@ -162,14 +168,18 @@ def admits(self, operation: str) -> bool: return operation not in self.refused -def _quantities(basis: BasisCapability) -> dict[str, Quantity]: +def _quantities(metadata: RunMetadata, basis: BasisCapability) -> dict[str, Quantity]: """Return what each derived quantity is on a basis. - Returns: the quantities: physical on an orthogonal modal basis, coordinate on a general one (whose physical energy is - :func:`solprop.processing.modal.physical_energy`) and on a spatial grid, whose - amplitudes are per transverse column. + :func:`solprop.processing.modal.physical_energy`). On a spatial grid only the + field, per transverse column in its route's stored unit: the readers that + would give the power, energy and spectral density read the columns as modes + and refuse the file (``refused_readers``). """ + if basis == "spatial": + return {"field": Quantity("coordinate", spatial_field_unit(metadata))} kind: QuantityKind = "physical" if basis == "modal" else "coordinate" return { "power": Quantity(kind, "W"), @@ -266,7 +276,7 @@ def _statement(handler: Output, resumable: Resumability | None) -> OutputCapabil return OutputCapabilities( basis, metadata.simulation_kind, - _quantities(basis), + _quantities(metadata, basis), kept, others, refused, diff --git a/src/solprop/driver/plane_view.py b/src/solprop/driver/plane_view.py index 0500c3047..00fd20fb3 100644 --- a/src/solprop/driver/plane_view.py +++ b/src/solprop/driver/plane_view.py @@ -107,9 +107,11 @@ def basis(self) -> BasisCapability: # numpydoc ignore=RT01 def quantities(self) -> dict[str, Quantity]: # numpydoc ignore=RT01 """What the plane's derived quantities are. - Whether its power, energy, density and field are physical fluxes. + Whether its power, energy, density and field are physical fluxes, as + :func:`solprop.capabilities.output` states them; a spatial plane has only + its field. """ - return _quantities(self.basis) + return _quantities(self._metadata, self.basis) def _reader(self) -> MemoryOutput: """Return the plane as a one-plane output, fetching its field once if needed.""" diff --git a/src/solprop/output/capability.py b/src/solprop/output/capability.py index 707f6886b..b919bbc29 100644 --- a/src/solprop/output/capability.py +++ b/src/solprop/output/capability.py @@ -64,6 +64,7 @@ "declared_capability", "reading_refusal", "refusal_for", + "spatial_field_unit", "withheld_accessors", "withheld_from", ] @@ -426,16 +427,44 @@ def reading_refusal( return refusal_for(capability, [measured], withheld=withheld_from(metadata)) -def _column_refusal(metadata: RunMetadata, name: str, recorded: Sequence[str]) -> str: - """Return a spatial file's refusal of a column reader, with its route's advice.""" +def _spatial_route(metadata: RunMetadata) -> str: + """Return a spatial record's route: radial, Cartesian, line or planar.""" kind = metadata.simulation_kind if kind == "free": # A line grid runs as the free transform with one y sample, and is the only # free grid measured per metre of y. per_width = metadata.transverse_measure == "per_unit_width" - route = "line" if per_width else "Cartesian" - else: - route = "planar" if kind == "slab" else kind + return "line" if per_width else "Cartesian" + return "planar" if kind == "slab" else kind + + +_SPATIAL_FIELD_UNITS: Mapping[str, str] = MappingProxyType( + {"radial": "V m", "Cartesian": "V/m", "line": "V/m", "planar": "sqrt(W/m)"} +) +"""The unit of a spatial route's stored field: a radial Hankel spectrum, a Cartesian +or line field, or a planar slab amplitude per y frequency.""" + + +def spatial_field_unit(metadata: RunMetadata) -> str: + """Return the unit of a spatial record's stored field. + + Parameters + ---------- + metadata : solprop.output.metadata.RunMetadata + The record of a spatial run (radial, free or slab). + + Returns + ------- + str + ``"V m"`` for a radial Hankel spectrum, ``"V/m"`` for a Cartesian or line + field, ``"sqrt(W/m)"`` for a planar slab amplitude per y frequency. + """ + return _SPATIAL_FIELD_UNITS[_spatial_route(metadata)] + + +def _column_refusal(metadata: RunMetadata, name: str, recorded: Sequence[str]) -> str: + """Return a spatial file's refusal of a column reader, with its route's advice.""" + route = _spatial_route(metadata) advice = COLUMN_READINGS[name].format( spectrum=_SPECTRUM_ADVICE.get(route, _SPECTRUM_ADVICE["radial"]), slab=" (planar_mode_energy per slab mode)" if route == "planar" else "", diff --git a/src/solprop/stepper/controller.py b/src/solprop/stepper/controller.py index 0bb18071d..14562e728 100644 --- a/src/solprop/stepper/controller.py +++ b/src/solprop/stepper/controller.py @@ -115,8 +115,9 @@ def weighted_norm_scale( .. note:: Provisional. Which weighting actually serves weak observables best -- fixed - per-band weights, per-mode weights or a rolling per-bin scale -- is the first - experiment of ``docs/dev/stepper.md`` section 9 and is not yet settled. + per-band weights, per-mode weights or a rolling per-bin scale -- has not been + measured: it is the open question "Error norms for observables far below the + pump" in ``docs/dev/open-questions.md`` (solprop issue #259). Parameters ---------- diff --git a/tests/processing/test_spatial_readers.py b/tests/processing/test_spatial_readers.py index c39ca7b43..44cdb5303 100644 --- a/tests/processing/test_spatial_readers.py +++ b/tests/processing/test_spatial_readers.py @@ -208,6 +208,7 @@ def test_a_plane_view_of_a_spatial_run_refuses_its_power_and_density(spatial_run OutputError, match=r"^get_spectral_density measures modal amplitudes" ): view.spectral_density() + assert set(view.quantities) == {"field"} assert view.spectrum()[1].shape[0] == output.fields().shape[1] finally: view._close() @@ -270,10 +271,22 @@ def test_the_advice_names_only_what_works_on_the_route(spatial_run): assert on_axis_spectrum(stored, flux).shape == (simulation.grid.n_omega,) +FIELD_UNITS = { + "radial": "V m", + "cartesian": "V/m", + "line": "V/m", + "planar": "sqrt(W/m)", +} +"""The unit each route stores its field in.""" + + def test_the_file_statement_keeps_the_statistic_and_states_the_readers(spatial_run): - _, _, output = spatial_run + route, _, output = spatial_run statement = capabilities.output(output) assert statement.basis == "spatial" + # Only the field, in the stored unit: the modal flux readers refuse the file. + field = capabilities.Quantity("coordinate", FIELD_UNITS[route]) + assert statement.quantities == {"field": field} # The energy statistic is the beam's physical flux, and stays admitted. assert statement.admits("energy") and statement.admits("peak_intensity") assert set(statement.refused_readers) == set(COLUMN_READINGS) diff --git a/tests/test_capabilities.py b/tests/test_capabilities.py index 48e885618..497a24ba9 100644 --- a/tests/test_capabilities.py +++ b/tests/test_capabilities.py @@ -16,16 +16,20 @@ from solprop.recipe import Recipe -def _spatial_output(): - """The example run relabelled as a radial one, as the compression tests do.""" +def _spatial_output(kind="radial", *, per_width=False): + """The example run relabelled as a spatial one, as the compression tests do.""" metadata = example_output().metadata assert metadata is not None + configuration = dict(metadata.configuration) + if per_width: + configuration["transverse_measure"] = "per_unit_width" spatial = MemoryOutput() spatial.open( replace( metadata, - simulation_kind="radial", - spatial={"geometry": "radial", "flux_model": "scalar-axial-admittance"}, + simulation_kind=kind, + spatial={"geometry": kind, "flux_model": "scalar-axial-admittance"}, + configuration=configuration, ) ) return spatial @@ -87,6 +91,39 @@ def test_a_spatial_file_refuses_compression_and_the_column_readers(): assert str(caught.value) == statement.refused_readers[name] +@pytest.mark.parametrize( + ("kind", "per_width", "unit"), + [ + ("radial", False, "V m"), + ("free", False, "V/m"), + ("free", True, "V/m"), + ("slab", True, "sqrt(W/m)"), + ], +) +def test_a_spatial_file_states_its_stored_field_unit_and_no_modal_flux( + kind, per_width, unit +): + statement = capabilities.output(_spatial_output(kind, per_width=per_width)) + assert statement.basis == "spatial" + assert statement.quantities == {"field": capabilities.Quantity("coordinate", unit)} + # The readers of the power, energy and spectral density refuse the file. + for reader in ("get_power", "energy", "get_spectral_density"): + assert reader in statement.refused_readers + + +def test_the_modal_statements_keep_their_units(): + builds = ((example_output, "physical"), (example_general_output, "coordinate")) + for build, kind in builds: + quantities = capabilities.output(build()).quantities + assert {name: one.unit for name, one in quantities.items()} == { + "power": "W", + "energy": "J", + "spectral_density": "J s/rad", + "field": "sqrt(W)", + } + assert {one.kind for one in quantities.values()} == {kind} + + def test_a_file_says_its_planes_its_recipe_and_whether_it_continues(tmp_path): recipe = Recipe.example("soliton") document = recipe.to_dict()