Skip to content

data-flow.md

Latest commit

 

History

History
73 lines (57 loc) · 3.61 KB

data-flow.md

File metadata and controls

73 lines (57 loc) · 3.61 KB
Raw

Architecture: Runtime Data Flow

How data moves through the two subsystems at run time. Static structure is in module-structure.md; this document traces the dynamic flows.

1. Learner flow — one online round

The learner's cycle is a tight explore→observe→update loop. Each observe consumes one cost signal and (once the window has two points) advances $c$.

FPTL learning round

Step by step (source src/learner.rs, design ../design/adaptive-learner.md):

  1. Explore. perturb_c / explore_c draws Laplace noise $\eta \sim \mathrm{Laplace}(0, 1/\epsilon)$, forms $\tilde c = \mathrm{clamp}(c + \eta)$, and stores $\tilde c$ as last_perturbed_c (the bug fix — attributing the next cost to $\tilde c$, not $c$).
  2. Score. The caller computes a cost at $\tilde c$ — e.g. via predict(query, target) (which internally builds MsmConfig::new(c̃) and calls distance), or via an external objective.
  3. Observe. observe(cost) pushes $(\tilde c, cost)$ into the bounded window, updates optional statistics, and — if the window has $\ge 2$ points — estimates the finite-difference gradient and takes a descent step $c \leftarrow \mathrm{clamp}(c - \epsilon, g)$.

No allocation happens on the hot path beyond the fixed-capacity window (a benchmark round is $\approx 56$ ns — see ../scientific/benchmarks.md).

2. WFST flow — building and searching the lattice

The cached engine builds a lazily-expanded lattice and answers distance via a tropical shortest-distance search.

WFST runtime data flow

  1. Build. MsmWfstBuilder::build constructs MsmWfst::new, which computes the data-driven quantization range, registers the initial state $(id, 0, 0)$ per target, and seeds an MsmWeight.
  2. Expand (lazy). A shortest-distance search calls expand / transitions_lazy on frontier states. For each state the engine decodes $(series, q, t)$, extends the MsmWeight via with_move/with_merge/ with_split, prunes successors whose accumulated cost exceeds max_cost, resolves successor ids in the registry, and caches the result (subject to CachePolicy).
  3. Answer. The tropical shortest path from the start to the final state $(m, n)$ equals MsmConfig::distance; the final state's final_weight holds the accumulated cost (../design/msm-wfst-embedding.md, proof ../formal/msm-wfst-embedding.md).

The stateless engine (MsmStateSource) flows differently: there is no build / mutation phase — compute_state(id) is a pure function of the positional id, and a generic lazy-composition driver drives the search. Edge weights are local and the driver's shortest-distance pass accumulates them.

3. How the subsystems combine

The learner and the WFST are independent, but compose naturally: the learner adapts $c$; a MsmConfig::new(c) built from the learned $c$ parameterises an MsmWfst for search. A typical pipeline —

observed costs ──▶ AdaptiveMsm.observe ──▶ learned c ──▶ MsmConfig::new(c)
                                                              │
                                                              ▼
                                    MsmWfstBuilder ─▶ MsmWfst ─▶ shortest distance

— is spelled out with runnable code in ../usage/cookbook.md.