sysid: independent + cross-model + external-hardware validation for the actuator recipes#2
Closed
machinavitalis wants to merge 4 commits into
Closed
sysid: independent + cross-model + external-hardware validation for the actuator recipes#2machinavitalis wants to merge 4 commits into
machinavitalis wants to merge 4 commits into
Conversation
The Servo/BLDC/SeriesElastic LM fitters were only checked against telemetry generated by the same fixed-step RK4 they use, so a shared discretization or sign error could pass silently. Add frozen reference traces produced by an independent integrator (SciPy Radau) over a standalone transcription of each device ODE, plus a recovery test that fits them through the recipe. SciPy is only needed to regenerate the traces, not to run the test. Also document the remaining sysid gaps in KNOWN_GAPS.md (external-ground-truth validation, transmission backlash, the BLDC R/L/Ke electrical split), which were previously only noted in source comments.
The Radau oracle rules out a shared integrator but still transcribes jaxterity's own device ODEs, so a physics model-form error copied into both the fitter and the transcription would survive. Close that for the series-elastic actuator with a cross-model oracle: ground truth built from jaxonomy's acausal rotational primitives (Inertia/Spring/Damper/TorqueSource), not from _sea_rhs at all. A sign or coupling error in jaxterity's SEA equations now fails to recover even if copied identically into the transcription. BLDC/Servo cross-model oracles (they embed controllers) and truly external-to-stack validation (third-party log / real bench) remain tracked in KNOWN_GAPS.md.
BLDC gets a genuine independent-physics oracle: jaxonomy's acausal IdealMotor (its own R/L/back-EMF/torque equations) driven open-loop, with the current-loop P controller folded exactly into an effective winding resistance (R+Kp_i) so no causal feedback crosses the acausal boundary (which otherwise recurses). Recovers Kt/J to ~0.3%. Servo gets an independent-implementation oracle: its nonlinear saturation feedback can't be a passive acausal network and jaxonomy's acausal sensor outputs can't sit in a causal feedback loop, so it is a jaxonomy causal block diagram (Integrator/Gain/Adder/tanh) on the BDF solver — a distinct engine but a transcription of the same equations, labeled honestly as the weaker level. generate_jaxonomy.py now emits all three oracles; two new nightly recovery tests. Docs (KNOWN_GAPS/CHANGELOG/README) updated; the only remaining sysid validation gap is ground truth external to the jax*y stack.
Adds the first sysid check whose ground truth is a real physical rig outside the jax*y stack: the EMPS positioning-rig benchmark (Janot/Gautier/Brunot 2019). The DC-motor recipe (V = Ks*sgn(v) + Kv*v + Ka*a) matches the rig's F = M*a + Fv*v + Fc*sgn(v) form; on real 1 kHz data it recovers mass and viscous/Coulomb friction to within 0.2% of the paper's own IDIM-LS identification and predicts a held-out validation trace (~13.6% RMS). This fills the one recipe with no in-stack oracle. Mode B (fetched, not vendored): the data has an unclear license, so fetch_emps.py downloads it on demand (sha256-pinned) into a gitignored dir and the test is marked external_data — skips without the data, excluded from check.sh/CI. Numbers recorded in RESULTS.md. Remaining gap: the BLDC/SEA/Servo LM recipes have no open real dataset and still need our own bench capture (tracked in KNOWN_GAPS).
Owner
Author
|
Closing: this work now lives on local main and will be pushed only on an explicit publish instruction. Removing the public branch/PR for now — no history is being carried publicly. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The
sysid.actuatorLM recipes (Servo / BLDC / SeriesElastic) were only ever checked by round-tripping telemetry generated with the same fixed-step RK4 the fitter uses, so a shared discretization or sign error passed silently — the fit recovered the bug, not the physics. This PR builds a validation ladder of increasing independence, ending in real hardware.The ladder
Radau(a different integrator than the fitter's RK4), applied to a standalone transcription of each device ODE. Rules out a discretization/sign bug shared between the fitter and its own data generator.IdealMotor(its own R/L/back-EMF/torque law, with the current-loop P folded into an effective resistance so no feedback crosses the acausal boundary). A sign/coupling error in jaxterity's own equations fails to recover here even if copied into the transcription.Notes for review
test/sysid/test_actuator_recipe_independent.py) load frozen.npzoracles; SciPy/jaxonomy are only needed to regenerate them (data/independent/generate*.py), not to run the tests.fetch_emps.pydownloads it on demand (sha256-pinned) into a gitignored dir; the test is markedexternal_data— skips without the data, excluded fromscripts/check.shand CI. Numbers recorded intest/sysid/external/RESULTS.md.KNOWN_GAPS.mdnow tracks the remaining sysid gaps explicitly (backlash, BLDC R/L/Ke electrical split, and external validation for the BLDC/SEA/Servo LM recipes — which have no open real dataset and need a bench capture).🤖 Generated with Claude Code