-
Notifications
You must be signed in to change notification settings - Fork 0
feat(tools): obs_inspect R-IEKF vs standard yaw-leak comparison #426
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -98,3 +98,34 @@ TEST_CASE("observability inspector: perturbing the window leaks yaw but not tran | |
| REQUIRE(yaw_small > 1e-6); | ||
| REQUIRE(yaw_big > yaw_small); | ||
| } | ||
|
|
||
| TEST_CASE("observability inspector: the R-IEKF parameterization does not leak yaw under perturbation", | ||
| "[tools][obs_inspect]") { | ||
| const ms::CameraExtrinsics<T> ext{}; // identity extrinsic | ||
| const Vec3 g{{0.0, 0.0, 1.0}}; | ||
| const Vec3 p_f{{0.2, -0.1, 6.0}}; | ||
| const auto truth = window(5); | ||
| const auto Ninv = bt::obs_build_N_invariant<T>(truth, p_f, g); | ||
|
|
||
| // Consistent point: the shipped invariant Jacobian annihilates the invariant gauge. | ||
| const auto [tr0, yaw0] = bt::obs_leak<T>(bt::obs_build_H_invariant<T>(truth, p_f, ext), Ninv); | ||
| REQUIRE_THAT(tr0, WithinAbs(0.0, 1e-9)); | ||
| REQUIRE_THAT(yaw0, WithinAbs(0.0, 1e-9)); | ||
|
|
||
| // Same clone-window perturbation that makes the STANDARD yaw leak (previous | ||
| // test). The invariant gauge directions are constants (rho = e_k, phi = g), so | ||
| // the leak must STAY ~0 — yaw observable by construction. This is the fix. | ||
| std::mt19937_64 rng(0x0B5E11ull); | ||
|
Comment on lines
+115
to
+118
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Line 117 wording inverts the gauge claim. “yaw observable by construction” conflicts with the invariant null-space argument being tested here; this path is about preserving the yaw unobservable gauge (leak stays ~0). Please reword to avoid future confusion. 🤖 Prompt for AI Agents |
||
| auto e = truth; | ||
| for (auto& c : e) { | ||
| c.R = | ||
| c.R * | ||
| SO3::exp(Vec3{{0.05 * bt::obs_urand<T>(rng), 0.05 * bt::obs_urand<T>(rng), 0.05 * bt::obs_urand<T>(rng)}}); | ||
| c.p = Vec3{{c.p[0] + 0.05 * bt::obs_urand<T>(rng), | ||
| c.p[1] + 0.05 * bt::obs_urand<T>(rng), | ||
| c.p[2] + 0.05 * bt::obs_urand<T>(rng)}}; | ||
| } | ||
| const auto [tr_p, yaw_p] = bt::obs_leak<T>(bt::obs_build_H_invariant<T>(e, p_f, ext), Ninv); | ||
| REQUIRE_THAT(tr_p, WithinAbs(0.0, 1e-9)); | ||
| REQUIRE_THAT(yaw_p, WithinAbs(0.0, 1e-9)); // flat — unlike the standard filter | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -355,12 +355,17 @@ per-clone σ from P, N at the unperturbed gauge). | |
| ``` | ||
|
|
||
| Output `observability.jsonl`: per-update window size, mean clone σ, and the | ||
| consistent/real yaw & translation leaks. On V2_03 the consistent leak is ~1e-15 | ||
| (the shipped Jacobian is correct), the translation leak is **structurally 0** | ||
| (the ±Hf cancellation protects it), and the **yaw leak is nonzero on 100% of | ||
| updates and grows with the clone window's attitude drift** — localizing the #212 | ||
| over-confidence to the yaw gauge on real data. The fix is the right-invariant | ||
| (R-IEKF) parameterization, whose gauge directions are estimate-independent. | ||
| consistent/real/invariant yaw & translation leaks. On V2_03 the consistent leak is | ||
| ~1e-15 (the shipped Jacobian is correct), the translation leak is **structurally 0** | ||
| (the ±Hf cancellation protects it), and the **standard yaw leak is nonzero on 100% | ||
| of updates and grows with the clone window's attitude drift** — localizing the #212 | ||
| over-confidence to the yaw gauge on real data. | ||
|
|
||
| It also measures the **right-invariant (R-IEKF) leak on the same perturbed clone | ||
| windows** (driving the shipped `build_invariant_measurement`): because the | ||
| invariant gauge directions are estimate-independent constants (ρ = e_k, φ = ĝ), the | ||
| invariant yaw leak stays ~0 where the standard one rises — the real-data evidence | ||
| that R-IEKF is observable-by-construction and is the fix the diagnosis points to. | ||
|
Comment on lines
+358
to
+368
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. README overstates invariant outputs and uses inverted observability wording. At Line 358, docs currently imply invariant yaw and translation leaks are reported, but the implementation context shows invariant yaw accumulation/reporting only. At Line 368, “observable-by-construction” reverses the gauge claim in this section. Proposed patch-consistent/real/invariant yaw & translation leaks. On V2_03 the consistent leak is
+consistent/real yaw & translation leaks, plus invariant yaw leak. On V2_03 the consistent leak is
@@
-invariant yaw leak stays ~0 where the standard one rises — the real-data evidence
-that R-IEKF is observable-by-construction and is the fix the diagnosis points to.
+invariant yaw leak stays ~0 where the standard one rises — the real-data evidence
+that R-IEKF preserves the unobservable gauge by construction and is the fix the diagnosis points to.🤖 Prompt for AI Agents |
||
|
|
||
| ## Layout | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -28,6 +28,11 @@ | |
| // uncertainty the filter actually reports on this window: given that | ||
| // uncertainty, how much yaw/translation information does the shipped H | ||
| // spuriously inject? | ||
| // * R-IEKF leak -- the SAME perturbed windows run through the shipped | ||
| // right-invariant Jacobian (build_invariant_measurement) + the invariant gauge, | ||
| // whose clone directions are estimate-independent constants. On V2_03 this is | ||
| // ~1e-16 where the standard leak is ~0.4 -- the candidate fix, validated on | ||
| // real data. | ||
| // | ||
| // Output: per-update JSONL (window size, mean clone sigma, consistent/real yaw & | ||
| // translation leak, NIS) + a summary localizing the leak to yaw vs translation on | ||
|
|
@@ -203,6 +208,7 @@ int main(int argc, char** argv) { | |
| std::uint64_t n_updates = 0, n_measured = 0; | ||
| double sum_cons_yaw = 0.0, sum_cons_tr = 0.0; | ||
| double sum_real_yaw = 0.0, sum_real_tr = 0.0, max_real_yaw = 0.0; | ||
| double sum_inv_yaw = 0.0, max_inv_yaw = 0.0; // R-IEKF leak at the SAME perturbations | ||
|
|
||
| est.backend().set_update_observer([&](const ms::State<T>& s, | ||
| const ms::FeatureTrack<T>& track, | ||
|
|
@@ -242,8 +248,12 @@ int main(int argc, char** argv) { | |
| const auto [cons_tr, cons_yaw] = bt::obs_leak<T>(bt::obs_build_H<T>(updater, cl, p_f), N); | ||
|
|
||
| // (2) Real leak: perturb the clone window by the filter's OWN per-clone | ||
| // sigma, N at the unperturbed gauge, averaged over seeded draws. | ||
| double dyaw = 0.0, dtr = 0.0; | ||
| // sigma, N at the unperturbed gauge, averaged over seeded draws. For each | ||
| // perturbed window also measure the RIGHT-INVARIANT (R-IEKF) leak — the | ||
| // candidate fix — on the SAME geometry: its gauge directions are constants, | ||
| // so it should stay ~0 where the standard one leaks. | ||
| const ms::DynMat<T> Ninv = bt::obs_build_N_invariant<T>(cl, p_f, g); | ||
| double dyaw = 0.0, dtr = 0.0, dyaw_inv = 0.0; | ||
| std::mt19937_64 rng(0x0B5E11ull ^ (n_updates * 0x9E3779B97F4A7C15ull)); | ||
| for (int d = 0; d < args.draws; ++d) { | ||
| std::vector<ObsPose<T>> pe = cl; | ||
|
|
@@ -258,9 +268,12 @@ int main(int argc, char** argv) { | |
| const auto [tr, yaw] = bt::obs_leak<T>(bt::obs_build_H<T>(updater, pe, p_f), N); | ||
| dyaw += static_cast<double>(yaw); | ||
| dtr += static_cast<double>(tr); | ||
| const auto inv = bt::obs_leak<T>(bt::obs_build_H_invariant<T>(pe, p_f, cal.extrinsics), Ninv); | ||
| dyaw_inv += static_cast<double>(inv.second); | ||
| } | ||
| dyaw /= args.draws; | ||
| dtr /= args.draws; | ||
| dyaw_inv /= args.draws; | ||
|
Comment on lines
+271
to
+276
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧹 Nitpick | 🔵 Trivial | 💤 Low value Invariant translation leak discarded — intentional? Line 272 accumulates only 🤖 Prompt for AI Agents |
||
|
|
||
| T mean_sth = T{0}, mean_sp = T{0}; | ||
| for (std::size_t i = 0; i < m; ++i) { | ||
|
|
@@ -275,8 +288,11 @@ int main(int argc, char** argv) { | |
| sum_cons_tr += static_cast<double>(cons_tr); | ||
| sum_real_yaw += dyaw; | ||
| sum_real_tr += dtr; | ||
| sum_inv_yaw += dyaw_inv; | ||
| if (dyaw > max_real_yaw) | ||
| max_real_yaw = dyaw; | ||
| if (dyaw_inv > max_inv_yaw) | ||
| max_inv_yaw = dyaw_inv; | ||
|
|
||
| json j{{"index", n_updates - 1}, | ||
| {"t", frame_t}, | ||
|
|
@@ -288,7 +304,8 @@ int main(int argc, char** argv) { | |
| {"leak_yaw_consistent", static_cast<double>(cons_yaw)}, | ||
| {"leak_trans_consistent", static_cast<double>(cons_tr)}, | ||
| {"leak_yaw_real", dyaw}, | ||
| {"leak_trans_real", dtr}}; | ||
| {"leak_trans_real", dtr}, | ||
| {"leak_yaw_invariant", dyaw_inv}}; | ||
| os << j.dump() << '\n'; | ||
| }); | ||
|
|
||
|
|
@@ -317,16 +334,18 @@ int main(int argc, char** argv) { | |
| os.flush(); | ||
|
|
||
| const double nm = n_measured ? static_cast<double>(n_measured) : 1.0; | ||
| const double ry = sum_real_yaw / nm, rt = sum_real_tr / nm; | ||
| const double ry = sum_real_yaw / nm, rt = sum_real_tr / nm, iy = sum_inv_yaw / nm; | ||
| std::cout << "obs_inspect: processed " << processed << " frames, " << n_updates << " updates (" << n_measured | ||
| << " measured)\n" | ||
| << " consistent leak (sanity, want ~0): yaw " << sum_cons_yaw / nm << " trans " << sum_cons_tr / nm | ||
| << " consistent leak (sanity, want ~0): yaw " << sum_cons_yaw / nm << " trans " << sum_cons_tr / nm | ||
| << "\n" | ||
| << " real leak @ filter's own clone sigma: yaw " << ry << " trans " << rt << "\n" | ||
| << " max real yaw leak: " << max_real_yaw << "\n" | ||
| << " STANDARD leak @ filter's own clone sigma: yaw " << ry << " trans " << rt << " (max yaw " | ||
| << max_real_yaw << ")\n" | ||
| << " R-IEKF leak @ the SAME perturbations: yaw " << iy << " (max yaw " << max_inv_yaw << ")\n" | ||
| << " -> " | ||
| << (ry > 4.0 * rt + 1e-12 ? "YAW dominates: over-confidence enters the yaw gauge" | ||
| : "no clear yaw dominance") | ||
| << (ry > 4.0 * rt + 1e-12 ? "YAW dominates the standard filter; " : "no clear standard yaw dominance; ") | ||
| << (iy < ry / 10.0 + 1e-12 ? "R-IEKF flattens it (observable by construction)" | ||
| : "R-IEKF does NOT flatten it") | ||
| << "\n records: " << args.out << "/observability.jsonl\n"; | ||
| return 0; | ||
| } | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Prevent a vacuous pass when invariant Jacobian construction degenerates.
At Lines 111 and 128, the test only checks
obs_leak(...) == ~0. Ifobs_build_H_invarianthits its degenerate path and returns zeros, these assertions pass without validating the invariant property. Add a non-degeneracy assertion onH_invariantbefore leak checks.Proposed patch
Also applies to: 128-130
🤖 Prompt for AI Agents