Skip to content

tools: MAVExplorer: add EKF3 variance, innovation, and sensor consistency graphs - #1708

Open
asikarastallion wants to merge 2 commits into
ArduPilot:masterfrom
asikarastallion:add-ekf-graphs
Open

tools: MAVExplorer: add EKF3 variance, innovation, and sensor consistency graphs#1708
asikarastallion wants to merge 2 commits into
ArduPilot:masterfrom
asikarastallion:add-ekf-graphs

Conversation

@asikarastallion

@asikarastallion asikarastallion commented Jul 23, 2026

Copy link
Copy Markdown

Adds comprehensive EKF and multi-sensor diagnostic preset graphs to mavgraphs.xml.

Closes #1595

Motivation

Evaluating EKF health, lane switching, and pre-filter sensor inconsistencies
currently requires manual plotting of multiple XKF*, NKF*, and raw sensor
messages. These presets allow immediate, one-click analysis of filter health
and hardware consistency directly from the MAVExplorer menu, and directly
address the gap reported in #1595 (no monitoring tool for EKF3 affinity and
lane switching).

Changes

  • EKF3/Variance: Plots normalized innovation variances (SV, SP, SH,
    SM, SVT). A 1.0 threshold line is included
    (1.0+0*XKF4[0].*) — this is not an arbitrary reference: it is
    ArduPilot's actual innovation-rejection gate (see "Verification" below).
  • EKF3/Innovations: Plots raw innovation (residual) values for Velocity,
    Position, Mag, and Yaw.
  • EKF3/Status: Plots solution status flags (SS) and estimated attitude
    errors (errRP).
  • Sensors: Adds pre-EKF hardware consistency checks, to help distinguish
    a genuine EKF/algorithm problem from an underlying sensor problem when a
    Variance or Innovation graph above shows something unhealthy:
    • IMU Accel/Gyro deltas between core 0 and 1.
    • GPS Position offsets (scaled by 1.0e7) and quality metrics (HDop,
      NSats).
    • Compass magnetic field magnitude consistency across instances.

Technical Details

  • Supports both modern EKF3 (XKF3/XKF4) and legacy EKF2 (NKF3/NKF4)
    log messages within the same views, so the graphs remain useful across
    older logs as well as current ones.
  • Evaluates multiple EKF lanes and sensor instances ([0], [1], [2])
    to support redundant hardware setups (dual/triple IMU, GPS, compass).
  • Uses degrees() for radian-to-degree conversions on Gyro and errRP
    data for readability.

Verification: the 1.0 threshold is ArduPilot's real rejection gate

Traced through AP_NavEKF3 source to confirm this isn't an arbitrary
reference line:

  • AP_NavEKF3_Outputs.cpp::getVariances() returns velVar = sqrtF(velTestRatio)
    (and the equivalent for pos/hgt/mag/tas) — this is exactly what gets logged
    as XKF4.SV/SP/SH/SM/SVT.
  • The filter's own health checks use the identical threshold, e.g.
    AP_NavEKF3_PosVelFusion.cpp: if (velTestRatio < 1.0), and
    AP_NavEKF3_MagFusion.cpp: magHealth = (magTestRatio[0] < 1.0f && ...).

So SV/SP/SH/SM/SVT crossing 1.0 in these graphs is the same
condition ArduPilot itself uses internally to start rejecting that
measurement — the graph is reading the filter's actual decision boundary,
not an approximation of it.

Testing

Verified all 17 graph expressions evaluate without errors (no NameError /
eval exceptions) against a real dual-IMU, dual-core EKF3 log
(178,979 messages, ~35 min flight):

  • All EKF3 (XKF3/XKF4) expressions returned full data (6,088 samples
    per field, both cores populated).
  • All IMU-consistency and Compass-consistency expressions returned full
    data (146,103 / 2,435 samples).
  • Rendered EKF3/Variance/Velocity to PNG to visually confirm the 1.0
    threshold line sits correctly against real SV values, which stayed in
    the 0–0.03 range with one genuine spike to 0.28 — axis scaling and
    threshold placement both looked correct.
  • Scanned 400 real dataflash logs on hand (home-directory-wide, filtered
    by dataflash magic bytes): the large majority contained XKF3/XKF4
    (EKF3), a small number of older logs contained NKF3/NKF4 (EKF2, see
    below), none contained a second GPS instance.

EKF2 (NKF3/NKF4) was additionally verified against two logs:

  • A real legacy log (ArduPlane V3.8.2-dev, ~2017/2018) found via a
    home-directory-wide scan.
  • A SITL-generated modern EKF2 log (EK2_ENABLE=1, EK3_ENABLE=0,
    AHRS_EKF_TYPE=2), to check current-format behavior directly.

Result: NKF3[0]/NKF4[0] bracket-instance syntax returns full, correct
data on modern EKF2 logs (2,477 samples per field, both cores populated,
PNG-rendered and visually confirmed) — see the coverage limitation below
for the one gap this surfaced.

Known test-coverage limitations

  • NKF3[0]/NKF4[0] instance syntax verified against two EKF2 logs:
    a real legacy log (ArduPlane V3.8.2-dev, ~2017/2018) and a SITL-generated
    modern log (EK2_ENABLE=1/EK3_ENABLE=0). Result: the bracket-instance
    syntax works correctly on modern EKF2 logs, where NKF3/NKF4 carry a
    core/lane field (C) — same as XKF3/XKF4. On pre-core-field EKF2 logs
    (roughly pre-2018, before multi-lane EKF2 existed), NKF3[0]/NKF4[0]
    silently returns empty data rather than raising an error, because there
    is no C field to index into. The plain (bracket-free) form — NKF3.IVN,
    NKF4.SV, etc. — does return data on these older logs.
    This is a narrow edge case (affects only genuinely old EKF2-only logs,
    not XKF3/XKF4, which have always had a core field), but worth knowing:
    a blank EKF2 graph on a very old log means "no core field," not
    "everything's fine."
  • Sensors/GPS/Position-Offset and Sensors/GPS/Quality render empty
    on any single-GPS setup
    (i.e. most vehicles) — this is expected
    behavior (no GPS[1] data to plot), not a bug, but worth knowing before
    assuming the graph is broken on a normal log.
  • These are preset diagnostic views, not automated lane-switch detectors —
    a lane switch event itself is not highlighted/annotated on the timeline;
    the graphs only make it visually apparent by eye.

On the EKF3 vs Sensors split

These are kept in one PR because the Sensors graphs exist specifically
to help answer why a Variance/Innovations graph shows a problem
(e.g. a Mag Variance spike traced back to a Compass Field-Consistency
mismatch). Happy to split into two PRs if maintainers would prefer
independent review — flagging as an option, not proposing it.

AI assistance disclosure: This contribution was developed with the assistance of Claude (Anthropic). I reviewed and tested the results as described above and understand the change.

Copilot AI review requested due to automatic review settings July 23, 2026 22:00

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

No monitoring tool for EKF3 Affinity and Lane Switching available

2 participants