Skip to content

fix(VideoManager): redact stream URLs in diagnostics - #14727

Open
alireza787b wants to merge 9 commits into
mavlink:masterfrom
alireza787b:agent/video-url-log-redaction
Open

fix(VideoManager): redact stream URLs in diagnostics#14727
alireza787b wants to merge 9 commits into
mavlink:masterfrom
alireza787b:agent/video-url-log-redaction

Conversation

@alireza787b

@alireza787b alireza787b commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Description

Several QGC-owned video diagnostics can expose URL credentials or query and fragment secrets. This change centralizes a logging-safe URL representation while preserving the connection details needed for field debugging.

The helper:

  • preserves scheme, host, port, and stream path;
  • removes URL user information;
  • preserves query keys while replacing every value with REDACTED;
  • replaces fragment content with a visible REDACTED marker;
  • keeps scheme-less values such as 5600 and camera.local:5600 useful; and
  • distinguishes empty and genuinely invalid URLs.

The helper is used by VideoSettings, VideoManager, VehicleCameraControl, the GStreamer source factory, and GstVideoReceiver. QGC-generated pipeline DOT files now retain topology, caps, media types, and states while omitting element property values, which can contain source credentials. Bad operator-entered stream URLs are reported as warnings rather than critical failures.

This does not change stream configuration, receiver lifecycle, or video transport behavior.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change
  • Documentation update
  • Refactoring
  • CI/Build changes
  • Other

Testing

  • Tested locally
  • Added/updated unit tests
  • Tested with simulator (not applicable)
  • Tested with hardware (no runtime behavior change)

Local checks on exact head 8e06c53cc545bd43528ddab8d8082580d664b7ee:

  • focused Qt probe covered bare ports, scheme-less host/port input, paths, user information, query values, fragments, empty input, invalid input, and both helper overloads;
  • focused GStreamer probe verified that the selected DOT flags retain graph topology and omit a secret element property;
  • git diff --check.

The same cases are split into five focused slots in a dedicated no-I/O QGCNetworkRedactionTest registered as Unit/Utilities, so QGC's normal CI executes them. GStreamerTest covers the DOT-property boundary. Exact-head QGC CI is complete: Linux x64/arm64 release builds, unit and integration tests, ASan/UBSan, coverage thresholds, Windows, macOS, iOS, Android, Docker, custom-plugin, and CodeQL checks passed. Linux evidence: run 30598293911.

Platforms Tested

  • Linux (focused Qt/GStreamer probes)
  • Windows (QGC exact-head CI build)
  • macOS (QGC exact-head CI build)
  • Android (QGC exact-head CI build)
  • iOS (QGC exact-head CI build)

Flight Stacks Tested

Not applicable; this change does not enter a vehicle command path.

Screenshots

Not applicable; there is no UI change.

Checklist

  • I have read the Contribution Guidelines
  • I have read the Code of Conduct
  • My code follows the project coding standards
  • I have added tests that prove the fix
  • New and existing unit and integration tests pass at the exact head in QGC CI

Related Issues

This is the first focused replacement for draft #13594. The HTTP MJPEG and WebSocket JPEG transports remain separate in #14730 and #14731.


By submitting this pull request, I confirm that my contribution is made under the terms of the project's dual license (Apache 2.0 and GPL v3).

@codecov

codecov Bot commented Jul 28, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 1.00000% with 99 lines in your changes missing coverage. Please review.
✅ Project coverage is 32.64%. Comparing base (f29efd3) to head (8e06c53).
⚠️ Report is 236 commits behind head on master.

Files with missing lines Patch % Lines
...anager/VideoReceiver/GStreamer/GstVideoReceiver.cc 0.00% 62 Missing ⚠️
src/Settings/VideoSettings.cc 6.25% 13 Missing and 2 partials ⚠️
...anager/VideoReceiver/GStreamer/GstSourceFactory.cc 0.00% 1 Missing and 8 partials ⚠️
src/VideoManager/VideoManager.cc 0.00% 8 Missing ⚠️
src/Camera/VehicleCameraControl.cc 0.00% 4 Missing ⚠️
...anager/VideoReceiver/GStreamer/GStreamerHelpers.cc 0.00% 0 Missing and 1 partial ⚠️

❌ Your patch check has failed because the patch coverage (1.00%) is below the target coverage (30.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #14727      +/-   ##
==========================================
+ Coverage   25.47%   32.64%   +7.17%     
==========================================
  Files         769      784      +15     
  Lines       65912    67593    +1681     
  Branches    30495    31311     +816     
==========================================
+ Hits        16788    22067    +5279     
+ Misses      37285    30657    -6628     
- Partials    11839    14869    +3030     
Flag Coverage Δ
unittests 32.64% <1.00%> (+7.17%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/Utilities/Network/QGCNetworkHelper.h 0.00% <ø> (ø)
...Manager/VideoReceiver/GStreamer/GstVideoReceiver.h 0.00% <ø> (ø)
...anager/VideoReceiver/GStreamer/GStreamerHelpers.cc 38.65% <0.00%> (+1.96%) ⬆️
src/Camera/VehicleCameraControl.cc 9.52% <0.00%> (+7.77%) ⬆️
src/VideoManager/VideoManager.cc 7.96% <0.00%> (+6.06%) ⬆️
...anager/VideoReceiver/GStreamer/GstSourceFactory.cc 30.74% <0.00%> (ø)
src/Settings/VideoSettings.cc 29.84% <6.25%> (+14.05%) ⬆️
...anager/VideoReceiver/GStreamer/GstVideoReceiver.cc 3.04% <0.00%> (+0.14%) ⬆️

... and 470 files with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ed09a05...8e06c53. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Build Results

Platform Status

Platform Status Details
Linux Passed View
Windows Passed View
MacOS Passed View
Android Passed View

All builds passed.

Pre-commit

Check Status Details
pre-commit Failed (non-blocking) View

Pre-commit hooks: 0 passed, 0 failed, 0 skipped.

Test Results

linux-coverage-integration: 37 passed, 0 skipped
linux-coverage-unit: 118 passed, 0 skipped
linux-sanitizers-integration: 20 passed, 0 skipped
linux-sanitizers-unit: 118 passed, 0 skipped
Total: 293 passed, 0 skipped

Code Coverage

Coverage: 69.4%

No baseline available for comparison

Artifact Sizes

Artifact Size
QGroundControl 221.80 MB
QGroundControl-aarch64 179.34 MB
QGroundControl-installer-AMD64 137.85 MB
QGroundControl-installer-AMD64-ARM64 80.36 MB
QGroundControl-installer-ARM64 109.11 MB
QGroundControl-linux 87.04 MB
QGroundControl-mac 87.04 MB
QGroundControl-x86_64 191.58 MB
No baseline available for comparison

Updated: 2026-07-31 02:47:57 UTC • Commit: 8e06c53 • Triggered by: Windows

@alireza787b

alireza787b commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

Coverage clarification for exact head 8e06c53cc: QGCNetworkRedactionTest is registered as a dedicated no-I/O Unit/Utilities test, and the exact-head Linux Test + Coverage workflow passed. The new GStreamer DOT-property regression also passed. Codecov reports 1% patch coverage because the changed lines are dominated by production logging call sites that the suite does not invoke (including 62 lines in GstVideoReceiver), not because the helper test is absent. Both codecov/patch and codecov/project concluded success, the project coverage threshold passed, and no coverage exclusions were added.

@DonLakeFlyer DonLakeFlyer added this to the Release V5.1 milestone Jul 29, 2026
@DonLakeFlyer

Copy link
Copy Markdown
Contributor

Review comments:

Design / debuggability

1. Over-redaction will hurt video debugging — the biggest concern. The helper strips path, query, and fragment, not just credentials. The RTSP path is the stream identity, not a secret in the common case: the #1 RTSP failure mode is a wrong mount point (/stream1 vs /axis-media/media.amp), and after this PR every such log reads rtsp://cam:554/<redacted>. "Invalid RTSP URI:" in GstSourceFactory.cc becomes almost useless — it says the URI is invalid while hiding which part is invalid. These are opt-in qCDebug categories that users enable precisely to produce bug-report logs; that's when you want fidelity. Recommend QUrl::RemoveUserInfo as the baseline (what the existing GStreamer code already did), optionally redacting query values only.

2. Concrete regression: <invalid-url> for the most common auto-stream case. In VideoManager::_updateAutoStream, pInfo->uri() for VIDEO_STREAM_TYPE_RTPUDP MAVLink cameras is frequently a bare port number (e.g. "5600") — see the QStringLiteral("udp://0.0.0.0:%1").arg(pInfo->uri()) fallbacks just below the log line. redactedUrlForLogging("5600") returns <invalid-url>, so the primary "Configure stream" diagnostic for PX4/ArduPilot camera auto-config is destroyed for the most common vehicle setup. Same problem for any scheme-less user-typed URL: the <invalid-url> sentinel hides exactly the malformed string needed to debug a parse failure.

3. VideoSettings changes remove the URL entirely. In the VideoSettings.cc hunks, "Testing configuration for UDP Stream:" loses its value completely. A UDP URL is 0.0.0.0:5600 — nothing secret — and the value is the whole point of the log (catching empty/typo'd config). At minimum log the redacted form, not nothing.

Missed coverage

4. src/Camera/VehicleCameraControl.cc lines 1820 and 1832 log raw pInfo->uri() ("Stopping stream:" / "Starting stream:") — the same URI the PR redacts in VideoManager::_updateAutoStream. If the goal is "QGC-owned URI logging," these are in scope and missed.

5. DOT pipeline dumps still leak credentials. GST_DEBUG_BIN_TO_DOT_FILE calls in GstVideoReceiver serialize element properties, including rtspsrc location=rtsp://user:pass@…. The PR body scopes out GStreamer-generated text, but the highest-fidelity credential leak path (the dumps users are told to attach for pipeline debugging) is untouched — the PR reduces, not eliminates, exposure.

Helper implementation

6. Silent query/fragment removal is asymmetric with path handling. Path redaction leaves a visible /<redacted> marker, but the test expects https://example.com/?token=abc123https://example.com/ — no indication anything was removed. A log reader can't distinguish "URL had no query" from "query was redacted," which itself misleads debugging. If query removal stays, add a marker.

7. Dead-store no-op in the helper: redactedUrl.setPath(hadPath ? QString() : redactedUrl.path()) — the else branch assigns the path to itself. Simpler: if (hadPath) { redactedUrl.setPath(QString()); }.

Log severity

8. qCCritical misuse on the touched lines (pre-existing). GstSourceFactory.cc lines 276, 316, 321, 339 use qCCritical for bad user-entered URLs while the identical failure class at line 508 ("Unsupported URI scheme:") uses qCWarning. Since the PR rewrites all four lines anyway, downgrading them to qCWarning here would be a zero-cost consistency fix (or explicitly defer as follow-up to keep scope tight).

GstVideoReceiver

9. _redactedUri() re-parses and redacts on every log call, including the watchdog and per-300-frames flow logs. Since _uri only changes in setUri(), caching the redacted string there would be cheaper and simpler at the call sites. Not blocking (qCDebug args aren't evaluated when the category is off).

Tests

10. Single monolithic test slot. QGCNetworkHelperTest follows one-slot-per-behavior; the new QGCNetworkRedactionTest packs 8+ distinct cases (credentials, encoded userinfo, IPv6, root-path query, invalid input…) into a single _testRedactedUrlForLogging slot, so a failure doesn't tell you which case broke.

11. Missing test cases: empty string, bare host:port without scheme (the comment-2 case — this should be a test, and it would expose the <invalid-url> regression), and the QUrl overload directly.

Summary

Comments 1–3 are the substantive objections (path/query stripping, <invalid-url> swallowing bare-port MAVLink URIs, VideoSettings logging nothing), and comment 4 shows the stated scope isn't fully covered. Suggest: redact userinfo (and at most query values), keep host/port/path, don't replace unparseable input with a sentinel without handling the bare-port case, and cover VehicleCameraControl.

@DonLakeFlyer

Copy link
Copy Markdown
Contributor

Most of the code review is relevant with my main concern being redacting urls to the point of making remote debugging impossible or less useful.

@alireza787b
alireza787b force-pushed the agent/video-url-log-redaction branch from fd4a35f to 2d149ee Compare July 31, 2026 01:26
@github-actions github-actions Bot removed the CMake label Jul 31, 2026
@alireza787b

alireza787b commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the detailed review. I addressed the substantive points in 2d149ee39 and rebased the branch onto current master.

  • Stream paths are preserved for mount-point debugging.
  • User information is removed; query keys remain visible with each value replaced by REDACTED; fragments retain a visible redaction marker.
  • Bare ports and scheme-less host/port values remain visible. Empty and genuinely invalid inputs have distinct sentinels.
  • VideoSettings values are present again, and VehicleCameraControl start/stop diagnostics use the same helper.
  • QGC DOT dumps now retain topology, media type, caps details, and states while omitting element property values. A GStreamer regression test verifies that a secret source property is absent.
  • The touched bad-input diagnostics are warnings rather than critical failures.
  • Redaction coverage is split into focused slots in a dedicated no-I/O Unit/Utilities test and includes empty input, bare port, scheme-less host/port, and the QUrl overload.

I left the optional cached redacted-URI suggestion out to avoid adding synchronized state for disabled debug categories; it can remain a separate optimization if profiling ever justifies it. The rebased exact-head workflows are running now.

@github-actions github-actions Bot added the CMake label Jul 31, 2026
@alireza787b

Copy link
Copy Markdown
Contributor Author

Small follow-up at a16dd0b46: while checking the CI label contract, I found that QGCNetworkHelperTest is labeled Network and is excluded by QGC's normal just test / check-ci path. I moved the same five focused, no-I/O redaction slots into a dedicated QGCNetworkRedactionTest registered as Unit/Utilities. This preserves the reviewed one-slot-per-behavior structure and makes the coverage executable in normal CI; production behavior is unchanged.

@alireza787b

Copy link
Copy Markdown
Contributor Author

Exact-head CI caught one consistency gap from the severity change: the existing negative UDP/TCP source-factory tests still ignored these diagnostics as QtCriticalMsg, while the reviewed implementation now emits QtWarningMsg. The new URL-redaction and DOT-property tests themselves passed. Commit 8e06c53cc updates only those strict log expectations, and the refreshed workflow is running now. I propagated the same narrow test correction to #14730 and #14731.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants