feat(logs-page): add a Logs page for browsing and fetching vehicle logs - #109
Open
dakejahl wants to merge 4 commits into
Open
feat(logs-page): add a Logs page for browsing and fetching vehicle logs#109dakejahl wants to merge 4 commits into
dakejahl wants to merge 4 commits into
Conversation
## Summary Adds a Logs page that lists every log logloader knows about -- on the vehicle, downloaded, uploaded -- and lets the operator pick which ones to fetch and publish. Uploaded logs link straight to the plot in the onboard Flight Review. ## Problem logloader used to decide on its own what to download and upload, and there was no way to see what it had or to ask for a particular log. Its counterpart change makes fetching request-driven, which needs somewhere to make requests from. ## Solution logloader now serves a small localhost API on :3005; the gateway proxies it at /api/logloader, including the SSE stream that drives live transfer progress. The page follows the existing house patterns: Options API, an axios service module, an inline EventSource with named events, and the SystemPage loading/error/retry triad. Select-all or a subset, then Download & Upload, Download Only, Upload, or Cancel. A log that is still being written by the vehicle never appears, so it cannot be selected at a size it will not keep. Verified against PX4 SITL and a local flight-review: first-launch queues only the newest log, select-all fetches and publishes the remaining four, and each lands byte-identical with a working plot link.
…ructure
## Summary
Adds a one-time translation of logloader.toml from its flat keys to the nested
layout, run from postinst before merge_configs.py.
## Problem
merge_configs takes its structure from the new template and prunes scalars the
template does not have. That is the right rule for a removed field and the wrong
one for a renamed field. Simulating an upgrade of a config with
remote_server = "https://review.arkelectron.com"
email = "pilot@example.com"
upload_enabled = true
public_logs = true
produced a merged file with [upload.remote] enabled = false and the stock
review.px4.io url: a vehicle configured to upload to a remote Flight Review
would silently stop after the upgrade. logloader still reads the flat keys as a
fallback, but that does not help when the merge has already deleted them.
## Solution
migrate_logloader_config.py rewrites the flat keys into their new homes, in both
the live config and the pre-upgrade backup, so what reaches merge_configs is an
ordinary same-shape merge. It is idempotent, leaves an already-nested file
alone, and never fails an upgrade.
…findings
## Summary
Follow-up to the Logs page covering an integration review, plus one bug that
turns out to affect every SSE consumer in the app, not just this page.
## Problem
The dev server gzips responses by default and its compression middleware
buffers until it has enough bytes, which for an SSE stream is never. Measured
through `npm run serve` with a browser's Accept-Encoding: 0 bytes and an
EventSource stuck at readyState 0; the same request through the gateway
returned immediately. That silently breaks the journal viewer, the network
stats stream and the firmware progress stream in dev as well.
Separately, http-proxy-middleware v3 reads handlers from `on`, so the v2
`onError`/`logLevel` options every proxy block used were dead: a stopped
upstream answered with the default plain-text 504 rather than the JSON the UI
parses, and the page reported "Request failed with status code 504".
## Solution
`compress: false` on the dev server, and `on: { error }` for all five proxies.
On the page: an upload records only the relative path Flight Review redirects
to, so targets now carry their base url and a remote plot link resolves against
the right origin instead of the ARK-OS host. A dropped stream is reported
rather than leaving a frozen table under a green "connected" dot; confirmations
expire; deleting a downloaded log asks first; an upload request names only the
targets that actually need the log; the select-all box is disabled on an empty
table; and the bokeh websocket is proxied in dev so a plot link works there too.
logloader's config tables are one level deep now, because the shared TomlEditor
renders exactly one level — with two, every upload setting silently vanished
from the Services page config editor. The editor also says so now instead of
rendering nothing, and the upgrade migration writes to the flattened names.
Depends on ARK-Electronics/logloader#32. The Logs page, the /api/logloader proxy and the restructured logloader.toml all need the binary that serves the API and reads the new schema; without the bump a deb built from this branch ships the old binary against the new config and the page 504s.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a Logs page listing every log logloader knows about — on the vehicle, downloaded, uploaded — and lets the operator pick which ones to fetch and publish. Uploaded logs link straight to the plot in the onboard Flight Review.
Depends on ARK-Electronics/logloader#32, which the submodule bump points at; merge that first.
Problem
logloader decided on its own what to download and upload, and there was no way to see what it had or to ask for a particular log. Its counterpart change makes fetching request-driven, which needs somewhere to make requests from.
Two things surfaced while wiring it up that are not specific to this page. The dev server gzips responses by default and its compression middleware buffers until it has enough bytes, which for an SSE stream is never — measured through
npm run servewith a browser'sAccept-Encoding: 0 bytes and anEventSourcestuck atreadyState 0, against 3332 bytes through the gateway. That has been silently breaking the journal viewer, the network-stats stream and the firmware-progress stream in dev too. Separately, http-proxy-middleware v3 reads handlers fromon, so the v2onError/logLeveloptions every proxy block uses are dead: a stopped upstream answers with the default plain-text 504 rather than the JSON the UI parses.Upgrading an existing install was also unsafe.
merge_configs.pytakes its structure from the new template and prunes scalars the template lacks — right for a removed field, wrong for a renamed one. Simulating an upgrade of a reallogloader.tomlproduced[upload_remote] enabled = falseand the stock url, so a vehicle configured to upload to a remote Flight Review would silently stop.Solution
logloader serves a localhost API on :3005; the gateway proxies it at
/api/logloader, including the SSE stream that drives live transfer progress. The page follows the house patterns: Options API, an axios service module, an inlineEventSourcewith named events, and the SystemPage loading/error/retry triad.Select all or a subset, then Download & Upload, Download Only, Upload, or Cancel. A log the vehicle is still writing never appears, so it cannot be selected at a size it will not keep. An upload records only the relative path Flight Review redirects to, so targets carry their base url and a remote plot link resolves against the right origin rather than the ARK-OS host.
compress: falseon the dev server,on: { error }for all five proxies, andmigrate_logloader_config.pyrun from postinst before the merge so a rename reachesmerge_configsas an ordinary same-shape merge.logloader's config tables are one level deep because the shared
TomlEditorrenders exactly one level; with two, every upload setting silently vanished from the Services page config editor. The editor now says so rather than rendering nothing.Verified against PX4 SITL and a local flight-review: first launch queues only the newest log, a log closing mid-run is picked up automatically, select-all fetches and publishes the rest, a restart re-downloads and re-uploads nothing, and a pre-FTP database plus old-format config carries its state and settings across.