Enhance UI performance for MQTT#118
Conversation
…I applications This document outlines performance considerations, optimization strategies, and implementation details for managing high-frequency MQTT messages in a UI context. It covers dispatcher queue behavior, memory management, and recommended settings for various throughput scenarios, along with alternative architectures for extreme throughput.
…dback for new messages
…UI item management
… capping max UI items to 50,000
There was a problem hiding this comment.
Pull request overview
This PR refactors MQTT ingestion + UI rendering to better support high-throughput scenarios by introducing a channel→reactive-stream pipeline, adding configurable message-processing profiles and counters, and updating several UI surfaces (status bar, Topic Explorer, recording toggles). It also adds centralized logging/exception handling hooks and extensive documentation for the new throughput model.
Changes:
- Introduces bounded/unbounded channel ingestion plus hot observable streams for messages/packets/logs, with per-subscriber buffering and batched UI inserts.
- Adds UI controls for throughput profiles and status-bar counters (Received/Notified/Buffered/Dropped), plus TreeDataGrid-based Topic Explorer with row highlight animation.
- Adds logging infrastructure (Microsoft.Extensions.Logging) and new docs explaining dispatcher pressure and channel handling.
Reviewed changes
Copilot reviewed 36 out of 36 changed files in this pull request and generated 15 comments.
Show a summary per file
| File | Description |
|---|---|
| docs/high-throughput-guide.md | Adds detailed performance guidance and rationale for the channel + Rx buffering approach. |
| docs/channel-handling.md | Documents channel full modes, packet inspection constraints, counters, and pipeline lifecycle. |
| Source/mqttMultimeter.csproj | Updates Avalonia packages; adds DynamicData/Logging/TreeDataGrid packages. |
| Source/Services/Mqtt/StreamConnectedEventArgs.cs | Adds event args to pass streams + pipeline settings to subscribers. |
| Source/Services/Mqtt/MqttClientService.cs | Implements channel ingestion, reactive session streams, counters, and packet inspection gating. |
| Source/Pages/TopicExplorer/TopicExplorerTreeNodeViewModel.cs | Adds concurrent child lookup for trie-based topic resolution. |
| Source/Pages/TopicExplorer/TopicExplorerTreeNodeView.axaml.cs | Fixes virtualization-related event subscription; updates row highlight to TreeDataGridRow. |
| Source/Pages/TopicExplorer/TopicExplorerPageViewModel.cs | Migrates to reactive message stream subscription; trie-based node creation; TreeDataGrid source + selection. |
| Source/Pages/TopicExplorer/TopicExplorerPageView.axaml.cs | Adds overlay/detail panel wiring based on SelectedItem changes. |
| Source/Pages/TopicExplorer/TopicExplorerPageView.axaml | Migrates UI from TreeView to TreeDataGrid; adds templates and detail overlay parts. |
| Source/Pages/TopicExplorer/TopicExplorerItemViewModel.cs | Fixes latest-message selection logic for TrackLatestMessage behavior. |
| Source/Pages/Subscriptions/SubscriptionsPageViewModel.cs | Yields before subscribe call (likely UI responsiveness tweak). |
| Source/Pages/PacketInspector/PacketViewModel.cs | Makes Number mutable (no longer init-only). |
| Source/Pages/PacketInspector/PacketInspectorPageViewModel.cs | Switches to reactive packet stream subscription; batches inserts with trimming. |
| Source/Pages/PacketInspector/PacketInspectorPageView.axaml | Updates recording toggle bindings and null-safe packet buffer binding. |
| Source/Pages/Log/LogPageViewModel.cs | Switches to reactive log stream subscription; batches inserts with trimming. |
| Source/Pages/Log/LogPageView.axaml | Updates toggle binding/command wiring for log recording. |
| Source/Pages/Inflight/InflightPageViewModel.cs | Switches to reactive message stream subscription; batches inserts with trimming; adds import batching. |
| Source/Pages/Inflight/InflightPageView.axaml.cs | Makes import handler async; simplifies exception handling. |
| Source/Pages/Inflight/InflightPageView.axaml | Updates recording toggle bindings to use a command + OneWay checked binding. |
| Source/Pages/Inflight/Export/InflightPageItemExportService.cs | Imports by building a list then bulk-inserting (reduces UI churn). |
| Source/Pages/Connection/MessageProcessingProfile.cs | Adds data-driven throughput profiles (Low/Medium/High/Custom). |
| Source/Pages/Connection/MessageProcessingOptionsViewModel.cs | Adds viewmodel for profile + channel/delay/buffer/max-items/counter settings. |
| Source/Pages/Connection/ConnectionPageViewModel.cs | Adds CanConnect/CanDisconnect gating; updates immediate connection state. |
| Source/Pages/Connection/ConnectionOptionsView.axaml | Adds “Message Processing” expander UI to configure profiles and pipeline settings. |
| Source/Pages/Connection/ConnectionItemViewModel.cs | Adds MessageProcessingOptions to each connection item. |
| Source/Pages/Connection/ConnectionItemView.axaml | Disables Connect/Disconnect buttons based on CanConnect/CanDisconnect. |
| Source/Pages/Connection/BoundedChannelFullModeOption.cs | Adds data model for full-mode options with display names. |
| Source/Main/MainViewModel.cs | Adds counters + timer lifecycle driven by message stream connect/disconnect events. |
| Source/Main/MainView.axaml.cs | Adds connect/disconnect toggle click handler. |
| Source/Main/MainView.axaml | Redesigns status bar: connection selector + toggle + counters + warning styling. |
| Source/Logging/LogObserverExceptionHandler.cs | Adds RxUI exception handler that logs and rethrows on UI thread. |
| Source/Extensions/CollectionTrimExtensions.cs | Adds SourceList.AddRangeAndTrim() batching helper. |
| Source/Controls/AutoGrid.cs | Improves dispatcher posting by passing explicit state and avoiding capturing instance members implicitly. |
| Source/App.axaml.cs | Adds logger factory + global exception hooks + Rx default exception handler + dev-tools logger integration. |
| Source/App.axaml | Adds TreeDataGrid Fluent theme include and highlight animation style. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
| catch (OperationCanceledException e) | ||
| { | ||
| _logger.LogCritical(e, "Channel processing tasks were canceled."); | ||
| } |
There was a problem hiding this comment.
Fixed by removing
| if (_mqttClient != null) | ||
| { | ||
| _mqttClient.ApplicationMessageReceivedAsync -= OnApplicationMessageReceived; | ||
| _mqttClient.DisconnectedAsync -= OnDisconnected; | ||
| _mqttClient.InspectPacketAsync -= OnInspectPacket; | ||
| _mqttClient.DisconnectedAsync -= OnDisconnected; |
| await _mqttClient!.DisconnectAsync().ConfigureAwait(false); | ||
| StopReactiveStreams(); | ||
| await StopChannelProcessingAsync().ConfigureAwait(false); |
| controls:AutoGrid.IsNextRow="True" | ||
| Classes="caption" | ||
| Content="Counter update (ms)" | ||
| ToolTip.Tip="How often the status-bar counters (Received, Notified, Buffered, Dropped) refresh. Uses Observable.Interval on the UI thread." /> |
| void OnUnhandledException(object sender, DispatcherUnhandledExceptionEventArgs e) | ||
| { | ||
| _logger.LogError(e.Exception, "Unhandled exception"); | ||
| ShowException(e.Exception); |
| [LoggerMessage(EventId = 0, Level = LogLevel.Error, Message = "Error in packet inspector.")] | ||
| public static partial void LogPackageInspectorError(this ILogger<MqttClientService> logger, Exception? exception); | ||
|
|
|
|
||
| if (sender is ToggleButton toggleButton) | ||
| { | ||
| toggleButton.IsChecked = viewModel.ConnectionPage.IsConnected; |
There was a problem hiding this comment.
Fixed by removing the event and promoting to a command.
| _logger.LogError(e.Exception, "Unobserved task exception"); | ||
| ShowException(e.Exception); |
| StartChannelProcessing(item.MessageProcessingOptions); | ||
| StartReactiveStreams(); | ||
|
|
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…event handlers on failure
…oltips in ConnectionOptionsView
…playing the exception on the UI thread
|
@chkr1011 this cover the #115 issue. I have write with copilot the guides and the process I follow to improve the performance. I have use the tool for the last months and I have not see any issue so far. I would apapreciate if you pay special attention to the tree topic explorer behavior just to make sure it provide the same featurs as before since I replace the entire component with the TreeView to get virtualization. Futher work can be made to bring virtualization to other Grids since I'm suspecting that virtualization is not implemented for Grids, we can use https://github.com/wieslawsoltes/ProDataGrid instead DataGrid since this one support virtualization. |
danielmeza
changed the title
…tgoing queue is full
|
@danielmeza I don't have much time for a review right now but I will have a look soon. It would he great if you can create multiple PR per topic/area so that I can approve smaller ones immediately. |
Since most of changes depends on changes made in the connection service, will be hard to split it in separated PRs, cause these base changes will break the way the app injects items in the UI in the previous version. What we can do is to separate the UI changes like the changes I did in the tool bar from the optimization changes. Other think we could try is to have a base branch where I can push small PRs, but I want to avoid that scenario since we will have to refactor the old code to make it build and that code will be removed at then to get this final result committed in this PR. I can expend some time with you in a live call if you wish to go over all the changes and discussed it. Or explain here in comments the changes in a self-review. |
This pull request introduces several improvements to the application's UI, error handling, and message counter logic. The most notable changes include enhanced error handling with logging, a more informative and interactive connection status bar, new message counter displays, and UI/UX enhancements for message lists and status warnings. Additionally, code quality improvements and new utility extensions are added.
Error handling and logging:
ILoggerFactoryand a customLogObserverExceptionHandlerfor handling and displaying unhandled exceptions and RxUI errors in the UI, improving diagnostics and stability. (Source/App.axaml.cs,Source/Logging/LogObserverExceptionHandler.cs) [1] [2] [3]UI/UX enhancements:
MainViewto include a connection selector, toggle button for connect/disconnect, and more detailed connection state and error messages. Added distinct styles for warning states and improved message counter visibility. (Source/Main/MainView.axaml,Source/Main/MainView.axaml.cs) [1] [2] [3]TreeDataGridand included the required theme, improving user feedback for incoming messages. (Source/App.axaml) [1] [2]Message counter and connection logic:
MainViewModelto track received, notified, buffered, and dropped messages separately, with new properties and UI bindings. Updated logic to sync counters with the MQTT message stream lifecycle. (Source/Main/MainViewModel.cs) [1] [2]Code quality and maintainability:
AutoGridby passing state explicitly to the UI thread dispatcher and referencing instance members directly. (Source/Controls/AutoGrid.cs) [1] [2]CollectionTrimExtensionsto efficiently add and trim items in lists, reducing UI update overhead for large message lists. (Source/Extensions/CollectionTrimExtensions.cs)