BT Ops Workspace

This workspace contains an AutoAPMS-compatible Foxglove BT Ops stack:

• btops_interfaces: ROS 2 messages and services exposed to Foxglove.
• btops_core: shared backend models, validation, and export helpers.
• btops_autoapms: AutoAPMS-aware provider for resource discovery and executor control.
• btops_groot2: BT.CPP v4 Groot2 transport and monitor hub.
• btops_gateway: ROS 2 gateway node that bridges AutoAPMS, Groot2, and Foxglove.
• btops_demo_trees: dedicated AutoAPMS tree resources used as stable Foxglove smoke tests.
• foxglove-btops-extension: Foxglove extension with ResourceExplorer, RuntimeMonitor, and SourceEditor.

Progress

Implemented

• Live AutoAPMS discovery is wired in through the provider.
• The provider now includes AutoAPMS internal mission trees in the catalog and clearly marks environments where no installed mission config resources are available.
• btops_gateway exposes the planned /btops/* topics and services.
• Runtime execution uses AutoAPMS StartTreeExecutor, CommandTreeExecutor, and parameter services.
• Groot2 transport is live for FULLTREE, STATUS, recording, transitions, and hook requests when pyzmq is installed.
• Foxglove extension packaging is on the official local-development flow and builds into a .foxe bundle.
• The workspace now ships dedicated Foxglove-first demo resources so live monitoring no longer depends on the more fragile upstream sample trees.
• RuntimeMonitor now renders a live BT graph from Groot2 FULLTREE data and colors nodes from recent transition state.
• RuntimeMonitor now shows per-node breakpoint and substitution badges directly on the graph and keeps hook state synchronized from backend execution snapshots.
• RuntimeMonitor now supports direct blackboard editing through /btops/set_blackboard.
• RuntimeMonitor now exposes /btops/set_recording controls and a replay workflow note for recording /btops/* topics into MCAP.
• RuntimeMonitor now distinguishes live-control versus replay/view-only sessions and includes an operator-facing capture checklist plus a ros2 bag command hint.
• SourceEditor now supports catalog-driven resource picking, validation markers, normalized source reuse, and diff preview.
• End-to-end validation has been run with tree_executor, simple_skill_server, btops_gateway, and foxglove_bridge.
• End-to-end mission validation and runtime apply were exercised against auto_apms_mission's orchestrator node.
• scripts/smoke_runtime.py now exercises live start, transition delivery, recording toggle, and hook-state reflection against the running gateway.
• scripts/smoke_substitution_profiles.py now verifies substitution profiles, hook-state persistence, and recording metadata reflection against a dedicated substitution demo tree.
• scripts/record_btops_topics.sh now records the replay-critical /btops/* topics into a rosbag/MCAP capture with one command.
• scripts/smoke_multi_executor.py now verifies catalog mission roles, simultaneous tree/mission executor monitoring, distinct Groot2 ports, and the guarded mission-runtime diagnostic path.

Recently stabilized

• Fixed a resource refresh race that could cause node_manifest_identities is empty during /btops/start_behavior.
• Fixed gateway shutdown so Ctrl-C no longer emits Destroyable because destruction was requested.
• Verified live /btops/transition_event output from Groot2-backed monitoring, with a STATUS-driven fallback when the installed BT.CPP runtime returns an empty Groot2 transition buffer.
• Verified /btops/set_blackboard round-trips back into /btops/blackboard_event.
• Mission YAML validation now canonicalizes AutoAPMS mission keys like mission: into the uppercase schema expected by the installed runtime.
• Inline mission failures now surface an actionable diagnostic in /btops/execution_snapshot even when the installed AutoAPMS action result message is empty.
• Verified backend-driven hook_state_json updates on /btops/execution_snapshot after live breakpoint and substitution-profile requests.
• Verified /btops/set_recording now persists recording session metadata (session_id, start/stop times, event counts) into recording_metadata_json.
• The Resource Explorer now defaults to the dedicated FoxgloveStableDemo tree and hides the older AutoAPMS sample trees that rely on self-parameter lookups.
• Mission starts are now guarded by default to avoid dispatching a known-broken orchestrator runtime path; guarded sessions still surface catalog and diagnostic context without crashing into the duplicate-registration failure.

Current gaps

• Mission validation and inline dispatch work, but the installed AutoAPMS mission runtime still fails while creating the tree due to upstream duplicate node registration inside orchestrator.
• The current workaround is a guard rail: the gateway blocks mission execution by default, returns a guarded snapshot, and reports the known runtime issue without dispatching the unsafe launch path.
• Replay depends on externally recording /btops/catalog, /btops/execution_snapshot, /btops/transition_event, and /btops/blackboard_event into MCAP through Foxglove or ros2 bag; there is not yet a dedicated import/export wizard inside the extension.
• Runtime substitution is now profile-based (always_*, pulse_*), but still maps to Groot2 result forcing rather than swapping arbitrary node implementations at runtime.

Recommended Environment

source /opt/ros/jazzy/setup.bash
source /home/arl/btops_ws/install/setup.bash
export ROS_AUTOMATIC_DISCOVERY_RANGE=LOCALHOST
unset ROS_LOCALHOST_ONLY

Run

Build the workspace:

cd /home/arl/btops_ws
colcon build --symlink-install

Run the gateway:

source /opt/ros/jazzy/setup.bash
source /home/arl/btops_ws/install/setup.bash
ros2 run btops_gateway btops_gateway

Run Foxglove bridge:

source /opt/ros/jazzy/setup.bash
source /home/arl/btops_ws/install/setup.bash
ros2 launch foxglove_bridge foxglove_bridge_launch.xml port:=8765 address:=127.0.0.1

Run the live runtime smoke test:

source /opt/ros/jazzy/setup.bash
source /home/arl/btops_ws/install/setup.bash
python3 /home/arl/btops_ws/scripts/smoke_runtime.py --transition-timeout 25

Recommended first live demo in Foxglove:

• Executor: tree_executor
• Resource identity: tree/btops_demo_trees::foxglove_stable_demo::FoxgloveStableDemo
• Start mode: auto (defaults to attached for the dedicated demo)

Additional focused demos:

• tree/btops_demo_trees::foxglove_substitution_demo::FoxgloveSubstitutionDemo for substitution-profile validation.
• tree/btops_demo_trees::foxglove_replay_demo::FoxgloveReplayDemo for replay/recording sessions with repeated transitions.

Record the replay topics into a bag:

/home/arl/btops_ws/scripts/record_btops_topics.sh btops_capture

Run the multi-executor and mission-diagnostic smoke test:

source /opt/ros/jazzy/setup.bash
source /home/arl/btops_ws/install/setup.bash
python3 /home/arl/btops_ws/scripts/smoke_multi_executor.py

Run the substitution-profile and recording-metadata smoke test:

source /opt/ros/jazzy/setup.bash
source /home/arl/btops_ws/install/setup.bash
python3 /home/arl/btops_ws/scripts/smoke_substitution_profiles.py --profile always_failure

If you must bypass the guarded mission runtime to reproduce the upstream failure directly:

export BTOPS_ALLOW_UNSTABLE_MISSION_RUNTIME=1

Next Steps

1. Keep mission execution on a guarded path until the installed AutoAPMS runtime duplicate-registration issue is fixed upstream or a safe external workaround is found.
2. Decide whether true dummy-node substitution is worth a protocol extension beyond the current result-forcing substitution profiles.
3. Consider a dedicated replay/import wizard inside the Foxglove extension if manual MCAP capture and reopen becomes a usability bottleneck.
4. Add one-command local bringup/teardown helpers around the current smoke scripts if repeated operator validation becomes common.

See also:

• /home/arl/btops_ws/docs/OPERATIONS.md
• /home/arl/btops_ws/docs/KNOWN_ISSUES.md