Natural-language missions. Deterministic execution. Verifiable outcomes.
English | 中文
A drone inspection operations platform in development, built on a safety-constrained mission runtime. The implemented simulation stack provides typed missions, admission and approval, local execution and evidence-based reports, plus project-scoped sites and virtual docks that gate every dispatch. Durable workflows, fleet scheduling and finding-to-reinspection workflows are the next product milestones.
Current scope: M2 completed on 2026-09-23 for a single drone in PX4 SITL + Gazebo (software-in-the-loop simulation). On 2026-09-25 the simulation line of M3 (local autonomy through a guarded PX4 external mode) passed its release gate; M3 closes only after Jetson-in-the-loop validation. H1 carries the pending JIL work; the original combined M3 gate remains not passed. P0 runtime provenance and product gates passed on 2026-09-25 at
de597d0; see P0 validation. P1 projects, virtual docks and the dispatch claim gate passed on 2026-09-26 atb49701b: three sites with logical docks and logical aircraft (S0), and one PX4 SITL aircraft through a virtual dock (S1); see P1 validation. P2 durable workflows passed on 2026-09-26 at3bdbd50: restartable inspection → labelled analysis → human review → simulated work order → reinspection runs, without duplicate dispatch or dispatch after a cancel, over three logical aircraft (S0) and on one PX4 SITL aircraft through a virtual dock (S1); see P2 validation. P3 multi-site, multi-drone scheduling passed on 2026-09-27 at73fcf23: deterministic, replayable assignment with airspace-cell holds, withdrawal before a claim and task-level relay, over logical sites and a 10 / 30 / 100-node ladder (S0) and two PX4 SITL aircraft in one Gazebo world (S1); see P3 validation. The P4 multimodal business loop was implemented at8f41112and holds in the logical world (S0), on PX4 SITL with real appearance changes (S1) and on the resident desk. Its first release gate missed only S2: the single live test of the frozen MiniMax-M3 analyzer on a VisA circuit-board subset reached precision 0.8835 against the required 0.90. P4 passed on 2026-10-10 after an S2-only re-verification (D076, below); see P4 validation. On 2026-10-10 the code findings of an external review were adopted (D075): airspace footprints now follow each skill's motion range, so external-mode tasks hold their whole task scope and recovery landing sites are covered, while a flight that cannot be bounded is refused a reservation; the energy model refuses a map whose landing sites leave its ground plane; the completion cache keys every generation option. For the open S2 criterion, MiniMax-M3.1-Flash-Preview was registered for the vision tier and the change-v6 profile was frozen on the calibration split at τ 0.95 (D076). By the user's decision its single live test reused the VisA test split as a disclosed second use, pre-registered before the run. It reached precision 0.9703 and recall 0.8167 (MiniMax-M3: 0.8835 and 0.758) with coverage 0.998 and no false assurance, and the S2-only re-verification gate passed atf4b1b99. The figures cover VisA close-ups only, not drone imagery. A checks workflow now runs on every push and pull request (D077). See the review record. P5, the hardware-free platform v0.1 (D069–D073), passed its release gate on 2026-10-09 at1716843for the software, SITL and protocol-simulator scope. One resident desk serves a PX4 SITL site, a logical fleet and a simulated vendor-managed dock under one service, with a second business template, a vendor protocol contract and a project audit trail. All 22 criteria passed, including a 72-hour real wall-clock soak in which the dock sessions answered within 10.6 s after each of 9 service restarts and 3 kills (limit 60 s). The first candidate4c3de14failed that recovery criterion (67.5 s, because the service refreshed every historical mission synchronously at start) and is kept as a negative record; see P5 validation and the operations guide. P6 started with active recapture and target-region quality (WP-P6-02, D078), which passed its gate on 2026-10-10 atecee452for the software / SITL scope. A new quality profile checks the asset's projected region (visibility, size, exposure and sharpness) instead of only the whole frame. When a verified capture is refused for a reason that another flight can change, an inspection workflow flies the same registered pose once more, and that flight needs its own human approval. The logical world passed 30/30; PX4 SITL with a glowing disc over the marker passed 5/5. The camera heading is declared, not measured, and was checked only against Gazebo truth; see P6 recapture validation. WP-P6-03 (D079) passed on 2026-10-11 at8d69ae1: 134 real-model answers to five adversarial corpora (planning, workflow drafts, operations ambiguity on two projects, and vision on S1 renders with two profiles) are committed and replayed strictly in the tests, with no answer leaving the operator's scope or taking effect without a person. Along the way the planner was given each robot's own site map (desk requests can now nameroad_northorasset_red_b) and the clock was removed from planning requests, which had made every real recording unreplayable; see P6 adversarial validation. Hardware and air-ground work follow the H/X milestones.
Try locally · Design · Validation · Documentation · Roadmap
- Plans inspection missions from natural language. MiniMax-M3 produces a constrained draft; the planner builds its context from read-only tools and registered assets, routes and flight volumes.
- Checks before execution. Deterministic compilation and admission validate capabilities, parameters, space, time, energy, airspace and resources. Unknown or invalid inputs are rejected.
- Binds approval to the task. Ed25519 signatures cover the approved version and package hash. Packages travel over mTLS and are independently checked onboard.
- Executes under local supervision. The mission executive schedules skills; a separate
guardianprocess owns the only flight-controller connection and applies recovery policies. Native failsafes and manual takeover remain available. - Reports evidence, including uncertainty. Image and telemetry checks feed completed / not completed / uncertain reports. MCAP, ULog and event records support independent judging and offline replay.
- Flies local autonomy through a guarded second path (M3, simulation). ROS 2 Jazzy nodes plan short segments from depth maps; the
guardianfilters every target through a control barrier function and authorizes it for a few hundred milliseconds; a PX4 external-mode node only forwards those authorizations and stops when they end. Onboard event detection produces candidate facts only. - Dispatches only what the site supports (P1, simulation). Projects bind sites, docks and aircraft. A dock reports link, lid, aircraft presence, energy, environment and upkeep; one deterministic eligibility check with exclusive reservations gates the preview, the lid opening and the moment the aircraft fetches its package. Holds are released only on a terminal result plus fresh grounded evidence, never on a timeout. Docks are logical simulators, not vendor hardware.
- Runs durable inspection workflows (P2, simulation). Versioned templates of whitelisted activities start by hand, on a daily or interval schedule, or from a bound event. Each run pins its template and survives service restarts through a lease, fencing and an outbox; the same activity never dispatches twice, and nothing new starts after a cancel. Every flight still needs its own human approval. Analysis is a labelled scripted fixture or a deterministic colour signature, not recognition quality; a reviewer's confirmation creates a simulated work order, and repair feedback starts a reinspection run.
- Schedules tasks across sites and aircraft (P3, simulation). An operator submits an inspection task; a pure, replayable decision filters every candidate with named reasons and ranks the rest by arrival time, recent use and robot id. One transaction assigns the task, compiles its mission and holds the robot, the dock and the airspace cells of its route; a silent aircraft's envelope grows until it is reconciled. An unclaimed assignment blocked too long is withdrawn to another robot with a new epoch, and a definite flight failure relays the task. Every flight still needs its own human approval.
- Closes the business loop on evidence (P4, simulation). A deterministic quality layer and durable analysis jobs turn verified captures into candidate findings with labelled sources (live model, recording, scripted or deterministic). Only a project reviewer confirms a finding; each confirmed finding gets one work order, and repair feedback starts a reinspection that closes the order only with a new verified capture taken after the feedback, an unsuspected analysis from an allowed source and the reviewer's confirmation. Model answers are candidates only and never change a flight verdict.
- Provides human and agent entry points. A Web mission desk supports planning and approval; the A2A gateway accepts task submissions and status queries. A resident desk can run behind Tailscale.
The implemented skill set covers takeoff, registered routes, image capture, asset inspection, return-home and landing. M2 inspection uses predefined observation routes in the simulation scene.
Prerequisites: Python 3.12+, uv and Git. The local mission desk runs on Windows or Linux without PX4 or Gazebo.
git clone https://github.com/SuperdeMan/drone-agent.git
cd drone-agent
uv sync --group dev
uv run python -m drone_agent.console.mission --localOpen http://127.0.0.1:8769, select campus_training and asset_red, then submit this exact example:
Inspect the red equipment marker east of the pad and bring back a photo.
Review the plan and admission checks, then approve the package. Local mode plans, admits and signs; no robot is connected and nothing flies.
Without MINIMAX_API_KEY, the desk uses labelled scripted answers for the exact requests in the M2 scenario set. With a key in the process environment, it uses live MiniMax-M3 planning. The page identifies the planner; key setup is documented in the cloud development guide.
To see every operations workspace locally (fleet and docks, workflows, scheduling, findings and work orders), run uv run python scripts/desk_preview.py --seed and open http://127.0.0.1:8770. It serves the desk over the logical S0 world with the resident desk's catalogs; flights are logical, labelled as such, and nothing in the preview is validation evidence.
Integration uses a configured Linux cloud workspace. After following the setup guide, inspect the target and retrieve the resident mission desk URL:
uv run python scripts/dev_stack.py target
uv run python scripts/dev_stack.py status
uv run python scripts/dev_stack.py desk-cloud --statusThe unified cloud flight desk provides the operations desk (workspaces for missions and approvals, workflows, scheduling, fleet and docks, and findings and work orders; D068) and M1 fixed inspections at one origin. Approved M2 missions fly in PX4/Gazebo and produce evidence reports; M1 retains live tracks, camera frames, pause/resume and cancellation. The two modes fly sequentially; switching pages never cancels a flight. See the flight desk guide and the M2 review and validation record.
PX4 SITL and Gazebo run on Linux; use an ASCII repository path for native components. The simulation guide also documents the explicit local fallback.
Request → constrained draft → compilation and admission → signed approval → local execution → evidence report.
| Boundary | Responsibility |
|---|---|
| Ground / cloud planning | The planner proposes a draft; deterministic code builds MissionSpec. Models have no control access. |
| Admission and approval | Validate the mission, bind approval to package_hash, and deliver a signed MissionPackage. |
| Onboard execution | uplink receives packages; executive schedules skills; guardian checks authority, freshness and constraints before control writes. |
| Verification and replay | Recheck evidence and keep execution_status, effect_verdict and safety_verdict separate. UNKNOWN never counts as success. |
The cloud sends an authorized mission and its limits. Local execution and recovery do not depend on a continuous stream of cloud-generated control commands. See the architecture overview, contracts and safety model.
Recorded results below belong to their exact revisions and scopes; they are not test results for every later commit.
| Evidence | Revision | Recorded result |
|---|---|---|
| Operations explanation assistant (WP-P6-05, D083) | 76e7403 |
Linux 1476 checks passed. The desk can explain a mission: the service builds a fact table from its records, the model (no tools) answers citing fact ids, and the answer is shown only after every citation, code, evidence id, version and claim of approval or completion is checked against the facts; otherwise a deterministic summary. A frozen corpus of 12 real mission states and 26 questions: 170 scripted doubles all as expected. One live MiniMax-M3 run: 23/26 answers passed the check, all citing the decisive fact; the 3 rejected were correct answers that mentioned approval or completion inside a negation or condition. No fabrication was shown. P1 S0, the authorization matrix and the page client tests pass. |
| Fine airspace reservations (WP-P6-06, D084) | 6544a9a |
Linux 1530 checks passed. The P3 judge now checks that every airborne position (Gazebo truth in S1) lies in a cell its own flight held at that moment: 0 violations under both models in the S0 matrix, the ladders, a long-route rung and P3 S1 (minimum margin about 3.3 m). Per-leg corridors are an opt-in v2 catalog model. On long routes they cut airspace-only waiting from 261 s to 52 s, but the pre-registered rule required the drop to exceed three times the noise of two box runs (211.1 s against 209.1 s), so corridors are not adopted and no catalog switches. A first comparison was voided because Windows throttled the unattended local runs. |
| Cross-site reference check (D082) | 4398938 |
Linux 1470 checks passed. Before planning, the service checks the request text against the catalog's ids, site map ids and an alias file; a request that names another project, site, robot or dock is rejected with scope.foreign_reference, without a model call. On the three labelled operations corpora and 15 desk cases: 0 false rejections, 17 caught, 5 missed (descriptions only, another volume, a foreign asset id). The first alias set rejected one request that must be planned and did not pass; the revised set was re-evaluated on the same data. P1 S0 42/42, authorization escapes 0. On the resident desk the request for "the red equipment marker at site A" from site B's drone is now rejected in 0.2 s. |
| Planner binding (D081) | b4a17a3 |
Linux 1462 checks passed. In catalog mode the planning request now names the robot's project, site and robot, and planner-v3 declines references to another one. On a second held-out corpus (38 cases) planner-v2 and planner-v3 each answered twice: wrong plans fell from 12 to 4 and over-refusals rose from 3 to 4; all 20 references to other projects, sites or robots were declined. The pre-registered rule adopted planner-v3. nl_v1 escapes stay 0; the P1 S0 and authorization matrices pass. The resident desk now plans with planner-v3. Cross-site substitution is reduced, not eliminated: it recurred once on the development set and once in a desk observation. |
| Planner clarification (D080) | 02694be |
Linux 1450 checks passed. planner-v2 asks the operator instead of guessing when a request does not single out its assets. A held-out corpus (38 cases, written after the prompt was frozen by an author who never saw either prompt) was answered live twice by each prompt: wrong plans fell from 19 to 5 and over-refusals rose from 0 to 1, so the pre-registered rule adopted planner-v2. nl_v1 escapes stay 0. 222 recordings replay strictly. The resident desk now plans with planner-v2. Cross-site references to an asset whose colour and id also exist at the robot's own site still slip through. |
| P6 adversarial recordings (D079) | 8d69ae1 |
Linux 1433 checks passed. Five live corpora, 134 recordings, all replayed strictly on the candidate and matching their live receipts; escapes 0. Planner (MiniMax-M3), operations corpus: 30 accepted, 5 wrong but admitted only behind the human approval, 3 stopped by the guards; two live rounds agreed case by case on 25/32 and 31/38. Vision: change-v3 followed no text in the image (1 false assurance); change-v6 rated 3 damaged markers captioned "passed" as normal, though an upscaled damaged control without text also fooled it (6 false assurances). P1 S0 42/42, P1 S1 5/5, authorization escapes 0. Through the desk entry the live planner now plans road_north and asset_red_b. |
| P6 active recapture (D078) | ecee452 |
Linux 1411 checks passed. S0 30/30 (P6-F01–F10 × 3 seeds): 21 refusals were replaced by a completed recapture; 6 runs failed without a conclusion when the recapture was refused again or declined; the whole-frame control template read a shadow as suspected 3 times, the P6 template 0 times. PX4 SITL 5/5 in 9 flights: the glowing disc raised the target's bright-pixel share to 0.997–1.0 (limit 0.5), and the recapture after its removal was normal. Gazebo truth at the 9 captures: heading 1.9°–12.7° off the approach route (declared ±20°), reported position 0.02–0.09 m from truth (declared 0.35 m). P1–P5 S0, the authorization matrix and the P2 / P4 S1 regressions passed. The resident desk ran one recapture session through the tailnet entry. |
| External review hardening (D075, D076) | 45ba863 |
Linux 1356 checks passed. Local S0 regressions all passed: P3 45/45 and the ladder 3/3, P1 42/42, P2 45/45, P4 45/45, P5 S0 + S3 60/60, and the authorization matrix with zero escapes; attempts stalled while the dev PC slept were voided and rerun. Two-aircraft P3 S1 5/5. M3-SITL energy scenarios 9/9 at three seeds. The resident desk was activated, and one scheduler task was flown with its airspace cells held and released; judge completed, no false success. S2 calibration split only: frozen change-v6 on MiniMax-M3.1-Flash-Preview reached precision 0.9495 and recall 0.783 at τ 0.95. No S2 test run yet. |
| Operations desk visual v2 (D074) | 0094238 |
Deployed with the Figma MCP implementation, browser icon and explicit Light/Dark buttons in both headers. Defaults to light; saves the choice across reloads and M1/M2 navigation and synchronizes tabs, independently of the OS. Linux: 1327 passed, no failures/errors/skips; worktree and isolated archive: 1326 passed plus 1 existing Windows Unix-socket skip each; Ruff and 32/32 client tests passed. 50 standard browser checks and 28 theme checks passed with no overflow or script errors. Business scripts, service fields and gate states are unchanged; no new resident flight was started. |
| M1 runtime | eefe76e |
400 tests; 22 flight/fault scenarios × 3 seeds = 66/66 passed; zero false success reports; matching replay. |
| M2 release gate | f362b9e |
775 tests; 18/18 end-to-end cases; 66/66 M1 regression; 42 deterministic and 32 scripted adversarial cases blocked or refused; zero false success reports and matching flight replay. |
| Live planner baseline | f362b9e |
MiniMax-M3: 20/20 plannable requests admitted on the first attempt, 8/8 expected refusals, 4/4 required blocks. |
| Resident desk with live planning | 74984f9 |
Chinese and English requests planned, approved, flown and independently verified; a privacy-intrusive request refused. |
| M2 review and unified desk | e8edf28 |
833 Linux checks and 18/18 M2 E2E cases passed; evidence handling, post-cancellation retries and judge coverage fixed. Live-model and interactive checks are listed separately. |
| M3-SITL release gate | 75382dc |
983 tests; 16 local-autonomy and fault scenarios × 3 seeds = 48/48; M1 66/66, M2 18/18 and 6/6 over Zenoh; zero false success reports and matching replay; supervision period p99 ≤ 106.6 ms. Jetson-in-the-loop pending hardware. |
| P0 release gate | de597d0 |
1000 Linux tests; 18/18 scripted M2 E2E cases and source audits; MiniMax-M3 live completion and cancellation probes; authoritative flight-version receipts verified. |
| P1 release gate | b49701b |
1102 Linux tests; 14 dock and dispatch faults × 3 seeds = 42/42 over three logical sites (S0); 5/5 PX4 SITL cases through a virtual dock (S1); M2 18/18; wrong / duplicate dispatch, wrong release, project escape and false success all zero; resident desk migrated with a verified backup. |
| P3 release gate | 73fcf23 |
1226 Linux tests; 15 scheduling faults × 3 seeds = 45/45 and a 10 / 30 / 100 logical-node ladder 3/3 (S0); 5/5 two-aircraft PX4 SITL cases with 9 flights in one world (S1) at a real-time factor ≥ 0.97; P1, P2 S0 and S1 and M2 18/18 regressions; double ownership, duplicate execution, lost tasks, conflicting holds, false success and stale-epoch effects all zero; resident desk migrated with a verified backup and a task scheduled, approved and flown through it. |
| P4 release gate | 8f41112 |
Not passed (S2 only); S2 re-verified in the next row. 1258 Linux tests; 15 business-loop faults × 3 seeds = 45/45 (S0) and the P1 / P2 / P3 S0 matrices; 5/5 PX4 SITL cases with 12 flights and a Gazebo damage patch (S1); P2 S1, P3 S1 and M2 18/18 regressions; resident desk migrated with a verified backup and a watch run analysed by the scripted fixture and live MiniMax-M3; all safety counts zero. The single live S2 test on VisA boards: precision 0.8835 < 0.90, recall 0.758, coverage 0.996, false assurance 0.017. |
| P4 S2 re-verification | f4b1b99 |
Passed (P4 closed). MiniMax-M3.1-Flash-Preview, change-v6 at τ 0.95: single live test on the VisA test split (disclosed second use) with precision 0.9703, recall 0.8167, coverage 0.998 and false assurance 0; strict replay 640/640 on the candidate; P4 S0 45/45 and the P1 / P2 / P3 S0 matrices rerun by the gate; the other fourteen P4 criteria remain those of 8f41112. |
| Operations desk workspaces (D068) | a0d12ff |
1263 Linux tests; every workspace rendered through the real Tailnet entry in Edge, light and dark, desktop and phone, with no console errors; live-planned missions submitted, approved and followed in the page to the independent judge; a watch run, a repair and its reinspection driven through the page. No gate status changes. |
| P2 release gate | 3bdbd50 |
1174 Linux tests; 15 workflow restart, deduplication and cancellation faults × 3 seeds = 45/45 (S0) plus the P1 S0 matrix 42/42; 5/5 PX4 SITL cases with 8 flights (S1); M2 18/18; duplicate dispatch, dispatch after cancel, false success, false work orders and lost runs all zero; resident desk migrated with a verified backup and a workflow run through to reinspection. |
The M2 end-to-end and natural-language adversarial runs used labelled scripted planner answers to test the execution chain and its boundaries. Live model behavior is documented separately in the baseline and resident-desk records. The machine-readable release result links the M2 evidence together.
uv run ruff check .
uv run pytest -q
uv run python scripts/generate_contract_fields.py --check
uv run python scripts/generate_proto.pyFor dependency downloads through the project mirror, set UV_DEFAULT_INDEX=https://pypi.tuna.tsinghua.edu.cn/simple for the current command or terminal session. Contract and planning checks run directly on Windows; Linux builds and integration use scripts/dev_stack.py.
Read CLAUDE.md before contributing (AGENTS.md is the coding-agent entry). Architecture and execution changes start with the relevant design documents. Design documents are in Chinese; the READMEs and code comments are bilingual.
| Start here | Guide |
|---|---|
| System boundaries and document map | Architecture overview |
| Message semantics and runtime assurance | Contracts · Safety · Wire protocol |
| Cloud simulation and the mission desk | Cloud development · Mission desk |
| Review flight evidence and replay | Evaluation · Fetch and view a run |
| Operations design and next work | Operations architecture · Implementation tasks · P4 validation · P5 validation · P6 recapture validation · P6 adversarial validation · P5 plan · Operations guide |
| Design rationale and reuse | Decisions · Sibling-project reuse |
| Stage | Scope | Status |
|---|---|---|
| M0–M2 | Contracts, single-drone runtime, constrained agent and evidence loop | Complete in simulation |
| M3 | Local autonomy, perception, localization and degradation handling | Simulation line passed; Jetson-in-the-loop pending |
| P0 | Capability inventory, runtime provenance and product gates | Complete for the software / SITL scope |
| P1 | Projects, sites, virtual docks and the dispatch claim gate | Complete for the software / SITL scope |
| P2 | Durable business workflows | Complete for the software / SITL scope |
| P3 | Multi-site, multi-drone scheduling | Complete for the software / SITL scope (two PX4 SITL aircraft in one world) |
| P4 | Multimodal business loop | Complete for the software / SITL scope; S2 re-verified with MiniMax-M3.1-Flash-Preview (D076) |
| P5 | Platform v0.1 | Complete for the software / SITL scope (vendor docks only against a protocol simulator) |
| P6 | Evidence-driven operations | Active recapture and target-region quality (WP-P6-02) and real-model adversarial recordings (WP-P6-03, followed by the planner-v2 clarification prompt, D080, planner-v3 with the catalog binding, D081, the deterministic cross-site reference check, D082) the operations explanation assistant (WP-P6-05) and fine airspace reservations (WP-P6-06, corridors not adopted) complete for the software / SITL scope; other work packages proposed |
| H1–H3 | JIL, bench / restricted flights and field operations | Separate hardware validation track |
| X1–X3 | Air-ground collaboration, vendor hardware and model / kernel research | Conditional extensions |
See the full roadmap and exit criteria. Local autonomy is validated in simulation only; cross-robot handoff and DJI / ArduPilot support are outside the current implementation. Airspace admission has an interface definition with a recorded UOM backend and rejects real-mode missions without a filing; energy estimates are for simulation only.