Notice
The operation of each component in this system requires corresponding data files. As one of the main contributions of this project lies in the design of the full system pipeline and data collection, and due to privacy and security considerations, we currently provide neither the complete data files nor the full implementation code. Critical functions involving data processing have been abstracted in the submitted code to protect sensitive operational details. After the paper is accepted, we will release (1) non-sensitive data samples and (2) a complete executable version with all functional modules. Detailed data preparation methods are described in the appendix.
Textual DCS serves as the interface bridging the high-fidelity simulation platform DCS World and LLMs. It provides both high-level and low-level APIs for LLMs. The code mainly demonstrates the collection and processing at the low-level; high-level logic and additional low-level details are stored in specific data files and are not provided at this time.
| Filename | Description |
|---|---|
dcs_export_udp.py |
Acquires in-game API data via UDP protocol, integrated with DCS-BIOS |
dcs_export_screen.py |
Captures data from specified screen regions |
dcs_export_hid.py |
Collects and saves HOTAS peripheral data |
retrieval_picture.py |
Implements multimodal RAG, retrieving semantically relevant images and descriptions |
dcs_memory.py |
Central data structure for unified storage of all above data |
FalconAgent is built upon Textual DCS and our unique bidirectional feedback mechanism, enabling end-to-end human decision support (from abstract tasks to concrete operations). Its main functionalities include:
| Filename | Description |
|---|---|
planner_global.py |
Generates operation sequences for crew members |
planner_schedule.py |
Optimizes operation sequences, including redundancy removal and multi-crew parallelization |
retrieval_data.py |
Agent retrieves the most relevant data for the current step via Textual DCS |
rag_test.py |
Supports FalconAgent in retrieving relevant documents and generating low-hallucination data descriptions |
Specially designed for the F-14A model, supporting dual-seat operation. Experiments require code tailored for multi-crew scenarios.
| Filename | Description |
|---|---|
main_multi.py |
Launches multi-crew environment, enabling data exchange between crew members over the network |
dcs_multiplayer.py |
Defines multiplayer communication protocols |
Files for launching and running the FalconCopilot system. Detailed configuration parameters can be found in the source code.
| Filename | Description |
|---|---|
main.py |
Starts the FalconCopilot system; startup mode can be adjusted via comments |
script/eval.ps1 |
Batch evaluation script for testing models on the test set |
ui.py |
Provides an interactive interface between FalconAgent and the user at runtime |
For quantitative evaluation of FalconCopilot’s data retrieval and bidirectional feedback capabilities using custom datasets.
| Filename | Description |
|---|---|
evaluation.py |
Evaluates data retrieval accuracy and feedback capability; supports LLMs simulating human feedback |
eval_ui.py |
Introduces real human feedback during evaluation, providing an interactive interface |
| Filename | Description |
|---|---|
model.py |
Custom interface for LLM invocation |
utils.py |
Data processing tools, e.g., automated extraction of high-level operations from manuals |
draw_task_diff.py |
Visualizes the effect of the pruning module on redundant task sequences |
config.ini |
Stores API keys for various LLMs |
For further details on data preparation, system architecture, or usage, please refer to the appendix of the paper and the source code comments.