FlyCo: Foundation Model-Empowered Drones for Autonomous 3D Structure Scanning in Open-World Environments

Chen Feng^1,*, Guiyong Zheng^2,*, Tengkai Zhuang³, Yongqian Wu³, Fangzhan He³, Haojia Li¹, Juepeng Zheng², Shaojie Shen¹, Boyu Zhou³

  ¹HKUST Aerial Robotics Group    ²SYSU    ³SUSTech STAR Lab   
^*Equal Contribution

📢 News

• [09/06/2026]: The released FlyCo codebase and development environment setup are now available.

📜 Introduction

This repository maintains the implementation of "FlyCo: Foundation Model-Empowered Drones for Autonomous 3D Structure Scanning in Open-World Environments", owned by Chen Feng and Guiyong Zheng.

    

FlyCo is a novel and principled system architecture that synergizes foundation models scene knowledge with drone flight skills for open-world aerial 3D structure scanning. Driven solely by simple human prompts, it achieves fully autonomous, safe, and efficient scanning without the laborious manual intervention required by existing methods, filling critical gaps in open-world practicality.

    

    

Please cite our paper if you use this project in your research:

@article{feng2026flyco,
  title={FlyCo: Foundation Model-Empowered Drones for Autonomous 3D Structure Scanning in Open-World Environments},
  author={Feng, Chen and Zheng, Guiyong and Zhuang, Tengkai and Wu, Yongqian and He, Fangzhan and Li, Haojia and Zheng, Juepeng and Shen, Shaojie and Zhou, Boyu},
  journal={arXiv preprint arXiv:2601.07558},
  year={2026}
}

License Notice: This project is released under the PolyForm Noncommercial License 1.0.0 and is intended for non-commercial use only; commercial use is not permitted without explicit permission from the copyright owners. Please see LICENSE for details.

Please kindly star ⭐️ this project if it helps you. We take great efforts to develop and maintain it 😁.

🛠️ Installation

Note: Please make sure you have at least 150 GB of free disk space before installation.

First, clone this repository:

git clone git@github.com:FC-Family/FlyCo.git
cd FlyCo

Then, please refer to the following module-specific instructions for step-by-step setup details:

• perception_predictor
• planner_simulator

Finally, download the UE simulation environments from Google Drive.

🚀 Quick Start

• Open a demo scenario

Launch UE and open the scenario you want to test from Content/scenarios/${DEMO_SCENARIO}/Maps:

Available Demo Scenarios:

arc	bahai	buddha	castle
christ	church	outpost	pagoda
pisa_cathedral	sacre_coeur	sant	schloss
statue	temple	tower	windmill

• Start the full FlyCo stack

cd scripts && ./run.sh -s ${DEMO_SCENARIO}

For example:

./run.sh -s bahai

A joystick is recommended for triggering the system:

1. Press the "LB" button to initialize all modules.
2. Press the "Y" button to start initial flight.
3. In the FlyCo-UI window, specify the target structure by using the middle mouse button to either drag a bounding box around it, or by clicking points on it with the left mouse button. Then, click send points or press d to publish the prompt. Afterwards, you can enter a textual prompt in the text box and similarly click send text to publish it.

4. Press the "X" button to enable all services.
5. Press the "RB" to start autonomous scanning flight.

• Stop all FlyCo services after the task is completed

cd scripts & ./kill.sh

🤗 FC-Family Works

1. What is FC-Family?

We aim to develop intelligent active perception flight to realize Fast and reliable Coverage / sCanning / reConstruction / inspeCtion etc.

2. Projects list

• PredRecon (ICRA2023): Prediction-boosted Planner for Aerial Reconstruction.
• FC-Planner (ICRA2024): Highly Efficient Global Planner for Aerial Coverage.
• SOAR (IROS2024): Heterogenous Multi-UAV Planner for Aerial Reconstruction.
• FC-Vision (RA-L2026): Online Visibility-aware Replanning for Occlusion-free Aerial Scanning.
• FlyCo: Complete and Prompt-Driven System for Open-World Aerial 3D Structure Scanning.