21 add models

README.md

@@ -8,7 +8,18 @@
 
 ## Documentation
 
-**The documentation for Orion is available [here](https://jonasrenault.github.io/orion/).**
+**Checkout Orion's [documentation](http://jrenault.fr/orion/) for help.**
+
+## Models
+
+Orion provides [YOLO12](https://docs.ultralytics.com/models/yolo12/) models fine-tuned on a [custom dataset](http://jrenault.fr/orion/datasets) of military vehicles with [4 classes](http://jrenault.fr/orion/classes).
+
+| Model                                                                                  | size<br><sup>(pixels) | params<br><sup>(M)   |
+| ------------------------------------------------------------------------------------   | --------------------- | -------------------- |
+| [orion12n](https://github.com/jonasrenault/orion/releases/download/v2.0.0/orion12n.pt) | 640                   | 2.6                  |
+| [orion12s](https://github.com/jonasrenault/orion/releases/download/v2.0.0/orion12s.pt) | 640                   | 9.3                  |
+| [orion12m](https://github.com/jonasrenault/orion/releases/download/v2.0.0/orion12m.pt) | 640                   | 20.2                 |
+| [orion12l](https://github.com/jonasrenault/orion/releases/download/v2.0.0/orion12l.pt) | 640                   | 26.4                 |
 
 ## Installation
 
@@ -42,15 +53,41 @@ When you install Orion in a virtual environment, it creates a CLI script called
 orion --help
 ```
 
-to see the various commands available (or take a look at the [documentation](https://jonasrenault.github.io/orion/) for examples).
+to see the various commands available (or take a look at the [documentation](http://jrenault.fr/orion/) for examples).
+
+#### Detect military vehicles in images
+
+The `predict` command will use one of orion's models to detect military vehicles in images. For example, if you downloaded the [orion12m.pt](#models) model file into your current directory, and want to use it to detect military vehicles in an image, run
+
+```bash
+orion predict ./orion12m.pt resources/test/afvs.jpg -s
+```
+
+The `predict` command with the `-s` option will save the annotated image:
+
+<div align="center">
+  <img src="docs/imgs/afvs.jpg" width="640"/>
+</div>
+
+#### Track military vehicles in videos
+
+The `track` command will use one of orion's models to track military vehicles in videos. For example, if you downloaded the [orion12m.pt](#models) model file into your current directory, and want to use it to track military vehicles in a video, run
+
+```bash
+orion track ./orion12m.pt resources/test/mev1.mp4
+```
+
+<div align="center">
+  <img src="docs/imgs/mev_tracking.gif" width="640"/>
+</div>
 
 ## Contents
 
 - The [orion](./orion/) directory contains the source code used to fetch and format datasets for fine-tuning a YOLO12 model for object detection.
 - The [resources](./resources/) directory contains video samples for vehicle detection task.
 - The [notebooks](./notebooks/) directory contains exemple notebooks on how to
   1. [Prepare](./notebooks/01_Prepare.ipynb) a custom dataset of images annotated for automatic target recognition of military vehicles.
-  2. [Train](./notebooks/02_Train.ipynb) fine-tune a YOLO12 model using the prepared dataset.
+  2. [Train](./notebooks/02_Train.ipynb) a YOLO12 model using the prepared dataset.
   3. [Evaluate](./notebooks/03_Evaluate.ipynb) a fine-tuned model on a realistic test dataset.
 
 ## Run the notebooks

docs/classes.md

@@ -0,0 +1,14 @@
+# Classes
+
+Orion uses four broad classes of military vehicles that our model can discriminate:
+
+1. **Armoured Fighting Vehicle (AFV)**
+2. **Armoured Personnel Carrier (APC)**
+3. **Military Engineering Vehicle (MEV)**
+4. **Light Armoured Vehicle (LAV)**
+
+## Class repartition
+
+![Class repartition](imgs/labels.jpg)
+
+The AFV class is overrepresented in Orion's [dataset](./datasets.md) since most open source image datasets only label military vehicles as tanks.

docs/datasets.md

@@ -0,0 +1,127 @@
+# Datasets
+
+Orion uses images from open source object-detection datasets to create a dataset of military vehicles and format it correctly for [YOLO](https://github.com/ultralytics/ultralytics) training. This page describes the various datasets used by Orion.
+
+## ImageNet
+
+The first dataset Orion uses is `ImageNet21k`. The `ImageNet21k` dataset is available from the [image-net website](https://image-net.org/download-images.php). You need to register and be granted access to download the images. We use the Winter 21 version since it gives the option of downloading the images for a single synset (a class) and we're only interested in images of specific classes (military vehicles).
+
+* The processed version of ImageNet21k is available on the [ImageNet21k repository](https://github.com/Alibaba-MIIL/ImageNet21K).
+* The class ids and names are available in [this issue](https://github.com/google-research/big_transfer/issues/7#issuecomment-640048775).
+
+Orion provides a `search` function to search ImageNet class names for a given query.
+
+```python
+def search(
+    keywords: list[str],
+    dir: Path = settings.ORION_HOME_DIR / "imagenet",
+):
+    """
+    Search image net classes matching the given keywords.
+
+    Args:
+        keywords (list[str]): List of keywords to search for.
+        dir (Path, optional): directory where files will be downloaded.
+            Defaults to ORION_HOME_DIR / "imagenet".
+    """
+```
+
+!!! note
+    The `search` function will download the list of class names and ids in the dataset `dir` if they are not already present.
+
+Orion also provides a convenience `download` function to download images and annotations for a specific class id.
+
+```python
+def download(
+    ids: list[str],
+    dir: Path = settings.ORION_HOME_DIR / "imagenet",
+):
+    """
+    Download ImageNet images and annotations for the given class ids.
+
+    Args:
+        ids (list[str]): the class ids to download.
+        dir (Path, optional): the dataset directory.
+            Defaults to settings.ORION_HOME_DIR / "imagenet".
+    """
+```
+
+!!! note
+    The `download` function will only download images for classes that actually have object detection annotations (a lot of classes in the ImageNet21k dataset do not have annotations).
+
+!!! info
+    Orion uses **378 annotated images** from the ImageNet dataset, coming from the `n04389033` class (tank, army tank, armored combat vehicle, armoured combat vehicle), which are all mapped to the `AFV` class.`
+
+## OpenImage
+
+The second dataset Orion uses is [Open Images](https://storage.googleapis.com/openimages/web/index.html) which contains images with `Tank` detection labels. Images from OpenImages are downloaded and managed with [fiftyone](https://docs.voxel51.com/integrations/open_images.html).
+
+!!! info
+    Orion uses **1246 annotated images** from the OpenImage dataset which are all mapped to the `AFV` class.
+
+## Russian Military annotated dataset
+
+Another dataset Orion uses is the [Russian Military vehicles](https://universe.roboflow.com/capstoneproject/russian-military-annotated) annotated dataset provided by Tuomo Hiippala from Digital Geography Lab on Roboflow. It contains 1042 annotated images of russing military vehicles with 10 classes which we map to either the `AFC` or the `APC` class.
+
+```python
+LABEL_MAPPING = {
+    "bm-21": "AFV",
+    "t-80": "AFV",
+    "t-64": "AFV",
+    "t-72": "AFV",
+    "bmp-1": "AFV",
+    "bmp-2": "AFV",
+    "bmd-2": "AFV",
+    "btr-70": "APC",
+    "btr-80": "APC",
+    "mt-lb": "APC",
+}
+```
+
+Orion provides a `download` function to download the images and annotations from this dataset and structure the directory to be imported into a `fiftyone` dataset.
+
+```python
+def download(dir: Path = settings.ORION_HOME_DIR / "roboflow"):
+    """
+    Downlad images and annotations from the russian military annotated dataset
+    on roboflow and format them to be imported into a fo.Dataset.
+
+    Args:
+        dir (Path, optional): the dataset dir.
+            Defaults to settings.ORION_HOME_DIR / "roboflow".
+    """
+```
+
+!!! info
+    Orion uses **1042 annotated images** from the Russian Military vehicles dataset which are mapped to the `AFV` or `APC` class.
+
+## Google Images
+
+To improve our training dataset, we also scraped images of military vehicles from Google Image and annotated them by hand. This sample dataset is available for download from Orion's github repository and contains 669 images of vehicles from all four classes (`AFV`, `APC`, `MEV` and `LAV`).
+
+!!! info
+    Orion uses **669 annotated images** scraped from Google Images for all four classes.
+
+## The Search 2
+
+The [The Search_2](https://figshare.com/articles/dataset/The_Search_2_dataset/1041463) consist of 44 high-resolution digital color images of different complex natural scenes, with each scene (image) containing a single military vehicle that serves as a search target. This dataset is not used by Orion for training; it is used instead for evaluating the models on **realistic long range automatic target recognition (ATR) samples**.
+
+## Command-line
+
+Orion provides a CLI command to download and setup a dataset of annotated military vehicles for training and development of automatic target recognition models. The `prepare` command will download images from the [ImageNet](#imagenet), [OpenImages](#openimage), [Russian military](#russian-military-annotated-dataset) and [Google Images](#google-images) sources and combine them into a single dataset on disk.
+
+The `prepare` command takes as an option the directory where all the source images will be downloaded and where the full combined dataset will be saved (by default, `~/.cache/orion`).
+
+```bash
+orion prepare --help
+
+ Usage: orion prepare [OPTIONS]
+
+ Prepare a dataset of annotated military vehicle images.
+
+╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────╮
+│ --dir   -d      DIRECTORY  Orion home directory. [default: ~/.cache/orion]                              │
+│ --ids           TEXT       List of class ids to download. [default: n04389033]                          │
+│ --help                     Show this message and exit.                                                  │
+╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯
+```

docs/models.md

@@ -0,0 +1,98 @@
+# Models
+
+Orion provides [YOLO12](https://docs.ultralytics.com/models/yolo12/) models fine-tuned on a [custom dataset](datasets.md) of military vehicles with [4 classes](classes.md).
+
+| Model                                                                                  | size<br><sup>(pixels) | params<br><sup>(M)   |
+| ------------------------------------------------------------------------------------   | --------------------- | -------------------- |
+| [orion12n](https://github.com/jonasrenault/orion/releases/download/v2.0.0/orion12n.pt) | 640                   | 2.6                  |
+| [orion12s](https://github.com/jonasrenault/orion/releases/download/v2.0.0/orion12s.pt) | 640                   | 9.3                  |
+| [orion12m](https://github.com/jonasrenault/orion/releases/download/v2.0.0/orion12m.pt) | 640                   | 20.2                 |
+| [orion12l](https://github.com/jonasrenault/orion/releases/download/v2.0.0/orion12l.pt) | 640                   | 26.4                 |
+
+## Training results
+
+### Orion12n
+
+![Orion12n training results](trains/orion12n/results.png)
+
+### Orion12s
+
+![Orion12s training results](trains/orion12s/results.png)
+
+### Orion12m
+
+![Orion12m training results](trains/orion12m/results.png)
+
+### Orion12l
+
+![Orion12l training results](trains/orion12l/results.png)
+
+## Usage
+
+To use one of Orion's models, download it from the [links above](#models) and then use one of Orion's CLI commands:
+
+### Detect military vehicles in images
+
+The `predict` command will use the model to detect military vehicles in images.
+
+```bash
+orion predict --help
+
+ Usage: orion predict [OPTIONS] MODEL_PATH DATA
+
+ Run predictions on a set of images using the given model.
+
+╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
+│ *    model_path      PATH  model path. [required]                                                                                   │
+│ *    data            PATH  data to make predictions on. [required]                                                                  │
+╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
+╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
+│ --save       -s                               save annotated images.                                                                │
+│ --save-txt       --no-save-txt                save detection results in a txt file. [default: save-txt]                             │
+│ --save-conf      --no-save-conf               save confidence score for each detection. [default: save-conf]                        │
+│ --output     -o                    DIRECTORY  save directory. [default: Path.cwd() /runs/predict]                                   │
+│ --help                                        Show this message and exit.                                                           │
+╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
+```
+
+For example, if you downloaded the [orion12m.pt](#models) model file into your current directory, and want to use it to detect military vehicles in an image, run
+
+```bash
+orion predict ./orion12m.pt resources/test/afvs.jpg -s
+```
+
+!!! tip
+    The `predict` command with the `-s` option will save the annotated image.
+
+![Annotated AFVs](imgs/afvs.jpg)
+
+### Track military vehicles in videos
+
+The `track` command will use the model to track military vehicles in videos.
+
+```bash
+orion track --help
+
+ Usage: orion track [OPTIONS] MODEL_PATH DATA
+
+ Track tanks in a video using a YOLO model and specified tracker.
+
+╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
+│ *    model_path      PATH  model path. [required]                                                                                   │
+│ *    data            PATH  input video. [required]                                                                                  │
+╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
+╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
+│ --conf     -c      FLOAT      confidence threshold for detections. [default: 0.5]                                                   │
+│ --tracker  -t      TEXT       tracker configuration file. [default: botsort.yaml]                                                   │
+│ --output   -o      DIRECTORY  save directory. [default: Path.cwd() /runs/track]                                                     │
+│ --help                        Show this message and exit.                                                                           │
+╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
+```
+
+For example, if you downloaded the [orion12m.pt](#models) model file into your current directory, and want to use it to track military vehicles in a video, run
+
+```bash
+orion track ./orion12m.pt resources/test/mev1.mp4
+```
+
+![MEV tracking](imgs/mev_tracking.gif)