Akida 2 VWW Model zoo example + Claude generator skill after feedback

akida2/model_zoo/vww/README.md

@@ -0,0 +1,239 @@
+<img src="../../../docs/assets/0.-BC-dev-hub-LOGO-flicker.svg" alt="BrainChip Dev Hub" width="200"/>
+
+# Visual Wake Words (VWW) — Akida 2
+
+## Model Card
+
+Float accuracy: **TBD** &nbsp;|&nbsp; Parameters: **TBD**
+
+The quantized variants below all share the same float backbone. On Akida 2 the
+model is quantized with **`quantizeml`**: 8-bit weights and activations need no
+quantization-aware training (QAT), while a lower-precision 4-bit variant (4-bit
+weights and activations, 8-bit input layer) uses QAT to recover accuracy — 4-bit
+PTQ accuracy is poor, so only the QAT result is reported.
+
+<table>
+  <thead>
+    <tr>
+      <th>Variant</th>
+      <th>Weights / Acts</th>
+      <th>QAT</th>
+      <th>Quantized acc.</th>
+      <th>Akida acc.</th>
+      <th>Sparsity</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td>8-bit</td>
+      <td align="center">w8 / a8</td>
+      <td align="center">-</td>
+      <td align="center">TBD</td>
+      <td align="center">TBD</td>
+      <td align="center">TBD</td>
+    </tr>
+    <tr>
+      <td>4-bit</td>
+      <td align="center">w4 / a4</td>
+      <td align="center">yes</td>
+      <td align="center">TBD</td>
+      <td align="center">TBD</td>
+      <td align="center">TBD</td>
+    </tr>
+  </tbody>
+</table>
+
+**Akida 2 hardware benchmark (FPGA @ 25 MHz)**
+
+Latency is measured on the Akida 2 FPGA reference platform, which runs at
+**25 MHz**. A projected latency at a higher target clock is also shown to
+indicate expected performance on faster silicon. The cycle count is fixed for a
+given model and mapping regardless of clock rate, so the projection is an exact
+rescale of the measured cycles.
+
+> **Note:** the projected clock is **provisional** — it is a placeholder pending
+> confirmation of the target Akida 2 silicon clock. Power measurement on the FPGA
+> platform is still under development, so only latency is reported at this time.
+
+<table>
+  <thead>
+    <tr>
+      <th>Variant</th>
+      <th>Mapping</th>
+      <th>NPs</th>
+      <th>Passes</th>
+      <th>Cycles</th>
+      <th>Latency @ 25 MHz (ms)</th>
+      <th>Projected @ 100 MHz (ms) <i>(provisional)</i></th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td rowspan="2">8-bit</td>
+      <td>Minimal</td>
+      <td align="center">TBD</td>
+      <td align="center">TBD</td>
+      <td align="center">TBD</td>
+      <td align="center">TBD</td>
+      <td align="center">TBD</td>
+    </tr>
+    <tr>
+      <td>AllNPs</td>
+      <td align="center">TBD</td>
+      <td align="center">TBD</td>
+      <td align="center">TBD</td>
+      <td align="center">TBD</td>
+      <td align="center">TBD</td>
+    </tr>
+    <tr>
+      <td rowspan="2">4-bit (QAT)</td>
+      <td>Minimal</td>
+      <td align="center">TBD</td>
+      <td align="center">TBD</td>
+      <td align="center">TBD</td>
+      <td align="center">TBD</td>
+      <td align="center">TBD</td>
+    </tr>
+    <tr>
+      <td>AllNPs</td>
+      <td align="center">TBD</td>
+      <td align="center">TBD</td>
+      <td align="center">TBD</td>
+      <td align="center">TBD</td>
+      <td align="center">TBD</td>
+    </tr>
+  </tbody>
+</table>
+
+The model is a standard **AkidaNet** (from `akida_models`) with
+width multiplier **alpha = 0.25** and input resolution **96 × 96**, built for
+Akida 2.
+
+## Requirements
+
+For environment requirements and setup, see the [Requirements](../../../README.md#requirements)
+section of the top-level README.
+
+## Dataset
+
+Visual Wake Words is a binary image classification benchmark, specifically
+designed to target edge deployment on resource-constrained devices. It is
+derived from the MS-COCO 2014 dataset. Each image is labelled **person**
+or **non-person** based on whether a person occupies at least 2% of the frame.
+Images are resized to **96 × 96 RGB**. The dataset contains approximately
+115k training images and 8k validation images.
+
+Reference: Chowdhery et al., *Visual Wake Words Dataset* (2019),
+[arXiv:1906.05721](https://arxiv.org/abs/1906.05721).
+
+## Dataset setup
+
+The dataset can be downloaded from the SiLabs ML benchmarks mirror:
+
+```bash
+wget https://www.silabs.com/public/files/github/machine_learning/benchmarks/datasets/vw_coco2014_96.tar.gz
+tar -xzf vw_coco2014_96.tar.gz
+```
+
+The scripts default to looking for the data at `./data/vw_coco2014_96`. If you
+want to store the dataset on a dedicated data drive, you can pass the path
+explicitly to each script (see `--data` / `-d` in the individual scripts).
+Alternatively, it may be more convenient to keep the dataset in its preferred
+location and create a symbolic link from the default path (one-off step):
+
+```bash
+ln -s /path/to/your/data/vw_coco2014_96 ./data/vw_coco2014_96
+```
+
+This way the scripts work out of the box without any extra arguments.
+
+## Pipeline
+
+Training produces a float model, then quantizes it with `quantizeml` into
+several variants, each converted to Akida format:
+
+| Stage | Description |
+|---|---|
+| Full-precision | Float32 training from scratch |
+| 8-bit quantization | `quantizeml quantize` to 8-bit weights and activations (8-bit input); no QAT required |
+| 4-bit quantization | `quantizeml quantize` to 4-bit weights and activations (8-bit input), with QAT fine-tuning — 4-bit PTQ accuracy is poor so only the QAT model is kept |
+| Conversion to Akida | Automated conversion of each quantized model to Akida 2 format with `cnn2snn convert` |
+
+## Reference Models
+
+Pretrained models are made available here, within the `pretrained_models/`
+folder. However, those are handled using the `git-lfs` package (git large
+file storage). For those to be downloaded with the repo, you will need to
+set up `git-lfs`. For further instructions, see the
+[Trained models](../../../README.md#trained-models) section of the top-level README.
+
+## Usage
+
+### Notebook
+
+Two notebooks are provided that walk through a) preparation of a trained Akida-compatible model and
+b) evaluation and benchmarking of that model on Akida.
+
+[vww_notebook_training.ipynb](vww_notebook_training.ipynb) walks through the 
+complete training pipeline end-to-end. It is written to expose and explain the Akida-specific
+aspects of the workflow: how the model is constructed for Akida 2 compatibility,
+what the quantization constraints mean in practice, and what the conversion
+step does. Start here if you want to understand *why* the pipeline is structured
+the way it is.
+
+[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Brainchip-Inc/brainchip_devhub/blob/main/akida2/model_zoo/vww/vww_notebook_training.ipynb)
+
+[vww_notebook_benchmark.ipynb](vww_notebook_benchmark.ipynb) walks through 
+evaluation of model accuracy on Akida and, if a hardware device is available, covers benchmarking
+of model latency.
+
+> **Note:** the hardware benchmark section requires a physical Akida 2 FPGA
+> platform with a connected board.
+
+### Script
+
+For straightforward reproduction of the training and evaluation results, run
+the full pipeline in one shot:
+
+```bash
+bash vww_train.sh [DATADIR]
+```
+
+The optional `DATADIR` argument overrides the default dataset location
+(`./data/vw_coco2014_96`).
+
+## Contributing and Maintenance
+
+This README is autogenerated from `docs/README.md.template`
+so that the accuracy and hardware benchmark values are written directly 
+by the code (via the `metrics.json` file, also in the docs folder).
+
+When the associated model or training pipeline is modified to improve
+performance, you should rerun the evaluations of the float and quantized
+model versions, plus the hardware benchmark, including the 
+`--save-metrics` argument, and then regenerate the README from the template
+using `update_readme.py`:
+```bash
+# Float model
+python vww_eval.py -l pretrained_models/akidanet_vww.h5 --save-metrics
+
+# 8-bit variant
+python vww_eval.py -l pretrained_models/akidanet_vww_i8_w8_a8.h5 --save-metrics
+python vww_eval.py -l pretrained_models/akidanet_vww_i8_w8_a8.fbz --save-metrics
+python vww_benchmark.py -l pretrained_models/akidanet_vww_i8_w8_a8.fbz --save-metrics
+
+# 4-bit variant (QAT)
+python vww_eval.py -l pretrained_models/akidanet_vww_i8_w4_a4_qat.h5 --save-metrics
+python vww_eval.py -l pretrained_models/akidanet_vww_i8_w4_a4_qat.fbz --save-metrics
+python vww_benchmark.py -l pretrained_models/akidanet_vww_i8_w4_a4_qat.fbz --save-metrics
+
+python update_readme.py
+```
+Then commit the changed files (template, metrics and updated README).
+
+Likewise, if you want to edit the contents of this README, you should
+not edit it directly, but instead edit `docs/README.md.template` and 
+then regenerate the README using
+``` bash
+python update_readme.py
+```

akida2/model_zoo/vww/colab_setup.py

@@ -0,0 +1,75 @@
+"""Colab setup for the VWW (Akida 2) training notebook.
+
+IF YOU ARE RUNNING THIS LOCALLY: you can ignore this file completely.
+It exists solely to make the "Open in Colab" badge work, and does nothing
+on a normal local run. It is not part of the VWW model/training code
+(see vww_data.py, vww_model.py, vww_train.py for that).
+
+If you ARE on Colab, this is the file that gets you running:
+  - Clones this repo (skipping Git LFS smudge, since pretrained weights
+    aren't needed for the default training path)
+  - Points Python at the right folders (repo root for brainchip_utils,
+    this example's folder for vww_data / vww_model / vww_train)
+  - Installs akida_models, tf_keras, quantizeml
+  - Downloads and extracts the VWW dataset (derived from MS-COCO 2014,
+    hosted by Silicon Labs) if it isn't already present
+
+Called from the notebook's first cell like:
+    import colab_setup
+    colab_setup.setup()
+"""
+import os
+import subprocess
+import sys
+
+REPO_URL = 'https://github.com/Brainchip-Inc/brainchip_devhub.git'
+REPO_DIR = 'brainchip_devhub'
+EXAMPLE_SUBDIR = 'akida2/model_zoo/vww'
+
+DATA_DIR = './data/vw_coco2014_96'
+DATASET_URL = ('https://www.silabs.com/public/files/github/machine_learning/'
+               'benchmarks/datasets/vw_coco2014_96.tar.gz')
+
+
+def _run(cmd):
+    print(f'$ {cmd}')
+    subprocess.run(cmd, shell=True, check=True)
+
+
+def setup():
+    """Set up a fresh Colab session to run this notebook. No-op if not on Colab."""
+    if 'google.colab' not in sys.modules:
+        print('Not running on Colab \u2014 nothing to do. (This step only matters '
+              'for Colab; local runs already have everything they need.)')
+        return
+
+    if not os.path.exists(REPO_DIR):
+        # Skip Git LFS smudge: pretrained weights aren't needed for the default
+        # (RUN_FLOAT_TRAINING = True) path, so avoid downloading them here.
+        os.environ['GIT_LFS_SKIP_SMUDGE'] = '1'
+        _run(f'git clone --depth 1 {REPO_URL} {REPO_DIR}')
+
+    os.chdir(os.path.join(REPO_DIR, EXAMPLE_SUBDIR))
+
+    # Repo root on sys.path for `brainchip_utils`; example folder for the
+    # local vww_data / vww_model / vww_train modules imported later in
+    # the notebook.
+    repo_root = os.path.abspath(os.path.join(os.getcwd(), '..', '..', '..'))
+    sys.path.insert(0, repo_root)
+    sys.path.insert(0, os.getcwd())
+
+    # Akida 2 quantization uses quantizeml (installed as a dependency of
+    # akida_models / cnn2snn, but pinned here for a clean Colab environment).
+    _run('pip install -q akida_models==1.14.0 tf_keras quantizeml')
+
+    if not os.path.exists(DATA_DIR):
+        os.makedirs('./data', exist_ok=True)
+        _run(f'wget -q {DATASET_URL}')
+        _run('tar -xzf vw_coco2014_96.tar.gz -C ./data')
+        print('Dataset downloaded and extracted to', DATA_DIR)
+    else:
+        print('Dataset already present at', DATA_DIR)
+
+    print(f'Colab setup complete. Working directory: {os.getcwd()}')
+    print('If TensorFlow was just installed/upgraded, restart the runtime '
+          '(Runtime > Restart session) and re-run this cell before continuing.')

akida2/model_zoo/vww/data/.gitignore

@@ -0,0 +1,4 @@
+# Git to Ignore everything in this directory
+*
+# Except this .gitignore file
+!.gitignore