An engineering prototype for detecting coffee-fruit maturity, estimating relative depth, and sending structured observations to a robot or mock controller.
中文说明 · Evaluation · Architecture
- YOLOv5 detection for six coffee-fruit classes:
diseased,dry,overripe,ripe,semi_ripe, andunripe. - MiDaS-based relative-depth estimation for detections inside a configurable region of interest.
- A PyQt5 real-time camera application and a Streamlit image/camera demo.
- A newline-delimited JSON TCP protocol plus a mock robot server.
- A modular, tested codebase separated into inference, rendering, configuration, protocol, and UI layers.
The supplied checkpoint was re-evaluated on the 99-image held-out test split (1,561 labeled instances) at 640 px on CPU:
| Precision | Recall | mAP@0.5 | mAP@0.5:0.95 |
|---|---|---|---|
| 0.650 | 0.282 | 0.229 | 0.143 |
These are honest test-set results from the included project checkpoint, not copied training claims. Performance is strongly class-imbalanced; see the full evaluation and limitations.
Requires Python 3.8+.
python -m venv .venv
# Windows: .venv\Scripts\activate
# macOS/Linux: source .venv/bin/activate
pip install -r requirements.txt
pip install -e .Download best.pt and place it at weights/best.pt, then copy the example configuration:
mkdir weights
# place best.pt in weights/
copy config.example.yaml config.yaml # WindowsThe first inference run uses torch.hub to fetch YOLOv5 and MiDaS model code/weights, so an internet connection is required.
streamlit run apps/web.pypython -m apps.desktop --config config.yamlIn one terminal:
python -m scripts.mock_robot --host 127.0.0.1 --port 8000Then enable robot transmission in either UI. Each detection is sent as one JSON line containing class, image coordinates, confidence, and normalized relative depth.
MiDaS is used here for monocular relative depth. The value is normalized per frame and is not a distance in meters, a calibrated world coordinate, or a complete 3D robot pose. Metric positioning would require camera calibration and a suitable depth/geometry pipeline.
I assembled the coffee dataset workflow, trained the six-class detector checkpoint, and developed the vision-to-robot prototype. For this public portfolio version I refactored the original experimental scripts into testable modules, added configuration and two user interfaces, defined a robust newline-delimited protocol, added a mock server, and corrected the depth representation so it does not imply uncalibrated metric distance.
The dataset is published on Roboflow Universe under CC BY 4.0. YOLOv5, MiDaS, PyTorch, Qt, and other dependencies remain subject to their own licenses. Repository source code is MIT licensed.
Datasets, virtual environments, generated runs, and model binaries are intentionally excluded from Git history. The checkpoint is distributed as a GitHub Release asset.