Sorting - #2
Closed
Crasher100 wants to merge 61 commits into
Closed
Conversation
Co-authored-by: JK <JulianKropp@users.noreply.github.com>
Co-authored-by: JK <JulianKropp@users.noreply.github.com>
Co-authored-by: JK <JulianKropp@users.noreply.github.com>
…t, and enhance Dockerfile setup
…e; update requirements for psutil
Collaborator
Author
|
@copilot resolve the merge conflicts in this pull request. We want to accept everything from sorting and overwrite main |
There was a problem hiding this comment.
Pull request overview
This PR restructures the project into a more complete end-to-end “AI-Sorter” pipeline: dataset capture/labeling, model training in Docker, and Raspberry Pi runtime control loop integrating camera + Coral inference + actuators.
Changes:
- Removed legacy/experimental scripts (old utils, camera/websocket wrappers, serial/GPIO tests, FLIR wrappers).
- Added new runtime modules (actuator controllers, PySpin camera wrapper, Tk display) and a new
main.pysorting loop. - Added documentation and a Docker-based training pipeline (TF Object Detection API → INT8 quantization → Edge TPU compilation).
Reviewed changes
Copilot reviewed 64 out of 143 changed files in this pull request and generated 19 comments.
Show a summary per file
| File | Description |
|---|---|
| utils/FLIR.py | Removed legacy FLIR helper. |
| utils/CameraWebsocketHandler.py | Removed legacy Tornado websocket handler. |
| utils/BiQuad.py | Removed legacy DSP filter code. |
| utils/init.py | Removed legacy mean_arr helper. |
| testneopixel.py | Removed standalone NeoPixel test script. |
| testgpiopin.py | Removed standalone GPIO PWM test script. |
| spinnaker_wrapper.cpp | Removed unused C++ Spinnaker wrapper. |
| socketserial.py | Removed serial↔socket bridge script. |
| requiremetns_freeze.txt | Removed old frozen requirements list. |
| python3.10.sh | Removed old startup script. |
| inference_process.py | Removed old inference script. |
| capture_process.py | Removed old capture script. |
| diagnoseserial.py | Removed serial diagnostics script. |
| newcoral.py | Removed old Coral test script. |
| FLIR/FLIR.py | Removed legacy FLIR module. |
| FLIR/init.py | Removed legacy FLIR package init. |
| old/updatet.py | Removed legacy pipeline script. |
| old/updatet2.py | Removed legacy pipeline script. |
| old/updatet3.py | Removed legacy pipeline script. |
| old/update4_camera.py | Removed legacy camera script. |
| old/update5_help.py | Removed legacy helper script. |
| old/try.py | Removed legacy import test. |
| old/test.py | Removed legacy wrapper test. |
| old/sorter.py | Removed legacy sorter prototype. |
| test_mixed.py | Added a local TFLite test script (currently with absolute paths). |
| test_coral.py | Added Coral inference smoke test script. |
| sensor/requirements-3.10.txt | Added sensor module Python deps. |
| sensor/README.md | Added sensor module setup docs. |
| sensor/out/.gitignore | Ignore captured output images in sensor/out/. |
| sensor/light_beam.py | Added MCP3008 light-beam test logger. |
| sensor/light_beam_2.py | Added alternative light-beam logging variant. |
| sensor/dockerfile | Added Dockerfile to build a sensor runtime image with Spinnaker SDK. |
| sensor/camera.py | Added PySpin camera wrapper with streaming + buffer flush utilities. |
| sensor/.python-version | Added Python version pin for sensor module. |
| sensor/.gitignore | Added sensor-local ignores for SDK artifacts/tarballs. |
| sensor/.dockerignore | Added sensor Docker ignore rules. |
| actuator/requirements-3.10.txt | Added actuator module Python deps. |
| actuator/README.md | Added actuator module setup docs. |
| actuator/led_neopixel.py | Added NeoPixel controller module + test harness. |
| actuator/elevator_motor.py | Added stepper elevator motor controller + test harness. |
| actuator/switch_solenoid_motor.py | Added solenoid controller module + test harness. |
| actuator/.python-version | Added Python version pin for actuator module. |
| display.py | Added fullscreen Tk display frontend for detections. |
| main.py | Added main sorter control loop integrating camera, TPU inference, and actuators. |
| create_unlabeled_dataset.py | Added dataset capture script (camera stream + filter + save). |
| label_dataset.py | Added auto-labeling script to generate Label Studio JSON tasks. |
| classify_mixed.py | Added batch classification helper for mixed/unlabeled images. |
| http-server.py | Added simple dataset HTTP server for Label Studio import. |
| requirements-3.10.txt | Added root requirements guidance for PC inference deps. |
| mypy.ini | Added mypy configuration. |
| .python-version | Added repo-level Python version pin. |
| .gitignore | Replaced placeholder with full ignore set (notably includes label_map.pbtxt). |
| README.md | Added new project-level docs and workflow overview. |
| docs/00-build-the-sorter.md | Added placeholder build guide. |
| docs/01-raspberry-pi-setup.md | Added Raspberry Pi setup guide. |
| docs/02-training-pc-setup.md | Added training PC setup guide. |
| docs/03-create-dataset.md | Added dataset creation + Label Studio guide. |
| docs/04-train-model.md | Added training pipeline guide. |
| docs/05-run-sorter.md | Added runtime guide. |
| pipeline.config | Added TF Object Detection API training config. |
| Dockerfile.train | Added end-to-end training Docker image definition. |
| docker/build_tfrecords.py | Added Label Studio JSON → TFRecord converter for container use. |
| docker/quantize.py | Added SavedModel → INT8 TFLite quantization script. |
| docker/entrypoint.sh | Added end-to-end training entrypoint script. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+64
to
+67
| # ---- Repo-level config + helper scripts -------------------------------------- | ||
| COPY pipeline.config /opt/sorter/pipeline.config | ||
| COPY label_map.pbtxt /opt/sorter/label_map.pbtxt | ||
| COPY docker/build_tfrecords.py /opt/sorter/build_tfrecords.py |
Comment on lines
+62
to
+65
|
|
||
| cp "$OUT_DIR/ssd_mobilenet_v2_quant_edgetpu.tflite" "$OUT_DIR/marbel_coral.tflite" | ||
| awk '{gsub(/\r/,"")} /id:/{n=$2} /name:/{gsub(/\047/,"",$2); print n, $2}' "$ASSETS/label_map.pbtxt" > "$OUT_DIR/labels.txt" | ||
|
|
Comment on lines
+261
to
+265
| # Solenoid ON → deflect to GREEN side (left) | ||
| # Solenoid OFF → marble falls to RED side (right, default) | ||
| display.update_detection(frame_bgr, label, confidence) | ||
| marble_shown = True | ||
|
|
Comment on lines
+38
to
+42
| def cleanup(self) -> None: | ||
| """Disables the motor and safely releases the GPIO pins.""" | ||
| self.disable() | ||
| GPIO.cleanup([self.step_pin, self.dir_pin]) | ||
|
|
Comment on lines
+154
to
+157
| # Initialize background subtractor BEFORE the loop | ||
| backSub = cv2.createBackgroundSubtractorMOG2(history=20, varThreshold=50, detectShadows=False) | ||
| kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5)) | ||
|
|
Comment on lines
+6
to
+9
| class CORSRequestHandler(SimpleHTTPRequestHandler): | ||
| def end_headers(self): | ||
| self.send_header('Access-Control-Allow-Origin', '*') | ||
| super().end_headers() |
Comment on lines
+11
to
+14
| dataset_dir = Path(__file__).parent / "dataset" | ||
| os.chdir(dataset_dir) | ||
|
|
||
| HTTPServer(('0.0.0.0', 1000), CORSRequestHandler).serve_forever() |
Comment on lines
+85
to
+86
| finally: | ||
| self.camera.EndAcquisition() |
Comment on lines
+140
to
+144
| frames_to_save: List[np.ndarray] = [] | ||
| frames_processed = 0 | ||
|
|
||
| try: | ||
| print("Turning LEDs on...") |
Comment on lines
+53
to
+57
| ├── label_dataset.py | ||
| ├── label_map.pbtxt # Single source of truth for class labels | ||
| ├── main.py # Entry point for the sorter | ||
| ├── Dockerfile.train # Docker image for training | ||
| └── pipeline.config # SSD MobileNet V2 training config |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.