Learning computer vision with OpenCV and MediaPipe, from classic rule-based detection to deep-learning-based landmark tracking. Part of my path toward robotics and embodied AI.
- Contour detection —
findContours, area filtering. Detection is only as good as the binary input ("garbage in, garbage out"). - Bounding boxes —
boundingRect, the universal(x, y, w, h)format shared across face, contour, and person detection. - Threshold & blur — Gaussian blur, Otsu thresholding, and why background-dependent rules break down in the real world.
- Person detection (HOG) —
HOGDescriptor, and its failure modes (false positives and false negatives).
- Hand tracking — 21 landmarks, robust to rotation and occlusion.
- Pose estimation — 33 body landmarks, a foundation for imitation learning in robotics.
- HOG vs MediaPipe — a head-to-head showing why learned models beat hand-tuned rules on hard inputs (odd poses, low contrast).
Refactored detection scripts into HandDetector / PoseDetector
with a shared interface (detect, draw, bounding_box), so the
main loop stays the same when swapping detectors.
A small interactive project: hands define a region, raised fingers select real-time filters (pixelate, thermal, edge, cartoon), composited with alpha blending. Explores the sense → decide → act loop.
The first step toward a person-following robot: detect one person and decide where they are and how far, using only a webcam. This is the sense → decide half of a robot control loop (the act half needs hardware, and comes later).
In scope (this is "enough"):
- Detect a single person (reusing the pose / bounding-box detector)
- Compute the box center → classify horizontal position:
LEFT/CENTER/RIGHT - Compute the box height → classify distance:
NEAR/FAR - Overlay the decision as text + draw the center point for visual debugging
Out of scope (deliberately excluded):
- Multiple people
- Re-identification (recognizing the same person after they leave frame)
- Motor / robot control
- Recording, fancy UI, filters
Positions use ratios of the frame, not fixed pixels, so it works at any resolution:
| Signal | Rule | Result |
|---|---|---|
| center x < 33% width | left third | LEFT |
| center x > 66% width | right third | RIGHT |
| otherwise | middle | CENTER |
| box height > 60% frame height | large box = close | NEAR |
| otherwise | FAR |
Height (not width) is used for distance because body height stays stable while arm movements change width. The thresholds (33%, 60%) are tunable.
- v0.1 — position + distance from a webcam (current)
- v0.2 — smooth the signal over time (reduce jitter)
- Later — connect to hardware for actual following
py -3.12 -m venv env
.\env\Scripts\Activate.ps1
pip install mediapipe==0.10.21 opencv-python numpyRun any detector through the virtual environment:
python main.pyPython 3.12 · OpenCV · MediaPipe · NumPy