SkyWrite ML Lab

An independent, portfolio-ready machine-learning repository for studying intent-aware webcam air writing. It does not modify or deploy the hackathon-submitted SkyWrite application.

The research problem has two separate parts:

1. Pen intent: for every webcam frame, predict whether the learner intends to draw (DRAW) or is repositioning (MOVE).
2. Whole-character validation: determine whether the cleaned, ordered strokes plausibly match the character assigned to the learner.

The Level 4 simulator combines these tasks with a 250 ms display buffer, hysteresis, smoothing, clutter rejection, and a 15-second inactivity reset. This makes model changes measurable before they are considered for any product.

Why this is a strong ML project

• Participant-disjoint evaluation prevents a writer's style leaking into validation.
• Every experiment is driven by a versioned YAML configuration.
• Private recordings and trained weights are excluded from Git.
• Offline replay measures frame classification and product behavior, not accuracy alone.
• The code separates learned behavior from transparent safety rules.

Repository map

configs/                 Versioned experiment settings
docs/                    Architecture, data card, model card, experiment log
scripts/                 Training and benchmark entry points
src/skywrite_ml/         Reusable data, model, metric, and Level 4 code
tests/                   Synthetic tests that contain no participant data
data/private/            Local-only recordings (git-ignored)
artifacts/               Local-only checkpoints and exports (git-ignored)

Quick start

python3 -m venv .venv
source .venv/bin/activate
pip install -e '.[dev]'
pytest

Copy validated collector JSON files into data/private/raw, or point --data at an existing private corpus. Never commit participant recordings or Blob credentials.

python scripts/train_intent.py --data data/private/raw --output artifacts/intent.pt
python scripts/benchmark_level4.py --data data/private/raw --checkpoint artifacts/intent.pt

Experimental method

The deterministic baseline split is 60% training, 20% development, and 20% test by participant, not by frame or recording. Development writers select checkpoints and thresholds; test writers are reserved for final reporting. Report precision, recall, F1, false-draw rate, transition delay, fragmented-stroke rate, and clutter-clear rate. A new model should replace a baseline only when it improves untouched test writers without increasing false marks.

See Architecture, Data Card, Model Card, and Experiment Guide.