CLIP Exit-Site Presence Binary Classification

This repository contains a minimal feasibility pipeline for:

• frozen CLIP image encoder (openai/clip-vit-base-patch32)
• embedding extraction
• logistic regression linear probe
• binary task: exit-site present vs exit-site absent/random

The goal is to test whether there is usable signal before investing in heavier model training.

Target and Decision Rule

• Task: reject random or non-exit-site photos while keeping exit-site photos.
• Positive class: exit-site visible (label=1).
• Negative class: non-exit-site (label=0) with three negative groups:
  • hard: skin redness/wound/scar but not exit-site
  • medium: normal abdominal skin
  • easy: non-medical images

Go/no-go criterion:

• If recall (positive class) is >= 0.8: signal is likely learnable; proceed to stronger models (fine-tuned CNN or detection model).
• If recall is < 0.8: refine dataset definition and negative sampling first; do not switch model family yet.

Data Layout

Expected working directories:

• data/positive/
• data/negative/hard/
• data/negative/medium/
• data/negative/easy/

The preparation script will generate:

• data/manifest.csv
• data/summary.json

Dataset Sources

Primary positive source (local):

• ../ntuh-pd-exit-site-classification/dataset

Suggested public sources for negatives (manual download + local unpack):

• hard negatives:
  • CO2Wounds-V2
  • WoundsDB
  • ISIC public collections
• medium negatives:
  • Abdominal Skin Segmentation Dataset (Apache-2.0)
• easy negatives:
  • CIFAR-10 (auto-downloaded by script through torchvision)

Always verify license terms for each downloaded dataset and keep citation/attribution records.

Setup

python3 -m venv .venv
source .venv/bin/activate
pip install -U pip
pip install -r requirements.txt

1) Build Dataset

You can build any size >=100. For fast feasibility, start with 200.
For stronger stability, use an expanded set (for example 480).

Example:

python scripts/prepare_mvp_dataset.py \
  --positive-source ../ntuh-pd-exit-site-classification/dataset \
  --hard-source /path/to/hard_negatives \
  --medium-source /path/to/medium_negatives \
  --target-total 480 \
  --positive-ratio 0.45 \
  --easy-ratio 0.08 \
  --hard-ratio 0.75 \
  --seed 42

Notes:

• Easy negatives are downloaded from CIFAR-10 by default.
• If hard/medium paths are missing, the script still runs but prints shortages.

2) Run CLIP Linear Probe

python scripts/run_clip_linear_probe.py \
  --manifest data/manifest.csv \
  --test-size 0.2 \
  --seed 42 \
  --augment-copies 2 \
  --augment-noise-std 0.01

To visualize training accuracy in Weights & Biases (W&B), use SGD probe mode:

python scripts/run_clip_linear_probe.py \
  --manifest data/manifest.csv \
  --probe-trainer sgd \
  --probe-epochs 40 \
  --wandb-enable \
  --wandb-project ntuh-exit-site-clip-probe \
  --wandb-run-name expanded-aug-seed42

For local testing without uploading to W&B cloud, set:

--wandb-mode offline

3) Save Model Bundle and Upload to Hugging Face

The script can save a deployable bundle and upload it directly.

python scripts/run_clip_linear_probe.py \
  --manifest data/manifest.csv \
  --test-size 0.2 \
  --seed 42 \
  --augment-copies 2 \
  --augment-noise-std 0.01 \
  --model-bundle-dir outputs/model_bundle \
  --hf-repo-id ruby0322/pd-exit-site-clip-linear-probe

Required auth for upload:

• hf auth login beforehand, or
• set HF_TOKEN in environment.

Uploaded bundle files:

• linear_probe.joblib
• bundle_config.json
• README.md

Outputs

The run script writes artifacts to outputs/:

• outputs/metrics.json
• outputs/confusion_matrix.png
• outputs/predictions.csv
• outputs/errors_false_negative.csv
• outputs/errors_hard_negative_fp.csv
• outputs/embedding_cache.npz
• outputs/train_curve.csv (epoch-level train_acc / test_acc)
• outputs/model_bundle/ (deployable probe bundle)

Focus on:

• false negatives (label=1, pred=0)
• hard negative failures (neg_group=hard, label=0, pred=1)

Current Reference Result (Expanded + Augmented)

With target-total=480, augment-copies=2, augment-noise-std=0.01, seed 42:

• accuracy = 0.9895833333333334
• recall_positive = 1.0
• false_negative_count = 0
• hard_negative_failure_count = 1

Production Decision

As of this update, the 98.9% model is approved as the current production pre-screen baseline.

• Production model repo:
  • ruby0322/pd-exit-site-clip-linear-probe
• Production URL:
  • https://huggingface.co/ruby0322/pd-exit-site-clip-linear-probe
• Decision basis:
  • accuracy = 0.9895833333333334
  • recall_positive = 1.0
  • false_negative_count = 0

Deployment note:

• Pin PRESCREEN_MODEL_REVISION in backend config for reproducibility.