AccessJEPA

AccessJEPA is a work-in-progress research repository.

This project explores how to build a visual world model for accessibility-aware navigation.
The practical objective is a classifier that can distinguish an accessible entry from a non-accessible entry.

---

Why this project exists

For navigation agents to be useful in real environments, they need to reason about accessibility constraints (steps, stairs, ramps, support structures, and potentially more nuanced cues later).
This repository is my experimentation space for that idea:

• learn visual representations from scene images with self-supervised learning,
• test whether those representations support accessibility classification,
• iterate from simpler to richer pretraining objectives.

---

Research storyline (VICReg -> I-JEPA)

The repository intentionally preserves the progression of the work:

1. VICReg first (training/main.py, training/cfgs/default.yaml)
   Started with a joint-embedding SSL baseline to test whether augment-based invariance can produce useful accessibility features.

2. Move to I-JEPA / IJEPA (training/ijepa_main.py, training/cfgs/ijepa.yaml)
   Shifted to predictive latent modeling with target masking: a context encoder predicts masked target representations generated by an EMA target encoder.

3. Current direction
   Use learned embeddings as the foundation for accessibility classification and, later, for a broader accessibility-aware world model.

---

Core concepts in this repo

• World model motivation: representations should encode actionable accessibility cues for navigation decisions.
• Target masking (I-JEPA): train by predicting latent features of masked image regions from visible context.
• Accessibility supervision: labels are used through linear probes / downstream classifiers to test representation quality.
• Exploration artifacts: selected crop-check visuals are kept in reports/bb_crop_check/ to document masking and target-region inspection.

---

Repository map

AccessJEPA/
├── training/
│   ├── main.py               # VICReg pipeline
│   ├── ijepa_main.py         # I-JEPA pipeline
│   ├── ijepa_model.py        # ViT encoder, predictor, masking, EMA target encoder
│   ├── dataset.py            # Data loading and transforms
│   ├── eval.py               # Probe/evaluation logic
│   └── cfgs/
│       ├── default.yaml      # VICReg config
│       └── ijepa.yaml        # I-JEPA config
├── app.py                    # Streamlit data exploration app
├── analyze_dataset.py        # Annotation/dataset analysis
├── export_accessibility_csv.py
├── reports/
│   ├── wm_annotations_summary.md
│   ├── wm_annotations_summary.json
│   └── bb_crop_check/        # Small curated visual checks for masking exploration
└── eb_jepa/                  # External module/submodule used by experiments

---

Quick start

Install dependencies

pdm install

Analyze annotations

python analyze_dataset.py

Run VICReg experiments

python -m training.main --fname training/cfgs/default.yaml

Run I-JEPA experiments

python -m training.ijepa_main --fname training/cfgs/ijepa.yaml

Run the Streamlit app

streamlit run app.py

---

Data and labels

Current experiments use image-level labels related to accessibility barriers, including:

• has_step
• has_stair
• has_grab_bar
• has_ramp

The long-term target remains the binary decision boundary: accessible entry vs non-accessible entry.

---

Current status and limitations

• The codebase is in active iteration and APIs/configs may change.
• Experiment tracking, metrics, and folder conventions are still being cleaned up.
• Results should be treated as exploratory, not benchmark-ready.
• This repository prioritizes transparency of research thinking over polished production packaging.