Skip to content

DATA.md

Latest commit

 

History

History
79 lines (62 loc) · 3.04 KB

DATA.md

File metadata and controls

79 lines (62 loc) · 3.04 KB
Raw

Data & Weights

Datasets, checkpoints, and experiment outputs are not stored in the repo. The data/, weights/, and output/ entries are git-ignored and are usually symlinks to a data volume, e.g.:

ln -s /path/to/volume/SceneQ/data   data
ln -s /path/to/volume/SceneQ/output output
ln -s /path/to/volume/SceneQ/weights weights

All paths below are configured in configs/scannet200.yaml.

Model weights (weights/)

The image–text encoders (CLIP, SigLIP 2) and the reasoning VLM (Qwen2.5-VL) are pulled from the HuggingFace / OpenAI hubs at runtime and do not need to be placed here. Only these local checkpoints are required:

weights/
├── sam_vit_h_4b8939.pth                 # Segment Anything (ViT-H)
└── CropFormer_hornet_3x_03823a.pth      # CropFormer / EntitySeg 2D masks
Checkpoint Source
sam_vit_h_4b8939.pth https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth
CropFormer_hornet_3x_03823a.pth CropFormer / EntitySeg model zoo

yoloworld/openyolo3d (only used to reproduce the OpenYOLO3D baseline comparison, not Scene-Q's own method) need a YOLO-World checkpoint too — see the note in INSTALL.md.

ScanNet200 data layout (data/)

data/scannetv2/
├── input/
│   ├── scannetv2_images/val/<scene>/         # posed RGB-D
│   │   ├── color/        *.jpg                # RGB frames
│   │   ├── depth/        *.png                # depth (scale 1000)
│   │   ├── pose/         *.txt                # per-frame camera poses
│   │   └── intrinsics/   intrinsic_color.txt
│   ├── scannetv2_pcds/val/<scene>.pth         # reconstructed point clouds
│   └── mesh_segmentation/0.05_20/             # superpoint segments (.segs.json)
├── process_saved/
│   ├── 2d_seg/cropformer/<scene>/  *.png      # precomputed 2D masks
│   └── depth_from_pc/                         # synthetic depth (optional)
├── scannet200_gt/instance_gt/validation/      # GT instance .txt (for eval / oracle)
└── scannet_preprocess/meta_data/scannetv2_val.txt

Oracle masks

To reproduce the oracle-mask experiments (paper Table II), convert the ScanNet200 GT instances into .pth mask arrays:

python make_oracle.py \
    --gt-dir  data/scannetv2/scannet200_gt/instance_gt/validation \
    --out-dir output/scannet200/oracle/inst_seg_pcd

Outputs (output/)

The pipeline writes per-experiment results under:

output/scannet200/<model_3d>/
├── inst_seg_pcd/                              # 3D instance masks (input proposals)
└── sem_inst_mask_array/<sem_model>/
    ├── sem_mask_array/                        # per-instance predicted labels
    └── eval_result/result_sem_inst.txt        # AP / AP@50 / AP@25

where <model_3d> ∈ {oracle, mask3d, oneformer3d} and <sem_model> ∈ {clip, siglip2_so, siglip2_giant}.