SLIDERS lets you reshape image search results through visual concepts learned directly from the embedding space.
Upload a query image, inspect the visual features activated by the model, and move semantic sliders to push results toward or away from specific concepts. Each steering axis is discovered by a sparse autoencoder and named automatically with a local vision-language model.
SLIDERS combines DINOv2, sparse autoencoders, FAISS and optional dual-index reranking to make image retrieval more controllable and easier to inspect.
The query panel exposes learned visual axes that can be adjusted before retrieval.
Traditional image retrieval systems return similar images but usually provide little control over why those results were selected.
SLIDERS exposes interpretable directions learned from the representation space. Instead of replacing the image query with text, the user can preserve the original query and modify individual visual properties through sliders.
Query image
↓
DINOv2 encoder
↓
Normalized image embedding
↓
Sparse autoencoder features
↓
Semantic slider steering
↓
FAISS and optional SAE retrieval
↓
Activation-aware reranking
↓
Image results
The main retrieval path works as follows:
- DINOv2 converts the query image into a normalized embedding.
- A sparse autoencoder exposes learned visual features.
- A local VLM assigns readable names to the most relevant features.
- Slider values move the query along selected SAE decoder directions.
- FAISS retrieves candidate images from the steered query.
- Optional SAE retrieval and activation-based reranking refine the results.
The repository also includes an experimental patch-level pipeline with DINOv2 register tokens and MaxSim late interaction for region-level retrieval.
| Area | Included |
|---|---|
| Visual backbone | DINOv2 CLS and patch embeddings |
| Interpretability | Sparse autoencoder feature discovery |
| Feature naming | Local vision-language model |
| Retrieval | FAISS similarity search |
| Steering | Positive and negative semantic directions |
| Reranking | SAE activation-aware scoring |
| Patch retrieval | MaxSim late interaction |
| Interface | Interactive visual search UI |
| Evaluation | Retrieval and feature analysis pipeline |
git clone https://github.com/Andrei-Stefan20/SLIDERS.git
cd SLIDERS
python -m venv .venvActivate the environment and install the dependencies.
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
.\scripts\run_pipeline.ps1The orchestrator can extract embeddings, train the SAE, name features, build the indexes, evaluate the retrieval pipeline and launch the interface.
For dataset preparation, GPU-specific installation instructions and individual pipeline stages, read the full run guide.
Feature inspection
Click an axis to inspect the images that activate it most and least.
Result inspection
Open a result at full resolution, copy its path or download the image.
SLIDERS is a research prototype focused on interpretable retrieval and query-time feature steering. The codebase includes both the stable CLS-based path and experimental patch-level retrieval components.
The current default example uses PlantVillage. The pipeline is configurable and can be adapted to other image collections by providing a dataset configuration and the corresponding files.
| Page | Content |
|---|---|
| Run guide | Setup, dataset preparation and full pipeline |
| Architecture | Runtime flow, modules and generated artifacts |
| Configuration | YAML configuration options |
| Troubleshooting | Known failure modes and fixes |
Released under the MIT License.