Pushing the boundaries of what's possible when it comes to product exploration using vector search
Semantic similarity and vector embeddings are amazing, but hard to explain to a layperson. In this project, I attempt to show the power of vector embeddings by representing relevant products on a 3D plane, in a friendly and familiar explorable world.
How it works: A user is looking to purchase products for an upcoming outdoor adventure trip. They indicate where they are going, what they plan to do, the temperature, and their experience.
The user is then presented with a 3D explorable world (ThreeJS/React Three Fiber), where e-commerce categories are presented as mountains (with subcategories as hills around the mountains). The mountain/hill (category/subcategory) locations are plotted based on their relationship to each other (e.g. the gloves mountain is next to headwear, which is next to footwear). This is via a one time vector embedding of the categories and subcategories and mapping to UMAP.
The height of each hill/mountain is based on the number of vector search results from Qdrant (top 230 products out of 1500 in the Qdrant database). Neo4J is then used to further refine this down to 50 relevant products for the trip using smart filtering, with climate and activity based relationships.
Finally, our trusty guide, Q-Bert (powered by LLMs) - selects 7 products from the curated 50 products chosen by Neo4J and crafts a 'tour' for the user: taking them on a guided trip through the mountains to explore these products.
The application combines multiple AI technologies to create an immersive product discovery experience:
- Vector Search Pipeline: Query → Mistral embedding → Qdrant similarity search → 230 products
- Graph Curation: Neo4J relationship filtering → 50 curated products
- 3D Visualization: Dynamic terrain heights + product placement
- AI Guide: Q-Bert mascot with LLM-powered product tours
- 1,500 products generated using Mistral Medium
- One time embedding of category/subcategory structure and mapping to 2D via UMAP
- 3D fully explorable world built in ThreeJS/React Three Fiber with AI generated icons
- Qdrant database with 1,500 products and facets embedded (and payloads)
- Neo4J Aura database containing the 1,500 products and 2,200 relationships
- User types query
- Mistral LLM acts as a search strategy agent, decides which queries to make to Qdrant, and which filters to use
- Qdrant returns ~250–350 unique products across multiple targeted searches
- Distribution of these across categories/subcategories sets terrain heights
- Neo4J narrows this down to 50 curated products using relationships (
LAYERS_UNDER|PAIRS_WITH|TRIP_COMPATIBLE) - World is constructed, river is dynamically created, and props are distributed
- The curated 50 products are placed across the world, in the subcategory location
- Mistral is used to create a pick of 7 highlighted products from the top 25 products, with a guided tour for the user as to why these are useful. Q-bert mascot then guides the user interactively through the products in the world
-
Generated 1500 mock outdoor/active e-commerce products using Mistral Medium LLM (with rebalancing across categories - see ./generate_parallel.py)
-
One time embedding of category and subcategory structure (terrain/generate_terrain_embeddings.py), and plotting to 2D plane via UMAP (./terrain/train_umap_model.py) - then hardcoded and stored as ./frontend/src/data/categoryTerrain.json
-
Built 3D world using threeJS/React Three Fiber. Creation of grid structure and placement of category/subcategory embedding onto the grid such that products can be placed. Dynamic generation of river travelling between points with the lowest number of products to create river valley. Creation of props (trees, cows, campfires, houses) and function to place them across the map, avoiding water. Creation of 3D category models for 40 or so subcategories. All 3D models created using AI: https://sloyd.ai
-
Set up Qdrant Database and embedded 1500 products, with payloads for filtering .
-
Set up Mistral LLM as a 'search strategy agent' - to decide which qdrant searches to make and which filters to use. Returning 230 products out of the 1500.
-
Dynamic hill and mountain height based on number of qdrant results per category.
-
Implemented Neo4J for intelligent further curation. Takes the 230 products returned by Qdrant and selects a subset of 50 products, applying smart relationship based filtering to find relevant products for different use cases (LAYERS_UNDER|PAIRS_WITH|TRIP_COMPATIBLE).
-
Display of the seleted 50 products on the world. With relevant category icons. Highlighting, zoom, on products.
-
Use Mistral to create a further curated selection of 7 products for the trip, with a guided tour among the products and why they are relevant. All text generated relevant to context by LLM (Gemini fallback)
- Map the products using UMAP again to dynamic locations, rather than pinning the mountain locations bsaed on category embeddings. Built the start of this but didn't work as well as I hoped and didn't have time to debug.
- Further use of Neo4J (first time using it, really cool in combo with Qdrant)
- Fixing bugs, more reliable placement of objects etc
- Framework: Next.js 15.5.2 with App Router
- 3D Engine: Three.js + React Three Fiber + React Three Drei
- Styling: Tailwind CSS v4
- Language: TypeScript with strict mode
- 3D Models: AI-generated models from Sloyd.ai
- Vector Database: Qdrant Cloud
- Graph Database: Neo4J Aura
- LLM: Mistral Medium/Large (search strategy, product generation, tours)
- Embeddings: Mistral Embed (1024 dimensions)
- Dimensionality Reduction: UMAP for 2D terrain mapping
pnpm run dev # Start development server
pnpm run build # Build for production
pnpm run lint # Run ESLintcd backend && pnpm startpython generate_parallel.py # Generate products
python terrain/generate_terrain_embeddings.py # Category embeddings
python terrain/train_umap_model.py # UMAP terrain mapping├── frontend/ # Next.js 3D application
│ ├── src/
│ │ ├── components/three/ # 3D visualization components
│ │ ├── data/ # Terrain and product data
│ │ └── utils/ # Terrain generation utilities
├── backend/ # Vector search API
├── docs/ # Documentation and setup guides
│ ├── SECURITY.md # Security configuration
│ ├── DEPLOYMENT.md # Deployment instructions
│ ├── SYSTEM_STATUS.md # System monitoring
│ └── TERRAIN_ARCHITECTURE.md # Technical architecture
├── embeddings/ # AI embedding generation
├── database/ # Qdrant operations
├── terrain/ # UMAP terrain mapping
└── generate_parallel.py # Product generation script
- Search Latency: <200ms for 230 product retrieval from Qdrant
- Neo4J Filtering: <100ms to curate 50 products from 230
- 3D Rendering: 60fps on modern hardware
- Terrain Generation: Real-time dynamic height adjustment
- Total Pipeline: Query → 3D visualization in <500ms
- Clone the repository
- Install dependencies:
pnpm install(root and frontend/back-end as needed) - Start the frontend:
pnpm run dev - Start the backend:
cd backend && pnpm start - Open: http://localhost:3000
The source code in this repository is licensed under the Apache License 2.0 (see LICENSE).
Third-party and generated assets (for example models under frontend/imported3dmodels/, category icons, or any Sloyd.ai generated assets) are NOT covered by the Apache 2.0 license. They remain subject to their original licenses and/or the Sloyd.ai terms. Redistribution, reselling, or AI training on those assets may be restricted by those terms.
Reproducing the demo: you can generate similar 3D assets via Sloyd.ai using equivalent templates, or replace them with your own assets.
Sloyd Plus Plan usage: Assets generated under the Sloyd Plus Plan are included here only for demonstration purposes and remain bound by Sloyd’s license. They may not be reused, redistributed, or sold. Do not upload these assets to public datasets or asset stores, and do not use them for training AI models.
See also: THIRD_PARTY_NOTICES.md for dependency attributions and licenses.