Multi-Modal Retrieval Project (Image search by image/text, Based on Milvus Lite)

🌐 Language: English | 中文

This project is a lightweight image search system based on Milvus Lite. Unlike the standard Milvus that requires Docker, Milvus Lite can run directly in a Python environment, greatly simplifying deployment and development.

This document will explain in detail how to run this project step by step based on your code and data.

---

⚙️ Code Execution Flow

The core code flow of this project is divided into three main steps, controlled by three independent Shell scripts. Please execute them in the following order:

1. Generate Embeddings (1_generate_embeddings.sh):

   • Purpose: Reads the original image files, extracts features using the specified CLIP model, and saves these features (embeddings) as a .pkl file.
   • This is the first step of the entire process and serves as the data foundation for subsequent steps.

2. Build Milvus Index (2_build_milvus_lite_index.sh):

   • Purpose: Reads the .pkl file generated in the previous step, creates a Collection in Milvus Lite, imports all vector data into it, and builds the index.
   • This step is responsible for loading the data into the vector database, preparing it for search.

3. Start the Search Application (search-run-milvus-lite.sh):

   • Purpose: Starts a Streamlit web application. This application connects to Milvus Lite and provides a user interface for real-time search via text or images.
   • This is the final deliverable of the project, providing interactive search functionality.

---

🛠️ Operation Guide

Step 1: Prepare Environment and Configuration

1. Install Dependencies:

   pip install -r requirements_milvus_lite.txt

2. Configure 1_generate_embeddings.sh:

   • Open the 1_generate_embeddings.sh file and must modify the following variables:
     • IMG_DIR: Points to your image dataset directory.
     • MODEL_PATH: Points to your local CLIP model directory.
     • DATABASE_DIR: Defines the storage location for the generated feature vector file (.pkl).

Step 2: Generate Feature Vectors

After configuration, run the first script:

./1_generate_embeddings.sh

• Upon successful execution, you should see a .pkl file in the DATABASE_DIR directory.

Step 3: Build Vector Index

1. Configure 2_build_milvus_lite_index.sh (Optional):

   • Open the 2_build_milvus_lite_index.sh file. Usually, you do not need to modify it, but you can pay attention to:
     • EMBEDDINGS_PATH: The script will automatically use the .pkl file path generated in the previous step.
     • COLLECTION_NAME: The name of the Milvus collection, which can be customized.
     • MILVUS_DATA_DIR: The storage location for Milvus Lite database files. Default is empty, can be modified via environment variables or script parameters.

2. Run the Script:

   ./2_build_milvus_lite_index.sh

• This script will automatically install Milvus Lite (if not already installed) and complete the index build.

Step 4: Start and Use the Search Application

1. Configure search-run-milvus-lite.sh (Optional):

   • Open the search-run-milvus-lite.sh file. Usually, you do not need to modify it, but you can pay attention to:
     • STREAMLIT_PORT: The access port for the web application, default is 8080.

2. Run the One-Click Startup Script:

   ./search-run-milvus-lite.sh

• The script will check if the previous steps are completed and then start the application.

3. Access the Application:
   • Open your browser and go to http://localhost:8080 (or your custom port) to start using the image search functionality.

📁 File Structure

.
├── app_milvus_lite.py             # Streamlit application main program
├── clip_image_search/             # Core search logic
│   ├── extract_embeddings.py      # Python file called by script 1: extracts image features
│   └── build_milvus_lite_index.py # Python file called by script 2: builds Milvus index
├── 1_generate_embeddings.sh       # Script 1: Generate embeddings
├── 2_build_milvus_lite_index.sh   # Script 2: Build index
├── search-run-milvus-lite.sh      # Script 3: One-click application startup
├── start_milvus_lite.py           # Milvus Lite service management tool (advanced)
├── requirements_milvus_lite.txt   # Python dependencies
└── README.md                      # This document

🌟 Features

• Zero Configuration: No Docker or complex configuration needed.
• Lightweight: Low resource usage, fast startup.
• Full-Featured: Supports vector search, indexing, and data persistence.
• Developer-Friendly: Ideal for rapid development and testing.

🐛 Common Issues

• Milvus Lite not available or No module named 'milvus': Please ensure all dependencies are installed: pip install -r requirements_milvus_lite.txt
• Collection not found: The Collection does not exist. Please run ./2_build_milvus_lite_index.sh first to build the index.
• Port Already in Use: If port 8669 is occupied, you can specify a new port when starting: streamlit run app_milvus_lite.py --server.port <new_port>
• Image File Not Found: Please check if the image path set in 1_generate_embeddings.sh is correct.

---

Note: For the Chinese version of this documentation, please see README-zh.md.

注意：本文档的中文版请参见 README-zh.md。