Thesis AlayaLite RAG Artifacts

This repository archives the code, experiment reports, thesis requirements, and templates for the undergraduate thesis:

LangChain 视角下的向量数据库（Vector DB）作用机制与系统架构讨论 —— 以 AlayaLite 集成为例

Student: 梁沛然
Student ID: 12210726
Major: 智能科学与技术
Advisor: 唐博

Contents

• code/langchain-alayalite-libs-alayalite/

  • LangChain partner-style AlayaLite integration package.
  • Includes vectorstores.py, retrievers.py, README, pyproject, and unit/integration tests.

• experiments/metadata_filtering/

  • AlayaLite metadata filtering experiment.
  • Validates single-field filtering, multi-field AND filtering, empty results, limit, retriever filtering, and metadata-based deletion.

• experiments/rag_deepseek/

  • Complete LangChain RAG demo using AlayaLite or FAISS as vector store and DeepSeek as LLM.

• experiments/rag_evaluation/

  • Quantitative RAG evaluation with retrieval metrics and DeepSeek-as-judge evaluation.
  • Includes the 10k AlayaLite/FAISS comparison report.

• experiments/hotpotqa_rag/

  • HotpotQA multi-hop retrieval and RAG experiment.
  • Uses official supporting_facts as gold supporting documents.
  • Compares AlayaLite and FAISS with BGE embeddings.

• midterm_workbench_results/

  • CSV results from the midterm workbench benchmark for AlayaLite, FAISS, Qdrant, and Chroma.

• reports/

  • Selected high-level experiment reports copied for quick reading.

• thesis_documents/

  • Thesis task book, midterm self-check, midterm report, school templates, AI declaration template, and filled thesis outline template.

Important Experiment Results

HotpotQA RAG experiment

Dataset: HotpotQA dev distractor
Samples: 7000
Documents: 69661
Embedding: BAAI/bge-small-en-v1.5 on CPU
LLM: deepseek-v4-flash

Vector store	k=10 any_hit	k=10 full_hit	k=10 recall	7000-query total latency	Mean latency
AlayaLite	0.9469	0.6453	0.7961	32.06s	4.58ms
FAISS	0.9786	0.6910	0.8348	32.12s	4.59ms

See experiments/hotpotqa_rag/results/.

Metadata filtering experiment

Corpus size: 50000 documents.

All filtering cases passed:

• single metadata key
• multi-key AND filtering
• no-match empty result
• limit truncation
• LangChain retriever filter
• metadata-based deletion

Precision and Recall@limit are both 1.0 for all exact-match filtering cases.

See experiments/metadata_filtering/results/.

10k DeepSeek RAG comparison

Knowledge base: 10008 documents/chunks
Answer model: deepseek-v4-flash
Judge model: deepseek-v4-pro

Metric	AlayaLite	FAISS
Retrieval hit rate	0.9975	0.9975
Mean retrieval recall	0.7808	0.7777
Mean retrieval latency	38.876 ms	6.705 ms
RAG mean judge score	4.55	4.70
Baseline mean judge score	1.9368	1.9250

See experiments/rag_evaluation/results_10k_comparison.md.

Reproduction Notes

This repository intentionally does not include:

• .venv
• DeepSeek API keys
• raw HotpotQA full dataset
• generated AlayaLite indexes
• FAISS binary indexes
• embedding cache files

Download HotpotQA automatically:

.\.venv\Scripts\python.exe experiments\hotpotqa_rag\hotpotqa_rag_pipeline.py `
  --data-path hotpot_dev_distractor_v1.json `
  --download-if-missing `
  --sample-count 7000 `
  --vector-store both

DeepSeek API keys are read from DEEPSEEK_API_KEY or local local_secrets.json. Do not commit real keys.

Related Repositories

• AlayaLite: https://github.com/AlayaDB-AI/AlayaLite
• langchain-alayalite: https://github.com/lpbirch/langchain-alayalite
• LangChain: https://github.com/langchain-ai/langchain