SPEC0207.US07 - Generate demo data for RAG testing

[wendellmva]

Description

Generate comprehensive demo data to test RAG (Retrieval-Augmented Generation) capabilities.

Goals

• Provide realistic data for vector search testing
• Demonstrate RAG system capabilities
• Enable performance testing
• Create examples for documentation

Tasks

Data Generation

• Create synthetic codebase data (C#, TypeScript, Python samples)
• Generate technical documentation snippets
• Add architecture decision records (ADRs)
• Create API endpoint descriptions
• Generate code review comments
• Add design pattern examples

Entity Population

• Create diverse NodeTypes with properties
• Generate realistic Nodes with relationships
• Add Properties with rich descriptions
• Create EdgeTypes for different relationship types
• Build complex graph structures

Vector Embeddings

• Generate embeddings for all properties
• Use multiple embedding models for comparison
• Add metadata for embedding provenance
• Test various vector dimensions

Data Seeder

• Create seeder service
• Add command-line tool for data generation
• Make data generation configurable (size, complexity)
• Add option to clear existing data

Test Scenarios

• Code generation queries
• Semantic search scenarios
• Graph traversal examples
• Multi-hop reasoning tests

Documentation

• Document data schema
• Provide example queries
• Add performance benchmarks

Sample Data Types

• Software architecture patterns
• Code snippets and examples
• Technical documentation
• API specifications
• Database schemas
• Design patterns

Related

Enables comprehensive testing of SPEC0206 (VectorRAG) features