Skip to content

Commit 0fb696b

Browse files
committed
feat: abstract embedding and LLM providers
Added abstract base classes for embedding and LLM providers. Therefore, contributors can now implement their own embedding and LLM providers by extending the abstract base classes. Also, I changed the dotenv loading to use the new config class. This works more like a traditional package structure and leaves the opinionated decisions to the application layer.
1 parent 7607509 commit 0fb696b

32 files changed

Lines changed: 2937 additions & 266 deletions

.env.example

Lines changed: 89 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
# Embedding Service Configuration
66
# =============================================================================
77

8-
# Provider selection: "local" (Ollama) or "openai"
8+
# Provider selection: "local" (Ollama), "openai", "cohere", or "huggingface"
99
# Default: local
1010
EMBED_PROVIDER=local
1111

@@ -24,14 +24,41 @@ OPENAI_API_KEY=your-openai-api-key-here
2424

2525
# OpenAI embedding model
2626
# Default: text-embedding-3-small
27-
# Options: text-embedding-3-small, text-embedding-3-large, text-embedding-ada-002
27+
# Options: text-embedding-3-small (1536 dims), text-embedding-3-large (3072 dims), text-embedding-ada-002 (1536 dims)
2828
OPENAI_EMBED_MODEL=text-embedding-3-small
2929

30+
# Cohere Configuration (when EMBED_PROVIDER=cohere)
31+
# Required if using Cohere provider
32+
COHERE_API_KEY=your-cohere-api-key-here
33+
34+
# Cohere embedding model
35+
# Default: embed-v4.0
36+
# Options: embed-v4.0 (1536 dims default, configurable 256-1536), embed-v3
37+
COHERE_EMBED_MODEL=embed-v4.0
38+
39+
# HuggingFace Configuration (when EMBED_PROVIDER=huggingface)
40+
# Required if using HuggingFace provider
41+
HUGGINGFACE_API_KEY=your-huggingface-api-key-here
42+
43+
# HuggingFace embedding model
44+
# Default: sentence-transformers/all-MiniLM-L6-v2
45+
# Options: any embedding model from HuggingFace Hub
46+
# Examples:
47+
# - sentence-transformers/all-MiniLM-L6-v2 (384 dims)
48+
# - sentence-transformers/all-mpnet-base-v2 (768 dims)
49+
# - BAAI/bge-small-en-v1.5 (384 dims)
50+
HUGGINGFACE_EMBED_MODEL=sentence-transformers/all-MiniLM-L6-v2
51+
52+
# HuggingFace Inference Endpoint URL (optional)
53+
# Default: auto-generated from HUGGINGFACE_EMBED_MODEL
54+
# Use this if you have a custom Inference Endpoint URL
55+
# HUGGINGFACE_EMBED_URL=https://api-inference.huggingface.co/pipeline/feature-extraction/sentence-transformers/all-MiniLM-L6-v2
56+
3057
# =============================================================================
3158
# LLM Service Configuration (for keyword extraction)
3259
# =============================================================================
3360

34-
# Provider selection: "local" (Ollama) or "openai"
61+
# Provider selection: "local" (Ollama), "openai", "anthropic", or "gemini"
3562
# Default: local
3663
LLM_PROVIDER=local
3764

@@ -46,9 +73,30 @@ LLM_MODEL=qwen3:8b
4673

4774
# OpenAI LLM model (when LLM_PROVIDER=openai)
4875
# Default: gpt-4o-mini
49-
# Options: gpt-4o-mini, gpt-3.5-turbo, gpt-4, etc.
76+
# Options: gpt-4o-mini, gpt-4o, gpt-3.5-turbo, gpt-4-turbo
5077
OPENAI_LLM_MODEL=gpt-4o-mini
5178

79+
# Anthropic Configuration (when LLM_PROVIDER=anthropic)
80+
# Required if using Anthropic provider
81+
ANTHROPIC_API_KEY=your-anthropic-api-key-here
82+
83+
# Anthropic Claude model
84+
# Default: claude-3-5-haiku-latest
85+
# Options:
86+
# - claude-3-5-haiku-latest (fastest)
87+
# - claude-sonnet-4-5 (best for agents and coding)
88+
# - claude-opus-4-5 (most capable)
89+
ANTHROPIC_LLM_MODEL=claude-3-5-haiku-latest
90+
91+
# Google Gemini Configuration (when LLM_PROVIDER=gemini)
92+
# Required if using Gemini provider
93+
GEMINI_API_KEY=your-gemini-api-key-here
94+
95+
# Google Gemini model
96+
# Default: gemini-2.5-flash
97+
# Options: gemini-2.5-flash (fast), gemini-2.5-pro (more capable), gemini-3-pro (latest)
98+
GEMINI_LLM_MODEL=gemini-2.5-flash
99+
52100
# =============================================================================
53101
# Quick Setup Guide
54102
# =============================================================================
@@ -65,4 +113,41 @@ OPENAI_LLM_MODEL=gpt-4o-mini
65113
# 3. Set OPENAI_API_KEY=your-key-here
66114
# 4. Optionally customize model names above
67115
#
116+
# For Cohere Setup:
117+
# 1. Set EMBED_PROVIDER=cohere
118+
# 2. Set COHERE_API_KEY=your-key-here
119+
# 3. Get API key from: https://dashboard.cohere.com/api-keys
120+
#
121+
# For HuggingFace Setup:
122+
# 1. Set EMBED_PROVIDER=huggingface
123+
# 2. Set HUGGINGFACE_API_KEY=your-key-here
124+
# 3. Get API key from: https://huggingface.co/settings/tokens
125+
# 4. Optionally set HUGGINGFACE_EMBED_MODEL to your preferred model
126+
#
127+
# For Anthropic Setup:
128+
# 1. Set LLM_PROVIDER=anthropic
129+
# 2. Set ANTHROPIC_API_KEY=your-key-here
130+
# 3. Get API key from: https://console.anthropic.com/
131+
#
132+
# For Google Gemini Setup:
133+
# 1. Set LLM_PROVIDER=gemini
134+
# 2. Set GEMINI_API_KEY=your-key-here
135+
# 3. Get API key from: https://aistudio.google.com/app/apikey
136+
#
137+
# =============================================================================
138+
# Important: LanceDB Dimension Requirements
139+
# =============================================================================
140+
#
141+
# LanceDB doesn't require specific dimensions - it only requires that query
142+
# embeddings match stored embeddings in dimension. Different providers return
143+
# different dimensions:
144+
#
145+
# - OpenAI: 1536 (text-embedding-3-small)
146+
# - Cohere: 1536 (embed-v4.0 default)
147+
# - HuggingFace: Varies (384-768 typically)
148+
# - Local: 4096 (qwen3-embedding default)
149+
#
150+
# If you switch providers, you'll need to re-embed all products with the new
151+
# provider's dimensions. See docs/PROVIDERS.md for details.
152+
#
68153
# =============================================================================

LICENSE

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
MIT License
22

3-
Copyright (c) 2025 VKRA
3+
Copyright (c) 2026 VKRA
44

55
Permission is hereby granted, free of charge, to any person obtaining a copy
66
of this software and associated documentation files (the "Software"), to deal

README.md

Lines changed: 80 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
11
# VKRA Protocol
22

3-
> Open Source LLMA Framework for Contextual Product Matching in LLM Applications
3+
> VKRA - Open Source Marketing Framework for Contextual Product Matching in LLM Applications
44
55
[![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/)
66
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
77

88
## Vision
99

10-
**Revolutionizing accessible advertisements in LLMs.** VKRA Protocol provides a modular, open-source framework for contextual product matching that enables developers to build monetization into their LLM applications while maintaining user trust and conversation quality.
10+
**Revolutionizing accessible advertisements in LLMs.** VKRA is an open-source marketing framework that provides a modular approach to contextual product matching, enabling developers to build monetization into their LLM applications while maintaining user trust and conversation quality.
1111

1212
## What is VKRA Protocol?
1313

@@ -72,7 +72,7 @@ pip install vkra-protocol
7272
Or install from source:
7373

7474
```bash
75-
git clone https://github.com/vkra/vkra-protocol.git
75+
git clone https://github.com/vkra-project/vkra-python
7676
cd vkra-protocol
7777
pip install -e .
7878
```
@@ -134,6 +134,8 @@ class MyUserProfileStore(UserProfileStore):
134134
ollama pull qwen3-embedding
135135
```
136136

137+
**Option A: Simple Service Usage (Environment Variables)**
138+
137139
```python
138140
from vkra_protocol import LLMAOrchestrator, EmbeddingService, LLMService
139141

@@ -148,6 +150,59 @@ llm_service = LLMService() # Uses local Ollama by default
148150
# OPENAI_API_KEY=your-key
149151
```
150152

153+
**Option B: Advanced Provider Usage (Dependency Injection)**
154+
155+
For more control and better testability, use providers directly:
156+
157+
```python
158+
from vkra_protocol import LLMAOrchestrator
159+
from vkra_protocol.providers import (
160+
EmbeddingService,
161+
LLMService,
162+
EmbeddingProviderFactory,
163+
LLMProviderFactory,
164+
)
165+
from vkra_protocol.providers.embedding import OpenAIEmbeddingProvider
166+
from vkra_protocol.providers.llm import OpenAILLMProvider
167+
168+
# Method 1: Use factory with config dict
169+
embedding_provider = EmbeddingProviderFactory.create("openai", {
170+
"api_key": "sk-...",
171+
"model": "text-embedding-3-small"
172+
})
173+
llm_provider = LLMProviderFactory.create("openai", {
174+
"api_key": "sk-...",
175+
"model": "gpt-4o-mini"
176+
})
177+
178+
# Method 2: Create providers directly
179+
embedding_provider = OpenAIEmbeddingProvider(
180+
api_key="sk-...",
181+
model="text-embedding-3-small"
182+
)
183+
llm_provider = OpenAILLMProvider(
184+
api_key="sk-...",
185+
model="gpt-4o-mini"
186+
)
187+
188+
# Wrap in services (or use providers directly if orchestrator supports it)
189+
embedding_service = EmbeddingService(provider=embedding_provider)
190+
llm_service = LLMService(provider=llm_provider)
191+
192+
# Method 3: Factory from environment (same as Option A)
193+
embedding_provider = EmbeddingProviderFactory.create_from_env()
194+
llm_provider = LLMProviderFactory.create_from_env()
195+
embedding_service = EmbeddingService(provider=embedding_provider)
196+
llm_service = LLMService(provider=llm_provider)
197+
```
198+
199+
**Benefits of Provider Pattern:**
200+
- ✅ Dependency injection for better testability
201+
- ✅ No hardcoded environment variables
202+
- ✅ Easy to mock in tests
203+
- ✅ Programmatic configuration
204+
- ✅ Consistent error handling (`ProviderError`, `ProviderConfigurationError`, `ProviderAPIError`)
205+
151206
### 3. Initialize Orchestrator
152207

153208
```python
@@ -211,6 +266,28 @@ for result in response.results:
211266
- `OPENAI_EMBED_MODEL=text-embedding-3-small` (default)
212267
- `OPENAI_LLM_MODEL=gpt-4o-mini` (default)
213268

269+
**Optional (Other Providers):**
270+
- `EMBED_PROVIDER=cohere``COHERE_API_KEY`, `COHERE_EMBED_MODEL`
271+
- `EMBED_PROVIDER=huggingface``HUGGINGFACE_API_KEY`, `HUGGINGFACE_EMBED_MODEL`
272+
- `LLM_PROVIDER=anthropic``ANTHROPIC_API_KEY`, `ANTHROPIC_LLM_MODEL`
273+
- `LLM_PROVIDER=gemini``GEMINI_API_KEY`, `GEMINI_LLM_MODEL`
274+
275+
## Supported Providers
276+
277+
### Embedding Providers
278+
- **OpenAI**: `text-embedding-3-small` (1536 dims, auto-detected)
279+
- **Cohere**: `embed-v4.0` (1024-1536 dims, auto-detected)
280+
- **HuggingFace**: Various models (dims vary, auto-detected)
281+
- **Local (Ollama/vLLM)**: Any local model (dims vary, auto-detected)
282+
283+
### LLM Providers
284+
- **OpenAI**: `gpt-4o-mini`, `gpt-4o`, etc.
285+
- **Anthropic**: `claude-3-5-haiku-latest`, `claude-sonnet-4-5`, etc.
286+
- **Google Gemini**: `gemini-2.5-flash`, `gemini-2.5-pro`, etc.
287+
- **Local (Ollama/vLLM)**: Any local model
288+
289+
**Note on Dimensions**: Embedding dimensions are detected automatically from the first API call. No need to hardcode dimension values!
290+
214291
## Module Configuration
215292

216293
Customize module behavior per request:

0 commit comments

Comments
 (0)