Skip to content

LLM_GUIDE.md

Latest commit

 

History

History
293 lines (228 loc) · 8.65 KB

LLM_GUIDE.md

File metadata and controls

293 lines (228 loc) · 8.65 KB
Raw

DevPal LLM Integration Guide

DevPal supports multiple Large Language Models (LLMs) for code analysis and generation. This guide explains how to configure and use different LLMs effectively.

Supported LLMs

1. Google Gemini (Recommended)

  • Provider: Google AI
  • Models: gemini-pro, gemini-pro-vision
  • Best for: Code analysis, explanation, and generation
  • Cost: Free tier available, then pay-per-use
  • Setup:
    1. Get API key from Google AI Studio
    2. Add to .env: GOOGLE_API_KEY=your_key_here

2. Anthropic Claude

  • Provider: Anthropic
  • Models: claude-3-sonnet-20240229, claude-3-opus-20240229, claude-3-haiku-20240307
  • Best for: Detailed code analysis, complex reasoning
  • Cost: Pay-per-use
  • Setup:
    1. Get API key from Anthropic Console
    2. Add to .env: CLAUDE_API_KEY=your_key_here

3. DeepSeek

  • Provider: DeepSeek
  • Models: deepseek-chat, deepseek-coder
  • Best for: Code generation, programming tasks
  • Cost: Pay-per-use
  • Setup:
    1. Get API key from DeepSeek
    2. Add to .env: DEEPSEEK_API_KEY=your_key_here

4. OpenAI GPT

  • Provider: OpenAI
  • Models: gpt-4, gpt-3.5-turbo, gpt-4-turbo
  • Best for: General code analysis and generation
  • Cost: Pay-per-use
  • Setup:
    1. Get API key from OpenAI Platform
    2. Add to .env: OPENAI_API_KEY=your_key_here

5. Ollama (Local)

  • Provider: Local installation
  • Models: codellama, llama2, mistral, etc.
  • Best for: Privacy, offline use, cost control
  • Cost: Free (runs locally)
  • Setup:
    1. Install Ollama
    2. Pull a model: ollama pull codellama
    3. No API key needed

6. HuggingFace

  • Provider: HuggingFace
  • Models: Various open-source models
  • Best for: Local deployment, custom models
  • Cost: Free (local) or pay-per-use (API)
  • Setup:
    1. For local: Install transformers library
    2. For API: Get key from HuggingFace
    3. Add to .env: HUGGINGFACE_API_KEY=your_key_here

Configuration

Environment Setup

Create a .env file with your preferred LLMs:

# Choose your primary LLM
DEFAULT_MODEL=claude

# API Keys (only add the ones you'll use)
GOOGLE_API_KEY=your_gemini_key
CLAUDE_API_KEY=your_claude_key
DEEPSEEK_API_KEY=your_deepseek_key
OPENAI_API_KEY=your_openai_key
HUGGINGFACE_API_KEY=your_hf_key

# Model Settings
EMBEDDING_MODEL=all-MiniLM-L6-v2
CHROMA_PERSIST_DIRECTORY=./chroma_store

Model Selection

You can specify which LLM to use for each command:

# Use Claude for code analysis
devpal ask "What does this function do?" --model claude

# Use DeepSeek for test generation
devpal test main.py --model deepseek

# Use Ollama for local processing
devpal patch utils.py --model ollama

LLM Comparison

Feature Gemini Claude DeepSeek GPT-4 Ollama HuggingFace
Code Analysis ⭐⭐⭐⭐⭐ ⭐⭐⭐⭐⭐ ⭐⭐⭐⭐ ⭐⭐⭐⭐ ⭐⭐⭐ ⭐⭐⭐
Test Generation ⭐⭐⭐⭐ ⭐⭐⭐⭐⭐ ⭐⭐⭐⭐⭐ ⭐⭐⭐⭐ ⭐⭐⭐ ⭐⭐
Error Explanation ⭐⭐⭐⭐ ⭐⭐⭐⭐⭐ ⭐⭐⭐⭐ ⭐⭐⭐⭐ ⭐⭐⭐ ⭐⭐
Documentation ⭐⭐⭐⭐ ⭐⭐⭐⭐⭐ ⭐⭐⭐⭐ ⭐⭐⭐⭐ ⭐⭐⭐ ⭐⭐
Speed ⭐⭐⭐⭐ ⭐⭐⭐ ⭐⭐⭐⭐ ⭐⭐⭐ ⭐⭐⭐⭐⭐ ⭐⭐⭐⭐⭐
Cost ⭐⭐⭐⭐ ⭐⭐⭐ ⭐⭐⭐⭐ ⭐⭐ ⭐⭐⭐⭐⭐ ⭐⭐⭐⭐⭐
Privacy ⭐⭐ ⭐⭐ ⭐⭐ ⭐⭐ ⭐⭐⭐⭐⭐ ⭐⭐⭐⭐⭐

Usage Examples

1. Code Analysis with Claude

# Claude excels at detailed code analysis
devpal ask "Explain the design patterns used in this codebase" --model claude

2. Test Generation with DeepSeek

# DeepSeek is excellent for code generation
devpal test src/main.py --model deepseek --framework pytest

3. Local Processing with Ollama

# Ollama for privacy-sensitive code
devpal patch src/utils.py --model ollama --dry-run

4. Error Analysis with GPT-4

# GPT-4 for comprehensive error analysis
devpal explain "TypeError: 'int' object is not callable" --model openai

Advanced Configuration

Custom Model Parameters

You can customize model behavior by modifying the LLM classes in src/models/llm_manager.py:

# Example: Customize Claude parameters
class ClaudeLLM(BaseLLM):
    def _initialize_model(self):
        self.model = ChatAnthropic(
            model="claude-3-opus-20240229",  # Use Opus for better performance
            anthropic_api_key=self.api_key,
            temperature=0.05,  # Lower temperature for more focused responses
            max_tokens=4096    # Increase token limit
        )

Model-Specific Prompts

Different LLMs may respond better to different prompt styles. You can customize prompts in src/models/prompts.py:

# Example: Claude-specific prompt
def claude_code_analysis_prompt(query: str, context: List[Dict[str, Any]]) -> str:
    return f"""You are an expert software developer. Please analyze the following code and answer the question.

Question: {query}

Code Context:
{format_context(context)}

Please provide a detailed analysis that includes:
1. Code structure and organization
2. Potential improvements
3. Best practices recommendations
4. Security considerations

Analysis:"""

Performance Optimization

1. Model Selection by Task

  • Code Analysis: Claude or GPT-4
  • Test Generation: DeepSeek or Claude
  • Error Debugging: Any model (Claude excels)
  • Documentation: Claude or GPT-4
  • Quick Questions: Gemini or Ollama

2. Batch Processing

For large codebases, consider using faster models for initial processing:

# Quick analysis with Gemini
devpal ask "Find all functions that handle user input" --model gemini

# Detailed analysis with Claude
devpal ask "Analyze the security implications of these functions" --model claude

3. Caching

DevPal automatically caches vector embeddings, but you can also cache LLM responses for repeated queries.

Troubleshooting

Common Issues

1. API Key Errors

# Check available models
devpal models

# Verify API key in .env file
cat .env | grep API_KEY

2. Model Not Available

# Check which models are working
devpal models

# Try a different model
devpal ask "test question" --model gemini

3. Ollama Connection Issues

# Check if Ollama is running
ollama list

# Start Ollama service
ollama serve

# Pull required model
ollama pull codellama

4. Rate Limiting

  • Use local models (Ollama, HuggingFace) for high-frequency usage
  • Implement request throttling for API-based models
  • Consider using multiple API keys for load balancing

Cost Optimization

1. Free Tier Usage

  • Gemini: 15 requests/minute free
  • Claude: Limited free tier
  • Ollama: Completely free (local)
  • HuggingFace: Free for local models

2. Model Selection Strategy

# Use free/local models for exploration
devpal ask "What files exist in this project?" --model ollama

# Use paid models for critical analysis
devpal ask "Find security vulnerabilities" --model claude

3. Prompt Optimization

  • Keep prompts concise and specific
  • Use few-shot examples for better results
  • Batch related questions together

Best Practices

1. Model Selection

  • Start with Gemini (free tier available)
  • Use Claude for complex analysis
  • Use DeepSeek for code generation
  • Use Ollama for privacy-sensitive work

2. Prompt Engineering

  • Be specific about the programming language
  • Include context about the codebase
  • Ask for structured responses when needed

3. Security

  • Never commit API keys to version control
  • Use local models for sensitive code
  • Review generated code before applying changes

4. Performance

  • Cache vector embeddings for large projects
  • Use appropriate models for different tasks
  • Consider parallel processing for multiple queries

Future Enhancements

Planned features for LLM integration:

  • Model Comparison: Side-by-side results from multiple models
  • Custom Model Support: Easy integration of new LLMs
  • Response Caching: Cache LLM responses for repeated queries
  • Model Performance Tracking: Monitor which models work best for different tasks
  • Automatic Model Selection: AI-powered model selection based on task type