Contributing to Agrolead

First, thank you for considering contributing to Agrolead! It's people like you that make Agrolead such a great tool.

Code of Conduct

This project and everyone participating in it is governed by our Code of Conduct. By participating, you are expected to uphold this code.

How Can I Contribute?

Reporting Bugs

Before creating bug reports, please check the issue list as you might find out that you don't need to create one. When you are creating a bug report, please include as many details as possible:

Use a clear and descriptive title
Describe the exact steps which reproduce the problem
Provide specific examples to demonstrate the steps
Describe the behavior you observed after following the steps
Explain which behavior you expected to see instead and why
Include screenshots and animated GIFs if possible
Include your OS, Python version, and dependency versions

Suggesting Enhancements

Enhancement suggestions are tracked as GitHub issues. When creating an enhancement suggestion, please include:

Use a clear and descriptive title
Provide a step-by-step description of the suggested enhancement
Provide specific examples to demonstrate the steps
Describe the current behavior and expected behavior
Explain why this enhancement would be useful

Pull Requests

Fill in the required template
Follow the Python style guide (PEP 8 + Black)
Include appropriate test cases
Update documentation as needed
End all files with a newline

Development Setup

1. Fork and Clone

# Fork on GitHub, then:
git clone https://github.com/m223rx/agrolead.git
cd agrolead
git remote add upstream https://github.com/original/agrolead.git

2. Create Virtual Environment

python3.12 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -e ".[dev]"

3. Install Pre-commit Hooks

pre-commit install

4. Create Feature Branch

git checkout -b feature/your-feature-name

Making Changes

Code Style

We follow PEP 8 and use:

Black for code formatting (100 character line length)
isort for import sorting
flake8 for linting
mypy for type checking

Format Your Code

# Automatically format
make format

# Check formatting
make lint

Write Tests

All new features must include tests:

# tests/test_feature.py
import pytest

@pytest.mark.asyncio
async def test_feature():
    # Your test here
    assert True

Run tests:

make test          # Run all tests
make coverage      # With coverage report

Type Hints

All functions should have type hints:

from typing import List, Optional

async def search_companies(
    query: str,
    country: Optional[str] = None,
    limit: int = 50
) -> List[Company]:
    """Search for companies.
    
    Args:
        query: Search query
        country: Optional country code
        limit: Maximum results
    
    Returns:
        List of companies found
    """
    pass

Docstrings

Use Google-style docstrings:

def calculate_score(company: Company) -> float:
    """Calculate lead score for a company.
    
    Scoring factors:
    - Importer: 25 points
    - Fresh produce: 20 points
    - Tomato mention: 20 points
    
    Args:
        company: Company to score
    
    Returns:
        Score between 0 and 100
    
    Raises:
        ValueError: If company data is invalid
    """
    pass

Commit Messages

Use clear, descriptive commit messages:

feat: Add email validation for contacts

- Implement EmailExtractor class
- Add email validation tests
- Update enrichment pipeline

Fixes #123

Format:

Use the imperative mood ("add feature" not "added feature")
Limit the first line to 72 characters
Reference related issues and PRs

Example: Adding a New Directory Adapter

Create the adapter

# agrolead/crawler/sources/my_adapter.py
from agrolead.crawler.base_adapter import AdapterConfig, HTTPAdapter

class MyDirectoryAdapter(HTTPAdapter):
    def __init__(self):
        config = AdapterConfig(
            name="my_directory",
            base_url="https://example.com"
        )
        super().__init__(config)
    
    async def search(self, query, country=None, limit=50):
        # Implementation
        pass
    
    async def crawl(self, start_urls, max_pages=None):
        # Implementation
        pass
    
    async def parse_company(self, url):
        # Implementation
        pass

Register the adapter

# agrolead/crawler/sources/adapters.py
ADAPTERS = {
    "my_directory": MyDirectoryAdapter,
}

Add tests

# tests/test_my_adapter.py
import pytest
from agrolead.crawler.sources.adapters import MyDirectoryAdapter

@pytest.mark.asyncio
async def test_my_adapter_search():
    adapter = MyDirectoryAdapter()
    async with adapter as adp:
        urls = await adp.search("test query")
        assert len(urls) > 0

@pytest.mark.asyncio
async def test_my_adapter_parse():
    # Test parsing
    pass

Update documentation

Add to README.md:

#### MyDirectory (`my_directory`)

- URL: https://example.com
- Coverage: Countries X, Y, Z
- Features: Search, crawl, parse

Pull Request Process

Create a feature branch from develop
Make your changes with clear commit messages
Add tests for new functionality
Format code with make format
Run tests with make test
Update documentation if needed
Push to your fork
Create a Pull Request with a clear description

PR Template

## Description
Brief description of changes

## Related Issue
Fixes #(issue number)

## Type of Change
- [ ] Bug fix
- [ ] New feature
- [ ] Documentation
- [ ] Performance improvement

## Testing
- [ ] Added tests
- [ ] All tests pass
- [ ] Tested locally

## Documentation
- [ ] Updated README
- [ ] Updated code comments
- [ ] Added docstrings

## Checklist
- [ ] Code follows style guidelines
- [ ] Self-review of own code
- [ ] Comments added for complex logic
- [ ] Documentation updated
- [ ] No new warnings generated
- [ ] Tests added/updated
- [ ] All tests passing

Styleguides

Git Commit Messages

feat: Add new feature
fix: Fix bug in component
docs: Update documentation
style: Format code
refactor: Refactor component
perf: Improve performance
test: Add tests
chore: Update dependencies

Python Naming

# Constants (UPPER_SNAKE_CASE)
MAX_RETRIES = 3
DEFAULT_TIMEOUT = 30

# Classes (PascalCase)
class LeadScorer:
    pass

# Functions and variables (snake_case)
def calculate_score():
    pass

# Private methods (_leading_underscore)
def _internal_method():
    pass

Comments

# Good: Explains WHY, not WHAT
# We need to sleep before the next request to respect rate limits
await asyncio.sleep(delay)

# Bad: Explains WHAT (obvious from code)
# Increment counter
counter += 1

Documentation

README Updates

When adding features, update README.md:

Add to features list
Include example usage
Document configuration options
Add troubleshooting if applicable

Docstring Format

Use Google style for consistency:

def example_function(arg1: str, arg2: int) -> bool:
    """Brief description.
    
    Longer description if needed. Can span multiple lines.
    
    Args:
        arg1: Description of arg1
        arg2: Description of arg2
    
    Returns:
        Description of return value
    
    Raises:
        ValueError: When something is wrong
    
    Example:
        >>> example_function("test", 42)
        True
    """
    pass

Performance Considerations

When contributing, consider:

Async Operations: Use async/await for I/O operations
Memory Usage: Large batches should be streamed
Database Queries: Use indexing and efficient queries
Rate Limiting: Respect target websites' terms
Caching: Cache frequently accessed data

Security

Never commit secrets or passwords
Validate user input
Use parameterized queries
Keep dependencies updated
Report security issues privately

Community

Be respectful and inclusive
Help other contributors
Share knowledge generously
Give credit to others' work

Additional Resources

Questions?

Feel free to open an issue with the question label or discuss in our community.

Thank you for contributing to Agrolead! 🌱

FilesExpand file tree

CONTRIBUTING.md

Latest commit

History