Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 16 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Inspamity — Agent Guidelines

AI-powered spam detection tool that integrates with rspamd using the Anthropic Claude API.
AI-powered spam detection tool that integrates with rspamd using AI providers (Anthropic Claude, OpenAI).

## Project Structure

Expand All @@ -9,7 +9,10 @@ AI-powered spam detection tool that integrates with rspamd using the Anthropic C
├── cli_toolbox.py # CLI tool for testing emails
├── email_utils/
│ ├── config.py # Shared config loader (system/local)
│ ├── anthropic_spam_check.py # Anthropic API integration
│ ├── prompts.py # Shared SYSTEM_PROMPT constant
│ ├── ai_spam_check.py # Provider dispatcher (check_spam_with_ai)
│ ├── anthropic_spam_check.py # Anthropic provider implementation
│ └── openai_spam_check.py # OpenAI provider implementation
│ └── process_email.py # Email parsing and formatting
├── rspamd/
│ └── external_ai_test.lua # rspamd Lua plugin
Expand All @@ -35,19 +38,27 @@ pip install -e ".[dev]"
- **Quotes**: double quotes (enforced by ruff)
- **Imports**: sorted by ruff (`I` rule), stdlib → third-party → local

## Before Submitting a PR

- Run `ruff check . && ruff format --check . && pytest` — all must pass
- Check if `README.md` needs updates to reflect your changes (new features, changed config, updated commands)

## Testing

- **Framework**: pytest 9.0.3
- **Run tests**: `pytest -v`
- **All changes must pass**: `ruff check . && ruff format --check . && pytest`
- Tests live in `tests/` and mirror the module structure
- Mock external APIs — never make real API calls in tests. Use `unittest.mock.patch` for the Anthropic client and `monkeypatch` for config paths
- Mock external APIs — never make real API calls in tests. Use `unittest.mock.patch` for API clients and `monkeypatch` for config paths
- Use `tmp_path` for temporary files, `monkeypatch` for config isolation

## Architecture Notes

- **Provider dispatch**: `ai_spam_check.py` reads the `provider` setting from config and dispatches to the appropriate provider module. Provider imports are lazy (inside if/elif branches) so only the selected SDK is loaded.
- **Adding a new provider**: Create `email_utils/<provider>_spam_check.py` with a `check_spam_with_<provider>(email_content: str) -> dict[str, Any]` function, add an elif branch in `ai_spam_check.py`, and add the config section.
- **System prompt** is shared across providers via `email_utils/prompts.py`. Do not duplicate it in provider modules.
- **Config loading** is centralized in `email_utils/config.py`. Don't duplicate config logic elsewhere. Config is read from `/etc/inspamity/config.ini` (system) or `config.ini` in the project root (local), in that priority order.
- **Entry points** (`email_ai_interface.py`, `cli_toolbox.py`) live at the project root because the rspamd Lua plugin references them by absolute path (`/usr/local/inspamity/`). Do not move them into a package.
- **Production deployment** uses a venv at `/usr/local/inspamity/.venv/` — the Lua script calls that venv's Python directly.
- The Anthropic API **temperature** parameter is only passed when explicitly set in config. Do not hardcode it.
- `check_spam_with_ai()` must always return a dict with `is_spam`, `confidence`, and `reason` keys, even on error.
- The **temperature** parameter is only passed to the API when explicitly set in config. Do not hardcode it.
- All provider functions must always return a dict with `is_spam`, `confidence`, and `reason` keys, even on error.
31 changes: 31 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,37 @@ After this, enable debugging in the config.ini

## ⚙️ Configuration

Copy `config.ini.default` to `config.ini` (or `/etc/inspamity/config.ini` for system-wide) and edit it:

```ini
[settings]
# AI provider: anthropic or openai
provider = anthropic

[anthropic]
api_key = your_api_key_here
model = claude-haiku-4-5-latest
temperature = 0.0
timeout = 20.0

[openai]
api_key = your_api_key_here
model = gpt-5.4-mini
temperature = 0.0
timeout = 20.0
```

### Supported Providers

| Provider | Default Model | Config Section |
|----------|--------------|----------------|
| Anthropic | `claude-haiku-4-5-latest` | `[anthropic]` |
| OpenAI | `gpt-5.4-mini` | `[openai]` |

Set `provider` in `[settings]` to switch between them. Only the selected provider's API key is required.

### rspamd Integration

By default, the rspamd Lua script is configured to:
- Run after all other checks (type postfilter)
- Skip emails already marked as spam
Expand Down
2 changes: 1 addition & 1 deletion cli_toolbox.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ def main() -> None:
# Check for spam using AI if requested
if args.check_ai:
try:
from email_utils.anthropic_spam_check import check_spam_with_ai
from email_utils.ai_spam_check import check_spam_with_ai

# Send formatted email content to AI
result = check_spam_with_ai(output)
Expand Down
19 changes: 14 additions & 5 deletions config.ini.default
Original file line number Diff line number Diff line change
@@ -1,12 +1,21 @@
[settings]
# AI provider to use: anthropic or openai
provider = anthropic
# Enable debug mode to save copies of processed emails and AI output
debug_mode = false
# Directory where debug files will be stored
debug_directory = /var/local/inspamity

[anthropic]
api_key = your_api_key_here
model = claude-haiku-4-5-latest
# Temperature for AI responses (0.0 = deterministic). Remove to use model default.
temperature = 0.0
timeout = 20.0

[settings]
# Enable debug mode to save copies of processed emails and AI output
debug_mode = false
# Directory where debug files will be stored
debug_directory = /var/local/inspamity
[openai]
api_key = your_api_key_here
model = gpt-5.4-mini
# Temperature for AI responses (0.0 = deterministic). Remove to use model default.
temperature = 0.0
timeout = 20.0
2 changes: 1 addition & 1 deletion email_ai_interface.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
from datetime import datetime
from pathlib import Path

from email_utils.anthropic_spam_check import check_spam_with_ai
from email_utils.ai_spam_check import check_spam_with_ai
from email_utils.config import load_config
from email_utils.process_email import format_email_content, get_email_content

Expand Down
35 changes: 35 additions & 0 deletions email_utils/ai_spam_check.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
from typing import Any

from email_utils.config import load_config


def check_spam_with_ai(email_content: str) -> dict[str, Any]:
"""Check if an email is spam using the configured AI provider.

Dispatches to the appropriate provider based on the 'provider'
setting in config.ini (defaults to 'anthropic').
"""
try:
config = load_config()
provider = config.get("settings", "provider", fallback="anthropic")

if provider == "anthropic":
from email_utils.anthropic_spam_check import check_spam_with_anthropic

return check_spam_with_anthropic(email_content)
elif provider == "openai":
from email_utils.openai_spam_check import check_spam_with_openai

return check_spam_with_openai(email_content)
else:
return {
"is_spam": "no",
"confidence": 0,
"reason": f"Unknown AI provider: {provider}",
}
except Exception as e:
return {
"is_spam": "no",
"confidence": 0,
"reason": f"Error checking spam with AI: {e}",
}
25 changes: 2 additions & 23 deletions email_utils/anthropic_spam_check.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,26 +4,10 @@
import anthropic

from email_utils.config import load_config
from email_utils.prompts import SYSTEM_PROMPT

SYSTEM_PROMPT = (
"You are a spam detection system. Analyze this email and classify it "
"as spam or not. Note that legitimate newsletters are not spam.\n\n"
"The attached email contains all headers, but is stripped from HTML "
"and attachments. It is also truncated if it's too long. At the end "
"it contains a summary of attachments, images and links that were "
"in the email.\n\n"
"Provide your analysis in JSON format with the following structure:\n"
"{\n"
' "is_spam": "yes|no",\n'
' "confidence": 0-100,\n'
' "reason": "brief explanation of key factors that led to this '
'classification"\n'
"}\n\n"
"Only output this JSON. Do not output anything else!"
)


def check_spam_with_ai(email_content: str) -> dict[str, Any]:
def check_spam_with_anthropic(email_content: str) -> dict[str, Any]:
"""Use Anthropic Claude API to check if an email is spam.

Args:
Expand Down Expand Up @@ -62,8 +46,3 @@ def check_spam_with_ai(email_content: str) -> dict[str, Any]:
"confidence": 0,
"reason": f"Error checking spam with AI: {e}",
}


if __name__ == "__main__":
test_content = "From: test@example.com\nSubject: Test\n\nThis is a test email"
print(check_spam_with_ai(test_content))
50 changes: 50 additions & 0 deletions email_utils/openai_spam_check.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
import json
from typing import Any

from openai import OpenAI

from email_utils.config import load_config
from email_utils.prompts import SYSTEM_PROMPT


def check_spam_with_openai(email_content: str) -> dict[str, Any]:
"""Use OpenAI API to check if an email is spam.

Args:
email_content: Processed and formatted email content.

Returns:
AI analysis result with is_spam, confidence, and reason.
"""
try:
config = load_config()

api_key = config.get("openai", "api_key")
model = config.get("openai", "model", fallback="gpt-5.4-mini")
timeout = config.getfloat("openai", "timeout", fallback=20.0)

client = OpenAI(api_key=api_key, timeout=timeout)

kwargs: dict[str, Any] = {
"model": model,
"max_completion_tokens": 100,
"messages": [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": email_content},
],
}

# Only pass temperature if explicitly configured
if config.has_option("openai", "temperature"):
kwargs["temperature"] = config.getfloat("openai", "temperature")

response = client.chat.completions.create(**kwargs)

return json.loads(response.choices[0].message.content)

except Exception as e:
return {
"is_spam": "no",
"confidence": 0,
"reason": f"Error checking spam with AI: {e}",
}
24 changes: 24 additions & 0 deletions email_utils/prompts.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
SYSTEM_PROMPT = (
"You are a spam detection system. Classify the following email as spam or not spam.\n\n"
"Spam includes: unsolicited commercial email, phishing, scam messages, "
"malware/link bait, and deceptive bulk mail.\n\n"
"Not spam includes: personal correspondence, legitimate newsletters and "
"mailing lists the recipient likely subscribed to, transactional emails "
"(order confirmations, password resets, shipping notifications), and "
"automated notifications from known services.\n\n"
"The email below has been pre-processed: HTML has been converted to plain text, "
"attachments have been removed, and the body may be truncated. A metadata summary "
"(images, links, attachments, DKIM/SPF results) is included at the top. "
"Note: long URLs and lists may be truncated by our preprocessing "
"(marked with [TRUNCATED]) — this is normal and not a spam indicator.\n\n"
"Useful signals to consider:\n"
"- DKIM/SPF failures on emails claiming to be from well-known organizations\n"
"- Suspicious URL domains (e.g. misspelled brands, free hosting sites), "
"especially combined with urgency or threats\n"
"- Pressure tactics (act now, account suspended, verify immediately)\n"
"- Mismatch between the claimed sender identity and the email content/headers\n\n"
"Respond with only this JSON object, no other text or markdown:\n"
'{"is_spam": "yes" or "no", "confidence": 0-100, "reason": "brief explanation"}\n\n'
"Confidence guide: 0-20 very unlikely spam, 30-50 uncertain, "
"60-80 likely spam, 85-100 clearly spam."
)
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ authors = [
dependencies = [
"anthropic>=0.90,<1.0",
"beautifulsoup4>=4.13",
"openai>=2.0,<3.0",
]

[project.optional-dependencies]
Expand Down
55 changes: 55 additions & 0 deletions tests/test_ai_spam_check.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
from unittest.mock import patch

from email_utils import config as config_module
from email_utils.ai_spam_check import check_spam_with_ai


def _mock_config(tmp_path, monkeypatch, provider="anthropic"):
"""Helper to set up a mock config file with provider selection."""
lines = f"[settings]\nprovider = {provider}\n"
lines += "[anthropic]\napi_key = test\n"
lines += "[openai]\napi_key = test\n"
config_file = tmp_path / "config.ini"
config_file.write_text(lines)
monkeypatch.setattr(config_module, "SYSTEM_CONFIG_PATH", tmp_path / "nonexistent.ini")
monkeypatch.setattr(config_module, "LOCAL_CONFIG_PATH", config_file)


class TestAiSpamCheckDispatcher:
@patch("email_utils.anthropic_spam_check.check_spam_with_anthropic")
def test_dispatches_to_anthropic(self, mock_anthropic, tmp_path, monkeypatch):
_mock_config(tmp_path, monkeypatch, provider="anthropic")
mock_anthropic.return_value = {"is_spam": "no", "confidence": 0, "reason": "test"}

result = check_spam_with_ai("test email")
mock_anthropic.assert_called_once_with("test email")
assert result["is_spam"] == "no"

@patch("email_utils.openai_spam_check.check_spam_with_openai")
def test_dispatches_to_openai(self, mock_openai, tmp_path, monkeypatch):
_mock_config(tmp_path, monkeypatch, provider="openai")
mock_openai.return_value = {"is_spam": "yes", "confidence": 80, "reason": "spam"}

result = check_spam_with_ai("test email")
mock_openai.assert_called_once_with("test email")
assert result["is_spam"] == "yes"

def test_unknown_provider_returns_error(self, tmp_path, monkeypatch):
_mock_config(tmp_path, monkeypatch, provider="unknown_provider")

result = check_spam_with_ai("test email")
assert result["is_spam"] == "no"
assert result["confidence"] == 0
assert "Unknown AI provider" in result["reason"]

def test_defaults_to_anthropic_when_no_provider_set(self, tmp_path, monkeypatch):
"""When provider is not in config, defaults to anthropic."""
config_file = tmp_path / "config.ini"
config_file.write_text("[settings]\n[anthropic]\napi_key = test\n")
monkeypatch.setattr(config_module, "SYSTEM_CONFIG_PATH", tmp_path / "nonexistent.ini")
monkeypatch.setattr(config_module, "LOCAL_CONFIG_PATH", config_file)

with patch("email_utils.anthropic_spam_check.check_spam_with_anthropic") as mock_anthropic:
mock_anthropic.return_value = {"is_spam": "no", "confidence": 0, "reason": "test"}
check_spam_with_ai("test")
mock_anthropic.assert_called_once()
Loading
Loading