Skip to content

Calibrate spam prompt confidence - #12

Merged
dtump merged 2 commits into
mainfrom
improve/system-prompt-calibration
Jul 30, 2026
Merged

Calibrate spam prompt confidence#12
dtump merged 2 commits into
mainfrom
improve/system-prompt-calibration

Conversation

@dtump

@dtump dtump commented Jul 30, 2026

Copy link
Copy Markdown
Owner

Summary

Make a small, generic calibration improvement to the shared spam-classification prompt without changing Inspamity's scoring contract.

Changes

  • Explicitly define confidence as confidence in the selected yes/no verdict; for example, is_spam=no with confidence 90 means strong confidence that the message is not spam and therefore maps to a large negative Rspamd score.
  • Prevent unsupported claims about recipient subscription history, domain reputation, or unfamiliar domains.
  • Treat absent preprocessed SPF/DKIM information as unknown rather than failed.
  • Clarify that authentication problems, promotional language, urgency, tracking infrastructure, image count, and awkward language are weak signals when isolated.
  • Require identity or URL mismatches to be considered with other suspicious evidence.
  • Add tests preserving these prompt and confidence semantics.

The prompt remains concise and generic: it contains no local domains, regional brands, sender allowlists, or user-specific preferences.

Validation

  • .venv/bin/ruff check .
  • .venv/bin/ruff format --check .
  • .venv/bin/pytest — 100 passed, 1 skipped

No live provider calls were made. Model-specific behavior should be compared before production deployment because prompt changes can still cause classification regressions.

@dtump
dtump merged commit 11034e0 into main Jul 30, 2026
4 checks passed
@dtump
dtump deleted the improve/system-prompt-calibration branch July 30, 2026 20:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant