dtump · dtump · Jul 21, 2026 · Jul 21, 2026 · Jul 21, 2026
@@ -0,0 +1,2 @@
+# Raw email fixtures legitimately contain quoted-printable whitespace.
+tests/fixtures/spam/*.eml -whitespace
@@ -47,11 +47,24 @@ pip install -e ".[dev]"
 ## Testing
 
 - **Framework**: pytest 9.0.3
-- **Run tests**: `pytest -v`
-- **All changes must pass**: `ruff check . && ruff format --check . && pytest`
+- **Run tests**: `.venv/bin/pytest -v` (or activate `.venv` first)
+- **All changes must pass**: `.venv/bin/ruff check . && .venv/bin/ruff format --check . && .venv/bin/pytest`
 - Tests live in `tests/` and mirror the module structure
 - Mock external APIs — never make real API calls in tests. Use `unittest.mock.patch` for API clients and `monkeypatch` for config paths
 - Use `tmp_path` for temporary files, `monkeypatch` for config isolation
+- `tests/fixtures/spam/` contains a reviewed corpus of 20 anonymised, distinct spam campaigns.
+  Use it for parser/formatting regression tests, and extend it only with clearly spam, non-duplicate
+  messages.
+- Fixture privacy is mandatory: remove delivery/authentication/`X-` headers and attachments, and
+  redact all recipient, organisation, domain, mail-server, and other identifying information from
+  decoded headers and text/HTML MIME parts. Keep the corpus privacy tests passing.
+- `tests/test_spam_corpus_live.py` is an opt-in LLM benchmark. It is skipped unless
+  `INSPAMITY_RUN_LIVE_LLM=1`; it must never make provider calls in ordinary tests or CI. A developer
+  with a private configured API key can run one fixture with
+  `INSPAMITY_RUN_LIVE_LLM=1 .venv/bin/pytest -m live_llm -v --durations=1 --log-cli-level=INFO`.
+  This logs each LLM result (classification, confidence, reason, and duration). Increase
+  `INSPAMITY_LIVE_LLM_FIXTURE_COUNT` (up to 20) for a more representative run. The local
+  `config.ini` needs only `settings.provider`, and the selected provider's `api_key` and `model`.
 
 ## Architecture Notes
 

@@ -125,7 +125,8 @@ timeout = 20.0
 [openai]
 api_key = your_api_key_here
 model = gpt-5.6-luna
-temperature = 0.0
+# GPT-5-class models do not support temperature; leave it unset for those models.
+# temperature = 0.0
 timeout = 20.0
 ```
 
@@ -156,6 +157,53 @@ You can adjust these settings by editing the installed plugin at `/etc/rspamd/pl
 4. Based on the response, rspamd adjusts the spam score accordingly
 5. Detailed logging provides insights into the decision-making process
 
+## 🧪 Testing and spam corpus
+
+The regular test suite includes an anonymised, reviewed corpus of 20 distinct
+spam campaigns in `tests/fixtures/spam/`. It exercises email parsing and
+formatting without calling an LLM, and verifies that recipient and mail-server
+identifiers are not present in the committed fixtures:
+
+```bash
+.venv/bin/pytest -v
+```
+
+For an opt-in local OpenAI benchmark, create the ignored `config.ini` in the
+project root with only these required settings:
+
+```ini
+[settings]
+provider = openai
+
+[openai]
+api_key = your_api_key_here
+model = your_model_id
+```
+
+`timeout` and `temperature` are optional; the benchmark needs neither. The
+project's `config.ini` is ignored by Git, and `/etc/inspamity/config.ini`, when
+present, takes precedence over it.
+
+Run one fixture by default:
+
+```bash
+INSPAMITY_RUN_LIVE_LLM=1 .venv/bin/pytest -m live_llm -v --durations=1 --log-cli-level=INFO
+```
+
+`--durations=1` only reports the single slowest test; it does not control the
+number of API calls. `--log-cli-level=INFO` displays one line per LLM response
+with its spam classification, confidence, reason, and duration. The default
+benchmark makes one API call. For a more representative run, increase the
+number of corpus fixtures (up to 20), for example:
+
+```bash
+INSPAMITY_RUN_LIVE_LLM=1 INSPAMITY_LIVE_LLM_FIXTURE_COUNT=20 \
+  .venv/bin/pytest -m live_llm -v --durations=20 --log-cli-level=INFO
+```
+
+The live tests expect every reviewed fixture to be classified as spam. They are
+skipped unless `INSPAMITY_RUN_LIVE_LLM=1` is set, so they never run in CI.
+
 ## 🐛 Troubleshooting
 
 - Check rspamd logs for errors: `sudo tail -f /var/log/rspamd/rspamd.log`

@@ -21,6 +21,6 @@ timeout = 20.0
 [openai]
 api_key = your_api_key_here
 model = gpt-5.6-luna
-# Temperature for AI responses (0.0 = deterministic). Remove to use model default.
-temperature = 0.0
+# GPT-5-class models do not support temperature; leave it unset for those models.
+# temperature = 0.0
 timeout = 20.0
@@ -30,6 +30,9 @@ include = ["email_utils*"]
 
 [tool.pytest.ini_options]
 testpaths = ["tests"]
+markers = [
+    "live_llm: tests that call a configured external LLM provider (opt-in only)",
+]
 
 [tool.ruff]
 target-version = "py310"

@@ -0,0 +1,39 @@
+# Anonymised spam corpus
+
+This directory contains 20 reviewed spam messages from July 2026.  It is a
+small, intentionally varied regression corpus for email parsing and optional
+local LLM benchmarking; every fixture's expected classification is **spam**.
+
+The source messages were selected as single representatives of their campaign.
+Near-identical deliveries (including messages differing only in recipient,
+transport headers, timestamps, or tracking identifiers) were excluded.
+
+All recipient and mail-server data was removed before committing the fixtures:
+
+- transport, authentication, delivery, and `X-` headers were discarded;
+- binary attachments were discarded;
+- recipient addresses, domains, names, and server IP addresses were replaced
+  in decoded headers and text/HTML MIME parts.
+
+| Fixture | Review rationale |
+| --- | --- |
+| `advance_fee_confidential_transaction.eml` | Vague lucrative confidential-transaction offer to undisclosed recipients. |
+| `advance_fee_fake_donation.eml` | Unsolicited EUR 4.5 million donation with a generic reply address. |
+| `advance_fee_fake_police_notice.eml` | Impersonates police/Europol to solicit a response. |
+| `advance_fee_fintrac_funds.eml` | FINTRAC-held-funds impersonation with an unrelated sender/reply address. |
+| `advance_fee_lottery_winner.eml` | Unsolicited lottery award and direct reply request. |
+| `phishing_benu_security_update.eml` | BENU security impersonation from an unrelated domain. |
+| `phishing_bitvavo_address_check.eml` | Bitvavo lookalike domain requesting an address check. |
+| `phishing_cloud_storage_payment.eml` | Generic cloud-payment suspension threat with unrelated links. |
+| `phishing_cloud_storage_renewal.eml` | Fake cloud-renewal expiry threat from an unrelated domain. |
+| `phishing_coinmerce_mica.eml` | Coinmerce/MiCA verification lure pointing to an unrelated target. |
+| `phishing_fake_card_transaction.eml` | Fake card transaction/security-alert social engineering. |
+| `phishing_fake_cjib_fine.eml` | CJIB fine impersonation with unrelated sender and payment link. |
+| `phishing_fake_coinbase_deposit.eml` | Unsolicited 18 BTC deposit notification with unrelated link. |
+| `phishing_fake_document_signature.eml` | Urgent document-signature impersonation. |
+| `phishing_fake_domain_termination.eml` | Fake domain termination/renewal notice. |
+| `phishing_fake_rvo_subsidy.eml` | RVO subsidy lure from an unrelated sender. |
+| `phishing_icloud_billing.eml` | Fake iCloud billing suspension with a short action deadline. |
+| `phishing_raiffeisen_pushtan.eml` | Raiffeisen pushTAN impersonation with unrelated links. |
+| `spam_wordpress_credential_injection.eml` | WordPress login message whose credentials contain crypto bait. |
+| `unsolicited_prescription_drugs.eml` | Unsolicited prescription-drug sales link. |
@@ -0,0 +1,122 @@
+MIME-Version: 1.0
+Reply-To: dawsondaniel1133@gmail.com
+From: dawsondanie <07898@srp2.ac.th>
+Date: Sat, 11 Jul 2026 15:32:10 +0530
+Message-ID:
+ <CAMzQiaTh6NPxp1S+QphvfcB+RSgdPbkQP0sNbRYS3=SqPxWRnw@mail.gmail.com>
+Subject: Hello,
+To: undisclosed-recipients:;
+Content-Type: multipart/alternative; boundary="00000000000019b4cf065652f45a"
+
+--00000000000019b4cf065652f45a
+Content-Type: text/plain; charset="utf-8"
+Content-Transfer-Encoding: quoted-printable
+MIME-Version: 1.0
+
+ *Hello,*
+
+*I would like to discuss a confidential and legitimate transaction
+opportunity with you that has the potential to be financially rewarding for
+both of us. I believe this proposal could lead to a mutually beneficial
+partnership, and I would appreciate the opportunity to share the details
+with you.*
+
+*If this is of interest, please reply at your earliest convenience, and I
+will provide you with more information.*
+
+*I look forward to hearing from you.*
+
+*Best regards,*
+Dawson Daniel
+My email id: dawsondaniel1133@gmail.com
+
+   -
+
+--00000000000019b4cf065652f45a
+Content-Type: text/html; charset="utf-8"
+Content-Transfer-Encoding: quoted-printable
+MIME-Version: 1.0
+
+<div dir=3D"ltr">
+<div class=3D"gmail-I_ZkbNhI gmail-D_FY gmail-W_6D6F"><div class=3D"gmail-msg=
+-body gmail-P_wpofO gmail-mq_AS"><div class=3D"gmail-jb_0 gmail-X_6MGW gmail-=
+N_6Fd5"><div><div id=3D"gmail-yiv3375765462"><div dir=3D"ltr">
+<div class=3D"gmail-yiv3375765462gmail-yiv0429902086gmail-yiv9804370622gmail-=
+yiv2371378472gmail-yiv0057422340gmail-yiv4302882220gmail-I_ZkbNhI gmail-yiv33=
+75765462gmail-yiv0429902086gmail-yiv9804370622gmail-yiv2371378472gmail-yiv005=
+7422340gmail-yiv4302882220gmail-D_FY gmail-yiv3375765462gmail-yiv0429902086gm=
+ail-yiv9804370622gmail-yiv2371378472gmail-yiv0057422340gmail-yiv4302882220gma=
+il-W_6D6F"><div class=3D"gmail-yiv3375765462gmail-yiv0429902086gmail-yiv98043=
+70622gmail-yiv2371378472gmail-yiv0057422340gmail-yiv4302882220gmail-msg-body =
+gmail-yiv3375765462gmail-yiv0429902086gmail-yiv9804370622gmail-yiv2371378472g=
+mail-yiv0057422340gmail-yiv4302882220gmail-P_wpofO gmail-yiv3375765462gmail-y=
+iv0429902086gmail-yiv9804370622gmail-yiv2371378472gmail-yiv0057422340gmail-yi=
+v4302882220gmail-mq_AS"><div class=3D"gmail-yiv3375765462gmail-yiv0429902086g=
+mail-yiv9804370622gmail-yiv2371378472gmail-yiv0057422340gmail-yiv4302882220gm=
+ail-jb_0 gmail-yiv3375765462gmail-yiv0429902086gmail-yiv9804370622gmail-yiv23=
+71378472gmail-yiv0057422340gmail-yiv4302882220gmail-X_6MGW gmail-yiv337576546=
+2gmail-yiv0429902086gmail-yiv9804370622gmail-yiv2371378472gmail-yiv0057422340=
+gmail-yiv4302882220gmail-N_6Fd5"><div><div id=3D"gmail-yiv3375765462gmail-yiv=
+0429902086gmail-yiv9804370622gmail-yiv2371378472gmail-yiv0057422340gmail-yiv4=
+302882220gmail-yiv9518327819"><div><div style=3D"font-size:12px;color:rgb(0,0=
+,0)">
+<div><strong>Hello,</strong></div>
+<div style=3D"font-family:tahoma;font-size:12px;color:rgb(0,0,0)"><strong>=C2=
+=A0</strong></div>
+<div><strong>I would like to discuss a confidential and legitimate=20
+transaction opportunity with you that has the potential to be=20
+financially rewarding for both of us. I believe this proposal could lead
+ to a mutually beneficial partnership, and I would appreciate the=20
+opportunity to share the details with you.</strong></div>
+<div style=3D"font-family:tahoma;font-size:12px;color:rgb(0,0,0)"><strong>=C2=
+=A0</strong></div>
+<div><strong>If this is of interest, <span style=3D"background-color:rgb(255,=
+255,255)">please</span> reply at your earliest convenience, and I will provid=
+e you with more information.</strong></div>
+<div style=3D"font-family:tahoma;font-size:12px;color:rgb(0,0,0)"><strong>=C2=
+=A0</strong></div>
+<div><strong>I look forward to hearing from you.</strong></div>
+<div style=3D"font-family:tahoma;font-size:12px;color:rgb(0,0,0)"><strong>=C2=
+=A0</strong></div>
+<div><strong>Best regards,</strong></div><div>
+Dawson Daniel
+
+</div>
+</div></div>
+</div></div></div></div><div class=3D"gmail-yiv3375765462gmail-yiv0429902086g=
+mail-yiv9804370622gmail-yiv2371378472gmail-yiv0057422340gmail-yiv4302882220gm=
+ail-jb_0 gmail-yiv3375765462gmail-yiv0429902086gmail-yiv9804370622gmail-yiv23=
+71378472gmail-yiv0057422340gmail-yiv4302882220gmail-X_6MGW gmail-yiv337576546=
+2gmail-yiv0429902086gmail-yiv9804370622gmail-yiv2371378472gmail-yiv0057422340=
+gmail-yiv4302882220gmail-N_6Fd5"></div></div>
+My email id: <a rel=3D"nofollow noopener noreferrer" class=3D"gmail-yiv337576=
+5462gmail-yiv0429902086gmail-yiv9804370622gmail-yiv2371378472gmail-yiv0057422=
+340gmail-yiv4302882220gmail-yiv3362108343gmail-yiv8566918399gmail-yiv30376457=
+46gmail-removed-link gmail-yiv3375765462gmail-yiv0429902086gmail-yiv980437062=
+2gmail-yiv2371378472gmail-yiv0057422340gmail-yiv4302882220gmail-yiv3362108343=
+gmail-yiv8566918399gmail-removed-link gmail-yiv3375765462gmail-yiv0429902086g=
+mail-yiv9804370622gmail-yiv2371378472gmail-yiv0057422340gmail-yiv4302882220gm=
+ail-yiv3362108343gmail-removed-link gmail-yiv3375765462gmail-yiv0429902086gma=
+il-yiv9804370622gmail-yiv2371378472gmail-yiv0057422340gmail-yiv4302882220gmai=
+l-removed-link gmail-yiv3375765462gmail-yiv0429902086gmail-yiv9804370622gmail=
+-yiv2371378472gmail-yiv0057422340gmail-removed-link gmail-yiv3375765462gmail-=
+yiv0429902086gmail-yiv9804370622gmail-yiv2371378472gmail-removed-link gmail-y=
+iv3375765462gmail-yiv0429902086gmail-yiv9804370622gmail-removed-link gmail-yi=
+v3375765462gmail-yiv0429902086gmail-removed-link gmail-yiv3375765462gmail-rem=
+oved-link gmail-removed-link">dawsondaniel1133@gmail.com</a>
+
+</div>
+</div></div></div></div><div class=3D"gmail-jb_0 gmail-X_6MGW gmail-N_6Fd5"><=
+/div></div><div class=3D"gmail-H_7jIs gmail-D_F gmail-ab_C gmail-Q_69H5 gmail=
+-E_36RhU"><div class=3D"gmail-D_F gmail-W_6D6F gmail-r_BN gmail-gl_C"><span c=
+lass=3D"em_N en_N"></span><div class=3D"gmail-D_F gmail-ab_C en_0 gmail-b_Z14=
+vXdP e_3mS2U gmail-I_ZkbNhI gmail-P_Z1otBpf"><ul tabindex=3D"-1" role=3D"menu=
+bar" class=3D"gmail-hd_n gmail-P_0 gmail-H_6D6F gmail-M_0 gmail-ir3_0 gmail-i=
+y_h gmail-A_6EqO gmail-c1AVi73_6FsP gmail-D_F gmail-ab_C"><li class=3D"gmail-=
+G_e gmail-p_R" role=3D"menuitem"><span class=3D"gmail-r_P gmail-D_F gmail-M_1=
+Eu5U7 gmail-cdPFi_Z1oI2WW gmail-cdPFi4_Z29WjXl gmail-rtlR_h_2gl0WC"></span><b=
+r></li></ul></div></div></div>
+
+<br></div>
+
+--00000000000019b4cf065652f45a--
@@ -0,0 +1,46 @@
+DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra.efbet.com 6547818BBCC4
+Date: Sat, 04 Jul 2026 22:01:24 +0000
+From: "Ms. Schaeffler" <pazardzhik.fm@efbet.com>
+Reply-To: "Ms. Schaeffler" <thegmbhfdic@outlook.com>
+Message-ID: <649954923.17839641.1783202484535.JavaMail.zimbra@efbet.com>
+Subject: AMN: Gefeliciteerd!
+MIME-Version: 1.0
+Content-Type: multipart/alternative;
+ boundary="=_373fc15c-dfbe-42ed-83b2-303e61988f67"
+Thread-Index: ORys/megSyCRN8BJ6IwvQ6z6jMQIWQ==
+Thread-Topic: Gefeliciteerd!
+
+--=_373fc15c-dfbe-42ed-83b2-303e61988f67
+Content-Type: text/plain; charset="utf-8"
+Content-Transfer-Encoding: quoted-printable
+MIME-Version: 1.0
+
+
+
+Gefeliciteerd!=20
+
+Uw e-mailadres is willekeurig geselecteerd en u ontvangt een donatie van =E2=
+=82=AC 4.500.000,00. Neem voor meer informatie contact met ons op via onderst=
+aand e-mailadres.=20
+
+Neem uitsluitend contact met ons op via onderstaand e-mailadres.=20
+
+E-mail: gmbhschaefflerag@gmail.com=20
+
+Schaeffler GMBH=20
+
+--=_373fc15c-dfbe-42ed-83b2-303e61988f67
+Content-Type: text/html; charset="utf-8"
+Content-Transfer-Encoding: quoted-printable
+MIME-Version: 1.0
+
+<html><body><div style=3D"font-family: arial, helvetica, sans-serif; font-siz=
+e: 12pt; color: #000000"><div><br></div><div><br></div><div data-marker=3D"__=
+SIG_PRE__">Gefeliciteerd!<br><br>Uw e-mailadres is willekeurig geselecteerd e=
+n u ontvangt een donatie van =E2=82=AC 4.500.000,00. Neem voor meer informati=
+e contact met ons op via onderstaand e-mailadres.<br><br>Neem uitsluitend con=
+tact met ons op via onderstaand e-mailadres.<br><br>E-mail: gmbhschaefflerag@=
+gmail.com<br><br>Schaeffler GMBH</div></div></body></html>
+
+--=_373fc15c-dfbe-42ed-83b2-303e61988f67--
+
@@ -0,0 +1,72 @@
+DKIM-Filter: OpenDKIM Filter v2.10.3 fut003.vmail.ae BEB80C2E9B19
+Date: Wed, 08 Jul 2026 09:44:45 +0400
+From: Korpschef JANNY KNOL <info@gltg.ae>
+Reply-To: korpschef.politienationale@myyahoo.com
+To:
+ korpschef politienationale <korpschef.politienationale@politienationale.nl>
+Message-ID: <972278551.1311896.1783489485764.JavaMail.zimbra@gltg.ae>
+Subject: =?utf-8?q?OFFICI=C3=8BLE?= OPROEP: FEDERALE POLITIE ZAAK #500-2025-NL
+MIME-Version: 1.0
+Content-Type: multipart/mixed;
+ boundary="----=_Part_1311889_2567609.1783489485757"
+Thread-Index: QvVSe0ajd0YRDpnz1jhMFAztma+XAg==
+Thread-Topic: =?utf-8?q?OFFICI=C3=8BLE?= OPROEP: FEDERALE POLITIE ZAAK
+ #500-2025-NL
+
+------=_Part_1311889_2567609.1783489485757
+Content-Type: multipart/alternative; 
+	boundary="=_757ed727-7b29-49b8-94aa-fbcd86facea1"
+
+--=_757ed727-7b29-49b8-94aa-fbcd86facea1
+Content-Type: text/plain; charset="utf-8"
+Content-Transfer-Encoding: quoted-printable
+MIME-Version: 1.0
+
+Geachte,=20
+Hierbij delen wij u mee dat er een offici=C3=ABle klacht tegen u is ingediend=
+ in het kader van een gezamenlijk onderzoek met de diensten van de Europese P=
+olitie (Europol).=20
+Het exacte onderwerp van deze klacht vereist uw dringende opheldering.=20
+U bent verplicht onmiddellijk contact op te nemen met de aangewezen onderzoek=
+sdienst door te reageren op dit e-mailadres: korpschef.politienationale@myyah=
+oo.com=20
+Gelieve deze kennisgeving als een wettelijke waarschuwing te beschouwen en de=
+ze niet te negeren.=20
+Hoogachtend,=20
+De Afdeling Juridische Zaken en Cyberbeveiliging (Onder gezag van Europol)=20
+
+BELANGRIJK: Technische instructie Indien dit bericht in uw ongewenste e-mail =
+(spam) is terechtgekomen, dient u het onmiddellijk te markeren als "geen spam=
+" en te verplaatsen naar uw inbox. Dit is noodzakelijk om de bijgevoegde docu=
+menten correct te kunnen lezen en verdere blokkades door uw provider te voork=
+omen. Het niet tijdig openen van dit dossier door technische filters ontslaat=
+ u niet van uw antwoordplicht binnen 72 uur.=20
+
+
+--=_757ed727-7b29-49b8-94aa-fbcd86facea1
+Content-Type: text/html; charset="utf-8"
+Content-Transfer-Encoding: quoted-printable
+MIME-Version: 1.0
+
+<html><body><div style=3D"font-family: arial, helvetica, sans-serif; font-siz=
+e: 12pt; color: #000000"><div data-marker=3D"__QUOTED_TEXT__"><div style=3D"f=
+ont-family:'arial' , 'helvetica' , sans-serif;font-size:12pt;color:#000000"><=
+div>=C2=A0Geachte,<br>Hierbij delen wij u mee dat er een offici=C3=ABle klach=
+t tegen u is ingediend in het kader van een gezamenlijk onderzoek met de dien=
+sten van de Europese Politie (Europol).<br>Het exacte onderwerp van deze klac=
+ht vereist uw dringende opheldering.<br>U bent verplicht onmiddellijk contact=
+ op te nemen met de aangewezen onderzoeksdienst door te reageren op dit e-mai=
+ladres: korpschef.politienationale@myyahoo.com<br>Gelieve deze kennisgeving a=
+ls een wettelijke waarschuwing te beschouwen en deze niet te negeren.<br>Hoog=
+achtend,<br>De Afdeling Juridische Zaken en Cyberbeveiliging (Onder gezag van=
+ Europol)<br>=C2=A0<br>BELANGRIJK: Technische instructie Indien dit bericht i=
+n uw ongewenste e-mail (spam) is terechtgekomen, dient u het onmiddellijk te =
+markeren als "geen spam" en te verplaatsen naar uw inbox. Dit is noodzakelijk=
+ om de bijgevoegde documenten correct te kunnen lezen en verdere blokkades do=
+or uw provider te voorkomen. Het niet tijdig openen van dit dossier door tech=
+nische filters ontslaat u niet van uw antwoordplicht binnen 72 uur.=C2=A0</di=
+v></div><br></div></div></body></html>
+
+--=_757ed727-7b29-49b8-94aa-fbcd86facea1--
+
+------=_Part_1311889_2567609.1783489485757--
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,2 @@
		# Raw email fixtures legitimately contain quoted-printable whitespace.
		tests/fixtures/spam/*.eml -whitespace