🚀: LLM classifier improvements

[mdecimus]

Discussed in #2777

^{Originally posted by unixfox February 8, 2026}
Hello,

I'm coming from Mailcow where Rspamd provide an LLM classifier too: https://docs.rspamd.com/modules/gpt/ and https://github.com/rspamd/rspamd/blob/master/src/plugins/lua/gpt.lua

I'm sharing some suggestions regarding the current implementation in Stalwart.

Those suggestions are also part of my experience tuning the Rspamd LLM classifier as writing in my article: https://cybercarnet.eu/posts/email-spam-llm/

• Stalwart should send the sender email address to the LLM. Even though the Stalwart phishing protection is already good, LLM are very good at recognizing phishing.
  For example, if they see "From Paypal" but the real email address is not paypal.com, they immediately see the red flag.
  It's probably not the best example but for lesser known services like local real estate agency, the LLM may know the agency and flag email that try to impersonate the agency.
• Stalwart shouldn't send the email content to the LLM if the content is already classified as HAM (negative score).
  Rspamd has such mechanism it's:

  allow_ham: When true, messages that already look like ham (negative score) are still evaluated.
  

  It saves on cost and avoid sending everything to the LLM. On my own mail server (still with mailcow), 80% of the email that I receive have a negative score.
• Stalwart should have the ability to give more context to the LLM.
  Rspamd GPT plugin has implemented "web search context", it sends to the LLM web searches for all the links found in the email content. It works really great if the LLM doesn't know a specific local business and this business is just sending you a legitimate invoice.
  This mirrors how a human would handle uncertain cases by searching online for related information before deciding whether a message is spam.
  From my experience and as written in my article, most of the time if the LLM sees a message with something like "Please find attached your payment notice for the coming month.". It will always return: "urges the recipient to open an attachment to view a bill, which is a classic pattern for malware delivery or phishing attempts".
  But if you include a web search, their behavior is entirely different, and instead they return that the email is legitimate.

In my opinion, LLMs have strong potential for spam detection, and the Rspamd GPT plugin is a good example of this in practice. Taking inspiration from Rspamd’s approach could help improve the accuracy and effectiveness of Stalwart’s LLM classifier.

Thank you in advance for considering this enhancement request.

[unixfox]

I’d like to suggest one more suggestion, making the execution order of spam filter stages configurable, specifically allowing the LLM classifier to run after user-defined rules.

Currently, the LLM step runs before user rules. In setups using a sieve script, emails may already be rejected early by a sieve script so running the LLM beforehand can add unnecessary cost and latency.

I'm talking about this part of the code:

stalwart/crates/spam-filter/src/analysis/score.rs

Lines 244 to 263 in 2266634

	// LLM classification
	#[cfg(feature = "enterprise")]
	self.spam_filter_analyze_llm(ctx).await;
	// SPDX-SnippetEnd
	// Spam trap
	self.spam_filter_analyze_spam_trap(ctx).await;
	// Pyzor checks
	self.spam_filter_analyze_pyzor(ctx).await;
	// Model classification
	self.spam_filter_analyze_classify(ctx).await;
	// User-defined rules
	self.spam_filter_analyze_rules(ctx).await;
	// Final score calculation
	self.spam_filter_finalize(ctx).await

[nomadturk]

@mdecimus

It would be great if we can call the LLM classifier through a sieve script, after running some other controls in order to reduce the amount of time and cost.

I don't like the Spam filter as it was mostly generating F/P for me, and I have it disabled on my system.
But I'd like to use it either parts of it, or after I run my own tests, just to be sure.

Can't we call either LLM checks or the spam filter, using vnd.stalwart.expressions and system scripts?
Or is it impossible?

[mdecimus]

@nomadturk It was already possible to call the spam classifier from sieve scripts in previous releases without enabling it globally.

[kanylbullen]

This fix would be much appreciated. I did a quick calculation on last 30 days of email and I would save 58% on token use if I could skip LLM on score <0.

[chuckmckinnon]

TypeSafe's new Jev model seems uniquely well-suited to email classification, because there's a big difference between a phishing email and unsolicited sales emails of the type "I came across Stalwart and wanted to explore whether your team is interested in increasing qualified B2B leads..."

It looks like it would be pretty trivial to get Jev to return structured output like:

classification:
    legitimate
    transactional
    newsletter
    unsolicited_sales
    bulk_spam
    phishing
    malicious

recipient_likely_requested_this:
    yes/no

sender_impersonation_suspected:
    yes/no

action:
    inbox
    spam
    quarantine

While it's not urgent, it would be pretty cool to have Stalwart support Jev-style classifications as they look to be much faster and cheaper than traditional LLMs.

[mdecimus]

Thanks for the suggestions, yes, all of them will be implemented shortly after the v1.0.0 release (late October/early November).