Appreciation for Cyberbro and sharing my Rust CLI implementation

[plang007kiku-svg]

Hi @stanfrbd,

I’ve been using Cyberbro for my cybersecurity research and threat intelligence work, and I wanted to express my sincere appreciation for such a well-structured and powerful project. It has been an invaluable foundation for my work.

As a way of giving back and exploring the codebase further, I've implemented a Rust CLI version to improve performance and enhance integration within my own workflow. I’m really enjoying the flexibility of your architecture.

Thank you for maintaining such a great tool for the community!

Best regards,
Kriangkrai Khatsom
Independent Cyber Security Researcher & OSCP Full-stack Web Developer at Hunter. Specialized (APT)

[stanfrbd]

Hi @plang007kiku-svg

Glad to see that you are using Cyberbro for your workflows!
I'm curious about the Go service and the Rust implementation of the CLI. I've also seen that you added multi-database support.
Could you elaborate on that if you have time?

[plang007kiku-svg]

Hi @stanfrbd, thanks for the great work on Cyberbro. I've integrated it into my personal research pipeline and wanted to share some observations.

1. Go microservice (lightweight pre-filter)
   In my lab, I generate a lot of netflow data (pcaps, firewall logs). Sending every single event directly to Cyberbro's Python engine would be overkill.
   I wrote a small Go service that:

Listens to the event stream (UDP or Redis pub/sub).

Applies cheap regex-based IOC extraction (IP, domain, hash patterns).

Forwards only the events that match any pattern to the main Cyberbro engine.

Why Go? Goroutines make it trivial to handle hundreds of concurrent flows, and the memory footprint stays under 50MB even under load.
In contrast, a Python multiprocessing approach would consume 3–5x more RAM for the same throughput.

2. Rust CLI (forensic & batch mode)
   I often get log files from various sources (SaaS exports, local IDS logs, etc.) that need to be analyzed offline, sometimes on machines without Python or Docker.
   I compiled a statically linked Rust binary that:

Reads log files line by line.

Extracts IOCs using the same logic as Cyberbro's engines/.

Outputs JSON lines (or CSV) for further processing.

Performance: Parsing a 2GB IIS log file takes ~11 seconds in Rust vs ~52 seconds in Python (on the same hardware). This becomes crucial when iterating over historical data or doing batch analysis across hundreds of log files.

3. Multi-database separation
   Originally, I used only SQLite (simplicity). But as the dataset grew, I ran into:

Lock contention when multiple processes tried to write simultaneously.

Performance degradation when storing large JSON payloads (full HTTP requests, process trees).

I switched to:

MySQL for structured, relational data (IOC lists, lookup tables).

MongoDB for document-oriented raw evidence (full API responses, PCAP metadata).

Redis for caching external API results (OTX, VT) to stay within rate limits and speed up repeated queries.

The beauty of Cyberbro's modular engine design is that I could implement these backends as plugins without touching the core IOC extraction logic.

Thanks again for a tool that's both practical and easy to hack on. It's been a great learning experience.

Best, Kriangkrai
Independent Cyber Security Researcher & OSCP Full-stack Web Developer at Hunter. Specialized (APT)