LLM-Streamliner

A Rust SDK for compressing LLM context into expandable memory modules, solving context window limitations.

Features

• Incremental Compression: Compress large contexts into compact memory modules
• On-Demand Expansion: Reconstruct original content when needed
• Multiple Algorithms: Support for various compression backends (Zlib by default, optional Gzip and LZ4)
• Serialization: Save/load compressed modules
• Async Ready: Designed for async workflows

Installation

Add to your Cargo.toml:

[dependencies]
llm-streamliner = "0.1"

Toolchain

The crate targets the Rust 2021 edition and is tested on the stable channel. Our CI runs cargo test, clippy, and fmt using stable Rust, while the optional benchmark workflow uses nightly only for the built-in benchmark harness.

Usage

use llm_streamliner::{MemoryModule, ZlibCompressor, ZlibExpander};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Compress context
    let context = "Your large LLM context here...";
    let compressor = ZlibCompressor;
    let module = MemoryModule::new(context, &compressor).await?;
    
    // Serialize
    let json = module.to_json()?;
    
    // Later... deserialize and expand
    let loaded_module = MemoryModule::from_json(&json)?;
    let expander = ZlibExpander;
    let expanded = loaded_module.expand(&expander).await?;
    
    println!("Original length: {}", context.len());
    println!("Compressed size: {}", json.len());
    println!("Expanded matches original: {}", expanded == context);
    
    Ok(())
}

Compression Algorithms

Algorithm	Feature Flag	Description
Zlib	-	Default compression (good balance of speed/ratio)
Gzip	gzip	Higher compression ratio
LZ4	lz4	Faster compression

Enable features in Cargo.toml:

llm-streamliner = { version = "0.1", features = ["gzip"] }

GzipCompressor/GzipExpander and Lz4Compressor/Lz4Expander are only available when their corresponding feature flags are enabled.

Benchmarks

Performance metrics available in BENCHMARKS.md:

Zlib Compression (1MB data)

• Avg time: ~1.7ms
• Compression ratio: ~60% (varies by content)
• Memory overhead: <1MB

Run benchmarks locally:

cargo bench --features=benchmarks

Contributing

Contributions welcome! Please see CONTRIBUTING.md for guidelines.

License

MIT - See LICENSE for details.