rusty-diff

A lightweight automatic differentiation library in Rust for building and training neural networks.

Features

• Automatic Differentiation: Compute gradients automatically using reverse-mode autodiff (backpropagation)
• Neural Network Components: Built-in MLP, neurons, layers, and activation functions (Tanh, ReLU, Sigmoid, Identity)
• Optimization: SGD optimizer with configurable learning rate
• Loss Functions: MSE loss for regression tasks
• BLAS Support: Optional BLAS/Accelerate integration for improved performance

Quick Start

use rusty_diff::Value;

// Create values and build computational graph
let x = Value::new(3.0);
let y = Value::new(4.0);
let z = &(&x + &y) * &x;  // z = (x + y) * x

// Compute gradients
z.backward();
println!("dz/dx = {}", x.grad());  // 10
println!("dz/dy = {}", y.grad());  // 3

Performance Features

By default, rusty-diff uses pure Rust implementations. For improved performance, you can enable BLAS (Basic Linear Algebra Subprograms) support:

Using Accelerate (macOS)

On macOS, you can use Apple's Accelerate framework for optimized linear algebra operations:

cargo build --features accelerate
cargo run --example sine_regression --features accelerate

Using OpenBLAS

For cross-platform BLAS support, you can use OpenBLAS:

cargo build --features openblas
cargo run --example sine_regression --features openblas

Adding to Your Project

Add rusty-diff to your Cargo.toml:

# Pure Rust (default)
[dependencies]
rusty-diff = "0.1"

# With Accelerate support (macOS)
[dependencies]
rusty-diff = { version = "0.1", features = ["accelerate"] }

# With OpenBLAS support
[dependencies]
rusty-diff = { version = "0.1", features = ["openblas"] }

Note: BLAS backends can significantly improve performance for large matrix operations, especially beneficial when training larger neural networks.

Examples

Run the examples to see rusty-diff in action:

# Basic arithmetic and autodiff
cargo run --example basic_usage

# Train MLP to approximate sine function
cargo run --example sine_regression

The sine regression example trains a neural network and exports loss curves to output/losses.csv for visualization.

License

See LICENSE for details.