Neural Network from Scratch in Rust

This project implements a simple neural network framework from scratch in pure Rust. It includes custom implementations of matrices, neurons, layers, forward propagation, backward propagation, and gradient descent—without relying on deep learning libraries like TensorFlow or PyTorch.

The goal is to create a fully functional neural network while understanding the low-level mechanics of training, weight updates, and activation functions.

---

🚀 Features

• Matrix Implementation (matrix.rs)

  • Matrix creation (zeros, random initialization).
  • Basic operations: addition, subtraction, multiplication, scalar multiplication.
  • Dot product, transpose, ReLU activation + derivative.
  • Mean Squared Error (MSE) loss and gradient computation.

• Neuron Implementation (neuron.rs)

  • Each neuron holds weights, biases, and connections to previous/next neurons.
  • Forward evaluation with ReLU activation.
  • Backpropagation with gradient updates for weights and biases.
  • Cycle detection in neuron graph (avoids infinite loops).

• Layer Implementation (layer.rs)

  • Multiple neurons grouped into layers.
  • Supports forward propagation (averaging across neurons).
  • Supports backward propagation with gradient updates.
  • Connects layers together into a feedforward network.

• Training Loop (main.rs)

  • Builds a network: input layer → hidden layers → output layer.
  • Runs forward + backward passes.
  • Uses random input data and trains the network to learn the function: output = 2 × input.
  • Prints loss per epoch.

---

🧩 Project Structure

.
├── matrix.rs     # Matrix struct + operations
├── neuron.rs     # Neuron struct + forward & backward propagation
├── layer.rs      # NetworkLayer struct + layer connections
├── main.rs       # Network setup + training loop

---

🔧 How to Run

1. Clone the repository:

   git clone https://github.com/SeerBlazeJ/rust-nn.git
   cd rust-nn

2. Run the project:

   cargo run

3. Example output:

   Epoch 1 - Loss: 1.9234
   Epoch 2 - Loss: 1.7421
   ...
   Epoch 6 - Loss: 0.4129
   

---

📚 Example: Training Task

Currently, the network is used to be trained for approximating a simple function:

Target:

y = 2 × x

Process:

• Input: Randomly generated matrix.
• Output: Forward pass through network.
• Loss: MSE between network output and target (scaled input).
• Backpropagation: Adjusts weights and biases with gradient descent.

---

🛠 Next Objectives

1. Multithreading Support (Planned)

   • Use Rust’s rayon or threading APIs to parallelize:

     • Matrix operations (multiplication, dot products).
     • Forward/backward passes across neurons/layers.

   • Expected result: Faster training on larger networks.

2. Improvements & Extensions

   • Add more activation functions (sigmoid, tanh, leaky ReLU).
   • Support mini-batch training.
   • Save/load trained weights.
   • Extend to classification tasks (softmax + cross-entropy loss).

---

🧠 Why This Project?

This project is designed to:

• Learn deep learning fundamentals by implementing every detail manually.
• Strengthen understanding of Rust memory safety, ownership, and concurrency.
• Provide a portfolio-ready example of combining systems programming with machine learning.

---

📜 License

This project is licensed under the MIT License.

---