Available Languages

• English
• Español

Table of Contents

• What is NovaNN?
• Project Philosophy
• Numerical Backend
• Educational and Technical Goal
• Introduction
  • Project Organization
• Quick Start
  • Autograd and Computational Graphs
  • Complete Neural Network Training
  • CNN Architectures for Computer Vision
  • Transfer Learning - Freezing Layers
  • Fine-Tuning with Differential Learning Rates
  • Model Saving and Loading
• Technologies Used
• Installation
  • Install from PyPI (Recommended)
  • Install from source
  • Run examples
  • Uninstallation
  • Troubleshooting
• Testing
  • Run All Tests
• Contribution
• License
• Author and Maintainer

What is NovaNN?

NovaNN is a Deep Learning framework developed from scratch in Python, designed to build, train, and evaluate neural networks in a modular, clear, and extensible way.

The main goal of NovaNN is not to compete with industrial frameworks, but to understand, implement, and demonstrate how modern frameworks like PyTorch or TensorFlow work internally, with special emphasis on the architecture of PyTorch, which served as the main inspiration.

NovaNN allows defining complete neural models, managing training, and performing automatic backpropagation through a dynamic autograd engine, all built explicitly without relying on external computation engines.

Project Philosophy

NovaNN was born with a clear idea:

Don't use the magic of existing frameworks, but build it.

Every component of the framework is designed to be readable, traceable, and testable, prioritizing deep understanding of:

• How computational graphs are built
• How gradients flow during backward
• How scalable ML frameworks are structured
• How clean and extensible APIs are designed

Numerical Backend

NovaNN uses NumPy as the main backend for numerical computation, leveraging:

• Efficient vectorized operations
• Explicit tensor manipulation
• Complete control over mathematical operations

This allows focusing on the logic of Deep Learning (autograd, layers, optimization, training) without overly abstracting the internal system behavior.

Educational and Technical Goal

This framework was created for educational and demonstrative purposes, with the aim of demonstrating solid knowledge in:

• Machine Learning and Deep Learning
• Mathematical fundamentals (linear algebra, calculus, optimization)
• Autograd and backpropagation
• System design and software architecture
• Unit testing and numerical validation
• Modular and extensible design
• Software engineering best practices
• Data preprocessing and model training

NovaNN is intended for people who want to understand how Deep Learning frameworks really work inside, beyond just using them.

⚠️ Note NovaNN does not aim to replace frameworks like PyTorch or TensorFlow in production environments. Its purpose is to serve as an advanced learning tool and as a technical demonstration of engineering applied to Deep Learning.

Introduction

NovaNN adopts a modular organization inspired by modern Deep Learning frameworks, with responsibilities clearly separated between data, models, training, and utilities. This structure favors both extensibility and clarity of workflow.

Project Organization

• examples/ Contains functional scripts showing the use of the framework in different scenarios:

  • Binary classification
  • Multiclass classification
  • Regression
  • Convolutional networks

• nova/ Contains the complete core of the NovaNN framework. Here tensors, the autograd engine, mathematical operations, neural network modules, optimizers, metrics, serialization, and internal utilities are implemented. It is organized in a modular way to clearly separate the different levels of the system: low-level (tensors and operations), autograd, high-level APIs (nn, optim, metrics) and auxiliary utilities. Each submodule has its own documentation to facilitate code navigation and maintenance.

• benchmarks/ Includes benchmarks designed to evaluate NovaNN's performance in different scenarios and compare it with other frameworks (mainly PyTorch). The benchmarks focus on:

  • Elementary operations and reduction
  • Autograd system cost
  • CPU training on small models
  • Memory usage and computational overhead This directory is not part of the framework runtime and is intended exclusively for performance analysis, technical validation, and comparative studies.

Quick Start

Build and train models with a syntax you already know. NovaNN looks like PyTorch, but runs on your own custom engine.

1. Autograd and Computational Graphs

Experiment with NovaNN's automatic differentiation engine. Create tensors, perform operations, and observe how gradients flow.

import nova
import nova.nn as nn

# Create tensors with gradient tracking
x = nova.tensor([[0.5, -0.2]], requires_grad=True)
w = nova.tensor([[1.0], [0.5]], requires_grad=True)
b = nova.tensor([0.1], requires_grad=True)

# Manual forward pass
y = x @ w + b
loss = (y ** 2).sum()

# Automatic backward pass
loss.backward()

print(f"Loss: {loss.item()}")           # Loss: 0.25
print(f"Gradient of x: {x.grad}")     # Automatically calculated gradients
print(f"Gradient of w: {w.grad}")
print(f"Gradient of b: {b.grad}")

# Or using nn.Module layers
model = nn.Linear(2, 1)
output = model(x)
output.backward()

2. Complete Neural Network Training

Train a simple binary classifier with all the framework's functionalities.

import nova
import nova.nn as nn
import nova.optim as optim
from nova.nn import functional as F

# Define the model
class BinaryClassifier(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(10, 64)
        self.fc2 = nn.Linear(64, 32)
        self.fc3 = nn.Linear(32, 1)
        self.dropout = nn.Dropout(0.3)

    def forward(self, x):
        x = F.relu(self.fc1(x))
        x = self.dropout(x)
        x = F.relu(self.fc2(x))
        x = F.sigmoid(self.fc3(x))
        return x

# Create model and optimizer
model = BinaryClassifier()
optimizer = optim.Adam(model.parameters(), lr=0.001)
criterion = nn.BCELoss()

# Example data
X_train = nova.randn(100, 10)  # 100 samples, 10 features
y_train = nova.randint(0, 2, (100, 1))

# Training loop
for epoch in range(50):
    # Forward pass
    predictions = model(X_train)
    loss = criterion(predictions, y_train)

    # Backward pass
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    if (epoch + 1) % 10 == 0:
        print(f"Epoch {epoch+1}/50, Loss: {loss.item():.4f}")

3. CNN Architectures for Computer Vision

NovaNN supports complex modules like 2D convolutions, batch normalization, and lazy layers.

import nova
import nova.nn as nn
import nova.nn.functional as F

class ConvNet(nn.Module):
    def __init__(self, num_classes=10):
        super().__init__()

        # Convolutional block 1
        self.conv1 = nn.Conv2d(3, 32, kernel_size=3, padding=1)
        self.bn1 = nn.BatchNorm2d(32, momentum=0.1)

        # Convolutional block 2
        self.conv2 = nn.Conv2d(32, 64, kernel_size=3, padding=1)
        self.bn2 = nn.BatchNorm2d(64, momentum=0.1)

        # Convolutional block 3
        self.conv3 = nn.Conv2d(64, 128, kernel_size=3, padding=1)
        self.bn3 = nn.BatchNorm2d(128, momentum=0.1)

        self.pool = nn.MaxPool2d(2, 2)

        # Fully connected layers (lazy to automatically infer dimensions)
        self.fc1 = nn.LazyLinear(256, bias=False)
        self.bn4 = nn.BatchNorm1d(256)
        self.fc2 = nn.Linear(256, num_classes)

        self.dropout = nn.Dropout(0.5)

    def forward(self, x):
        # Feature extraction
        x = self.pool(F.relu(self.bn1(self.conv1(x))))  # 32x32 -> 16x16
        x = self.pool(F.relu(self.bn2(self.conv2(x))))  # 16x16 -> 8x8
        x = self.pool(F.relu(self.bn3(self.conv3(x))))  # 8x8 -> 4x4

        # Flatten
        x = x.view(x.size(0), -1)

        # Classification head
        x = F.relu(self.bn4(self.fc1(x)))
        x = self.dropout(x)
        x = self.fc2(x)

        return x

# Create model and process images
model = ConvNet(num_classes=10)
batch_images = nova.rand(8, 3, 32, 32)  # Batch of 8 RGB 32x32 images
logits = model(batch_images)

print(f"Output: {logits.shape}")  # Shape: (8, 10)

4. Transfer Learning - Freezing Layers

Leverage pre-trained models by freezing layers to use them as fixed feature extractors.

import nova
import nova.nn as nn
import nova.optim as optim
from nova.nn import functional as F

# Suppose we have a pre-trained model
class PretrainedFeatureExtractor(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(3, 64, 3, padding=1)
        self.conv2 = nn.Conv2d(64, 128, 3, padding=1)
        self.pool = nn.MaxPool2d(2, 2)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        x = self.pool(x)
        x = F.relu(self.conv2(x))
        x = self.pool(x)
        return x

# Complete model with transfer learning
class TransferLearningModel(nn.Module):
    def __init__(self, num_classes=5):
        super().__init__()
        # Pre-trained backbone (frozen)
        self.backbone = PretrainedFeatureExtractor()

        # New classification head (trainable)
        self.classifier = nn.Sequential(
            nn.LazyLinear(512),
            nn.ReLU(),
            nn.Dropout(0.5),
            nn.Linear(512, num_classes)
        )

    def forward(self, x):
        x = self.backbone(x)
        x = x.view(x.size(0), -1)
        x = self.classifier(x)
        return x

# Create model
model = TransferLearningModel(num_classes=5)

# Freeze feature extractor layers
for param in model.backbone.parameters():
    param.requires_grad = False

# Only train the classifier
optimizer = optim.AdamW(
    filter(lambda p: p.requires_grad, model.parameters()),
    lr=0.001
)

# Verify which parameters are trainable
trainable = sum(p.numel() for p in model.parameters() if p.requires_grad)
total = sum(p.numel() for p in model.parameters())
print(f"Trainable parameters: {trainable}/{total}")

5. Fine-Tuning with Differential Learning Rates

Train different parts of the network with different learning rates for optimal fine-tuning.

import nova
import nova.nn as nn
import nova.optim as optim
from nova.nn import functional as F

class FineTuneModel(nn.Module):
    def __init__(self):
        super().__init__()
        # Base layers (pre-trained)
        self.base_layers = nn.Sequential(
            nn.Linear(100, 256),
            nn.ReLU(),
            nn.Linear(256, 128),
            nn.ReLU()
        )

        # Middle layers
        self.mid_layers = nn.Sequential(
            nn.Linear(128, 64),
            nn.ReLU()
        )

        # Classification head (new)
        self.head = nn.Linear(64, 10)

    def forward(self, x):
        x = self.base_layers(x)
        x = self.mid_layers(x)
        x = self.head(x)
        return x

model = FineTuneModel()

# Configure differential learning rates per layer group
optimizer = optim.Adam([
    {'params': list(model.base_layers.parameters()), 'lr': 1e-5},  # Very low for base layers
    {'params': list(model.mid_layers.parameters()), 'lr': 1e-4},   # Medium
    {'params': list(model.head.parameters())}
], lr=1e-3) # High for new head

# Example data
X = nova.randn(32, 100)
y = nova.randint(0, 10, (32,))

# Training
for epoch in range(100):
    logits = model(X)
    loss = F.cross_entropy(logits, y)

    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    if (epoch + 1) % 20 == 0:
        print(f"Epoch {epoch+1}: Loss = {loss.item():.4f}")

6. Model Saving and Loading

Serialize your trained models to reuse them later.

import nova
import nova.nn as nn

# Train model
model = nn.Sequential(
    nn.Linear(10, 64),
    nn.ReLU(),
    nn.Linear(64, 10)
)

# Save complete model
nova.save(model, 'model.pt')

# Save only parameters (state_dict)
nova.save(model.state_dict(), 'model_weights.pt')

# Load complete model
loaded_model = nova.load('model.pt')

# Load only parameters into a new model
new_model = nn.Sequential(
    nn.Linear(10, 64),
    nn.ReLU(),
    nn.Linear(64, 10)
)
new_model.load_state_dict(nova.load('model_weights.pt'))

Technologies Used

The NovaNN framework is built using the following main technologies and libraries:

• Language: Python >= 3.14, < 3.15
• Dependency Management: Poetry (for package management and virtual environments)
• Main Libraries:
  • numpy: Efficient numerical operations and multidimensional arrays
  • pandas: Tabular data handling and analysis (for dataset loading)
  • matplotlib: Graph and result visualization
  • seaborn: Aesthetic enhancement of statistical visualizations
  • scikit-learn: Classical Machine Learning tools and utilities
  • pyarrow: Efficient backend for pandas DataFrames (reduces memory usage)
  • pyyaml: to manipulate YAML files
  • requests: To make web queries
  • tqdm: To show progress bar
• Development Tools:
  • pytest: Unit testing framework
  • pytest-cov: Code coverage in tests
  • ipykernel: Jupyter kernel for notebooks
  • black: Code formatter to maintain consistent style
• Benchmarking Tools
  • torch: Deep learning framework
  • torchvision: Extra torch package for vision tasks

Installation

NovaNN is available on PyPI and can be easily installed using pip or poetry. You can also install it from source if you want to contribute or explore the framework in depth.

Option 1: Install from PyPI (Recommended)

The easiest way to install NovaNN is via pip or poetry:

# With pip
pip install novann

# With poetry
poetry add novann

Verify installation

import nova
print(nova.__version__)  # Should display: 4.0.4

System requirements

• Python: >= 3.12, < 4.0.0
• Operating system: Windows, Linux, macOS

Option 2: Install from source

If you want to contribute to the project or explore the source code, you can clone the repository and install using Poetry.

1. Clone the repository

git clone git@github.com:JOSE-MDG/NovaNN.git
cd NovaNN

2. Install Poetry (if you don't have it)

• Windows (PowerShell):

(Invoke-WebRequest -Uri https://install.python-poetry.org -UseBasicParsing).Content | python -

• Linux/macOS:

# With curl
curl -sSL https://install.python-poetry.org | python3 -

# With pipx
pipx install poetry

Add Poetry to PATH:

• On Linux/macOS:

# Bash/Zsh (temporary)
export PATH="$HOME/.local/bin:$PATH"

# Bash (permanent - add to ~/.bashrc)
echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc

# Zsh (permanent - add to ~/.zshrc)
echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.zshrc
source ~/.zshrc

• On Windows

# PowerShell (temporary for current session)
$env:Path += ";$env:APPDATA\Python\Scripts"

# PowerShell (permanent - current user)
[System.Environment]::SetEnvironmentVariable("Path", $env:Path + ";$env:APPDATA\Python\Scripts", "User")

# PowerShell (permanent - system)
[System.Environment]::SetEnvironmentVariable("Path", $env:Path + ";$env:APPDATA\Python\Scripts", "Machine")

# Command Prompt (temporary)
set PATH=%PATH%;%APPDATA%\Python\Scripts

# Command Prompt (permanent)
setx PATH "%PATH%;%APPDATA%\Python\Scripts"

3. Install project dependencies

# Write lock file
poetry lock

# Install all dependencies (including development)
poetry install

# Production dependencies only
poetry install --without dev,benchmark

# With benchmarking tools
poetry install --with benchmark

4. Activate virtual environment

# Install shell plugin
poetry self add poetry-plugin-shell

# Activate shell with virtual environment
poetry shell

# Alternatively, run commands directly without activating shell:
poetry run python examples/binary_classification.py

Run examples

Once NovaNN is installed (from PyPI or source), you can run the included examples:

# If installed from source
poetry run python examples/binary_classification.py
poetry run python examples/multiclass_classification.py
poetry run python examples/conv_example.py
poetry run python examples/regression.py

# If installed from PyPI, create your own scripts
python my_neural_network.py

Uninstallation

# If installed from PyPI with pip
pip uninstall novann

# If installed with poetry
poetry remove novann

# If installed from source
# Simply remove Poetry's virtual environment
poetry env remove python

Troubleshooting

Error: "Python version not compatible"

NovaNN requires Python >=3.12, <4.0.0 Check your version:

python --version

If you have multiple Python versions, use:

poetry env use python3.14 # or python3.12/13

Error: "Module nova not found"

If installed from source, make sure you're in Poetry's virtual environment:

poetry shell

Or use poetry run before your commands.

Problems with NumPy or dependencies

If you have dependency conflicts, try:

pip install --upgrade pip
pip install novann --no-cache-dir

Testing

The framework includes a complete unit test suite in the tests/ directory that verifies correct implementation of all components covering 87% of the module. For more information go to Unit Tests

Run All Tests

# All tests
poetry run pytest

# Verbose tests
poetry run pytest tests/ -v

# Tests with coverage
poetry run pytest --cov

# Tests with html report
poetry run pytest --cov --cov-report=html

Contribution

To know how to contribute to NovaNN go to contributions

License

This project is under the MIT License. See the LICENCE file for more details.

MIT License Summary:

• Free software to use, copy, modify, merge, publish, distribute
• Can be used for commercial purposes
• License includes original copyright
• No warranty and authors are not responsible for damages

Author and Maintainer

Juan José - Developer & Machine Learning Enthusiast (16 years)

• GitHub: https://github.com/JOSE-MDG
• Email: josepemlengineer@gmail.com

About Me: At only 16 years old, I built NovaNN from scratch as an educational project to demonstrate my passion and deep understanding of deep learning. This framework represents months of self-study, experimentation, and dedication, implementing each algorithm mathematically from original papers.

Acknowledgments:

• Inspired by PyTorch and other deep learning frameworks
• Open source community for shared tools and knowledge
• Research papers that form the basis of implementations