ANN from scratch

This project is part of course 22110 at Denmarks Technical University (DTU). The project aims at creating a well functioning artificial neural network (ANN) from scratch, using pure python. That is, no NumPy, Pandas, Keras, Tensorflow or any other supporting libraries.

The data:

The data is a part of a project at DTU HealthTech, which is about predicting whether certain variations of a Single-Nucleotide Polymorphism (SNP) will lead to a disease or not. The data folder consists of four files in which each line is a single datapoint. Three of the data sets are used for training and one is used for model evaluation.

The input for training is whitespace separated floating point numbers. There are 27 input variables and a single target value (1 = disease, 0 = healthy), as such:

0.075 0.3 0.075 0.225 -0.3 -0.075 0.15 -0.15 0 -0.3 -0.3 0.15 -0.3 -0.225 0 -0.15 -0.15 -0.3 -0.225 -0.3 -0.075 0.3 0.193746064573032135 1.011 0.7525 0.744312561819980218 0.612 0
-0.075 0.075 0.15 0.15 -0.375 -0.15 0.375 -0.15 -0.225 -0.375 -0.15 0.15 -0.3 -0.375 -0.3 -0.225 -0.225 -0.375 -0.3 -0.375 -0.225 0.15 0.173058043883721677 1.011 0.7525 0.744312561819980218 0.340 0
-0.3 -0.225 -0.15 -0.3 -0.45 -0.15 -0.225 -0.375 0.75 -0.375 -0.075 -0.225 -0.3 -0.3 -0.375 -0.3 -0.3 -0.375 -0.075 -0.375 -0.075 0.75 0.336279812283658667 1.011 0.7525 0.744312561819980218 0.100 1
-0.15 -0.3 0 -0.3 -0.225 0.075 -0.225 -0.375 0.6 -0.075 -0.15 -0.225 0 0.3 -0.225 -0.3 -0.3 -0.225 0.3 -0.3 -0.15 0.6 0.303728369615393505 1.011 0.7525 0.744312561819980218 0.212 1

The data set is made specifically for network training. It is therefore not dirty or corrupted in any way.

Modules:

• weights_and_biases.txt: .txt-file containing weights and biases describing the trained model
• -> utilities:
  • helpers_matrix.py: Utility functions for matrix and vector operations.
  • train_utils.py: Utilities for training, as well as some globals describing the structure of the ANN
  • predict_evaluate_utils.py: Utilities for prediction and evaluation
  • file_utils.py: Utilities for writing weights and biases to a txt-file and reading + transforming them back to matrices
• read_data.py: Reads the data, transforms it into a the correct datatype and shape, and divides it into training and test datasets.
• train.py: Initializes, creates and trains the neural net based on the training data. The model is implemented with a bias neuron on each layer - that way it learns better.
• predict_evaluate.py: Predicts the test data by using the tuned weights and biases from file, and evaluates those predictions.

Additional reading:

For improved understanding, see the following sources:

• Youtube series about deep learning basics
• Short high level article explaining the key concepts of an ANN
• Some calculus behind the backpropogation algorithm
• Explanation of stochastic gradient descent

End note

The main focus of this project is to build a well functioning artificial neural network - tuning hyperparameters and optimizing the accuracy of the model has therefore only been done to a certain satisfactory level. The prediction errors reflects this fact.

By: Eirik Runde Barlaug and Sara Heiberg