Deep Learning with elixir
This project is called Deep Pipe(DP)

Network example (See test.ex)

defnetwork init_network2(_x) do
  _x |> f(5,5) |> flatten
  |> w(576,100) |> b(100) |> sigmoid
  |> w(100,10) |> b(10) |> sigmoid
end

Install:
  sudo apt install build-essential
  sudo apt-get install build-essential erlang-dev libatlas-base-dev
  mix deps.get


Usage:
  iex -S mix

module DP is Deep Pipe(DP) module
module CTensor is code for CNN data operation
module Cmatrix is code for Matrix
module MNIST is code for MNIST data set
module Time is time/1 for measure execution time

I implemented backpropagation and numerical-gradient
Now I'm testing small data set.


expample:
iex(1)> require Time

iex(1)> require(Time)
Time
iex(2)> Time.time(Test.momentum(30,100))
preparing data
ready
2.866287227629866
2.5600212240059506
...
0.04318082027257467
0.029026173275906994
0.03131037967594155
0.06550367669302301
accuracy rate = 0.879
"time: 55248299 micro second"
"-------------"
:ok

specification:

data structure
matrix  data structure of Matrex(CBLAS) e.g. m[2*3]
row_vector data structure of Matrex(CBLAS)  e.g. m[1*3]
tensor e.g. [m[2*3],n[2*3]]
network e.g. [{:weight,w,lr,v},{:bias,b,lr},{:function,f,g}]
weight {:weight,w,lr,v} w is matrix, lr is learning rate, v is for momentum,adagrad,adam
bias   {:bias,b,lr,v} b is row vector
filter {:filter,w,lr,st,v} st is stradd for convolution
pad    {:pad,n} n is size of padding
pool   {:pool,st} st is stradd
function {:function,f,g,h} f is function, g is differential function, h is function name
softmax {:softmax,f,_} f is function, only output layer softmax is set with cross_entropy

module macros
defnetwork is macros to describe network
argument must have under bar to avoid warning message

w(m,n)  weight matrix size(m,n). elements are Gaussian distribution random float
w(m,n,lr) lr is learning rate (default is 0.1)
w(m,n,lr,z) z is multiple for Gaussian distribution random float. (default is 0.1)

b(n) bias row_vector size(n). elements are all zero
b(n,lr) lr is learning rate (default is 0.1)

function sigmoid,relu,ident,softmax

f(m,n) filter matrix size(m,n). elements are Gaussian distribution random float
f(m,n,lr) lr is learning rate
f(m,n,lr,z) z is multiple for Gaussian distribution random float.(default is 0.1)
f(m,n,lr,z,st) st is stradd

pad(n) padding n is size of padding

pool(st) pooling stradd size is st

module DP
forward/2 forward calculation for batch data
forward(x,network) x is data(matrix or tensor) , network

numerical_gradient/3 calculate gradient by numerical differentiation
numerical_gradient(x,network,t) x is data, t is train data, loss function is mean_square
numerical_gradient(x,network,t,:cross) loss function is cross_entropy
numerical_gradient(x,network,t,:square) loss function is mean_square

gradient/3 caluculate gradient by backpropagation
gradient(x,network,t)  x is data, t is train data

learning/2, /3 update network with gradient
learning(network,gradient)  update with sgd
learning(network,gradient,:sgd)
learning(network,gradient,:momentum) update with momentum method
learning(network,gradient,:adagrad) update with adagrad method
learning(network,gradient,:adam) update with adam method

print/1 print data
newline/0 print LF
save/2 save network data to file
save(filename,network)
load/1 load network data from file
load(filename)

mean_square/2 loss function
mean_square(y,t) y is row vector of result and t is row vector of train
cross_entropy/2 loss function
cross_entropy(y,t) y is row vector of result and t is row vector of train

module MNIST
train_image(n)  get train image data size of n. Each data is 28*28 matrix
train_image(n,:flatten) get train image data size of n. Each data is 784 row_vector
train_label_onehot(n) get train label data aas onehot row_vector

test_image(n)  get test image data size of n. Each data is 28*28 matrix
test_image(n,:flatten) get test image data size of n. Each data is 784 row_vector
test_label_onehot(n) get test label data aas onehot row_vector

module time
time/1
time(func) e.g. time(1+2)