ML4all/index.html at master · s4ac/ML4all · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
<!DOCTYPE html>
<html>

<head>
	<meta charset="UTF-8">
	<meta http-equiv="X-UA-Compatible" content="IE=edge">
	<meta name="viewport" content="width=device-width, initial-scale=1">

	<title>simplePerceptron</title>
	<script type="text/javascript" src="http://latex.codecogs.com/latexit.js"></script>
	<script src='https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/MathJax.js?config=TeX-MML-AM_CHTML'></script>
	<script src="jquery-3.3.1.min.js"></script>
	<script src="simplePerceptron/libraries/p5.js"></script>
	<script src="simplePerceptron/libraries/p5.dom.js"></script>
	<script src="simplePerceptron/libraries/p5.sound.js"></script>
	<script src="simplePerceptron/perceptron.js"></script>
	<script src="simplePerceptron/trainer.js"></script>
	<script src="simplePerceptron/sketch.js"></script>
	<link rel="stylesheet" href="style.css">
	<script src="codemirror/lib/codemirror.js"></script>
	<link rel="stylesheet" href="codemirror/lib/codemirror.css">
	<script src="codemirror/mode/javascript/javascript.js"></script>
	<style>
	</style>
</head>

<body>
	<div class="container">
		<h1>Semi serious attempt to understand the core concepts of Machine Learning</h1>

		<h2>What is a PERCEPTRON</h2>
		A perceptron is an artificial neuron, it was developed between the 50’s and 60’s by the scientist
		<a href="https://en.wikipedia.org/wiki/Frank_Rosenblatt">Frank Rosenblatt</a>. But before we start to describe how an artificial neuron works it is worth to understand how our
		biological neurons operate.
		<br>
		<blockquote>
			“The biological neuron is a nerve cell that provides the fundamental functional unit for the nervous system of all animals”
			<a href="#1">[1]</a>
		</blockquote>

		<h2>That said how does it really work?</h2>
		<img class="images" src="img/NEuron.png" alt="">
		<br> Before getting into the functionalities let’s see how a neuron is made. Every neuron is composed by a cell body called
		soma. The soma itself is attached to many dendrites but to only one axon. (Fig. 1) The Axon is a very long fiber that eventually
		branches to connect to other neurons. The connection between the axon and the neuron is composed by two parts: the synapse
		and the dendrite. The synapse process the impulse from the axon and transmits it to the dendrite that will deliver the
		impulse to the soma or cell body. The soma reads an interprets the information carried by the impulse and sends a new impulse
		to the axon. This procedure goes on and on endlessly. Neurons basically communicate each other via electro-chemical impulses.
		The impulse to proceed from one neuron to the other must
		<blockquote>“be strong enough to activate the the release of chemicals across a synaptic cleft.“
			<a href="#2">[2]</a>
		</blockquote> The impulse must surpass a the threshold of the synapses otherwise the chemicals will not be released.

		<h2>How does an artificial neuron work?</h2>
		<img class="images" src="img/perceptron.png" alt="">
		<br> As already said at the end of the 50’s Frank Rosenblatt developed the first artificial neuron modeled onto the biological
		one: the perceptron. The perceptron is a linear model for binary classification. It takes some inputs, those inputs are
		than processed with dot product between themselves and the weights. The result is than fed to the step function that returns
		0 or 1. The nature of the output gives the perceptron the possibility to learn to classify between two distinct states.
		This means that it is a linear classifier, it can classify things that in a graph can be separated by a line.(???) Before
		we deepen into the mathematical and algorithmic concepts regarding the perceptron it is worth to describe what the weights
		are. The weights were introduced by Rosenblatt in his perceptron model. Those numbers express
		<blockquote>
			“the importance of the respective inputs to the output.”
			<a href="#3">[3]</a>
		</blockquote>Therefore the weights define which input is more important in order to get a desired output.
		<u>The whole purpose of machine learning is to tune the weights of artificial neurons, so that given a set of inputs the neuron
			will output a correct answer.</u>
		It is important to make clear how important it is to understand the role of the weights. If we look again at the fig. 2 we
		see a perceptron. Let’s assume this perceptron can classify whether an animal is a dog or a cat given two inputs value
		like size and sound/pitch (assuming that is possible to encode sound in a unique number). The value of the weights determines
		the output. In the case of the dog the weights favor the size as a cat has always lower size while cats weights favor the
		pitch as cats have an higher pitch than dogs. Of course chiwawas might be confused as dogs. But this because we would need
		much more inputs to really classify between the two species.
		<h2>How does the weights tuning happen?</h2>
		One of the methodologies for tuning the weights is with the method of
		<b>supervised learning</b>. Here there is an great explanation by
		<a href="http://natureofcode.com/book/chapter-10-neural-networks/">Daniel Shiffman</a>
		<blockquote>
			With this method[supervised learning], the network is provided with inputs for which there is a known answer. This way the
			network can find out if it has made a correct guess.
			<u>If it’s incorrect, the network can learn from its mistake and adjust its weights.</u> The process is as follows:
			<br>
			<ol>
				<li>Provide the perceptron with inputs for which there is a known answer.
				</li>
				<li>Ask the perceptron to guess an answer.
				</li>
				<li>Compute the error. (Did it get the answer right or wrong?)
				</li>
				<li>Adjust all the weights according to the error.
				</li>
				<li>Return to Step 1 and repeat!
					<a href="#4">[4]</a>
				</li>
			</ol>
		</blockquote>

		Let’s dig into the mechanics of the perceptron. The first thing to understand is this formula, the dot product:
		<br>
		<br>
		<p class="math">$$\sum_{i=1}^{n} x_{i} w_{i} $$</p>
		<br> Given a perceptron with 5 inputs the result of the dot product will be
		<br>
		<br>
		<p class="math">$$w_{1}x_{1}+w_{2}x_{2}+w_{3}x_{3}...+w_{5}x_{5}$$</p>
		<br> And here the
		<code>JavaScript</code> implementation
		<br>
		<br>
		<div class="snippet" id="code1">
			<script>
				setCode('simplePerceptron/feedForward.txt', 'code1');
			</script>
		</div>
		<br> The the result of the dot product is the sum of the product of each input with its own weight. There is an important thing
		to add: the bias. As you can see above the we are dealing with a sum of products. Therefore if we have an input value equal
		to 0 it will always return 0. To avoid that we insert a third input called the bias that we set to 1.
		<br>
		<img class="images" src="img/perceptron2.png" alt="">
		<br> On the the other hand the step function will look like this
		<br>
		<br>
		<p class="math">
			$$f(x)=\begin{cases}\phantom{}0 & x > 0 \\ 1 & x \leq 0\end{cases}$$
		</p>
		<br> And here the
		<code>JavaScript</code> implementation
		<br>
		<br>
		<div class="snippet" id="code2">
			<script>
				setCode('simplePerceptron/activation.txt', 'code2');
			</script>
		</div>
		<br> Therefore if the result of the dot product is greater or equal than 0 the result will be 1 otherwise it will be 0.
		<br> So far so good, now we need the last piece, a way to train our perceptron. To do this we will teach the the artificial
		neuron to learn from his errors with the following formula:
		<br>
		<br>
		<p class="math">
			$$\Delta W = error \times x_{i}$$
		</p>
		<br> And here the
		<code>JavaScript</code> implementation
		<br>
		<br>
		<div class="snippet" id="code3">
			<script>
				setCode('simplePerceptron/train.txt', 'code3');
			</script>
		</div>
		<br> As already mentioned the perceptron is good in classifying linearly separable problems. Than lets consider a very simple
		situation where we have a two-dimensional space separated by a wall and we want our perceptron to be able to classify if
		the a random point placed in the space is either on one side or on the other.
		<br> First we need to build our perceptron.
		<br>
		<br>
		<div class="snippet" id="code4">
			<script>
				setCode('simplePerceptron/perceptron.js', 'code4');
			</script>
		</div>
		<br> The
		<code>Trainer class</code> will help us in classifying data and labeling it
		<br>
		<br>
		<div class="snippet" id="code5">
			<script>
				setCode('simplePerceptron/trainer.js', 'code5');
			</script>
		</div>
		<br>
		<br> The first thing to do will be to create a linear function like this one
		<br>
		<br>
		<div class="snippet" id="code6">
			<script>
				setCode('simplePerceptron/function.txt', 'code6');
			</script>
		</div>
		<br>
		<br>this will separate the canvas into two parts. Than we create our data-set with labels with the following code
		<br>
		<br>
		<div class="snippet" id="code7">
			<script>
				setCode('simplePerceptron/data.txt', 'code7');
			</script>
		</div>
		<br>
		<br> And here is our perceptron classifying if a dot is on one side or the other of the line
		<br>
		<br>
		<div id="perceptronCanvas">

		</div>
		<button onclick="play()">PLAY/PAUSE</button>
		<button onclick="displayPerceptron()">SHOW PERCEPTRON</button>
		<br>
		<br>If you visualize the perceptron you can see that after enough training the weights will stabilize. When this point is reached
		the perceptron is trained and will always recognize if a point is above or below the line.
		<br>
		<br>
		<h1>If you made it until here than tap on your shoulder, you understood how a perceptron works!</h1>
		<br>
		<br>
		<h2>Neural Networks</h2>
		<br>
		<br>We already mentioned that a a single perceptron can ony solve
		<b>linearly separable</b> problems. But what does it mean?
		<br> Let's look at the image below:
		<br>
		<br>
		<img class="images" src="img/linear.png" alt="">
		<br>
		<br>The image shows two data-sets, the one on the left is linearly separable, similar to our experiment above. The data-set
		on the right shows data that is not linearly separable.
		<b>Linearly separable</b> data-sets are ones where it is possible to separate the data with a straight line, or a plane in
		a three dimensional dataset. However if the data is distributed like in the example on the right side than it is not linearly
		separable anymore, and therefore a perceptron will not be able to solve such a problem. A famous non linearly separable
		problem is the XOR truth table.
		<br>
		<br>
		<img class="images" src="img/XOR.png" alt="">
		<br>
		<br> The XOR logical operator returns true if one of the two elements is false, and returns false if they are both either true
		or false. It is impossible to draw a single line that separates the true from the false, therefore one perceptron can't
		solve such a problem. To do so we need
		<b>multi-layered perceptron</b>, a network of many perceptrons.
		<br>
		<br>
		<img class="images" src="img/multiLayer.png" alt="">
	</div>
	<div class="footnotes">
		<ol>
			<li id="1">
				Gibson, Adam, and Josh Patterson. Deep Learning: A Practitioner’s Approach. Sebastopol CA: O’Reilly, 2017.
			</li>
			<li id="2">
				Ibid.
			</li>
			<li id="3">
				<a href="http://neuralnetworksanddeeplearning.com">Nielsen, Michael A. “Neural Networks and Deep Learning,” 2015.</a>
			</li>
			<li id="4">
				<a href="http://natureofcode.com/book/chapter-10-neural-networks/">Shiffman, Dan. “The Nature of Code.”</a>
			</li>
		</ol>
	</div>
	<script>

	</script>
</body>

</html>