Single Layer Perceptron : Implementing a single layer neuron from scratch using python

After going through this article ,you should be able to:

1. Understand how the weights of a neural network gets updated

2. To implement and train your model for any given linearly separable data.

What’s a linearly separable data?

The given data is said to be linearly separable if the points belonging to different classes can be separated by hyperplanes.

To explain this let’s take an example of a 2 input neural network that classifies each input as 1 or 0.

Let’s consider the given data to be the truth table of NAND gate.

Given Inputs and expected outputs

Let d(expected outputs)=y and Inputs be x1 and x2.

Then,

X=[x1 x2], W=[w1 w2] , Y=[y]

Here,

X=[[0,0,1,1],[0,1,0,1]], d=[[1],[1],[1],[0]], initially W=[0 , 0]

Bias b=1

From my previous article on Perceptron ,

v= Xᵗ.W+b

Y=Φ(v)

Here the activation function used is hardlimit function (Φ(v)=1 ;v≥0 and Φ(v)=0 ;v<0)

The equation of the hyperplane in our case is similar to that of a straight line and is given by

Y=w1*x1+w2*x2+b

The hyperplanes classifies every point lying above it as 0 and every point below it as 1.

1. Suppose it miss classifies a given point that belongs to 0 as 1, this means that the straight line has to be pushed downwards and weights are updated as W=W-n*X
2. Suppose it miss classifies a given point that belongs to 1 as 0, this means that the straight line has to be pushed upwards and weights are updated as W=W+n*X
3. Suppose the straight line classifies all the lines properly .In that case no updation takes place W=W

Weight Updation Rule

Now let’s program a neural network to perform these operations using Python:

1. Importing required libraries

2. Defining a function pre_out that gives the output of the activation (hardlimit) function.

3. Defining a function update that calculate v and updates the weights.

4. Initializing the required matrices and parameters X(test_x), W, n(learning rate) and bias(b)

5. Train the neural network.

6. Test the trained neural network.

The following will be the output

You can try changing the values of the learning rate and bias , and observe the changes the weights and the convergence rate.