Hebbs Rule

[00:00:00]Welcome to the session on Hebbian learning rule.

[00:00:03]The Hebbian learning rule, also known as Hebb's learning rule, was proposed by Donald O. Hebb and is one of the easiest learning rules in neural networks. It is used for pattern classification. It is a single-layer neural network with one input layer and one output layer. The input layer can have n number of units, while the output layer has only one unit. The Hebbian rule works by updating the weights between neurons in the neural network for each training sample.

[00:00:35]Hebb's principle can be described as a method of determining how to alter the weights between model neurons.

[00:00:43]When two neurons increases, if the two neurons activate simultaneously and reduces if they activate separately.

[00:00:52]Nodes that tend to be either positive or both negative at the same time have strong positive weights, while those that tend to be opposite have strong negative weights.

[00:01:03]One of the key characteristics of Hebb's rule is that it doesn't need a teacher or a target label. It only looks for patterns and correlations in the data.

[00:01:14]In its simplest mathematical form, the change in weight between an input neuron and an output neuron is proportional to their simultaneous activation.

[00:01:25]W I of new is equal to W I old plus X I Y.

[00:01:30]There, W is the weight, X is the input, and Y is the output.

[00:01:36]Update of weight and bias based on error is known as learning.

[00:01:42]The training algorithm of Hebb's rule is given as follows. Step one, set the initial weight to zero. W is equal to zero for I is equal to one to n, where I is the total number of input neurons, and the initial bias B is equal to zero.

[00:02:00]Step two, update the weight and bias. W I new is equal to W I old plus X I Y.

[00:02:09]Where, X I is the input and Y is the output.

[00:02:13]B new is equal to B old plus Y.

[00:02:16]Change in weight is given by del W I is equal to X I Y.

[00:02:23]Change in bias is given by del B equal to Y.

[00:02:29]Now, we shall see a simple numerical to solve the given input sequence pattern using Hebb's rule.

[00:02:37]Calculate the new weights and bias by using the Hebb's rule to perform the following classification of the given input sequence pattern.

[00:02:45]Plus symbol represents a value one.

[00:02:48]And, the empty sequence indicates minus one.

[00:02:51]Consider I belongs to a member of the class has a target value one.

[00:02:57]And, a O does not belong to the member of a class, so has target value minus one.

[00:03:04]The given sequence pattern for class I is plus plus plus empty plus empty plus plus plus.

[00:03:15]Similarly, for class O the given sequence pattern is plus plus plus plus empty plus plus plus plus.

[00:03:26]So, it is stated in our numerical that the plus symbol is represented by the value one. So, all the plus we can represented by giving a value one. And, the empty sequence we have to denote it as minus one.

[00:03:42]Now, we shall generate the input data for the given sequence pattern I and O.

[00:03:50]So, input vector for class I is given as I is equal to 1 1 1 -1 1 -1 1 1 1 Similarly, the input vector for class O can be given as 1 1 1 1 -1 1 1 1 1 The target for class I is given as 1 and target for class O is given as -1.

[00:04:27]Now the input sequence pattern as per the numerical has nine values, so it can be represented from X1 to X9 and the target can be represented by Y.

[00:04:43]There are two classes I and O. We have represented the data values for the given input sequence pattern for class I and class O. Class I, the data is +1 1 1 -1 1 -1 1 1 1. Similarly, class O, the data is 1 1 1 1 -1 1 1 1 1. The target for class I is +1 and the target for class O is -1.

[00:05:17]So, as per our Hebb's rule, the initial weight is assumed to be zero, so the initial weight is given as zero.

[00:05:26]Now, we shall determine the change in weight for class I.

[00:05:31]So, change in weight, the formula is Xi into Y. So, X1 into Y will be del W1.

[00:05:38]Similarly, X2 into Y will be del W2. X3 into Y will be del W3. X4 into Y will be del W 4 X5 into Y will be del W5 etc. So, del W1 is X1 X1 is 1 into Y is 1. So, 1 into 1 is 1. Similarly, 1 into 1 is 1.

[00:06:05]Similarly, -1 del W4 -1 into 1 is -1.

[00:06:10]So, similarly, we have calculated the change in weight for class I as 1 1 1 -1 1 -1 1 1 1. Now, we shall determine the updated weight for class I, which is given by WI new is equal to WI old plus XIY. So, WI old, that is the initial weight, which is 0, and the change in weight is given by del WI. So, we have to add the values present in this row and present in this row to determine the updated weight WI new. So, 0 + 1 gives us 1. Similarly, 0 + 1 1 0 + 1 1 0 + -1 -1 0 + 1 1 0 + -1 -1 0 + 1 1 0 + 1 1 0 + 1 1. And this is the updated weight for class I.

[00:07:15]Now, we shall determine the change in weight for class O, which is given by the same formula X1Y X2Y etc. up to X9Y.

[00:07:26]Now, X1 is 1. Y is this one. So, 1 into -1 gives us -1. 1 into -1 gives us -1. 1 into -1 gives us -1.

[00:07:42]One into minus one, again minus one.

[00:07:45]Minus into minus one, which is one. One into minus one, minus one. One into minus one, minus one. One into minus one, minus one. One into minus one, minus one. This is the change in weight for class four. Now, we have to calculate the updated weight for class four. So, for updated weight for class four calculation, we have to use the WIO as the updated weight of class I. So, summation of these row values and these row values is going to give us the updated weight for class four.

[00:08:26]So, one plus of minus one is zero. One plus of minus one is zero. One plus of minus one is zero. Minus one plus of minus one is minus two. One plus of one is two. Minus one plus of minus one is minus two. One plus of minus one is zero. One plus of minus one is zero. And one plus of minus one is zero.

[00:08:51]This is the updated weight for class four.

[00:08:57]So, now we have calculated the updated weight for the given input sequence patterns, class I and class O.

[00:09:06]Hope you have understood this sum.

[00:09:08]Next example, we shall see how to update the weight and bias of a AND NOT logic gate using bipolar logic.

[00:09:18]So, first we shall see the truth table of AND NOT logic.

[00:09:23]So, the output expression for AND NOT logic is Y is equal to X1 into X2 bar.

[00:09:31]So, here we have considered two input AND gate. So, there are two inputs, X1 and X2. So, input values are input combinations are 0 0, 0 1, 1 0, and 1 1.

[00:09:45]To determine the output expression, Y is equal to X1 into X2 bar. X2 bar is 1, so 0 into 1 0. Similarly, 0 into 1 bar that is 0, so output is 0. 1 into 0 bar that is 1, so 1 into 1 is 1, and 1 into 0 which is gives us 0.

[00:10:09]So, same can be converted in a bipolar form. In bipolar form, zeros are represented by minus one. So, the zero here will be presented as minus one.

[00:10:21]Similarly, the zero is represented as minus one. So, the truth table is as follows. -1 -1 -1, -1 1 -1, 1 -1 1, 1 1 -1.

[00:10:38]Now, we shall see how to update the weight and bias using Hebb's logic for this truth table of the AND NOT logic in Now, we shall see how to solve the AND NOT logic using Hebb's rule.

[00:10:53]The truth table of the AND NOT logic is updated here. X1, X2, Y, and change in weight, change in bias, updated weight, and updated bias is going to be calculated. So, for the first input combination -1 -1, the target is -1. So, del W1 is -1 into -1 that is 1. Del W2 is -1 into -1 1.

[00:11:23]Del B is Y. Y is equal to -1.

[00:11:27]W1, the updated weight is nothing but old weight plus change in weight. Old weight initial weight is considered as zero, so our old weight will be zero.

[00:11:39]So, zero plus one, that is one.

[00:11:42]Similarly, W2 zero plus del W2, that is one. And bias is zero plus minus one, that is minus one. For the second input comb- tion, minus one, one, minus one. Del W1 is minus one minus one, that is one.

[00:12:03]One into minus one, that is minus one.

[00:12:07]And B is minus one.

[00:12:09]The new weight or the updated weight is one plus one, that is two. One plus minus one, that is zero. Minus one plus minus one, that is minus two. For the third input combination, one, minus one, the target is one. So, change in weight will be one into one is one for W1.

[00:12:33]And minus one into one, minus one for del W2.

[00:12:37]And one for del B.

[00:12:40]That is the target.

[00:12:43]And the new weight W1 will be two plus one, that is three.

[00:12:49]W2 will be zero plus minus one, that is zero.

[00:12:53]And B will be minus two plus one, that is minus one.

[00:12:57]For the last input combination, one, one, the target is minus one. And del W1 will be one into minus one, that is minus one.

[00:13:08]W2 will be one into minus one, that is minus one. And del B is nothing but Y, that is minus one. The new weight will be W1 is equal to three W1 old, that is three. Plus change in weight, minus one. So, three plus of minus one, which is two.

[00:13:29]And W2 will be minus one plus of minus one, that is minus two.

[00:13:35]And updated bias will be minus one plus of minus one, that is minus two.

[00:13:42]So, the updated weight for AND NOT logic is two minus two minus two.

[00:13:50]So, hope you would have enjoyed the session on Hebb's learning rule.